Cisco

Introduction to
Load Balancing
BRKAPP-1001
Session_ID
Presentation_ID © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 2
Agenda
 Introduction
 Load Balancing and Health Monitoring
 Flow Management
 Server Offload
 High-Availability
 Deployments
 Geographic Load Balancing
 What’s Next ?
Session_ID
Cisco Application Delivery Networks
Network Classification Application Scalability Application Networking
• Quality of service • Server load-balancing • Message transformation
• Network-based app recognition • Site selection • Protocol transformation
• Queuing, policing, shaping • SSL termination and offload • Message-based security
• Visibility, monitoring, control • Video delivery • Application visibility
WAN
Application Acceleration WAN Acceleration Application Optimization

• Latency mitigation • Data redundancy elimination • Delta encoding
• Application data cache • Window scaling • FlashForward optimization
• Meta data cache • LZ compression • Application security
• Local services • Adaptive congestion avoidance • Server offload
Session_ID
The Application Delivery Journey
Client/Server Web-enabled SOA/Web 2.0

Application
Trends Centralized Decentralized Distributed
Few connections 1000s of connections Exponential increase in
connections
Early Application End-to-End Application

Technologies Aware Networks Delivery Networks
L4-7 Switching Message Visibility

Cisco Load balancing Virtualization
Solution Web
Acceleration Deep Packet
Inspection
QoS
WAN Multi-Gigabit
Optimization Performance
1995–2000 2000–2006 2006 and Beyond
Session_ID
How It All Started
Direct Communication Clients/Servers
IP TCP http Data
X
Web Server
Benefit
 Simple solution
Issue
 No Fault tolerance
 Limited performance and scalability
Session_ID
Scaling to a Few Servers
The Software Approach
S/W Load Balancer

Clustering Technologies
Benefit
 Addresses some of the fault tolerant and performance issues
Issue
 Still limited in scale/performance.
 Leverages server resources for LB and HA
 Proprietary clustering technologies
Session_ID
Scale and High Availability for Larger
Deployments: The Hardware-Based Solution
Benefit
 Addresses fault tolerant, performance and scalability issues
 Future-proof: architecture includes hardware co-processors to
support resource-intensive features (i.e. SSL, compression)
Session_ID
The Main Functions of a Load Balancer
Clients Load Web
Balancer/ Servers
Content
Switch
Database
 Represents multiple server farms

with public IP addresses
Virtual IP’s or VIP’s (which clients Streaming
resolve via DNS)
 Monitors the health of servers
 Intelligently distributes incoming requests
according to configurable rules
Session_ID
Terminology
Load Balancing
Content Algorithm
Clients (Predictor)
Switch
- Round Robin
Load
Balancer Serverfarm
Servers
Client-Side Keepalive
Gateway (Probe)
Class-Map
URL = /news
Virtual IP Address User-Agent = WindowsCE
(VIP) Client = 192.0.0.0/8
172.16.2.100 XML
TCP port 80 Policy-Map Gateways
If match class-map X
then use serverfarm X
Session_ID else use serverfarm y
Devices Being Load balanced
 Servers
 Proxies
 Accelerators (compression engines, SSL offloaders)
 Caches (reverse and transparent)
 Firewalls (Layer 3 and Layer 2)
 VPN concentrators
 Routers
 Generic IP device requiring
load distribution and/or redundancy
Session_ID
Traffic Being Load balanced
 Generic IP traffic (i.e. IPsec tunnels)
 Generic UDP and TCP (i.e. proprietary protocols)
 Network services (i.e. LDAP, DNS, Radius)
 HTTP (i.e. Web Presentation Layer, Web Services, SOAP/XML)
 Voice & Video (i.e. RTSP, SIP, H.323)
 Remote terminals (i.e. Windows Terminal Services)
 Multi-connection protocols (i.e. FTP, RTSP)
 Multi-tier packaged applications (i.e. SAP, Oracle, Microsoft, BEA)
 Vertical specific applications (i.e. medical, finance, education)
Ethernet IP TCP HTTP Ethernet

Payload
Header Header Header Header Trailer
Layer 2 Layer 3 Layer 4
Layer 5-7
Session_ID
HTTP
The Most Common Load Balanced Protocol
 RFC 2616,HTTP 1.1 IETF draft standard:

―The hypertext transfer protocol (HTTP) is an
application-level protocol for distributed,
collaborative, hypermedia information systems‖
 Three important elements of an HTTP request:
Method (GET, POST, …)
URI
Headers (include cookies)
 Carried over TCP
Multiple HTTP requests can be ―tunneled‖ over the same TCP
connection
Session_ID
HTTP 1.0—Single Request
Client Web
Server
SYN
SYN_ACK
ACK
GET / HTTP 1.0
ACK
HTTP/1.0 200 OK
Continuation
ACK
FIN
FIN_ACK
ACK
Session_ID
HTTP 1.1—Two Requests, No Pipelining
Client Web
Server
SYN
SYN_ACK
ACK
GET /a.gif HTTP 1.1
ACK
HTTP/1.1 200 OK
ACK
GET /b.jpg HTTP 1.1
ACK
HTTP/1.1 200 OK
Continuation
ACK
FIN
FIN_ACK
ACK
Session_ID
HTTP 1.1—Building an Entire Page
TCP 3101 > 80
index.html
TCP 3102 > 80
logo1.gif globe.gif footpage.jpg
TCP 3103 > 80

/cgi-bin/count
The behaviour
depends TCP 3104 > 80
on the browser
bannertop.jpg menu.jpg
Session_ID
FTP—File Transfer Protocol
A Multi-Connection Protocol
Active FTP
C:>ftp
Client test.cisco.com
FTP server test
User: abc FTP
Password: xxx
230 User abc
Server
3016 21
1
2
3017 20
3
4
Session_ID
FTP—File Transfer Protocol
A Multi-Connection Protocol
Passive FTP
C:>ftp
Client test.cisco.com
FTP server test
User: abc FTP
Password: xxx
230 User abc
Server
3018 21
1
2
3019 2036
3
4
Session_ID
Load Balancing and
Health Monitoring
How Connections Are Distributed to the Best Available Servers
Session_ID
Load Balancing Algorithms
Client Serverfarm
How to Distribute Requests Across Servers ?

Enhanced Predictors Improve Serverfarm Efficiency
Session_ID
Load Balancing Algorithms
 (Weighted) Round Robin
Very simple, servers receive equal (or proportional) amount of requests
 (Weighted) Least Connections
Dynamic, based on open connections, optimizes load across servers
 Hash on IP (source/destination, with mask)
No state required for persistence
 Hash on URL or portion of URL
Useful for transparent cache redirection
 Based on Load
Server load retrieved via SNMP or feedback protocols
 Fastest
Based on response time: fastest servers receive newer connections
 Least Bandwidth
Real-time amount of traffic considered to select less active server
Session_ID
Session Persistence—Stickiness
The ―Shopping Cart‖ Problem
Browse
I’ll Never
Shop Here 1
Again!
Select
3 Buy
Empty?!?
Session_ID
Session Persistence—Stickiness
 Session: logical aggregation of multiple simultaneous or
subsequent connections
 Sessions are limited in time (timeout)
 Servers might keep session state locally
 Load distribution across multiple servers introduces the problem
The content switch needs to identify a session and send

connections belonging to the same session (i.e. from the
same client) to the same server
Methods to identify the session or client:

Source IP address, HTTP session cookie, SIP session ID,
SSL ID, generic protocol session data, …
Session_ID
Health Checking
 The content switch needs to continuously monitor the
back-end servers
 Failed servers have to be identified and removed from rotation:
the load balancing algorithms adapt to the change
 Server failures should be transparent to clients
 Servers recovering from failures should be checked and put back in
the available pool, avoiding flapping
 Any failure affecting client-server interaction should be detected:
connectivity, application or back-end servers malfunctions
Clients Serverfarm
X
Session_ID
Presentation_ID © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
X 24
Active Probing—Keepalives
 Intended to run periodically
 Generated by the load balancer: a correct reply is expected
 Either predefined health checks or user-configurable scripts
 Examples: ICMP (L3 connectivity), TCP (stack), HTTP (application)
 For each probe:
Interval, retry times
Maximum TCP open time
Maximum receive time (max response time)
Failed retry time, successful retries before back in service
Serverfarm
Session_ID
In-Band Health Monitoring
 The load balancer monitors server-to-client ―inband‖ traffic and
keep counters for consecutive errors
 Can catch basic errors:
No replies from server
RST’s from server
 For HTTP traffic, can perform return error code checking (i.e. 500-
type errors should remove servers from rotation)
Clients Serverfarm
Session_ID
Flow Management
―Layer 4‖ and ―Layer 7‖ processing
Session_ID
Flows, Connections, Sessions
A Load Balancer Maintains

Much More State than a Router
on a per-Flow Basis
 3 main types of flows

TCP: IP protocol, src/dst IP, src/dst L4 port, TCP state
UDP: IP protocol, src/dst IP, src/dst L4 port
Generic IP: source/destination IP
 TCP flows (connections) require setup

 Multiple flows between the same client and server might be
logically grouped into a session
Session_ID
Layer 4 Switching
 L2–L4 information is always present in the first packet
of the flow (unless it is a fragment!)
IP protocol
Source/destination IP addresses
Source/destination L4 ports (for TCP/UDP)
Source VLAN, MAC address
 The load balancing decision can be made on the first packet
Session_ID
Layer 4 Flow Setup—Basic Load Balancing
Decisions Made on First Packet
Matches VIP
SYN Selects Server
Rewrites
L2/L3/L4
Matches Existing
Flow
Rewrites L2/L3/L4 SYN_ACK
Shortcut
ACK
Shortcut
Data
Shortcut
Data
Shortcut
Session_ID
Layer 7 Switching
 L5–L7 information is only received after the TCP setup and might
span multiple packets
HTTP URLs, cookies, header fields
SSL session ID
FTP data channel port
Generic application data
 Requires TCP termination and buffering!
Session_ID
Layer 7 Flow Setup for HTTP (1/3)
Load Balancing Decisions Require More Data
Matches VIP w/L7 rule

SYN Chooses SEQ #
Replies w/SYN_ACK
ACK Starts
Buffering
Data
ACK’s Client Packets

Keeps Buffering
Session_ID
Parses the Data

Data Selects Server
Initiates TCP
SYN_ACK
Acts as Client
Does Not Forward
SYN_ACK
Empties Buffer
Sends Data to Server
Session_ID
Does Not Forward ACK

Ready to
Splice the Flows ACK
Matches Existing Flow
Rewrites L2/L3/L4
and SEQ/ACK Data
Shortcut
ACK
Shortcut
Data
Shortcut
Session_ID
Layer 7 Flow Setup—Full Proxy
The Most Flexible Approach
SYN
SYN_ACK
ACK
GET/HTTP 1.1
Data
ACK SYN
Independent client &

server connections
SYN_ACK
ACK
Full Proxy
Data—GET
ACK
HTTP/1.1 200 OK HTTP/1.1 200 OK Data
ACK
… …
Data
Client connection Server connection
Session_ID
Content Switching Metrics
 Connections per Second (CPS)
L4 vs. L7
 HTTP requests per Second (―CPS‖)
HTTP 1.1 vs. 1.0
 Concurrent Connections (CC)
 Bandwidth (in Gbps) and Packets per Second
 Latency
 Keepalives per second
 Number of virtual servers/real servers
 Number of policies/rules
Session_ID
Server Offload
Freeing Up Server CPU and Resources
Session_ID
Server Offload Overview
 What is it ?
Perform resource intensive functions on application traffic in the content switch
on behalf of the server. Often hardware accelerated.
 Why ?
Servers can dedicate more resources to processing and serving client requests:
faster application response!
 What can be offloaded ?
SSL processing, TCP setup/close, HTTP compression, XML processing,…
Application Servers
Switch
Session_ID
Offloading SSL
 Offload CPU-intensive SSL processing
Servers resources are dedicated to serving requests and running applications,
rather than encrypting data
 Centralized key/certificate storage/management
 Allows advanced content switching (URL-based, cookie-sticky,
payload parsing) and inspection of SSL traffic
 Scalability: easy to add more SSL ―performance‖
Content
Switch
Encrypted to Clear Text to

VIP:443 Servers:80 Web
Servers
Session_ID
SSL—Handshake
Full Abbreviated
Re-use same SSL session ID
Less latency - Faster applications
Client Hello
Server Hello
Certificate *
Server Key Exchange *
Certificate Request * Client Hello
Server Hello Done
Server Hello
* Certificate Change Cipher Spec
Client Key Exchange Finished
* Certificate Verify
Change Cipher Spec Change Cipher Spec
Finished Finished
Change Cipher Spec

Finished Application Data
Application Data
Session_ID
Building an Encrypted Web Page
TCP 3101 > 443
SSL ID index.html
123
TCP 3102 > 443
SSL ID logo1.gif globe.gif footpage.jpg
123
TCP 3103 > 443

SSL ID bannertop.jpg menu.jpg
123
Session_ID
SSL Offload Metrics
 New transactions per second (TPS)

Full SSL setup (asymmetric)
Depends on key size
Different from chipset RSA operations
 Raw throughput (in Mbps or Gbps)
Symmetric
 Concurrent connections (CC)

 Number of SSL ID cached entries (for SSL ID re-use)
 Number of services
 Number of certificates
Session_ID
Offloading TCP
TCP Reuse (Multiplex)
 Offload TCP (HTTP) setup processing from servers
Servers resources are dedicated to serving requests and running
applications, rather than opening and closing TCP connections
 TCP connections to the server are kept open
(HTTP 1.1 Connection Keepalive)
 Client requests multiplexed to existing server connections
TCP1
TCP1 Pool1
TCP2
TCP2 Pool2
TCP3
Session_ID
High Availability
Protecting Against Single Points of Failure
Session_ID
Redundancy
Heartbeat and State

Synchronization link
BACKUP
Internet VIP Active IP Interface

192.1.1.100 10.1.1.254
ACTIVE
Session_ID
Terminology
BOX-TO-BOX
PER-VIP REDUNDANCY
REDUNDANCY
An Entire Load Balancer Is
Either Active or Standby Each VIP Can
GRANULARITY Independently Be Active or
All VIPs Are Standby
in the Same State
ACTIVE-ACTIVE ACTIVE-STANDBY
Only One Entity Can

Multiple Entities Can Process Traffic at Any Given
STATE Process Traffic at the Same Time
Time (The Other Is
Standby/Monitoring)
Session_ID
Redundancy—Statefulness
STICKY
STATELESS FULL STATEFUL
STATEFUL
LB
COMMUNICATION
Sync/Monitor Sticky Tables Full Flow Tables
IDEAL FOR Stateless Content Session Stateful Long Living Flows
LB RESOURCES Low Medium High
Adaptive Redundancy
Stateful Level Configurable
Independently on Each Policy
Session_ID
Deployments
Network Integration Options and Examples
Session_ID
Router Mode
Servers Default Gateway:

Content Switch IP
Content Switch ―Routing‖
Subnet A Subnet B
 Servers in private IP subnet
 VIPs usually in different, routable subnet from servers
 Requires two IP subnets
 Easy to deploy with many server IP subnets
Session_ID
Bridge Mode
Upstream Router
Content Switch ―Bridging‖
Subnet A
 Servers in routable IP subnet
 VIP’s can be in the same or different subnet
 Requires one IP subnets for each farm
 Easy deploy for firewall or cache load balancing
Session_ID
L3 One-Arm Mode
Subnet B
Upstream Router
 L2-rewrite not possible

 Content switch not inline Subnet B
Does not see unnecessary traffic
 Requires PBR, server default gateway pointing
to load balancer or client source NAT
The return traffic is needed!
 Not as common as bridge or routed mode due to problems
with forcing traffic back to CSM in return direction
PBR—Policy Based Routing, NAT—Network Address Translation
Session_ID
L3 One-Arm Mode—Flows
VIP
Server
IP
1 3’ 2
4 3
1 Just Routing Traffic to the VIP
2 Just Routing Traffic to the Server IP
3 L2 to the Server Default Gateway
3’ Routing Would Break; Need to Use Either PBR, SNAT,
or Server Default Gateway
4 Just Routing to the Client IP
PBR—Policy Based Routing, sNAT—Source Network Address Translation

Session_ID
L2 One-Arm Mode
Return Traffic Bypassing Load Balancer
Servers
Default Gateway:
Upstream Router
Same IP Subnet
 Bypass for return traffic: high throughput!
 Requires MAC rewrite, L2 adjacency
 Servers need identical loopback addresses (one per VIP)
 TCP termination not possible: no L7 features!
 Load balancer blind to return traffic (inband, accounting)
Session_ID
A Multi-Tier Example of Deployment
Application Server Suite 10g
3 serverfarm in 3 distinct APPHosts
IP subnets configured in
bridge mode
Application Servers
(portal, Java,
caching)
IDMHosts
Identity Management
(login functions)
DBHosts
OIDHosts
Separate Data-Base
farm not requiring Internet Directory
load balancing (LDAP)
Session_ID
Firewall Load Balancing
FWLB + SLB
Internal Load Balancer

Inside
distributes traffic to servers
and stores source MAC address Network
for return traffic to firewalls
1
Internal
2 3 Load Balancer
8
7 6
External 4
Load Balancer
5
Firewall Serverfarm
farm
Session_ID
Geographic
Load Balancing
Disaster Recovery and Load Distribution Across Data Centers
Session_ID
Distributed Data Center Topology
Internal Internet
Network Service Service
Provider A Provider B Internal
Network
Front-End Tier
(Web)
Application
Tier
Database
Tier
Session_ID
Site Selection Mechanisms
 Site selection mechanisms depend on the technology

or mix of technologies adopted
for request routing:
1. HTTP Redirect
2. DNS Based
3. Route Health Injection and L3 Routing
 Health of servers and applications need to

be taken into account
 Optionally, also other metrics (like load and distance)
can be measured and utilized for a better selection
Session_ID
DNS-Based Site Selection
Root DNS for/ Root DNS for .com
DNS Proxy 2
3 4 Authoritative DNS
5
cisco.com
1 6
10 7
8
Client 9 Authoritative
DNS
http://www.cisco.com/ www.cisco.com
Data Center 1 Data Center 2

Session_ID
DNS-Based Site Selection
Root DNS for/ Root DNS for .com
DNS Proxy 2
3 4 Authoritative DNS
5
cisco.com
1 6
10 7
8
Client 9 Authoritative
TCP:80
DNS
http://www.cisco.com/ www.cisco.com
Data Center 1 Data Center 2

Session_ID
What’s Next ?
Load Balancing, Content Switching, Application

Delivery … and Cisco Products
Session_ID
Advanced Requirements: From Load
Balancing to Application Delivery
 Server Offload
Free up server CPU and resources
 Application Acceleration
Better user experience, faster transactions
 Bandwidth Reduction
Efficient WAN resources utilization
 Application and Protocol Inspection
Protection against sophisticated application-specific attacks
 Virtualization
One physical device behaves as many: maximum deployment
flexibility and separation of resources
 Flexible Network Management
Allows multiple users, with different responsibilities, to
simultaneously manage the device
Session_ID
Cisco Application Control Engine Family
XML Switching and PCI Application Switching
Multi-Module
(64 Gbps)
ACE XML Gateway
30,000 TPS
Module
(4-16 Gbps) +
ACE Web ACE

Module
Application 16 Gbps
Firewall
ACE
Appliance Module
(1-2 Gbps) 8 Gbps
ACE
Module ―One-Click‖
4 Gbps
ACE 4710 Migration
2 Gbps Tools
ACE XML
ANM Gateway
ACE 4710 Manager
1 Gbps
ACE GSS
20K DNS RPS
CSS 11501
Up to 1 Gbps Global Products and Tools
Session_ID
Session_ID

Cisco

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco

Uploaded by

Copyright:

Available Formats

Introduction to

Application Acceleration WAN Acceleration Application Optimization

Client/Server Web-enabled SOA/Web 2.0

Early Application End-to-End Application

L4-7 Switching Message Visibility

1995–2000 2000–2006 2006 and Beyond

IP TCP http Data

S/W Load Balancer

 Represents multiple server farms

Ethernet IP TCP HTTP Ethernet

 RFC 2616,HTTP 1.1 IETF draft standard:

TCP 3101 > 80

TCP 3103 > 80

How Connections Are Distributed to the Best Available Servers

How to Distribute Requests Across Servers ?

The content switch needs to identify a session and send

Methods to identify the session or client:

―Layer 4‖ and ―Layer 7‖ processing

A Load Balancer Maintains

 3 main types of flows

 TCP flows (connections) require setup

 The load balancing decision can be made on the first packet

 Requires TCP termination and buffering!

Matches VIP w/L7 rule

ACK’s Client Packets

Parses the Data

Does Not Forward ACK

Independent client &

Client connection Server connection

Freeing Up Server CPU and Resources

Encrypted to Clear Text to

Change Cipher Spec

TCP 3101 > 443

TCP 3103 > 443

 New transactions per second (TPS)

 Concurrent connections (CC)

Protecting Against Single Points of Failure

Heartbeat and State

Internet VIP Active IP Interface

Only One Entity Can

IDEAL FOR Stateless Content Session Stateful Long Living Flows

LB RESOURCES Low Medium High

Network Integration Options and Examples

Servers Default Gateway:

Content Switch ―Bridging‖

 L2-rewrite not possible

PBR—Policy Based Routing, sNAT—Source Network Address Translation

Internal Load Balancer

Disaster Recovery and Load Distribution Across Data Centers

 Site selection mechanisms depend on the technology

 Health of servers and applications need to

Data Center 1 Data Center 2

Data Center 1 Data Center 2

Load Balancing, Content Switching, Application

ACE Web ACE

You might also like