Professional Documents
Culture Documents
Agenda
L2 challenges and limitations Spanning tree protocol - traditional approach Multichassis Etherchannel "Routing" at L2 - Fabricpath and TRILL
Cisco Public
BRKDCT-2049
Cisco Public
Traditional approach
L2 Requires a Tree
11 Physical Links
S2
5 Logical Links
S1
S3
Spanning Tree Protocol (STP) typically used to build this tree Tree topology implies:
Wasted bandwidth increased oversubscription Sub-optimal paths Conservative convergence (timer-based) failure catastrophic (fails open)
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
What is Spanning-Tree ?
Why do we need it ?
A redundant connection kills a bridged network: No TTL at layer 2, A single packet can take the whole bandwidth Though, we want to keep parallel links for redundancy
Cisco Public
What is Spanning-Tree ?
Why do we need it ?
The Spanning-Tree is a layer-2 algorithm was originally designed by Radia Perlman while working for DEC in 1985. Adopted into IEEE 802.1D 1990 with updates in 1998 and 2004 This protocol provides the following: Loop-free network Keeps the redundancy in case of failure Operates in a plug & play fashion
2011 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Cisco Public
Bridge 1 ROOT
Bridge 2
Bridge 3
Bridge 4
Bridge 5
Bridge 6
Bridge 7
IEEE solution:
802.1w/RSTP (Rapid Spanning Tree Protocol)
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
Optimizing L2 Convergence
PVST+, Rapid PVST+ or MST
Rapid-PVST+ greatly improves the restoration times for any VLAN that requires a topology convergence due to link UP Rapid-PVST+ also greatly improves convergence time over backbone fast for any indirect link failures PVST+
Traditional spanning tree implementation
Rapid PVST+
Scales to large size (~10,000 logical ports) Easy to implement, proven, scales
MST
Permits very large scale STP implementations (~30,000 logical ports)
Not as flexible as rapid PVST+
Cisco Public
10
Layer 2 Hardening
Spanning Tree Should Behave the Way You Expect
Place the root where you want it
Root primary/secondary macro
LoopGuard STP Root
RootGuard LoopGuard
11
Multichassis Etherchannel
BRKDCT-2049
Cisco Public
12
Feature Overview
How does it help with STP? (1 of 2) Before
STP blocks redundant uplinks VLAN based load balancing Loop Resolution relies on STP Protocol Failure
Primary Root Secondary Root
After
No blocked uplinks Lower oversubscription
Cisco Public
13
Feature Overview
How does it help with STP? (2 of 2) Reuse existing infrastructure
Cisco Public
14
BRKDCT-2049
Cisco Public
15
802.3ad or PagP
802.3ad
802.3ad or PagP
802.3ad
Server
Server
Server
Minimizes traffic disruption from switch or uplink failure with Deterministic subsecond
Stateful and Graceful Recovery (SSO/NSF)
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
16
VS Header
L2 Hdr L3 Hdr
Data
CRC
VSL
PFC
SF
RP
SF
RP
PFC
Active Supervisor
CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards
SSO Synchronization
CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards
Cisco Public
18
Si
Si
Switch 2 Slot 5 Processor Information : ---------------------------------------------Current Software state = STANDBY HOT (switchover target) <snip>
Switch1
Switch2
Cisco Public
19
VSS Domain 10
VSS Domain 20
VSS Domain 30
Use a UNIQUE VSS Domain-ID for each VSS Domain throughout the network. Various protocols use Domain-IDs to uniquely identify each pair.
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
VSS
Both LACP and PAGP Etherchannel protocols and Manual ON modes are supported
Blue Traffic destined for the Server will result in Link 1 in the MEC link bundle being chosen as the destination path
Orange Traffic destined for the Server will result in Link 2 in the MEC link bundle being chosen as the destination path
Link 1
Link 2
Cisco Public
22
High Availability
Dual-Active Detection
If the entire VSL bundle should happen to go down, the Virtual Switching System Domain will enter a Dual Active scenario where both switches transition to Active state and share the same network configuration (IP addresses, MAC address, Router IDs, etc) potentially causing communication problems through the network
3 Step Process detection (using one or more of 1 Dual-Active three available methods - ePAgP, VLSP Fast
Hello, IP BFD) Switch1 Switch2
2 3
Recovery Period - Further network disruption is avoided by disabling previous VSS active switch interfaces connected to neighboring devices . Dual-Active Restoration - when VSL is restored , the switch that has all its interfaces brought down in the previous step will reload to boot in a preferred standby state Active Recovery Standby
VSL Active
Cisco Public
24
Supervisor failure events therefore require manual intervention for recovery of the affected chassis
Uplinks are not active when the Supervisor is in ROMMON mode Undeterministic outage time Relies on manual process to install and convert the new Supervisor with current VSS configuration
Cisco Public
25
STANDBY COLD
Si
Si
VSL
Cisco Public
26
Switch-1
Switch-2
Active Active
STANDBY COLD
Si
VSL
Active Active Si
Cisco Public
27
1
Active VSS supervisor incurs a hardware failure
SSO Active
RPR-Warm
VSL
SSO
STANDBY COLD
Si
Si
100 %
Available Bandwidth
SW1
50%
SW2
= Line Cards Active 1
SW2
Duration
28
Cisco Public
Switch-2
1. SSO failover to the hot-standby supervisor in switch-2 2. Switch-1 reloads and comes back online. 3. 50% bandwidth is available during switch-1 reload
R
SSO
STANDBY COLD
SSO Active
Si
RPR-Warm
VSL
Si
100 %
= Reload
Available Bandwidth
SW1
50%
SSO
= SSO Switchover
SW2
SW2
SW2
1
Cisco Public
Duration
29
3
1. Switch-1 comes online 2. Previous RPR warm supervisor resumes SSO hot standby state
Switch-2
STANDBY COLD
Si
VSL
Si
100 %
Available Bandwidth
R
SW1
50%
SW1
= Reload
= Line Cards Active
SW2
SW2
SW2
SW2
1
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
Duration
30
100%
100%
50%
50%
1
SW2
4
SW1/SW2
5
SW1
SW2
SW1
4 SW1
At step 3 during RPR switchover, bandwidth will be dropped to 0% for 1-2 minutes
2011 Cisco and/or its affiliates. All rights reserved.
With EFSU, a minimum of 50% bandwidth is available throughout the software upgrade process
Cisco Public
31
L3 Core
No FHRPs No Looped topology Policy Management
L2/L3 Distribution
Access
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
32
L2/L3 Core
L2 Distribution
Dual-Homed Servers, Single active uplink per VLAN (PVST), Fast L2 convergence
L2 Access
Cisco Public
33
BRKDCT-2049
Cisco Public
34
Feature Overview
vPC Definition
Allow a single device to use a port channel across two upstream switches
Eliminate STP blocked ports and uses all available uplink bandwidth Dual-homed server operate in active-active mode Provide fast convergence upon link/device failure Reduce CAPEX and OPEX Available on all current and future generation cards
Logical Topology without vPC
35
Feature Overview
vPC Terminology
vPC Peer-keepalive link vPC Domain vPC peer-link
vPC Domain - pair of vPC switches vPC peer - vPC switch, one of the pair
CFS protocol
vPC member port - one of the set of ports that form a vPC
vPC - the combined port channel between the vPC peers and the downstream device vPC peer-link - link used to synchronize state between vPC peer devices, must be 10GbE vPC peer-keepalive link - the keepalive link between vPC peer devices (backup to the vPC peer-link)
Cisco Public
36
Single-Sided vPC
root
vPC on the N7k Root
N7k02 2/10
logical equivalent
2/1
Po51,2 N5k01
2/2
2/1
2/2
N5k02
Cisco Public
37
Double-Sided vPC
root
vPC on the N7k
Root
N7k02 2/10
logical equivalent
2/1
2/2
2/1
2/2
Po51
N5k01
Po10
Peer Link N5k02
primary
secondary
38
Recommendations:
Use LACP when available for graceful failover and mis-configuration protection
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
P S P S
1. Dual Attached
40
7k1 Po1
7k2
P
L3 ECMP
Routing Protocol Peer Dynamic Peering Relationship
Router
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
Router
41
Requirements:
Needs to remain enabled, but doesnt dictate vPC member port state
Best Practices:
Make sure all switches in you layer 2 domain are running with Rapid-PVST or MST (IOS default is non-rapid PVST+), to avoid slow STP convergence (30+ secs) Remember to configure portfast (edge port-type) on host facing interfaces to avoid slow STP convergence (30+ secs)
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
vPC vPC STP is running to manage loops outside of vPCs direct domain, or before initial vPC configuration
42
L3 L2
44
Feature Overview
vPC and VSS Comparison
Functionality
Multi-Chassis Port Channel Loop-free Topology STP as a fail-safe protocol Control Plane Support for Layer 3 portchannels Control Plane Protocols 10GE ports in the Channel Device Configuration Non Disruptive ISSU Support
Cisco Public
45
BRKDCT-2049
Cisco Public
46
47
FabricPath Introduction
BRKDCT-2049
Cisco Public
48
FabricPath IS-IS
FabricPath IS-IS replaces STP as control-plane protocol in FabricPath network Improves failure detection, network reconvergence, and high availability
Introduces link-state protocol with support for ECMP for Layer 2 forwarding
Exchanges reachability of Switch IDs and builds forwarding trees
STP BPDU
STP BPDU
FabricPath IS-IS
STP
FabricPath
Cisco Public
49
Why IS-IS?
A few key reasons: Has no IP dependency no need for IP reachability in order to form adjacency between devices Easily extensible Using custom TLVs, IS-IS devices can exchange information about virtually anything Provides SPF routing Excellent topology building and reconvergence characteristics
Cisco Public
50
Ethernet
FabricPath Header
STP FabricPath
FabricPath Interface Interfaces connected to another FabricPath device Send/receive traffic with FabricPath header No spanning tree!!! No MAC learning Exchange topology info through L2 ISIS adjacency Forwarding based on Switch ID Table
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
DSID20
SSID10 DMACB SMACA Payload
S10
Ingress FabricPath Switch
S20
Egress FabricPath Switch
Payload
SMACA
DMACB
MAC A
MAC B
Ingress FabricPath switch determines destination Switch ID and imposes FabricPath header Destination Switch ID used to make routing decisions through FabricPath core No MAC learning or lookups required inside core Egress FabricPath switch removes FabricPath header and forwards to CE
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
A B C D
S100
S101
FabricPath
S200
MAC A
MAC B
MAC C
MAC D
Cisco Public
54
S101
FabricPath
S200
S100
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
S101
S200 55
S300
S100
MAC C
FabricPath Core
S200 MAC A
MAC B
Cisco Public
58
Multidestination traffic constrained to loop-free trees touching all FabricPath switches Root switch assigned for each multidestination tree in FabricPath domain Loop-free tree built from each Root and assigned a network-wide identifier (Ftag) Support for multiple multidestination trees provides multipathing for multi-destination traffic
Two trees supported in NX-OS release 5.1
S100
S101
FabricPath
S200
S100
S20
S100
S10
S10
S101
S30
S40
S101
S20
Root
S200
S40
Root
S200
S30
Logical Tree 1
2011 Cisco and/or its affiliates. All rights reserved.
Logical Tree 2
Cisco Public
59
S3
FabricPath
L2
Introducing VPC+
S1
L1
CE F1 F1 F1
po3
VPC+ F1
F1 S2 F1
VPC+ allows dual-homed connections from edge ports into FabricPath domain with active/active forwarding
CE switch, Layer 3 router, dual-homed server, etc. Physical
Host A
Host AS4L1,L2
F1 F1 F1 S2
VPC+ creates virtual FabricPath switch for each VPC+-attached device to allow loadbalancing within FabricPath domain
VPC+
S4
po3
Host A
Cisco Public
61
S10
S20
S30
S40
S100
FabricPath
S200
MAC A
MAC B
Cisco Public
MAC C 62
S10
S20
S30
S40
S1000
S100
FabricPath
S200
MAC A
MAC B
Cisco Public
MAC C 63
HSRP Standby
SVI
S10 S20
SVI
S30 S40
DMAC0002 SMACHSRP
Payload S1000
po1 po2
S100
FabricPath
S200
1/30
MAC A
MAC B
MAC C
Cisco Public
64
BRKDCT-2049
Cisco Public
65
http://datatracker.ietf.org/wg/trill/
Cisco Public
66
FabricPath initial release runs in a Native mode that is Cisco-specific, using proprietary encapsulation and control-plane elements
Nexus 7000 F1 I/O modules and Nexus 5500 HW are capable of running both FabricPath and TRILL modes
Cisco Public
67
TRILL
Yes No No No No Point-to-point OR shared
Cisco Public
68
Conclusion
BRKDCT-2049
Cisco Public
69
Cisco Public
70
Thank you.
Backup slides
BRKDCT-2049
Cisco Public
72
VSL bandwidth should be greater than or equal to the largest bandwidth connection to a single attached device (downlink)
Consider the bandwidth on a per VSS chassis basis
Si
Si
Consider the bandwidth for any Service Modules and SPAN sessions Distribute the VSL interfaces across multiple modules for added resiliency Include at least one VSL interface from the Supervisor module for faster VSL bring-up during reloads
Cisco Public
73
S10
S20
S30
S40
Ftag
1 2
Payload
L9
Broadcast
1 2
S100
S101
FabricPath
Multidestination Trees on Switch 200
Payload
S200
Tree
IF L9 L9,L10,L11,L12
Ftag
MAC A
1 2
MAC B
Cisco Public
74
S10
S20
S30
S40
Ftag
1 2
L9
Ftag
1 2
S100
S101
FabricPath
Multidestination Trees on Switch 200
Payload
S200
Tree
IF L9 L9,L10,L11,L12
Unknown
MAC A
1 2
A
B
e1/1 (local)
S200 (remote)
MAC B
A
B e12/2 (local)
Cisco Public
75
S10
S20
S30
S40
S200
S200
DMACB
SMACA
Payload
L9
FabricPath
Payload
S200
S101
S200
S200
MAC A
S200
S200
MAC B
IF/SID
e1/1 (local) S200 (remote)
B
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
76