IBM System p - Virtual I/O Server Physical and Virtual Ethernet Configuration 802.

1q (VLAN tagging) enabled configuration Dual networks

User Ethernet Network

Dean Rowswell, IBM March 19, 2008

Cisco Switch configuration example for the User Ethernet network interface Port-channel83 description LACP channel VIO server #1 User network no ip address switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 100 switchport mode trunk switchport nonegotiate storm-control broadcast level 10.00 storm-control multicast level 10.00 spanning-tree portfast trunk spanning-tree bpduguard enable ! interface GigabitEthernet7/46 description LACP VIO server #1 ethernet adapter #1 no ip address switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 100 switchport mode trunk switchport nonegotiate spanning-tree portfast trunk spanning-tree bpduguard enable channel-protocol lacp channel-group 83 mode passive ! interface GigabitEthernet8/46 description LACP VIO server #1 ethernet adapter #2 no ip address switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 100 switchport mode trunk switchport nonegotiate spanning-tree portfast trunk spanning-tree bpduguard enable channel-protocol lacp channel-group 83 mode passive ! NOTES: The Storage Ethernet network configuration uses different interface and port channel numbers and vlan 200 but is otherwise the same config

Storage Ethernet Network

Virtual I/O Server #1

Virtual I/O Server #2

en

en

en

en

t0 en

t2 en

t0 en

t2 en

Shared Ethernet Adapter configuration Define primary and backup roles for 2 virtual ethernet adapters in each VIOS VIOS #1 ent10 : SEA failover trunk priority=1 ent11 : SEA failover trunk priority=2 VIOS #2 ent10 : SEA failover trunk priority=2 ent11 : SEA failover trunk priority=1

AIX or Linux LPAR

AIX or Linux LPAR

ent8 (8023ad) ent18 (virtual) ent10 (virtual) ent14 (SEA) ent12 (ctl chan)

Ethernet adapters on each Virtual I/O Server ent0 : physical connection to User network ent1 : not used (bottom port of 2-port adapter) ent2 : physical connection to Storage Network ent3 : not used (bottom port of 2-port adapter) ent4 : physical connection to User network ent5 : not used (bottom port of 2-port adapter) ent6 : physical connection to Storage network ent7 : not used (bottom port of 2-port adapter) ent8 : 8023ad Link Aggregation device for ent0 and ent4 User network ent9 : 8023ad Link Aggregation device for ent2 and ent6 Storage network ent10 : virtual ethernet adapter with 802.1Q enabled / VLAN ID: 1, 100 / Access external network ent11 : virtual ethernet adapter with 802.1Q enabled / VLAN ID: 2, 200 / Access external network ent12 : virtual ethernet adapter VLAN ID: 91 (control channel) ent13 : virtual ethernet adapter VLAN ID: 92 (control channel) ent14 : Shared Ethernet Adapter (SEA) - bridges external User network with Hypervisor at TCPIP Layer 2 ent15 : Shared Ethernet Adapter (SEA) - bridges external Storage network with Hypervisor at TCPIP Layer 2 ent16 : VIO server VLAN with TAG ID: 100 ent17 : VIO server VLAN with TAG ID: 200 ent18 : virtual ethernet adapter - VIOS connection to VLAN 100 for TCP/IP config including ssh and DLPAR

t4

ent9 (8023ad)

ent11 (virtual) ent15 (SEA) ent13 (ctl chan)

t6

ent8 (8023ad) ent18 (virtual) ent10 (virtual) ent14 (SEA) ent12 (ctl chan)

t4

ent9 (8023ad)

ent11 (virtual) ent15 (SEA) ent13 (ctl chan) ent0 or eth0 ent1 or eth1 ent0 or eth0 ent1 or eth1

t6

VLAN ID = 100

VLAN ID = 200

IBM POWER Hypervisor

Port security must be disabled to allow multiple MAC addresses (VIO client LPARs) on the VIO server ethernet ports This config assumes that the native VLAN is 1 (because it is not displayed above). This is required because LACP packets must not be tagged on the switch.

VIO Server commands for this config Run the first 6 commands on both VIO servers: $ mkvdev -lnagg ent0,ent4 -attr mode=8023ad hash_mode=src_dst_port ent8 Available $ mkvdev -lnagg ent2,ent6 -attr mode=8023ad hash_mode=src_dst_port ent9 Available

Steps to add a new VLAN ID (ie. 500) to the User Ethernet network Add new VLAN ID to the Cisco Switch config: switchport trunk allowed vlan 100,500 On each VIO Server run command: $ mkvdev -vlan ent14 -tagid 500 On each VIO client LPAR use DLPAR to add a new virtual ethernet adapter with VLAN ID=500 and configure an IP address on the new ethernet interface

$ mkvdev -sea ent8 -vadapter ent10 -default ent10 -defaultid 1 -attr ha_mode=auto ctl_chan=ent12 ent14 Available $ mkvdev -sea ent9 -vadapter ent11 -default ent11 -defaultid 2 -attr ha_mode=auto ctl_chan=ent13 ent15 Available $ mkvdev -vlan ent14 -tagid 100 ent16 Available $ mkvdev -vlan ent15 -tagid 200 ent17 Available $ mktcpip -hostname

-inetaddr X.X.X.X -interface en18 -netmask X.X.X.X -gateway X.X.X.X