XOS 9.5.4.0 Xos Configuration Guide

XOS Configuration Guide
Version #: XOS 9.5.1
Part Number 03392P March 2011
Copyright and Trademark Information

Copyright 2011 by Crossbeam Systems Boxborough, MA, USA All Rights Reserved The products, specifications, and other technical information regarding the products contained in this document are subject to change without notice. All information in this document is believed to be accurate and reliable, but is presented without warranty of any kind, expressed or implied, and users must take full responsibility for their application of any products specified in this document. Crossbeam Systems disclaims responsibility for errors that may appear in this document, and it reserves the right, in its sole discretion and without notice, to make substitutions and modifications in the products and practices described in this document. This material is protected by the copyright and trade secret laws of the United States and other countries. It may not be reproduced, distributed, or altered in any fashion by any entity (either internal or external to Crossbeam Systems), except in accordance with applicable agreements, contracts, or licensing, without the express written consent of Crossbeam Systems. For permission to reproduce or distribute please contact your Crossbeam Systems account executive. This product includes software developed by the Apache Software Foundation: http://www.apache.org. Crossbeam, Crossbeam Systems, X-Series, XOS, X20, X30, X45, X60, X80, X80-S, and any logos associated therewith are trademarks or registered trademarks of Crossbeam Systems, Inc. in the U.S. Patent and Trademark Office, and several international jurisdictions. All other product names mentioned in this manual may be trademarks or registered trademarks of their respective companies.
Contents
About This Guide
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cautions, Warnings, and Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IPv6 Address Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 11 12 12 13 14 16
Chapter 1: Introduction
Hardware Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Module Link Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Application Processor (VAP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Greenlight Element Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated Workflow System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Greenlight Element Manager (GEM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Element Management System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 18 18 18 19 19 19 20 20 21 21 22 23 23 23 24
Chapter 2: Initial Configuration

Configuration Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Before You Begin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the Interview Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Working with Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Validating the Running Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Saving the Running Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing Default Usernames and Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing Default SSH Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 26 26 27 28 29 29 30 30 31
Chapter 3: Configuring System Settings

Configuring Basic System Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the Host Name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the IP Domain Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the Control Network IP Address and Subnet Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying the Current Control Network and Subnet Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the System Identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the System Time Zone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setting the System Calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling the Greenlight Element Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Facility Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Access Facility Alarm Settings in the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring and Displaying the System Operational Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring an NTP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a DNS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33 33 34 34 34 35 35 35 35 36 36 37 37 38
3
Chapter 4: Configuring Administration Settings

Managing User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a User Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the CPM Root Password and Expiration Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Root Expiration Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Root Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recovering from an Expired CPM Root Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Web Server SSL Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing the Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling SSH Access to VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 39 41 41 41 41 42 43 43 43 44
Chapter 5: Configuring VAP Groups

Overview of VAP Groups in XOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Number of VAPs in a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Master VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Master VAP Reselection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VAPs and Serialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VAPs and APMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Standby VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IPv6 Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jumbo Frame Support and VAP Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scatter Gather and Fragmented Ethernet Frame Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RAID Capability and Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VAP Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VAP Backup and Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure VAP Group for Ethernet Jumbo Frame Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mapping a Management Circuit to a VAP Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of IPv6 Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IPv6 Circuit Considerations and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Supported IPv4 to IPv6 Tunnels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VAP Group Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Environment VAP Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Additional Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure a Virtual Environment VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying VAP Group Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying VAP Group Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 46 46 46 46 46 47 47 47 47 47 47 47 48 48 49 49 50 51 51 51 52 52 55 58 58 59 59 60
Chapter 6: Configuring Interfaces

Data Interface Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Physical Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Logical Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interface Internal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Network Device (VND) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Domains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Group Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redundant Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VND Tap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Interfaces on the NPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents
61 62 62 62 62 63 63 65 65 65 66 66 66 68
4
Packet Mirroring on the NPM Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Access Control Lists (ACL Interface) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . configure acl-interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Access Control Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Chassis Resource Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resource Protection Workflow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disabling Flow Table Limit Action per Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Redundant Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a Multi-Link Group Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
73 73 73 75 78 81 82 83 85
Chapter 7: Configuring Management Interfaces

Physical Management Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Management Physical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Access Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Define an Access List to a Management Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Display Management Access Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 90 91 94 94
Chapter 8: Configuring Flow Provisioning

Flow Rule Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 VAP IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 VAP Non-IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Default VAP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 System IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Interaction Between System and VAP IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Multi-VAP Group IP Flow Rule Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 System Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Priorities for IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Configuring IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Configuring Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Configuring System IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Troubleshooting IP Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Configuring System Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Chapter 9: Configuring Protocols

Configuring Static IP Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring ARP Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying ARP Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Neighbor Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying Neighbor Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a RADIUS Server Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring an LDAP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the LDAP Version Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring RSA SecurID to Authenticate Administrative Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring RSA SecurID to Authenticate Remote Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integrating an ACE/Server with VPN-1/FireWall-1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring ACE/Server in a VPN-1/Firewall-1 Clustered Environment . . . . . . . . . . . . . . . . . . . . . Configuring an ACE Server to Work with a High Availability Cluster . . . . . . . . . . . . . . . . . . . . . . . . 107 109 109 109 110 110 111 111 112 113 113 113 114
Chapter 10: Configuring Bridge Mode

Configuring Bridge-Mode Bridges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Additional Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Circuit Restrictions for Bridges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VLAN Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuration Using Multi-Link Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 117 117 117 118 119
Chapter 11: Configuring CPM Redundancy

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CP Redundancy Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CP Redundancy Hardware Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CP Redundancy and RAID Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CP Redundancy Runtime States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing CPMs and Configuring Redundancy in a New System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing a Second CPM into an Existing System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing CP Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying the CP Redundancy State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the CP Redundancy Administrative State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshooting CP Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the CBS PSI Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a Redundant CPM Management Virtual IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redundant CPM Management Virtual IP Address Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual IP Address Interaction during In-Service Upgrade (ISU) . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 122 122 122 123 124 125 125 125 126 126 127 128 128 128 128
Chapter 12: Configuring Multi-System High Availability

VRRP Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Failover Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Router (VR). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VRRP Priority. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OSPF Failover in Conjunction with VRRP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control Link Port (HA Port) and Management Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VRRP Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VRRP Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Control Link Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating an Active-Standby Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manually Force a VRRP Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . View VRRP Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example Dual Box High Availability Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example VRRP Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Complex Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring VRRP and Automatic ARP for NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 129 130 130 131 131 131 133 134 136 141 142 143 144 144 145 148
Chapter 13: Configuring Secure Flow Processing

What Is Secure Flow Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Serial Traffic and Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a Serial Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Serialized VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Secure Flow Processing on Layer 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Parallel Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 151 152 153 154 156
Chapter 14: Configuring RMON and SNMP Traps

Configuring RMON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . System Owned RMON Alarms and Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring RMON Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring RMON Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying RMON Alarms, Events, and Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying Disk Monitoring Event and Alarm Entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Simple Network Management Protocol (SNMP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring an SNMP Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SNMP MIB Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Allowing SNMPv3 User Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 159 160 160 161 162 163 163 164 166 167
Contents
Chapter 15: Using UNIX Commands on VAP Groups and Upgrading Applications
Executing UNIX Commands on a Designated VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Executing UNIX Commands on All VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Copying a File from the CPM to all the VAPs in a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Chapter 16: Viewing Performance, Statistics, and Status

Using GEM to View Module Information and Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Module Information and Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Disk Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Heartbeat Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Module Utilization and Load Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Interface Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Switch Data-Path (SDP) Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . View VRRP Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . show vrrp status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . show vrrp detail-status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Show Remote Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Flow Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . show flow active. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . show flow distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . show flow-path active. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 174 175 175 176 177 178 178 178 179 179 180 180 181 182
Chapter 17: Viewing Alarms, Events, and Logs

Viewing Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using GEM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring a Remote Syslog Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Message Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Alarm Details from the Message Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing Audit Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 186 187 190 190 191 192
Chapter 18: Upgrading XOS Software and Firmware

XOS Automated Workflow System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XOS shar Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Copying and Extracting the shar File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading with the Extracted XOS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Files Generated During the Software Upgrade Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading the XOS Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manually Upgrading Firmware on an NPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manually Upgrading Firmware on an APM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manually Upgrading Firmware On a CPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XOS Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Applications Supported for Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrade Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 196 196 197 199 200 200 202 202 204 205 205 208 209
Chapter 19: Configuring Routing Protocols

Routing Software Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Converting Static IP Routes From a Previous RSW Version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing a Routing Protocol on a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Routing Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Accessing NSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring FIB Retain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redistributing Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IPv6 Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 212 213 214 214 215 215 216 217
6to4 Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ISATAP Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IPv6IP Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GRE Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
217 219 221 222
Chapter 20: Backups, Upgrades, and Reinstallation

VAP Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Preparing for Safe Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manual Safe Upgrade Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading to CPM-8600 or APM-8600 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrade a CPM-8400 to a CPM-8600. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrade an APM-8400 to an APM-8600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a New Configuration Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reinstallation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Disk Partitioning Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 226 227 229 229 229 230 230 231
Appendix A: In-Service Upgrade

In-Service Upgrade (ISU) Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . In-Service Upgrade Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modes of In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understanding Optimal Batch Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sample Configuration Analyzer Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating In-Service Upgrade Batches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Executing a Sample In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Executing a Non-Interactive In-Service Upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aborting a Non-Interactive In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 233 234 234 234 235 235 235 236 240 240
Appendix B: RAID-Related Hard Drive Configuration and Repair

RAID Feature Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Additional Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prerequisites for RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Important RAID Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disk Drive and RAID Information for CPM-8600s, APM-8600s, and APM-8650s . . . . . . . . . . . . . . Disk Drive and RAID Information for CPM-9600s and APM-9600s . . . . . . . . . . . . . . . . . . . . . . . . . RAID Information for CPM-8600s Only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RAID Information for CPM-9600s Only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Important RAID Information for APM-8600s, APM-8650s, and APM-9600s . . . . . . . . . . . . . . . . . . Obtain Hard Drive Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using mdstat to Determine RAID-Related Drive Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error Situations and Error Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure RAID on an Existing Non-RAID System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding a Second Drive to a CPM-8600 and Enabling RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding Hard Drives to an APM-8600, APM-8650, or APM-9600 and Configuring RAID. . . . . . . . . Change the Existing RAID Setting on a CPM-8600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group . Configuring Non-RAID on a CPM-8600 with Existing RAID 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Detect and Replace Failed Drives on an Existing RAID System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Replace Failed Drive on a CPM-8600 or CPM-9600 Configured for RAID 1. . . . . . . . . . . . . . . . . . Replace Failed Drive on an APM-8600 with RAID 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Replace Failed Drive on an APM-9600 with RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured . . . . . . . . . 242 242 242 243 243 243 243 243 244 244 244 247 248 248 249 250 251 251 253 253 254 254 255
Appendix C: Swatch Dynamic Display Tool

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Understanding the Swatch Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 Performance Monitoring with the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Contents 8
Starting and Stopping the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using Single Key Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swatch Tool Configuration Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a Swatch Tool Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swacquire Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swread Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swcalc Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swaggr Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swscreen Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the swdisplay Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Logging Screen Data to a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using Predefined TCL Variables in a Swatch Tool Configuration File. . . . . . . . . . . . . . . . . . . . . . . Using Swatch Tool Variables in All Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using Existing Swatch Tool Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swatch Tool Child Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Single Shared-Memory Region and Semaphore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshooting the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Not Being Displayed or Updated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data is Not Displayed in the Correct Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swatch Tool is Affecting CPM Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swatch Tool is Affecting Performance of Remote APMs or NPMs . . . . . . . . . . . . . . . . . . . . . . . . .
259 259 260 260 260 261 261 262 262 262 263 263 263 264 264 264 265 265 265 265 265 265
Appendix D: Crossbeam SNMP MIB Reference

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using a MIB browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using HP OpenView Network Node Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . New Crossbeam MIB Entries in XOS V9.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Greenlight Element Manager (GEM) MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XOS Alarms and SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CROSSBEAM-SYSTEMS-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the CROSSBEAM-SYSTEMS-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CROSSBEAM-SYSTEMS-MIB product entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-HARDWARE-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the CBS-HARDWARE-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-HARDWARE-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-HARDWARE-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-MODULE-RESOURCES-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the CBS-MODULE-RESOURCES-MIB OIDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-MODULE-RESOURCES-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-MODULE-RESOURCES-MIB Trap Entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VAPGROUP-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the CBS-VAPGROUP-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VAPGROUP-MIB entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VAPGROUP-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VRRP-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying the CBS-VRRP-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VRRP-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CBS-VRRP-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Standard MIBs on the Crossbeam X-Series Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 268 268 268 269 275 280 280 280 281 282 282 285 288 288 288 292 293 293 293 294 294 294 294 296 296
Appendix E: Maintenance Mode

Using Maintentance Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Contents
10
About This Guide

This guide provides step-by-step instructions for configuring and troubleshooting the X-Series Platform running XOS Version 9.5.x or later. For a complete software version compatibility matrix, refer to the XOS Release Notes for the specific XOS version that you are using. IMPORTANT: For the latest updates and revisions to X-Series Platform documentation, log into the Crossbeam Customer Support Portal at http://www.crossbeam.com/support/online-support/.
Intended Audience
This guide is intended for qualified service personnel responsible for installing, configuring, and managing software on Crossbeam X-Series Platforms.
Related Documentation
The following documents are provided on the Crossbeam Systems USB Installer (USBI) or are available on the Crossbeam Systems Customer Support Portal located at http://www.crossbeam.com/support/online-support/.
z z z z z z z z z z z
APM, CPM, and NPM Installation Notice X80-S Platform Hardware Installation Guide X60 Platform Hardware Installation Guide X20 and X30 Platform Hardware Installation Guide XOS Command Reference Guide Multi-Application Serialization Configuration Guide Multi-System High Availability Configuration Guide Install Server User Guide USB Installer User Guide RSW Installation Guide (available with the RSW kit, purchased separately) XOS V9.5.1 Release Notes
Configuration Guide
11
Conventions
Typographical Conventions
For paragraph text conventions, see Table 1 on page 12. For command-line text conventions, see Table 2 on page 13.
Table 1.
Typographical Conventions Used in Paragraph Text

Types of Information Elements on the graphical user interface. Usage Examples In the IP Address field, type the IP address of the first VAP in the group. Click OK to close the dialog. Select the Print to File check box.
Typographical Convention Bold
Courier
Keys on the keyboard. File names, folder names, and command names. Any information that you must type exactly as shown. Program output text.
Press Esc to return to the main menu. Save the user.txt file in the user_install directory. Use the start command to start the application. In the Username field, type Administrator. The XOS CLI show calendar command displays the system calendar: Fri Feb 25 13:32:03 2011
Courier Italic
File names, folder names, command names, or other information that you must supply. A sequence of commands from the task bar or menu bar.
In the Version Number field, type 8.5.patch_number.
>
From the taskbar, choose Start > Run. From the main menu, choose File > Save As... Right-click on the desktop and choose Arrange Icons By > Name from the pop-up menu.
About This Guide
12
Table 2.
Typographical Conventions Used in Command-Line Text

Types of Information User prompts and program output text. Usage Examples CBS# show calendar Fri Feb 25 13:32:03 2011 [root@xxxxx]# md crossbeam
Typographical Convention Courier
Courier Bold Information that you must type in exactly as shown. <Courier Italic> Angle brackets surrounding Courier italic text indicate file names, folder names, command names, or other information that you must supply. Square brackets contain optional information that may be supplied with a command. Separates two or more mutually exclusive options. Braces contain two or more mutually exclusive options from which you must choose one.
[root@xxxxx]# md <your_folder_name>
[]
CBS# configure dns server <IP_address> [vap-group <VAP_group_name>]
CBS# cp-unknown-state {cp1|cp2}
{}
CBS# configure vap-group <vap_group_name> CBS(config-vap-grp)# raid {0|1}
Cautions, Warnings, and Notes

Caution: Lists precautions that you must take to avoid temporary data loss or data unavailability. Warning: Lists precautions that you must take to avoid personal injury, permanent data loss, or equipment damage.
IMPORTANT: Lists important steps that you must perform properly or important information that you must take into consideration to avoid performing unnecessary work. NOTE: Provides special information or tips that help you properly understand or carry out a task.
Configuration Guide
13
IPv6 Address Notation

Within this guide, two notation types are used to represent IPv6 addresses.
z z
Standard Notation Compressed Notation
NOTE: XOS V9.5.x does not support mixed notation IPv6 addresses.
Standard notation
The standard notation for IPv6 addresses is eight 16-bit hexadecimal words separated by colons. For example: 2001:BA95:AC10:0000:CF6A:000D:2145:3713 Specifying leading zeros is not required, provided that there is at least one numeric value in each field of the address. The above address can also be represented as: 2001:BA95:AC10:0:CF6A:D:2145:3713
Compressed notation
Many IPv6 addresses contain multiple fields of zeros. Use the double colon (::) notation to represent a single contiguous group of zero fields within an IPv6 address. For example: 1458:0:0:0:0:A03:5:AC17 AF06:0:0:0:BC:0:0:4 0:0:0:0:0:0:0:1 0:0:0:0:0:0:0:0 These can be represented as: 1458::A03:5:AC17 AF06::BC:0:0:4 ::1 :: NOTE: The AF06:0:0:0:BC:0:0:4 example is represented as AF06::BC:0:0:4 because only one ::'' is allowed in an address. In this guide, the example IPv6 addresses are taken from:
z z
The Unique Local Unicast address range (FC00::/7) because these addresses are not propagated over the Internet The 2002: address range as part of the prefix for IPv6 6to4 tunnels
About This Guide
14
NOTE: In accordance with the IANA IPv6 address space allocations, an XOS VAP of type xslinux_v3 ("V3"), xslinux_v5 ("V5"), or xslinux_v5_64 ("V5_64") will treat some address ranges as reserved. At this writing, the IANA range assignments are as follows (excerpted from http://www.iana.org/assignments/ipv6-address-space/): IPv6 Prefix 0000::/8 0100::/8 0200::/7 0400::/6 0800::/5 1000::/4 2000::/3 4000::/3 6000::/3 8000::/3 A000::/3 C000::/3 E000::/4 F000::/5 F800::/6 FC00::/7 FE00::/9 FE80::/10 FEC0::/10 FF00::/8 Allocation Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Global Unicast Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Reserved by IETF Unique Local Unicast Reserved by IETF Link Local Unicast Reserved by IETF Multicast Reference [RFC4291] [RFC4291] [RFC4048] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4291] [RFC4193] [RFC4291] [RFC4291] [RFC3879] [RFC4291]
With regard to reserved address blocks, some kernel versions make address-type determinations that are different from those made by other kernel versions. It should be noted that, in particular, V3 VAPs treat the following ranges as strictly reserved (that is, these addresses are reserved and are not treated as Global Unicast addresses), whereas V5 and V5_64 VAPs treat these ranges as reserved Global Unicast addresses:
z z z z
0000::/8 0100::/8 0200::/7 0400::/6
For purposes of XOS circuit/interface configuration, it is recommended that the above "Reserved by IETF" ranges NOT be used unless warranted by special circumstances. In the general case, user-assigned XOS IPv6 unicast circuit/interface addresses should be allocated from the 2000::/3 (Global Unicast) and/or the FC00::/7 (Unique Local Unicast) address block(s).
Configuration Guide
15
Customer Support
Crossbeam Systems offers a variety of service plans designed to meet your specific technical support requirements. For information on purchasing a service plan for your organization, please contact your account representative or refer to http://www.crossbeam.com/support/technical-support/. If you have purchased a Crossbeam Systems product service plan and need technical assistance, you can report issues by telephone: United States: +1 800-331-1338 OR +1 978-318-7595 EMEA: + 33 4 8986 0400 Asia Pacific: +1 978-318-7595 Latin America: +1 978-318-7595 You can also report issues via e-mail to support@crossbeam.com. In addition, all of our service plans include access to the Crossbeam Customer Support Portal located at http://www.crossbeam.com/support/online-support/. The Crossbeam Customer Support Portal provides you with access to a variety of resources, including Customer Support Knowledgebase articles, technical bulletins, product documentation, and release notes. You can also access our real-time problem reporting application, which lets you submit new technical support requests and view all your open requests. Crossbeam Systems also offers extensive customer training on all of its products. For current course offerings and schedules, please refer to the Crossbeam Education Services Web pages located at http://www.crossbeam.com/support/training-services/.
About This Guide
16
1
Introduction
The X-Series Platform running the XOS software is an open networked application server designed to deliver enhanced application services while providing high-performance and availability. The modular design of the X-Series Platform allows it to run multiple applications, while providing multi-gigabit throughput performance. The XOS software, using secure standard interfaces with multiple levels of controlled access, enables cost effective integration of the X-Series Platform into the overall operation of your network. This chapter provides an overview of the various features and functions of the X-Series Platform.
z z z z z z z z z
Hardware Overview on page 17 Virtual Application Processor (VAP) on page 19 Traffic Flow on page 19 Networking on page 20 High Availability on page 20 Application Overview on page 21 Greenlight Element Manager on page 21 Automated Workflow System on page 22 Management Options on page 23
Hardware Overview
There are various XOS hardware platforms for different business needs. Each hardware platform contains three types of modules:
z z z
Network Processor Module (NPM) for packet reception, classification and load balancing. Control Processor Module (CPM) maintaining overall system configuration, management and integrity. Application Processor Module (APM) for hosting applications and network services that process packets belonging to individual flows.
These modules contain standard network interfaces including 10/100/1000 Ethernet, and RS-232 ports. When populated with two CPMs, two to four NPMs, and two to ten APMs, the X-Series Platform operates without a single point of failure. The X-Series Platform also supports multiple, redundant power supplies (1200W or 2700W). For more information about power supply compatibility, refer to the X-Series AC Power Supply Installation and Replacement Guide.
17
Network Processor Module

The NPM provides network connectivity, and handles the flow of traffic into and out of the system. The X-Series Platform supports up to four NPMs. The NPM uses both a network processor and a microprocessor. The NPM-8650 and the NPM-8650-BG support Gigabit Ethernet and 10 Gigabit Ethernet interfaces. The NPM-8620 supports only Gigabit Ethernet interfaces. The ports are located on the front panel of the module. The NPM makes load balancing decisions based on the following feedback loop: Each APM runs a flow agent (cbsflowagentd) that collects OS performance information from the /proc file system, such as /proc/uptime (how long the system has been up since the last reboot) and /proc/net/dev (network information). The load agent communicates with the load calculator on the CPM. The CPM flow calculator (cbsflowcalcd) uses the information from the load agent to compute the scheduling vector (a table that assigns weights to each APM). The CPM sends the scheduling vector to the NPM once every second. The NPM uses this scheduling vector to determine how to load balance the traffic to each VAP.
Control Processor Module

The CPM provides all the generic system-wide functions, including:
z z z z z z z z z
Switched Ethernet control network for all slots in the chassis. 10/100/1000 external Ethernet management port, logging port, console port, and chassis interconnect port (for dual-box high availability). Management of the system boot process. Management of insertion or removal of new modules into the X-Series Platform. Software upgrade. Service provisioning. CP redundancy. RAID data protection or hard drive maximization. Management of alarms.
The X-Series Platform supports up to two CPMs, with both CPM slots providing identical control plane connectivity. The CPM designated as the Primary CPM can manage and support all NPMs and APMs in chassis. When two CPMs are present, one module remains in a standby (secondary) state until needed, which provides CP redundancy. The CPM communicates to the other modules through a dual-redundant, private, switched control plane. The switching elements for the control plane are housed on the CPM, with all the point-to-point connections from each of the other slots in the chassis arriving at the CPM through the backplane connector.
Application Processor Module

The APM completes the application level processing of traffic flows. All traffic is load balanced between the APMs by the NPM. An APM processes the incoming network traffic and sends all outgoing traffic to the NPM. The APM contains a high-performance processor and the associated hardware needed for network flow processing. Depending on the hardware model, the X-Series Platform supports up to ten APMs and can provide APM redundancy when two or more modules are present. For APMs with two hard drives installed, the RAID options support data duplication or enhanced disk drive efficiency. When an APM is inserted into the chassis, it automatically boots and sends DHCP queries to get an IP address. Once this is received from the CPM, the system retrieves a kernel using TFTPBoot. The APM boots this kernel then mounts its file system using NFS. Optionally, the APM can have a hard disk drive for IDS and A/V engines.
Introduction
18
Module Link Status

Each module in the chassis maintains a link with the other modules. The state of this unidirectional link is based on the number of heartbeat messages received over time. The CPM maintains the state of all connections in the system. The APM and NPM keep track of connectivity from remote modules to themselves. Heartbeats are sent four times a second. If no heartbeats are received within one second, the link is declared down. Otherwise, the link state number (0 to 100%) represents how many heartbeats on average have been received within a 4-second interval. A link state of 0 indicates the link is down; 100% indicates the link is fully operational. Heartbeats have their own Ethernet protocol ID; they are not sent as IP packets. There are two types of heartbeat packets:
z z
Hellos: Broadcast tracking states of links interconnecting modules. Election: Unicast between two CPMs to negotiate which CPM is to become the primary.
Use the show heartbeat command to view the heartbeat status.
Virtual Application Processor (VAP)

A Virtual Application Processor (VAP) is an application operating environment that is run on an APM. A VAP consists of the operating system, the system software, and an application. A VAP group is a collection of VAPs configured with identical policies running the same application, grouped together to provide redundancy and increased throughput. The VAP group modules act in concert with one another as a virtual processing engine. The NPM load balances packet flows across the group based on a feedback loop from the APMs and CPMs. The assignment of VAPs to physical APMs is controlled using several parameters. The number of VAPs in the group is set using the vap-count parameter. Additional capacity is achieved by adding APMs and increasing the VAP count. The CPM tracks which VAPs are currently loaded, compared to the VAP count. The operating system and file structure for each VAP is stored on the CPM. To run an application, the APM loads the appropriate image to its memory. This allows for faster reload and reboot when preemption is enabled. The CPM also works in conjunction with the VAPs to provide load balancing information to the NPMs.
Traffic Flow
The X-Series Platform dynamically creates internal flow paths for each serviced flow. On the arrival of the first packet of a new flow, a new path is created based on the flow rule associating user traffic with provisioned application services. All subsequent packets of the flow follow the same path through the system and are processed by the same set of applications. The X-Series Platform can parallelize or serialize traffic. As shown in Figure 1, parallelization is copying a traffic flow to one or more passive applications. For example, traffic can be passed to a Firewall and IDS simultaneously. This eliminates the need for external taps or SPAN ports. You configure parallelization by mapping a circuit to two VAP groups. Also shown in Figure 1, serialization is defined as moving traffic through two or more active applications. For example, passing a flow first to an intrusion protection application then to a Firewall. Serialization is configured with the use of an internal circuit. Flow rules determine how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, configured by the user.
19
Figure 1.
Example of Traffic Flow

XOS System VAP Group 1
VAP VAP VAP
Physical Port
Traffic Logical
Pa
Circuit
ra
lle
ra lT
c ffi
VAP Group 2
VAP VAP VAP
Parallel Traffic
NPM
Serial Traffic
VAP VAP VAP
VAP Group 3
Networking
The X-Series Platform supports these networking features:
z z z z z
Bridge mode for firewall and IDS/IPS Virtual Local Area Network (VLAN) IP static routes Equal Cost-MultiPath (ECMP) support for static routes Dynamic routing protocols, BGP, OSPF, OSPFv3, RIP, RIPNG, PIM-SM, and IGMP; however, these protocols require the optional Routing Software (RSW) kit (purchased separately)
High Availability
The X-Series Platform provides a number of features to ensure that the system remains operational. The features include:
z z z z
CP Redundancy. Each chassis contains two CPMs, where one is Active and the other is Standby. Interface Redundancy. A physical interface can be configured as a backup to another physical interface. Failover occurs quickly and is transparent to the application. LACP (Link Aggregation Control Protocol). Multiple physical interfaces can be combined into one interface. A failure of one physical link does not prevent traffic flow. VAP Load Balancing. Each VAP corresponds to an APM. In a VAP group, each VAP runs an instance of the application, and traffic is load balanced among the VAPs. Should an APM fail, the system automatically redistributes the traffic flow among the remaining VAPs. VAP Load Balancing with Synchronization. State synchronization may be necessary when switching a flow from one VAP to another while stateful packet processing is being performed. Using Check Point FireWall-1 synchronization, when a flow is reassigned to another VAP within the group, that VAP will have the proper FireWall-1 connection state to process the arriving packets. There may be a delay in synchronizing the new connection state to other VAPs so some packets may be dropped by the Firewall on the new VAP until the connection state is created.
Introduction
20
z z z
Standby VAP. An unused VAP is configured as Standby. Should a VAP in any VAP group fail, the standby VAP automatically joins that group. APM Preemption Mode. VAP groups are assigned a priority. Should a higher priority VAP group have a VAP failure, a VAP from a lower priority application is taken and reloaded into the high priority group. Multi-System High Availability. Using VRRP (Virtual Router Redundancy Protocol), two or more X-Series Platforms can be configured to provide High Availability (HA).
Application Overview
The X-Series Platform supports these basic application configurations:
z z
Multiple instances of the same application on different VAP groups. For example, you can have many distributed firewalls and have distinct policies in each instance. One VAP group can span multiple Application Processor Modules (APMs). The flow classification and distribution allow for even load balancing across all VAPs, thereby increasing the performance of a single application. Multiple applications can be configured to run concurrently. The multi-application configurations can use the Check Point OPSEC APIs to allow configurations of Check Point and one or more OPSEC-compliant applications. For example, Check Point FW-1/VPN-1 can be configured to use the Squid and Websense URL filtering applications using the Check Point OPSEC APIs. These applications would run in separate VAP groups. Other non-OPSEC compliant multi-application scenarios are supported as well. Multiple applications can be configured to run in series. Multiple, different applications can be linked through an interface-internal, allowing traffic to be inspected by each application. Traffic is passed from one application to the next, allowing for multi-layered, in-depth inspection by best-of-breed security applications within a single X-Series Platform.
Security applications are provided separately.
Greenlight Element Manager

The Greenlight Element Manager (GEM) provides a view into the components and health of your X-Series Platform. GEM is a Web application that displays system and component information in an easy to reference format. GEM provides system health monitoring capabilites only; future releases will incorporate additional tools and capabilities. Individual views provide module status, load averages, flow rates, VAP group information, application data, system alarm information, DBHA and failover group status, and more, in a single glance. The tabbed interface can be customized to show specific module information or system historical data. Detailed user assistance provides additional information about the views and steps to add or customize the display for your specific needs. The Greenlight Element Manager application has the following client system requirements:
z z z
2 Ghz, dual-core processor, with 2GB RAM Minimum screen resolution of 1024x768 Microsoft Windows 7, Vista, or XP

with Internet Explorer 7 or 8 with Firefox 3.0 or 3.5
z z
Apple Mac 10.5 with Firefox 3.0 or 3.5 Linux with Firefox 3.0 or 3.5
21
To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser. https://<ip_address>
Third Party Pop-ups on GEM

Third party pop-up windows may appear while using GEM. The appearance of pop-ups can be controlled on individual browsers, and are based on user preferences. Users running GEM on a Microsoft Windows system using Internet Explorer may see content related to Microsoft's Windows Live toolbar add-on. Specifically, when attempting to create an APM view, a "Get an instant stock quote" button may display. Use the following workaround to prevent the appearance of these pop-ups. Disable the Windows Live add-on as follows: 1. 2. 3. 4. In Internet Explorer, select Tools --> Manage Add-ons. The Manage Add-ons dialog box appears. For each of the Show views, select all items beginning with "Windows Live" and then click "Disable". Click OK to close the dialog box. Restart Internet Explorer.
Automated Workflow System

The Automated Workflow System (AWS) is a menu driven, command line interface tool, designed to simplify the process of updating XOS versions, performing firmware upgrades, and viewing the current status of your X-Series Platform. To use the AWS, start the Automated Workflow System from the command line: CBS# automated-workflow-menu Welcome to the X-Series Platform Automated Workflow System! Version: 1.1.0-x 1. 2. 3. 4. 5. Configure XOS... Upgrade XOS software and firmware... View system configuration and status... Applications... Custom...
Select a submenu to view available automated workflows. Enter x to exit or ? for help. Please Enter Selection: Use the question mark to access the help on any of the menu or submenu commands.
Introduction
22
Management Options
XOS provides the following system management options:
z z z
Greenlight Element Manager Command Line Interface (CLI) Element Management System (EMS)
A user account has CLI, GEM, AWS, and EMS access. However, the level of access for CLI and EMS is configured differently. XOS supports Access Control Lists (ACLs) for the Ethernet management port.
Greenlight Element Manager (GEM)

GEM is a Web application that displays system and component information in an easy to reference format. To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser. https://<ip_address>
Command Line Interface

The Command Line Interface (CLI) is accessed locally using an RS-232 serial connection on the CPM. The CLI can be accessed remotely by Telnet or SSH through the Ethernet management port on the CPM. You can log into the system as a system administrator, which accesses the system commands, or as a UNIX root user, as shown in Figure 2. Although some UNIX commands are used, generally, configuration changes using the UNIX environment are not recommended or supported. You can assign a user a CLI access of 0 to 15, where 15 is the highest level of access. The access levels are used to restrict users to specific sets of commands. Users at a particular access level can access any commands whose level is at or lower than their user level. For example a user at level 6 can access any commands whose level is 0 to 6. Each CLI command is also assigned an access level. All CLI commands are set at either level 0 or 15 by default. Most of the configuration commands are set at level 15 while most show commands are at level 0. For a description on using the CLI, refer to the XOS Command Reference Guide.
Figure 2.
Login as Admin or Root

Login: admin root
CLI Prompt CBS#
unix su
Unix Prompt [root@xxxx admin]# rsh fw_1 rsh fw_2 rsh fw_3
VAP Unix Prompt fw_1(xxxx): /root$
23
Element Management System

The Element Management System (EMS) is a Java based application that offers a browser-based interface for configuring and monitoring the X-Series Platform. The XOS software and server reside on the CPM and are accessed through the CPM management interface. All communications are XML-based, and secured through SSL. Using EMS, you can define, configure, and monitor X-Series Platforms, applications, and users throughout your network. Using the EMS client, you can:
z z z z z
Configure and monitor the NPM, APM, and CPM Configure and monitor circuits View information about modules Maintain user accounts and access levels View performance statistics
EMS requires Java Runtime Environment (JRE) 1.6.x or higher on the client system. If you are using Internet Explorer as your browser, you will be automatically taken to Sun's web site to download JRE the first time you start EMS. Otherwise, you can download it manually from http://java.sun.com. NOTE: EMS cannot be used to install applications. This can only be done using the CLI. Also, EMS cannot configure offline modules; this must also be done using the CLI. EMS access levels differ from the CLI access levels. Each user account is assigned a specific role, restricting the user to view and configure only certain windows within EMS. EMS is displayed in a browser window that directly connects to a web server running on the X-Series Platform. Multiple X-Series Platforms can be managed from a workstation by using separate browser windows. Communication between the workstation and each individual X-Series Platform uses an HTTPS-based private interface and exchange XML-formatted data. To access EMS, configure the Web server during the XOS software installation, or from the command line after install. NOTE: The log on process to EMS has changed. Enter the IP address of the X-Series Platform followed by /ems (i.e. https://<ip_address>/ems) into your browsers address field. The user name and password are case sensitive. The connection between the X-Series Platform and a dedicated workstation is made secure by the Secure Socket Layer (SSL). SSL provides authentication, integrity, and security between a browser and the XOS web server. The EMS window contains three sections:
z z z
The menus in the top frame provide access to administrative functions, configuration, and other tasks. The navigation tree in the left frame displays folders to open windows for configuring, provisioning, and monitoring the system. The work space in the right pane displays information windows about a folder or component you select in the tree view. For example, when you click on the System Configuration folder, a window is displayed showing configuration properties for the system.
Introduction
24
2
Initial Configuration
When you log in to the X-Series Platform for the first time, the system runs the Interview program. This program prompts you to configure various parameters. The Interview does not run on subsequent startups, unless you reset the system to factory defaults. By default, DHCP and SSH are enabled for quick access and initial setup. This chapter contains the following topics:
z z z z
Configuration Overview on page 25 Using the Interview Process on page 28 Changing Default Usernames and Passwords on page 30 Changing Default SSH Keys on page 31
Configuration Overview
The basic steps to configure the X-Series Platform are as follows: 1. 2. Gather information about your network and security applications. Use the Interview program to configure the basic parameters, such as:

CPM management IP Address and subnet mask Activate the Web Server to provide access to Greenlight Element Manager and EMS Create and configure user accounts Create and configure Access Control Lists (ACLs) Internal IP network address and subnet mask NOTE: The internal network IP address must be unique to your network. CPM default gateway and host name Time server IP address System Identifier, which is a unique value from 1 to 255 given to each X-Series Platform Operating mode
3. 4. 5. 6.
Create Virtual Application Processors (VAPs). Define circuits, which are interfaces to the VAP. Define the physical interfaces and map them to the circuits. Define the IP and non-IP flow rules.
The following configuration steps can be performed in any order:

z z z z
Load and configure the various applications. Configure facility alarms. Configure the routing features, such as IP static routes and their cost, and the optional routing protocols (OSPF, OSPFv3, RIP, RIPNG, BGP, and PIM-SM). Configure High Availability. This can include single or multi-box redundancy, CPM redundancy, and redundant interfaces.
25
Before You Begin

Before beginning the software installation or configuration of your X-Series Platform, gather the following:
z z
Network topology information Application information
Network Topology
Gather or create the following network topology information: 1. 2. Create a diagram containing as much information as possible about your network topology. List all known IP, VLAN, and media information.
Table 3.
NPM
Network Topology Information

Interface Port Number IP Address Network Mask Interface Type VLAN Information Circuit Name
3.
Specify the VAP group default gateway route for all XOS VAP groups.
Table 4.
VAP Group Default Gateways

Default Gateway
VAP Group Name
26
4.
Specify all static routes to be configured on the X-Series Platform.
Table 5.
Static Route Specifications

Next Hop IP Address
Destination Network
5.
Specify information about other devices that are directly connected to your X-Series Platform. Add them to your diagram and supply their IP and/or VLAN information if applicable.
______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________
6.
Specify all information about remote access to your Security Service Switch. For example:

Hostname/IP Address Username Password
Application Information
Gather the following application specific information: 1. Specify the platform you are using, for example:

Windows Linux Solaris IPSO HP-UX AIX
27
2.
Specify the intended VAP group names used with the applications.
Table 6.
Applications and Their VAP Group Names

VAP Group Name
Application
3.
Specify all keys for the various applications. Application Key
Using the Interview Process

When you start the X-Series Platform for the first time, log in as an administrator (Login ID and Password are both admin). The system automatically runs the Interview program. This program prompts you to configure a number of parameters. The Interview program does not run on subsequent startups, unless you reset the system by erasing the database and rebooting the system. Follow the Interview program to configure the basic parameters, including:
z z z z z z z
CPM management IP Address Subnet mask System Identifier Interfaces Internal IP network address and subnet mask CPM default gateway and host name Time server IP address
28
These features can also be configured using the Command Line Interface (CLI) or the graphical Element Management System (EMS) utility. Each feature is described in detail in the appropriate chapter of this configuration guide. Refer to specific chapters as necessary. Some reasons you would exit the Interview program:
z z
To configure VAP groups and circuits. To import an existing configuration file using FTP.
If you choose to exit the Interview program, you can refer to the Configuration Overview on page 25 as a guideline to complete the system configuration.
Working with Configuration Files

On a running system, there are two copies of the X-Series Platform configuration file: Startup Configuration The system startup configuration file is applied anytime the system boots or reboots. Running Configuration This is the active (running) configuration. This is a copy of the base Startup Configuration with additional configuration changes made since the last system boot or save of the running configuration to the startup configuration. IMPORTANT: When you save configuration changes, the changes are saved to the Running Configuration. The changes are not applied to the Startup Configuration. When the system reboots, the changes are not seen unless you copy the Running Configuration to the Startup Configuration prior to reboot. You can save X-Series Platform configuration files locally to the CPM disk as text files consisting of CLI commands. You can copy the Startup or Running configuration to a named configuration file for later analysis and comparison. However, direct booting from a named configuration file is not allowed. Configuration files include sections for each subsystem (system, module, interface, mappings, and services). The Running Configuration Validation feature allows you to instantly review system configuration settings. You can perform the following tasks with configuration files:
z
Display the running and startup configuration files as follows: show running-config show startup-config
NOTE: If you are using EMS, use the System Config menu to display the running or startup configuration file.
z z z
Display the running configuration validation Copy the running configuration to the startup configuration or to a file Copy the startup configuration to a named file
Validating the Running Configuration

To run the validation using the CLI, enter the following command: CBS# validate-configuration In EMS, use the Running Configuration Validation window to instantly review system configuration settings. For example, messages are supplied if no logical interface is configured for a given physical interface. To run the validation using EMS, go to the System Config menu and select Show Running Config Validation. The Show Running Configuration Validation window is displayed.
29
Saving the Running Configuration

The running configuration can be saved to either the startup configuration or a designated file. If you are using EMS, you can copy the running configuration to the startup configuration from the System Config menu. If using the CLI, use the following command: copy running-config startup-config You can save the startup or running configuration to a designated file for later analysis and comparison. Direct booting from the designated file is not allowed. To save the running configuration to a designated file from EMS, use the System Config menu. At the CLI, use one of the following commands: copy running-config <path-file> [-flat] [echo-password] [-sort] [include-default] copy startup-config <path-file> [-flat] [echo-password] [-sort] [include-default] Where <path-file> is the desired path and file name for the copied running configuration, -flat copies CLI commands with complete context, echo-password includes the encrypted passwords, -sort sorts VAP groups by vap-group-name in the vap-group section and sorts circuits by circuit-name in the circuit section, and include-default includes parameters that still have their default values. For example: copy running-config backup-file
Changing Default Usernames and Passwords

The X-Series Platform has the following default usernames and passwords. For security, you should change them.
z
To access the Command Line Interface (CLI), the username is admin, and the password is admin. From the admin account, use the following command to change the password. You are prompted for a new password. CBS# configure password
NOTE: The admin user can be deleted using the configure no username admin command. Before deleting the admin user, make sure that another user name with the same privileges exists. If the admin user is deleted, this command appears in the running configuration file after an XOS upgrade to prevent the admin user from being recreated by default.
z z
To access the X-Series Platform from the EMS web server, the username is admin and the password is admin. Use the configure password command to change the password. The GRUB password is MD5-encrypted. The password to access GRUB is x40rocks. Perform the following to change the password: a. b. Log into the system as root which will place you at the Linux prompt. Run /sbin/grub-md5-crypt to create the MD5-crypted password, for example: [root@xxxxx root]# /sbin/grub-md5-crypt Password: $1$LiTbt0$30FLNL0TWxFWWyBBhoYw6. c. d. Edit the /boot/grub/grub.conf file and replace the old MD5-crypted password with the new MD5-crypted password string. Save the /boot/grub/grub.conf file.
NOTE: When upgrading GRUB, the password is preserved.
30
Changing Default SSH Keys

The X-Series Platform provides default SSH keys. These should be changed as follows: 1. 2. Log into the system as root. Generate a new RSA1 key: /usr/bin/ssh-keygen -q -t rsa1 -f /etc/ssh/ssh_host_key -C '' -N '' The files /etc/ssh/ssh_host_key and /etc/ssh/ssh_host_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files. 3. Generate a new RSA key: /usr/bin/ssh-keygen -q -t rsa -f /etc/ssh/ssh_host_rsa_key -C '' -N '' The files /etc/ssh/ssh_host_rsa_key and /etc/ssh/ssh_host_rsa_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files. 4. Generate a new DSA key: /usr/bin/ssh-keygen -q -t dsa -f /etc/ssh/ssh_host_dsa_key -C '' -N '' The files /etc/ssh/ssh_host_dsa_key and /etc/ssh/ssh_host_dsa_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files.
31
32
3
Configuring System Settings
The main topics in this chapter include:
z z z z z
Configuring Basic System Settings on page 33 Configure Facility Alarms on page 36 Configuring and Displaying the System Operational Mode on page 37 Configuring an NTP Server on page 37 Configuring a DNS Server on page 38
Configuring Basic System Settings

When you first booted your X-Series Platform, you completed a configuration interview. This section describes how to change those settings using the Command Line Interface (CLI). Specifically, how to:
z z z z z
Change and display device and domain name Change the network and subnet mask addresses Change system information Change time zones Change external access ability
NOTE: To access the device settings using EMS, expand Configuration in the Navigation Tree and open Wizard. In the Configuration Wizard window, click Open next to Device Configuration. The following sections describe how to access the device settings using the CLI.
Defining the Host Name

The host name is the unique name given to the X-Series Platform during the initial system interview. For systems with two CPMs, each CPM (CP1 and CP2) has a host name. To define the host name, use the following command: configure hostname <name> [cp1|cp2] The name is an alphanumeric string, which may include characters such as dashes and symbols. Specify CP1 or CP2 to define a hostname for that CPM. To display the current hostname, use the following command: show hostname The following is an example of the show hostname command: CBS# show hostname CP1 Hostname: myhost1 CP2 Hostname: myhost2
33
Defining the IP Domain Name

The IP Domain Name is the name given to the X-Series Platform. The default domain name is crossbeam. Use the no parameter to restore this default setting. NOTE: To make a domain name searchable by Domain Name Servers (DNS), use configure dns search-name. To define the IP Domain Name, use the following command: configure ip domainname <name> The name is an alphanumeric string, which may include characters, such as dashes, dots, and symbols. The following is an example of the hostname command: CBS# configure ip domainname mycompany.com To display the current domain name, use the following command: show ip domainname
Defining the Control Network IP Address and Subnet Mask

The default value for the XOS system-internal-network is 1.1.0.0/16. IANA has recently (January 2010) allocated 1.0.0.0/8 to APNIC for use on the Internet. IMPORTANT: Be sure to reset the internal network value to an appropriate unused network, a private address range, or unallocated address blocks defined by IANA. This configuration change requires a chassis reload. XOS restricts IP routes whose destination network is the same or a more specific network match with the system-internal-network IP address. Traffic that matches this criteria will not pass through the X-Series Platform. To define the internal control network and subnet mask used by the X-Series Platform to communicate between devices, use the following command: CBS# configure system-internal-network {<ip-addr> <netmask>|<ip-addr/ netmask>} Where: <ip-addr> <netmask> <ip-addr/netmask> IP address for internal control network. Network mask for this internal control IP address. The last two bytes are ignored. Default is 255.255.0.0. IP address and network mask in CIDR format, for example, 2.2.0.0/16.
The following is an example of this command: CBS# configure system-internal-network 2.2.0.0 255.255.0.0 NOTE: The configuration of system-internal-network is restricted from Class D or E internal networks. An internal network configured as Class D or E will cause boot problems with the modules in the X-Series Platform. If the configured network is not unique in the system, an error message will display.
Displaying the Current Control Network and Subnet Mask

To display the current control network and subnet mask, use the following command: show system-internal-network
34
This command displays both configured and operational system-internal-network (IP and mask). Configured System Internal Network Configured System Internal Netmask Operational System Internal Network Operational System Internal Netmask (1 row) : : : : 2.2.0.0 255.255.0.0 2.2.0.0 255.255.0.0
Defining the System Identifier

To define a system identifier used when connecting multiple X-Series Platforms, use the following command: configure system-identifier <system-id> Where system-id must be unique per X-Series Platform. Valid range is 1 to 255. NOTE: If changes are made to the system identifier and/or internal network address, the X-Series Platform must be rebooted.
Defining the System Time Zone

To change the time zone specified during the configuration interview, use the following command: configure timezone <name> Where name is the case-sensitive time zone. If you hit enter without typing a timezone, a list of available timezones that you can choose from is displayed. This is an actual time zone, as specified in the configuration. The following is an example of the time zone configuration command. This command requires a reboot for the change to implemented. CBS# configure timezone America/New_York To display the current time zone setting, use the following command: CBS# show timezone Time Zone America/New_York
Setting the System Calendar

The system calendar runs continuously, even if the system is powered off or rebooted. To set the system calendar, use the following command: calendar <hh:mm:ss> <date> <month> <year> Where: <hh:mm:ss> <day> <month> <year> Current time in hours (military format), minutes, and seconds. Current day of the month (1 to 31). Current month, in three letter format. Current year (4 digits - no abbreviation)
The following example sets the system calendar to 1:32 p.m. on December 23, 2010: calendar 13:32:00 23 dec 2010 NOTE: This command will not run when the system is configured with an NTP server.
Enabling the Greenlight Element Manager

To enable the Greenlight Element Manager (GEM), use the following command: configure web-server
XOS Configuration Guide 35
To access the Greenlight Element Manager, use a secure connection (https://) to enter the system IP address or name in a Web browser. https://<ip_address>
Enable Access to EMS

Use the configure web-server command to enable access to the legacy EMS application. Use the show web-server command to display whether the Web server is enabled or disabled. To access EMS, use a secure connection (https://) to enter the system IP address or name in a Web browser followed by /ems. https://<ip_address>/ems
Configure Facility Alarms

Facility alarms monitor processor usage on each of the modules, and are divided into 3 categories: Minor, Major, and Critical. Each category has an upper and lower threshold, with an upper threshold default value. All threshold values can be set by the user. The following table lists the user-configurable facility alarms.
Table 7.
Facility Alarms:
Upper/Lower Threshold Default Values Minor Major Critical
Parameter
Description and Units
cpu cpu-core disk-usage-root disk-usage-boot disk-usage-mgmt disk-usage-cbconfig disk-usage-tftpboot disk-usage-var free-memory
CPU cycles, percent of capacity CPU core usage, percent of capacity /root partition disk usage, percent of capacity /boot partition disk usage, percent of capacity /mgmt partition disk usage, percentage of capacity /cbconfig partition disk usage, percent of capacity /tftboot partition disk usage, percent of capacity /var partition disk usage, percent of capacity APM free memory page count multiplier
80% / 0% 80% / 0% 70% / 0% 70% / 0% 70% / 0% 70% / 0% 70% / 0% 70% / 0% N/A / 4
90% / 0% 99% / 0% 90% / 0% 99% / 0% 80% / 0% 97% / 0% 80% / 0% 97% / 0% 80% / 0% 97% / 0% 80% / 0% 97% / 0% 80% / 0% 97% / 0% 80% / 0% 97% / 0% N/A / 2 N/A
There are additional facility alarms that are not user configurable, but provide output in the message logs. For information about these alarms and how to view the complete list, refer to the show alarms command in the XOS Command Reference Guide.
Access Facility Alarm Settings in the CLI

To access the facility alarms threshold settings using the CLI, use the configure facility-alarm command as described in the XOS Command Reference Guide. To display the current threshold alarm settings, use the show facility-alarm command.
Configuring System Settings 36
Configuring and Displaying the System Operational Mode

To configure the X-Series Platform operational mode, use the following command: configure operating-mode {single-np|dual-np|quad-np} {series-6} Where: single-np dual-np quad-np Single NPM mode system. Available on the X20, X30, X45, and X60 platforms. Dual NPM mode system. This is the default value for the X80-S platform, and optional on the X45 and X60. Quad NPM mode system. This is an optional mode for the X80-S platform.
To display the current operational mode, use the following command: CBS# show operating-mode Chassis Type Configured Operating Mode Operational Mode Configured NPM Mode Operational NPM Mode : : : : : X80 dual-np dual-np series-6 series-6
Configuring an NTP Server

The Network Time Protocol (NTP) is used to synchronize the time of a computer client or server to another server or reference time source, such as a radio or satellite receiver or modem. It provides accuracies typically within a millisecond on LANs and up to a few tens of milliseconds on WANs relative to Coordinated Universal Time (UTC) through a Global Positioning Service (GPS) receiver, for example. Typical NTP configurations utilize multiple redundant servers and diverse network paths in order to achieve high accuracy and reliability. Some configurations include cryptographic authentication to prevent accidental or malicious protocol attacks and some provide automatic server discovery using IP multicast. To access the NTP server settings using EMS, open Configuration > System Configuration > NTP Server in the Navigation Tree. The following sections describe how to access the NTP server settings using the CLI. To configure an NTP server address on the X-Series Platform, use the following command: configure ntp server <server-address> Where server-address is the IP address of the NTP server. The following is an example of this command: CBS# configure ntp server 172.16.31.101 To display the current NTP server configured on the X-Series Platform, use the following command: show ntp-server
37
Configuring a DNS Server

The Domain Name System (DNS) is a global network of servers that translate host names like www.mycompany.com into numerical IP (Internet Protocol) addresses, such as 24.62.13.19. The DNS server commands add servers in the order they are entered. The first DNS server added will be the first nameserver entry in /etc/resolv.conf, the next will be the second listed, and so on. These are equivalent to Windows Preferred DNS Server and Alternate DNS Server respectively. The search-name parameters correspond to UNIX search directive or Windows appended suffixes options. Multiple DNS entries for each can exist, and the entries can be different for CPM and VAP groups. To access the DNS server settings using EMS, open Configuration > System Configuration > DNS Server in the Navigation Tree. The following describes how to access the DNS server settings using the CLI.
z
To configure a DNS server address on the X-Series Platform, use the following command: configure dns server <server-address> Where server-address is the IP address of the DNS server. For example: CBS# configure dns server 172.16.31.101 To configure a DNS server address for a VAP group, use the following command: configure dns server <server-address> vap-group <vap-group-name> Where server-address is the IP address of the DNS server, and vap-group-name is the name of the VAP group associated with the server address. The following is an example of this command: CBS# configure dns server 192.168.30.81 vap-group fw To configure a DNS server search name for a VAP group, use the following command: configure dns search-name <name> vap-group <vap-group-name> Where name is the search-name for the specified VAP group, and vap-group-name is the name of the VAP group associated with the server address. The following is an example of this command: CBS# configure dns search-name mycompany.com vap-group fw To display the current DNS server configured on the X-Series Platform, use the following command: show dns-server DNS Server Address 192.168.30.81 192.168.30.81 VAP Group fw
To display the a DNS server search-name assigned to a VAP group, use the following command: show dns-search-name DNS Search Name mycompany.com VAP Group
38
4
Configuring Administration Settings
This chapter contains the following major administration setup topics:
z z z z
Managing User Accounts on page 39 Changing the CPM Root Password and Expiration Interval on page 41 Changing the Web Server SSL Certificate on page 43 Enabling SSH Access to VAPs on page 44
Managing User Accounts

Each system user is defined by the users name, password, and access level. Collectively, these properties define each users profile. Each user account has both CLI and web access. However, the access is configured differently. If you are a system administrator, you can:
z z
Create or modify a user profile. Delete a user profile.
The following sections describe how to perform these tasks using the CLI. To manage user accounts using the Element Management System (EMS), select Administration from the menu bar in the main window then select User Admin. From the Administration menu, you can also change your password, view system users, and view users logged in from the web.
Creating a User Account

To create or modify a user account using the CLI, complete the following: 1. Create a user name, using the following command: CBS# configure username <name> Where name is a case sensitive alphanumeric string. 2. 3. 4. If you are creating a new user, you will be asked to enter a password. The password is a case sensitive alphanumeric string. Retype the password to confirm it. Define the user CLI access level, using the following command: CBS# configure username <name> privilege <level> Where the level is the access level assigned to the user. Valid values are from 0 to 15, with 15 being the highest. The default value is 0.
39
5.
Define the user EMS access level, using the following command: CBS# configure username <name> gui-level <level> Where level is the access level assigned to the user. The predefined levels restrict the user to view and configure specific windows within EMS in the table below.
Table 8. EMS Access Levels

Level unauthorized guest network-operator service-operator administrator Perform Configuration No login Read only Yes Read only Yes Perform Flow Provisioning No login Read only Read only Yes Yes Perform Administration No login No No No Yes
The administrator level can view and configure all windows. Administration tasks include managing user accounts, upgrading software, and configuring system parameters. Configuration tasks include configuring the system, module, VAP groups, interfaces and protocols. Flow Provisioning tasks include managing rate limiters and IP flow rules. The following is an example of how to create an account named rudolph: CBS# configure username rudolph privilege 15 gui-level administrator Password : Retype Password: To delete a user account from the CLI: CBS# configure no username <name> To change a users password, use the following command, and enter the new password. CBS# configure reset-password <username> NOTE: You can use the enable level command to change your user access level for this session. This command may require a password for higher levels. The no parameter restores the default privilege level. To create, modify, or delete a user account from EMS, go to the Administration menu and select User Admin. Also use EMS to specify how long web users can remain logged in. If an individual web user does not request a new page within the specified number of minutes, the user session is cancelled and the user must log in again. This setting applies to all users who login to the X-Series Platform via HTTP. To specify the timeout value for all user web sessions, go to the Administration menu and select Web Session Timeout. The default value is 20 minutes.
40
Displaying User Accounts

The XOS software allows you to view accounts for a single user or all users. To view a single user account, use the following command: CBS# show username [name] To view all user accounts, use the following command: CBS# show usernames Using EMS, go to the Administration menu and select System Users or Web Users to view users logged in at the CLI or via HTTP, respectively.
Changing the CPM Root Password and Expiration Interval

By default, the CPM root password expires after 30 days. The password and expiration interval can be changed at the CPM root level. The procedure is different depending on whether the system uses GRUB as the boot loader. NOTE: The password and its expiration interval are independent of each other. On new installations of XOS V9.x (non-migration from 8.x, a password is required (default is admin).
Changing the Root Expiration Interval

To modify the number of days before the CPM root password expires, login to the CPM as root and enter the following command followed by the user name, at the root prompt: [root@xxxxx admin]# chage -M <XX> <username> Where XX is the number of days before the password expires for the named user. Use 99999 to set the password to never expire.
Changing the Root Password

To change the root password, use the following command: [root@xxxxx admin]# passwd <username> Changing password for user <username> New UNIX password: <password> passwd: all authentication tokens updated successfully [root@xxxxx admin]# The default username is root, and the default password is admin. For XOS V9.5, a password is required.
41
Recovering from an Expired CPM Root Interval

To recover from an expired CPM root interval, perform the following: 1. If the administrator account has not expired, reload the primary CPM as follows. The slot could be 6, 7, 13, or 14. CBS# reload module <slot> 2. 3. If the administrator account has expired along with the root password, you need to physically pull out and re-seat the primary CPM to reboot the system. Watch carefully for the GRUB menu.
NOTE: To use the backspace function in GRUB, enter CTRL-h. a. b. c. d. At the GRUB menu, type p on the keyboard then enter the password, x40rocks. At the GRUB prompt, type e to edit. Change the line number by pressing the down arrow until you reach the grub line starting with kernel. Edit the line to add single for single user mode. For example: grub edit> kernel /vmlinuz-x.x.xx-x ro root=/dev/hda7 console=ttyS0,9600n8 single e. f. 4. Type b to boot the CPM in single user mode. At the root prompt, you can use the passwd <password> command to change the password, or refer to Changing Default Usernames and Passwords on page 30 to change the GRUB password.
Shutdown and reboot the X-Series Platform, using the following command: [root@xxxx admin]# shutdown -r now
When the X-Series Platform reboots, use the vap-group-password command if you desire to propagate the new CPM root password to one or more of the VAPs.
42
Changing the Web Server SSL Certificate

Some web browsers, such as Internet Explorer Version 7, give a security certificate warning message when opening an EMS session. This occurs because the certificate for EMS is self-published. The warning message can be ignored. As an alternative, you can purchase or create a host-specific certificate from a trusted Certification Authority (CA) then install it as described in this section. The process of obtaining a certificate is defined by the third-party CA.
Overview
The XOS web server resides on Tomcat, which is an open-source implementation of Java Servlet and JavaServer Pages technologies. The web server provides a Secure Socket Layer (SSL) certificate that users must accept to access the XOS web server. If you choose to purchase a certificate, you can use a public CA such as VeriSign (at www.verisign.com), Thawte (at www.thawte.com), Baltimore (at www.baltimore.com), or various other public CAs. To control the certification process, you can become your own CA, which requires that you install certificate server software and issue your own certificates. When you submit a request for a server certificate from a CA, you can use the CAs web site to generate part of the certificate, such as the identification information, the public key, and the private key. You then submit the identification information and the public key to the CA for signing. If you generate the certificate request through a web browser, you submit a .PEM or .DER file that contains the encoded PKCS10 information, which is your identification information and the public key. Once you generate the certificate request, submit the request to the CA for signing. When the certificate returns, you can download it out of the directory the CA specified and copy it into a text editor. The signed certificate contains your identification information, your public key, and the CAs signature. This part of the certificate is still in .PEM or .DER format, but it is now encoded X.509 information. If you generated your certificate request in a browser, you can export a PKCS12 file from your browser, which includes your private key and your certificate. You export the file as a password protected PKCS12 file with a .p12 file extension (or a .pfx file extension if you export it using Internet Explorer). You can then import this file into the CA server. The procedure for getting your certificate signed depends on the CA you use and whether your organization generates and signs its own certificates. If you chose to use a public CA, go to the CAs web site to get a signed certificate. You can submit the certificate that you generated to your CA over the Internet. They will then contact you with information on how to get your signed certificate. For instructions on generating a new certificate and key, refer to the following web site: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/tomcat-ssl-howto.html#s6
Installing the Certificate

The SSL certificate parameters are stored in the /usr/os/web/conf/server.xml file. For example: <Connector className="org.apache.tomcat.service.PoolTcpConnector"> <Parameter name="handler" value="org.apache.tomcat.service.http.HttpConnectionHandler"/> <Parameter name="port" value="8443"/> <Parameter name="socketFactory" value="org.apache.tomcat.net.SSLSocketFactory"/> <Parameter name="keystore" value="/var/tomcat/conf/keystore" /> <Parameter name="keypass" value="changeit"/> <Parameter name="clientAuth" value="true"/> </Connector>
43
In this example, the keystore is file /var/tomcat/conf/keystore. The keystore password is changeit. Once you have the certificate and key, copy the new keystore file to a safe location on the X-Series Platform then modify server.xml file to point to the new keystore file. The following example shows the default location on the X-Series Platform: <Parameter name="keystore" value="/usr/os/web/conf/keystore" /> If you used a password other than changeit, edit this line in server.xml: <Parameter name="keypass" value="changeit"/>
Enabling SSH Access to VAPs

To allow individual users to directly access specific VAP members, enable SSH access by executing the following steps. Before starting, ensure that the increment-per-vap parameter has been configured on the management interface for the VAP group so that each VAP member has a unique IP address. Execute all steps on each member of the VAP group. 1. Create user accounts on each APM. test_1 (group1):~# useradd <username> test_1 (group1):~# passwd <username> Changing password for user <username>. New UNIX password: passwd: all authentication tokens updated successfully. test_1 (group1): ~# 2. Start SSHd on each member of the VAP by executing service sshd start. For example: test_1 (group1): service sshd start Starting sshd: 3. 4. 5. Configure SSHd to start after a reboot. [root@REG80B2 root]# chkconfig --add sshd Configure SSHd to create the host.allow file. [root@REG80B2 root]# chkconfig sshd on Modify the /etc/hosts.allow file to permit either all hosts or specified hosts access to the SSH daemon. For example, to allow any host connection, specify sshd: ALL or to allow a specific host connection, specify sshd: <host IP address>. [ OK ]
Disable SSH Access

To disable access to the VAP members, use the following procedure. 1. Modify the /etc/hosts.allow file and remove the host from the SSH daemon access list. For example, to disable all hosts from connecting, specify sshd: NONE. To block a specific host connection, remove the host IP address: sshd: <host IP address>. Unconfigure SSHd from starting at boot. [root@REG80B2 root]# chkconfig --del sshd 3. Stop the service on each VAP. test_1 (group1): service sshd stop Stopping sshd: 4. Delete the user account on the APM. test_1 (group1):~# userdel <username> [ OK ]
2.
44
5
Configuring VAP Groups
A Virtual Application Processor (VAP) is the application operating environment running on an APM. A VAP consists of the OS, the system software, and an application installed and running on the APM. VAP groups consist of one or more APMs running the same application. XOS provides the ability to perform load balancing and high availability within a VAP group. An X-Series Platform supports a maximum of 10 VAP groups. This chapter contains the following main topics:
z z z z z z z
Overview of VAP Groups in XOS on page 45 Overview of IPv6 Traffic on page 51 Configuring a VAP Group on page 52 VAP Group Example on page 55 Virtual Environment VAP Groups on page 58 Displaying VAP Group Parameters on page 59 Displaying VAP Group Mapping on page 60
Overview of VAP Groups in XOS

A Virtual Application Processor (VAP) is an application operating environment that is run on an APM. A VAP consists of the operating system, the system software, and an application. A VAP group is a collection of VAPs configured with identical policies running the same application, grouped together to provide redundancy and increased throughput. The VAP group modules act in concert with one another as a virtual processing engine. The NPM load balances packet flows across the group based on a feedback loop from the APMs and CPMs. The assignment of VAPs to physical APMs is controlled using several parameters. The number of VAPs in the group is set using the vap-count parameter. Additional capacity is achieved by adding APMs and increasing the VAP count. The CPM tracks which VAPs are currently loaded, compared to the VAP count. The operating system and file structure for each VAP is stored on the CPM. To run an application, the APM loads the appropriate image to its memory. This allows for faster reload and reboot when preemption is enabled. The CPM also works in conjunction with the VAPs to provide load balancing information to the NPMs. The NPM assigns traffic flows to VAPs. Once a flow is assigned, the flow is entered in an Active Flow Table. This table is synchronized between the NPMs and includes information such as source and destination address, time to live, and VAP assignment. The load and preemption priorities of a VAP group are set to 0 by default. Load priority defines the order in which VAPs are loaded onto APMs. Preemption priority defines which applications can be re-purposed and their APMs reconfigured with a higher priority application. Changing this value must be done carefully because a VAP group with a higher priority value seeks the resources of the other VAP groups if the required resources are not available in its group. For example, a firewall VAP group configured for three VAPs preempts a lower priority VAP group if its resources drop below three VAPs.
45
Number of VAPs in a Group

The vap-count parameter determines how many VAPs to create, while the max-load-count parameter determines how many VAPs to load. This makes it possible to create more VAPs than initially required, thus keeping a prepared VAP in reserve. If you set the vap-count to 5 but max-load-count to 4, then an extra VAP will be available and can be brought online with a single command. NOTE: Some applications require that the vap-count and max-load-count be the same. The following are guidelines for the vap-count and max-load-count: 1. 2. The vap-count cannot be zero (minimum is 1). The max-load-count cannot be greater than the VAP Count.
When decrementing the vap-count the max-load-count must be adjusted accordingly downward.
Master VAP
When members of a VAP group load onto APMs, the first VAP to become available is designated the Master VAP. The main function of the Master VAP is to handle routing protocols. For example, the OSPF daemon only runs on the Master VAP to prevent multiple routing tables should the daemon run on all VAPs. The Master VAP is also used for various protocol responses, such as ARP and Spanning Tree. For example, all VAPs respond to an ARP request, but only the Master VAPs response is sent out by the X-Series Platform. Should the Master VAP fail, another VAP in the group is elected Master.
Master VAP Reselection

When configuring Layer 2 applications with multiple VAP groups, the master-failover-trigger application parameter must be configured. This parameter tells the system to re-elect a new master VAP if the application fails on the current master VAP, thereby preventing Layer 2 looping issues on the network. Use the master-holddown parameter to specify the amount of time that the current master VAP remains down before XOS gracefully elects a new master VAP. Configure these parameters for each VAP group, after configuring the ap-list.
VAPs and Serialization

Two or more VAP groups running different applications can be connected to one another in series. This allows traffic passing through the X-Series Platform to be inspected by both applications, and then passed on to the internal network or the Internet. For general information about serializing applications, see Configuring Secure Flow Processing on page 151. To learn how to configure specific serialized topologies, see the Multi-Application Serialization Configuration Guide included in this release.
VAPs and APMs

The ap-list parameter defines the list of APMs that are allowed to load VAPs in a group. Generally this list will show ap1 through ap10 when all APMs are identical. However, if an APM is different you may wish to change this list. You cannot mix APM product types within a VAP group. If your system includes four APM-8650s and two APM-8600s, you would create two VAP groups, one group of four 8650s and another group of two 8600s. It is also important that the APMs in a VAP group have identical hardware, such as processors, hard disk drives, and memory. Use the ap-list parameter and show chassis command to display module name and hardware type information needed for creating unmixed VAP groups.
46
Standby VAP
The standby VAP is not part of a VAP group. This VAP runs on an APM, checking the hardware integrity. If an APM running a VAP in a group fails, the standby VAP is dynamically allocated to the VAP group. The standby VAP is preempted and rebooted to run the image of the failed VAP. As an alternative to configuring a standby VAP, preemption priority can be assigned to each VAP group. This allows you to configure an application in a VAP group as a higher or lower priority compared to another application. Should a higher priority VAP group have a failure, it takes a VAP from the lower priority group. This configuration works well when you have a mission critical application and a non-mission critical application running in the same X-Series Platform.
IPv6 Traffic
Use the enable-ipv6 command from the vap-group context to enable IPv6 support on the VAP group. A non-ip-flow-rule is automatically configured and activated, passing IPv6 traffic directly to the Master VAP for processing by the application. After IPv6 is enabled on the VAP group, the user configures circuits, interfaces, and static routes for handling IPv6 traffic. For more information about IPv6 traffic, see Overview of IPv6 Traffic on page 51.
Load Balancing
With load balancing, the NPM distributes the traffic across multiple VAPs, maximizing the overall performance of the X-Series Platform. If a VAP fails, the NPM reclassifies the traffic flow and load balances it among the remaining VAPs. In addition, use the load-balance-vap-list parameter to define the APMs that are allowed to receive new flows. To gracefully shut down a VAP, remove its entry from this list. Existing flows will continue to be handled by the VAP, but once these flows cease the VAP will have no traffic.
Jumbo Frame Support and VAP Groups

VAP groups can be enabled to process Ethernet frames greater than the standard maximum size of 1500 bytes. If you enable jumbo frames for a VAP group, you should set the MTU size on each circuit associated with the VAP group. The minimum should be 1500, and the maximum value is 9000 .
Scatter Gather and Fragmented Ethernet Frame Support

Use the scatter-gather parameter to enable or disable fragmented Ethernet frame support at the physical Ethernet driver.
Flow Rules
A flow rule determines how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, which is configured by the user. There are flow rules for IP and non-IP traffic. Refer to Configuring Flow Provisioning on page 95 for information.
RAID Capability and Settings

APM-8600s and APM-8650s can accommodate dual hard disk drives. With two drives installed you can, optionally, configure the drives for RAID (redundant array of independent disks) operation. Depending on the RAID configuration, the second drive of the pair can be used either to act as a backup, storing identical information, or act as a second available drive, used whenever the other drive of the pair is busy. The first setting increases data security, the second setting produces maximum drive speed/efficiency. The RAID setting for the VAP group can be modified at any time as needed.
47
Requirements for RAID Feature

To configure the drives for RAID operation, your system must meet the following requirements:
z z
Two hard drives are required in an APM-8600, APM-8650, and APM-9600 to use the RAID feature. Use the show module status disk command to check for the presence and size of the disk drives. Both disks must be of the same type and have identical partition sizes. Use the cat /proc/partitions (from root) command to check them.
RAID Settings
Configure RAID settings using the raid parameter of the configure vap-group command. There are three possible RAID settings:
z
configure vap-group <vapgroupname> no raid This command turns raid operation off. The drives function independently. This is the default setting. configure vap-group <vapgroupname> raid 0 This command configures the system to write to whichever drive of the pair is not currently busy. This setting increases efficiency and speed but does not duplicate data. Also, both drives are required in order to access the full content of the stored data.
configure vap-group <vapgroupname> raid 1 This command configures the system to write the same data to both disks, creating a complete backup in case of one drives failure.
See RAID-Related Hard Drive Configuration and Repair on page 241 for more information on RAID configuration.
VAP Operating Systems

The table shows the VAP Operating Systems and the modules on which they are supported.
Table 9.
Name xslinux_v3 xslinux_v5
VAP Operating Systems

X-Series Kernel X-Series Linux v3 X-Series Linux v5 X-Series Linux v5, 64-bit X-Series Virtual Environment Supported Module Default on APM-8600/8650 APM-8600/8650, APM-9600 APM-8600/8650, APM-9600 APM-9600
xslinux_v5_64 xsve
You can create different VAP groups running different kernels in one X-Series Platform. Refer to the XOS V9.5 Release Notes or specific application installation guides for additional information.
VAP Backup and Restore

The VAP backup and restore feature allows you to save your VAP group configurations to the CPM. Should you need to restore your VAP group configurations for any reason, this feature provides a safeguard against unrecoverable failure.
Backup of VAP File Systems

You can create a VAP group archive for each VAP group on the CPM. Individual archives are named for the VAP group and numbered (from 1 to 99). You can save more than one backup file for each VAP group on the CPM, based on the available disk space.
48
Restoring VAP File Systems

When you restore a VAP groups file system, you can specify an archive name, or use the most recent backup (default). However, the archive file used must be from an identical VAP group configuration (number of VAPS and hardware) to the current configuration.
Application Monitoring
Application monitoring is enabled by default, and directs XOS to poll application health on each VAP in a group every five seconds to verify that every application is able to process traffic. If the application is not active on a VAP in the group, the system notifies the NPM to stop new flows (load balancing) to that VAP. The NPM performs this process dynamically. When a VAP is booted, the NPM will wait until the application is fully operational before sending traffic to the VAP. When Application Monitor is disabled, the NPM sends traffic to the VAP immediately upon the APM being Up, regardless of the application state. If the application goes down, the NPM will continue to send traffic as long as the APM remains up, and there is no change in load balancing. With the master-failover-trigger application command configured on the VAP group, XOS will re-elect a new master VAP any time application monitoring detects an application failure on the master VAP. NOTE: Application monitoring cannot detect process hangs. If a process is not functioning but the application is running, the XOS health system continues to report that the application is running.
Configure VAP Group for Ethernet Jumbo Frame Support

To configure a VAP-group to support jumbo Ethernet frames, use the jumbo-frame parameter from the vap-group context. Once this setting is enabled, you will need to reload the VAP to enable the functionality. CBS(config-vap-grp)# jumbo-frame CBS(config-vap-grp)# exit Jumbo Ethernet frame support also needs to be configured at the circuit level using the setting of the MTU on each circuit associated with the vap-group. For example, set the MTU for inf1 to a value of 9000 to enable jumbo frame usage on this VND: circuit inf1 circuit-id 1026 device-name inf1 vap-group jumbo ip-forwarding mtu 9000 ip 11.0.0.1/8 11.255.255.255 Once you have set the MTU value, you can observe the usage of jumbo frames. Use an RSH session on the VAP and use the ifconfig command from root: root# ifconfig inf1 inf1 Link encap:Ethernet HWaddr 00:03:D2:E0:0D:01 inet addr:11.0.0.1 Bcast:11.255.255.255 Mask:255.0.0.0 UP BROADCAST DEBUG RUNNING MULTICAST MTU:9000 Metric:1 RX packets:3 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:180 (180.0 b) TX bytes:192 (192.0 b) root#
49
Mapping a Management Circuit to a VAP Group

Managing an application installed on an X-Series Platform is often done using an individual connection (a single physical interface) to the VAP group. This connection allows you to update policies and monitor the application on the VAPs. Some applications require that you explicitly define the circuit as a management circuit (refer to specific application install guides for more information). Configuring the management circuit on a VAP group can be done two ways. If the management circuit has already been configured, you can use a single line command to create the connection: Command: CBS# configure circuit mgmt vap-group bluefw management-circuit In this example, mgmt specifies the management circuit applied to the VAP group, and bluefw is the name of the VAP group to which the management circuit is applied. The second way to map a management circuit to a VAP group is done while configuring the management circuit. 1. Create a management circuit, so that remote application management utilities can interface with the application.
Command: CBS# configure circuit mgmt CBS(conf-cct)# 2. Assign a device name to the circuit. For clarity, the device name should be the same as, or based on, the circuit name.
Command: CBS(conf-cct)# device-name mgmt CBS(conf-cct)# 3. Associate the VAP group with a circuit. Some applications (such as IBM ISS) require that you designate this circuit as the management-circuit. Refer to the application documentation for more information.
Command: CBS(conf-cct)# vap-group bluefw CBS(conf-cct-vapgroup)# management-circuit CBS(conf-cct-vapgroup)# 4. Assign an IP address for VAP group management. Use increment-per-vap to assign a unique IP address per VAP member, allowing individual management connections. When configuring the management IP addresses it is recommended to leave some unused IP addresses so that additional APMs and VAPs can be added as the system grows.
Command: CBS(conf-cct-vapgroup)# ip 172.16.19.63/24 increment-per-vap 172.16.19.67 CBS(conf-cct-vapgroup-ip)# end CBS#
50
Overview of IPv6 Traffic

XOS supports IPv6 addresses in both an IPv6/IPv4 dual-stack, and a pure IPv6 environment. XOS supports the tunneling of IPv6 traffic through an IPv4 network. In a simple scenario, IPv6 traffic arrives from an external network, passes through an interface on the NPM to a circuit attached to a VAP group with an application installed and running. The application processes the traffic, and passes it out another circuit and interface on the NPM. The traffic is sent to its destination on the client subnet. Configure VAP groups to accept IPv6 addresses using the enable-ipv6 command. Applications running on an IPv6-enabled VAP group must be able to process IPv6 traffic. After enabling IPv6 on the VAP group, configure IPv6 addresses on the circuits and the interfaces. When you enable IPv6 on the VAP group, a default non-ip-flow-rule is automatically created as part of the command: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate The default flow rule passes all IPv6 traffic to the master VAP for processing. Refer to Configuring a VAP Group on page 52 for the steps to configure IPv6 on a VAP group.
Load Balancing
IPv6 traffic is not load-balanced across all VAPs in the group. However, IPv6 traffic can be load-balanced across multiple cores on a the master VAP, reducing resource utilization and increasing throughput. Use the multi-proc-processing parameter under the core-assignment context to modify the default IPv6 non-ip flow rule (ipv6_rule). This parameter will configure load balancing across multiple processors and multiple cores on the master VAP. For core-assignment configuration steps, refer to Configuring a VAP Group on page 52. For a list and explanation of each of the core-assignment parameters, refer to Chapter 6 of the Command Reference Guide.
IPv6 Circuit Considerations and Limitations

Consider the following information when configuring your X-Series Platform for IPv6:
z z z z z
Enable IPv6 on the VAP group before assigning an IPv6 address to the circuit. If IPv6 is not enabled, a warning will be displayed when the IPv6 address is assigned to the circuit. IPv6 addressing may be used on VLANs and virtual IP addresses. Increment-per-vap cannot be configured for an IPv6 address. IPv6 addressing is not supported on the management network. You must specify an IP address for a circuit as an IPv4 address before assigning an IPv6 address or alias address on that circuit. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses. The minimum MTU size for IPv6 enabled circuits is 1280 and the maximum is 9000. An error message appears when enabling IPv6 on a circuit with an MTU of less than 1280, or when setting the MTU to less than 1280 on an IPv6-enabled circuit. The Crossbeam default MTU value is 1500. IPv4 / IPv6 mixed mode addresses cannot be configured using the CLI. For example, configuring the following address on a circuit will result in an error. FC00:0:0:0:0:FFFF:204.9.54.119
51
Supported IPv4 to IPv6 Tunnels

The following transition mechanisms, or tunnels, are provided to allow early adapters of the IPv6 protocol a method to deliver IPv6 traffic over an IPv4 network segment. Support of these tunnels on the X-Series platform uses a standards (RFC) -based approach and is not Crossbeam proprietary. IPv6 transition tunnels should not be confused with VPN, ssl, or other IPSec tunnels. There are a number of standard protocols that provide transition mechanisms for allowing IPv6 traffic to traverse an IPv4 network segment. XOS in conjunction with RSW 8.0 supports the following four IPv6 to IPv4 tunneling protocols:

6to4 ISATAP IPv6IP GRE
The process uses a dual stack router or a firewall on either side of an IPv4 network. These serve as endpoints for the tunnel. IPv6 traffic is encapsulated within IPv4 traffic, and crosses the network as native IPv4 traffic. When the encapsulated packets reach the destination router or firewall, the IPv6 packets are de-encapsulated and routed appropriately. For additional tunnel information, refer to IPv6 Tunneling on page 217
Configuring a VAP Group

To create a VAP group using EMS, open Configuration > Modules and VAP Groups > VAP Groups. To create a VAP group using the CLI use the following steps. Not all the steps are required, but are shown here to illustrate the commands used for configuration. A simple VAP group configuration follows in the next section. 1. Create a VAP group named bluefw using the default xslinux_v3 operating system. The command line interface is case sensitive. Different capitalizations (Bluefw and blueFW) are considered to be different names. To avoid confusion, use unique names. Because commands and keywords are lower case, using at least one upper case letter in a name will reduce confusion. NOTE: The xslinux_v3 is loaded by default. To install an application that requires a different VAP operating system, specify the Linux kernel in the configure command. Command: CBS# configure vap-group bluefw Are you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group bluefw. May take several minutes.................... %WARNING: X-Series xslinux_v3 is not supported on APM9600 or above CBS(config-vap-grp)# 2. Define the number of VAPs in this group. Multiple VAPs provide redundancy and additional capacity. Command: CBS(config-vap-grp)# vap-count 3 Are you sure you want to adjust vap-count to 3? <Y or N> [Y]: Y Adjusting vap-count. May take several minutes............................ CBS(config-vap-grp)# 3. Specify the maximum number of VAP members in the VAP group. This number cannot exceed the VAP count. In order to install some applications, the max-load-count must match the VAP count. Command: CBS(config-vap-grp)# max-load-count 3 CBS(config-vap-grp)#
52
4.
Assign APMs to support the VAP group. This command specifies the list of APMs to be loaded. All VAP members should be running on identical APMs. Use show module status from the CLI to verify the configuration of each APM. Command: CBS(config-vap-grp)# ap-list ap3 ap4 ap5 CBS(config-vap-grp)# NOTE: Use the show chassis command to determine the module names and types associated with each APM. Use the ap-list parameter to associate like module types (for example, just APM-8650s) into a VAP group of identical module types.
5.
Define the VAP group load-balance-vap-list. The load-balance-vap-list is a list of VAP indexes that the NPM uses to load balance new flows. For example, if you have a VAP group with 3 VAPs (bluefw_1, bluefw_2, and bluefw_3), the NPM load-balances over all three VAPs by default. Use the load-balance-vap-list to force the NPM to load balance over specific VAPs as follows. Command: CBS(config-vap-grp)# load-balance-vap-list 1 2 3 CBS(config-vap-grp)# The NPM ignores indexes included in the command that do not exist. For example, the NPM will ignore indexes 4 5 6 7 8 9 in the command for a VAP group that only has 3 indexes.
6.
Enable IPv6 address support if required. If you do not require IPv6 support, skip to Step 8. CBS(config-vap-grp)# enable-ipv6 Enabling IPv6 automatically configures the following non-ip-flow-rule: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate
7.
Use the core-assignment parameter to load-balance IPv6 traffic across multiple processors and multiple cores on the master VAP. CBS(config-vap-grp)# non-ip-flow-rule ipv6_rule core-assignment multi-proc-processing
8.
Configure the default flow-rule for the VAP group and return to the VAP group context. There are four steps to configure the load balancing flow rule.

Create the load balancing flow rule for the bluefw VAP group. Set flow rule action to load-balance traffic to all available VAP members. Set the activate flag to enable the action. Return to the main CLI context to prepare for the next step.
CBS(config-vap-grp)# ip-flow-rule bluefw_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)# 9. Define the VAP group load-priority level. This command determines which VAP group is given priority when assigning VAPs to physical APMs. Valid values are 0 to 255, with 0 being the default value. A zero sets the lowest priority level. Command: CBS(config-vap-grp)# load-priority 5 CBS(config-vap-grp)#
53
10. Define the VAP group preemption-priority level. The application running on one VAP group is assigned a higher priority than an application on another VAP group (for example, a firewall assigned a higher priority than an IPS). If the higher priority VAP group requires additional APMs to load its VAPs, this value determines from which group APMs can be taken. If the preemption priority for this group is higher than the load priority for a currently running VAP, that VAP is shutdown and an APM is loaded to the higher priority VAP group instead. Additionally, should a higher priority VAP group have a failure, it takes a VAP from a lower priority group. This configuration works well when you have a mission critical application and a non-mission critical application running in the same X-Series Platform. Valid values are 0 to 255, with 0 being the default value. A zero sets the lowest priority level. Command: CBS(config-vap-grp)# preemption-priority 5 CBS(config-vap-grp)# 11. Define the delay period this VAP group waits before load balancing. Assign the <interval> (number of seconds) a VAP group waits before beginning load balancing. Valid values are from 1 to 3600 seconds, with a default value of no delay. Command: CBS(config-vap-grp)# delay-flow 10 CBS(config-vap-grp)# 12. Enable or disable the Reverse Path (RP) filter. Disabling rp-filter allows a management server external to the network to enter through the load balancing filter and access the correct VAP. This anti-spoofing mechanism is resident in the Linux kernel. Disable the rp-filter using the no command. Command: CBS(config-vap-grp)# no rp-filter CBS(config-vap-grp)# 13. Enable or disable logging of martians (packets with a source IP address but no known route). Packets with source addresses not routable by the system are referred to as martians (packets from Mars) on the grounds that they are of no evident terrestrial (that is, normal) source. Martian packets can occur from network equipment malfunction, mis-configuration of a host, or simple coexistence of two logical networks on a single physical layer. For example, if the IP networks 192.168.35.0/24 and 10.1.2.0/24 operate on the same Ethernet segment, packets from 10.1.2.5 are martians to system 192.168.35.8, and vice versa. When enabled, martians are logged to syslog. Command: CBS(config-vap-grp)# log-martians CBS(config-vap-grp)# 14. Configure vg-reset-wait-time. This command sets the wait time (in seconds) before resetting a VAP when no connectivity to the VAP has been detected. In the case of an application that is busy processing and fails to send heartbeats for a period of time, this command will extend the wait time before the VAP resets. Command: CBS(config-vap-grp)# vg-reset-wait-time 15 CBS(config-vap-grp)# 15. Configure the reload-timeout value. By default, the system waits 300 seconds for a VAP group to reload. If the reload does not occur, the system resets the slot. For some system configurations, you may need to have the system wait a specific number of seconds for the VAP group to reload. For example, a firewall application may only require the system to wait 120 seconds before reloading. Valid values are 60 to 18000 seconds. Command CBS(config-vap-grp)# reload-timeout 120 CBS(config-vap-grp)#
Configuring VAP Groups 54
VAP Group Example

The following is an example of a minimal VAP group configuration: CBS# configure vap-group bluefw CBS(config-vap-grp)# vap-count 2 CBS(config-vap-grp)# max-load-count 2 CBS(config-vap-grp)# ap-list ap1 ap2 CBS(config-vap-grp)# ip-flow-rule fw_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate The following is the same VAP group configuration, with IPv6 and multi-core processing enabled: CBS# configure vap-group bluefw xslinux_v5 CBS(config-vap-grp)# vap-count 2 CBS(config-vap-grp)# max-load-count 2 CBS(config-vap-grp)# ap-list ap1 ap2 CBS(config-vap-grp)# enable-ipv6 CBS(enable-ipv6)# exit CBS(config-vap-grp)# non-ip-flow-rule ipv6_rule core-assignment multi-core-processing CBS(non-ip-flow-rule)# exit CBS(config-vap-grp)# In Figure 3, the X-Series Platform is configured to handle multiple VAPs, each having access to other networks and the Internet through either VAP group.
Figure 3.
VAP Group Example
Computer
Circuit13/Port3 10.133.2.193 Switch Circuit11/Port1 VAPgroup1 VAP1/AP1
15.0.0.193
Internet
Computer 20.0.0.193 Internet Circuit12/Port2 Circuit14/Port4 44.0.0.195 Switch Computer
VAPgroup2 VAP2/AP2 Computer
The following is an example of the commands in a configuration based on this figure: CBS# configure vap-group redfw Are you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group Firewall. May take several minutes.................... %WARNING: X-Series xslinux_v3 is not supported on APM9600 or above CBS(config-vap-grp)# max-load-count 1 CBS(config-vap-grp)# load-priority 2 CBS(config-vap-grp)# ap-list ap1 CBS(config-vap-grp)# ip-flow-rule Firewall_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)# jumbo-frame CBS(config-vap-grp)# exit
CBS# configure vap-group bluefw Are you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group Firewall. May take several minutes.................... %WARNING: X-Series xslinux_v3 is not supported on APM9600 or above CBS(config-vap-grp)# max-load-count 1 CBS(config-vap-grp)# load-priority 5 CBS(config-vap-grp)# preemption-priority 2 CBS(config-vap-grp)# ap-list ap2 CBS(config-vap-grp)# ip-flow-rule Firewall_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)# exit CBS# configure circuit purple CBS(conf-cct)# vap-group redfw CBS(conf-cct-vapgroup)# ip 10.133.2.193 255.255.255.0 CBS(conf-cct-vapgroup)# end CBS# configure circuit red CBS(conf-cct)# vap-group bluefw CBS(conf-cct-vapgroup)# ip 20.0.0.193 255.0.0.0 CBS(conf-cct-vapgroup)# end CBS# configure circuit yellow CBS(conf-cct)# vap-group redfw CBS(conf-cct-vapgroup)# ip 15.0.0.183 255.0.0.0 CBS(conf-cct-vapgroup)# end CBS# configure circuit blue CBS(conf-cct)# vap-group bluefw CBS(conf-cct-vapgroup)# ip 44.0.0.195 255.0.0.0 CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log11 CBS(intf-gig-logical)# circuit purple CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log12 CBS(intf-gig-logical)# circuit red CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log13 CBS(intf-gig-logical)# circuit yellow CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log14 CBS(intf-gig-logical)# circuit blue CBS(intf-gig-log-cct)# end CBS# 1/1
1/2
1/3
1/4
You can use the show running-config vap-group <name> command to view the VAP group configuration to verify your steps. Similarly, use the show running-config command with circuit <name>, vrrp-failover-group <name>, or group-interface <name> parameters to view specific running configurations for those parameters.
56
The following is an example of the commands to create an IPv6 configuration based on Figure 3 on page 55: CBS# configure vap-group redfw xslinux_v5 Are you sure you want to create a new vap-group with OS version xslinux_v5? <Y or N> [Y]: CBS(config-vap-grp)# max-load-count 1 CBS(config-vap-grp)# load-priority 2 CBS(config-vap-grp)# ap-list ap1 CBS(config-vap-grp)# enable-ipv6 CBS(enable-ipv6)# end CBS# configure vap-group bluefw xslinux_v5 Are you sure you want to create a new vap-group with OS version xslinux_v5? <Y or N> [Y]: CBS(config-vap-grp)# max-load-count 1 CBS(config-vap-grp)# load-priority 5 CBS(config-vap-grp)# preemption-priority 2 CBS(config-vap-grp)# ap-list ap2 CBS(config-vap-grp)# enable-ipv6 CBS(enable-ipv6)# end CBS# configure circuit purple CBS(conf-cct)# vap-group redfw CBS(conf-cct-vapgroup)# ip 2002:C:4:0:0:0:0:B3/64 %WARNING: IPv6 primary address was configured. No allowed for this circuit. CBS(conf-cct-vapgroup)# end CBS# configure circuit red CBS(conf-cct)# vap-group bluefw CBS(conf-cct-vapgroup)# ip 2002:1:F:0:0:0:0:F5/32 %WARNING: IPv6 primary address was configured. No allowed for this circuit. CBS(conf-cct-vapgroup)# end CBS# configure circuit yellow CBS(conf-cct)# vap-group redfw CBS(conf-cct-vapgroup)# ip 2002:C:4:0:0:0:0:B8/64 %WARNING: IPv6 primary address was configured. No allowed for this circuit. CBS(conf-cct-vapgroup)# end CBS# configure circuit blue CBS(conf-cct)# vap-group bluefw CBS(conf-cct-vapgroup)# ip 2002:1:F:0:0:0:0:11/32 %WARNING: IPv6 primary address was configured. No allowed for this circuit. CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log11 CBS(intf-gig-logical)# circuit purple CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log12 CBS(intf-gig-logical)# circuit red CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log13 CBS(intf-gig-logical)# circuit yellow CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet CBS(conf-intf-gig)# logical log14 CBS(intf-gig-logical)# circuit blue CBS(intf-gig-log-cct)# end CBS# 1/1
IPv4 aliases will be
1/2
1/3
1/4
57
Virtual Environment VAP Groups

The Virtual Environment provides a platform that allows non-linux based applications to run on XOS. When you configure an X-Series Virtual Environment (XSVE) VAP group, XOS creates a VAP group configured to run one or more virtual machines. During the application install, the virtual machine is created on the APM (the host). The application (guest application or guest) is installed and configured in the same manner as other applications on the X-Series platform. Use the following command to create the virtual environment VAP: configure vap-group <name> xsve For example: CBS# configure vap-group vbluefw xsve Are you sure you want to create a new vap-group with OS version xsve? <Y or N> [Y]: Y Creating vap-group vfwall. May take several minutes.................... %WARNING: X-Series xsve is only supported on APM9600 or above CBS(config-vap-grp)# An APM configured as an X-Series VE VAP supports a single virtual machine running a single guest virtual appliance (application). The X-Series virtual environment is only supported on the APM-9600 or above.
Additional Commands
The following commands are used when configuring virtual environment VAP groups. These commands fall under the configure vap-group context. For additional information on these and other virtual environment-specific commands, refer to the XOS Command Reference Guide or to the individual application installation and configuration guide.
Flow Proxy
In a virtualized environment, the flow-proxy parameter changes the flow processing algorithm to improve performance. IMPORTANT: This parameter is disabled by default, but is required for any VAP group that uses the XSVE VAP operating system. CBS# configure vap-group vbluefw CBS(config-vap-grp)# flow-proxy CBS(config-vap-grp)# To disable the flow-proxy command after it has been enabled, use the no flow-proxy parameter.
Fail to Host
In a case where the virtual application (guest) fails, fail-to-host reroutes traffic through the host, allowing packets to be forwarded by the VAP. This parameter is disabled by default and should only be enabled when required by an application (guest). Refer to your application information for specific requirements. CBS# configure vap-group vbluefw CBS(config-vap-grp)# fail-to-host CBS(config-vap-grp)# To disable the fail-to-host command after it has been enabled, use the no parameter.
58
Incoming Circuit Groups

For configurations using circuits that are part of an incoming circuit group (ICG), some applications require the use of an incoming circuit group name. For more information about configuring a circuit as part of an incoming circuit group refer to Configuring Circuits on page 68, or the XOS Command Reference Guide. Use the following command to configure an incoming circuit group name. CBS# configure incoming-circuit-group-name <ICG number> <ICG name>
Configure a Virtual Environment VAP

The following is an example XSVE VAP configuration. For the VAP group configuration procedure, refer to Configuring a VAP Group on page 52. vap-group vbluefw xsve vap-count 2 max-load-count 2 ap-list ap7 ap8 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 flow-proxy fail-to host ip-flow-rule loadbalance action load-balance incoming-circuit-group any activate # incoming-circuit-group-name 2 internal incoming-circuit-group-name 3 external # circuit red circuit-id 1025 device-name red incoming-circuit-group 2 vap-group vbluefw ip 50.0.0.2/24 50.0.0.255 increment-per-vap 50.0.0.10 circuit blue circuit-id 1026 device-name blue incoming-circuit-group 3 vap-group vbluefw ip 51.0.0.2/24 51.0.0.255 increment-per-vap 51.0.0.10
Displaying VAP Group Parameters

To display configuration information about a VAP group, use the following command: show vap-group <vap-group-name> Where vap-group-name is the name of the VAP group. The following is an example display. CBS# show vap-group bluefw VAP Group Operating System Load Priority Preemption Priority AP List VAP Count Max Load Count Max Reload Count Load Balance VAP List IP Forwarding (true/false) Delay Flow (seconds) : : : : : : : : : : : bluefw xslinux_v3 0 0 ap2 ap3 ap4 2 2 3 1 2 3 4 5 6 7 8 9 10 f 0
59
Backup Mode Reload Timeout (seconds) RP Filter (true/false) Log Martians (true/false) DHCP Relay Server List RAID Jumbo Frame (true/false) Scatter Gather (true/false) Master HoldDown Timer (in seconds) Master Failover Trigger Application Monitoring (true/false) IPv6 Enabled (true/false) IPv6 IP Forwarding (true/false) Fail To Host (true/false) Flow Proxy (true/false) Reset Wait Time (seconds) (1 row)
: : : : : : : : : : : : : : : :
none 300 f f none f f 0 application t t f f f 5
Displaying VAP Group Mapping

You can view the relationship between VAP groups and the chassis slots occupied by the APMs. To do this with EMS, open Configuration > Modules and VAP Groups > VAP Groups. To use the CLI, enter the following command: show ap-vap-mapping The following is a sample display created by this command. Module AP1 AP2 AP3 AP4 AP5 Where: Module Number of the APM. Slot APM location in the chassis. Status Module status: Active, Up, Down, Initializing, Booting, Diagnostics, Maintenance, or Unavailable. VAP IP Address IP address assigned to the VAP VAP group Name of the VAP group. Index Assignment of the VAP within the VAP group. Master True indicates that this is the Master VAP in the group; otherwise, false is shown. Slot Status 3 4 5 6 7 Active Active Active Active Booting VAP IP Address VAP Group 30.1.4.101 30.1.4.102 30.1.4.103 30.1.4.104 30.1.4.105 redfw bluefw bluefw bluefw redfw Index 1 1 2 3 2 Master true true false false false
60
6
Configuring Interfaces
The physical data interfaces are located on the Network Processor Modules (NPMs). An XOS system can be configured with only one NPM type; however, the NPM type must also be defined in the software using the configure operating-mode command. This chapter includes the following major topics:
z z z z z z
Data Interface Overview on page 61 Configuring Interfaces on the NPM on page 66 Packet Mirroring on the NPM Interfaces on page 73 Configuring Chassis Resource Protection on page 78 Configuring Redundant Interfaces on page 83 Configuring a Multi-Link Group Interface on page 85
Data Interface Overview

The data interfaces consist of physical interfaces, logical interfaces, circuits, and Virtual Network Devices (VNDs). The physical interface is mapped to a logical interface, which is mapped to a circuit. The circuit is seen as a VND by the VAPs. Virtual Local Area Networks (VLANs) are associated with a logical interface.
Figure 4.
Data Interfaces
VAP Group
VND1
Logical
Circuit
VND1
VND1
NPM
APM
APM
APM
61
Physical Interfaces
The physical data interfaces on the NPMs are identified by type (gigabitethernet or 10gigabitethernet), chassis slot number of the NPM, and physical port number. For example, the gigabitethernet interface on the fourth port of the NPM in slot 1 would be designated as gigabitethernet 1/4. Depending on the NPM model, the physical interface can be a copper or fiber interface and support speeds to 10 Gb.
Logical Interfaces
Physical interfaces are mapped to logical interfaces. A physical interface can be mapped to several logical interfaces. The application has no knowledge of the physical interface being used. When a physical interface fails, the logical interface and any circuits configured to that logical interface can be moved to another physical interface.
Interface Internal
Serial connections between VAP groups, and connections between individual VAPs for synchronization are configured using the interface-internal command.
Circuits
The circuit provides the path packets follow from the physical interfaces on the NPMs to the VAP groups. These paths are dynamically updated each time a VAP boots onto an APM. When a circuit connects a physical interface to a VAP group, the state of the circuit depends on the physical link state. The circuit goes down when the physical interface fails or is unavailable. Circuits can also provide internal communication channels between VAPs in a VAP group (synchronization traffic), serial connections for packets to travel between different applications on separate VAP groups, and parallel connections to multiple VAP groups sending traffic to more than one application simultaneously. Circuits between VAPs (sync circuits) or between VAP groups (serial or parallel circuits) are configured on an interface-internal. When Dual-box High Availability is configured, an external interface is needed to provide connectivity to the sync circuit between VAP groups on separate systems. In this case, the sync circuit is configured using the link-state-resistant parameter. This parameter allows the circuit to continue passing internal traffic between the VAP members, even when the communication between systems stops due to a physical interface failure.
Circuits and IP Addressing

For a circuit servicing a Layer 3 VAP group, IP addresses can be associated with that circuit in different ways:
z
Multiple addresses associated with a circuit on a VAP is often referred to as aliasing or multi-netting. The first IP address configured on a circuit is the primary IP; the remaining addresses are aliases. These IP addresses can either be the same across VAPs, or different. This configuration is identical for all VAPs within a VAP group. It is possible to have all VAPs use the same primary IP address. All VAPs within the same group can be assigned the same IP address associated with the circuit.
Create a circuit by specifying a circuit name for the circuit. Then map the circuit to a VAP group and assign an IP address. Optionally, you can place the circuit into a VLAN by specifying a default outbound VLAN tag.
62
Serialization and Failover

By using VRRP to identify circuits, serialized applications can be configured into failover groups. VRRP allows you to configure Next Hop Health Check to verify the state of the device on the other end of the circuit. For more information refer to the Mutli-Application Serialization Configuration Guide.
Virtual Network Device (VND)

Each VAP in the group sees the circuit as a VND, and it appears as a standard Ethernet interface to any processes or applications running on the VAP. The VNDs are identical on each VAP in the VAP group. IP addresses are assigned to circuits, and thus to the VNDs. Identical IPs on each VAP allow for load balancing across the VAPs. Optionally, unique IP addresses can be assigned to the VND on each VAP. A unique IP address is usually assigned for application management, since management software must be able to address a specific member of a cluster. The VND appears as a device name in the circuit configuration. XOS assigns a default device name to each VND; VNDxxxx, where xxxx is a unique number. Use the device-name parameter to change the default VND name; often the circuit-name and device-name are configured as the same.
VLANs
A logical interface can be configured to pass traffic with a specific VLAN tag, a range of VLAN tags, or all VLAN and non-VLAN traffic. The ingress-vlan-tag parameter for the configure interface...logical or configure group-interface...logical command associates a VLAN tag or range of tags to circuits. NOTE: Logical interfaces with overlapping VLAN ID ranges cannot coexist on the same physical interface. Additionally, when using interface redundancy, the backup interface cannot be assigned a VLAN ID that is assigned to the master interface. The logical-all parameter for the configure interface command configures a logical interface to pass all VLAN and non-VLAN traffic. Each physical interface can have only one logical interface configured with the logical-all parameter. In a VLAN configuration using an 802.1q trunk, a single physical interface may handle several subnets. The X-Series Platform can be configured to handle a VLAN trunk by assigning many logical interfaces to a single physical interface. Each logical interface is associated with a VLAN tag or range of VLAN tags. On ingress, a tagged packet is passed only to the logical interface that allows that VLAN tag. The logical interface then passes the packet to its configured circuit, which sends the packet to a VAPs VND interface. The VND interface strips the VLAN tag and passes the normal IP packet to the VAPs IP stack. By using the default-egress-vlan-tag parameter with the configure circuit command, egress packets can be tagged by associating a VLAN tag with the circuit. When a packet egresses through a particular VND, that VND will assign a tag to that packet based on the circuits associated VLAN. The tagged packet is passed to the circuit, then to the logical interface which sends the tagged packet out the corresponding physical interface. Use the hide-vlan-header optional argument under the default-egress-vlan-tag parameter to remove the VLAN tag from the header. This is used with any application where the VLAN tag must be removed for proper operation. Use the replace-vlan-tag <id> parameter with the configure circuit <name> vap-group command to replace an existing VLAN tag on the circuit traffic with another VLAN tag. This is useful to map traffic at an incoming port with a VLAN ID of x to a VLAN ID of y, especially if connected to an external switch. For security, the X-Series Platform drops any incoming packet tagged with a VLAN tag that is not associated with any of the configured logical interfaces.
In the following example, a single physical interface has 2 logical interfaces, and each logical interface has a single circuit. The physical interface is connected to a VLAN trunk that carries VLANs 400 and 500. interface gigabitethernet 1/9 logical vlan400 ingress-vlan-tag 400 circuit vlan_400 logical vlan500 ingress-vlan-tag 500 circuit vlan_500 In this example, when a tagged packet comes into the physical interface, logical interfaces are checked for associated VLAN tags. A packet tagged with VLAN 400 is passed to circuit vlan_400. A packet tagged with VLAN 500 is passed to circuit vlan_500.
MAC Address Inheritance

A single MAC address (user-defined or system generated) can be assigned to multiple VLANs on a VAP group using the logical-all parameter under the configure interface command. If you do not specify a MAC address in Step 1, a system generated MAC address will be used. The following steps describe this process. 1. Create a base circuit with a user-defined MAC address. circuit int100base device-name int100base vap-group fw mac-addr 44:44:44:44:44:44 ip 100.100.100.100/24 100.100.100.255 2. Create a VLAN circuit. circuit int100v circuit-id 1106 device-name int100v vap-group fw ip-forwarding default-egress-vlan-tag 100 ip 101.101.101.101/24 101.101.101.255 vap-group L2 promiscuous-mode active 3. Create the interface, and assign a logical-all to the base circuit. interface gigabitethernet 1/4 int100 logical-all int100base circuit int100base logical int100v ingress-vlan-tag 100 100 circuit int100v Use the show circuit command on the VAP group to verify the MAC address assignment. Once the base VLAN is configured with the logical-all and a user defined MAC address, the additional VLANs inherit the MAC address automatically. If the user changes the MAC address on the base circuit, the VLANs will inherit the new MAC address. Assigning a logical to the base circuit in Step 3 prevents changes to the base circuit MAC address from being applied to the VLANs. The MAC addresses on the VLANs remain as assigned in the initial configuration. To display configured VLAN information, use the show vlan command.
64
Domains
A domain is used to differentiate traffic that uses overlapping IP addresses. For example, gigabitethernet 1/2 and gigabitethernet 1/3 both receive traffic from an IP address of 10.10.101.10; however, the sources are from different LANs. This can occur in an ISP configuration. To differentiate traffic, assign Domain 1 to the gigabitethernet 1/2 circuit and Domain 2 to the gigabitethernet 1/3 circuit. A domain ID can also be used to differentiate multicast traffic, where one incoming packet goes out on multiple interfaces. For example, the X-Series Platform configured with PIM-SM passes ICMP multicast traffic. This traffic may be sent out on different interfaces for clients that have joined a multicast group. To differentiate the traffic, you assign a different domain ID for each circuit carrying the multicast traffic. In this case, you would also need to create an IP flow rule that specifies the domain ID or range of domains.
Group Interfaces
The X-Series Platform supports grouping multiple physical ports into a multi-link group interface using LACP (Link Aggregation Control Protocol, IEEE 802.3ad). Configure the group interface as you would a physical interface, by assigning the group to logical interfaces and assigning the logical interfaces to circuits. To create a multi-link group, you must first create a non-VLAN circuit to use for that group. The X-Series Platform supports assigning a multi-link group interface to multiple VAP groups, which allows you to tap traffic from the group interface. There is a limit of 8 ports to a group interface. Circuits used in one group-interface cannot be used in another group-interface. For scalability, you can add a multi-link group interface to a bridge or transparent interface. This allows greater throughput of traffic. However, the multi-link group cannot have any configured logical interfaces. Refer to Configuring Bridge Mode on page 115 for more information on bridge and transparent modes.
Redundant Interfaces
A physical interface can be configured as a backup for another physical interface in case of failure. By specifying a backup physical interface, logical interfaces assigned to the master physical interface are also backed up. Switching to the backup interface is transparent to the application. Failover occurs very quickly since the physical interface state is monitored by the X-Series Platform controller daemon. NOTE: The failover succeeds only if the backup interface is in the UP state. When failover occurs, the backup port always assumes the master ports MAC address, and sends a gratuitous ARP to update Layer 2 switch tables. NOTE: A Master interface cannot be part of a mode multi-link group-interface. The backup interface can be configured as a Standby or Active interface. If configured as active and the master interface fails, the backup interface assumes the failed interfaces traffic in addition to its own. This is known as an active-active configuration. A backup interface in an active-active configuration cannot be configured to pass the same type of traffic as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or pass untagged traffic. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANS tagged 300.
Restrictions for Overlapping VLANs and Redundancy Interfaces

1. 2. The Master interface cannot be in standby-only state. Masters sharing the same backup cannot have overlapping VLANS. Overlapping VLANS include: a. b. c. Completely covering the range (100-200 and 50-250). Includes partially overlapping (100-200 and 200-201). Excludes logical-all.
65
3. 4. 5.
Overlapping VLANS are checked for ALL involved logicals. Masters can be part of a group-interface. Validation is done in all paths including configuring redundancy-interfaces, set/unset standby-only, add/remove VLANs.
VND Tap
A VND tap enables an application, such as IDS, to view traffic on the circuit specified. Configuring a circuit between two VAP groups allows serial communication between them, as described in Configuring Circuits on page 68. To configure a VND tap, add a third VAP group to the circuit and configure the VAP group for promiscuous mode. When a tap is configured while there are existing traffic flows, the tap will not see the established flows. If it is necessary to view established flows, use the clear flow-active command. IMPORTANT: Using the clear flow-active command halts all traffic to the system for approximately 60 to 90 seconds. Do not use this command unless necessary. A flow rule is required to see any traffic on a tap interface. Adding a tap to a logical line requires that the VAP group be added to the parent circuit first. There are four steps to create a flow-rule for the tap VAP group.
z z z z
Create the default flow rule for the tap VAP group. Set flow rule action to load-balance tap traffic to all available VAP members. Set the activate flag to enable the action. Return to the main CLI context to prepare for the next step. CBS(config-vap-grp)# ip-flow-rule tap_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# end CBS#
Configuring Interfaces on the NPM

The following sections provide the procedures to configure traffic flow on the NPM.
Configuring Physical and Logical Interfaces

There are two types of physical interfaces on the NPM: gigabitethernet and 10gigabitethernet. Each physical interface is identified by the NPM slot number and the port on the NPM. Each physical interface can contain one or more logical interfaces. Generally, there is a separate logical interface for each VLAN tag or range of VLAN tags on an Ethernet interface. To configure a physical interface using EMS, open Configuration > Interfaces > Data Interfaces. Click on an interface row and click the Edit button. From the Command Line Interface, use the configure interface command. Not all settings are used on the 10gigabitethernet interface, as noted.
66
Table 10.
Setting
NPM Physical Interfaces

Definition Automatically matches best speed for multi-speed devices and uses automatic sensing to support full duplex operation. A fiber gigabitethernet port must have auto negotiation enabled. Only applicable for gigabitethernet. Configures half or full duplex mode. Only applicable to gigabitethernet ports configured for a copper connection.
auto-negotiate
duplex-mode
logical <name> Defines a logical interface. Enter a unique name to create a logical interface, or specify ingress-vlan-tag an existing logical name to change its settings. <low> [<high>] The ingress-vlan-tag passes VLAN traffic with a specific VLAN ID, as specified by [circuit <name>] <low>, or a range of VLAN IDs, as specified by <low> and <high>. Valid values are 0 to 4094. Optionally, the circuit parameter assigns an existing circuit to the logical interface. logical-all media-speed pause-frame standby-only Configures a logical interface to pass traffic regardless of VLAN tags. A physical interface can have only one logical interface configured with the logical-all. Configures media speed; only applicable to gigabitethernet ports configured for a copper connection. This setting overrides the auto-negotiation speed. Configures pause frame, also known as flow control. Only use if the interface is a backup in an interface redundancy configuration. The interface will not be used for active traffic unless the master interface fails.
You can disable and enable any interface with the [no] enable command. The following example is a simple gigabitethernet interface configuration. The NPM is in slot 1 and the physical interface is port 4. By default, the interface is enabled, auto-negotiation is enabled, duplex mode is full, and the interface is not in standby mode. CBS# configure interface gigabitethernet 1/4 Use the show command in the config-intf context to display the current configuration. For example: CBS# configure interface gigabitethernet 1/4 CBS(conf-intf-gig)# show Interface : gigabitethernet 1/4 Enable (true/false) : t Auto Negotiate Enabled (true/false) : t Media Speed (Mbits) : auto Duplex Mode : auto Pause Frame (true/false) : t Standby Only (true/false) : f To display a physical interfaces current operational status, use the show interface command.
Interface-Internal
An interface-internal is created to provide internal connectivity between VAPs (synchronization or sync circuits) or between VAP groups (serialization). An interface-internal can be segmented into logical lines in the same way physical interfaces are assigned logical lines, and handles VLAN traffic, non-VLAN traffic, or all traffic. The following example configures an interface-internal to pass all traffic. CBS# configure interface-internal serial CBS(conf-intf-internal)# logical-all log_ser CBS(conf-intf-internal-log-all)# circuit serialize CBS(conf-intf-int-log-all-cct)# end CBS#
67
Configuring Circuits
The circuit provides the path packets must take from the physical interfaces on the NPMs to the VAP groups. To configure circuits using EMS, open Configuration > Interfaces > Data Interfaces > Circuits. Use the New button to create a circuit or to map the circuit. Use the Help button for details. To create a circuit using the CLI, use the configure circuit <circuit-name> command, where circuit-name is the unique name given to the circuit. Use additional parameters to further define the circuit. To configure or re-configure an existing circuit, specify the circuits name, and use additional parameters to further define the circuit.
General Circuit Configuration Considerations

z z z z z z
A circuit cannot be added to more than one logical interface. Enable IPv6 on the VAP group before you assign an IPv6 address to the circuit. If IPv6 is not enabled, a warning will be displayed when the IPv6 address is assigned to the circuit. IPv6 addressing may be used on VLANs and virtual IP addresses. Increment-per-vap cannot be configured for an IPv6 address. IPv6 addressing is not supported on the management network. If you specify the IP address for a circuit as an IPv4 address, alias addresses for that circuit can be any mixture of IPv4 and IPv6 addresses. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses. The minimum MTU size for IPv6 enabled circuits is 1280. An error message appears when enabling IPv6 on a circuit with an MTU of less than 1280, or when setting the MTU to less than 1280 on an IPv6-enabled circuit. The Crossbeam default MTU value is 1500. IPv4 / IPv6 mixed mode addresses cannot be configured using the CLI. For example, configuring the following address on a circuit will result in an error. FC00:0:0:0:0:FFFF:204.9.54.119
Circuits and WRP Interfaces

XOS exposes WRP circuits to allow you to configure Next Hop Health Checks and virtual system failover. Circuits can have a device-name starting with 'wrp' to identify that they are part of a Check Point VSX configuration. By using VRRP to assign virtual IP addresses to the WRP circuits and interfaces, XOS can monitor these circuits. A device name prefixed with 'wrp' can be used on multiple circuits. NOTE: Users cannot change this device name once it is established. Instead, you must delete the circuit and create it again. For information about how WRP circuits are used, see Configuring Multi-System High Availability on page 129. When you install and configure Check Point VSX, the application script produces additional WRP circuit configurations. These additional circuit configurations are not necessary to the Crossbeam configuration, and can make the configuration difficult to read. If you do not use, or need to review the WRP circuits, use the exclude-wrp parameter to hide the WRP circuits from show-running config.
Circuit Parameter
Table 11 lists the parameters available for configuring circuits in the NPM Mode.
Table 11.
NPM Circuits
Parameter Definition (Optional) User-defined domain (default is 1). (Optional) Circuit ID to be used, advised to use range <1-1024>
domain <1-4095> circuit-id <1-4095>
68
Parameter device-name
Definition Configures device name, also known as interface name. The device-name of a circuit or interface cannot be lo, gre0, or sit0. Device names cannot begin with eth. If necessary, specify an Incoming Circuit Group (ICG) identifier for the circuit. The ICG is used by flow rules as an additional method to identify traffic from one or more specific circuits. Valid values are 2 to 255. When enabled, Proxy ARP allows the circuit to reply to all ARP requests with known IP addresses. Specifies an existing VAP group to assign to this circuit.
incoming-circuit-group <number> proxy-arp vap-group
Configuring Circuits on a VAP Group

A VAP group is assigned while creating a circuit using the configure circuit <circuit-name> vap-group <group-name> command. The following settings are applicable after assigning a VAP group: NOTE: MAC Address Restrictions: The use of multicast mac-addresses is not allowed, unless the system-reserve option is specified. Source MAC addresses with a (binary) 1 in the least significant bit of the first octet are multicast MACs. The use of all zeros (0:0:0:0:0:0) or all ones (ff:ff:ff:ff:ff:ff) as a MAC address is not supported.
Table 12.
Circuit VAP Group Settings

Definition Assign a primary address and broadcast address to the circuit. If an IPv6 address is used, verify that enable-ipv6 has been set on the VAP group, and that the address is entered in CIDR format. NOTE: When configuring circuits, the broadcast address of the circuit or circuit alias cannot be outside of the subnet. XOS will display an error message if this condition is detected. Defines a range of IP addresses so that each VAP in the group has a unique IP address. The end-ip-addr defines the end of the IP address range. Increment-per-vap is not supported with IPv6.
Parameter ip {<ip-addr> <netmask>|<ip-addr/ netmask>} [<broadcast-addr>]
increment-per-vap <end-ip-addr>
alias {<ip-addr netmask>|<ip-addr/0-32>} [broadcast-addr] [increment-per-vap <ip-addr>] | [floating]
Assigns an alias IP address as the primary IP address for the VAP group. The VAP group must have a previously defined IP address. For example, this command assigns an IP address to 3 members of a VAP group: CBS(conf-cct-vapgroup)# ip 20.2.2.102/24 increment-per-vap 20.2.2.104 CBS(conf-cct-vapgroup-ip)# alias 20.2.2.101/24 NOTE: If you specify the IP address for a circuit as an IPv4 address, alias addresses for that circuit can be any mixture of IPv4 and IPv6 addresses. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses The floating parameter assigns the alias IP address to the master VAP, allowing traffic, cluster management, and synchronization communication to go directly to the master VAP. If a new master VAP is elected, the address floats to the new master. NOTE: Only one floating address can be used on a circuit. This parameter cannot be used with increment-per-vap.
69
Parameter default-egress-vlan-tag <id> [hide-vlan-header]
Definition (Optional) Define the egress VLAN tag, where id is an integer value ranging from 0-4094. By default, packets are sent without VLAN tags. NOTE: Some vendors do not tag VLAN 1 packets going over a VLAN trunk because they handle all non-tagged packets as VLAN 1. If the X-Series Platform is configured to look only for tagged packets, it will drop these untagged packets. To avoid this situation, configure two circuits on the same subnet, where one looks for VLAN 1 tagged packets and the other looks for non-tagged packets. [hide-vlan-header] - Optional argument under default-egress-vlan-tag. Removes the VLAN tag from the header. Used with any application where the VLAN tag must be removed for proper operation.
dhcp-relay-server-list <IP_address> <IP_address> dhcp-relay icmp-redirect ip-flow-rule-priority <value> ip-forwarding mac-addr <addr> [system-reserved]
Specify the IP addresses of the DHCP relay servers that the VAP group will use. See Note below. Enables the specified circuit to pass DHCP traffic to a relay server. See Note below. Enables ICMP Redirect Change the default IP flow rule priority. Valid values are 0 to 31. Default is 21. Use no to restore the default value. Enables IP forwarding on the circuit. System generated MAC addresses are selected from the range 00:03:D2:EX:XX:YY to 00:03:D2:FX:XX:YY. The MAC addresses in the system-reserved address pool have sequential values for X:XX, starting at 0:01. The value for YY is the system ID assigned to the X-Series Platform. Use the system-reserved parameter to specify a user-defined MAC address that is within the range of MAC addresses generated by the system. Using the existing format, change the X:XX values as required. The YY value must match the system ID, and is updated automatically when the system ID is changed. The MAC address should be changed only if there are external devices that expect a specific MAC address.
management-circuit mtu <size>
Maps a management circuit to a VAP group. Use no to delete the mapping. Specifies the MTU size for the circuit on that VAP group. The same circuit with a different VAP group can have a different MTU. Default is 1500. IPv6 circuits must have an MTU size equal to or greater than 1280.
promiscuous-mode [active]
Circuits configured in promiscuous mode receive copies of all the data packets on the logical interconnect; other circuits only receive packets destined to that VAP. Circuits with ip-forwarding should use no promiscuous-mode. Circuits used as members in a bridge should use promiscuous-mode active. Circuits used in a tap configuration should use promiscuous-mode.
replace-vlan-tag <id>
Specifies a VLAN tag that the circuit will use to replace the existing VLAN tag on the circuit traffic. This is useful to map traffic at an incoming port with a VLAN ID of x to a VLAN ID of y, especially if connected to an external switch. The <id> is a value in the range of 1 to 4094.
70
Parameter verify-next-hop-ip <ip-addr>
Definition Directs the VAP group that you are configuring to verify connectivity to the specified next-hop IP address and to perform a redundant interface failover in the event that a next-hop IP address health check fails. This option is effective only if the circuit is assigned to an interface that has been configured for redundancy. IPv6 addresses can be used if at least one address (primary or alias) is an IPv6 address.
NOTE: To enable a VAP group to use DHCP relay servers, first configure circuits with the dhcp-relay parameter, and then map those circuits to the interfaces for the DHCP server and DHCP client. Then use the dhcp-relay-server-list parameter under configure vap-group to configure the VAP group to use one or more specific DHCP servers. The following is an example of one-to-one mapping. This example ties the first gigabitethernet interface in slot 1 to a circuit called dmz with an IP address of 192.168.30.1. This circuit will appear as an interface on each of the VAPs in the VAP group called firewall. Running the ifconfig command from the CLI for each VAP in the group would show the same interface with the same IP address on each VAP. CBS# configure circuit dmz CBS(conf-cct)# device-name dmz CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip 192.168.30.1/24 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface gigabitethernet 1/1 CBS(conf-intf-gig)# logical dmz CBS(intf-gig-logical)# circuit dmz CBS(intf-gig-log-cct)# end The following example configures the same circuit, but assigns increment-per-vap and an alias. CBS# configure circuit dmz CBS(conf-cct)# device-name dmz CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip 101.1.1.2/24 increment-per-vap 101.1.1.4 alias 101.1.1.1/24 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface gigabitethernet 1/1 CBS(conf-intf-gig)# logical dmz CBS(intf-gig-logical)# circuit dmz CBS(intf-gig-log-cct)# end The following example configures a circuit between two VAP groups to serialize traffic: CBS# configure circuit serialize CBS(conf-cct)# device-name serialize CBS(conf-cct)# vap-group iss CBS(conf-cct-vapgroup)# promiscuous-mode active CBS(conf-cct-vapgroup)# exit CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip 2.2.2.1/24 increment-per-vap 2.2.2.2 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface-internal serial CBS(conf-intf-internal)# logical-all log_ser CBS(conf-intf-internal-log-all)# circuit serialize The following example assigns a circuit to two VAP groups to parallelize traffic. Traffic coming into the circuit goes to both VAP groups simultaneously.
71
CBS# configure circuit circuit1 CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip-forwarding CBS(conf-cct-vapgroup)# ip 100.123.10.1/8 CBS(conf-cct-vapgroup-ip)# end CBS# configure circuit circuit1 CBS(conf-cct)# vap-group ids CBS(conf-cct-vapgroup)# promiscuous-mode CBS(conf-cct-vapgroup)# end The following example configures a circuit for synchronization between VAPs in the VAP group, firewall. The device-name parameter changes the Linux interface name from vndxxxx to sync. CBS# configure circuit fw_sync CBS(conf-cct)# device-name sync CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip 2.2.2.1/24 increment-per-vap 2.2.2.2 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface-internal sync_circuit CBS(conf-intf-internal)# logical-all sync_circuit CBS(conf-intf-internal-log-all)# circuit fw_sync Use the show running-config circuit <name> command to display the circuit configuration parameters.
IPv6 Examples
To run traffic using IPv6 addressing, the following example assumes that the VAP group has IPv6 enabled (see IPv6 Traffic on page 47). The difference is the IP address specified for the circuit. CBS# configure circuit dmz CBS(conf-cct)# device-name dmz CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip fd00:1545:be72:e5af::cf33:54aa/64 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface gigabitethernet 1/1 CBS(conf-intf-gig)# logical dmz CBS(intf-gig-logical)# circuit dmz CBS(intf-gig-log-cct)# end The following example configures an IPv4 address on a VAP group that has previously been IPv6 enabled, and assigns an IPv6 alias. CBS# configure circuit dmz CBS(conf-cct)# device-name dmz CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip 101.1.1.2/24 CBS(conf-cct-vapgroup-ip)# alias fd00:1545:be72:e5af::cf33:33ff/64 CBS(conf-cct-vapgroup-ip)# end CBS# configure interface gigabitethernet 1/1 CBS(conf-intf-gig)# logical dmz CBS(intf-gig-logical)# circuit dmz CBS(intf-gig-log-cct)# end Use the show running-config circuit <name> command to display the circuit configuration parameters.
72
Packet Mirroring on the NPM Interfaces

Packet mirroring provides the ability to selectively copy traffic to another interface. Traffic is filtered by defining specific VLANs, MAC addresses, and other criteria. If the destination is an external NPM interface, an IDS or packet sniffer can be attached to perform an analysis of the traffic. Additionally, you can copy all packets to the eth2 interface on the NPM, and use tcpdump to inspect the packets. The pass-through feature copies filtered traffic directly out another interface, bypassing any applications or functions on the X-Series Platform. This can be invaluable to maintaining traffic flow in the event you need to troubleshoot an application. Specific traffic can be dropped at the NPM interface. If traffic coming from a VLAN, or originating from a MAC address has been identified as malicious, the NPM can be instructed to drop the traffic. All of these filters are defined using an NPM Access Control List (ACL), and assigned to an interface.
Access Control Lists (ACL Interface)

An access control list (ACL) consists of a filter and an action. The ACL is applied to an interface on the NPM, where it defines how traffic is handled. When you define an ACL for an interface, the NPM inspects all packets arriving on that interface and performs the ACL defined action on packets that match the criteria. There are two components to an access control list, the filter that defines the traffic, and the action to be taken on the defined traffic. NOTE: When you define an ACL for a group interface, the NPM applies the ACL to all physical interfaces in the group.
configure acl-interface
The configure acl-interface command creates and configures a new ACL filter. This command is also used to modify an existing ACL filter. Use the show acl-interface command to display the filtering criteria defined for each ACL filter that is currently configured on the X-Series Platform. Use the no parameter to delete the specified ACL filter. Before deleting an ACL filter, all interface ACL definitions that include that filter must be deleted. Use the show acl-interface-mapping command to display a list of the ACLs defined for each physical interface and group interface configured on the X-Series Platform. configure [no] acl-interface <ACL_filter_name>
Filters
You can define the following filters for an ACL:
z
Direction

ingress-only: NPM applies ACL action only to traffic flowing into the X-Series Platform. This is the default flow direction filtering criteria defined when you create a new ACL filter. egress-only: NPM applies ACL action only to traffic flowing out of the X-Series Platform. bidirectional: NPM applies ACL action to all traffic flowing into and out of the X-Series Platform. <VLAN_tag> Defines VLAN tag filtering criteria, using the specified VLAN tag. The NPM applies the ACL action to a packet passing through the interface only if the packets VLAN tag matches the specified VLAN tag. The VLAN tag is specified in decimal (0 to 4095) or hexidecimal (0x000-0xFFF) format.
VLAN
73
<VLAN_tag> <mask> Defines VLAN tag filtering criteria, using the specified VLAN tag and mask.The NPM applies the ACL action to a packet passing through the interface only if the packets VLAN tag matches the specified VLAN tag when the specified mask is applied. The X-Series Platform applies the mask in binary format, where 0s indicate wildcard bits. A packets VLAN tag matches the specified VLAN tag if all their non-wildcard bits match. To apply the ACL action only to packets with the specified VLAN tag, use a mask of 0xFFF. To apply the ACL action without considering a packets VLAN tag, use a mask of 0x000.
Ethernet protocol (ether-type)
<Ethernet_type_code> Defines Ethernet type filtering criteria. Specify the Ethernet type code in hexidecimal format. Valid values are from 0x0000 to 0xFFFF. The NPM applies the ACL action to a packet passing through the interface only if the packets Ethernet type code matches the specified Ethernet type code. <Ethernet_type_code> <mask> Defines Ethernet type filtering criteria, using the specified Ethernet type code and the specified mask. specify the Ethernet type code and mask in hexidecimal format. Valid values for <Ethernet_type_code> are from 0x0000 to 0xFFFF, valid values for <mask> are from 0x0000 to 0xFFFF. The X-Series Platform applies the mask in binary format, where 0s indicate wildcard bits. A packets Ethernet type code matches the specified Ethernet type code if all their non-wildcard bits match. To apply the ACL action only to packets with the specified Ethernet type code, use a mask of 0xFFFF. To apply the ACL action without considering a packets Ethernet type code, use a mask of 0x0000.
Source-mac
<source_MAC_address> The NPM applies the ACL action to a packet only if its source MAC address matches the specified source MAC address. Specify the source MAC address using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff). NOTE: Configure an ACL filter with the source-mac command using the direction ingress-only command. A warning will be displayed if the direction is specified as egress-only or bidirectional.
<source_MAC_address> <mask> You must specify the source MAC address and mask using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff). NOTE: The X-Series Platform applies the mask in binary format, where 0s indicate wildcard bits. A packets source MAC address matches the specified source MAC address if all their non-wildcard bits match. To apply the ACL action only to packets with the specified source MAC address, use a mask of ff:ff:ff:ff:ff:ff. To apply the ACL action without considering a packets source MAC address, use a mask of 00:00:00:00:00:00.
Destination-mac NOTE: Configure an ACL filter with the destination-mac command using the direction ingress-only command. A warning will be displayed if the direction is specified as egress-only or bidirectional.
<destination_MAC_address> The NPM applies the ACL action only to packets whose destination MAC addresses match the specified filtering criteria. You must specify the destination MAC address using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff). <destination_MAC_address> <mask> The NPM applies the ACL action to a packet only if its destination MAC address matches the specified destination MAC address when the specified mask is applied. You must specify the destination MAC address and mask using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff). NOTE: The X-Series Platform applies the mask in binary format, where 0s indicate wildcard bits. A packets destination MAC address matches the specified destination MAC address if all their non-wildcard bits match. To apply the ACL action only to packets with the specified destination MAC address, use a mask of ff:ff:ff:ff:ff:ff. To apply the ACL action without considering a packets destination MAC address, use a mask of 00:00:00:00:00:00.
74
Actions
The actions direct the NPM about how to handle the traffic that meets the ACL filter criteria, and are specified inline while configuring the ACL with the interface. See Packet Mirroring on the NPM Interfaces on page 73 for details about each of the actions and configuring the interfaces.
Configuring Access Control Lists

Once the ACL has been created and the filters defined, use the following command to assign the ACL to an interface and define the action the NPM will take with the traffic that meets the filter criteria. acl-interface-mapping <ACL_filter_name> {drop | mirror {gigabitethernet | 10gigabitethernet} <slot>/<port> | pass-through {gigabitethernet | 10gigabitethernet} <slot>/<port> | capture} The following actions can be specified:
z
Mirror: This action copies packets to one or more interfaces. Often the interface is connected to an IDS or packet sniffer that performs an analysis of the interface traffic. This functionality is identical to SPAN port functionality. Pass-through: This action will bypass any applications or functions of the X-Series Platform and will pass traffic directly to an outgoing interface or interfaces. One consideration when using pass-through; Pass-through traffic will not be sent to internal interfaces or VAPs. This includes traffic for protocols that are critical to network and interface connectivity, such as LACP and ARP. Do not configure an ACL to filter and pass-through all traffic on an interface or the network may shut down. Drop: Instructs the NPM to drop traffic that meets the filter criteria. If traffic coming from a VLAN, or originating from a MAC address has been identified as malicious, the ACL can specify that location and instruct the NPM to drop the traffic. As with the pass-through action, use care when specifying the drop action. If necessary network protocol information is dropped from the interface it may cause the network, or section of the network, to shut down. Capture: Primarily a troubleshooting function, it instructs the NPM to copy all packets that match the criteria defined in the ACL filter to the local eth2 interface on the NPM. You can use tcpdump to inspect packets captured. By default, the eth2 interface is down. You will have to use ifconfig to bring the eth2 interface up and perform the tcpdump. Once the dump is complete, return the eth2 interface to the down state.
ACL Configuration Considerations

ACLs can be used in the following manner:
z z z z z z z z z z
Remote mirroring (the mirrored port is on a different NPM than the source port) is not supported. Remote pass-through (the destination port is on a different NPM than the source port) is not supported. An interface or multi-link group-interface must be configured before it is used as a target to pass-through or mirror traffic. Multiple ACL filters can be configured for the same interface. When a packet arrives on an interface, the NPM applies the filters to the packet one at a time, in the order that the filters were configured. To reconfigure the precedence of acl-interface-mappings, delete the mappings and redefine them in the order you want. Multiple interfaces can be configured with the same ACL filter. An ACL filter can be configured to mirror traffic to multiple interfaces on the same NPM. An interface cannot mirror or pass-through traffic to itself. An interface can be configured to mirror or pass-through traffic on a port that is part of a multi-link group-interface, so long as both interfaces are on the same NPM. A multi-link group-interface can mirror traffic to a single interface, if all group member interfaces and the target mirror interface are on the same NPM.
75
An interface which is part of a multi-link group interface cannot mirror traffic to another interface. Instead, mirror the traffic from the whole group to another interface
Use the no acl-interface command to remove the specified ACL from a physical interface. IMPORTANT: If multiple ACLs are defined for the physical interface, the no parameter will only delete the most recently defined ACL. Use the following show commands to view the criteria for the ACL:
z z z
show running-config to determine the order in which ACLs are defined for a physical interface. show acl-interface to display the conditions defined for each ACL that is currently configured on the X-Series Platform. show acl-interface-mapping to display a list of the ACLs assigned to each physical interface, and display the action configured for each ACL.
Single Outgoing Interface Example

The following example configures an ACL interface filter named single_acl to filter only packets on ingress from VLAN 201. The filter is then applied to the interface below, and assigned an action. CBS# configure acl-interface single_acl CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# vlan 201 The following command configures gigabitethernet port number 1 on the NPM installed in slot number 1 to pass traffic between the X-Series Platform and an external network. This command also places you in the conf-intf-gig context in which you can configure settings for the gigabitethernet interface, and create and configure logical interfaces for that interface. CBS# configure acl-interface-mapping CBS(conf-acl-intf-map)# interface gigabitethernet 1/1 CBS(conf-acl-map-intf-gig)# acl-interface single_acl mirror CBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 1/10 CBS(conf-acl-map-intf-gig-mirror)# The show command displays the ACL configuration: CBS# show acl-interface-mapping Primary (interface/group) : gigabitethernet 1/1 ACL Interface : single_acl action : mirror Destination interface(s) : gigabitethernet 1/10 The NPM inspects all packets arriving on gigabitethernet interface 1/1, and copies all packets that match the criteria defined in the ACL interface filter called single_acl to gigabitethernet interface 1/10.
Multiple Outgoing Interface Example

The following example configures an ACL interface filter named multiple_acl to filter only packets on ingress. The filter is then configured to mirror traffic to multiple ports on the same NPM. IMPORTANT: Packet mirroring cannot send traffic to remote NPMs. Packets can only be mirrored to ports on the same NPM. CBS# configure acl-interface multiple_acl CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# exit CBS# configure acl-interface-mapping CBS(conf-acl-intf-map)# interface gigabitethernet 1/1 CBS(conf-acl-map-intf-gig)# acl-interface multiple_acl mirror CBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 1/10 CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 1/11
Configuring Interfaces 76
CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 1/12 CBS(conf-acl-map-intf-gig-mirror)# end The show command displays the ACL configuration: CBS# show acl-interface-mapping Primary (interface/group) : gigabitethernet 1/1 ACL Interface : multiple_acl action : mirror Destination interface(s) : gigabitethernet 1/10, 10gigabitethernet 1/11, 10gigabitethernet 1/12 (1 row)
Group Interface Example

The following example configures an ACL interface filter named vlan70, and then mirrors all vlan 70 traffic from group-interface mlt to port 2/7. CBS# configure acl-interface vlan70 CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# vlan 70 CBS(conf-acl-intf)# end CBS# configure group-interface mlt CBS(conf-group-intf)# interface-type gigabitethernet CBS(conf-grp-intf-gig)# exit CBS(conf-group-intf)# mode multi-link circuit gr10 CBS(conf-group-intf)# interface 2/9 CBS(conf-group-intf)# interface 2/10 CBS(conf-group-intf)# logical vlan70 circuit vlan70 CBS(conf-group-intf)# end CBS# configure acl-interface-mapping CBS(conf-acl-mapping)# group-interface mlt CBS(conf-acl-map-group-intf)# acl-interface vlan70 mirror CBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/7 CBS(conf-acl-map-group-intf-mirror)# end
Overlapping ACL Filters Example

The following example illustrates the behavior when different ACL filters are configured on the same port. In this case, traffic coming into port 2/1 on vlan 70 (the acl filter vlan70 explicitly handles vlan 70) is mirrored to ports 2/6 and 2/9. All other traffic coming in to port 2/1 is mirrored to port 2/5. The order in which the filters are mapped to the ports determines the priority in which they are applied to the traffic on the port. CBS# configure acl-interface vlan70 CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# vlan 70 CBS(conf-acl-intf)# exit CBS# configure acl-interface all CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# exit CBS# configure acl-interface-mapping CBS(conf-acl-mapping)# interface gigabitethernet 2/1 CBS(conf-acl-map-group-intf)# acl-interface vlan70 mirror CBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/6 CBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/9 CBS(conf-acl-map-group-intf-mirror)# exit CBS(conf-acl-map-group-intf)# acl-interface all mirror CBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/5 CBS(conf-acl-map-group-intf-mirror)# end
Overlapping Port Example

The following is an example of overlapping port mirroring. In this case, vlan 101 is mirrored to the interfaces gigabitethernet 4/10, and 10gigabitethernet 4/11, 4/12. An additional acl-interface filter is configured to mirror traffic from port 4/1 to port 4/11. Since the first filter configured (vlan101) includes 4/11, traffic will be mirrored to all ports. CBS# configure acl-interface vlan101 CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# vlan 101 CBS(conf-acl-intf)# exit CBS# configure acl-interface vlan102 CBS(conf-acl-intf)# direction ingress-only CBS(conf-acl-intf)# vlan 102 CBS(conf-acl-intf)# exit CBS# configure acl-interface-mapping CBS(conf-acl-intf-map)# interface gigabitethernet 4/1 CBS(conf-acl-map-intf-gig)# acl-interface vlan101 mirror CBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 4/10 CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/11 CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/12 CBS(conf-acl-map-intf-gig-mirror)# exit CBS(conf-acl-map-intf-gig)# acl-interface vlan102 mirror CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/11 CBS(conf-acl-map-intf-gig-mirror)# end
Configuring Chassis Resource Protection

Chassis resource protection provides configuration parameters to prevent malicious traffic from consuming critical NPM resources. User configurable parameters based on TCP flow validation and flow table limits are set to monitor and filter traffic flow. Unvalidated flows can be dropped or aggressively aged-out, preventing the overflow of buffers and flow tables. Additional settings are provided to make effective use of resources for handling fragmented TCP and UDP packets. Basic per packet validation of TCP/IP packets can be configured to identify and potentially drop invalid packets at line rate to protect APM and end-hosts. These features must be enabled by the user, and are off by default.
TCP Flow Validation

When a TCP packet enters the system, XOS creates a new entry in the flow table and sets the status to unvalidated. In the unvalidated state the uni-directional flow has a short idle time-out of 5-60 seconds. When both directions of a TCP connection pass through the chassis, and the three way handshake has completed, both directions of the connection are validated. They then receive a user-specified idle time-out, which may be as much as one hour. By validating TCP flows in this manner, and aggressively aging out the unvalidated flows, the NPM flow table is protected from TCP-based denial of service attacks such as a SYN flood. Both directions of a TCP connection must flow through the chassis in order to validate the TCP connection. The TCP packet ordering must also be maintained for TCP flow validation on the chassis. For example, in external TAP configurations where the traffic does not flow through the chassis, the NPM may receive re-ordered TCP frames and fail to validate TCP connections. After a TCP connection has been validated, the TCP FINs and RSTs are validated before being acted upon. This prevents premature and incorrect termination of TCP connections due to spoofed RSTs.
Flow Table Limits

TCP flow validation alleviates denial of service due to TCP based flood attacks. To protect NPM resources from UDP and ICMP based attacks, flow table limits are configured for individual protocols.
78
UDP and ICMP flows are stateless, and do not have flow setup or termination schemes. New flows are assigned the idle time-out specified in the ip-flow rule. By specifying flow table limits on each flow type, these flows do not consume critical NPM resources, preventing the setup of other flows.
Establishing a Traffic Baseline

Use the Greenlight Element Manager to monitor traffic and establish a traffic baseline. This baseline allows you to profile your traffic and identify anomalous traffic patterns, or put measures in place which will prevent attacks such as TCP SYN floods. 1. 2. Open a Web browser and navigate to the Greenlight Element Manager using a secure connection and the IP address or name of your X-Series Platform (https://<IP_address>). Review the Flows View for an overview of flows per protocol.
Figure 5.
GEM Flows View
3.
Double-click on the pie chart to open the Flow Utilization by Protocol graph. Use this tool to monitor traffic on the X-Series Platform. For additional information on configuring specific utilization graphs, refer to the Greenlight User Assistance.
Figure 6.
GEM Flow Utilization by Protocol
79
Use the following list to understand the type of traffic your network is processing, and the type of traffic that could be a threat.

What is the most common type of traffic: TCP, UDP, ICMP, or Other-IP. Identify a baseline for normal traffic. Use the Historical data option to look back over time. Identify the high end of a range. Are there consistent spikes or low spots in the data?
Use the information gathered to configure parameters to monitor, pass, or drop traffic of a specific protocol based on the number of flows passing through the system. For example, if the flow table limit action is set to drop UDP traffic, the following scenario occurs: When the number of new UDP flows reaches a preliminary threshold, new flows are monitored (counted). As the number of new flows increases toward the user-defined limit, the chance of the preemptive dropping of a new flow increases. When the total number of active flows reaches the predefined limit, all new UDP flows will be dropped. Traffic associated with established flows will continue to pass. The system default is to pass all traffic. If the flow table limit is configured for a protocol, and the action is set to pass, then new flows of that protocol will not be dropped even when the protocol flow table limit is passed. Traffic will continue to pass until the flow table resources are consumed.
Fragment Handling
With IP traffic, packets are mapped to a flow based on protocol, source and destination IP addresses, and source and destination ports. Mapping TCP and UDP packets to flows requires the source and destination port. When TCP and UDP packets are fragmented, the individual fragments must be associated with source and destination ports before mapping the fragments to flows. Some fragmented TCP and UDP traffic is expected and is handled without impact on the system. A high number of fragmented packets will consume resources and reduce the effective throughput of traffic. Examples of fragmented or nonconforming packets that may be part of an attack are packets with:
z z z z z
Bad or overlapping offsets Spoofed packet size High numbers of missing fragments Duplicate Head of Packet A high number of small fragments
Fragment handling employs heuristics to process the fragments. This logic makes decisions whether to drop fragments, aggressively age-out fragmented packets, or forward the fragments. When under processing pressure, nonconforming packets are more likely to be dropped. The fragment handling heuristics are off by default. Crossbeam recommends enabling fragment handling on your X-Series Platform. Once enabled, it can be specifically disabled on a per-logical line basis. For instance, for an IPS application that inspects all traffic, disable fragment handling on the interfaces or logical lines attached to the application. NOTE: IPv6, ICMP, and Other-IP traffic are not affected by fragment handling settings.
TCP/IP Packet Validation

TCP/IP Packet validation is an inspection process to detect invalid TCP/IP frames. Packet Validation performs the following checks:
z z z z z
TCP/IP header information TCP/IP checksum IP option length Packet size Flags
A parameter of drop or pass is configured for invalid packets. The default action is to pass all packets. Statistics are maintained for all packet validation checks regardless of the drop or pass parameters.
80
Resource Protection Workflow

One example of a denial of service attack is a SYN flood. Without TCP flow setup validation enabled, this type of attack will fill the NPM flow table with unvalidated traffic, consuming the available resources. With no space in the flow table for new flows, new traffic is dropped. By using a combination of the Greenlight Element Manager to identify a traffic profile, and resource protection parameters to configure flow table settings, you can limit resources devoted to TCP traffic and prevent a denial of service situation resulting from a SYN flood. This workflow provides configuration steps to monitor and preserve NPM resources in the event of flooding-based denial of service attacks. Configuration steps for packet validation and fragment handling are included as a best practice. These settings will preserve NPM resources against other malicious traffic. 1. 2. Using GEM and the steps in the Traffic Baseline section, establish a baseline for TCP traffic. See Establishing a Traffic Baseline on page 79. From the command line interface, enable resource protection for the chassis. Command: CBS# configure chassis-resource-protection CBS(conf-resource-protection)# enable CBS(conf-resource-protection)# NOTE: To restore all the resource protection settings to default values, use the no chassis-resource-protection command. 3. TCP flow validation is enabled by default when you enable chassis resource protection. With TCP flow validation enabled, you have the option of disabling validation on a per-flow-rule basis, using bypass-tcp-flow-setup-validation. Typically, bypass should be set when the topology prevents TCP packets from being recieved in the correct order, for example, when an external tap is configured. When bypassed, the TCP sequence numbers are updated, and FIN and RST sequence numbers are validated before flow removal. 4. Configure the partitions in the NPM flow table using the information from the baseline exercise. Use the traffic flow baseline for the threshold value. For instance, if your typical traffic pattern uses 60% of your NPM capacity, set that as the flow table threshold. The TCP threshold value reflects how much more of the NPM resources you want to devote to TCP traffic above the 60%. If your network typically processes UDP traffic, set a limit value here. The longer timeouts assigned to stateless protocols makes them equally susceptible to flood attacks. Assigning a limit guarantees these flows do not consume all the NPM resources and prevent the setup of other flows. All protocols must be configured, even if the value entered is 0. The flow table partition threshold range is 0-85%. Individual protocol limits are specified as a percentage of the total flow table, and the total for all values must equal 100. Command: CBS(conf-resource-protection)# configure flow table partition threshold 60 tcp 30 udp 10 icmp 0 other-ip 0 CBS(conf-flow-table-partition)# 5. Configure the action taken when the protocol (TCP in this case) limit is reached. To protect against a SYN flood, set the action to drop. Command: CBS(conf-flow-table-partition)# flow-table-profile tcp CBS(conf-rp-table-profile)# table-limit action drop CBS(conf-rp-table-profile)# exit CBS(conf-flow-table-partition)#
81
6.
Configure the action to be taken when UDP partition limit is reached. Set the action to drop. Command: CBS(conf-flow-table-partition)# flow-table-profile udp CBS(conf-rp-table-profile)# table-limit action drop CBS(conf-rp-table-profile)# exit CBS(conf-flow-table-partition)# end CBS#
7.
Configure TCP/IP packet validation, and set the following actions to drop. Command: CBS# configure packet-validation CBS(conf-pkt-validation)# validate-tcp-packet action drop CBS(conf-pkt-validation)# validate-tcp-checksum action drop CBS(conf-pkt-validation)# validate-ip-packet action drop CBS(conf-pkt-validation)# end CBS# When packet validation is enabled, the individual validation checks are enabled with a default action of pass. These checks verify that the packets are well-formed, and are not corrupted. Setting the action to drop removes invalid packets at the NPM interface and reduces resource consumption on the APMs and the end-hosts. Statistics are maintained for all checks regardless of the configured action. These checks are applied to all circuits that are connected to external interfaces. Individual circuits can be configured independently of the global settings.
8.
Configure fragment handling parameters. Fragment handling settings are designed to identify common anomalies and, under stressful conditions allow NPM resources to be directed toward good traffic. It is recommended to enable fragment handling on the X-Series Platform. In the case of an IPS application that requires visibility into all the traffic, this functionality can be disabled. Command: CBS(conf-resource-protection)# fragment-handling-options CBS(conf-rp-frag-handling)# exit CBS(conf-resource-protection)# end CBS# Fragment handling settings are set globally and may be disabled per logical interface.
Resource protection is now configured globally on your X-Series Platform. All traffic will be subject to these parameters.
Disabling Flow Table Limit Action per Interface

Some interfaces, such as those handling application management traffic, are not typically subject to attack. However, all NPM network interfaces are subject to the global resource protection settings. In the case of a SYN flood on an external interface, if the flow table partitioning action for the protocol is set to drop when the limit is reached, all TCP traffic on all interfaces is dropped. The normal application management traffic will be blocked. For internal traffic that passes through an interface on the NPM, you can disable the flow table limit action settings on a per-interface basis.
Workflow
To disable individual flow table limit action settings on an existing, configured interface, use the following steps: 1. From the CLI, enter the configure interface context. CBS# configure interface gig 1/1
2.
Enter the logical and circuit for the interface. CBS(conf-intf-gig)# logical mgmt CBS(intf-gig-logical)# circuit mgmt
3. 4.
Override the flow table limit action for the individual management circuit. CBS(intf-gig-log-cct)# no flow-table-limit Use end to return to the main CLI context. Repeat the steps for each interface required. CBS(intf-gig-log-cct)# end CBS#
Additional resource protection settings can be disabled on individual interfaces using the procedure above. CBS# configure interface gig 1/1 CBS(conf-intf-gig)# logical mgmt CBS(intf-gig-logical)# circuit mgmt CBS(intf-gig-log-cct)# ? [no] flow-table-limit [no] fragment-handling-options Overrides the global configuration and disables flow table limit per interface Overrides the global configuration and disables fragment handling options per interface Overrides the global configuration and disables packet-validation per interface
[no] packet-validation
Configuring Redundant Interfaces

To create redundant interfaces, specify the name of the master physical interface being backed up, name of the backup physical interface, and failover mode. Both master and backup must be existing configured interfaces. Upon failover, the original MAC address and circuit IP addresses switch over to the backup port. The gratuitous ARP forces Layer 2 devices to relearn the port on which they have learned this MAC address. IMPORTANT: When configuring interface redundancy, a backup interface in an active-active configuration cannot be configured to pass traffic in the same broadcast domain as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or both pass untagged traffic. In addition, an interface configured with logical-all cannot be used as a redundant interface. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANs tagged 300. Use the following command to define a master interface and its backup interface, where slot/port is the actual NPM slot and port location: configure redundancy-interface master {{gigabitethernet | 10gigabitethernet} {slot/port}} backup {{gigabitethernet | 10gigabitethernet} {slot/port}} mac-usage master failovermode <mode> The failover mode can be set to the parameters described in Table 13.
Table 13.
Failover Mode Parameters

Definition Failover does not occur. Switch to backup interface now. Upon failure of the master physical interface, the backup interface services traffic. The original Master resumes being Master when it becomes available.
Parameter no-failover manual-failover preemption-on
83
Parameter preemption-off
Definition Upon failure of the master physical interface, the backup interface services traffic. The original Master does not resume being Master when it becomes available, unless the backup fails. Upon failure of the master physical interface, the backup interface services traffic. However, the traffic does not switch over to the original master even should the backup interface fail.
manual-swback
The following example configures the interface at port 1 of the NPM in slot 1 to be backed up by port 2 of the same NPM. Should the master interface fail, the backup interface services traffic but the traffic does not switch back to the original master should the backup fail. CBS# configure redundancy-interface master gigabitethernet 1/1 backup gigabitethernet 1/2 mac-usage master CBS(conf-intf-redun)# failovermode manual-swback NOTE: The mac-usage master parameter is required for the redundancy-interface command. Here is an example of configuring an interface for redundancy: CBS# configure interface gigabitethernet 4/6 CBS(conf-intf-gig)# logical ispxena2 CBS(intf-gig-logical)# circuit ispxena2 CBS(intf-gig-logical)# end CBS# configure interface gigabitethernet 4/7 CBS# configure redundancy-interface master gigabitethernet 4/6 backup gigabitethernet 4/7 mac-usage master failovermode preemption-off If you configure a bridge-mode interface for redundancy, the backup interface must be configured as standby-only, as shown in the following example: CBS# configure interface gigabitethernet 4/6 CBS(conf-intf-gig)# logical ispxena2 CBS(intf-gig-logical)# circuit ispxena2 CBS# configure interface gigabitethernet 4/7 CBS(conf-intf-gig)# standby-only CBS# configure redundancy-interface master gigabitethernet 4/6 backup gigabitethernet 4/7 mac-usage master failovermode preemption-off To display the currently configured redundant interface, use the following command: CBS# show redundancy-interface Master Intf Backup Intf Active Intf ------------------------------gig 1/3 gig 2/3 Master gig 1/7 gig 2/7 Backup MacUsage -------master master FailOverMode -----------preemption-on preemption-off
84
Configuring a Multi-Link Group Interface

You can aggregate physical links into a group interface using the multi-link mode. All interfaces in a group interface must be of the same physical interface type (gigabitethernet or 10gigabitethernet). The links are aggregated using Link Aggregation Control Protocol (LACP). To create a multi-link group, you must first create a non-VLAN circuit to use for that group. NOTE: A multi-link group interface implies a promiscuous-mode and logical-all statement. Multi-link tagged bundles should only be sent across the required VLANs to avoid unnecessary processing. To view and create group interfaces using EMS, open Configuration > Interfaces > Group Data Interfaces then select Group Physical Interface or Group Logical Interface. Use the Help button for details. In the Command Line Interface, use the configure group interface command. NOTE: If you have a multi-link group connected to a multi-link group on a different X-Series Platform, each system must have a unique system ID, as set with the configure system-identifier <system-id> command. Not all settings are used on the 10gigabitethernet interface, as noted.
Table 14.
Group Interface Settings for the NPM

Definition multi-link Aggregates physical ports into one interface using Link Aggregation Control Protocol (LACP). You must first create a non-VLAN circuit to use for that group. The circuit <name> assigns an existing circuit to the group interface. Once assigned, this circuit cannot be removed unless you delete then recreate the group interface. Be aware that there are restrictions to the number of circuits that can be assigned to a bridge, as described in Circuit Restrictions for Bridges on page 117.
Setting mode {multi-link } circuit <name>
interface-type auto-negotiate
Specify the physical interface type for the group. Choices are gigabitethernet or 10gigabitethernet. All interfaces in the group must be the same type. Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Matches best speed for multi-speed devices and uses automatic sensing to support full duplex operation. If using fiber gigabitethernet ports, auto negotiation must be enabled. Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures half or full duplex mode. For gigabitethernet ports, this is only applicable when the interface is configured for a copper connection. Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures media speed. For gigabitethernet ports, this is only applicable when the interface is configured for a copper connection. This setting overrides the auto-negotiation speed. Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures pause frame, also known as flow control. Configured under the interface-type context. The group interface can be disabled or enabled. Default is enabled. Add individual interfaces to the group. Settings applied to the group are automatically written to the individual interfaces. Configured under each interface context. An interface is enabled if both the group interface and individual interface are enabled. Individual physical interfaces may also be disabled / enabled independent of the group interface.
85
duplex-mode
media-speed
pause-frame
enable (group interface) interface slot/port enable (individual interfaces)
You can disable and enable a group interface with the [no] enable command: CBS# config group-interface test interface-type gigabitethernet no enable You can enable or disable individual physical devices using the enable command, once the device has been specified: CBS# config group-interface test interface 1/1 no enable NOTE: When configuring physical link parameters such as speed, MAC address, auto-negotiate, duplex mode, and pause frame for the group interface, the parameters are applied to each physical interface in the group. By default, the interface is enabled, auto-negotiation is enabled, duplex mode is full, and the interface is not in standby mode. The following example configures circuit1 for VLAN 2, and configures a second non-VLAN circuit (required). The non-VLAN logical line must be created before the VLAN logical line. When specifying the group interface mode, you must specify a circuit. The circuit is automatically configured with a logical interface that passes all VLAN and non-VLAN traffic. CBS# configure group-interface lab CBS(conf-group-intf)# interface-type gigabitethernet CBS(conf-grp-intf-gig)# exit CBS(conf-group-intf)# mode multi-link circuit circuit2 CBS(conf-group-intf)# interface 1/1 CBS(conf-grp-intf-intf)# exit CBS(conf-group-intf)# interface 1/2 CBS(conf-grp-intf-intf)# exit CBS(conf-group-intf)# interface 1/3 CBS(conf-grp-intf-intf)# exit CBS(conf-group-intf)# interface 1/4 CBS(conf-grp-intf-intf)# exit CBS(conf-group-intf)# logical logical11 ingress-vlan-tag 2 CBS(conf-group-intf-logical)# circuit circuit1 To have traffic coming into this group interface directed to two applications, assign Circuit1 to two VAP groups. For example: CBS# configure circuit circuit1 CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# default-egress-vlan-tag 2 CBS(conf-cct-vapgroup)# ip-forwarding CBS(conf-cct-vapgroup)# ip 100.123.10.1/8 CBS(conf-cct-vapgroup-ip)# exit CBS# configure circuit circuit1 CBS(conf-cct)# vap-group ids CBS(conf-cct-vapgroup)# promiscuous-mode In the example of directing traffic to two or more VAP groups, you must also assign the LACP circuit to all the VAP groups. Continuing the previous example, Circuit2 is the LACP circuit, and you would assign Circuit2 to both VAP groups as follows: CBS# configure circuit circuit2 CBS(conf-cct)# vap-group firewall CBS(conf-cct-vapgroup)# ip-forwarding CBS(conf-cct-vapgroup)# exit CBS# configure circuit circuit2 CBS(conf-cct)# vap-group ids CBS(conf-cct-vapgroup)# promiscuous-mode To view existing group interfaces from the CLI, use the show group-interface command.
86
MAC Address Inheritance

If you apply a user-defined MAC address to the mode multi-link circuit, additional circuits (e.g. VLAN circuits) mapped to the same VAP group inherit the mode circuits MAC address automatically. If the user changes the MAC address on the mode circuit, the VLAN circuits will inherit the new MAC address. If you use the logical command to map additional circuits (e.g. VLAN circuits) to a mode multi-link group interface, the VLAN circuit does not inherit the mode circuits MAC address. If the user changes the MAC address on the mode circuit, the VLAN circuits will retain their originally configured MAC addresses.
Interface Status Group

This command ties the status of interfaces or group-interfaces together. If all interfaces and group-interfaces in an interface-status-group are UP, then the state of the interface-status-group is UP. If any interface or group-interface in the interface-status-group is DOWN, then the interface-status-group state is DOWN. CBS# configure interface-status-group <group_name> CBS(conf-intf-status-group)# gigabitethernet <slot/port> | 10gigabitethernet <slot/port> | group-interface <name> When interface-status-group command is enabled, all members of the interface status group will stop passing traffic if one or more members fails. When interface-status-group is disabled, healthy members will continue to pass traffic if one group interface member fails. Use show interface-status-group to display the interface status groups and the associated interfaces. Status grouping is disabled by default.
87
88
7
Configuring Management Interfaces
This chapter provides information on configuring interfaces, including access lists. It includes the following major topics:
z z z z z
Physical Management Interfaces on page 89 Configure Management Physical Interfaces on page 90 Configure Access Lists on page 91 Define an Access List to a Management Interface on page 94 Display Management Access Lists on page 94
Physical Management Interfaces

The physical management interfaces are provided by the Control Processor Modules (CPMs). The management interfaces allow you to manage the configured interfaces. The physical interfaces are identified by chassis slot number of the CPM (13 or 14), physical port number (1 or 2) of the CPM, and interface type (gigabitethernet). Table 15 lists the CPM physical interfaces for each platform.
Table 15.
Physical Interfaces for X-Series Platforms
X80 / X60 X20/X30 X80-S Chassis Chassis Port # Interface Type Chassis Slot # Slot Slot # 13 13 14 14 6 6 7 7 7 7 1 2 1 2 gigabitethernet gigabitethernet gigabitethernet gigabitethernet
Description
Gigabit Ethernet management port Gigabit Ethernet logging port on CP1 Gigabit Ethernet management port Gigabit Ethernet logging port on CP2
NOTE: Inside NAT does not failover from one CPM management interface to the other CPM. The inside NAT should be explicitly configured identically on both CPM management interfaces. IMPORTANT: If you configure two management interfaces on the same slot, they cannot be on the same subnet.
89
Configure Management Physical Interfaces

The following is an example of displaying the management physical interface configuration: CBS(conf-mgmt-gig)# show Management Interface Enabled (true/false) IP Address Broadcast Address Auto Negotiate Enabled (true/false) Input Access List Output Access List (1 row) To configure a management physical interface: 1. Configure the interface type and CPM slot location and port. Command: CBS# configure management gigabitethernet 13/2 CBS(conf-mgmt-gig)# 2. Define the MAC address for the management interface using the MAC address of the physical port. If you do not supply the MAC address, the system will either use the NIC MAC, or will automatically generate the MAC address. Command: CBS(conf-mgmt-gig)# mac-addr 40:03:d4:55:20:23 CBS(conf-mgmt-gig)# 3. Define the Maximum Transfer Unit (MTU) size for the management interface. Specify the size in bytes; the range is 68 to 1536, or use the system default of 1500. Command: CBS(conf-mgmt-gig)# mtu 1500 CBS(conf-mgmt-gig)# 4. Define the IP address and network mask for the management physical port. NOTE: You may provide the IP address and the Network Mask in either CIDR format (172.16.10.10/24), or in dotted-decimal format followed by space on the same line (172.16.10.10 255.255.255.0) Command: CBS(conf-mgmt-gig)# ip-addr 172.16.10.10 255.255.255.0 CBS(conf-mgmt-gig)# 5. 6. Enable the interface as follows: CBS(conf-mgmt-gig)# enable Define the access list. The access list defines which subnets or specific IP addresses are allowed to connect to and manage the CPM management interface. To create an access list see Configure Access Lists on page 91. Command: CBS(conf-mgmt-gig)# access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255 CBS(conf-mgmt-gig)# access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255 : : : : : : : gigabitethernet 14/1 t 172.16.28.42/24 172.16.28.255 t 1001 1002
90
Configure Access Lists

The access list allows connectivity to the CPM interface from specified networks and hosts, and allows you to specify IP addresses that have access to the CPM management interfaces. By default, all access is denied. In order to manage the X-Series Platform, create an access rule allowing you to perform management functions. An example of a simple access rule would be one that allows:
z z z z z
Access to the system (ssh, telnet, https) The ability to transfer files or policies (scp/ftp) ICMP to ping the system when troubleshooting A list of the management source addresses with access to the system Permissions for those ip addresses
During the XOS Install Interview, you can choose to allow all traffic, which will automatically create an allow/all rule. This may be safest in a non-production environment, where external access to the system is limited. The following is an example of an allow/all access list. configure access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255 configure access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255 TCP/UDP configure access-list <number> {deny|permit} {tcp|udp} {{source-ip <src-ip-address> <source-wildcard>}|source-any} {source-port-any|source-port-name <name>|source-port <0-65535> [<0-65535>]} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} {destination-port-any|destination-port-name <name>| destination-port <0-65535> [<0-65535>]} [log] ICMP configure access-list <number> {deny|permit} icmp {{source-ip <src-ip-address> <source-wildcard>}|source-any} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} {icmp-message <message>|icmp-type <0-255>} [log] IP/Protocol Number configure access-list <number> {deny|permit} {ip|protocol-number <0-255>} {{source-ip <src-ip-address> <source-wildcard>}|source-any} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} [log] NOTE: A wildcard setting of 255.255.255.255 permits any address regardless of the IP address configured. To configure the access list to permit a specific host address, the wildcard used should be 0.0.0.0. Where: <number> deny permit ip|tcp|udp|icmp <src-ip-address> <source-wildcard> Number of an access list. This is a decimal number from 0 through 65535. Denies access if the conditions are matched. Permits access if the conditions are matched. Protocol name or number. IP address of the network or host from which the packet is being sent. Use a 32-bit quantity in four-part dotted-decimal format. Wildcard bits applied to the source. Use a 32-bit quantity in four-part dotted-decimal format. Places ones in the bit positions you want to ignore.
91
source-any source-port-any source-port-name <name>
Abbreviation for a source and source-wildcard of 0.0.0.0 255.255.255.255. Configures the source port to be any. Specifies the source low/high port-name; names are as follows:
z z z z z z z z z z z z z z z z z z z z
source-port <number> Source low/high port-number value. Valid numbers are from 0 - 65535 ftp-data File Transfer Protocol Data (20) ftp File Transfer Protocol (21) ssh Secure SHell (22) telnet Telecommunications Network Protocol (23) smtp Simple Mail Transfer Protocol (25) time Time (37) domain Domain Name Server (53) bootps Bootstrap Protocol Server Protocol (67) bootpc Bootstrap Protocol Client Protocol (68) tftp Trivial File Transfer Protocol (69) http Hyper Text Transfer Protocol or www or www-http (80) rtelnet Remote Telecommunications Network Protocol (107) pop3 Post Office Protocol Version 3 (110) nntp Network News Transfer Protocol (119) ntp Network Time Protocol (123) imap Internet Message Access Protocol (143) snmp Simple Network Management Protocol (161) ldap Lightweight Directory Access Protocol (389) https Secure Hyper Text Transfer Protocol (443) isakmp Internet Security Association and Key Management Protocol (500)
<des-ip-address>
IP address of the network or host receiving the packet being sent. There are two alternative ways to specify the destination:
z z
Use a 32-bit quantity in four-part dotted-decimal format Use any as an abbreviation for a source and source-wildcard of 0.0.0.0 255.255.255.255
<destination-wildcard> Wildcard bits applied to the destination. Use a 32-bit quantity in four-part dotted-decimal format. Places ones in the bit positions you want to ignore. destination-any destination-port-any destination-port-name <name> destination-port <number> Abbreviation for a destination and destination-wildcard of 0.0.0.0 255.255.255.255. (TCP/UDP Only) Configures the destination port to be any. (TCP/UDP Only) Specifies the destination low/high port-name. Names are the same as the source-port names. (TCP/UDP Only) Specifies the destination low/high port number. Values are 0 to 65535.
92
icmp-message <message-name>
Valid message names as listed below: NOTE: Enter the text string, for example, network-redirect. As an alternative, see the next row in this table for information on how to enter the icmp-type followed by the type number. A list of names, types, and codes is located here: http://www.iana.org/assignments/icmp-parameters
z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z
echo-reply echo-reply (type 0) destination-unreachable unreachable (type 3) network-unreachable net-unreachable (type 3, code 0) host-unreachable host-unreachable (type 3, code 1) protocol-unreachable protocol-unreachable (type 3, code 2) port-unreachable port-unreachable (type 3, code 3) fragmentation-needed packet-too-big (type 3, code 4) source-route-failed source-route-failed (type 3, code 5) network-unknown network-unknown (type 3, code 6) host-unknown host-unknown (type 3, code 7) network-prohibited dod-net-prohibited (type 3, code 9) host-prohibited dod-host-prohibited (type 3, code 10) TOS-network-unreachable net-tos-unreachable (type 3, code 11) TOS-host-unreachable host-tos-unreachable (type 3, code 12) communication-prohibited administratively-prohibited (type 3, code 13) host-precedence-violation host-precedence-unreachable (type 3, code 14) precedence-cutoff precedence-unreachable (type 3, code 15) source-quench source-quench (type 4) redirect redirect (type 5) network-redirect net-redirect (type 5, code 0) host-redirect host-redirect (type 5, code 1) TOS-network-redirect net-tos-redirect (type 5, code 2) TOS-host-redirect host-tos-redirect (type 5, code 3) echo-request echo (type 8) router-advertisement router-advertisement (type 9) router-solicitation router-solicitation (type 10) time-exceeded time-exceeded (type 11) ttl-zero-during-transit ttl-exceeded (type 11, code 0) ttl-zero-during-reassembly reassembly-timeout (type 11, code 1) parameter-problem parameter-problem (type 12) ip-header-bad general-parameter-problem (type 12, code 0) required-option-missing option-missing (type 12, code 1) timestamp-request timestamp-request (type 13) timestamp-reply timestamp-replace (type 14) address-mask-request mask-request (type 17) address-mask-reply mask-reply (type 18)
93
icmp-type <type-number>
Configures the ICMP message matching criteria for the ACL to include only packets with the specified message type. The X-Series Platform applies the ACLs action (deny or permit) to a packet only if its message type number matches the specified message type number. Valid message type numbers are from 0 to 255.
log
Enables logging for this access list.
Define an Access List to a Management Interface

To apply an access list to a management interface, use the following command: configure management <interface> <slot/port> access-list <list-number> [input|output] Where: interface slot/port list-number input/output Type of interface, gigabitethernet or high-availability Actual CPM slot and port location of the interface Access list to use on this interface Specifies that the access list affects incoming or outgoing traffic on this interface
Display Management Access Lists

To display the current configured access lists, use the following command: CBS# show access-list Access List Access Protocol Source IP and Wildcard Destination IP and Wildcard Log Enable (true/false) : : : : : : 1002 permit ip 0.0.0.0 255.255.255.255 172.16.0.0 0.0.255.255 f
Final Steps
If you are going to be managing the X-Series Platform from an external subnet, you must configure a management port default gateway. While this is generally understood for external management, it is included in the Install Interview questions, and worth noting here. Command: CBS# configure management default-gateway 172.16.19.63 CBS(conf-mgmt)# end CBS#
94
8
Configuring Flow Provisioning
This chapter describes using and creating IP and non-IP flow rules and QoS rate-limiter rules. It contains the following major topics:
z z z z
Flow Rule Overview on page 95 Configuring IP Flow Rules on page 101 Configuring System IP Flow Rules on page 103 Configuring System Non-IP Flow Rules on page 106
Flow Rule Overview

A flow rule determines how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, configured by the user. There are four types of flow rules:
z z z z
IP Flow Rule - processes IP traffic at the VAP level. Non-IP Flow Rule - processes non-IP traffic at the VAP level. System IP Flow Rule - affects all IP traffic at the system level, which affects all VAPs. System Non-IP Flow Rule - affects all non-IP traffic at the system level.
All IP flow rules are part of the IP flow classifier system module. The IP flow classifier is designed to be an IP delivery, policy enforcement, and provisioning module. Its default behavior is to direct IP traffic destined to the X-Series Platform to the correct slot (or VAP) for all the configured IP addresses.
VAP IP Flow Rules

The NPM uses the IP flow rules configured for a VAP group to determine how to process IP traffic destined for the members of that VAP group. Each flow rule is associated with a specific VAP group, and multiple flow rules may be created for each VAP group. These flow rules can identify traffic based on the five fields in the IP packet:
z z z z z
IP source address IP destination address Protocol, which indicates the destination upper-layer protocol layer for this datagram. For example, ICMP is 1, IGMP is 2, TCP is 6, and UDP is 17. The default is all protocols (1-255). Source port Destination port
In addition, traffic can be identified by the Domain ID and Incoming Circuit Group (ICG) values that define the physical location of the flow in the X-Series Platform. Each of these values can be a single value or a range of values. For example, a flow rule can be applied to a range of IP source addresses such as 192.100.100.000 to 192.100.100.255. This would be noted in CIDR format as 192.100.100.000/24.
95
The Incoming Circuit Group (ICG) is used by the NPM as an additional method to classify traffic for one or more circuits. When an ICG number is assigned or changed in the circuit configuration, it will impact flow classification on the NPM. If you change an ICG circuit assignment, verify that the appropriate flow rules have been configured or updated. The flow of traffic through a VAP or VAP group can be configured using the following actions and other criteria:
z z z z z z z z z z z z z z
load-balance drop allow pass-to-master pass-to-vap broadcast direction skip-port-protocol generate-reversed-flow source-addr destination-addr source-port destination-port core-assignment
Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules. Each flow rule is assigned a priority, as described in Priorities for IP Flow Rules on page 100. A flow can match different flow rules. However, only the flow rule with the highest priority is used. It is possible to have many rules at the same priority as long as they do not conflict. Flow rules are considered to be in conflict if they have the same priority, are assigned to the same VAP group, and a packet could match both flow rules. For example, these two flow rules are in conflict if they are applied to the same VAP group:
z z
Rule 1: Dest-IP 20.0.0.1, Priority 10 (default) Rule 2: Dest-Port 80, Priority 10
Since it is possible to receive a packet destined to 20.0.0.1 on port 80, the rules are in conflict. Simply changing the priority of either rule removes the conflict. To check for flow rule conflicts, enable the check-flow-rule command. When enabled, each time a flow rule is activated, XOS checks for policy conflicts between that flow rule and the flow rules that are currently activated on the X-Series Platform. If XOS detects a policy conflict, the activate operation fails and the CLI issues an error message. NOTE: If flow rule checking has been disabled and is then enabled, all the activated flow rules are checked for conflicts. The following parameters are secondary actions that can be configured for flow rules:
z
The skip-port-protocol parameter is often used for load balancing purposes. When enabled (default), the NPM ignores the destination port, source port, and protocol in the IP packet when deciding how to load balance a new flow. Therefore, any flows with the same source and destination IP addresses will be assigned to the same VAP. The timeout for a VAP-level IP flow rule determines the idle timer for a flow. If traffic for any flow is idle for this period of time, the NPM deletes that flow from the Active Flow Table. If another packet belonging to that flow is sent after this deletion, the NPM considers this an unknown flow and processes the packet to determine which flow rules apply and where to send the flow. This could result in moving the flow to another VAP, which could be a problem when using a stateful firewall.
96
Timeout values can be set from 30 seconds to 1 hour, or auto (default). The auto timeout value is set by the system. By default, the system defines a timeout value of 10 minutes for all TCP packets, 1 minute for UDP packets, and 30 seconds for all other protocols. VAP IP flow rules are configured with the configure vap-group <group-name> ip-flow-rule <name> command. After creating a flow rule, you must use the activate command to enable it; otherwise the flow rule does not take affect. When you activate a flow rule, it is not applied to existing flows, it only affects new flows.
VAP Non-IP Flow Rules

Non-IP flow rules are configured at the VAP group level. The non-IP flow rules are configured with the configure vap-group <group-name> non-ip-flow-rule command and apply to all non-IP traffic coming into the VAP group. These flow rules identify non-IP traffic by encapsulation type. The encapsulation can be Ethernet, LSAP, or SNAP, along with a protocol number. The Ethernet and SNAP protocols can be 1519 to 65535, and LSAP with DSAP or SSAP can be 0-255. The any keyword can be used with each protocol to allow any protocol number (wildcard). For LSAP, the any keyword denotes all DSAP and SSAP combinations. The non-IP flow rules can be configured using the following criteria:
z z z z z z z z
action drop action broadcast action pass-to-master encapsulation ethernet encapsulation lsap encapsulation snap activate core-assignment
The VAP non-IP flow rule settings can be seen with the show non-ip-flow command. Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules.
IPv6 Non-IP Flow Rule

When you enable IPv6 on the VAP group, a non-ip flow rule is automatically created. The non-ip flow rule ipv6_rule is configured as follows: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate IPv6 traffic is classified as non-ip traffic and passed to the master VAP. Because this traffic is passed to the master VAP, traffic is not load balancing across multiple VAPs. If you anticipate a high volume of IPv6 traffic, use the core-assignment parameters under the VAP group context to load-balance IPv6 traffic across multiple cores on the master VAP. For VAP group and core-assignment configuration steps, refer to Configuring a VAP Group on page 52.
Default VAP Flow Rules

The IP flow classifier automatically generates an IP flow rule when you assign an IP address to a VAP group, using the configure circuit command. Depending on the IP address, one of the following rules is created:
97
z z z
The IP Flow Rule action will be set to broadcast when the IP address subnet is set to a broadcast address. Secondary actions are set to default and the priority is 21. For other IP addresses, the action is to load balance, using the IP address as the destination address. Secondary actions are set to default and the priority is 21. When using the IP address with increment-per-vap, the original address is sent to VAP index 1. The next IP address in sequential order is sent to VAP index 2, and so on. The flow is not load balanced. Secondary actions are set to default, timeout is set to auto, and the priority is 21. When using the IP address with increment-per-vap, the IP flow rule action is set to broadcast for packets with a broadcast address. Otherwise, the default flow-rule is used. Priority is 21.
These rules cannot be modified directly; however, you can modify the default priority using the ip-flow-rule-priority parameter when assigning a circuit to a VAP group. For example, you can change the default priority of 21 using the following command: configure circuit <name> vap-group <name> ip-flow-rule-priority <value> Use the show default-ip-flow-rule command to view the default IP flow rule settings.
System IP Flow Rules

System-wide IP flow rules are configured with the configure system-ip-flow-rule command and apply to all traffic coming into the system. The system IP flow rules can be configured using the following actions and other criteria:
drop allow pass-to-masters broadcast direction skip-port-protocol generate-reversed-flow source-addr destination-addr source-port destination-port
Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules. System IP flow rules have the same parameters as the VAP IP flow rules, which are discussed in the previous section. When a flow matches a system flow rule and a VAP-level flow rule, the flow rule with the higher priority takes precedence. If the priority is the same, the system IP flow rule applies. The exceptions, such as the timeout value, are explained in Interaction Between System and VAP IP Flow Rules on page 99. System IP flow rules must be configured when local IP host addresses (or pools of addresses) exist on the X-Series Platform by virtue of a third party proxy or VPN application that are not otherwise visible to the X-Series Platform configuration.
98
The default system IP flow rule settings can be seen with the show default-ip-flow-rule command. You can also change the default timeout values for specific IP addresses, destination ports, and so on. Refer to the XOS Command Reference Guide for details on XOS command usage, syntax, options, and output. CBS# show system-ip-flow-rule System IP Flow Rule Destination Address Destination Address High Destination Port Destination Port High Source Address Source Address High Source Port Source Port High Incoming Circuit Group Protocol Protocol High Domain Domain High Action Activate (true/false) Priority Skip Protocol (true/false) Skip Port (true/false) Skip Port Protocol (true/false) Timeout : : : : : : : : : : : : : : : : : : : : : myrule 0.0.0.0 255.255.255.255 0 65535 0.0.0.0 255.255.255.255 0 65535 1 1 255 1 4095 drop f 10 t t t auto
If the default timeout value for a specific rule is changed, you must verify that the timeout is set properly for all return flows. In the following example, the system default has been changed to 1 hour for all TCP packets. Because HTTP (TCP port 80) typically has short timeouts, create a higher priority rule with a destination port of 80 for the TCP packets, and assign a timeout value of 3 minutes. You must also create a flow rule using source port 80 with a timeout of 3 minutes. Without this flow rule, the return flow, which is not bound for destination port 80, resets the timeout of the active flow entry from 3 minutes to 1 hour. This occurs because the original flow and the return flow share the same active flow entry even though they use two different rules.
Interaction Between System and VAP IP Flow Rules

IP flow rules are matched only if all of the corresponding IP fields and incoming circuits are within the range of the IP flow rule. If more than one IP flow rule is matched, the flow rule with the highest priority rule (with primary action other than none) is selected and only independent secondary actions matching the flow rule are incorporated or merged into this rule. The timeout value is normally set by the system IP flow rule. However, if a flow matches both a VAP-level flow rule and system IP flow rule that ONLY defines the timeout value, the timeout value of the VAP-level flow rule applies regardless of priority. For example, you create and activate a VAP group flow rule that has a timeout interval of 30 seconds for TCP packets and a priority of 15. You also have a system IP flow rule with a timeout of 20 minutes for TCP packets and a priority of 25. The system IP flow rule does not define any parameter other than the timeout value. The result is that all TCP packets for that flow timeout in 30 seconds. If a flow only matches a VAP-level IP flow rule and system flow rule that both have a timeout value of auto, the IP flow classifier decides the best timeout from a scaling point of view, currently set at 1 minute.
Multi-VAP Group IP Flow Rule Match

An IP flow rule is assigned to a specific VAP. If an IP flow rule is not specifically assigned, it is a system rule. In multi-VAP group environments, the flow rule classifier builds basic flow rule matches per VAP group, then merges these matches from each VAP group and system VAP group.
99
Merging matched IP flow rules from different VAP groups results in different actions, selected based on the circuit of the classified packet. Classification always occurs on a particular interconnect of the X-Series Platform and is a function of the VAP group reachability from this interconnect. Table 16 summarizes the merger/match process.
Table 16.
Summary of Merger/Match Process

VAP Group Directly Reachable through Ingress Circuit VAP Group NOT Directly Reachable Only the primary action from this match is considered but not applied on the current circuit. System Rules Result of the basic match is considered.
Circuit of the VAP group is in promiscuous mode. Circuit of the VAP group is in non-promiscuous mode; destination MAC-address is local or broadcast.
Result of the basic match from this VAP group is considered. Result of the basic match from this VAP group is considered only if the primary action is of the highest priority in comparison with other matches; otherwise, the result is the same as for a NOT directly reachable VAP group. None of the rules are considered.
Circuit of the VAP group is in non-promiscuous mode; destination mac-address is external.
When matches from multiple groups are merged, secondary actions are merged based on the secondary action merge flow rule. Each primary action from selected matches is considered in the context of the corresponding VAP group.
System Non-IP Flow Rules

The system non-IP flow rules are configured with the configure system-non-ip-flow-rule command and apply to all non-IP traffic coming into the system. These flow rules identify non-IP traffic by encapsulation type. The encapsulation can be Ethernet, LSAP, or SNAP, along with a protocol number. The Ethernet and SNAP protocols can be 1519 to 65535, and LSAP with DSAP or SSAP can be 0-255. The any keyword can be used with each protocol to allow any protocol number (wildcard). For LSAP, the any keyword denotes all DSAP and SSAP combinations. The system non-IP flow rules can be configured to take the following actions: broadcast drop pass-to-masters Send the traffic flow to all VAPs. Should be used when non-IP traffic is meant to terminate on the XOS system. Drop packets. This is the default action. Pass the traffic to the master VAPs of all VAP groups. Should be used when non-IP traffic is meant to pass through the XOS system.
The system non-IP flow rule settings can be seen with the show system-non-ip-flow-rule command.
Priorities for IP Flow Rules

Each IP flow rule has an associated priority level. This priority is used to differentiate rules with over-lapping criteria. Valid priority level values are from 0 (lowest priority) to 31 (highest priority) with some being allocated to users and others to the IP flow classifier. Only priorities in the range of 10 through 20 and 25 through 30 are configurable. The rest are for IP flow classifier use only.
100
NOTE: The ip-flow-rule-priority parameter is used when assigning a circuit to a VAP group. This is the only time you can configure a priority for a flow rule. The priority ranges are 10-20 and 25-30. The lower range of priority (10 through 20) is designed for all normal operations of the X-Series Platform. You can define flow rules to load-balance traffic across particular VAPs of the VAP group or select VAPs based on other criteria, like load. The higher range (25 to 30) is designed to over-write the system default configuration; to redefine normal IP forwarding rules; or to restrict a particular type of the traffic. Flow rules with priorities in this range are considered critical rules. NOTE: If there is an overlap of a VAP group level flow rule and a system non-IP flow rule where both handle protocol or encapsulated traffic, the VAP group flow rule always has the higher priority. In addition, a flow rule that contains a specific protocol or encapsulation type has a higher priority than a flow rule that uses the any parameter to specify the protocol or encapsulation type.
Configuring IP Flow Rules

To configure VAP group-specific IP flow rules using the EMS, open Configuration > Flow Provisioning > IP Flow Rules. The following example configures IP flow rules using the CLI. In this example, there are three flow rules, all assigned to VAP group firewall. Flow rule fw_lb load balances all IP traffic. It will match any protocol, and source and destination addresses and ports. It is configured for a 10 minute timeout interval, where if traffic for this flow is idle for 10 minutes then the NPM will delete that flow from the Active Flow Table. Flow rule bad_host drops all traffic from source address 10.1.2.3, and flow rule no_icmp drops all traffic destined for ICMP (protocol 1). CBS# configure vap-group firewall ip-flow-rule fw_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# timeout 10-minutes CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# end CBS# configure vap-group firewall ip-flow-rule bad_host CBS(ip-flow-rule)# action drop CBS(ip-flow-rule)# priority 29 CBS(ip-flow-rule)# source-addr 10.1.2.3 CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# end CBS# configure vap-group firewall ip-flow-rule no_icmp CBS(ip-flow-rule)# action drop CBS(ip-flow-rule)# priority 30 CBS(ip-flow-rule)# protocol 1 CBS(ip-flow-rule)# activate All of these rules can match ICMP; the first 2 rules default to protocols 1-255 while the last rule matches protocol 1 explicitly. The rules fw_lb and bad_host both match the source address of 10.1.2.3. If these rules had the same priority, they would conflict. Therefore, the priority of the fw_lb rule is at the default of 10, bad_host is at 29, and no_icmp is at 30. Refer to the XOS Command Reference Guide for the full syntax of the ip-flow-rule command. The following IP flow rule example shows how to ensure that all FTP traffic (which uses ports 20 and 21) is sent to VAP #1 in the group. Since a packet can possibly have both a source AND destination of port 20, the rules must be set to different priorities. The same is true for port 21. However, since a packet cannot have a destination port of both 20 and 21, these rules can be the same priority.
101
vap-group vap1 xslinux_v3 max-load-count 1 ap-list ap3 ap4 ap5 ap6 ap7 ap8 ap9 ap10 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 ip-forwarding ip-flow-rule vap1_lb action load-balance activate ip-flow-rule ftp_data_src action pass-to-vap 1 priority 11 source-port 20 20 activate ip-flow-rule ftp_data_dst action pass-to-vap 1 priority 12 destination-port 20 20 activate ip-flow-rule ftp_ctrl_src action pass-to-vap 1 priority 11 source-port 21 21 activate ip-flow-rule ftp_ctrl_dst action pass-to-vap 1 priority 12 destination-port 21 21 activate Use the show ip-flow-rule <flow-rule-name> command to display an IP flow rule and its parameters. If no flow rule is specified, all system IP flow rules are displayed. CBS# show ip-flow-rule IP Flow Rule VAP Group Destination Address Destination Address High Destination Port Destination Port High Source Address Source Address High Source Port Source Port High Incoming Circuit Group Protocol Protocol High Domain Domain High Action Activate (true/false) Priority Skip Protocol (true/false) Skip Port (true/false) Skip Port Protocol (true/false) Timeout Trace (true/false) Generate Reversed Flow (true/false) Direction Bypass-tcp-flow-setup-validation (true/false) Core Assignment : : : : : : : : : : : : : : : : : : : : : : : : : : : flow1 fw 0.0.0.0 255.255.255.255 0 65535 0.0.0.0 255.255.255.255 0 65535 1 1 255 1 4095 load-balance t 10 t t t auto f f both f random-single-core
102
Configuring Non-IP Flow Rules

To configure VAP group-specific non-IP flow rules using the EMS, open Configuration > Flow Provisioning >Non IP Flow Rules. The following example configures a non-IP flow rule using the CLI. In this example, the drop_traffic flow rule drops snap traffic. CBS# configure vap-group red non-ip-flow-rule drop_traffic CBS(ip-flow-rule)# encapsulation ethernet type 2000 CBS(ip-flow-rule)# action drop CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# end
Configuring System IP Flow Rules

To configure system level IP flow rules using the EMS, open Configuration > Flow Provisioning > System IP Flow Rules. The following example configures IP flow rules using the CLI. In this example, the tcp_traffic flow rule sets the timeout period to 20 minutes for all TCP (protocol 6) traffic. CBS# configure system-ip-flow-rule tcp_traffic CBS(conf-system-ip-flow)# action drop CBS(conf-system-ip-flow)# timeout 20-minutes CBS(conf-system-ip-flow)# protocol 6 CBS(conf-system-ip-flow)# activate Use show system-ip-flow-rule <flow-rule-name> to display a system IP flow rule and its parameters. If no flow rule is specified, all system IP flow rules are displayed. CBS# show system-ip-flow-rule System IP Flow Rule Destination Address Destination Address High Destination Port Destination Port High Source Address Source Address High Source Port Source Port High Incoming Circuit Group Protocol Protocol High Domain Domain High Action Activate (true/false) Priority Skip Protocol (true/false) Skip Port (true/false) Skip Port Protocol (true/false) Timeout Trace (true/false) Generate Reversed Flow (true/false) Direction Core Assignment (1 row) : : : : : : : : : : : : : : : : : : : : : : : : : testrule 0.0.0.0 255.255.255.255 0 65535 0.0.0.0 255.255.255.255 0 65535 1 1 255 1 4095 drop f 10 t t t auto f f both random-single-core
103
Troubleshooting IP Flows
You can trace VAP group and system IP flow rules using the trace option. When the trace option is enabled on an ip-flow-rule (configure vap-group <group-name> ip-flow-rule <name> -t), the system marks all new flows established using the specified flow rule with a trace flag. The trace flag is cleared only when the flow is cleared from the flow table by either timing out or by issuing the clear flow-active command.
Flow Processing Log Information

The flow processing daemon logs debug messages related to flow setup, timeout, and re-routing activity on flows marked with the trace flag. IMPORTANT: When troubleshooting flows, it is important that you create specific IP flow rules with the trace option enabled for running in a production environment. Avoid enabling the trace option on a flow unnecessarily. The logging activity will cause a severe reduction in performance. The flow processing daemon on the NPM logs trace messages to the syslog using daemon.notice facility/ severity level. You can control the logging level per NPM using the following command (issued from the NPM): / # cbs_query_flowd log fp notice [on|off] The default setting for notice log level under fp (flow processing) is on. Setting it is not persistent (does not survive an NPM reboot). Changing the NPM logging level does not change the trace functionality, but filters any outgoing log messages. System performance will not suffer as significantly with notice set to off while you are tracing flows. On the CPM, these messages go to /var/log/messages by default but this can be controlled by editing the /etc/syslog.conf file and restarting the syslog daemon. The APM also logs messages during the setup and re-routing of flows that are marked with the trace flag. The APM will log a message every time a flow (re)setup is needed for a packet marked with the trace flag. The trace logs are stored in /var/log/messages as kernel-level entries from a specific VAP. If a flow is originated from an APM, the flow initially cannot be traced. However, if the flow experiences a policy lookup on the NPM, the NPM may identify that the flow should be traced. If so, the APM will store this fact, and will then log a message every time a flow (re)setup occurs for the APM originated flow.
Clear-Flow Active Command

The clear-flow active command deletes all active flow connection and load-balancing information from all Network Processor Modules (NPMs) installed in the X-Series Platform, or from a specified NPM. This command stops all traffic, and in most cases, causes a complete service interruption that may last for several minutes. Consider these risks carefully before entering this command. Crossbeam Systems recommends that you use this command only after consulting with a Crossbeam Systems Customer Support or Professional Services representative.
Handling Flows When an Application is Down

When the Application Monitor identifies an application as down on an APM, it will stop flows to that VAP. Flows are automatically reassigned to active applications, and new flows are balanced across the active VAPs. When the application returns to an Active state, it is eligible to receive new flows distributed by the NPM.
104
Dropping Flows
Flows can be dropped for several reasons, as listed below. Use the show flow active command to display information currently stored in the Active Flow Table (AFT) on the Network Processor Modules (NPMs) running on the X-Series Platform. The verbose parameter display additional information, including Drop reasons about all active flows. For a list of filter parameters to use with the show flow active command, see Chapter 6 of the XOS Command Reference Guide. Possible values for <drop_reason_ID> are:
No L2 policy match The destination MAC in the packet did not match the MAC address of any VND on the circuit where the packet entered the system. NOTE: Circuit information is not displayed for flows with this drop reason ID. No L3 policy match There are no IP flow rules that apply to this layer 3 flow. NOTE: Circuit information is not displayed for flows with this drop reason ID. L3 drop policy This layer 3 flow matches the conditions defined in the access control list (ACL) for an IP flow rule configured with the action, drop. PS2Master failed A VAP group IP flow rule configured with the action, pass-to-master, or a system-level IP flow rule with the action, pass-to-masters, applies to this flow. The NPM attempted to send this flow to one or more master VAPs, but the operation failed because none of the master VAPs were in the Active state. PS2IDX failed A VAP group IP flow rule configured with the action, pass-to-vap, applies to this flow. The NPM attempted to send this flow to the appropriate VAP, but the operation failed because the VAP was not in the Active state. Load-balance failed A VAP group IP flow rule configured with the action, load-balance, applies to this flow. The NPM attempted to load balance this flow across the VAPs in the appropriate VAP group, but the operation failed because there were no active VAPs in the group or because there were no VAPs in the VAP groups load-balance VAP list. Broadcast failed A flow rule configured with the action, broadcast, applies to this flow. The NPM attempted to broadcast this flow to all VAPs in one VAP group or to all VAPs in all VAP groups, but the operation failed because none of the VAPs to which the NPM sent the flow were in the Active state. No Reason One or more flow rules were successfully applied to this flow, and none of those IP flow rules are configured with the action, drop.
105
Configuring System Non-IP Flow Rules

To configure system non-IP flow rules using the EMS, open Configuration > Flow Provisioning > System Non-IP Flow Rules. Configuring system non-IP flow rules is similar to configuring system IP flow rules with the exception of the packet encapsulation used and the lack of IP addresses. The following example configures system non-IP flow rules using the CLI. In this example, the snap_traffic flow rule drops all SNAP packets with a protocol number of 2000 and an Organization Unique Identifier (OUI) of 10. CBS# configure system-non-ip-flow-rule snap_traffic CBS(conf-system-non-ip-flow)# encapsulation snap type 2000 oui 10 CBS(conf-system-non-ip-flow)# action drop CBS(conf-system-non-ip-flow)# activate Use the show command to view the system non-IP flow rule snap_traffic: CBS(conf-system-non-ip-flow)# show System Non IP Flow Rule : snap_traffic Encapsulation : snap Type : 2000 OUI : 10 Action : drop Activate (true/false) : t (1 row)
106
9
Configuring Protocols
This chapter contains the following major topics:
z z z z z z z z z z
Configuring Static IP Routes on page 107 Configuring ARP Entries on page 109 Displaying ARP Entries on page 109 Configuring Neighbor Discovery on page 109 Displaying Neighbor Discovery on page 110 Configuring a RADIUS Server Host on page 110 Configuring an LDAP Server on page 111 Defining the LDAP Version Number on page 111 Configuring RSA SecurID to Authenticate Administrative Users on page 112 Configuring RSA SecurID to Authenticate Remote Users on page 113
Configuring Static IP Routes

Static IP routes are user-defined routes that cause packets moving between a source and a destination to take a specific path. Static IP Routes are defined per routing domain and apply to all VAP groups which have circuits in this domain, unless the VAP group is explicitly specified. Static routes are automatically load-balanced using an Equal-Cost Multi-Path (ECMP) algorithm. NOTE: Static IP routes are configured in the same manner for IPv4 and IPv6 protocols. To configure static IP routes, you need to specify the routes Destination IP Network Address and Network Mask. The Destination Network IP Address is the IP address of the network or host to which the packet is being sent. You also need to specify the Next Hop IP address. This is the IP address of the next hop that can be used to reach that network. When specifying an IP route you can also specify to verify the health of the next hop. When specified, probes (ARP requests) are sent to the next hop IP address. If the probes fail, route cost is increased so that an alternative route with a lower cost may be chosen. Use the following command to configure static IP routes: configure ip route {<ip-addr> <netmask>|<ip-addr/netmask>} <next-hop-ip-addr> [domain <domain-id>] [circuit <circuit-name>] [vap-group <groupname>] [metric <metric-value>] [verify-next-hop] Where: <ip-addr> <netmask> <ip-addr/netmask> IP address of the route. Network Mask associated with this IP address. Destination IP address and network mask. Can be either CIDR or decimal format. IPv6 addresses use the CIDR format: 2001:BA95:AC10:0:CF6A:D:2145:3713/64
107
<next-hop-ip-addr> <domain-id> <circuit-name> <groupname> <metric-value>
IP address of the next hop that is used to reach that network. User defined domain, ranging from 1 to 4095, with a default value of 1. Name of the circuit. VAP group for this address. Default value is all VAP groups. Metric value that you want to assign to the static IP route that you are configuring. Valid values are: IPv4: 1 to 255, inclusive, with a default of 0 (zero) IPv6: 257 to 511, inclusive with a default value of 256
verify-next-hop
Verifies a next hop route for a default network before using it.
The following example configures a static route, its netmask, and the address of the next hop: CBS(config)# ip CBS(config-ip)# route 190.1.1.0 255.255.255.0 192.213.212.111 The following example configures the same static route, but also verifies the health of the next hop: CBS(config)# ip CBS(config-ip)# route 190.1.1.0 255.255.255.0 192.213.212.111 vap-group firewall verify-next-hop The following example configures an IPv6 static route and verifies the health of the next hop: CBS(config)# ip CBS(config-ip)# route 2002:BA95:AC10:0:CF6A:D:2145:3000/64 FC00:C959:CC10:0:CF6A:D:2145:0700 vap-group firewall circuit thru1 verify-next-hop To display configured static routes, use the following command: show ip route [<destination-ip> | <first-destination-ip> <last-destination-ip>] [sort-by-destination-address] The following is an example display of the show-ip-route command. Module group1_1 group1_1 group1_1 group1_1 group1_1 group1_1 Destination 192.168.20.30/32 30.1.0.0/16 20.0.0.0/8 21.0.0.0/8 22.0.0.0/8 23.0.0.0/8 Gateway 30.1.0.20 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 Metric 0 0 0 0 0 0 Device eth0 eth0 vnd1 vnd2 vnd3 vnd4
If you have IPv6 configured, show ip route will display the IPv6 routing table. NOTE: XOS will restrict IP routes whose destination network is the same or a more specific network match with the system internal network IP address. For example, the route a.b.c.0/24 is a more specific match than route a.b.0.0/16. XOS displays the following error when a conflict is found: CBS# config management ip-route 1.1.1.234 255.255.255.255 192.168.32.2 %CONF-ERR: Invalid value Detail: Conflict found with internal network 1.1.0.0 255.255.0.0 CBS#
108
Configuring ARP Entries

Define static Address Resolution Protocol (ARP) entries by relating an IP address to a MAC address. To define ARP entries, use the following command: configure arp <ipaddr> <mac-addr> [domain <domain-id>] [vap-group <groupname>] [circuit <circuit-name>] Where: <ipaddr> <mac-addr> <domain-id> <groupname> <circuit_name> IP address of the ARP entry. MAC address of the ARP entry. Domain for this host. The range is 1to 4095, with a default of 1. VAP group to which this IP address is assigned. Default is all VAP groups. The CLI displays the existing VAP group names. Specifies a circuit. The CLI displays existing circuit names when the user types ?
The following is an example of this command: CBS(config)# arp 192.178.25.10 00:05:c2:13:44:71
Displaying ARP Entries

To display ARP entries, use the following command: CBS# show arp Module primarycpm 15Net_1 npm1 npm2 1.1.11.35 192.178.25.10 192.178.25.15 192.178.25.47 Address 1.1.11.20 1.1.11.101 1.1.11.1 1.1.11.2 1.1.11.35 192.178.25.10 192.178.25.15 192.178.25.47 Hardware Addr 00:05:c2:00:0b:05 00:05:c2:00:0b:04 00:05:c2:00:0b:11 00:05:c2:00:0b:12 00:05:c2:00:0b:15 00:05:c2:13:44:71 00:05:c2:15:0c:a8 00:d1:c2:b5:71:e5 Type dynamic dynamic dynamic dynamic dynamic static dynamic dynamic Interface eth0 eth0 eth0 eth0 eth0 firewall eth2 eth2
Configuring Neighbor Discovery

Neighbor discovery specifies a static association between an IPv6 address and an ethernet layer MAC address to ensure proper forwarding of frames. By default, an NPM can apply a neighbor-discovery static entry to any packet passing between the members of a VAP group and a physical interface on an NPM. Use the following command to configure neighbor discovery: configure neighbor-discovery <IP_address> <MAC_address> [domain <domain_ID>] [vap-group <VAP_group_name>] [circuit <circuit_name>] configure no neighbor-discovery <IP_address> <MAC_address> For additional details on how to configure the parameters of this command, please refer to Chapter 7 of the Command Reference Guide. The following command configures a new static entry that maps the IP address fd00::330:f3cb:fb31:56ab to the MAC address 00:03:d2:00:02:0d. This command also configures the new entry to apply only to packets destined for VNDs that XOS creates for the circuit called testcct on the VAP group called testvapgroup:
CBS# configure neighbor-discovery fd00::330:f3cb:fb31:56ab 00:03:d2:00:02:0d vap-group testvapgroup circuit testcct CBS# NOTE: You cannot specify a MAC address that contains only 0s or only fs.
Displaying Neighbor Discovery

This command displays the entries in the neighbor discovery table, along with status information for each discovered node. Use the following command to display the neighbor discovery table: show neighbor-discovery For additional details about the output of this command, please refer to Chapter 12 of the Command Reference Guide. The following is a sample output of this command. CBS# show neighbor-discovery Neighbor Entry Module one_2 Domain 0 IP address 2002:1::3 State REACHABLE Type UNICAST Flags HW Address 00:03:d2:20:0c:69 Device enter Neighbor Entry Module one_2 Domain 0 IP address 2002:2::1 State REACHABLE Type UNICAST Flags HW Address 00:03:d2:1f:fb:b9 Device thru Neighbor Entry Module one_2 Domain 0 IP address fe80::203:d2ff:fe1f:fbb9 State STALE Type UNICAST Flags HW Address 00:03:d2:1f:fb:b9 Device thru
Configuring a RADIUS Server Host

The X-Series Platform supports RADIUS Authentication by selecting a host to use as the RADIUS server and the desired authentication port. Define the RADIUS authentication host and its attributes as follows: configure radius-server host {<hostname>|<ip-address>} [auth-port <auth-port-value>] [timeout <timeout-value>] [key <key-word>]
110
Where: <hostname> <ip-address> DNS name of the RADIUS server host. IP address of the RADIUS server host. The IP address of the RADIUS server host can be on the same network as a management interface or circuit. However, the IP address:
z z
Cannot be on the XOS system's internal network, as defined by the configure system-internal-network command Cannot be an address that is already defined in the XOS configuration, such as a circuits address or a management address.
<auth-port-value> UDP destination port for authentication requests. Valid values are 0 to 65535. Default is 1812. <timeout-value> <key-word> Number of seconds the X-Series Platform waits for a RADIUS server host to reply before timing out. Valid values are 0 to 3600 seconds. Default is 3 seconds. Authentication and encryption key used for all RADIUS communications between the host and the RADIUS daemon. This key must match the encryption used on the RADIUS daemon.
Configuring an LDAP Server

The X-Series Platform supports the Lightweight Directory Access Protocol (LDAP), which is an Internet protocol that email programs use to look up contact information from a server. LDAP servers index all the data in their entries, and filters may be used to select just the person or group you want, and return just the information you want. For example, search for all people located in Chicago whose name contains Fred that have an email address and return their full name, email, title, and description. To configure an LDAP Server, use the following command: configure ldap-server {<hostname>|<ip-address>} [auth-port <auth-port-value>] Where: <hostname>| <ip-address> <auth-port-value> Enter the LDAP server DNS name or IP address. Specifies the UDP destination port for authentication requests. Valid values are 0 to 65535, with a default value of 389.
NOTE: Configuration for the LDAP Server are written to the /etc/ldap.conf.This file will be overwritten when the CPM is rebooted. To preserve these files, execute the CLI command copy running-config.
Defining the LDAP Version Number

To define an LDAP Host Server's Version number, use the following command: configure ldap-parameter version {2|3} [distinguished-name <name>] Where: version {2|3} <name> The LDAP servers version number, 2 or 3. Default is 3. List of distinguished names to search.
111
Configuring RSA SecurID to Authenticate Administrative Users

RSA SecurID is an authentication solution built on a method called two-factor authentication using the RSA ACE/Server authentication engine. The normal password authentication methods provides a low level of authenticity. The SecurID solution adds a second level of authentication in addition to a predefined PIN. A user is required to provide an additional passcode which is dynamically generated during the authentication process making it hard to break into a SecurID system. The SecurID system issues individually registered devices to authorized employees, that generate single-use passcodes (tokens), which change based on a timecode algorithm. A new tokencode is generated every 60 seconds and the RSA ACE/Server authentication engine validates these changing tokencodes. A user is then required to provide this dynamic tokencode in combination with a predefined PIN assigned to them during the authentication process. The RSA ACE/Server is used to issue authenticators (token generator devices) to individuals, setup security policies and create logs of user accesses to the network resources managed by the system. In addition, RSA Security provides device specific software agents called RSA Ace/Agents, to enable strong authentication in network resources and web servers. The RSA Security system also provides the RSA ACE/Server Application Programming Interface that allows developers to integrate RSA SecurID Authentication into your applications. Prerequisites: Before starting this procedure, obtain the ACEAgent_50_PAM.tar from the SecurID Website. This is not included as part of the XOS distribution. IMPORTANT: After enabling the SecurID pam authentication, the only method available to recover from a loss of communication between the Ace server and CPM is by logging in as root via the console. To configure RSA SecurID, complete the following: 1. 2. 3. On the CPM, create the following directory: mkdir /var/ace Transfer the file sdconf.rec from the RSA server to the /var/ace directory. Open the sdconf.rec file and edit the following line: edit /etc/profile Add the line: VAR_ACE=/var/ace/sdconf.rec 4. 5. On the CPM, install the ACEAgent_50_PAM.tar, using the following command: install_pam.sh Open the /etc/sshd/sshd_config file and complete the following: a. b. 6. a. b. 7. Set the PAMAuthenticationViaKbdInt parameter to Yes. Set the Restart sshd: line to Restart sshd: "service sshd restart Comment out the following: auth required /lib/security/pam_stack.so service=system-auth Add the following: auth required /lib/security/pam_securid.so reserve Execute the CLI command copy running-config to preserve these files. Otherwise, the files in the /etc/pam.d/ directory will be overwritten when the CPM is rebooted.
Open the /etc/pam.d/sshd file.
112
8.
In some cases, the agent libraries (client side) use the wrong interface IP address in the decryption and authentication fails. To overcome this issue, place a new text file sdopts.rec next to sdconf.rec with the following line included: CLIENT_IP=x.x.x.x Where x.x.x.x is the agents primary IP address, as defined on the server (the IP of the interface to which the server is routed).
Configuring RSA SecurID to Authenticate Remote Users

Integrating an ACE/Server with VPN-1/FireWall-1
To integrate an ACE/Server with the Check Point VPN-1/FireWall-1: 1. 2. Copy the sdconf.rec file from the ACE/Server system to /var/ace on each VAP within the FireWall-1 VAP group. On each VAP in the FW-1 VAP group, edit the /etc/services file, adding the lines: securid 5500/udp securidprop 5510/tcp 3. On the ACE/Server, define the VPN-1/Firewall-1 system using the Add Client option. If you did not define Sites (optional), leave this field blank. If you want to authenticate all users on the ACE Server, in the Client Type select Communication Server and Open to All Locally Known Users. Make sure you allow these services in the VPN-1/Firewall-1 policy and that the clocks are synchronized on both systems (ACE/Server and VPN-1/Firewall-1).
4.
Configuring ACE/Server in a VPN-1/Firewall-1 Clustered Environment

1. 2. Define each cluster member separately on the ACE server with its unique IP address. To prevent the cluster member from hiding (NATing) its unique IP address, add the following line to the "$FWDIR/lib/table.def" file on the mgmt and install policy: "no_hide_services_ports = { .., <5500, 17> };". (Note: this line is already a part of the table.def file as of NG AI R55 based SmartCenter). 3. If the agent libraries (client side) use the wrong interface IP address in the decryption and authentication fails, place a new text file, sdopts.rec, next to sdconf.rec with the following line added: "CLIENT_IP=x.x.x.x" where "x.x.x.x" is the agent's primary IP as defined on the server (the IP of the interface that the server is routed to). NOTE: After a SmartCenter upgrade, any changes to the *.def files are lost.
113
Configuring an ACE Server to Work with a High Availability Cluster

When the ACE Server first communicates with Check Point VPN-1/Firewall-1, it sends a Node Secret. The Node Secret is sent to the published Name and IP address of each VAP in the FW-1 VAP group. To define the X-Series Platform name and IP address: 1. Determine each VAPs name and IP address, using the ping command as follows: On Windows: ping <the host name of the machine> On Solaris: ping -s <the host name of the machine> 2. Add each VAP name and IP address to the hosts file on the VPN-1/Firewall-1 ACE server. The X-Series Platform name will be the first name and all other interfaces follow. You also have to rename it; for example, if the computer name is "Bob," assign the name to the published IP (the IP shown while pinging the system). Then name the rest of the interfaces bob-1, bob-2 and so on. On the ACE Server, add a client with the name of the system on which VPN-1/Firewall -1 is installed. Under the Secondary Node, add the rest of the interfaces. Each time a different interface. Complete the previous step for all VAPs in the FW-1 VAP group.
3. 4. 5.
114
10
Configuring Bridge Mode
You can configure the X-Series Platform for Layer 2 bridging. This chapter describes the components used to configure bridges in XOS.
z z
Configuring Bridge-Mode Bridges on page 115 Additional Configurations on page 117
Configuring Bridge-Mode Bridges

The X-Series Platform supports bridging multiple circuits on different segments of the same LAN as part of a bridge-mode device. To create a bridge-mode bridge, three steps need to be completed. 1. 2. 3. Create required circuits: A bridge circuit and at least two interface circuits in promiscuous-mode active. Assign the interface circuits to physical ports. Group the circuits using the bridge-mode command.
NOTE: There are restrictions to the number of circuits that can be assigned to a bridge. Refer to Circuit Restrictions for Bridges on page 117. If you are using EMS, open Configuration > Interfaces > Data Interfaces > Bridge Mode. To create a bridge-mode bridge using the CLI, complete the following: 1. Configure a circuit to be used as the bridge-mode bridge. CBS# configure circuit <bridge_circuit_name> CBS(conf-cct)# 2. Configure a device name for the bridge. CBS(conf-cct)# device-name <device_name> CBS(conf-cct)# 3. Assign a VAP group to the circuit: CBS(conf-cct)# vap-group <vapgroupname> CBS(conf-cct-vapgroup)# NOTE: For applications requiring an IP address on the bridge, assign the IP address to the template circuit before creating the bridge. If the application does not specifically require an IP address on the bridge, do not assign a primary IP address to the template circuit. Refer to your application documentation for IP address requirements. 4. Configure the IP address if required. CBS(conf-cct-vapgroup)# ip 10.70.10.1/24 10.70.10.255 CBS(conf-cct-vapgroup-ip)# end CBS# 5. Configure the first of two member circuits. CBS# configure circuit <name1> CBS(conf-cct)#
115
6.
Configure device name to the circuit. CBS(conf-cct)# device-name <device_name> CBS(conf-cct)#
7.
Assign the bridged VAP group to the circuit. CBS(conf-cct)# vap-group <vapgroupname> CBS(conf-cct-vapgroup)#
8.
Set the circuit on the VAP group to promiscuous mode active. Do not configure IP information, as these circuits will be grouped into the above circuit by using the bridge-mode feature. CBS(conf-cct-vapgroup)# promiscuous-mode active CBS(conf-cct-vapgroup)#
9.
Repeat steps 4 through 7 for the second member circuit. Proceed with Step 9 after the second circuit has been configured.
10. Assign these circuits to the physical interfaces, using the following command: CBS# configure interface gigabitethernet 1/1 logical-all <logical name> circuit <name1> CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet 1/2 logical-all <logical name> circuit <name2> CBS(intf-gig-log-cct)# end CBS# 11. Group the circuits into bridge-mode, using the following command: CBS# configure bridge-mode <bridge_circuit_name> CBS(conf-bridge-mode)# circuit <name1> CBS(conf-bridge-mode)# circuit <name2> CBS(conf-bridge-mode)# end CBS# The following is an example of a bridge-mode configuration: CBS# configure circuit bridgecct CBS(conf-cct)# device-name bridgecct CBS(conf-cct)# vap-group ids CBS(conf-cct-vapgroup)# ip 10.70.10.1/24 10.70.10.255 CBS(conf-cct-vapgroup-ip)# end CBS# configure circuit circuit1 vap-group ids promiscuous-mode CBS# configure circuit circuit2 vap-group ids promiscuous-mode CBS# configure interface gigabitethernet 1/1 logical logical_1 circuit1 CBS(intf-gig-log-cct)# end CBS# configure interface gigabitethernet 1/2 logical logical_2 circuit2 CBS(intf-gig-log-cct)# end CBS# configure bridge-mode bridgecct CBS(conf-bridge-mode)# circuit circuit1 CBS(conf-bridge-mode)# circuit circuit2 active active circuit
circuit
116
Additional Configurations
The following section provided examples of additional bridge-mode bridge configurations. These include a basic bridge configuration, a VLAN configuration, and a bridge configuration using multi-link group interfaces. A bridge can be connected to another application in a different VAP group. To do this, configure an interface-internal to connect the bridge to the applications VAP group. Also, you can direct traffic to the applications VAP group circuits based on VLAN tags using a combination of ingress-vlan-tag in the logical line and default-egress-vlan-tag in the circuit configuration.
Circuit Restrictions for Bridges

A circuit should not be assigned to more than two bridges. More specifically, a circuit and VAP group pair with promiscuous-mode active should not be included in more than two bridging configurations. (This applies to bridge-mode transparent, as well as bridge-mode bridge.) In a VAP group with multiple VAPs, traffic on promiscuous circuits cannot be sent from one VAP to another VAP. This is true even when the IP address on the bridge-mode circuit is configured as increment-per-vap. Furthermore, a circuit is restricted to one bridge in the following configurations:
z z
A circuit should not be assigned to more than one bridge if it also mapped to a physical interface. A circuit should not be assigned to more than one bridge if the circuit and VAP group pair is configured for no promiscuous-mode.
NOTE: If you disable a bridge that has attached multi-link group-interfaces, the attached interfaces are not disabled automatically. NOTE: If you enable an MLT group-interface that is attached to a bridge that is disabled, the bridge will remain disabled. Before creating a bridge configuration, you need to determine:
z z
Whether the application requires bridge or transparent mode for the bridge-mode configuration Whether VLAN support is required
Basic Configuration
The following configuration example provides a basic bridge configuration using the transparent mode. vap-group is1 xslinux_v3 vap-count 2 max-load-count 2 ap-list ap4 ap5 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 11 12 ip-flow-rule def_rule_is1 action load-balance priority 15 activate circuit bridge10 device-name bridge10 vap-group is1 circuit vpt1_1 device name vpt1_1 vap-group is1 promiscuous-mode active circuit vpt1_2 device-name vpt1_2
117
vap-group is1 promiscuous-mode active interface gigabitethernet 1/1 logical-all vpt1_1 circuit vpt1_1 interface gigabitethernet 1/2 logical-all vpt1_2 circuit vpt1_2 bridge-mode bridge10 transparent circuit vpt1_1 circuit vpt1_2
VLAN Configuration
The following configuration example expands the basic configuration so that traffic is serialized from an IPS application to a firewall. VLANs are also supported. vap-group is2 xslinux_v3 vap-count 5 max-load-count 2 ap-list ap7 ap8 ap9 ap10 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 delay-flow 120 reload-timeout 1200 ip-flow-rule ips_def_rule_is2 action load-balance priority 15 activate vap-group fw xslinux_v3 vap-count 3 max-load-count 2 ap-list ap2 ap3 ap4 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 delay-flow 120 reload-timeout 1200 ip-flow-rule fw_def_rule action load-balance priority 15 activate system-non-ip-flow-rule IS_stp encapsulation lsap dsap 66 ssap 66 action broadcast activate circuit fwext device-name fwext vap-group fw ip-forwarding ip 172.16.11.10/24 172.16.11.255 circuit vlanbridge1 device-name vlanbridge1 vap-group is2 circuit intvlancirc1 device-name intvlancirc1 vap-group fw
118
ip-forwarding vap-group is2 promiscuous-mode active circuit vlan20 device-name vlan20 vap-group fw default-egress-vlan-tag 20 ip-forwarding ip 172.16.20.10/24 172.16.20.255 circuit vpt1_5 device-name vpt1_5 vap-group is2 promiscuous-mode active interface gigabitethernet 1/2 logical FWEXT circuit fwext interface gigabitethernet 1/5 logical vpt1_5 circuit vpt1_5 interface-internal GRBR5 logical-all GRBR5 circuit intvlancirc1 logical vlan20 ingress-vlan-tag 20 20 circuit vlan20 bridge-mode vlanbridge1 transparent circuit intvlancirc1 circuit vpt1_5
Configuration Using Multi-Link Groups

The following configuration creates two multi-link groups, MLT1 and MLT2, then adds these groups instead of physical interfaces to the bridge group. This allows greater throughput of traffic. However, the multi-link group cannot have any configured logical interfaces. vap-group is1 xslinux_v3 vap-count 2 max-load-count 2 ap-list ap4 ap5 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 11 12 ip-flow-rule ips_def_rule_is1 action load-balance priority 15 activate circuit MLTa device-name MLTa vap-group is1 promiscuous-mode active circuit MLTb device-name MLTb vap-group is1 promiscuous-mode active
119
circuit MLTBRIDGE1 device-name MLTBRIDGE1 vap-group is1 group-interface MLT1 interface-type gigabitethernet mode multi-link circuit MLTa gigabitethernet 1/1 gigabitethernet 1/2 gigabitethernet 1/3 group-interface MLT2 interface-type gigabitethernet mode multi-link circuit MLTb gigabitethernet 2/1 gigabitethernet 2/2 gigabitethernet 2/3 bridge-mode MLTBRIDGE1 transparent circuit MLTa circuit MLTb
120
11
Configuring CPM Redundancy
This chapter explains how to configure redundant CPMs. It includes the following major topics:
z z z z z z z
Overview on page 121 Installing CPMs and Configuring Redundancy in a New System on page 124 Installing a Second CPM into an Existing System on page 125 Managing CP Redundancy on page 125 Troubleshooting CP Redundancy on page 126 Using the CBS PSI Tool on page 127 Configuring a Redundant CPM Management Virtual IP Address on page 128
NOTE: APM redundancy is covered by creating redundant VAPs, as described in Configuring VAP Groups on page 45. NPM redundancy is covered by creating redundant interfaces, as described in Configuring Interfaces on page 61 Multi-system redundancy allows one X-Series Platform to be redundant for another, which is accomplished using VRRP, as described in Configuring Multi-System High Availability on page 129.
Overview
CPMs provide physical resources, data, and services to the X-Series Platform, as well as outside management clients. To provide continuous operation of the system in the event of a CPM failure, CP redundancy should be configured between CPMs. The X-Series Platform architecture supports a Primary and a Secondary CPM. Whenever two CPMs are present in the X-Series Platform, the CPMs (and other modules in the system) learn about each others states through periodic heartbeat messages. Either CPM can act as the Primary/Secondary (Active/Standby) CPM from either slot. However, at any given time, the system can have only one Primary CPM. If the Primary CPM detects a catastrophic module failure and crashes; or if the Secondary CPM detects that the Primary CPM is no longer operational, the following occurs:
z z z z z
Secondary CPM forces a hardware reset of the Primary CPM slot. Secondary CPM declares itself as the new Primary CPM. Primary CPM IP address is installed on the control bus IP interface of the Secondary CPM. New Primary CPM sends out a gratuitous ARP message with its own MAC address. New Primary CPM starts all daemons that run on the Primary CPM.
To make the failover virtually transparent to the other modules in the system, the Primary and Secondary CPMs are assigned unique IP addresses on the control network. These addresses are role based, not slot based. This makes the Primary CPM control network IP address the same, independent of the slot where the CPM resides. (This is different than the management interfaces IP address, which is slot-based.) To the APMs and NPMs, the CPM failover looks like a quick CPM reboot.
121
NOTE: When using a Primary/Secondary in the admin state for two CPMs and the Primary CPM is rebooted, the Secondary CPM attempts to become the Primary. During this process, if the Primary finishes rebooting before the Secondary becomes the Primary, the Secondary is put into the single user mode, making it inaccessible and displaying the following message: An error occurred during the file system check. Dropping you to a shell. Use the fsck command ... Use caution when using the Primary/Secondary admin state, and if this scenario occurs, use the commands displayed in the error message to place the Secondary CPM back online. Configure the IP NAT inside and outside settings on each management interface. When configuring for redundancy, make sure that the IP NAT settings are the same for the interfaces on both CPMs.
CP Redundancy Software Components

The XOS CP redundancy software uses the following components:
z z z z z
Heartbeat Driver - Maintains the state of all the modules in the system and implements the Primary CPM election protocol. Two-Port Multiplexor Driver (TPM) - Creates an abstraction of one reliable control network out of two control Ethernets. CPRM (Control Processor Redundancy Management) Daemon (cbscprmd) - Executes the CP redundancy state machine. Disk Mirroring Driver (drbd) - Executes disk mirroring. Shell Scripts - Execute procedures necessary for CP redundancy state transitions.
CP Redundancy Hardware Requirements

To use CP redundancy you must have the following hardware installed on your X-Series Platform:
z
Backplane EEPROM should be identical on both CPM slots. To check the backplane EEPROMs, use the following command: /usr/os/bin/eepromdiag -c backplane -rs NOTE: Execute this command on both CPMs to ensure that the results are the same.
Valid board EEPROM on both CPMs. To check the board EEPROMs, use the following command: /usr/os/bin/eepromdiag -c board -rs Partition size must be identical on both CPMs. Use the following command to perform the check: cat /proc/partitions
CP Redundancy and RAID Limitations

This section describes requirements and limitations regarding RAID and CP redundancy configurations.
z z z z
Both CPMs must have identical RAID hard disk drive (HDD) configurations A mix of non-RAID and RAID configurations is not supported in CP redundancy. The mirrored partition sizes on each HDD on both CPMs must be identical. If you add a CPM-8600 to an X-Series Platform with an existing CPM-8600, you must make sure the new CPM-8600 has the same configuration as the original. To configure CP redundancy with RAID, perform a fresh install on each CPM-8600.
122
CP Redundancy Runtime States

The following are possible runtime states for the Primary and Secondary CPMs:
z
Unknown The CPM is in an unknown state and may be in the process of booting. If the offline CPM is in an Unknown state for a long period of time, use the cp-unknown-state command to instruct the primary CPM to ignore the offline CPM. Initialization The CPM is in its initial state. Un-synchronized

z z
Occurs when the CPM crashes during re-synchronization of the mirrored disk partitions. After going into this state, the CPRM daemonizes itself and allows normal Linux initialization to complete. Occurs during the repair process, if the Primary CPM crashes.
In both cases, the CPRM waits for another CPM to become the Primary CPM. The CPRM learns about this event from the heartbeat driver. The CPRM on the un-synchronized CPM makes sure the control network interface has the same subnet mask as that of the Primary CPM and reconfigures it if it does not. The CPRM then goes into the Repair state and waits for the connection from the Primary CPM.
z
Repair The Secondary CPM is being synchronized to the Primary CPM. When the CPRM goes into the Repair state it allows Linux initialization to complete. The CPRM then waits for a TCP connection to be established from the Primary CPM. After TCP connection is established, the CPMs exchange information to access what repairs are needed. During synchronization, any commands issued against the CPM are blocked to prevent the disruption of the synchronization process. This includes attempting to execute the following commands:

upgrade reload cpm modules change cp-redundancy setting, including disabling or setting either CPM offline.
Offline The X-Series Platform is operating in the stand-alone CPM state. The CPRM goes into the Offline state either during initialization or when the System Administrator requests it through the CLI. When the CPM is in this state, other system processors ignore it as if the module is not plugged into the chassis. The only XOS subsystem that runs on the CPM in this state is the heartbeat driver, as it needs to communicate the Offline state to the other CPM to avoid the other CPM from causing a reboot. The Offline state is useful in preventing hard drive data corruption when a CPM is accidentally plugged into the wrong chassis or when the System Administrator needs to perform diagnostics or maintenance operations. To return the Offline CPM to another state, the module must be rebooted or plugged into the proper chassis. Primary The Primary CPM runs all software subsystems and replicates all data on mirrored partitions and configuration data on the CPM system partition. At 10 second intervals, the CPRM updates the Persistent State Information (PSI) with a new timestamp. The CPRM, on the Primary CPM, also learns through the heartbeat driver the state of the other CPM. States include: a. b. c. CPM is in the Offline state or not plugged in. This module is ignored. CPM is in the Un-synchronized or Election state. If this persists for the configured timeout (the default value is 60 seconds), the other CPM is reset. CPM is in the Primary state. A major error has occurred. When this occurs there is connectivity to other processing elements and many of them agree that the current primary is correct and that the second CPM is reset. If however, the majority of elements point to the second CPM as the primary, the current primary is reset. If no other processing elements are running, the CPM with a smaller slot number resets the other CPM. CPM is in the Repair state. In this case, the CPRM establishes a TCP connection to the other CPM and exchanges information to determine what repairs are needed. The CPRM then starts data replication once it learns from the other CPM that it is ready to do so. It also informs the other CPM when data replication is complete.
d.
123
e. f.
z
CPM is in the Secondary state. In this case, the CPRM watches the Secondary CPM to determine if it is healthy. The CPRM resets the Secondary CPM if it is found to be unhealthy or failed. CPM is plugged in but does not come up for more than the configured timeout (Default value is 20 minutes). The CPM is reset.
Secondary The CPM is a standby, ready for fail-over state. The CPM in this state maintains a replicated copy of data on mirrored partitions and configuration data on the CPM system partition. At 10 second intervals, the CPRM updates the CP_PSI with a new timestamp. The goal of the Secondary CPM is to determine when the Primary CPM fails or becomes unavailable in another manner. When the CPRM on the Secondary CPM detects such a condition, it resets the Primary CPM and sets the Secondary CPM to the Primary state. This change of the system state is propagated through the heartbeat to other processing elements. The CPM components also learn about the state change from the heartbeat driver and take appropriate component specific actions. In particular, the CPRM instructs the mirroring driver (DRBD) to run in primary mode and mount the corresponding devices. It then assigns a CPM the Primary CPM IP address to a control network interface.
Installing CPMs and Configuring Redundancy in a New System

Perform the following procedure to install and configure the X-Series Platform for CP Redundancy. Depending on your hardware platform, the slots for the CPMs are 6 and 7, or 13 and 14. NOTE: When configuring CP Redundancy, dual CPMs will synchronize only if each CPM is running the same XOS Version and Kit number. Previous versions of XOS allowed kit variations, as long as the release version was the same. 1. 2. 3. Insert the CPM that is intended to be the Secondary CPM into the desired slot. If unfamiliar with this process, refer to the hardware installation guide for your hardware platform. Install the XOS software. Log into XOS as root: CBS# unix su Password: [root@xxxxx admin]# 4. 5. 6. 7. 8. Halt the CPM: # halt Insert the CPM that is intended to be the Primary CPM into the desired slot. Install the XOS software. Refer to the Install Server User Guide for detailed instructions. When prompted, make sure to enable CP Redundancy on this CPM. Configure CP redundancy <Y or N>[N]: y Reload the entire X-Series Platform. CBS# reload all This command may take a few minutes. Any unsaved configuration will be lost. Do you want to save it to startup-config? <Y or N>[Y]: #Start Configuration Validation ## #End Configuration Validation Do you still want to save the current configuration? <Y or N>[Y]: Proceed with reload? <Y or N> [Y]: Make sure that the Primary CPM is up and running before continuing. 9. At the secondary CPM, press the reset pin located at the front to reboot, or remove and re-seat the CPM.
124
Installing a Second CPM into an Existing System

To install a second CPM and configure CP Redundancy on an existing system, complete the following: 1. Prevent the current CPM from monitoring the second CPM for redundancy: CBS# cp-unknown-state {cp1|cp2} ignore Enter the Primary CPM in this command. For example, use the following command when the Primary CPM is currently in the first CPM slot and the intention is to install a CPM in the second CPM slot: CBS# cp-unknown-state cp1 ignore 2. 3. 4. 5. Insert the new CPM into the chassis. Using the Install Server or USB Installer, install the same version of XOS software that is on the primary CPM. Do not reboot the new CPM when the installation process has completed. Configure CP redundancy on the current primary CPM using the following command: CBS# configure cp-redundancy Reload all modules. Choose yes to save the configuration when prompted. CBS# reload all This command may take a few minutes. Any unsaved configuration will be lost. Do you want to save it to startup-config? <Y or N>[Y]:Y #Start Configuration Validation ## #End Configuration Validation Do you still want to save the current configuration? <Y or N>[Y]:Y Proceed with reload? <Y or N> [Y]:Y 6. When the primary CPM has been reloaded, reboot the newly installed CPM. When the new secondary CPM boots up the first time, it will discover the chassis system ID and reboot. After this reboot, it will start the synchronization process. 7. Have the CPMs monitor each other for redundancy: CBS# cp-unknown-state {cp1|cp2} monitor
Managing CP Redundancy
For redundancy purposes, each CPM can be administratively configured to the following states:
z z
Election The CPMs determine the correct Primary CPM. Offline A CPM is set to offline for offline work.
Setting the CPM state to Offline takes effect immediately.
Displaying the CP Redundancy State

To determine the current administrative state, use the following command: CBS# show cp-redundancy Administrative State: CP1 (this cp) is ELECTION CP2 (other cp) is ELECTION CP Redundancy is ENABLED
125
Operational State: CP1 (this cp) is PRIMARY CP2 (other cp) is REPAIR CP Redundancy is ENABLED Synchronization Status: Disk synchronization is 20% completed NOTE: If the show cp-redundancy command is executed during an In Service Upgrade, the following warning message will be displayed: WARNING: In-Service Upgrade Is in Progress.
Configuring the CP Redundancy Administrative State

To change or manage CP redundancy, use the following command: configure cp-redundancy set {cp1|cp2|this_cp|other_cp} {election|offline} Where: cp1 cp2 this_cp other_cp election offline Configures CP1 Configures CP2 Configures the CPM from which you are issuing the command Configures the other CPM Sets specified CPM to election state Sets specified CPM to offline state
To take a CPM out of the offline state, execute the following commands on the Offline CP: CBS# configure cp-redundancy set this_cp election CBS# copy running-config startup-config If there is a Primary CPM in this system, the configuration on the Primary CPM must match the configuration on the other CPM. To ensure this, execute the following command on the Primary CPM: CBS# configure cp-redundancy set other_cp election After executing the previous command, reboot the offline CPM. If the configuration of the CPM being rebooted does not match that of the Primary CPM, the CPM being rebooted comes up in an offline state.
Troubleshooting CP Redundancy
To troubleshoot potential CP redundancy, use the following commands and tools:
z z z
show cp-redundancy command from the CLI Persistent State Information tool (PSI) CPRMD log files located in /usr/os/logs/cbscprmd.log and cbscprmd.log.saved.
To use the PSI tool use the following command: [root@xxxxx /root]# /usr/os/bin/cbspsitool <options> The PSI usage options are: -r, rbcnt <value>. Sets the reboot count to the designated value. -x, xos <ser_num>. Sets the XOS serial number to the designated serial number, which is described in Using the CBS PSI Tool on page 127. -g, debug <level>. Executes the program with debug set to the specified level.
126
-v, version. Displays the version of this program and exits the PSI tool. -h, help. Prints this help message and exits the PSI tool. The PSI.dat file contains the count of successive unsuccessful reboots. If there are excessive reboots, the CPM automatically goes into the Offline state. If this occurs, use the following command to reset the count to zero: /usr/os/bin/cbspsitool -r 0 The /usr/os/logs/cbscprmd.log and cbscprmd.log.saved files contain important CPRM log messages for the current and previous versions of the CPRM daemon, respectively.
Using the CBS PSI Tool

During the installation of a new CPM into an existing chassis, it may be necessary to reset the PSI (Persistent State Information) value on the CPM hard disk. This value is a record of the chassis backplane serial number which is learned by, and stored on, the CPM. At boot time, the CPM compares this value to the current backplane serial number. If the numbers match, the CPM boots completely. This means that a primary CPM boots, starts daemons, manages the other modules in the chassis; and a secondary CPM boots and starts synchronizing with the primary CPM. If the numbers do not match, the CPM boots in the standby mode only. This means that the CPM boots but does not synchronize, nor attempt to manage the other modules in the chassis. This process prevents a loss of configuration. However, if the PSI value is zero, the CPM learns the current backplane serial number and boots completely. To run the CBS PSI tool, complete the following: 1. Exit to the UNIX prompt, using the following command: CBS# unix su password: UNIX_password [root@xxxx admin]# 2. Run the CBS PSI tool to reset the box_ser_num value to zero as follows. This causes the CPM to relearn the current backplane serial number upon the next boot. [root@xxxx admin]# /usr/os/bin/cbspsitool -x 0 A display similar to the following should be displayed on the console. The backplane serial number is identified by the box_ser_num. CCpNvPsi class information psi_fname: /usr/os/.PSI.dat ------ Persistent State Information -----------version: 1 box_ser_num: 0 state: CP_PSI_PRIMARY admin_state: CP_ADMIN_ELECTION timestamp: 1035216100 - Mon Oct 21 12:01:40 2002 update_seqn: 27306 reboot_cnt: 0 xsum: 0x00000bfa ---- mirror id ---ocp_ser_num: A2330003 timestamp: 1034973673 random: 1804289383 This command can be used before you unplug the CPM or after you plug the CPM into a different X-Series Platform. In the first case, you should stop the CPRM daemon before resetting the serial number. In the latter case, reboot the CPM after resetting the serial number. When CPRM finds the X-Series Platform serial number in the file is zero, it replaces it with correct serial number that it reads from the EEPROM.
127
Configuring a Redundant CPM Management Virtual IP Address

A single virtual IP address (VIP) can be configured to persist from CPM to CPM during failover. This virtual IP address can be used for any communication to the device (SSH, EMS, CLI, etc.) as well as a single-source virtual IP address for SNMP traps, irrespective of which CPM is the active Primary. NOTE: Configuration of a management VIP address is allowed either in the Primary or offline state. If configured in an offline state, the management VIP address will not become active until the CPM becomes Primary. The virtual management IP address utilizes the Active CPM interface MAC address.
Limitations
The following lists the management VIP address restrictions:
z z z
The underlying management interface must already be defined and assigned a static IP address. If an active / standby CPM is present, both the primary and the secondary static management IP address must be on the same network as each other. The management VIP address must have the same network as the underlying static management IP address. Although the management VIP address inherits its subnet mask from the static management IP address, the management VIP address itself must be different from the static management IP address. The specified management VIP address must not already be in use:

On another management interface As a hop in the management routing table In the management host table As a management ip-alias
z z
The management VIP address is only used for the management interface, it is not used for any of the other interfaces on the CPM. In a dual-system configuration, the management VIP address cannot be the same as any CPM IP address used in the configure remote-box command.
Redundant CPM Management Virtual IP Address Configuration

Perform the following: 1. Execute the following command: CBS# configure management vip-addr <ip address> Where <ip address> is the desired virtual IP address. 2. 3. Validate the configuration of the virtual IP address by executing the following command: CBS# show management-vip To modify the virtual IP address that you configured, delete it using the following command then configure the new virtual IP address: CBS# configure management no vip-addr <ip address> Where <ip address> is the IP address to be removed.
Virtual IP Address Interaction during In-Service Upgrade (ISU)

When performing an ISU with a virtual management IP address configured, the virtual IP address will remain associated with the CPM running the original version of the XOS software. The virtual IP address will only transition to the CPM running the new version of the XOS software when ISU completes and the old CPM is upgraded to the new XOS software. This transition occurs when the user selects Option 3 during the final question of the Interactive In-Service upgrade. Refer to the In-Service Upgrade (ISU) Overview on page 233.
Configuring CPM Redundancy 128
12
Configuring Multi-System High Availability
This chapter provides information about configuring two or more X-Series platforms in a redundant manner using VRRP and Failover Groups to achieve a Multi-System High Availability configuration. The following topics are covered in this chapter:
z z z z z
VRRP Components on page 129 VRRP Configurations on page 133 Example Dual Box High Availability Configuration on page 143 Example VRRP Configurations on page 144 Configuring VRRP and Automatic ARP for NAT on page 148
You can configure multiple X-Series Platforms for High Availability (HA) using the Virtual Router Redundancy Protocol (VRRP). However, you do not need to configure one chassis as master and another as standby. Instead, create a failover group containing one or more VAP groups and circuits, and create a similar failover group with the same ID on one or more separate chassis. Should one group fail, the counterpart group on another chassis becomes master. For detailed steps to configure an Active-Standby or an Active-Active Dual-box, high availability configuration, refer to the Multi-System High Availability Configuration Guide provided with your XOS documentation.
VRRP Components
The following sections describe the various components used when configuring VRRP.
Failover Group
Failover groups and Virtual Routers (VRs) are used only in High Availability configurations. A failover group is a grouping of one or more VRs. A VR identifies the circuits and associated VAP groups for high availability. Only a failover group, not the entire system or an individual VAP group, can fail over to a backujp failover group on another system. Failover groups operate in pairs, one on each chassis, and are usually assigned different VRRP priorities. The failover group with the higher priority is the master. Each failover group needs a counterpart failover group on another chassis participating in VRRP multi-system redundancy. For example, a failover group on an X-Series Platform may contain the VAP groups and circuits used for a firewall application. The same firewall application and configuration would be on a second X-Series Platform, also contained in a failover group. Both failover groups must have the same group identifier; however, they must have different priority values. The failover group with the higher value assumes Master status. You cannot use two failover groups on the same chassis to back up one another.
129
Virtual Router (VR)

A virtual router can be attached to a single circuit only, and can include only one VAP group attached to that circuit. In addition, the VR can assign individual IP addresses to the circuit and the VAP group interface. For circuits already configured with an IP address, the VR can also assign a virtual IP address. This virtual IP address allows you to configure failover groups using the same virtual IP address on more than one chassis.
VRRP Priority
Each failover group is assigned a VRRP priority. The failover group with the higher priority is designated as the Master. All failover groups with the same ID must be configured with different VRRP priorities. A failure within a failover group does not necessarily cause a failover to another chassis. Instead, the VRRP priority is reduced by a pre-configured value, called a priority-delta. This minimizes or eliminates the problem of failing over to a chassis that has an even more diminished capacity. After the failure has been rectified, the VRRP priority returns to its configured value. Each of the following events can be assigned a priority-delta:
z
Failure of an interface that is associated with a monitored VR circuit Examples include a redundant interface, an LACP interface, or a bridge interface. If the chassis has been configured with redundant interfaces, as described in Redundant Interfaces on page 65, an interface failure that causes a failover to a redundant interface in an UP state does not cause the VRRP priority to be decremented.
Unreachable next hop IP address Failure to reach the next-hop IP address is based on the ability of the address to answer ARPs or, for IPv6, neighbor solicitations. The next-hop IP to be verified is configured within the VRRP/virtual-router/ VAP group. If you configure a failover group with a next hop health check and you also configure a monitored circuit with an associated interface, an unreachable next hop caused by a failure of the interface does not cause two decrements of the VRRP priority delta (one for the interface failure and one for the next hop becoming unreachable). The VRRP priority delta is decremented once for the interface failure and XOS prevents two decrements of the VRRP priority for a single cause. NOTE: If you configure redundant interfaces and associate them with a monitored interface, and both the master and backup interfaces fail, the situation is the same as that described in the previous paragraph. If the redundant interface passes a subsequent next hop health check, the VRRP priority is restored to its previous value.
Less than the desired number of active VAPs per VAP group You can configure the minimum number of active VAPs that you require in the VAP group. If the VAP group is configured as part of the failover group, then when the number of VAPs falls below the limit, the VRRP priority of the failover group is decremented.
Interface failure in a monitored circuit. Circuits that connect physical interfaces to VAP groups can be monitored using VRRP. If a failure on the monitored circuit is detected, the priority delta (value) decreases the configured priority (value) of the failover group. If the priority delta drops below the priority of the failover group on the other chassis, the failover group loses its master status, and fails over to the backup. With preemption enabled, after the monitored circuit recovers, the failover group resumes its role as master. NOTE: By default, preemption is OFF to prevent the second failover (with its associated delay) that would restore the original Master Failover Group as Master.
When an event causes a failover, the Interface Redundancy Manager (IRM) daemon deactivates IP addresses in the Master failover group then activates those IP addresses in the backup failover group.
130
OSPF Failover in Conjunction with VRRP

Crossbeam uses VRRP to provide failover between devices that do not participate in a routing protocol exchange and do not notice link or node-level failures. However, the VRRP status is not propagated to dynamic routing protocol entities such as OSPF and this may result in broken or asymmetric IP forwarding. To solve this problem, Crossbeam enables the VAP to communicate per-circuit VRRP status to the associated OSPF daemon. This is done through the ospf-cost-increment parameter in the vrrp failover-group CLI context. When the VRRP status of an interface is backup, XOS increases the cost of the corresponding OSPF interface. When the VRRP status of the interface changes to master, XOS removes the incremented cost from the OSPF configuration. For more information about OSPF configuration, refer to the RSW Installation Guide.
High Availability (HA) Port and Management Interfaces

The X-Series Platform requires a communication link between all the X-Series Platforms in a High Availability (HA) configuration. Crossbeam recommends using the CPM High Availability port and both Management ports on the primary CPMs and both Management ports on the secondary CPMs when connecting between chassis. In a dual-system configuration, each pair of ports (HA, Management 1, and Management 2) are connected to separate network broadcast domains. Crossbeam recommends that customers do not connect the High Availability ports or Management ports directly to each other. Connecting the ports directly can lead to scenarios that do not provide full redundancy. NOTE: Typically, you configure High Availability and Management ports for auto-negotiation. If you connect any of these ports to a switch and auto-negotiation does not work, use the configure management high-availability or configure management gigabitethernet command to manually set up the communication parameters.
VRRP Example
Systems do not have to be configured with identical failover groups. For example, as shown in Figure 7 System A has these failover groups:
z z z z
Failover Group 1 contains the VAP group and circuits used for a firewall and has VRRP priority 150. Failover Group 2 contains the VAP group and circuits used for an intrusion prevention system application and has VRRP priority 150. Failover Group 3 contains the VAP group and circuits used for a URL filtering application and has VRRP priority 100. Failover Group 4 contains the VAP group and circuits used for a web security application and has VRRP priority 100.
System B has:
z z z z
Failover Group 1 contains the VAP group and circuits used for a firewall and has VRRP priority 100. Failover Group 2 contains the VAP group and circuits used for an intrusion prevention system application and has VRRP priority 100. Failover Group 5 contains the VAP group and circuits used for an e-mail security application and has VRRP priority 150. Failover Group 6 contains the VAP group and circuits used for an intrusion detection system application and has VRRP priority 150.
131
System C has:
z z z z
Failover Group 5 contains the VAP group and circuits used for an e-mail security application and has VRRP priority 100. Failover Group 6 contains the VAP group and circuits used for an intrusion detection system application and has VRRP priority 100. Failover Group 3 contains the VAP group and circuits used for a URL filtering application and has VRRP priority 150. Failover Group 4 contains the VAP group and circuits used for a web security application and has VRRP priority 150.
Figure 7.
VRRP Example
System A
System B
System C
Failover Group 1: Firewall Failover Group 2: IPS Failover Group 3: URL Filter Failover Group 4: Web Security
Priority 150 Priority 150 Priority 100 Priority 100
Failover Group 1: Firewall Failover Group 2: IPS
Priority 100
Priority 100 Failover Group 5: E-mail Security Priority 150 Failover Group 6: IDS Priority 150
Failover Group 5: Failover Group 6: Failover Group 3: Failover Group 4:
E-mail Security IDS URL Filter Web Security
Priority 100 Priority 100 Priority 150 Priority 150
Layer 2 Switch
In this example, each of the systems hosts two Master failover groups and two backup failover groups. Under normal operating conditions, System A runs the Firewal and IPS applications, System B runs the E-mail Security and IDS applicaitons, and System C runs the URL Filter and Web Security applications.. With all systems running properly, a catastrophic problem with System A causes Failover Groups 1 and 2 to fail over to System B. Similarly, with all systems running properly, a catastrophic failure on System B causes Failover Groups 5 and 6 to fail over to System C. If System A suffers an interface failure within Failover Group 1, reducing its VRRP priority to 125, no failover would occur. If additional failures on System A reduce the VRRP priority of Failover Group 1 below 100, a failover to System B would occur. NOTE: This example shows independent applications, where none of the configurations serialize traffic from one application to another. When traffic is serialized between two applications, such as Firewall and IPS, both applications are placed in the same failover group.
132
VRRP Configurations
There are two basic configurations: Active-Standby and Active-Active. In both configurations, one failover group is Master and the counterpart failover group is in backup mode. With Active-Standby, the failover group on one chassis passes traffic while the backup group on the other chassis does not. With Active-Active, the Master failover group on each chassis passes traffic. The corresponding failover groups on each chassis are in backup mode. To achieve an Active-Active configuration, attach one circuit to two VRs, where each VR is in a different failover group. The basic configuration, as shown in Figure 8, includes:
z
Failover Group 1:

On System A, this failover group has a VR attached to a circuit connected to a VAP group. Since the circuit does not have an IP address assigned, the VR assigns a primary IP address. On System B, the failover group has a VR with the same configuration; however, the VR assigns a virtual IP address, which is the same as the primary IP address on System As VR. The failover group on System A is given a higher VRRP priority than System Bs failover group. Thus System As failover group is initially the Master. On System B, this failover group has a VR attached to a circuit connected to a VAP group. Since the circuit does not have an IP address assigned, the VR assigns a primary IP address. On System A, the failover group has a VR with the same configuration; however, the VR assigns a virtual IP address, which is the same as the primary IP address on System Bs VR. The failover group on System B is given a higher VRRP priority than System As failover group. Thus System Bs failover group is Master.
Failover Group 2:

In this configuration, each Master group has a unique IP address; thus, traffic can flow to both addresses. Should one group fail, the failover group on the other chassis automatically handles all traffic going to both addresses. Should the circuit be configured with an IP address in this configuration, both VRs would assign virtual addresses instead of one VR assigning a primary address.
Figure 8.
VRRP Active-Active Configuration
133
Before creating a VRRP configuration, determine:

z z z z
Which X-Series Platforms are to be involved What applications or configurations require redundancy Definition of each failover group Which X-Series Platform will host the master failover group, and which will host the bacup failover group
Next, perform the following steps, described in the subsequent sections: 1. 2. 3. Physically cable and configure the High Availability ports. Create failover groups. Configure VRRP on the VAP groups.
IMPORTANT: Each X-Series Platform must have a unique identifier to be in a VRRP configuration, which is set with the configure system-identifier <system-id> command. NOTE: A VRRP Virtual Router (VR) configuration can have only one VAP group assigned. A VAP group is assigned to a VR using the vrrp virtual-router vrrp-id <id> vap-group <name> command. During the XOS upgrade you will be prompted to change your configuration. If you do not make this change, the system keeps only the first vap-group and issues a warning message.
Configuring High Availability Ports

There are a few options when cabling the High Availability ports:
z z z
In a dual-system configuration, the High Availability ports can be connected directly to each other or can be connected using a switch. For 3 or more systems, the High Availability ports are connected to a switch as shown in Figure 7 on page 132. If you have CP redundancy, you can connect all four High Availability ports (2 chassis, with primary and secondary CPMs on each chassis) into a single switch. This allows physical connectivity between the two primary CPMs, one in each chassis. Should a primary CPM fail in one chassis, the backup CPM will still be able to communicate with the remote chassis. To prevent packet loops, the secondary CPM automatically drops all packets coming into its High Availability port.
Figure 9.
Multi-System VRRP Hardware Configuration
134
Redundant Interfaces for the High Availability Ports

For additional High Availability port redundancy, you can configure the two CPM management ports as redundant interfaces for the High Availability port. However, each management interface must be reachable via IP routing from the corresponding remote chassis. You can use the remote-box command to specify up to five IP addresses that could be used to connect to each remote chassis, each IP address corresponding to a port on one CPM on the remote chassis. The first address is used to establish the initial UDP session to the remote box. By specifying more than one IP address, you provide redundancy in the event the UDP session over the first IP address fails. NOTE: The remote-box command requires that you have interconnected CPMs on the two chassis. The IP address that you specify depends on how you have interconnected the CPMs. If you use the High Availability port, specify the internal IP address ( for example, 1.1.45.20) associated with the remote Primary CPM (obtained by running show internal-ip on the remote chassis). If you use either or both of the management ports, specify the external IP address(es) associated with the port(s). Crossbeam recommends that you connect the CPMs using separate network broadcast domains for each pair of ports (the HA ports, the Management 1 ports, and the Management 2 ports). Connecting the ports directly can lead to scenarios that do not provide full redundancy. With this method, in the absence of any remote chassis configuration, VRRP exchanges between the IRM daemons still occur.
Show Remote Box

Use the show-remote-box command to view the path and path status for each of the interconnected ports on the two systems. CBS# show remote-box 22 Local System ID: 85 Remote System ID: 22 Remote IP Local Intf Local IP 192.168.211.89 14/1 192.168.211.85 10.10.22.20 HA port 10.10.85.20 (2 rows)
Status Time In State Active 0 days, 00:00 Standby 0 days, 00:00
Link Qual 0 0
135
Creating an Active-Standby Configuration

This section describes one way to use VRRP to build an Active-Standby failover configuration. The steps assume an existing single VAP configuration and configure virtual routers, virtual IP addresses, and assign failover priority values to the circuits and interfaces. NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management networks are configured with IPv4.
Configuring the Remote System ID

A remote system is identified by its system ID. Use the configure remote-box command to configure the remote system ID and IP address. configure remote-box <system-id> <1st-ip-address> [<2nd-ip-address>] [<3rd-ip-address>] [<4th-ip-address>] [<5th-ip-address>] If you connect Chassis 1 to Chassis 2 using only the High Availability (HA) port, you do not need to configure the remote system identifier. The two chassis use the HA port to exchange system-id information. However, if you do this, the HA port connection represents a single point of failure. Crossbeam recommends that you connect the HA ports and both management interfaces on both the primary and secondary CPMs in both chassis. Then use the remote-box command on both chassis to configure all five ports on the remote chassis. Configure the remote system ID and IP address using the configure remote-box command. NOTE: The remote-box command requires that you have interconnected CPMs on the two chassis. Crossbeam recommends that you specify the following IP addresses:

For the High Availability port, specify the internal ip address (1.1.46.20) associated with the remote Primary CPM (obtained by running show-internal-ip on the remote chassis). For the management ports, specify the external IP addresses associated with the ports. Ideally, specify all three addresses associated with the remote primary CPM and the two management addresses associated with the remote secondary CPM. Crossbeam recommends that you connect the CPMs using separate network broadcast domains for each type of port (the HA ports, the Management 1 ports, and the Management 2 ports) on both the Primary CPMs and the Secondary CPMs. Connecting the ports directly can lead to scenarios that do not provide full redundancy.
CBS# configure remote-box 46 10.10.46.20 192.168.50.46 192.168.51.56 CBS(conf-remote-box)# end NOTE: The example configure remote-box command specifies only the IP addresses of the management 1 and 2 interfaces for the primary CPM on chassis 2. Crossbeam recommends that you connect the management interfaces of the other CPM on Chassis 2 and that you add the IP addresses for those interfaces to the configure remote-box command.
Configuring VRRP Systems Linked Through a Switch

NOTE: When two CPMs on two different X-Series Platforms are linked through a switch in a multi-box redundancy configuration, both X-Series Platforms could become VRRP master if the switch is not configured for multicast. Routers are responsible for forwarding multicast traffic to other subnets in the network but use multicast routing protocols other than IGMP. To guarantee that multicast forwarding requirements are met for the other subnets, multicast traffic must always be forwarded to the router ports. The port on the switch that has a router connected to it can be configured so that it receives all multicast traffic streams. Normally, switches discover the router ports automatically. However, there may be circumstances where you must manually add router ports. The following example shows how to manually define router ports for a 3COM 4300 switch by using the addPort command on the routerPort menu:
136
1.
At the Top-level menu, enter: bridge multicastFilter routerPort addPort Select router unit (1):
2. 3.
Enter the unit number (1). The following prompt is displayed: Select router port (1-48): Enter the desired router port number.
Create Failover Groups

Before creating a failover group, you need to decide which configuration elements to include, specifically the circuits and VAP groups. The following procedure describes how to create a failover group. NOTE: When configuring VRRP, the broadcast IP address for the virtual router must be on the same subnet as the virtual router. XOS displays an error message if this condition is not met. To create a failover group using EMS, open Configuration > Redundancy > VRRP > VRRP Failover Group. Use the Help button for details. 1. Create the failover group by assigning it a name and ID. The ID must be unique on this chassis, and must be the same on its counterpart failover group on the remote chassis (Chassis 2). Syntax: configure vrrp failover-group <name> failover-group-id <1-255> Command: CBS# configure vrrp failover-group vrrp_FW failover-group-id 200 CBS(conf-vrrp-group)# 2. Set the VRRP priority. VRRP priority should be different for each chassis; the failover group with the higher priority is the master. Certain events such as an interface failure or a change in VAP group member count can be configured to decrement the VRRP priority. A chassis failover occurs when the VRRP priority value of one failover group drops below the priority of the failover group on the other chassis. VRRP priority values are 1 to 255, and the default is 100. Syntax: priority <1-255> NOTE: In this sample configuration, set the VRRP priority to 150 on Chassis A, and 100 on Chassis B. Chassis A is the master. Command: CBS(conf-vrrp-group)# priority 150 CBS(conf-vrrp-group)# 3. If you are using OSPF with RSW, assign an OSPF cost increment to the circuits configured in the VRRP failover group. This will force OSPF to fail over to the backup chassis at the same time that a decremented priority value causes VRRP to fail over. Syntax: CBS(conf-vrrp-group)# ospf-cost-increment circuit <circuit_name> <increment-cost> Command: CBS(conf-vrrp-group)# ospf-cost-increment circuit lan02 10 CBS(conf-vrrp-group)# ospf-cost-increment circuit lan04 10 CBS(conf-vrrp-group)# 4. Specify the interval time (in seconds) between VRRP advertisements to other systems. Only the Master failover group sends advertisements. The number of seconds can be 1 to 255. The default is 2 seconds. Since a backup failover group can become Master, it is recommended that you use the same setting on all chassis.
137
Syntax: advertise-interval <seconds> Command: CBS(conf-vrrp-group)# advertise-interval 30 CBS(conf-vrrp-group)# 5. Configure preemption to specify how you want VRRP to function on failure. When preemtion is enabled, failure of the master failover group causes traffic to be re-routed to the backup failover group (which then becomes Master). The original Master resumes Master status when it becomes available. When disabled (using no), upon failure of the master failover group, the backup failover group services traffic. The original Master does not resume being Master when it becomes available, unless the backup fails. NOTE: By default, preemption is OFF to prevent the second failover (with its associated delay) that would restore the original Master Failover Group as Master. The examples in this chapter do not enable preemption. 6. Determine which circuits to monitor and set a priority-delta for each circuit. A monitored circuit can affect the failover group VRRP priority; however, monitored circuits (usually a physical interface) do not fail over when VRRP switches to a backup failover group. The priority-delta decrements the failover group VRRP priority whenever an interface on the monitored circuit fails. In this example, physical interfaces are given a priority-delta of 30. When an interface failure occurs, the priority value (150) is decremented by the priority-delta (30) to 120. Since this is not less than 100 (priority on Chassis B), failover will not occur until more than one circuit has failed. Only then does the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the interface returns to the Up state.The priority-delta can be 0 to 255. Default is 1. Syntax: monitor-circuit <circuit_name> priority-delta <0-255> Command: CBS(conf-vrrp-group)# monitor-circuit lan02 CBS(conf-vrrp-failover-cct)# priority-delta 30 CBS(conf-vrrp-failover-cct)# exit CBS(conf-vrrp-group)# monitor-circuit lan03 CBS(conf-vrrp-failover-cct)# priority-delta 30 CBS(conf-vrrp-failover-cct)# exit CBS(conf-vrrp-group)# monitor-circuit lan04 CBS(conf-vrrp-failover-cct)# priority-delta 30 CBS(conf-vrrp-failover-cct)# exit CBS(conf-vrrp-group)# NOTE: If your system incorporates group interfaces, use the monitor-group-interfaces parameter in this context to set a priority delta. Additionally, you can set a minimum number of ports (in the group interface) using the dist-port-threshold parameter to monitor. If the number of ports in that state falls below this threshold, the priority delta value is subtracted from failover group priority. See the XOS Command Reference Guide for specific command information. 7. Create a Virtual Router, assign an ID, and attach it to an existing circuit. If more than one VR is attached to the same circuit, each VR must have a unique ID. Syntax: virtual-router vrrp-id <1-4096> circuit <circuit_name> Command: CBS(conf-vrrp-group)# virtual-router vrrp-id 101 circuit v101 CBS(conf-vrrp-failover-vr)# a. If you require the VR circuit interface to remain in an UP state when the failover group is in backup mode, use the backup-stay-up command. The circuit does not send or receive traffic while in the backup state. CBS(conf-vrrp-failover-vr)# backup-stay-up
Configuring Multi-System High Availability 138
NOTE: In a VSX configuration, VSX automatically enables backup-stay-up for all VLAN interfaces that are part of a VRRP group. b. Assign a MAC address to the VRRP circuit. Use the vrrp-mac feature to automatically generate a unique vrrp-mac. In the event of a failover, the MAC address moves with the virtual IP address to the new chassis. By keeping the MAC address consistent, it reduces the time it takes for upstream routers to converge routing tables. Alternatively, use interface (default) to use the MAC address configured on the physical interface. Syntax: mac-usage {vrrp-mac|interface} Command: CBS(conf-vrrp-failover-vr)# mac-usage vrrp-mac c. Assign a priority-delta to the VR. In this serialized topology, each virtual router is given a priority-delta of 55. When a VR fails, the failover groups VRRP priority of 150 is decremented by the priority-delta value of 55 to a new priority value of 95. Since this is less than 100 (the priority on Chassis B), the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the VR recovers. Syntax: priority-delta <1-255> Command: CBS(conf-vrrp-failover-vr)# priority-delta 55 CBS(conf-vrrp-failover-vr)# d. Assign a VAP group to the VR. Specifying the VAP group of the VR allows you to assign a virtual IP address to the VAP group. The circuit must already be mapped to the VAP group with the configure circuit <name> vap-group <name> command. Syntax: vap-group <vap-group-name> Command: CBS(conf-vrrp-failover-vr)# vap-group fw CBS(conf-vrrp-vr-vapgroup)# exit CBS(conf-vrrp-failover-vr)# e. Assign a unique IP address to the virtual router only when its circuit is NOT configured with an IP address. This address is considered to be the primary address. The no parameter deletes the IP address. Syntax: ip {<ip-addr> <netmask>|<ip-addr/netmask>} [<broadcast-ip-addr>] [increment-per-vap <ip-addr>] This command assigns the fw VAP group to the virtual router and an IP address of 172.16.20.103 with a mask of 24 in CIDR format. Command: CBS(conf-vrrp-failover-vr)# vap-group fw CBS(conf-vrrp-vr-vapgroup)# ip 172.16.20.103/24 CBS(conf-vrrp-vr-vapgroup)# f. If the circuit already has an assigned IP address, assign a virtual IP address as follows. Note that the number of virtual IP addresses assigned to an IP address is limited to 99. NOTE: Use the same format to assign virtual IPv6 addresses. Syntax: virtual-ip {<ip-addr> <netmask>|<ip-addr/netmask>} [<broadcast-ip-addr>] [increment-per-vap <ip-addr>]}
139
Command: CBS(conf-vrrp-vr-vapgroup)# virtual-ip 10.0.101.1/24 CBS(conf-vrrp-vr-vapgroup)# exit The optional floating parameter assigns the virtual IP address to the master VAP, allowing traffic, cluster management, and synchronization communication to go directly to the master VAP. If a new master VAP is elected, the address floats (is assigned) to the new master. In a VRRP configuration, the floating parameter assigns the virtual IP address to the master VAP on the new master chassis in the event of a failover. g. Configure a next hop health check and a priority-delta. The next hop health check is an optional setting to verify IP connectivity to the next hop gateway. In this example, the gateway is given a priority-delta of 55. If the next hop health check fails, the failover group VRRP priority of 150 is decremented by the priority-delta value of 55 to a new priority value of 95. Since this is less than 100 (the priority on Chassis B), the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the interface recovers. Specify the IP address of an external host (external to the X-Series Platform). The IP address 10.10.101.20 is used here as an example. Using exit twice returns you to the correct CLI context to repeat the VR configuration process. NOTE: An IP route whose next-hop destination network is exactly the same (same network and netmask) as the circuit's or virtual circuit's network is considered to be invalid. If you try and set such a next hop destination, an error message is displayed. Command: CBS(conf-vrrp-vr-vapgroup)# verify-next-hop-ip 10.10.101.20 CBS(conf-vrrp-vr-verify-next-hop)# priority delta 55 CBS(conf-vrrp-vr-verify-next-hop)# exit CBS(conf-vrrp-vr-vapgroup)# exit CBS(conf-vrrp-group)# h. 8. Repeat Step 6, a through g, to add additional VRs to the failover group. Note that each VR can be assigned to only one VAP group.
Create this failover group on one or more additional X-Series Platforms.
Configure the FW VAP group for Failover

VRRP monitors the fw VAP group for failure of individual VAPs. By setting the active-vap-threshold and the priority-delta, individual VAP failures can decrement VRRP priority and cause a comparison with the VRRP priority on the remote chassis. Failover occurs when the priority-delta value decrements the priority value of the master failover group below the priority value of the associated failover group on the remote chassis. In our configuration, the priority value for the master failover group is 150 and the priority value for the backup group is 100. A priority-delta of 60 is used for each of the VAP groups. To configure the fw VAP group for failover: 1. Enable VRRP on the fw VAP group. CBS# configure vrrp vap-group fw CBS(conf-vrrp-vap-group)# 2. Assign the VRRP enabled fw VAP group to a failover group list (required). CBS(conf-vrrp-vap-group)# failover-group-list failover1 CBS(conf-vrrp-vap-group)# 3. Optionally, set the hold down timer. Configure the hold-down-timer to 120, which forces the VAP group to wait for two minutes while the application fully boots. This wait prevents the failover group from becoming VRRP master before the application is fully active. CBS(conf-vrrp-vap-group)# hold-down-timer 120 CBS(conf-vrrp-vap-group)#
4.
Optionally, set the active VAP threshold and return to the main CLI context. The active-vap-threshold monitors the number of active VAPs in the VAP group. If the number of active VAPs drops below the threshold, the failover groups priority value is decremented by the priority-delta (defined in Step 5) and a comparison is done between the priorities of the failover groups on the two chassis. Failover occurs when the priority-delta decrements the priority value of the master failover group below the priority value of the backup group. CBS(conf-vrrp vap-group)# active-vap-threshold 3 CBS(conf-vrrp vap-group)# end CBS#
5.
Set the priority-delta value for the VAP group (optional). Assign a priority-delta to the VAP group. VRRP decrements the priority of the failover group whenever the number of active VAPs falls below the active-vap-threshold. The priority-delta value can be any number between 1 and 255 and the default value is 1. When the VAP returns to the Active state, the priority-delta value is added back to the priority value. CBS(conf-vrrp-vap-group)# priority-delta 60 CBS(conf-vrrp-vap-group)#
Manually Force a VRRP Failover

It may be useful to force a VRRP failover from the Master failover group to the backup (for example, if you want to update the policies on the VAP group in the primary failover group). To force a failover, use the vrrp-relinquish-master <VRRP_failover_group> command. Command: CBS# vrrp-relinquish-master vrrp_fw_primary Do you really want to relinquish VRRP Master?[N] You must enter Y for the command to execute. To manually return the primary failover group to master status, issue the vrrp-relinquish-master command on the currently active master: CBS# vrrp-relinquish-master vrrp_fw_backup Executing this command will force a VRRP failover. Are you sure? [N]
141
View VRRP Status

VRRP (DBHA) component status can easily be reviewed using the DBHA View in the Greenlight Element Manager (GEM). The DBHA View displays failover group status and remote box connections, including alarm conditions. Any alarm conditons are linked to details in the GEM Alarms Panel, simplifying identification and classification of the alarms. Refer to the GEM user assistance for additional information about the view.
Figure 10.
DBHA View in GEM
You can also view the status of VRRP components and connections using the following XOS CLI commands:
z z z
show vrrp status [<failover_group_id>] show vrrp detail-status [<failover_group_name>] show remote-box
See View VRRP Status on page 178 for details on using these commands. Also, see the XOS Command Reference Guide for all information regarding usage, options, and syntax for these commands.
142
Example Dual Box High Availability Configuration

The following is an example of configuring two chassis in a Dual Box High Availability system, where the two management ports are configured as redundant interfaces to the High Availability ports. NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management and control networks are configured with IPv4. System 1 system-identifier 1 remote-box 2 1.1.2.20 192.168.30.2 192.168.100.2 192.168.30.22 192.168.100.22 management gigabitethernet 13/1 ip-addr 192.168.30.1/24 192.168.30.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 13/2 ip-addr 192.168.100.1/24 192.168.100.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 14/1 ip-addr 192.168.30.11/24 192.168.30.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 14/2 ip-addr 192.168.100.11/24 192.168.100.255 enable access-list 1001 input access-list 1002 output System 2 system-identifier 2 remote-box 1 1.1.1.20 192.168.30.1 192.168.100.1 192.168.30.11 192.168.100.11 management gigabitethernet 13/1 ip-addr 192.168.30.2/24 192.168.30.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 13/2 ip-addr 192.168.100.2/24 192.168.100.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 14/1 ip-addr 192.168.30.22/24 192.168.30.255 enable access-list 1001 input access-list 1002 output management gigabitethernet 14/2 ip-addr 192.168.100.22/24 192.168.100.255 enable access-list 1001 input access-list 1002 output
143
Example VRRP Configurations

This section presents two example VRRP configurations - a simple Active-Active example, and a complex Active-Standby example. NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management and control networks are configured with IPv4.
Simple Configuration
The following is a simple example of two chassis in an active-active configuration: System A vrrp failover-group firewall failover-group-id 1 priority 101 monitor-circuit firewall virtual-router vrrp-id 100 circuit gig21 vap-group fw virtual-ip 192.250.10.100/24 192.250.10.255 virtual-router vrrp-id 101 circuit gig22 vap-group fw virtual-ip 192.251.10.100/24 192.251.10.255 vrrp failover-group ips failover-group-id 2 priority 201 virtual-router vrrp-id 200 circuit gig31 vap-group ips virtual-ip 192.150.10.100/24 192.150.10.255 virtual-router vrrp-id 201 circuit gig32 vap-group ips virtual-ip 192.151.10.100/24 192.151.10.255 # vrrp vap-group fw priority-delta 2 active-vap-threshold 3 # vrrp vap-group ips priority-delta 2 active-vap-threshold 3 System B vrrp failover-group firewall failover-group-id 1 priority 201 virtual-router vrrp-id 100 circuit gig11 vap-group fw virtual-ip 192.250.10.100/24 192.250.10.255 virtual-router vrrp-id 101 circuit gig12 vap-group fw virtual-ip 192.251.10.100/24 192.251.10.255 vrrp failover-group ips failover-group-id 2 priority 101 virtual-router vrrp-id 200 circuit gig31 vap-group ips virtual-ip 192.150.10.100/24 192.150.10.255 virtual-router vrrp-id 201 circuit gig32 vap-group ips virtual-ip 192.151.10.100/24 192.151.10.255
# vrrp vap-group fw priority-delta 2 active-vap-threshold 3 # vrrp vap-group ips priority-delta 2 active-vap-threshold 3
Complex Configuration
The following is a example of two chassis in an Active-Standby configuration with VLANs.
Figure 11.
Two System Active-Standby Configuration
System A vap-group fw xslinux_v3 # circuit inside3 circuit-id 1025 device-name inside3 vap-group fw default-egress-vlan-tag 3 hide-vlan-header ip 192.168.1.10/24 192.168.1.255 circuit inside4 circuit-id 1026 device-name inside4 vap-group fw default-egress-vlan-tag 4 hide-vlan-header
145
ip 4.4.4.10/24 4.4.4.255 circuit outside5 circuit-id 1027 device-name outside5 vap-group fw default-egress-vlan-tag 5 hide-vlan-header ip 5.5.5.10/24 5.5.5.255 circuit outside6 circuit-id 1028 device-name outside6 vap-group fw default-egress-vlan-tag 6 hide-vlan-header ip 10.10.1.10/24 10.10.1.255 interface gigabitethernet 1/1 logical inside3 ingress-vlan-tag 3 3 circuit inside3 logical inside4 ingress-vlan-tag 4 4 circuit inside4 interface gigabitethernet 1/2 logical outside5 ingress-vlan-tag 5 5 circuit outside5 logical outside6 ingress-vlan-tag 6 6 circuit outside6 vrrp failover-group left failover-group-id 1 priority 200 monitor-circuit inside3 priority-delta 10 monitor-circuit inside4 priority-delta 10 monitor-circuit outside5 priority-delta 10 monitor-circuit outside6 priority-delta 10 virtual-router vrrp-id 103 circuit inside3 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 192.168.1.50/24 192.168.1.255 virtual-router vrrp-id 104 circuit inside4 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 4.4.4.50/24 4.4.4.255 virtual-router vrrp-id 105 circuit outside5 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 5.5.5.50/24 5.5.5.255 virtual-router vrrp-id 106 circuit outside6 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 10.10.1.50/24 10.10.1.255 # vrrp vap-group fw priority-delta 20 active-vap-threshold 3 System B vap-group fw xslinux_v3
146
# circuit inside3 circuit-id 1025 device-name inside3 vap-group fw default-egress-vlan-tag 3 hide-vlan-header ip 192.168.1.11/24 192.168.1.255 circuit inside4 circuit-id 1026 device-name inside4 vap-group fw default-egress-vlan-tag 4 hide-vlan-header ip 4.4.4.11/24 4.4.4.255 circuit outside5 circuit-id 1027 device-name outside5 vap-group fw default-egress-vlan-tag 5 hide-vlan-header ip 5.5.5.11/24 5.5.5.255 circuit outside6 circuit-id 1028 device-name outside6 vap-group fw default-egress-vlan-tag 6 hide-vlan-header ip 10.10.1.11/24 10.10.1.255 interface gigabitethernet 1/1 logical inside3 ingress-vlan-tag 3 3 circuit inside3 logical inside4 ingress-vlan-tag 4 4 circuit inside4 interface gigabitethernet 1/2 logical outside5 ingress-vlan-tag 5 5 circuit outside5 logical outside6 ingress-vlan-tag 6 6 circuit outside6 vrrp failover-group right failover-group-id 1 priority 190 monitor-circuit inside3 priority-delta 10 monitor-circuit inside4 priority-delta 10 monitor-circuit outside5 priority-delta 10 monitor-circuit outside6 priority-delta 10 virtual-router vrrp-id 103 circuit inside3 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 192.168.1.50/24 192.168.1.255 virtual-router vrrp-id 104 circuit inside4 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 4.4.4.50/24 4.4.4.255 virtual-router vrrp-id 105 circuit outside5 priority-delta 20 mac-usage vrrp-mac vap-group fw virtual-ip 5.5.5.50/24 5.5.5.255 virtual-router vrrp-id 106 circuit outside6 priority-delta 20
147
mac-usage vrrp-mac vap-group fw virtual-ip 10.10.1.50/24 10.10.1.255 # vrrp vap-group fw priority-delta 20 active-vap-threshold 3
Configuring VRRP and Automatic ARP for NAT

In some specific VRRP configurations, if a firewall application is configured with automatic ARP for NAT both chassis may reply to the NAT ARP requests. This section describes configuration options for DBHA using VRRP and dynamic routing so that both chassis do not reply to the NAT ARP requests.
NAT MAC Usage

The Checkpoint VPN-1 Power NGX application uses the interface MAC as the virtual MAC when creating NAT entries. On the X-Series Platform, the interface is a circuit or VND (virtual network device). This VND MAC is either a circuit defined MAC, or an interface level MAC. Beginning with XOS 8.0, the MAC address manual definition is at the circuit VAP group (mac-addr - Sets MAC address) level: From the CBS CLI CBS# show circuit inside Circuit Name : inside Circuit-Id : 1 Device Name : inside ... MAC Address : 00:03:d2:f5:48:02 (system-reserved) ... Ifconfig from the VAP inside Link encap:Ethernet HWaddr 00:03:d2:f5:48:02 To prevent an incorrect NAT associated MAC configuration between the two chassis, the same VRRP MAC is defined at circuit level. With the VND level MAC address set the same as the VRRP IP address, VRRP gratuitous ARP generation upon failover will update the adjacent switch CAM table. This will ensure NAT automatic ARP failover is also transparent and performed efficiently.
Interface MAC Limitations

When configuring the X-Series Platform, you can associate multiple VNDs with a single interface differentiated by VLAN tags. These VNDs may belong to the same or different VAP groups. When using NAT with automatic ARP within VPN-1, the VND MAC address can be configured with the same address as the virtual VRRP MAC. When configuring Interface level MAC or circuit level MAC options with the same MAC address as the VRRP virtual MAC address, all of the associated VRRP failover groups priorities have to be the same, to provide consistent master or backup status. The MAC replication configuration mentioned above, in addition to having no support for a VND level IP address, prevents the user from switching VRRP master/backup and backup/master statuses between chassis and VAP groups.
148
VRRP Configuration
To prevent the VRRP backup circuit from responding to automatic ARP requests, legacy style VRRP should be used. Legacy VRRP uses the IP address at the circuit level. There is no concept of virtual and interface level IP addresses; only the virtual IP (VIP) address. When the circuit is in backup, the VND remains down, not populating the routing table or responding to ARP/IP requests. The user also has the option of configuring both a circuit level IP address and a VIP. However, in this scenario the circuit level ip address should not be defined, nor should the VRRP backup-stay-up command be used. When configuring legacy VRRP the IP is no longer defined at the circuit level, but at the VRRP VAP group level as an IP address (not a virtual-ip.). virtual-router vrrp-id 2 circuit outside vap-group fw ip 192.168.102.1/24 192.168.102.255 After the legacy VRRP option has been defined, allowing the addition of more IP addresses, you can create more VRRP failover groups and define additional virtual-ip addresses. Both the legacy and virtual (sub interface) circuits will remain down when VRRP is in backup. From the CLI: vrrp failover-group fw failover-group-id 1 virtual-router vrrp-id 2 circuit outside vap-group fw ip 192.168.102.1/24 192.168.102.255 vrrp failover-group fw2 failover-group-id 2 virtual-router vrrp-id 3 circuit outside vap-group fw virtual-ip 192.168.102.2/24 192.168.102.255 One drawback to this type of configuration is not being able to use the VRRP next hop health check. VRRP next hop health check uses the circuit level IP address to check the availability of a pre-configured gateway. If the IP address is not reachable, a priority-delta is deducted.
149
Dynamic Routing, VRRP, and NAT Circuits

There are several ways to configure dynamic routing on the X-Series Platform: one example is to configure OSPF on the same IP stack as VRRP, with OSPF configured at the circuit level, and VRRP using a sub circuit or virtual router. This does not work with NAT and Automatic ARP. The most effective method is to keep VRRP and dynamic routing separate. When designing a solution with OSPF, VRRP, and automatic ARP, VRRP circuits should be defined as OSPF passive. Or they should be configured as an AS external and different circuits used for OSPF. When VRRP migrates the OSPF process will send an update LSA to its adjacent devices advertising the stub networks availability. With OSPF running on different circuits, the update by the VRRP master and backup chassis would be immediate. The worst circumstance is the VRRP master chassis dies so hello packets are no longer received from the original master, and the VRRP backup chassis switches VRRP from backup to master. In this case the OSPF transitive paths have already been calculated because these circuits had previously formed adjacency, now the second chassis is the only one advertising the VRRP circuit network. To prevent the earlier automatic ARP issue, the circuit level IP address cannot be active, this prevents the user from configuring OSPF, VRRP and enabling automatic ARP for NAT on the same circuit(s). As described above, you can define OSPF on an alternative transitive pair of circuits and VRRP in legacy mode on another circuit, enabling the user to have all three working together on the same firewall.
150
13
Configuring Secure Flow Processing
You can configure the X-Series system to run multiple applications in series and in parallel. This chapter includes the following major topics:
z z z z z
What Is Secure Flow Processing on page 151 Serial Traffic and Circuits on page 151 Managing Serialized VAPs on page 153 Secure Flow Processing on Layer 3 on page 154 Parallel Traffic Flow on page 156
What Is Secure Flow Processing

One aspect of Secure Flow Processing is the concept of configuring VAP groups in series and parallel to more efficiently pass traffic. Serializing data traffic through multiple applications allows you to inspect incoming traffic at multiple levels. There are two significant advantages to serializing applications on the Crossbeam X-Series Platform:
z z
The ability to move traffic / data between VAP groups within the X-Series Platform. There is no requirement of a physical interface to pass traffic between the APMs. Once IP traffic has been classified by the NPM, serialized traffic uses the speed of the data plane to move between APMs; it is not limited to the speed of the physical interface.
Serialization provides the ability to configure and link multiple VAP groups running different applications. These specific applications are linked in series, allowing traffic to be inspected by each application. Multiple VAP groups are created (one per application) with circuits configured between the VAP groups. Traffic is passed from one VAP to the next, allowing for multi-layered, in-depth inspection by best-of-breed security applications within a single device. Previous versions of XOS that supported serialization were restricted to certain applications working on a Layer 3 routing level. XOS provides numerous enhancements to allow secure flow processing by bridging Layer 2 applications linked to Layer 3 applications. For detailed serialization configuration information, refer to the Multi-Application Serialization Configuration Guide.
Serial Traffic and Circuits

To create a virtual connection between two or more VAPs, a circuit is configured on two VAP groups, and assigned to an interface-internal. In a serial configuration, traffic is directed through one application, across the circuit, and then through a second application. XOS exposes these internal interfaces and virtual circuits to the CLI, allowing users to to configure serial connections from Layer 2 to Layer 3, and across Layer 3 applications. Additionally, user access to the internal interfaces and virtual circuits allows the serialized configuration to be included in a VRRP / Dual Box High Availability configuration, and in the case of module or interface failures, failed over to a back up system. For information about configuring VRRP and failover groups, refer to the Multi-System High Availability Configuration Guide.
151
IPv6 addressing is supported on serialized configurations, with an IPv6-enabled VAP group. This configuration can include VLANs and virtual IP addresses. IPv6 is not supported on the management network.
Configuring a Serial Circuit

Serial circuits between VAP groups are created in the same manner as regular circuits. The circuit is attached to two VAP groups, and later assigned to an interface-internal. The following configuration example shows the circuit between connecting two Layer Three VAP groups, and then defined as part of an interface-internal. circuit between device-name between vap-group fw ip 9.0.0.1/24 vap-group ips ip 9.0.0.2/24 interface-internal between logical-all between circuit between The following is an example of IPv6 addressing: circuit between device-name between vap-group fw ip 2002:1:F:0:0:0:0:F5/32 vap-group ips ip 2002:1:F:0:0:0:0:E9/32 interface-internal between logical-all between circuit between To configure serialization between a Layer 2 and a Layer 3 application, the serial circuit is configured as part of the Layer 2 bridge. The configuration is slightly different from the Layer 3 configuration, since the Layer 2 VAP group does not have an IP address, and must be in promiscuous mode. The following example assumes that the two VAP groups (IPS and FW) have already been configured, and shows only the high level steps for configuring serialization. For detailed procedures, refer to the Multi-Application Serialization Configuration Guide. 1. Configure the circuits.

LAN (on the client subnet side) Br1 (the layer two bridge) Ser1 (the serial circuit between the VAP groups) WAN (connected to an external network) Br1 (bridge-mode transparent) LAN (physical interface) WAN (physical interface) Interface-Internal (the interface that connects the circuit internally to the VAP groups and the bridge)
2. 3.
Configure Bridge Mode
Configure Interfaces

152
A sample Layer 2 to Layer 3 serialization configuration: circuit LAN device-name LAN vap-group IPS promiscuous-mode active circuit Br1 device-name Br1 vap-group IPS circuit Ser1 device-name Ser1 vap-group IPS promiscuous-mode active vap-group FW circuit WAN device-name WAN vap-group FW ip 172.16.20.7/24 # bridge-mode Br1 transparent circuit LAN circuit Ser1 # interface gigabitethernet 1/1 logical LAN circuit LAN interface gigabitethernet 2/1 logical WAN circuit WAN interface-internal Br1 logical-all Ser1 circuit Ser1
Managing Serialized VAPs

Managing of multiple VAPs is often done using individual physical connections to the modules. With serialized applications it is more efficient to manage both VAP Groups using a single physical interface. To accomplish this, you only need to assign both VAP Groups to the same circuit. circuit mgmt device-name mgmt vap-group fw ip 192.168.0.1/24 increment-per-vap 192.168.0.3 vap-group ips ip 192.168.0.6/24 increment-per-vap 192.168.0.8 When configuring the management IP addresses it is recommended to leave some unused IP addresses to allow for future growth. In the example above, the addresses 192.168.0.4 and 192.168.0.5 are left unassigned, thus allowing for additional VAPs to be added to the fw VAP group.
153
Secure Flow Processing on Layer 3

The following example explains a simple serial traffic flow, and provides basic circuit and interface configuration steps. For detailed steps to configure a basic serialized topology, please refer to the Multi-Application Serialization Configuration Guide.
Figure 12.
Example Layer 3 Serialization Topology
1.
Create individual VAP groups. vap-group L3_VAP_1 vap-count 2 max-load-count 2 ap-list ap1 ap2 ip-flow-rule L3_default action load-balance activate # vap-group L3_VAP_2 vap-count 1 max-load-count 1 ap-list ap3 ip-flow-rule L3_default_2 action load-balance activate #
2.
Create a circuit for incoming traffic. circuit incoming device-name incoming vap-group L3_VAP_1 ip 10.0.0.1/24
154
3.
Create a circuit between VAP groups, and associate the circuit with both VAP groups. circuit l2l3 device-name l2l3 vap-group L3_VAP_1 ip 5.5.5.1/24 vap-group L3_VAP_2 ip 5.5.5.2/24
4.
Create a circuit for outgoing traffic, and associate it with the VAP group it will be leaving through. circuit outside device-name outside vap-group L3_VAP_2 ip 20.0.0.1/24
5.
Define an interface-internal and assign the serial circuit l2l3. interface-internal l2l3 logical-all l2l3 circuit l2l3
6.
Configure the route between VAPS config ip route 20.0.0.1/24 ip 5.5.5.2 vap-group L3_VAP_1 config ip route 10.0.0.1/24 ip 5.5.5.1 vap-group L3_VAP_2
7.
Create and configure management circuit. circuit mgmt device-name mgmt vap-group L3_VAP_1 ip 192.168.0.1/24 increment-per-vap 192.168.0.3 vap-group L3_VAP_2 ip 192.168.0.6/24 increment-per-vap 192.168.0.8
8.
Create the interfaces for each circuit. interface gigabitethernet 1/1 logical incoming circuit incoming interface gigabitethernet 2/1 logical outside circuit outside interface gigabitethernet 1/8 logical management circuit mgmt This will create a unique IP address on each VAP in the group, which is required for a management circuit. The circuit must have the increment-per-vap parameter, even if the VAP group contains only one VAP. Once the VAP groups, circuits, management circuits, and all interfaces are configured, you can install the applications on each VAP. Refer to your application documentation for installation information.
155
Parallel Traffic Flow

Parallel traffic flow copies traffic flow to one or more passive applications, simultaneously. This flow is achieved by mapping a circuit from the physical interface to multiple VAP groups. IP flow rules at the VAP level can be used to move traffic simultaneously through multiple security engines, such as a Layer 3 firewall and IDS. IPv6 addressing is supported on parallel configurations, with an IPv6-enabled VAP group. This configuration can include VLANs and virtual IP addresses. IPv6 is not supported on the management network.
Figure 13.
Example Parallelization Topology
1.
First, create individual VAP groups. vap-group L3_FW vap-count 2 max-load-count 2 ap-list ap1 ap2 load-balance-vap-list 4 5 6 7 8 9 10 1 2 3 ip-flow-rule L3_default action load-balance activate # vap-group IDS_VAP vap-count 1 max-load-count 1 ap-list ap3 load-balance-vap-list 4 5 6 7 8 9 10 1 2 3 ip-flow-rule L3_default_2 action load-balance activate #
156
2.
Next create a circuit for incoming traffic. circuit incoming device-name incoming
3.
The following example assigns a circuit to two VAP groups to parallelize traffic. Traffic coming into the circuit goes to both VAP groups simultaneously. configure circuit incoming vap-group L3_FW ip 10.10.10.20/24 vap-group IDS_VAP promiscuous-mode
4.
Create a circuit for outgoing traffic, and associate it with the VAP group it will be leaving through. circuit outgoing device-name outgoing vap-group L3_FW ip 5.5.5.15/24
5.
Create and configure management circuit. circuit mgmt device-name mgmt vap-group L3_FW ip 192.168.0.1/24 increment-per-vap 192.168.0.3 vap-group IDS_VAP ip 192.168.0.6/24 increment-per-vap 192.168.0.8
6.
Associate virtual circuits with physical interfaces. Interface gigabit 1/1 logical incoming circuit incoming interface gigabit 1/2 logical outgoing circuit outgoing interface gigabit 1/3 logical mgmt circuit mgmt
7.
Once the VAP groups, circuits, and management circuits are configured, you can install the applications on each VAP. Refer to your application documentation for installation information.
157
158
14
Configuring RMON and SNMP Traps
The X-Series Platform supports Remote Monitoring (RMON) Event and Alarm groups. You can configure Remote Monitoring (RMON) and simple network management protocol (SNMP) to monitor your X-Series Platform. Use RMON for industry-standard management information base (MIB) entries and use SNMP traps to implement Crossbeam MIB entries for system monitoring. The following topics are covered in this chapter:
z z z z z z z z z z z z
System Owned RMON Alarms and Events on page 159 Configuring RMON Events on page 160 Configuring RMON Alarms on page 160 Displaying RMON Alarms, Events, and Logs on page 161 Displaying Disk Monitoring Event and Alarm Entries on page 162 Configuring Simple Network Management Protocol (SNMP) on page 163 Supported SNMP Traps on page 164 SNMP Traps and Informs on page 164 Configuring Trap Destinations on page 165 Displaying the SNMP Trap Log on page 165 SNMP MIB Dependencies on page 166 Allowing SNMPv3 User Access on page 167
Refer to Crossbeam SNMP MIB Reference on page 267 for a comprehensive guide to Crossbeam MIB entries and their corresponding object identifiers (OIDs).
Configuring RMON
NOTE: RMON configuration is not supported on an offline CPM.
System Owned RMON Alarms and Events

Some RMON alarm and event entries are system owned and can not be modified or deleted by the user. The owner string for these entries reads system and their alarm index range is 65000 and above. A user defined alarm entry should use indices lower than 65000. If a system owned alarm entrys type indicates trap then a trap is sent out to all the configured trap destinations, irrespective of the community string used in that system owned alarm entry. Look for the industry standard RMON MIBs in the /usr/share/snmp/mibs directory.
159
Configuring RMON Events

Each RMON event entry includes one of the following action types:
z z z
Log and Trap Log only Trap only
Use the event action types to configure the RMON Alarm entries, and dictate the action taken by the RMON agent when an alarm entry crosses its configured threshold. After you configure RMON, save the configuration settings with copy running-config start-config so that the configuration persists across system restarts. To configure the RMON event action type, use the following commands: configure rmon event 1 trap <community-string> description trap only configure rmon event 2 log description log only owner <name> configure rmon event 3 log trap <community-string> description log and trap For example: CBS# configure rmon event 1 trap public description "trap only" CBS# configure rmon event 2 log description "log only" owner user1 CBS# configure rmon event 3 log trap public description "log and trap" NOTE: The community string cannot contain whitespace characters (space or tab).
Configuring RMON Alarms

RMON alarms provide for periodical statistical sampling of variables, and comparing them with previously configured thresholds. If the monitored variables cross the specified thresholds, RMON generates an event. For each alarm entry, you must specify the following:
z z z z z z
SNMP variable Sampling interval Delta vs. absolute value Rising and falling threshold values Rising and falling event entries Optionally the owner for the entry
To configure RMON alarms, use the following command: configure rmon alarm <number> <variable> <interval> {delta|absolute} rising-threshold <value> [<event-number>] falling-threshold <value> [<event-number>] [owner <owner>] For example: CBS# configure rmon alarm 1 ifEntry.ifInOctets.1 10 absolute rising-threshold 80 3 falling-threshold 60 3 owner user1 CBS# configure rmon alarm 2 ifOutUcastPkts.1 10 delta rising-threshold 80 3 falling-threshold 60 3 owner user1
160
Displaying RMON Alarms, Events, and Logs

To display an RMON trap or log, use the following commands: show rmon log show rmon alarms show rmon events For example: CBS# show rmon log Event Index 3 3 Log Index 1 2 Log Time 00:00:01 00:00:11 Log Description Alarm [index 1] crossed FALLING threshold. alarmValue=0, threshold=60 Alarm [index 2] crossed FALLING threshold. alarmValue=0, threshold=60
CBS# show rmon alarms Alarm No. : 65000 Interval (secs) : 60 Variable : .1.3.6.1.4.1.2021.9.1.9.1 Sample Type : Absolute Value : 28 Startup : rising Rising Threshold : 70 Falling Threshold : 60 Rising Event : 65000 Falling Event : 65001 Owner : system Entry Status : valid Alarm No. Interval (secs) Variable Sample Type Value Startup Rising Threshold Falling Threshold Rising Event Falling Event Owner Entry Status Alarm No. Interval (secs) Variable Sample Type Value Startup Rising Threshold Falling Threshold Rising Event Falling Event Owner Entry Status : : : : : : : : : : : : : : : : : : : : : : : : 65001 60 .1.3.6.1.4.1.2021.9.1.9.2 Absolute 26 rising 70 60 65000 65001 system valid 65002 60 .1.3.6.1.4.1.2021.9.1.9.3 Absolute 29 rising 70 60 65000 65001 system valid
161
Alarm No. : 65003 Interval (secs) : 60 Variable : .1.3.6.1.4.1.2021.9.1.9.4 Sample Type : Absolute Value : 17 Startup : rising Rising Threshold : 70 Falling Threshold : 60 Rising Event : 65000 Falling Event : 65001 Owner : system Entry Status : valid (4 rows) CBS# show rmon events Event Number : 65000 Event Type : Log_n_trap Community : Last Time Sent : 00:00 Owner : system Description : Disk Usage Crossed Upper Threshold Event Number Event Type Community Last Time Sent Owner Description (2 rows) : : : : : : 65001 Log_n_trap 00:00 system Disk Usage Crossed Lower Threshold
Displaying Disk Monitoring Event and Alarm Entries

The X-Series Platform tracks two system owned events for disk usage crossing the upper and lower thresholds values and four alarms (for four partitions) for monitoring disk usage on CPM. Additional information about the partitions whose disk-usage crosses these thresholds can be obtained through SNMP by reading the diskTable MIB or by using the CLI/GUI commands/views on disk usage/history. The four partitions are (using the snmpwalk dump on diskTable): dskTable.dskEntry.dskPath.1 dskTable.dskEntry.dskPath.2 dskTable.dskEntry.dskPath.3 dskTable.dskEntry.dskPath.4 = = = = / /boot /cbconfig /tftpboot = = = = /dev/hda7 /dev/hda1 /dev/hda6 /dev/hda8
dskTable.dskEntry.dskDevice.1 dskTable.dskEntry.dskDevice.2 dskTable.dskEntry.dskDevice.3 dskTable.dskEntry.dskDevice.4
The X-Series Platform also monitors the dskPercent for these four partitions using system owned RMON alarm entries. dskTable.dskEntry.dskPercent.1 dskTable.dskEntry.dskPercent.2 dskTable.dskEntry.dskPercent.3 dskTable.dskEntry.dskPercent.4 You can display RMON alarms and events, as described in Displaying RMON Alarms, Events, and Logs on page 161.
162
Configuring Simple Network Management Protocol (SNMP)

In order for the X-Series Platform to communicate with a SNMP-based network management system on your network, you must configure the SNMP settings appropriately. Access the SNMP configuration parameter in the EMS GUI from navigating to Configuration > Protocols > SNMP. The basic command in the XOS CLI
Configuring an SNMP Server

Use the configure snmp-server community command to define an SNMP community on the server consisting of this X-Series Platform (the SNMP server) and one or more SNMP management stations (the SNMP clients). The SNMP management station(s) included in the community use the specified community string as a password to read from the SNMP agent on the X-Series Platform. configure snmp-server community <community_string> {<IP_address> | {<IP_address> <subnet_mask> | <IP_address>/<0-32>}} See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command. Table 17 lists the parameters used with this command.
Table 17.
Parameter
configure snmp-server parameters

Description Community string, expressed as a text string, assigned to the SNMP community that you are defining. The SNMP management stations use the specified community string as a password to read the SNMP agent on the X-Series Platform. NOTE: The community string cannot contain whitespace characters.
<community_string>
<IP_address>
Configures the SNMP community to include the SNMP agent on the X-Series Platform (the SNMP server) and the single SNMP management station (SNMP client) with the specified IP address. Configures the SNMP community to include the SNMP agent on the X-Series Platform (the SNMP server), and the SNMP management station within the specified IP address range. You can specify the IP network and subnet mask address in CIDR format (for example, 10.15.0.0/16), or non-CIDR format (for example, 10.16.0.0 255.255.0.0).
{<IP_address> <subnet_mask> | <IP_address>/<0-32>}
163
Configuring SNMP Traps

The X-Series Platform supports SNMP Versions 1, 2c, and 3 while supporting the notification types of either Trap or Inform (Version 2c only). This section describes how to configure the X-Series Platform to capture and view traps generated by the X-Series Platform. The main topics in this section are:
z z z z z z
SNMP Traps and Informs on page 164 Supported SNMP Traps on page 164 Configuring Trap Destinations on page 165 Displaying the SNMP Trap Log on page 165 SNMP MIB Dependencies on page 166 Allowing SNMPv3 User Access on page 167
Refer to Crossbeam SNMP MIB Reference on page 267 for a list of Crossbeam-specific MIB entries and their corresponding object identifiers (OIDs). Refer to SNMP MIB Dependencies on page 166 for a list of the SNMP MIB dependencies and compile orders required by some SNMP applications. Refer to the SNMP application documentation for specific requirements.
SNMP Traps and Informs

There are two types of SNMP notification messages: SNMP traps and SNMP informs. You can configure the SNMP notification receiver host to accept one of these two types of messages. The host confirms receipt of SNMP informs, but does not confirm receipt of SNMP traps. Therefore, if you want to send critical notifications to the host, you should configure the host to receive SNMP informs. NOTE: SNMP informs are only available for use with devices that support SNMP version 2c.
Supported SNMP Traps

The XOS software supports the following SNMP traps. For a complete list of all SNMP traps, refer to the Crossbeam SNMP MIB Reference on page 267:
z z z z z z z
Cold and warm starts. Chassis temperature exceeded/normal - indicates the chassis temperature status, as measured at the upper fan tray. This is indicated as normal or exceeded. System alarm state - indicates the highest current alarm state of the X-Series Platform. Module temperature exceeded/normal - indicates the module temperature status as normal or exceeded. Module insert/remove - indicates that a module has been removed or inserted in the XOS backplane. Module state changes - indicates the various states of each module in the X-Series Platform. Valid states include: unavailable, down, initializing, up, standby, and bootwait. CPU load exceeded/normal - indicates the CPU load of the processor modules and indicates whether the CPU is seeing normal or excessive loads. This is the current value of the one minute load on the CPU module that is experiencing normal or excessive loads. Link State - indicates the operational state (up or down) of the control link. Power supply failed/recovered - indicates that a redundant power supply has either failed or been replaced. Power supply feed failed/recovered - indicates that a redundant power supply feed has either failed or become operational. Fan tray insert/remove - indicates that a fan tray has been removed or inserted in the X-Series Platform.
164
z z z z
z z z z z z z z
Fan failed and recovered - indicates the various states of each fan in the fan trays. Voltage - Indicates whether CPU and Module voltages are within or exceeding normal ranges. Module Memory Usage - indicates memory usage per module. CPU Utilization - indicates usage of all cores on the specified module. CPU Core Utilization - indicates percentage of total use of CPU core, per core, on a specified module. CPU core utilization can be tracked for APMs and CPMs. NPM Flow Table Usage - indicates total flow table usage. NPM Flow Table Protocol - indicates the flow table limit status (exceeded / met / normal) of flows by protocol. Protocols include TCP, UDP, ICMP, and Other. Module SDRAM check
Configuring Trap Destinations

Use the following to configure a trap destination:
z z z z
The SNMP Host IP Address Specify the SNMP Version (V1 or V2c) Notification Type (traps or informs) Community String to identify the trap.
To configure a trap destination, use the following command: configure snmp-server host <host_ip_addr> [traps|informs] [version 1|2c] <community-string> [udp-port <port-number>] NOTE: The community string cannot contain whitespace characters (space or tab). For example: configure snmp-server host 10.1.1.29 traps version 1 private configure snmp-server host 10.1.1.29 informs version 2c public To delete a host, use: configure no snmp-server host <host_ip_addr> <community-string> NOTE: If the host that you wish to delete currently receives informs, you must specify the informs parameter with this command.
Displaying the SNMP Trap Log

XOS maintains a rotating log of the last 100 SNMP traps issued by the X-Series Platform. A trap is counted only once even though it may have been sent to several destinations. Associated trap variables, for example, sysUpTime, as well as Date and Time are also recorded in the log. The following example shows a partial display of an SNMP trap log: CBS# show traplog Trap Description : Trap OID : sysUpTime : Time & Date : Num of variables : Variable 1 : Variable 2 : Other Variables : cbsHwModuleStatusChanged .1.3.6.1.4.1.6848.4.1.14 23:12:10 2006-11-02 23:00:44.90 1 cbsHwModuleStatus.1 = up(4)
165
Trap Description Trap OID sysUpTime Time & Date Num of variables Variable 1 Variable 2 Other Variables
: : : : : : : :
cbsHwModuleStatusChanged .1.3.6.1.4.1.6848.4.1.14 23:10:50 2006-11-02 22:59:25.09 1 cbsHwModuleStatus.1 = initializing(3)
NOTE: The sysUpTime value is the time accumulated since the SNMP agent was configured.
SNMP MIB Dependencies

Certain SNMP applications require that SNMP MIB dependencies and compile orders be defined. The following lists detail these dependencies for Crossbeam-specific MIBs (CBS-*). Base SNMP MIBs required for CBS MIBs, with dependencies:
z z z z z z z z
SNMPv2-SMI: root SNMPv2-TC: SNMPv2-SMI SNMPv2-CONF: SNMPv2-SMI SNMPv2-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF IANAifType-MIB: SNMPv2-SMI, SNMPv2-TC IF-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF, SNMPv2-MIB, IANAifType-MIB INET-ADDRESS-MIB: SNMPv2-SMI, SNMPv2-TC HOST-RESOURCES-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF, IF-MIB
CBS MIBs with dependencies:

z z z z z
CROSSBEAM-SYSTEMS-MIB: SNMPv2-SMI CBS-HARDWARE-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB CBS-MODULE-RESOURCES-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB CBS-VAPGROUP-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB CBS-VRRP-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB, INET-ADDRESS-MIB, CBS-VAPGROUP-MIB
Compile order based upon above dependencies (items on same line may be compiled in any order): 1. 2. 3. 4. 5. 6. 7. 8. SNMPv2-SMI SNMPv2-TC, SNMPv2-CONF, CROSSBEAM-SYSTEMS-MIB SNMPv2-MIB, IANAifType-MIB, INET-ADDRESS-MIB IF-MIB HOST-RESOURCES-MIB CBS-HARDWARE-MIB CBS-MODULE-RESOURCES-MIB, CBS-VAPGROUP-MIB CBS-VRRP-MIB
166
Allowing SNMPv3 User Access

You can allow SNMPv3 users to have read-only access to the X-Series Platform. To configure access: configure snmp-user <username> [no-passwords] [auth-type [md5|sha|none]] [priv-type [des|none]] [oid <access>] Where: <username> no-passwords auth-type [md5|sha|none] priv-type [des|none] oid <access> Must be a unique name. Do not prompt the user for a password. By default, the user must enter the passwords configured with auth-type and priv-type. Specifies the authentication method for the user. Choices include no authentication (none), MD5 checksum, and SHA authentication. The default is none. Use des to encrypt data or none (default) to not encrypt data. If using des, auth-type must be md5 or sha. Specifies the MIB subtree that the user can access. For example, specify iso for the whole tree, or mib-2 to limit access to just the MIB objects that are part of the mib-2 tree. Smaller portions of the MIB can be selected, such as entering interfaces to restrict access to just the interface table. The default is .iso. The following OID formats are allowed:
z z z
numeric oids, such as .1.3.6.1 fully qualified oid names, such as .iso.org.dod names directly under mib-2, such as system, interfaces, at, and ip
167
168
15
Using UNIX Commands on VAP Groups and Upgrading Applications
This chapter describes the UNIX commands used on all VAPs within a VAP group. It contains the following major topics:
z z
Executing UNIX Commands on a Designated VAP on page 169 Executing UNIX Commands on All VAPs on page 170
The /usr/os/bin/cbs_rsh tool can be used to execute any interactive UNIX command on all available VAPs in a VAP group or on a particular VAP within a VAP group.
Executing UNIX Commands on a Designated VAP

To execute UNIX commands on a designated VAP in a VAP group, use the following command: /usr/os/bin/cbs_rsh <VapGroupName> <VapIndex> This command takes you to an rsh session on the specified VAP. Once executed, you can execute any UNIX command on that VAP. The following is an example for a VAP with an index of 2 in a VAP group named vgSync. In this example, the command would be entered as follows: [root@xxxxx]# /usr/os/bin/cbs_rsh vgSync 2 Last login: Mon Mar 08 17:33:09 from primarycpm vgSync_2(xxxxx):root$ vgSync_2(xxxxx):root$ ps PID 1651 1652 1672 TTY pts/0 pts/0 pts/0 TIME 00:00:00 00:00:00 00:00:00 CMD login bash ps
vgSync_2(xxxxx):root$ vgSync_2(xxxxx):root$ exit logout rlogin: connection closed. [root@xxxxx]#
169
Executing UNIX Commands on All VAPs

To execute UNIX commands on a all VAPs within a VAP group, use the following command: /usr/os/bin/cbs_rsh <vapGroupName> This command causes the following prompt to display: vapGroupName>> Commands entered at this prompt are either specific to cbs_rsh, or executed directly on each VAP within the VAP group. Once executed, the command results are displayed. When the execution of all commands has completed you are returned to the VAP Group Name prompt. Commands specific to the cbs_rsh are as follows:
Table 18.
Commands Specific to cbs_rsh

Description Exits the cbs_rsh session. Displays help menu. Starts saving commands for those VAPs that are down. When these VAPs come back up, all the save commands are executed on that VAP. Stops saving commands for those VAPs that are down. Copies a file on the CPM to all VAPs within the VAP group. Execute previous UNIX command.
Command quit /q ? /help cbs_start_s cbs_stop_s cbs_cp !p
Copying a File from the CPM to all the VAPs in a VAP Group
To copy a file from the CPM to all the VAPs within a VAP group, use the cbs_cp command at the VAP group prompt. The following example copies the file SomeFile to the VAPs in the FW VAP group. 1. Issue the cbs_rsh command with the desired VAP group (FW in our example) to access the VAP group prompt. Help for the cbs_rsh commands is displayed along with the VAP group prompt: [root@xxxxx admin]# /usr/os/bin/cbs_rsh FW !p quit/q cbs_cp cbs_start_s cbs_stop_s FW>> 2. Enter the cbs_cp command at the VAP group prompt; the system prompts for the source of the file to be copied. Enter the source with its full path as shown in the example below. FW>>cbs_cp Source File on CPM (with full path) :/tftpboot/.private/home/admin/SomeFile 3. When prompted, enter the full destination path. Destination File on VAP (with full path) :/root/SomeFile ----------Execute previous UNIX command Exit this program Copy a file on CPM to all vaps within a vap-group Start saving commands for VAPs currently not present Stop saving commands for VAPs currently not present
Using UNIX Commands
170
4.
The system then issues a copy command for each VAP in the VAP group by appending VAP group index number to the VAP group name (in this example: FW_1, FW_2, etc).
The full example is shown below: [root@xxxxx bin]# /usr/os/bin/cbs_rsh FW !p Execute previous UNIX command quit/q Exit this program cbs_cp Copy a file on CPM to all vaps within a vap-group cbs_start_s Start saving commands for VAPs currently not present cbs_stop_s Stop saving commands for VAPs currently not present FW>>cbs_cp Source File on CPM (with full path) :/tftpboot/.private/home/admin/SomeFile Destination File on VAP (with full path) :/root/SomeFile Copying :/tftpboot/.private/home/admin/SomeFile to /tftboot/FW_1//root/ SomeFile Copying :/tftpboot/.private/home/admin/SomeFile to /tftboot/FW_2//root/ SomeFile FW>> quit [root@xxxxx admin]#
171
Using UNIX Commands
172
16
Viewing Performance, Statistics, and Status
This chapter describes how to view the XOS performance, statistics, and status. It contains the following major topics:
z z z z z z z z z
Using GEM to View Module Information and Status on page 173 Viewing Module Information and Status on page 174 Viewing Disk Usage on page 175 Viewing Heartbeat Status on page 175 Viewing Module Utilization and Load Averages on page 176 Viewing Interface Statistics on page 177 Viewing Switch Data-Path (SDP) Statistics on page 178 View VRRP Status on page 178 Viewing Flow Distribution on page 180
Using GEM to View Module Information and Status

The Greenlight Element Manager (GEM) provides a view into the components and health of your X-Series Platform. GEM is a Web application that displays system and component operational information in an easy to reference format. GEM provides system monitoring capabilites only, and does not allow you to configure your system. Individual views provide module status, connection rates, load averages, flow rates, VAP group information, DBHA and Failover Group status, application data, and more in a single glance. The tabbed interface can be customized to show specific module information or system historical data. User assistance provides additional details about the views and steps to add or customize the display for your specific needs. To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser. https://<ip_address>
173
Viewing Module Information and Status

To view the status of all modules in the chassis, use the following command: CBS# show module status The following is a partial display. This display continues for each module in the chassis. NA = Not Available, DP = Data Plane, CP = Control Plane cp = Control Processor, ap = Application Processor, np = Network Processor Slot 3: Board Type AP8650 Board Part Number 004910 Board Serial Number L829H004 Board Revision 8 Control FPGA Revision 0x600 Focus FPGA Revision 0x600 Board OCODE A000 Dual-CPU Capacity Available CPU Presence Present CPU2 Presence Not Present CPU Voltage 1.14(V) 3.3v Supply 3.31(V) 1.8v Supply 1.82(V) 1.5v Supply 1.52(V) CPU Temperature 31(C) Intake Air Temperature 28(C) FPGA Temperature 38(C) Exhaust Air Temperature 33(C) SDRAM 1 Size 1048576(KB) SDRAM 2 Size 1048576(KB) SDRAM 3 Size 0(KB) SDRAM 4 Size 0(KB) Total Memory 2073904(KB) Used Memory 430116(KB) Free Memory 1643788(KB) Shared Memory 0(KB) Buffers Memory 0(KB) Cached Memory 111604(KB) Memory Utilization 15.36% Total High Memory 262144(KB) Free High Memory 15860(KB) Total Low Memory 1811760(KB) Free Low Memory 1627928(KB) Active LED On Standby LED Off Failure LED Off Hard Disk N/A Second Hard Disk N/A Flash N/A Hard Drive Error None Second Hard Drive Error None RAID Status Not Active, RAID none Line 3/4 Up Line 3/5 Up Control Bus A Up Control Bus B Up ap2a Link Down ap2b Link Up ap2c Link Down
174
ap2d Link CPU Speed CPU Up Time Threshold Acceleration Card 1 Acceleration Card 2 Ethernet Daughter Card
Down 2327 MHz 96179 secs Not Present Not Present Not Present
EMS provides a Module Memory Usage graph under Status > System Status in the navigation tree.
Viewing Disk Usage

The X-Series Platform tracks two system events for disk usage crossing the upper and lower thresholds values and four alarms (for four partitions) for monitoring disk usage on the CPM. It also lists the 5 largest files on each VAP. This information is collected once a day. To display the disk usage history using EMS, open Status > System Status > Disk Usage History Stats in the navigation tree. To display the current disk usage using the CLI: CBS# show disk-usage To display a history of disk usage using the CLI: CBS# show disk-usage history ====================================================================== Top Disk Users Report for Thu Mar 18 04:02:03 EDT 2010 ====================================================================== Filesystem 1K-blocks /dev/sda12 11812024 /dev/sda9 102740 /dev/mapper/d2vg0-lv0 1971472 /dev/mapper/d2vg1-lv0 37353008 /dev/mapper/d2vg1-lv1 2011792 Used Available Use% Mounted on 1902472 9309528 17% / 7772 89752 8% /boot 127200 7801048 83136 1744128 27654488 1826464 7% /cbconfig 23% /tftpboot 5% /mgmt
In case of critical disk errors, you can configure the CPM to automatically go offline as follows: CBS# configure cp-action {cp1|cp2} disk-error offline
Viewing Heartbeat Status

The Heartbeat is a mechanism to determine the state of the system connections (on the control and data planes) between modules, which verifies whether these modules can communicate with one another. The Heartbeat Status reports the condition of these connections. The system processor modules (CPM, APM and NPM) use Heartbeat Status to determine the state of their connections to other processor elements in the system. The Heartbeat Status shows whether each processing element is exchanging heartbeats with another processor. The CPMs run heartbeats with all other processors over the control plane. The NPMs run heartbeats with APMs over the control plane, as well as over the data plane. APMs run heartbeats among themselves over the data plane. Eight missed heartbeats in a row causes a connection to be declared unusable.
175
To view the Heartbeat Status using EMS, open Status > System Status > Heartbeat Status in the navigation tree. For the CLI, use the following command: CBS# show heartbeat Link Quality TO: 1 FROM 1 2 ON ports CB A: NA 100% CB B: NA NA DP A: 100% 100% DP B: NA NA DP C: NA NA DP D: NA NA
3 100% NA 100% NA NA NA
4 100% NA 100% NA NA NA
5 100% NA NA NA NA NA
6 NA NA NA NA NA NA
8 100% NA 100% NA NA NA
10 100% NA 100% NA NA NA
11 100% NA 100% NA NA NA
12 NA NA NA NA NA NA
13 100% NA 100% NA NA NA
14 NA NA NA NA NA NA
This display continues for each module in the chassis.
Viewing Module Utilization and Load Averages

Module Load Averages show the CPU % load average for all APMs and CPMs in the system during the previous 1, 5, and 15 minute intervals, indicating the number of computable processes. EMS provides a Module Utilization Averages graph under Status > System Status in the navigation tree. To view the Module Load Averages from the CLI, use the following command. CBS# show cpu CPU utilization average for np1: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00 CPU utilization average for np2: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00 CPU utilization average for ap2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.11 CPU utilization average for ap4: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00 CPU utilization average for cp2: for last 1 minute: 0.98 for last 5 minutes: 1.64 for last 15 minutes: 1.22 CPU load average for np1: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00 CPU load average for np2: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00
176
CPU load average for ap2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00 CPU load average for ap4: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00 CPU load average for cp2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00 Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt ---- ------ ----- ----- ----- ----- ----- ----- ----- ----3 ap1 CPU 0.0 0.0 0.0 100.0 0.0 0.0 0.0 3 ap1 CPU0 0.0 0.0 0.0 100.0 0.1 0.1 0.0 3 ap1 CPU1 0.0 0.0 0.0 100.0 0.0 0.0 0.0
Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt ---- ------ ----- ----- ----- ----- ----- ----- ----- ----4 ap2 CPU 0.0 0.0 0.0 100.0 0.0 0.0 0.0 4 ap2 CPU0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 4 ap2 CPU1 0.0 0.0 0.0 100.0 0.0 0.0 0.0 Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt ---- ------ ----- ----- ----- ----- ----- ----- ----- ----13 cp1 CPU 0.1 0.0 0.0 0.0 0.0 0.1 98.4
Viewing Interface Statistics

View the statistics for the interfaces on the CPM or NPM as follows: CBS# show interface detail Gigabitethernet 6/1 is up Interface is in use Hardware address is 00:03:d2:2f:52:7a MTU 1500 bytes, BW 100 Mbits, full-duplex, auto-negotiation is enabled Last clearing of "show interface" counters never PHY stats: Statistics on physical line Received: Total frames 53610 (bytes 42983430) Broadcast frames 0 Undersized frames 0 Oversized frames 0 Throttles 0 Total errors 0 Frame check sequence (FCS) errors 0 Frame errors 0 Overrun errors 0 Ignored errors 0 Transmitted: Total frames 17662 (bytes 1594894) Underrun errors 0 Total errors 0 Collisions 0
177
Viewing Switch Data-Path (SDP) Statistics

SDP Statistics are the statistics for each APM and CPM. View SDP statistics as follows: CBS# show switch-data-path In Slot Mod SDPx Packets 10 VAP 0 268708390 10 VAP 1 251429612 10 VAP 2 891383778 10 VAP 3 209923713 10 VAP 4 0 10 VAP 5 0 10 VAP 6 0 10 VAP 7 0 10 VAP total 1621445493 Slot mod SDPx In Packets In Errors In Drops Out Packets Out Errors Out Drops In Errors 0 0 266 0 0 0 0 0 266 In Drops 0 0 266 0 0 0 0 0 266 Out Packets 307895769 293391354 5438364 6220223 0 0 0 0 612945710 Out Errors 0 0 0 0 0 0 0 0 0 Out Drops 0 0 0 0 0 0 0 0 0
Physical slot number Module type Switch data path number Number of packets received Number of packets received with errors Number of packets dropped while receiving packets Number of packets sent Number of errors occurring during transmission Number of drops occurring during transmission
To clear these counters, use the following command: clear switch-data-path {all|module <low> [<high>]} Where: all <low> <high> Clears the counters from all modules. A specific module. Valid values are 1 to 14. If high is not specified, this is the only module. The low parameter specifies the first module (lowest number) and high specifies the last module (highest number). All modules in the range are included. Valid values are 1 to 14.
View VRRP Status

If you have two or more X-Series Platforms in a VRRP configuration, you can determine the state of VRRP on an individual system with the following commands:
z z z
show vrrp status [<failover-group-id>] show vrrp detail-status [<failover-group-id>] show-remote-box
See Configuring Multi-System High Availability on page 129 for details on configuring VRRP.
show vrrp status

The following is an example of this command.
Viewing Performance, Statistics, and Status 178
CBS# show vrrp status Priority is Actual/Configured FG-ID Priority Status Preempt 2 200/200 Master on 5 0/100 Down off (2 rows)
Master Sys ID 222 1
Master Priority 200
The Status column displays the state of the failover group, which can be:
z z z z z
backup Failover group is in backup mode. down Failover group is not functioning. init Failover group is initializing. master Failover group is in master mode. unknown The state of the failover group cannot be determined.
The Priority column displays the failover groups current priority / configured VRRP priority, in that order. If everything is running normally, these numbers are the same. Otherwise, one or more events caused the priority to be decremented by pre-configured priority-deltas, as described in VRRP Priority on page 130. If there is a problem, use the ID in the Sys ID column and enter the show vrrp virtual-router <id> or show vrrp circuit-ip vr-id <id> command. These displays will provide additional information to help you determine which event caused the priority to be decremented. If necessary, use the show vrrp vap-group <vap-group-name>, show vrrp verify-next-hop, and show vrrp monitor-circuit commands to find the problem. If using EMS, you can access the VRRP status by opening Status > Network Status > VRRP Stats in the navigation tree.
show vrrp detail-status

The following is an example of this command. CBS# show vrrp detail-status Execute show vrrp detail-status-help to see help for this command. FG_ID Status 200 Master 200 Master 200 Master 200 Master 200 Master (5 rows) CBS# Priority 198/198 198/198 198/198 198/198 198/198 Delta 1 1 1 1 50 Type vr vr mi mi vg Component ymlt/20 ser/10 gigabitethernet 4/4 gigabitethernet 2/4 vsx
Show Remote Box

Use the show-remote-box command to view the path and path status for each of the interconnected ports on the two systems. CBS# show remote-box System ID : 45 First IP Address : 1.1.45.20 Second IP Address : 192.168.50.45 Third IP Address : 192.168.51.55 Fourth IP Address : 0.0.0.0 Fifth IP Address : 0.0.0.0 Active IP Address : 1.1.45.20 (1 row)
179
Viewing Flow Distribution

Use the following commands to monitor traffic flows from the NPMs to VAPs on the APMs:
z z z
show flow active show flow distribution show flow-path active
show flow active

The show flow active command displays information currently stored in the Active Flow Table (AFT) on the NPMs running on the X-Series Platform. Use this command to determine whether flows are arriving on the NPMs, and if they are being dropped or sent to the appropriate VAP groups. Use the verbose parameter to change the format that the CLI displays the output of the command, and to display additional information about each active flow. The show flow active command is a snapshot of the current state of the active flows when the command is issued. Information is not updated when the state of an existing flow changes or when a new flow arrives on the X-Series Platform. Use the poll parameter to continuously poll the NPMs and display updated information at regular intervals. Use this parameter to continuously monitor traffic over time, observing changes in the states of existing flows and obtaining information about new flows that arrive on the X-Series Platform. NOTE: Press Ctrl-y to stop updating the command output and return to the CLI prompt. By default, this command displays information about all active flows. Use one or more of the following parameters to filter the command output and display specific information:
z z z z z z z z
source-address destination-address source-port destination-port protocol domain circuit-id module
z z z z z z z z
master-npm fast-path-only verbose poll sort validated validation-pending no-validation
By default, this command lists the active flows in the order in which they appear in the AFT. Use the sort parameter to sort the list of flows.
180
Syntax
show flow active [source-address {<IP_address> | <lowest_IP_address> <highest_IP_address>} ] [destination-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [source-port {<port_number> | <lowest_port_number> <highest_port_number>}] [destination-port {<port_number> | <lowest_port_number> <highest_port_number>}] [protocol {<protocol_number> | <lowest_protocol_number> <highest_protocol_number>}] [domain {<domain_ID_number> | <lowest_domain_ID_number> <highest_domain_ID_number>}] [circuit-id {<circuit_ID_number> | <lowest_circuit_ID_number> <highest_circuit_ID_number>}] [module {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [master-npm {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [fast-path-only] [verbose] [poll <polling_interval>] [sort] [validated] [validation-pending] [no-validation]
Default Output
By default, the show flow active command displays information in a table, using the following format: . Module <NPM_or_VAP_name1> Source <IP>:<port> Destination <IP>:<port> Prot <#> Dom TTI/MAX <ID#> <mm:ss>/<mm:ss> rx packets <#>
Modules <VAP_name1>, <VAP_name2> ... rx circuit <ID#> Master <NPM_name> <NPM_or_VAP_name2> <IP>:<port> <IP>:<port>
Fast Path {Y|N} <#>
<ID#> <mm:ss>/<mm:ss> rx packets <#>
{Re-routing | Drop(<drop_reason_ID>)} rx circuit <ID#> Master <NPM_name>
Fast Path {Y|N}
The format shown in the entry for <NPM_or_VAP_name1> is used if the flow is received by an NPM or VAP and then transferred to a VAP for processing. The format shown in the entry for <NPM_or_VAP_name2> is used if the flow is received by an NPM or VAP and then rerouted to an external system or dropped. For detailed information about this command and parameters, refer to the XOS Command Reference Guide.
show flow distribution

The show flow distribution command displays the number of flows that each Network Processor Module (NPM) has assigned to each virtual application processor (VAP) in each VAP group, and displays the rates at which each NPM assigns new and existing flows to each VAP. The following table describes the information provided in each column/row. Column/Row Heading NP Information Provided Name of the NPM. Use the show chassis command to display the NPM names assigned to the NPMs installed in your X-Series Platform.
181
Column/Row Heading Uptime
Information Provided Amount of time that the NPM has been in the UP state, in days, hours, and minutes. Hours and minutes are expressed in the format: mm:ss. For example, 9 hours and 7 minutes is 09:07. Slot number for the APM assigned to the VAP to which the NPM assigns flows. Name of the VAP to which the NPM assigns flows. A VAP name has the format: <VAP_group_name>_<VAP_Index_Number> Use the show ap-vap-mapping command to display the index numbers assigned to the VAPs in each VAP group configured on the X-Series Platform.
Slot VAP
New Flows Rate Aged Flows Rate Flows
The change in the number of new flows assigned to the VAP, since the last second. The change in the number of existing flows assigned to the VAP, since the last second. Total number of flows currently assigned to the VAP.
There are two parameters for sorting the show flow distribution output:
z z
sort vap-group sorts flows by vap-group name, then index-in-group, then apm slot sort apm-slot sorts flows by apm-slot, then vap-group name, then index-in-group
The following is an example: CBS# show flow distribution New Flows Rate ========== 0 7344 0 6340 0 5733 0 5223 Aged Flows Rate ========== 0 5133 0 5538 0 5670 0 5041 Flows ========= 0 72043 0 79002 0 69182 0 68423
NP np1 np2 np3 np4 np1 np2 np3 np4
0 3 0 3 0 3 0 3
Uptime days, 00:00 days, 19:17 days, 00:00 days, 19:17 days, 00:00 days, 19:17 days, 00:00 days, 19:17
Slot 7 7 7 7 8 8 8 8
VAP testvapgroup_1 testvapgroup_1 testvapgroup_1 testvapgroup_1 testvapgroup_2 testvapgroup_2 testvapgroup_2 testvapgroup_2
show flow-path active

The show flow-path active command displays the flow paths for the active flows that the X-Series Platform is currently processing. A flow path is the path that a flow takes when it goes through the X-Series Platform. A flow path has the following basic elements:
z z z z z
Flow classification information source and destination IP addresses, source and destination port numbers, protocol number, and domain ID number Network Processor Module (NPM) on which the active flow enters the X-Series Platform Circuit(s) on which the active flow enters an active virtual application processor (VAP) group(s) Active VAP(s) that processes the flow NPM interface on which the active flow leaves the X-Series Platform
182
By default, this command displays information only about the initial entry path and the final egress NPM interface for each flow. That is, the command displays information only about the NPM, circuit, and VAP on which the flow first enters the X-Series Platform and the NPM interface on which the flow exits the X-Series Platform. Use the show flow-path active command output to determine whether NPMs are dropping flows when they arrive on the X-Series Platform, to make sure traffic is successfully passing through the X-Series Platform, and to determine where the NPM sends flows that it does not drop. Use the verbose parameter to display information about the full path for each active flow, from the ingress NPM to the egress NPM interface. Use this parameter to monitor flows when:
z z
The flows pass through more than one VAP group configured on the X-Series Platform. The flows pass through a VAP group that is configured with separate circuits and NPM interfaces for ingress and egress traffic.
The show flow-path active command is a snapshot of the current state of the active flows when the command is issued. Information is not updated when the state of an existing flow changes or when a new flow arrives on the X-Series Platform. Use the poll parameter to continuously poll the NPMs and display updated information at regular intervals. Use this parameter to continuously monitor traffic over time, observing changes in the states of existing flows and obtaining flow path information for new flows that arrive on the X-Series Platform. NOTE: Press Ctrl-y to stop updating the command output and return to the CLI prompt. By default, this command displays flow path information for all active flows. You can use one or more of the following parameters to filter the command output to display flow path information only for active flows that match the criteria that you specify with the parameters:
z z z z z
source-address destination-address source-port destination-port protocol
z z z z z
domain circuit-id module master-npm fast-path-only
By default, this command lists the active flows in the order in which they appear in the AFT.
Syntax
show flow-path active [verbose] [source-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [destination-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [source-port {<port_number> | <lowest_port_number> <highest_port_number>}] [destination-port {<port_number> | <lowest_port_number> <highest_port_number>}] [protocol {<protocol_number> | <lowest_protocol_number> <highest_protocol_number>}] [domain {<domain_ID_number> | <lowest_domain_ID_number> <highest_domain_ID_number>}] [circuit-id {<circuit_ID_number> | <lowest_circuit_ID_number> <highest_circuit_ID_number>}] [module {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [master-npm {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [fast-path-only]
183
Default Output
By default, the show flow-path active command displays information in a table, using the following format: Module <NPM_name1> Source:port <IP>:<port> Destination:port <IP>:<port> Prot <#> Dom <ID#>
rx circuit <ID#> rx active <VAP_name> [rx passive] tx_port <slot>_<port> master <NPM_name> <NPM_name2> <IP>:<port> <IP>:<port> <#> <ID#>
rx circuit <ID#> master <NPM_name>
Drop(<drop_reason>)
The output has the format shown in the entry for <NPM_name1> if the originating NPM transfers the flow to a VAP for processing. The output has the format shown in the entry for <NPM_name2> if the originating NPM drops the flow. The following example displays the initial entry path and the egress NPM interface for every active flow that the X-Series Platform is currently processing. In this example, a VAP group called testvapgroup is currently configured on the X-Series Platform and is running a firewall application. CBS# show flow-path active This command may take a few minutes. Do you want to continue? <Y or N> [Y]: Y Module Source:port Destination:port Prot Dom np2 172.16.10.100:2009 172.16.20.240:80 6 1 rx circuit 1027 rx active testvapgroup_2 tx_port 4_2 master np2 np4 172.16.20.240:80 rx passive
172.16.10.144:53814 rx passive
rx circuit 1028 rx active testvapgroup_1 tx_port 2_2 master np2 np2 rx circuit 1029 master np2 CBS# 172.16.10.207:31754 Drop(PS2IDX failed)
172.16.20.240:80
For additional detailed information about the parameters used with this command, as well as additional examples, refer to the XOS Command Reference Guide.
184
17
Viewing Alarms, Events, and Logs
This chapter describes how to view the X-Series Platform message and audit logs. It contains the following major topics:
z z z z
Viewing Alarms on page 185 Configuring a Remote Syslog Server on page 190 Viewing Message Logs on page 190 Viewing Audit Logs on page 192
Viewing Alarms
XOS records alarm data and the Alarms Manager provides the data to GEM, the CLI, SNMP, and system log files for end user consumption. GEM provides extensive alarm information in multiple views in the user interface including the Alarms Panel, the Alarm Details view, and the XOS Alarms Model Information Pages. The CLI provides several options under the show alarms command, providing the same level of granularity as the GEM Alarm views. The Message Log maintains a history of software events that have occurred on the XOS console. This historical log can be searched and the information displayed as well. SNMP traps and MIB tools can be configured to output system alarms into third party tools.
Figure 14.
XOS Alarm Notification

XOS provides consistent alarm notifications through a variety of interfaces.
185
Using GEM
In GEM, the Alarms Panel displays information about each of the alarms logged on the X-Series Platform. The list includes active alarms and optionally, historical alarms. To show historical alarms, check the Include Historical Alarms check box. The alarm list can be filtered to show all alarms, or only alarms related to Dual-box high availability (DBHA). Click the Reset Filter button to remove the filter from the alarm list. The Alarms Panel displays the columns listed below by default, and is updated when a new alarm is generated. To sort a column, click on the header.
z z z z z
ID: The number assigned to the alarm by XOS. Date: Date and Time of alarm. Severity: Critical, Major, Minor, Info, or Clear (indicates the alarm has been cleared). Source: The location where the alarm occurred. The location can be a module or an assembly. Description: A brief description of the alarm.
Colors highlight the alarm severity; red indicates a critical alarm, orange indicates a major alarm, yellow indicates a minor alarm, green indicates a cleared alarm, and gray indicates an informational alarm. To mark one or more alarms as read, right-click the alarm and select Mark as Read. For additional details on an alarm, double-click the alarm to open the Alarm Details view. Additional columns are available by clicking the down arrow in the a column header and selecting from the secondary list. These columns display information that may be specific to certain types of alarms. For more information, refer to the GEM user assistance.
Alarm Details
The Alarm Details window displays additional information to support the data shown in the Alarms panel, including a probable cause and suggested repair actions. The Probable Cause is a starting point for investigating the alarm, and the information should be used with the Suggested Repair Actions to resolve the issue. Repair information may include how to clear the alarm, CLI commands to gather more information about the alarm conditions, and suggestions to prevent the alarm from occurring again.
Clearing Alarms
A small set of informational alarms are clearable using the XOS CLI. These are listed in the optional User Clearable column. Other alarms can only be cleared through user action, such as a configuration change or rebooting a module. For alarm repair actions, refer to the Suggested Repair information in the Alarm Details view. Double clicking any alarm in the Alarms View will display the Alarm Details view.
186
Reviewing Alarm Information

XOS and GEM provide information about the current and historical alarms on the X-Series Platform. An additional resource for alarm information is the XOS Alarm Model, which is a complete list of all the alarms implemented in XOS. The Alarm Model can also be viewed by using the show alarm model command from the Command Line Interface.
Historical Data
Displaying historical data allows you to look back over a period of time, up to and including the current GEM session. It provides perspective on current conditions, and can help identify trends or events that resulted in an alarm.
Using the CLI

Use the CLI to display currently active alarms, or an alarm history that includes both active and past alarms. Use the active parameter to display currently active alarms. Use the history parameter to display all active and past alarms. Use the model parameter to display the XOS alarms model. The alarms model provides detailed information about every alarm that can be raised on an X-Series platform. CBS# show alarms active CBS# show alarms history CBS# show alarms model When you use the show alarms active or show alarms history command, you can use a number of additional parameters to filter the output by severity, source, id, and date/time. You can add the verbose parameter to view additional detail, including suggested repair actions. No additional parameters are available for show alarms model.
Show Alarms Active

The active parameter displays an active alarm status summary. Specify one or more of the minor, major, or critical filters to display the active alarm summary, followed by a list of the conditions that triggered each severity of active alarm. Use the following CLI command to view active alarms: CBS# show alarms active Active Alarms Summary: Source -----cp2 np1 uprfan Total Critical -------1 0 0 1 Major ----3 0 1 4 Minor ----0 2 0 2
XOS displays a summary of all alarm levels, showing the module and the number of alarms for each alarm severity. View specific alarm information by including the alarm category in the show alarm command, as follows: CBS# show alarms active major Active Alarms Summary: Source -----cp2 np1 uprfan Total Critical -------1 0 0 1 Major ----3 0 1 4 Minor ----0 2 0 2
187
* indicates an alarm that can be cleared with the 'clear alarms' CLI command Major: ID -96 *95 94 69 Date ---Nov 11 Nov 11 Nov 11 Nov 10 Source -----cp2 cp2 cp2 uprfan Description ----------Failover group xyz priority 49 Failover group xyz status master No remote box configured Fan tray mismatch
16:02:43 15:34:16 15:34:10 09:18:23
XOS displays the alarm status summary for each module. The alarm status summary is followed by information for each active alarm in the category specified. Use the verbose parameter to display complete alarm information, including suggested repair actions: CBS# show alarms active major verbose Active Alarms Summary: Source -----cp2 np1 uprfan Total Critical -------1 0 0 1 Major ----2 0 1 3 Minor ----0 2 0 2
Major: ================================================================================ Alarm Id : 94 Brief Description : No remote box configured Date : Thu Nov 11 15:34:10 2010 Severity : Major Alarm Name : noRemoteBoxConfigured Alarm Source : cp2 Slot : slot 7 Module : cp2 Information : Probable Cause : Configuration Or Customization Error Event Type : other User Clearable : false Extended Description : VRRP is configured but no remote box is configured, : which could indicate an incomplete configuration. Repair Action : If you have VRRP configured, you must configure at : least one remote box. Either add a remote box or : delete the VRRP configuration. -------------------------------------------------------------------------------You can further filter the data by source, ID, or date and time by using additional parameters. See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command.
Show Alarms History

The history parameter displays a list of the last 1000 alarms on the system, active and historical. The most recent alarms appear first. Use the following command to view all alarms, active and historical: CBS# show alarms history
Viewing Alarms, Events, and Logs 188
ID -425 424 423 422 421 420
Date ---Oct Oct Oct Oct Oct Oct
7 7 7 7 7 7
17:20:10 17:01:23 11:01:20 10:44:59 10:43:54 10:35:59
Severity -------Minor Clear Minor Info Major Minor
Source -----bay1 pwrA cp1 system lwrfan np1
Description ----------Power supply missing Power supply failure Firmware mismatch slot: 1,5,13 New system alarm level (major) Fan tray mismatch Flow table median threshold(tcp)
XOS displays a summary of all alarms, starting with the most recent. Use additional parameters to filter the output of the show alarms history command by ID, severity, source, and date. Use the verbose parameter to display complete alarm information, including suggested repair actions. See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command.
Show Alarms Model

The Alarms Model provides detailed information about all alarms that can be generated on the XOS Platform, including an extended description and suggested repair actions. This information is not related to the currently active or historical alarms on your X-Series Platform. It is a list of all the alarms that can be generated by the X-Series Platform, and is provided for informational purposes. Use the following command to view the Alarms Model: CBS# show alarms model XOS displays the alarms model in the following format: CBS# show alarms model Alarm Name Managed Objects : : : Parameters : : : : : : : : : Information : Default Severity : Probable Cause : Event Type : User Clearable : Brief Format : Extended Description : : Repair Action : : : Targets : : : applicationDown Slot Module APP CPM Host Name APP CPM IP Address APP Name APP New State APP Old State APP Release APP VAP Group Name APP VAP Index APP Version minor Out Of Service Processing Error Alarm false Application down The application running on the specified APM is down. Verify the state of the application on the APM (use GEM System view). Restart the application if necessary. GEM LOG SNMP
189
User Clearable Alarms

A small number of alarms can be cleared from the CLI by using the clear alarms command. Active alarms that can be cleared by using the clear alarms command are indicated by an asterisk in the ID column of the output of the show alarms active command when used with one or more parameters. NOTE: You must use at least one parameter (e.g. critical, major, or minor) in the show alarms active command to see this output. Otherwise the show alarms active command displays only the summary table. In this example, the alarm with ID 95 is user-clearable. ID -96 *95 94 69 Date ---Nov 11 Nov 11 Nov 11 Nov 10 Source -----cp2 cp2 cp2 uprfan Description ----------Failover group xyz priority 49 Failover group xyz status master No remote box configured Fan tray mismatch
16:02:43 15:34:16 15:34:10 09:18:23
Cleared alarms remain in the alarms history, and can be viewed by executing the show alarms history CLI command. Use the clear-alarms command to clear these alarms. You can clear all alarms by using the all parameter, or you can specify an alarm or group of alarms by using the id parameter. The following command clears the user-clearable alarm with ID 95. CBS# clear alarms id 95 CBS# When you clear an alarm, a new alarm appears in the alarms history with a severity of Clear. The cleared alarm (ID 95) remains in the alarm history. See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command. NOTE: The system also clears alarms automatically, either because the condition that generated the alarm no longer exists, or because the alarm has been superseded by a later alarm.
Configuring a Remote Syslog Server

The X-Series Platform supports a remote logging server. To configure the remote server, enter the following: CBS# configure logging server {<hostname>|<ipaddress>} To view any configured remote logging servers: show logging server
Viewing Message Logs

The Message Log maintains a history of software events that have occurred on the XOS console. Use it to store a record of system events or to diagnose operational problems. Log files are stored in the /var/log directory of the CPM system. All APMs and NPMs are configured to log their events to this location. Log file rotation can occur on a periodic basis or when the log file reaches a specified size. Logs are rotated by saving the current log file to an archive copy. The current log or any of the archive copies can be retrieved by a Network Management Server for long-term storage. You can also create several log files containing messages from different subsystems, or different severity levels to be directed to different files. To support proactive notification of error condition log messages, a generic method is provided to send log messages as traps. To display the Message Log:
Viewing Alarms, Events, and Logs 190
CBS# show logging console Jul 7 20:33:33 crossbeam Received msg from hm with Jul 7 20:33:33 crossbeam 512->1024 Jul 7 20:33:33 crossbeam unknown slot 3 Jul 7 20:33:33 crossbeam Received msg from hm with Jul 7 20:33:33 crossbeam Received msg from hm with Jul 7 20:33:35 crossbeam unknown slot 7 Jul 7 20:33:35 crossbeam unknown slot 3 Jul 7 20:33:35 crossbeam unknown slot 7
cbssysctrld[1777]: [W] int CHmSession::read () opcode=5 0 entries cbssysctrld[1777]: [W] HM Reallocating sbd cbssysctrld[1777]: [W] pollHiChange Poll rx'd from cbssysctrld[1777]: [W] int CHmSession::read () opcode=5 0 entries cbssysctrld[1777]: [W] int CHmSession::read () opcode=5 0 entries cbssysctrld[1777]: [W] pollHiChange Poll rx'd from cbssysctrld[1777]: [W] pollHiChange Poll rx'd from cbssysctrld[1777]: [W] pollHiChange Poll rx'd from
Console debug messages for the APMs and NPMs are disabled by default. To enable console messaging from the APM enter the Linux command: echo "6 4 1 7" > /proc/sys/kernel/printk NOTE: To make this change permanent following a reboot, edit the APM /etc/rc.local file by commenting out the line echo "1 2 1 1" > /proc/sys/kernel/printk. Reporting messages to the console may cause performance degradation. Excessive verbose messaging may cause the module to reset. Kernel messages can always be viewed in dmesg on the APM and the system log on the CPM.
Viewing Alarm Details from the Message Log

To view alarm details in the message log, you can grep the log. For example, start by grepping the log by AlarmID, then to see more detail about major alarms, grep the term "| major |". To see all of the alarms in the messages file, grep the output for AlarmID. CBS# unix su Password: [root@TechPubs-X45 admin]# [root@TechPubs-X45 admin]# grep AlarmID /var/log/messages Nov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 82 | Sat Nov 20 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status down Nov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 83 | Sat Nov 20 2010 | critical | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 0 Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 84 | Sat Nov 20 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 199 Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 85 | Sat Nov 20 2010 | major | cp2 | noRemoteBoxConfigured | No remote box configured Nov 20 15:09:29 TechPubs-X45 cbsalarmlogrd: AlarmID 86 | Sat Nov 20 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status master Nov 20 15:09:39 TechPubs-X45 cbsalarmlogrd: AlarmID 87 | Sat Nov 20 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 100 16:08:31
16:08:31
16:09:23
16:09:23
16:09:29
16:09:39
191
Nov 20 17:37:34 TechPubs-X45 cbsalarmlogrd: AlarmID 88 | Sat Nov 20 18:37:34 2010 | clear | uprfan | fanTrayIncompatible | Fan tray mismatch | CorrelationID 53 Nov 20 17:37:37 TechPubs-X45 cbsalarmlogrd: AlarmID 89 | Sat Nov 20 18:37:37 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatch To see the CLI command show alarms in the messges file, grep the output for "| major |" [root@TechPubs-X45 admin]# grep AlarmID /var/log/messages | grep major systemAlarm | New system alarm level (major) Nov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 82 | Sat Nov 20 16:08:31 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status down Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 84 | Sat Nov 20 16:09:23 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 199 Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 85 | Sat Nov 20 16:09:23 2010 | major | cp2 | noRemoteBoxConfigured | No remote box configured Nov 20 15:09:29 TechPubs-X45 cbsalarmlogrd: AlarmID 86 | Sat Nov 20 16:09:29 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status master Nov 20 15:09:39 TechPubs-X45 cbsalarmlogrd: AlarmID 87 | Sat Nov 20 16:09:39 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 100 Nov 20 17:37:37 TechPubs-X45 cbsalarmlogrd: AlarmID 89 | Sat Nov 20 18:37:37 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatch Nov 20 20:37:39 TechPubs-X45 cbsalarmlogrd: AlarmID 91 | Sat Nov 20 21:37:39 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatch Nov 22 14:06:37 TechPubs-X45 cbsalarmlogrd: AlarmID 92 | Mon Nov 22 15:06:37 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group xyz status down
Viewing Audit Logs

The Audit Log allows you to track user-configuration events showing which users are making what changes to the system. To display the Audit Log, use the following command: show audit-trail The following is an example of this command. Aug 23 14:04:59 crossbeam cli: USER: admin, COMMAND: CBS# configure timeout > configure no timeout Aug 23 14:05:08 crossbeam cli: USER: admin, COMMAND: CBS# configure timeout > configure no timeout Aug 23 14:05:10 REG80A1 cli: USER: admin, COMMAND: CBS# configure hostname > configure hostname REG80A1 cp1 Aug 23 14:05:11 REG80A1 cli: USER: admin, COMMAND: CBS# configure hostname > configure hostname REG80A2 cp2 Aug 23 14:05:11 REG80A1 cli: USER: admin, COMMAND: CBS# configure ip telnet > configure ip telnet
192
Aug 23 14:05:12 REG80A1 cli: USER: admin, COMMAND: CBS# configure ip ftp > configure ip ftp Aug 23 14:05:13 REG80A1 cli: USER: admin, COMMAND: CBS# configure system-identifier > configure system-identifier 101 #Warning: WARNING: Command takes effect next system reload Aug 23 14:05:14 REG80A1 cli: USER: admin, COMMAND: CBS# configure operating-mode > configure operating-mode quad-np series-6 #Warning: WARNING: Command takes effect next system reload Aug 23 14:05:15 REG80A1 cli: USER: admin, COMMAND: CBS# configure access-list ip source-ip destination-ip > configure access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255 Aug 23 14:05:16 REG80A1 cli: USER: admin, COMMAND: CBS# configure access-list ip source-ip destination-ip > configure access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255
193
194
18
Upgrading XOS Software and Firmware
This chapter provides migration and upgrade information. It includes the following major topics:
z z z z
XOS Automated Workflow System on page 195 XOS shar Upgrade on page 196 Upgrading the XOS Firmware on page 200 XOS Migration on page 204
If you are upgrading to XOS 9.5.x on an X-Series Platform currently running XOS 9.0 or above, use the Automated Workflow System to perform an upgrade. If you are upgrading to XOS 9.5.x on an X-Series Platform currently running XOS 8.x, use the Migration Process to upgrade to XOS 9.5.x.
XOS Automated Workflow System

The Automated Workflow System (AWS) combines multiple manual steps into simple menu selections to automate the upgrade of XOS software and firmware. Crossbeam recommeds using the Automated Workflow System to perform upgrades. This automated process performs system checks and upgrades automatically, saving significant time over the manual upgrade process. AWS allows you to upgrade XOS software versions beginning with XOS V9.0. To view the currently installed XOS version prior to upgrading, use the show release command from the upgrade context. CBS# upgrade show release Crossbeam: 9.0.0-131 (current) CBS# To verify system compatibility for an upgrade, perform the following system check. After copying the .shar file onto your system, run the verify-system command from the upgrade context. CBS# upgrade verify-system 9.x.x If incompatibilities are detected, error messages will be displayed. Perform the necessary changes (if any) and start the Automated Workflow System to perform the upgrade. The Automated Workflow System will not upgrade XOS V8.x to XOS V9.0; use the Migration Process. The Automated Workflow System provides a method to upgrade software versions after V9.0, and can be used to upgrade firmware on modules in the X-Series Platform after XOS 9.0 has been installed. To access the Automated Workflow Menu, enter the following command from the XOS CLI: CBS# automated-workflow-menu Welcome to the X-Series Platform Automated Workflow System! Version: 1.1.0-x 1. 2. 3. 4. 5. Configure XOS... Upgrade XOS software and firmware... View system configuration and status... Applications... Custom...
195
Select a submenu to view available automated workflows. Enter x to exit or ? for help. Please Enter Selection: Enter your selection to begin the automated process. Help is available at each level by using the ?. AWS supports five automated workflows:
z z
Software upgrade: This feature is available to upgrade from XOS V9.0 to a future release. You cannot upgrade to XOS V9.0 using AWS. Firmware compatibility check: The firmware compatibility check compares the current versions of installed firmware against the recommended version, and displays the result (Up to date, or Update required). Firmware update: This workflow will perform the firmware update automatically. Prepare for Rollback: This workflow saves the currently installed XOS version to a separate distribution (distribution two) on the disk. Distribution one is prepared for the installation of a new version of XOS. Rollback: This workflow will return the X-Series Platform to the previous version of XOS should an upgrade fail.
z z z
Information generated during an automated workflow is saved to the AWS log file, available at /var/log/aws.log. You can also watch the progress during a workflow using the show automated-workflow-progress command. During the upgrade process, AWS will lock the configuration. If a user attempts to access the configuration file to make changes during the upgrade, an error message stating the configuration is locked by the Automated Workflow System will be displayed.
XOS shar Upgrade

With XOS V9.x installed, you can use AWS to upgrade your XOS versions. However, there may be situations where you choose to perform a shar upgrade of the software or firmware. Those procedures are included in the next sections. IMPORTANT: The shar upgrade procedure will not work to upgrade from pre-V9.0 to XOS V9.x. To upgrade to XOS V9.x, you must use the XOS Migration on page 204. To view the currently installed XOS version prior to upgrading, use the show release command from the upgrade context. CBS# upgrade show release Crossbeam: 9.0.1-xxx (current) Crossbeam: 9.5.x-xxx CBS#
Copying and Extracting the shar File

To FTP and extract the XOS software, complete the following: 1. Log into XOS as root: CBS# unix su Password: [root@xxxxx admin]# 2. Change to the rpm directory, using the following command: # cd /usr/os/rpm
196
3. 4. 5. 6.
FTP or copy the xos-upgradepack-A000-X.Y.Z-W.shar.gz file from your XOS CDROM or software package to /usr/os/rpm. Note that A000-X.Y.Z-W represents Ocode-Major.Minor.Maintenance-Kit.shar Enter the following command at the root prompt while in /usr/os/rpm: # gzip d xos-upgradepack-A000-X.Y.Z-W.shar.gz Change the extracted file to an executable as follows: # chmod 777 xos-upgradepack-A000-X.Y.Z-W.shar Once the above command completes, enter the following command: # ./xos-upgradepack-A000-X.Y.Z-W.shar
Upgrading with the Extracted XOS Software

To avoid upgrade problems during the procedure, do not execute any CLI commands that are not within the following upgrade procedure. During the upgrade, the system displays informational messages regarding verifications and checks. As these verifications and checks are being performed, you may temporarily be unable to issue any CLI commands.
Single CPM Configurations

1. 2. 3. 4. 5. Start a CLI session. If using CP redundancy, skip to Dual CPM Configurations below. From within the CLI, enter the Upgrade context using the following command: CBS# upgrade Display the available software versions as follows: CBS(upgrade)# show new-release Enter the desired software version. For example: CBS(upgrade)# install <xos_version> Answer the questions as prompted. If in the Offline state, saving the configuration is optional. 6. After the successful upgrade message is displayed, reload the system as follows: CBS# reload all
Dual CPM ConfigurationsUpgrading XOS

NOTE: Changing the CPM Redundancy configuration may require a reboot of the CPMs before the configuration changes take effect. You cannot have two CPMs synchronized in the X-Series Platform while running the XOS upgrade. Installations to CPMs must be performed one at a time. The recommended process for upgrading an XOS system with Control Processor redundancy is to: 1. 2. 3. 4. 5. Take the secondary CPM offline. While the primary CPM is still running, upgrade the secondary CPM. Configure the platform so that, upon reload, the current secondary CPM boots as the primary and the original primary is taken offline. Now upgrade the offline CPM, which had been the original master, and re-enable election/sync. Reboot the offline CPM. The two CPMs synchronize and CPM redundancy is restored.
To upgrade a Dual CPM configuration: 1. Start a CLI session on the primary CPM.
197
2. 3. 4. 5. 6. 7.
If you are using CP redundancy, bring the secondary CPM offline. CBS# configure cp-redundancy set other_cp offline Start the CLI on the secondary CPM. Note that it may take a moment or two for the secondary CPM to come offline. Perform the Copying and Extracting the shar File on page 196 to copy the file to the offline CPM. From within the CLI on the offline CPM, enter the Upgrade context using the following command: CBS# upgrade From the secondary CPM, display the available software versions as follows: CBS(upgrade)# show new-release From the secondary CPM, enter the desired software version. For example: CBS(upgrade)# install <xos_version> Where xos_version is the specific version of the software that you wish to load. Answer the questions as prompted.
8. 9.
Configure the secondary CPM for election. CBS# configure cp-redundancy set this_cp election Save the configuration on the secondary CPM.
10. Start the CLI on the primary CPM then configure the secondary CPM for election from the primary CPM. CBS# configure cp-redundancy set other_cp election 11. Reload the secondary CPM: CBS# reload offline-cp 12. Reload all APM and NPM modules. For the X80 system, reload modules 1 to 12 as follows; otherwise, reload modules 1 to 5. This command takes all modules offline; no traffic is processed during this time. CBS# reload modules 1 12 13. Take the Primary CPM offline. CBS# configure cp-redundancy set this_cp offline 14. When prompted, log back in to the CPM that was the original Primary CPM. 15. Save your configuration. CBS# copy running-config startup-config NOTE: The secondary CPM reloads with the new version of the XOS software and becomes the Primary CPM. 16. Connect to CLI on the current Primary CPM and verify that the system is operating correctly before upgrading the other CPM. 17. Repeat the procedure in Copying and Extracting the shar File on page 196 on the offline CPM. 18. From within the CLI on the offline CPM, enter the Upgrade context using the following command: CBS# upgrade 19. From the offline CPM, display the available software versions as follows: CBS(upgrade)# show new-release 20. From the offline CPM, enter the desired software version. For example: CBS(upgrade)# install <xos_version> Where xos_version is the specific version of the software that you wish to load. Answer the questions as prompted. 21. Set the offline CPM to Election mode. CBS# configure cp-redundancy set this_cp election
Upgrading XOS Software and Firmware 198
22. Save your configuration. CBS# copy running-config startup-config 23. Connect to the current Primary CPM and reload the offline CPM. The CPM becomes the secondary CPM. CBS# reload offline-cp After the reload is complete CPM redundancy is restored. The CPM roles are reversed from when you started this procedure.
Dual CPM ConfigurationsInstalling Applications

For dual CPM configurations, to install an application, do the following: For RPM: 1. 2. Execute the rpm -i command for the application on both CPMs. Execute the application install command from the CLI only on the primary CPM.
For CBI: 1. 2. Copy the .cbi bundle to /usr/os/apps/archive/ on the primary CPM only. Install the application from the CLI as normal.
Files Generated During the Software Upgrade Procedure

The following configuration files are created during the upgrade process.
Table 19.
Files Created During an Upgrade

File Name upgrade-saved-cfg.<release>.orig upgrade-saved-cfg.<release>.flat upgrade-saved-cfg upgrade-cfg-error(s) Description Original configuration before upgrade. Original configuration before upgrade in flat format. Configuration after upgrade in flat format. All CLI commands plus errors. CLI Upgrade log file. History log file. CLI errors only.
Directory /tmp
/usr/os/logs
cli_conversion_log SystemSoftwareReleaseHistory
/var/log
cli-upgrade-errors
199
Upgrading the XOS Firmware

Crossbeam recommeds using the Automated Workflow System to perform firmware upgrades. This automated process performs system checks and upgrades automatically, saving significant time over the manual upgrade process. Refer to the XOS V9.5.x Release Notes for the updated list of the minimum firmware version requirements.
Caution: Manual Firmware and FPGA upgrades must be executed at the same time before reloading. Attempting to reboot the system after a firmware upgrade without the corresponding FPGA upgrade will fail.
Use /crossbeam/bin/revs_check to display the current status of the firmware on your system. Information is provided for each installed module (NPM, CPM, APM). Additionally, you can review the Firmware Status view in the Greenlight Element Manager for the same information. There are two types of firmware to upgrade: FPGA and FLASH. To see the current firmware status, the minimum revision necessary, and the recommended version from the command line, execute the revs_check command from the /crossbeam/bin directory with the following parameters:

-c: Display the current firmware revision. -m: Display modules which do not meet minimum requirement. -u: Display modules whose firmware revisions are not up-to-date. -h: Display latest version of the files.
CBS# unix su [root@xxxx admin]# cd /crossbeam/bin [root@xxxx admin]# ./revs_check -c -m -u The flashit tool needed to perform an upgrade is located within the /crossbeam/bin directory. If there are multiple flash or FPGA files, the file ending in the highest number is the latest. If you are unsure of the type of firmware, enter -h to determine the usage.
Manually Upgrading Firmware on an NPM

Upgrade the firmware on an NPM-86xx using a CPM as follows: 1. Set the NPM to maintenance state: CBS# configure module <slot-number> maintenance NOTE: The module must be in maintenance mode; otherwise, the procedure may leave the module in an unstable state. 2. 3. 4. Go to the unix prompt. CBS# unix su Change directory to /usr/os/etc/Firmware/NP. [root@xxxx admin]# cd /usr/os/etc/Firmware/NP Locate then copy the Flash and FPGA files to the /tftpboot/npm6_shared directory. For example: [root@xxxx NP]# cp npm6xbprc_R0010_130.dat /tftpboot/npm6_shared/ [root@xxxx NP]# cp npm6FlashImage0014.dat /tftpboot/npm6_shared/ [root@xxxx NP]# cp npm6sysctrl_R04.dat /tftpboot/npm6_shared/
200
5.
Telnet to the NPM. [root@xxxx admin]# telnet npm<slot> The NPM can be in slots 1-4. For example, enter npm1 to connect to the NPM in slot 1.
6.
Run the flashit -w <filename> command for each file. The following filenames are examples only: /# flashit -w /cbs/shared/npm6xbprc_R0010_130.dat /# flashit -w /cbs/shared/npm6FlashImage0014.dat /# flashit -w /cbs/shared/npm6sysctrl_R04.dat
7. 8.
When the upgrade is completed, return to the CPM prompt: /# exit Verify that the upgrade completed successfully. a. b. Make sure that there were no NPM error messages during the flashit command. Check the syslog for the message, npm<slot> flashit: program terminates normally.
9.
Exit the unix prompt and return to the CLI.
10. Enable the module. CBS# configure module <slot-number> enable 11. Use the show version detail command to verify that the firmware is upgraded. For example: CBS# show version detail Copyright (c) 2000-2011 by Crossbeam Systems, Inc. All rights reserved. Version: XOS 9.5.x [Mar 1 2011 13:56:26] (bldmgr) gcc: gcc version 2.96 20000731 (Linux 7.3 2.96-112) CVS_Label: XOS-9_5_x_x-20101205_2 Kit_Number: 88 CPU at 1995 Mhz processor with 3592316K bytes of memory 3566660K bytes of memory in use Uptime is 3 day(s) 21 hour(s) 40 min(s) 36 sec(s) Hard disk is 120(GB) Second Hard disk is not present Flash is not present
Details per slot: Revision for slot 1 Boot Strap version Bootloader version Diagnostics version SysCtl FPGA version Focus FPGA version CPLD version Board version Board serial number Board type Board part number : : : : : : : : : : 2.0.0.10 2.0.0.8 2.1.0.2 0x4 0xa 0x5 F G801B022 NP8600 003927
201
Manually Upgrading Firmware on an APM

To upgrade APM firmware, the APM must be part of a VAP group. Upgrade the APM firmware, as follows: 1. 2. Determine the APM VAP mapping structure: CBS# show ap-vap-mapping Set the APM to maintenance state: CBS# configure module <slot-number> maintenance NOTE: The module must be in maintenance mode; otherwise, the procedure may leave the module in an unstable state. 3. 4. 5. Wait momentarily for the APM to reboot. Verify that the APM is fully booted in Maintenance mode: CBS# show ap-vap-mapping Log into the VAP mapped to the APM. [root@xxxx admin]# rsh <vap_index> The firmware image files are located under the /usr/os/etc/Firmware/AP directory. The flash image file has Flash in the filename. The following filenames are examples only: apcp6FlashImage0010.dat (flash image file for an APM-8600) apcp6_fpga_60.dat (FPGA image file for an APM-8600) For 86xx modules, from the VAP UNIX prompt, execute the following commands: # /usr/os/bin/flashit -w <flash image file> # /usr/os/bin/flashit -w <image file> 6. 7. After the success messages display, exit out of the VAP rsh. After the reload completes, enable the APM: CBS# configure module <slot-number> enable
Manually Upgrading Firmware On a CPM

The firmware upgrade procedure for a CPM depends on the whether a single or dual CPMs are present.
Upgrading Firmware for a Single CPM

The firmware image files are located in the /usr/os/etc/Firmware/CP directory. The flashit image file has Flash in the filename. The following filenames are examples only: apcp6FlashImage0010.dat (flash image file for a CPM-8600) apcp6_fpga_60.dat (FPGA image file for a CPM-8600) To upgrade the firmware on a single CPM, complete the following steps: For CPM-86xx modules, from the UNIX prompt, execute the following commands: # /usr/os/bin/flashit -w <flash image file> # /usr/os/bin/flashit -w <image file>
202
1. 2. 3.
After the success messages display, halt the CPM as follows: # halt Press the reset pin located at the front of the CPM to reboot it, or remove and re-seat the CPM. After the CPM reboots successfully, verify that the firmware is upgraded: CBS# show module status revision {cp1|cp2}
Upgrading Firmware for Dual CPMs

The following upgrade procedure is valid for systems running XOS V8.0 or later. For earlier versions, refer to the XOS Configuration Guide for that release. Also, the dual CPMs must be in a Primary / Secondary state before the upgrade starts. The firmware image files are located in the /usr/os/etc/Firmware/CP directory. The flashit image file has Flash in the filename. The following filenames are examples only: apcp6FlashImage0010.dat (flash image file for a CPM-8600) apcp6_fpga_60.dat (FPGA image file for a CPM-8600) To upgrade the FPGA firmware for a dual CPM system, complete the following: 1. Make sure that CP redundancy is enabled and that both CPMs are set to the Election state: CBS# show cp-redundancy Take note which CPM is Primary and which is Secondary. If the Administrative state of the CPMs is not set to Election, perform the following at the Primary CPM: CBS# config cp-redundancy set this_cp election CBS# configure cp-redundancy set other_cp election CBS# wr NOTE: Although a message warns that the system must be reloaded, you do not need to reload the modules. 2. Upgrade the Secondary CPM firmware. For CPM-86xx modules, from the UNIX prompt, execute the following commands: # /usr/os/bin/flashit -w <flash image file> # /usr/os/bin/flashit -w <image file> The firmware image files are located under the /usr/os/etc/Firmware/CP directory. 3. 4. 5. After the success messages display, halt the CPM: # halt Reboot the CPM by pressing the reset pin located in the front of the CPM. If necessary, you can also remove and re-seat the CPM to reboot it. After the CPM reboots successfully, verify that the firmware is upgraded. From the primary CPM use the following CLI command: CBS# show module status revision {cp1|cp2} 6. 7. Reboot the Primary CPM so the upgraded CPM becomes Primary. Upgrade the non-upgraded CPM firmware. For CPM-86xx modules, from the UNIX prompt, execute the following commands: # /usr/os/bin/flashit -w <flash image file> # /usr/os/bin/flashit -w <image file> The firmware image files are located under the /usr/os/etc/Firmware/CP directory. 8. 9. After the success messages display, halt the CPM as follows: # halt Reboot the CPM by pressing the reset pin located in the front of the CPM. If necessary, you can also remove and re-seat the CPM to reboot it.
203
10. After the CPM reboots successfully, verify that the firmware is upgraded. From the primary CPM use the following CLI command: CBS# show module status revision {cp1|cp2} NOTE: At the completion of this procedure, the CPMs have swapped their Primary and Secondary states. To restore the Primary state to the original CPM, wait for CPM synchronization, and then reboot the current Primary CPM.
XOS Migration
Starting with XOS V9.0, Crossbeam implemented the RHEL5 64-bit kernel and compatible userspace packages. These improvements provide updated support and additional security provisions. Versions of XOS prior to V9.0 use a 32-bit operating system. Upgrading from XOS V8.x or earlier to V9.5.x requires running the XOS migration feature. The migration process combines elements of a fresh installation with the upgrade process to automatically migrate your applications and system configuration to the latest XOS software. The following migration scenarios are supported:
z z z
XOS V8.5 to XOS V9.x XOS V8.x with Series-6 (86xx and higher) hardware to XOS V9.x XOS V7.3 on a CPM-86xx with no configuration to XOS V9.x
X-Series Platforms with older modules (pre-86xx) require upgrades prior to migration. Migration is an automated process that takes place in three stages. 1. 2. 3. Stage One saves the existing configurations and necessary files, and prepares the new distribution. Stage Two copies the required files to the new distribution, and converts files to the RHEL5 kernel. Stage Three converts the configuration file, upgrades all VAPs, saves the configuration, and reboots the system.
Because the migration process is an automated task, user input must be provided for the following items before the migration begins.
z z
A system reboot is required at the end of the first stage of the process. Choose whether to automatically reboot the system, or to halt the system and perform the reboot manually. If an error occurs during migration, choose whether to halt the migration and remain incomplete with the new version while repairing the error, or revert to the previous version.
Throughout the migration process, the current state of the CPM is saved in a system file. The CPM invokes different scripts during the process, using responses to the above questions when required.
XOS V9.x Migration Minimum Requirements:

z z
Hardware modules (APM, CPM, and NPM) of 8600 or above CPM-8600 or later with 4GB RAM (four 1-Gigabit DIMMs)
If your X-Series Platform is not running 86xx-series modules, you must upgrade the modules before migrating to V9.x. Additionally, the CPM-86xx must have four 1-Gigabit DIMMs installed to meet the memory requirements. Refer to the hardware upgrade instructions and RAM upgrade instructions provided with the modules for upgrade procedures. Once your system meets these minimum requirements, follow the appropriate procedure in the Workflows section to perform the migration. The migration process performs a number of system checks, and provides information when user action is required. If the application versions currently installed on your X-Series Platform are not supported on 9.x, you will be asked to exit the migration process and upgrade the applications. See Applications Supported for Migration for application information.
If your current configuration requires changes, you will be asked to exit the migration process and update the configuration. For information on configuration requirements for migration, see Upgrade Considerations on page 208.
Applications Supported for Migration

The following applications are supported for migration to XOS 9.0. For information regarding application migration during an upgratde to XOS V9.X. and higher, please refer to the XOS Release Notes.
z z z z z z z z z
Check Point SecurityGateway R70/R71 Check Point VPN-1 Power NGX R65 Check Point VPN-1 Power VSX NGX R65/R67 Imperva SecureSphere 7.0 IBM Proventia N-IPS 2.0 Trend IWSS 3.1 Trend IMSS 7.0 Websense 6.3.2 RSW 8.0
Workflows
The following workflows provide information and steps to execute the migration process.
Migration from XOS V8.5.x to V9.x

The migration process performs a number of system checks, and provides information when user action is required. If your system does not meet the minimum requirements, an error message will be displayed, and you will be given options to make the necessary upgrades before the migration.
Single CPM Migration

1. Copy the following file to the /crossbeam/rpm directory on your X-Series Platform. CBS# unix su Password: [root@xxxxx admin]# mv xos-migratepack-9.5.x-xxx /crossbeam/rpm/ 2. Execute the following commands to begin migration: [root@xxxxx admin]# cd /crossbeam/rpm [root@xxxxx rpm]# chmod 777 xos-migratepack-9.5.x-xxx [root@xxxxx rpm]# ./xos-migratepack-9.5.x-xxx The script executes. Several system checks are performed, and messages are displayed when the migration requirements are not met. The system provides information in many cases about the recommended course of action. After each requirement is met, restart the migration process. 3. When all checks are passed, the following general messages will appear: !!!!!!!!!!!!!!! !!! WARNING !!! !!!!!!!!!!!!!!! Distribution 2 will be overwritten by the migration process. Would you like to proceed? (y/n):y !!!!!!!!!!!!!!!
205
!!! WARNING !!! !!!!!!!!!!!!!!! If you use acl-interface to mirror or pass-through traffic from the source interface to the target interface, you must ensure that the target acl-interface is configured. If you fail to configure the target interface, the migration will proceed but your acl-interface configuration will not transfer into XOS V9.5.x. Please see the Release Notes or Configuration Guide for more information. Would you like to proceed? (y/n):y Any unsaved configurations will be lost during migration process. Do you want to save the configuration before continuing? (y/n):y [I] Saving configuration. CBS# copy running-config startup-config Saving configuration ... Please be patient... . [I] Performing pre-migration checks. [I] Executing pre_upgrade_check. WARNING !!! WARNING: XOS 9.5.x only supports the following modules: * CPM-8600 * APM-8600, APM-8650, APM-9600 * NPM-8600, NPM-8620, NPM-8650, NPM-9600 Only supported modules will boot after upgrading to XOS 9.5.x. Do you want to reboot the CPM automatically, when the migration process requires a reboot? (y/n):y If an error occurs during the migration process, do you want to revert to the existing XOS release? (y/n):y The system performs the migration, saves your configuration and reloads all modules at the completion of the process.
Dual CPM Migration

If you have a redundant dual CPM configuration, use the following procedure to migrate to XOS V9.5.x. 1. Switch to the linux prompt and login to the system as root. CBS# unix su Password: 2. Copy the following file to the /crossbeam/rpm directory on both the primary and secondary CPMs on your X-Series Platform. [root@xxxxx admin]# mv xos-migratepack-9.5.x-xxx /crossbeam/rpm/ 3. 4. 5. Start a CLI session on the primary CPM. Verify that both CPMs are fully synced using the show cp-redundancy command. CBS# show cp-redundancy From the primary CPM, bring the secondary CPM offline. CBS# configure cp-redundancy set other_cp offline
6. 7.
Start the CLI on the offline (secondary) CPM. Note that it may take a moment or two for the secondary CPM to come offline. Execute the following commands to begin migration: [root@xxxxx admin]# cd /crossbeam/rpm [root@xxxxx rpm]# chmod 777 xos-migratepack-9.5.x-xxx [root@xxxxx rpm]# ./xos-migratepack-9.5.x-xxx The script executes. Several system checks are performed, and mesages are displayed when the migration requirements are not met. The system provides information in many cases about the recommended course of action. After each requirement is met, restart the migration process.
8.
When all checks are passed, the following general messages will appear: !!!!!!!!!!!!!!! !!! WARNING !!! !!!!!!!!!!!!!!! Distribution 2 will be overwritten by the migration process. Would you like to proceed? (y/n):y Any unsaved configurations will be lost during migration process. Do you want to save the configuration before continuing?(y/n):y [I] Saving configuration. CBS# copy running-config startup-config Saving configuration ... Please be patient... . [I] Performing pre-migration checks. [I] Executing pre_upgrade_check. WARNING !!! WARNING: XOS 9.5.x only supports the following modules: * CPM-8600, CPM-9600 * APM-8600, APM-8650, APM-9600, APM-2030 * NPM-8600, NPM-8620, NPM-8650, NPM-9600, NPM-20, NPM-30 Only supported modules will boot after upgrading to XOS 9.x.x. Do you want to reboot the CPM automatically, when the migration process requires a reboot? (y/n):y If an error occurs during the migration process, do you want to revert to the existing XOS release? (y/n):y The system performs the migration.
9.
From the CLI on the offline CPM, configure the offline (secondary) CPM for election. CBS# configure cp-redundancy set this_cp election
10. Save the configuration on the offline (secondary) CPM. CBS# copy running-config startup-config 11. From the primary CPM, configure the offline (secondary) CPM for election. CBS# configure cp-redundancy set other_cp election 12. From the primary CPM, reload the offline (secondary) CPM: CBS# reload offline-cp 13. From the primary CPM, verify the offline (secondary) CPM is rebooting.
CBS# show chassis 14. From the primary CPM, set the primary CPM to offline. This may take a few moments. CBS# configure cp-redundancy set this_cp offline The secondary CPM reloads with the new version of the XOS software and becomes the primary CPM. 15. Save your configuration. CBS# copy running-config startup-config 16. Connect to the CLI on the new primary CPM and reload the NPMs and APMs to run with the XOS 9.x software. CBS# reload all 17. Verify that the system is operating correctly before upgrading the offline CPM. 18. Repeat Step 7 and Step 8 on the offline CPM. 19. Enter the CLI on the primary CPM, and set the secondary CPM to election. CBS# configure cp-redundancy set other_cp election 20. From the primary CPM, reload the secondary CPM. CBS# reload offline-cp
Migration from XOS V7.3.x or V8.x with Series- 6 Hardware

If your X-Series Platform has XOS V8.x and Series-6 hardware, the migration procedure is the same as the previous workflow. You are not required to upgrade to 8.5 prior to migration. If XOS V7.3 is installed, and there are no VAP groups or interfaces configured on the system, and the system identifier is a value between 1-255, it is possible to migrate directly to XOS V9.x using the Workflows on page 205.
Migration with an NPM-82x0 or an APM-8400

If your X-Series Platform has a mixture of Series-6 and non-Series-6 hardware, the migration will fail. XOS V9.x requires Series-6 hardware. In most cases, the upgrade process to Series-6 hardware requires application and configuration updates. Before deciding to perform a migration, consider if it is more efficient to migrate your current configuration or to perform a clean installation of XOS V9.x and reconfigure your system. Perform the following upgrades to bring your system up to the base-level configuration for Migration. 1. 2. 3. 4. Upgrade the modules according to the Series-6 upgrade instructions included with your hardware. Upgrade your configuration. See Upgrade Considerations on page 208 for more information. Upgrade or uninstall/reinstall the applications. See Applications Supported for Migration on page 205. After the upgrades are complete, perform the XOS V9.x Migration procedure.
Upgrade Considerations
When migrating XOS, a script checks the configuration file for compatibility problems. If any issues are discovered, you have the option of stopping the upgrade and correcting them. You can also perform these checks manually before migrating. The following checks are performed by the migration script:
z
Interfaces with overlapping ingress VLANs cannot be redundant and must be reconfigured. A backup interface in an active-active configuration cannot be configured to pass the same type of traffic as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or pass untagged traffic. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANS tagged 300. Refer to VLANs on page 63 for more information.
208
If your configuration uses the acl-interface command to mirror or pass-through traffic from a source interface to a target interface, the following configurations must be updated. If these configurations are not updated, your acl-interface configuration will not transfer into XOS V9.x:

The target acl-interface must be configured. Remote mirroring and pass-through (when the target port is on a different NPM than the source port) is not supported. An interface cannot mirror or pass-through traffic to itself.
z z
The MTU setting is no longer applicable to interfaces. Instead, the MTU must be assigned to circuit and VAP group with the circuit <name> vap-group <name> MTU command. Virtual router configurations can no longer have multiple VAP groups mapped to a single circuit. Make sure that all virtual routers have only one VAP group assigned. A VAP group is assigned to a virtual router using the vrrp virtual-router vrrp-id <id> vap-group <name> command. The promiscuous-mode active parameter should be used for circuits used as members in a bridge. Previously, the member circuits were set to promiscuous-mode using the circuit <name> vap-group <name> command. The system ID must be configured if upgrading to the NPM-8600. You can configure the system ID with the configure system-identifier <system-id> command, where valid values are 1 to 255. CPM versions prior to CPM-8600 are not supported. Upgrades from a version prior to XOS V8.0 are not supported, with the exception of an unconfigured XOS V7.3. The series-2 operating-mode is not supported. Please configure the operating-mode to series-6 prior to upgrading. The xslinux_v1 VAP OS is not supported. Please remove any VAP groups using the xslinux_v1 OS. The xslinux_v4 VAP OS is not supported. Please remove any VAP groups using the xslinux_v4 OS. UNI kernels are not supported. Please change these VAP groups to 'kernel-smp auto' A system-identifier of 0 is not supported. Please configure the system-identifier to <1-255> prior to upgrading. During an upgrade or migration, if your running configuration includes a flow rule with either a source or destination IP address in the form a.b.c.d/0 where a.b.c.d is not 0.0.0.0, the address will be converted to 0.0.0.0/0. Only the following modules are supported on XOS V9.5.x:

z z z z z z z z z
CPM-8600, CPM-9600 APM-8600, APM-8650, APM-9600, APM-2030 NPM-8600, NPM-8620, NPM-8650, NPM-20, NPM-30
Refer to the XOS Command Reference Guide for a list of all commands that have changed in XOS V9.5.x.
Log Files
Major milestones on the migration process are logged in the syslog of the running distribution. More detailed information is logged into the following files: /crossbeam/logs/migration_s1.log This log file will contain information for stage one of the migration process. It will be created and updated on the old distribution, and then copied to the new distribution at the end of stage one. /crossbeam/logs/migration_s2.log This log file will contain information for stages two and three of the migration process. It will be created and updated on the new distribution. The log files include a time and date stamp for each event.
209
210
19
Configuring Routing Protocols
This chapter describes how to install routing protocols on the X-Series Platform. The routing protocols include:
z z z z z z
Routing Information Protocol Version 2 (RIPv2) Routing Information Protocol Version Next Generation for IPv6 (RIPng) Open Shortest Path First (OSPF) Open Shortest Path First for IPv6 (OSPFv3) Border Gateway Protocol (BGP) Protocol Independent Multicast-Sparse Mode (PIM-SM)
Crossbeam provides these routing protocols in a Routing Software (RSW) package that is sold and installed separately from XOS. Refer to the RSW Installation Guide for detailed information. This chapter contains the following major topics:
z z z z z
Routing Software Overview on page 211 Installing Routing Protocols on page 213 Installing a Routing Protocol on a VAP Group on page 214 Managing Routing Configurations on page 214 IPv6 Tunneling on page 217
Routing Software Overview

Regardless of the protocol that you install and use on the Crossbeam Platform, RSW installs a Network Services Module (NSM) that coordinates the routing information between the routing daemons and the operating system on the X-Series Platform. NSM shares the routing information between all members in a VAP group. NSM collects and distributes all routing information from the routing protocols, the kernel, and static routes. In XOS V9.x, the routing software only uses static routes configured in the XOS CLI. NSM is installed automatically with the routing protocols. Therefore, a routing protocol such as OSPF cannot run independently without an active NSM daemon. If the NSM daemon accidentally stops, all routing protocols on the platform stop. The routing daemons run on a single, master-VAP in a VAP group so that external routers see the VAP group as a single neighbor. The routing daemons communicate with neighbor devices and pass calculated routing information to the routing information base (RIB). NSM also collects information about directly connected networks and static routes from the XOS operating system. After NSM on the master VAP accumulates routing information, it writes the routing information to the XOS Forwarding Information Base (FIB), and sends collected routing information to the NSM instances running on all other members in the VAP group. The NSM instances on all other VAPs write the routing information to the respective XOS FIBs. This architecture ensures that every member in a VAP group contains identical routing information.
211
Understanding Routing Services Failover on the X-Series Platform

Routing services stop when failures occur on the master VAP that cause a routing protocol daemon or NSM to stop. To minimize the impact of the failure, XOS selects another master VAP and starts the routing protocol on the newly elected master. The NSM daemon on the new master VAP begins relearning routing information and distributing it within the VAP group. During the transition to a new master, all existing routes are stale and unused and the new master VAP must relearn all routes. During the relearning period, the FIB Retain feature in NSM allows the use of stale routes (default of 90 seconds). See Configuring FIB Retain on page 215 for more information.
Converting Static IP Routes From a Previous RSW Version

In earlier versions of RSW, static routes were created and managed by NSM. In XOS V9.x, XOS writes static routes into the VAP operating system and NSM uses only XOS CLI-created static routes. The conversion script in XOS V9.x, nsm_routes_to_cli.sh, converts routes configured in the nsm.conf file on the first VAP in the VAP group into configuration commands for XOS. The conversion script validates each route in nsm.conf for a next hop IP address. The script discards any route that does not have a next hop IP address and records the discarded route in the conversion script output. Usage: /crossbeam/bin/nsm_routes_to_cli.sh [OPTIONS] Convert NSM IP routes to XOS IP routes. Options: -o <file>: Output file to store result -h : Print out this help
Converting Static IP Routes to XOS Commands

Follow these steps to convert static IP routes configured in nsm.conf to XOS route configuration commands. NOTE: Make backup copies of the XOS running-config and each NSM and protocol configuration before converting static IP Routes to XOS commands. See Configuring Static IP Routes on page 107 for more information. 1. 2. Log into the CPM: CBS# Change to root: CBS# unix su [root@test-chassis admin]# Change directory to /crossbeam/bin: [root@test-chassis admin]# cd /crossbeam/bin [root@test-chassis bin]# Execute the route conversion command: [root@test-chassis bin]# ./nsm_routes_to_cli.sh -o <file> [root@test-chassis bin]# The script finds the first VAP and converts the routes in nsm.conf to XOS commands saved in the specified <file>. The script output indicates any routes that do not have a next hop IP address specified.
3.
4.
Executing Converted Static IP Routes to Configure XOS

1. 2. Log into the CPM: CBS# Execute the commands in the file with converted routes: CBS# exec /<path>/<file> CBS#
212
NOTE: If any routes are invalid, the XOS CLI returns an error message. 3. Copy the running-config to startup-config: CBS# copy running-config startup-config CBS#
Verifying Static IP Routes

Verify the static routes before and after the conversion and configuration into XOS. Do this by comparing the static routes defined in the XOS running-config and the NSM configuration file.
Installing Routing Protocols

To install the routing protocols onto the XOS CPM: 1. Log into the X-Series Platform as root, using the following commands: CBS# unix su Password: [root@xxxxx admin]# For the xslinux_v3 VAP operating system, copy the following RPMs to the /usr/os/rsw/rpm directory: zebos-common-x.x.x-x.x.x.x.EL.i686.rpm zebos-bgp-x.x.x-x.x.x.x.EL.i686.rpm zebos-ospf-x.x.x-x.x.x.x.EL.i686.rpm zebos-ospf6-x.x.x-x.x.x.x.EL.i686.rpm zebos-rip-x.x.x-x.x.x.x.EL.i686.rpm zebos-ripng-x.x.x-x.x.x.x.EL.i686.rpm zebos-pim-x.x.x-x.x.x.x.EL.i686.rpm For the xslinux_v5 or xslinux_v5_64 VAP operating system (designated by an EL5 in the file name), copy the following RPMs to the /usr/os/rsw/rpm directory: zebos-common-x.x.x-x.x.x.x.EL5.i686.rpm zebos-bgp-x.x.x-x.x.x.x.EL5.i686.rpm zebos-ospf-x.x.x-x.x.x.x.EL5.i686.rpm zebos-ospf6-x.x.x-x.x.x.x.EL5.i686.rpm zebos-rip-x.x.x-x.x.x.x.EL5.i686.rpm zebos-ripng-x.x.x-x.x.x.x.EL5.i686.rpm zebos-pim-x.x.x-x.x.x.x.EL5.i686.rpm 2. Exit from the root: [root@xxxxx rpm]# exit
213
Installing a Routing Protocol on a VAP Group

Perform the following to install and configure a routing protocol: 1. Create a VAP group for the routing protocol (refer to Configuring a VAP Group on page 52). If a VAP group was already created, you need to know which kernel is installed as displayed by the show vap-group command. Install a protocol on a VAP group: CBS# routing-protocol <protocol> vap-group <vap-group-name> install Where <protocol> is rip, ripng, ospf, ospf6, pim, or bgp. 3. 4. After the protocol has been installed, enable and start the protocol: CBS# configure routing-protocol <protocol> vap-group <vap-group-name> start Configure the protocol: CBS# routing-protocol <protocol> vap-group <vap-group-name> configure The last command launches the ZebOS CLI. You are prompted for a password. Refer to the RSW Installation Guide for the default password, information on how to configure the routing protocols, and to change the default password. When you have finished using the ZebOS CLI, type exit to return to the XOS CLI.
2.
Managing Routing Configurations

The following commands allow you to manage the routing protocol configurations: routing-protocol routing-protocol routing-protocol routing-protocol routing-protocol routing-protocol routing-protocol Where: <protocol> install configure uninstall update Can be rip, ripng, ospf, ospf6 pim, or bgp. Installs a protocol on the VAP group. Launches the routing protocol prompt. Refer to the RSW Installation Guide to configure the protocol. Removes the protocol from the VAP group. Updates routing information for the vap-group. Use this command after members have been added to the VAP group or after the VAP group has been deleted and recreated. Saves an existing routing configuration on a VAP group to a file. Restores a saved routing configuration from a file to a VAP group. <protocol> <protocol> <protocol> <protocol> <protocol> <protocol> <protocol> vap-group vap-group vap-group vap-group vap-group vap-group vap-group <vap-group-name> <vap-group-name> <vap-group-name> <vap-group-name> <vap-group-name> <vap-group-name> <vap-group-name> install configure uninstall update save <path_and_file> restore <path_and_file> status
save <path_and_file> restore <path_and_file>
Caution: Restoring an RSW configuration file from a lower or higher RSW version may cause unpredictable results.
status Displays the status of a routing protocol on a VAP group.
The following commands write the protocol status to the XOS running-config and take the related action on the protocol daemon. configure routing-protocol <protocol> vap-group <vap-group-name> {start|restart|stop}
Where: <protocol> start stop restart Can be rip, ripng, ospf, ospf6, pim, or bgp. Starts the protocol. Stops the protocol. Restarts the protocol. This forces a running protocol, such as OSPF, to rebuild its routing table.
The following command enables access to the routing protocol NSM daemon on the master VAP from the CLI. routing-protocol-services vap-group <VAP_group_name> configure Any changes made to NSM are written to all active VAP group members. In NSM you can modify ACLs, configure FIB Retain, and troubleshoot general routing issues.
Accessing NSM
Use the following steps to access the CLI for NSM. 1. From the XOS CLI command prompt, enter: CBS# routing-protocol-services vap-group <VAP_group_name> configure Password: Enter the password (default is admin): Password: ***** Router> Enter privileged mode: Router>enable Password: Enter the privileged mode password (default is admin): Password: ***** Router#
2.
3.
4.
Configuring FIB Retain

Crossbeam recommends configuring FIB Retain on the VAP group to allow for graceful failover if the NSM daemon restarts or if the master VAP in a VAP group goes down. The default FIB Retain time is 90 seconds. Because the master VAP builds the RIB used in all VAP members, any interruption to the operation of the master VAP can result in stale routes in the FIBs on slave VAP members. When XOS elects a new master VAP member, the routing protocols must start, form adjacencies, and exchange routing information. The new master VAP learns routes for the entire VAP group. Slave VAP members must wait for RIB updates from the new master VAP. The resulting delay can cause traffic to be dropped or misrouted. FIB Retain minimizes the impact of this delay, by allowing you to specify a period of time during which existing routes are preserved while the routing protocol on the newly elected master refreshes routes in the FIB. If your system uses a single VAP, FIB Retain can temporarily preserve existing routes for IPv4 and IPv6 traffic in the event NSM restarts. NOTE: Only IPv4 routing information is synchronized across VAPs in the VAP group. IPv6 traffic is processed only on the master VAP. IPv6 routes learned by the master VAP are not shared with other members of the VAP group, but must be relearned by the new master VAP in the event of a failure. Follow these steps to configure FIB Retain if the default time of 90 seconds is not suitable for your configuration.
215
1. 2. 3.
From the XOS CLI command prompt, enter: CBS# routing-protocol-services vap-group <VAP_group_name> configure Enter the password (default is admin): Password: ***** Enter privileged mode: Router>enable Password: Enter the privileged mode password (default is admin): Password: ***** Router# Enter the configure terminal context to configure the FIB Retain value: Router#configure terminal Router(config)# Enter the FIB Retain configuration context: Router(config)#fib retain {time <number_of_seconds> | forever} Router(config)# Enter either the number of seconds or forever to retain the FIB values.
4.
5.
6.
7.
Exit the terminal configuration context: Router(config)#end Router# Write the configuration to nsm.conf: Router#write Configuration saved to /usr/os/etc/zebos/nsm.conf Router# Exit NSM to synchronize the changes to all VAP members: Router#exit CBS#
8.
9.
10. CBS#
Redistributing Routes
Protocols in the X-Series Platforms can be configured to redistribute routes to other protocols. Refer to the RSW Installation Guide for information on how to access the protocol and turn on route redistribution.
216
IPv6 Tunneling
IPv6 tunneling provides a method to transmit and receive IPv6 traffic over an IPv4 network. Typically, this kind of tunnel is used to connect to an IPv6-enabled destination (host or network). In the Crossbeam implementation of IPv6 tunneling, one end of the tunnel is a Crossbeam X-Series chassis and the other end is a dual stack router. IPv6 traffic is encapsulated in IPv4 packets and passed over the IPv4 network. When an IPv4 packet reaches the destination, it is unencapsulated and the IPv6 packet is routed appropriately. On the X-Series chassis, the traffic-processing VAP group must be configured to handle IPv6 traffic.
6to4 Tunnel
One IPv6 tunneling protocol is 6to4 tunneling. Using 6to4, you do not have to specify the destination address of the tunnel. You configure the local (source) end of tunnel with an IPv4 address and an IPv6 address. The IPv6 address contains an encoding of the IPv4 address. The major steps to configure the local/source end of the tunnel are:
z z z z
Create a VAP group and enable it to handle IPv6 traffic. Create a circuit with an IPv6 address and associate it with the VAP group. Create a circuit with an IPv4 address and associate it with the VAP group. Create the tunnel, specifying 6to4 as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, and specifying the address of the IPv4 circuit as the source-address.
Figure 15.
6to4 Tunnel
Example
To configure a 6to4 tunnel using the parameters in Figure 15, perform these steps: 1. On the X-Series chassis configure a VAP group, and enable IPv6 traffic processing. CBS# configure vap-group vg1 enable-ipv6
217
NOTE: A non-ip flow rule is automatically configured with these parameters: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate 2. Identify the IPv4 address that you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.16.20.30.
NOTE: This address will be the source address that you specify when configuring the 6to4 tunnel. 3. Configure the first of two circuits with these characteristics: a. b. c. Choose a name for the circuit (in this example, tunnel_a). CBS# configure circuit tunnel_a Associate the circuit with the VAP group (in this example, vg1) CBS(conf-cct)# vap-group vg1 Assign an IPv6 address to the circuit. The address must be constructed as follows:

The first 16-bit field must be 2002 to identify the IPv6 address as a 6to4 tunnel address. The next two fields must contain a decimal-to-hexadecimal conversion of the IPv4 address that you identified in Step 2. In this example, converting 172.16.20.30 to two hexadecimal numbers results in AC10:141E (172 converts to AC, 16 converts to 10, and so on). In the last 16-bit field, specify the host address that you want to associate with this circuit (for example, 3).
CBS(conf-cct-vapgroup)# ip 2002:AC10:141E:0:0:0:0:3/48 %WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit CBS(conf-cct-vapgroup-ip)# end 4. Configure a second circuit with these characteristics:
Choose a name for the circuit (in this example, tunnel_b). CBS# configure circuit tunnel_b Associate the circuit with the VAP group (in this example, vg1) CBS(conf-cct)# vap-group vg1 Assign the local IPv4 address that is being used by the VAP group to send traffic to the external IPv6 network. CBS(conf-cct-vapgroup)# ip 172.16.20.30/24 CBS(conf-cct-vapgroup-ip)# end
5.
Configure the 6to4 tunnel. CBS# configure ipv6-tunnel 6to4 circuit tunnel_a vap-group vg1 CBS(conf-tunnel-6to4)# source-address 172.16.20.30 CBS(conf-tunnel-6to4)#
6.
Optionally, configure a time-to-live value for packets sent through the tunnel. Values range from 1 to 256 seconds, and the default is 64. CBS(conf-tunnel-6to4)# time-to-live 120 Optionally, enable Maximum Transmission Unit (MTU) discovery for the path that the tunnel uses. Discovery of the MTU helps to avoid fragmentation. CBS(conf-tunnel-6to4)# path-mtu-discovery CBS(conf-tunnel-6to4)# end CBS#
7.
218
ISATAP Tunnel
ISATAP (Intra-Site Automatic Tunnel Address Protocol) is used for a site that has not fully implemented an IPv6 network. An ISATAP host talks to an ISATAP router through the ISATAP automatic tunnel over the existing IPv4 network. Typically, this kind of tunnel enables IPv6-enabled hosts to communicate over a local IPv4 network. The ISATAP tunnel interface treats underlying IPv4 as an NBMA (Non Broadcast Multi Access) network. Each ISATAP interface is assigned the same IPv6 prefix. The Router Advertisements (RA) are used to automatically assign prefixes. The major steps to configure the local/source end of the tunnel are:
z z z z
Create a VAP group and enable it to handle IPv6 traffic. Create a circuit with an IPv6 address and associate it with the VAP group. Create a circuit with an IPv4 address and associate it with the VAP group. Create the tunnel, specifying isatap as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, and specifying the address of the IPv4 circuit as the source-address.
Figure 16.
ISATAP Tunnel
Example
To configure an ISATAP tunnel using the parameters in Figure 16, perform these steps: 1. On the X-Series chassis configure a VAP group, and enable IPv6 traffic processing. CBS# configure vap-group vg2 enable-ipv6 NOTE: A non-IPv4 flow rule is automatically configured with these parameters: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate
219
2.
Identify the IPv4 address that the you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.32.30.40.
NOTE: This address will be the source address that you specify when configuring the ISATAP tunnel. 3. Configure the first of two circuits with these characteristics: a. b. c. Choose a name for the circuit (in this example, tunnel_1). CBS# configure circuit tunnel_1 Associate the circuit with the VAP group (in this example, vg2) CBS(conf-cct)# vap-group vg2 Assign an IPv6 address to the circuit. The address must be constructed as follows:

The prefix fields can be any valid IPv6 prefix, but must include the reserved identifier for ISATAP, which is 0:5efe. This example uses FC00:0:0:0:0:5EFE: as the prefix. The last four fields must contain a decimal-to-hexadecimal conversion of the IPv4 address that you identified in Step 2. In this example, converting 172.32.30.40 to two hexadecimal numbers results in AC20:1E28: (172 converts to AC, 32 converts to 20, and so on).
CBS(conf-cct-vapgroup)# ip FC00:0:0:0:0:5EFE:AC20:1E28/16 %WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit CBS(conf-cct-vapgroup-ip)# end 4. Configure a second circuit with these characteristics:
Choose a name for the circuit (in this example, tunnel_2). CBS# configure circuit tunnel_2 Associate the circuit with the VAP group (in this example, vg2) CBS(conf-cct)# vap-group vg2 Assign the local IPv4 address that is being used by the VAP group to send traffic to the external IPv6 network. CBS(conf-cct-vapgroup)# ip 172.32.30.40/24 CBS(conf-cct-vapgroup-ip)# end
5.
Configure the isatap tunnel. CBS# configure ipv6-tunnel isatap circuit tunnel_1 vap-group vg2 CBS(conf-tunnel-isatap)# source-address 172.32.30.40 CBS(conf-tunnel-isatap)#
6.
Optionally, configure a time-to-live value for packets sent through the tunnel. Values range from 1 to 256 seconds, and the default is 64. CBS(conf-tunnel-isatap)# time-to-live 120 Optionally, enable Maximum Transmission Unit (MTU) discovery for the path that the tunnel uses. Discovery of the MTU helps to avoid fragmentation. CBS(conf-tunnel-isatap)# path-mtu-discovery CBS(conf-tunnel-isatap)# end CBS#
7.
NOTE: For ISATAP hosts to obtain automatic IPv6 addresses, you must turn on Router Advertisements in the RSW software on the X-Series chassis. See the RSW documentation for details.
220
IPv6IP Tunnel
An IPv6IP tunnel requires that you configure an IPv4 address for both ends of the tunnel (on the X-Series chassis and on the destination router). Typically, this kind of tunnel is used as a static connection between two IPv6-enabled networks that are separated by an Ipv4 network. The major steps to create the local/source end of the tunnel are:
z z z z
Create a VAP group and enable it to handle IPv6 traffic. Create a circuit with an IPv6 address and associate it with the VAP group. Create a circuit with an IPv4 address and associate it with the VAP group. Create the tunnel, specifying ipv6ip as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, specifying the address of the IPv4 circuit as the source-address, and specifying the IPv4 address of the destination router as the destination-address.
Figure 17.
IPv6IP Tunnel
Example
To configure an IPv6IP tunnel using the parameters in Figure 17, perform these steps: 1. On the X-Series platform configure a VAP group, and enable IPv6 traffic processing. CBS# configure vap-group vg3 enable-ipv6 NOTE: A non-IPv4 flow rule is automatically configured with these parameters: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate 2. Identify the IPv4 address that the you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.48.40.50.
NOTE: This address will be the source address that you specify when configuring the IPv6IP tunnel. 3. Configure the first of two circuits with these characteristics: a. b. c. Choose a name for the circuit (in this example, tunnel_x). CBS# configure circuit tunnel_x Associate the circuit with the VAP group (in this example, vg3) CBS(conf-cct)# vap-group vg3 Assign an IPv6 address to the circuit (in this example, FD00:BC00:FFFF:2:0:0:0:2/48).
221
NOTE: Unlike the IPv6 addresses that are associated with 6to4 and ISATAP tunnels, this address has no pre-defined prefix and does not contain any encoding of the IPv4 source-address. CBS(conf-cct-vapgroup)# ip FD00:BC00:FFFF:2:0:0:0:2/48 %WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit CBS(conf-cct-vapgroup-ip)# end 4. Configure a second circuit with these characteristics:
Choose a name for the circuit (in this example, tunnel_y). CBS# configure circuit tunnel_y Associate the circuit with the VAP group (in this example, vg3) CBS(conf-cct)# vap-group vg3 Assign the local IPv4 address that is being used by the VAP group to send traffic to the external IPv6 network. CBS(conf-cct-vapgroup)# ip 172.48.40.50/24 CBS(conf-cct-vapgroup-ip)# end
5.
Configure the IPv6IP tunnel. CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 source-address 172.48.40.50 destination-address 172.48.40.51 CBS#
6.
Optionally, configure a time-to-live value for packets sent through the tunnel. Values range from 1 to 256 seconds, and the default is 64. CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 time-to-live 120 CBS#
7.
Optionally, enable Maximum Transmission Unit (MTU) discovery for the path that the tunnel uses. Discovery of the MTU helps to avoid fragmentation. CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 path-mtu-discovery CBS#
GRE Tunnel
A GRE tunnel requires that you configure an IPv4 address for both ends of the tunnel (on the X-Series chassis and on the destination router). Typically, this kind of tunnel is used as a static connection between two IPv6-enabled networks that are separated by an Ipv4 network. The major steps to create the local/source end of the tunnel are:
z z z z
Create a VAP group and enable it to handle IPv6 traffic. Create a circuit with an IPv6 address and associate it with the VAP group. Create a circuit with an IPv4 address and associate it with the VAP group. Create the tunnel, specifying gre as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, specifying the address of the IPv4 circuit as the source-address and the IPv4 address of the destination router as the destination-address.
222
Figure 18.
GRE Tunnel
Example
To configure a GRE tunnel using the parameters in Figure 18, perform these steps: 1. On the X-Series chassis configure a VAP group, and enable IPv6 traffic processing. CBS# configure vap-group vg4 enable-ipv6 NOTE: A non-IPv4 flow rule is automatically configured with these parameters: non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate 2. Identify the IPv4 address that the you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.64.50.60.
NOTE: This address will be the source address that you specify when configuring the IPv6IP tunnel. 3. Configure the first of two circuits with these characteristics: a. b. c. Choose a name for the circuit (in this example, tunnel_m). CBS# configure circuit tunnel_m Associate the circuit with the VAP group (in this example, vg4) CBS(conf-cct)# vap-group vg4 Assign an IPv6 address to the circuit (in this example, FD00:DE00:AAAA:5:0:0:0:5/48).
NOTE: Unlike the IPv6 addresses that are associated with 6to4 and ISATAP tunnels, this address has no pre-defined prefix and does not contain any encoding of the IPv4 source-address. CBS(conf-cct-vapgroup)# ip FD00:DE00:AAAA:5:0:0:0:5/48 %WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit CBS(conf-cct-vapgroup)# end 4. Configure a second circuit with these characteristics:
Choose a name for the circuit (in this example, tunnel_n). CBS# configure circuit tunnel_n
223
Associate the circuit with the VAP group (in this example, vg4) CBS(conf-cct)# vap-group vg4 Assign the local IPv4 address that is being used by the VAP group to send traffic to the external IPv6 network. CBS(conf-cct-vapgroup)# ip 172.64.50.60/24 CBS(conf-cct-vapgroup-ip)# end
5.
Configure the GRE tunnel. CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 source-address 172.64.50.60 destination-address 172.64.50.61
6.
Optionally, configure a time-to-live value for packets sent through the tunnel. Values range from 1 to 256 seconds, and the default is 64. CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 time-to-live 120 CBS#
7.
Optionally, enable Maximum Transmission Unit (MTU) discovery for the path that the tunnel uses. Discovery of the MTU helps to avoid fragmentation. CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 path-mtu-discovery CBS#
224
20
Backups, Upgrades, and Reinstallation
This chapter describes how to backup the configuration, upgrade XOS hardware, and reinstall XOS software. This information is provided to assist with upgrading to 86xx modules so that migration to XOS V9.x can take place. You cannot use the upgrade or reinstallation procedures to upgrade to XOS V9.x. The major topics in this chapter include:
z z z z z
VAP Backup on page 225 Preparing for Safe Upgrade on page 226 Upgrading to CPM-8600 or APM-8600 Modules on page 229 Creating a New Configuration Database on page 230 Reinstallation on page 230
VAP Backup
The Backup tool can backup everything under the specified directory tree for a CPM or all members of a VAP group. This information is tarred and zipped, with the output being placed in /root. To use the Backup tool, complete the following: 1. Log in as root. CBS# unix su Password: [root@xxxxx admin]# 2. Enter the following command at the root prompt. /usr/os/bin/archive_dir -v <vap-group-name> -d <directory-name> Command line options: -v <vap-group-name> -d <directory-name> -h -V If the -v option is used, you must specify the <vap-group name>. If the -v option is not used, then the CPM will be archived. Default = /usr/os (the directory may show as /crossbeam; it is the same as / usr/os) Help version
If you do not provide any options at the command line, the Backup tool runs in interactive mode and prompts for the VAP group name, directory to be backed up, and archive name. If you leave VAP group name blank, the directory from CPM is backed up. In backing up a VAP group, enter the path as it would show while logged in to a VAP. To restore a file: 1. 2. Unzip the file gzip d <targz_file> Move to: /tftpboot/[<vap-group-name>_<index>]
3.
Untar the file: tar xvf <tar_file> If restoring a VAP group directory, repeat this process for all the VAP group indexes, including the <vap-group-name>_common directory.
NOTE: If you backed up a <vap-group-name> / without specifying another directory, you may see a Cannot mkdir: No such file or directory message. This can be ignored.
Preparing for Safe Upgrade

The Automated Workflow System can be used to upgrade from XOS V9.0 to XOS V9.x. See XOS Automated Workflow System on page 195 for more information. You cannot perform an upgrade to XOS V9.0. Upgrading to XOS V9.0 from pre-V9.0 can only be done using the Migration process. See XOS Migration on page 204 for more information. The Safe Upgrade feature partitions the disks on the CPMs into two distributions so that two different XOS configurations can be installed at the same time, and the CPM can boot in either distribution. This allows you to install and use a later XOS release while keeping the existing release. It also allows two different configurations using the same XOS release. Dividing the CPM hard disk into 2 distributions does not degrade system performance. The Automated Worflow System (AWS) provides a simple procedure for preparing for an upgrade, and upgrading your XOS version. In the case where an upgrade fails, having a snapshot of your system also allows for an easy rollback using AWS. AWS automatically performs prerequisite checks and notifies you of any incompatibilities. All backup procedures are then performed automatically. To perform the safe upgrade process prior to an XOS upgrade, use the following procedure. 1. Open the Automated Workflow System. Command: CBS# automated-workflow-menu Welcome to the X-Series Platform Automated Workflow System! Version: 1.1.0-11 1. 2. 3. 4. 5. Configure XOS... Upgrade XOS software and firmware... View system configuration and status... Applications... Custom...
Select a submenu to view available automated workflows. Enter x to exit or ? for help. 2. Select 2. Upgrade XOS software and firmware... Please Enter Selection: 2 Automated Workflow Menu: Upgrade XOS software and firmware 1. 2. 3. 4. Upgrade XOS software and install recommended firmware Prepare system for a possible rollback Install XOS firmware only Roll back XOS software and firmware
226
5. Verify XOS software and firmware compatibility Select a task to execute an automated workflow. Enter b to go back, x to exit or ? for help. 3. Choose 2. Prepare system for a possible rollback Please Enter Selection: 2 Automated Workflow: Prepare system for a possible rollback Press ? for assistance on questions Analyzing system configuration... VRRP State : non-master CPM Redundancy Status : single Current XOS Software Release : 9.5.x-67 All external management interfaces will be shut off. Please use the console interface to monitor the system. WARNING: Rollback preparation will automatically reload the chassis! Begin rollback preparation now (y/n)? [n]: y Once the rollback (safe upgrade) procedure completes, return to the main AWS menu and perform the XOS upgrade.
Manual Safe Upgrade Procedure

In a single CPM system, take the system offline. In a dual CPM system, the system can remain active. To partition the CPM disk drive for the safe upgrade feature, you need to run in single user mode which requires a console connection to CPM. If you have a Dual Box High Availability configuration, repeat this procedure on each system.
Prerequisites
z
A minimum 80GB hard drive on the CPM.
Perform the following steps: 1. 2. 3. 4. 5. 6. Reboot the CPM. Watch carefully for the GRUB menu. At the GRUB menu, type p on the keyboard then enter the password, x40rocks. At the GRUB prompt, type e to edit. Change the line number by pressing the down arrow until you reach the grub line starting with kernel. Add single to the end of this line, and press Enter to save the edit. Type b to boot the CPM in single user mode. Run the following command to copy from distribution 1 (d 1) to distribution 2 (d 2): /usr/os/bin/xos-copy-dist -d 2 Everything on the running partition is copied to the other partition. This could take up to two hours. 7. Reboot the CPM. The GRUB boot loader will show the option to boot from the second distribution.
227
8.
In dual CPM system, repeat this procedure to partition the disk in the second CPM. However, the CPM being partitioned must not be monitored by the other CPM for redundancy. To prevent monitoring, enter: CBS# cp-unknown-state {cp1|cp2} ignore For example, the following command forces CP2 to take no action while CP1 is being partitioned: CBS# cp-unknown-state cp2 ignore You would repeat this step when the other CPM is being partitioned, for example: CBS# cp-unknown-state cp1 ignore After the partitioning, you need to have the CPMs monitor each other for redundancy: CBS# cp-unknown-state {cp1|cp2} monitor
To change or list the kernel from either partition (d1 or d2) to be used on next boot: /usr/os/bin/xos-toggle-boot {d1|d2} When using ./xos-toggle-boot from the /crossbeam/bin directory to switch distributions and boot into another version of XOS, use ./xos-toggle-boot -l to see all the kernel versions available. When you are in XOS V8.x, V9.x will not be specified in the list. The kernel will be available, but XOS V8.x will not recognize it as V9.x. Always choose the first kernel listed without an XOS version number. Under XOS V8.x: [root@xxxxx bin]# ./xos-toggle-boot -l System is running on Distribution 1(Release 8.5.0) Default kernel for next boot is Distribution 1 1: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0) Available kernels: 0: Linux (2.4.21-47.EL) (Distribution 1 - 8.5.0) 1: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0) 2: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0) 3: Linux (2.6.18-53.el5_64smp) (Distribution 2) <-- Choose this kernel 4: Linux (2.6.18-53.el5_64smp) (Distribution 2) [root@TechPubs-X45 bin]# Under XOS V9.x: [root@xxxxx bin]# ./xos-toggle-boot -l System is running on Distribution 2 (Release 9.x.x) Default kernel for next boot is Distribution 2 3: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x) Available kernels: 0: Linux (2.4.21-47.EL) (Distribution 1 - 8.5.0) 1: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0) 2: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0) 3: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x) 4: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x) [root@xxxxx bin]# To change the default kernel from the CLI, use the cp-next-boot {cp1|cp2} distribution x command, where x is 1 or 2, then reboot. The show cp-next-boot CLI command displays which partition will be used to boot the CPM the next time the CPM boots.
228
NOTE: If you are using xos-copy-dist to copy from D2 to D1, the default boot disk will be D2 at the completion of the command. This is not the case if you are copying from D1 to D2.
Partitioning the CPM Hard Disk

You can divide the CPM hard disk into two partitions and use the xos-copy-dist command. Creating a second partition provides the following benefits:
z
Enable the Safe Upgrade feature This feature enables you to install a future upgrade on one partition of the CPM hard disk, while your existing XOS version and configuration remain on the other partition. You can then choose to boot the CPM in either partition. This enables you to try a new version while having the option to easily return to your existing version in case of issues.
z z
Run the same XOS version on both partitions and use the partitions to try different configurations. Create a backup of the partition.
NOTE: Dividing the CPM hard disk into 2 partitions does not degrade system performance. For a detailed partitioning procedure, refer to Manual Safe Upgrade Procedure on page 227. NOTE: If you are using xos-copy-dist to copy from D2 to D1, the default boot disk will be D2 at the completion of the command. This is not the case if you are copying from D1 to D2.
Upgrading to CPM-8600 or APM-8600 Modules

The following instructions outline the steps needed to upgrade to CPM-8600 or APM-8600 hardware. NOTE: If installing CPM-8600s to replace earlier CPMs and you want to enable the RAID feature on the new CPMs (assuming the boards have dual disk drives installed), or to reconfigure the RAID setting on previously installed CPM-8600s, you must perform a reinstallation. Refer to Reinstallation on page 230.
Upgrade a CPM-8400 to a CPM-8600

NOTE: XOS V9.x requires that a CPM-8600 have 4 GB of memory To upgrade your CPM-8400 to CPM-8600, complete the following steps: 1. If you currently have CPM redundancy configured, skip this step. If you have a single CPM, configure CPM redundancy to facilitate the upgrade process. Refer to Configuring CPM Redundancy on page 121 for more information. This process takes about eighty (80) minutes. NOTE: You must reload the chassis when changing from non-CPM redundancy to CPM redundancy. 2. 3. 4. 5. 6. Upgrade all existing CPM modules to XOS Version 8.1 or later. Insert the new CPM-8600 module into the secondary CPM slot. Refer to the X80 Platform Hardware Installation Guide for more information about inserting modules. After the synchronization is complete, halt the old CPM module and remove it from the chassis. This causes CPM failover to occur. The secondary CPM (the 8600 module) becomes the primary CPM. Insert a new CPM-8600 module. Configure this CPM-8600 module as the backup.
Upgrade an APM-8400 to an APM-8600

NOTE: A single VAP group cannot contain APM-8400 modules and AMP-8600 modules, except during the upgrade process described below.
229
The following procedure assumes there are two empty slots. To upgrade APM-8400 to APM-8600, complete the following steps: 1. 2. 3. 4. Upgrade all existing APM modules to XOS Version 8.1 or later. Install the first new APM-8600 into an empty APM slot. Refer to the X80 Platform Hardware Installation Guide for more information about installing modules. Configure the APM-8600 into the VAP group. Bring down an APM-8400 module using the following command: configure module <slot-number> maintenance This command automatically initializes an APM-8600 into the VAP group. NOTE: For more information about the use of maintenance mode, see Maintenance Mode on page 297. 5. 6. Remove the APM-8400 that is in maintenance mode and replace it with another APM-8600. Repeat steps 3 through 5 for each APM-8400 until all new members in the VAP group are APM-8600 modules.
Creating a New Configuration Database

Caution: The following command erases all configuration data, followed by a system reboot going through the first-time boot sequence.
To create a new configuration database and delete the old which includes deleting all VAPs, use the following command: CBS# reset-configuration
Reinstallation
Reinstallation is accomplished through the use of either the Install Server, or the USB Installer (USBI).For complete information on using the Install Server, refer to the Install Server User Guide. For complete information on using the USB Installer, refer to the USB Installer User Guide. If you have previously configured and run the Install Server, you may use the following procedure to re-install the XOS software. If you have not created your Install Server, refer to the Install Server User Guide. 1. From the system console on the CPM, reboot the CPM and issue an <Esc> <Ctrl-r> command during the reboot as follows: a. To reboot the CPM, issue the command: reload module <slot # x> Where x the slot number of your master CPM. b. At the start of the reboot process and during the reboot you will need to respond to prompts including: Do you want to save (your configuration) to startup-config? ... Proceed with reload? c. During the reboot process, there is a momentary pause with a message similar to the following: Crossbeam System, Inc. Bootstrap Complete Rev x.x.x.x This is described as the spinning | prompt. d. At the spinning | prompt, press <Esc> <Ctrl-r>.
230
e.
From the system console, use the following command to boot the XOS diskless CPM: ROM> boot net The boot net command sends out a broadcast message to locate the Install Server. The X-Series Platform is forced to boot from the Install Server, using the diskless partition with the matching MAC address.
2. 3.
From the system console, login as root. The default password is admin. Install the XOS system software, as follows, where <system dirname> is the directory name containing the system software release to be installed. # /usr/os/bin/install-xos /common/<system dirname> [-r1] NOTE: The -r1 is a RAID option. Prerequisites and RAID details are provided in RAID-Related Hard Drive Configuration and Repair on page 241. RAID options can only be enabled on a CPM during an install or reinstallation, as follows: default, no switch included, the system writes only to the primary drive. -r1, dual disk drives in a CPM-8600, the system writes identical data to both drives, creating a complete backup in case of one drives failure. You can use the RAID-1 option (-r1) in combination with the CPM SYNC feature.
4.
When the install-xos script completes, enter reboot at the prompt. The CPM then reboots from its own disk and prompts you for a login name and password.
Changing the Disk Partitioning Scheme

Begininning with XOS V9.5.0, you can change the disk partitoning scheme on the CPM using the cp-disk-scheme command. The CPM supports partitions of 80, 120, 250, and 500 Gigabytes. cp-disk-scheme 500 The partitioning scheme that you select must be equal to or smaller than your CPM disk size and must be larger than the current partitioning scheme. If you want to configure cp-redundancy between two CPMs, both must have the same disk partitioning scheme. After you configure a new partitioning scheme, you must reboot the CPM. Use the show cp-disk-scheme command to see the current and the configured disk partitioning schemes. CBS# show cp-disk-scheme Current scheme: 250GB Configured scheme: 500GB
231
232
A
In-Service Upgrade
This appendix describes In-Service Upgrades (ISU), an alternate method of upgrading XOS software while minimizing downtime. It includes the following major topics:
z z z z z z
In-Service Upgrade (ISU) Overview on page 233 In-Service Upgrade Process Overview on page 234 Modes of In-Service Upgrade on page 234 Understanding Optimal Batch Configuration on page 235 Executing a Sample In-Service Upgrade on page 236 Executing a Non-Interactive In-Service Upgrade on page 240
In-Service Upgrade (ISU) Overview

You cannot perform an In-Service Upgrade to bring your X-Series Platform up to XOS V9.x. You must use the Migration process. See XOS Migration on page 204 for more information. The process of an ISU virtually splits the chassis in half. One half of the chassis is responsible for forwarding traffic, while the other half is upgraded. The first step in the process is upgrading the secondary CPM to the new version of XOS software and rebooting into primary mode. During this time, there will be no interruption in traffic. Once the upgraded CPM has initialized as a primary CPM, APMs and NPMs will be moved in batches to the half of the chassis controlled by the upgraded CPM. The batches of APMs and NPMs will be grouped to minimize downtime and impact to traffic flows. The final stage of ISU includes upgrading the old primary CPM and reloading it into secondary mode. NOTE: During the upgrade, the NPMs and APMs are rebooted by the system.
Prerequisites
The ISU feature has several requirements for successful completion, including redundant CPMs, APMs, and NPMs. The following are the minimum prerequisites to perform an In-Service Upgrade.
z z z z z
XOS V8.0 or later. Two CPMs. Two (or more) APMs per VAP group are recommended. Circuits configured with redundancy across two (or more) NPMs are recommended. If needed for applications, configure Application Monitor.
Resolve any issues listed below before starting In-Service Upgrade.

z z z z
Verify that CPM redundancy is enabled and that CPMs are synchronized. Note that CPM synchronization may take between 45 and 90 minutes depending on your systems configuration. Verify that both CPMs are in election mode. Verify that the current boot partition is the same as cp-next-boot setting. Verify that adequate disk space (greater than 2GB) is available.
233
Limitations
The following are limitations of In-Service Upgrade:
z z z z z z z
Applications are not upgraded during the process only the XOS operating system. Dynamic routing tables are not preserved during upgrade. The XOS chassis does not operate at full capacity during the upgrade. No CPM redundancy exists during the upgrade. There is no state sharing between the upgraded and un-upgraded CPMs within a chassis. The configuration database is locked during the In-Service Upgrade. Interfaces without redundancy are not supported.
In-Service Upgrade Process Overview

The basic steps for In-Service Upgrade are:
z z z z z z z z z z
Prepare for the upgrade Copy, unzip, set permissions, and unpack the shar file Enter In-Service Upgrade Configure batches Issue the install command through CLI Save system configuration Save the batch configuration Transition the secondary CPM into offline mode and upgrade Reload the offline CPM into primary state Move batches Upgrade the old CPM
Modes of In-Service Upgrade

In-Service Upgrade has two modes; Interactive and Non-Interactive. Interactive mode allows the user to control the timing of each step during upgrade. Interactive mode is the default mode. Non-Interactive mode executes the upgrade with no user input. The Interactive In-Service Upgrade can be aborted at any time by entering Ctrl-Y or answering N at any prompt in the upgrade interview process. Once the In-Service Upgrade is aborted, all APMs and NPMs are transitioned back to the original state. Non-Interactive In-Service Upgrades can be executed from either the CLI or from the XOS Graphical User Interface. Non-Interactive In-Service Upgrades begun using the XOS Graphical User Interface are executed automatically using the default settings. Those Non-Interactive In-Service Upgrades begun through the CLI can use either default or custom settings. The Non-Interactive In-Service Upgrade can be aborted at any time. To end a Non-Interactive In-Service upgrade, switch to an Interactive In-Service Upgrade by executing the CLI command upgrade in-service install <XOS version> and then entering Ctrl-Y. NOTE: If there is a possibility that the CPM may need to be rolled-back to the previous version, Safe Upgrade is the recommended procedure. For more information on Preparing for Safe Upgrade on page 226. NOTE: If the show cp-redundancy command is executed during an In Service Upgrade, the following warning message will be displayed: WARNING: In-Service Upgrade Is in Progress.
In-Service Upgrade 234
Understanding Optimal Batch Configuration

In-Service Upgrade (ISU) uses the concept of batches to collect APMs and NPMs into groups. A batch consists of one or more APMs or NPMs selected to be taken out of active service and upgraded to the new XOS software. These batches are automatically created through the Configuration Analyzer. The results of the Configuration Analyzer should be reviewed in the context of your current system topology. As APMs and NPMs are upgraded and transitioned to the new CPM, there is a potential for interruption in traffic. By grouping specific APMs and NPMs into specific batches, the disruption can be minimized. The default batch configuration can be modified as required to a maximum of ten custom batches.
Sample Configuration Analyzer Output

This section describes the typical Configuration Analyzer output. Batch1 will consist of one half of the APMs associated with each VAP group, excluding the master VAP. These APMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. Any flows currently processed by the APMs in this group will fail-over to the remaining APMs for that VAP group during the transition as defined by the flow-rules for that group. By design, all down and standby VAPs are moved as part of Batch1. Batch2 will consist of one half of the NPMs associated with redundant circuits. These NPMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. Any flows processed by the NPMs in this group will fail-over to the remaining NPMs during the transition. Optimally, NPMs controlling LACP circuits, bridges and active redundancy-interfaces will be processed during Batch3 to minimize traffic interruption. Batch3 will consist of the remaining NPMs. These NPMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. At the start of this batch, all active flows will be transitioned to the upgraded half of the chassis. There will be an interruption in traffic due to this transition. Batch4 will consist of the remaining APMs in the chassis. Theses APMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. There is no interruption in traffic during this transition. Once all APMs have initialized on the upgraded half of the chassis, the system will be operating at full capacity on the upgraded software.
Creating In-Service Upgrade Batches

There are two methods for batch creation Configuration Analyzer and custom batch lists.
Configuration Analyzer
The Configuration Analyzer creates a default batch list using a predefined logic. Review the Understanding Optimal Batch Configuration on page 235 before proceeding. The default batch list can be viewed by executing the following commands: CBS# upgrade in-service CBS(in-service-upgrade)# show default-batches To use the default batch list, execute the following command to set the batch list: CBS# upgrade in-service CBS(in-service-upgrade)# batch-default
235
Standby and down modules are the first to be moved since these modules are not passing traffic. These modules are placed in the first batch, which cannot be modified. To view the modules in this batch, enter the following commands: CBS# upgrade in-service CBS(in-service-upgrade)# show standby-modules
Custom Batch List

A custom batch list can be created if required. It is important to understand the impact specific APMs and NPMs have on traffic flows before assigning a slot to a batch. Review Understanding Optimal Batch Configuration on page 235 before proceeding. Before defining custom batches, clear the current batch list using the following commands: CBS# upgrade in-service CBS(in-service-upgrade)# clear-batches Once the batches have been cleared, you can define the custom batch configuration. Any NPMs and APMs that are not assigned to a specific batch will be placed in a default batch and moved after the last defined batch has been moved. The following example assigns np2 (slot 2) to Batch2: CBS(in-service-upgrade)# batch-2 np2 Batch2: np2(2) The following example assigns multiple slots to a batch: CBS(in-service-upgrade)# batch-1 fw_1 fw_2 Batch1: fw_1 fw_2 Batch2: np2(2) NOTE: A slot can only be assigned to one batch. If moving a slot that is already defined in another batch, remove the slot from the previous batch before assigning it to the new batch.
Executing a Sample In-Service Upgrade

This section describes a sample In-Service Upgrade that uses a four-batch scenario. The following illustration represents the state of the sample XOS chassis before the upgrade begins.
Figure 19.
Sample Chassis Before In-Service Upgrade
In-Service Upgrade
236
The example configuration contains a single VAP group consisting of two members and redundant circuits split across NPM1 and NPM2. 1. The shar file contains the files needed to execute the upgrade. These files must be copied to the correct location (/usr/os/rpm), unpacked, and have the correct permissions applied (chmod 777). This step must be performed on the primary CPM only. For more information, see Backups, Upgrades, and Reinstallation on page 225. At the unix prompt on one of the CPMs, execute the following commands: [root@xxx admin]# echo isu_child_tmo=10800 >> /usr/os/etc/cbscprmd.cf [root@xxx admin]# killall -HUP cbscprmd 3. 4. Repeat the previous step for the other CPM. Begin an Interactive In-Service Upgrade as shown below. The command batch-default clears any custom batches, and uses only system generated batches. CBS# upgrade in-service CBS(in-service upgrade)# batch-default Batch1: FW_1(3) Batch2: np2(2) Batch3: np1(1) Batch4: FW_2(4) CBS(in-service upgrade)# install <XOS version> The system displays informational messages regarding verifications and checks. Correct any issues before proceeding with the In-Service Upgrade. 5. Save the configuration to ensure that the upgraded half of the chassis is synchronized with the running configuration. Any unsaved configuration will be lost. Do you want to save it to startup-config? <Y or N>[Y]: Enter Y to save the configuration. 6. The current batch configuration is displayed as shown in the following example: Do you want to save the current batch configuration? <Y or N>[Y]: Batch1: fw_1(3) Batch2: np2(2) Batch3: np1(1) Batch4: fw_2(4) Done with batches? <Y or N> [Y]: Enter Y to indicate that you are done with batches or enter N to reconfigure the batches. NOTE: Please review the batch order, it cannot be changed during the In-Service Upgrade following completion of this step. 7. To continue the In-Service Upgrade, the secondary CPM will be reloaded into offline mode and upgraded, enter Y at the system prompt to continue. Proceed with install? <Y or N>[Y]: NOTE: The system automatically copies the shar file to the new CPM, the new CPM goes offline and performs the upgrade. This step takes about 30 minutes to complete. During this time, a series of informational messages are displayed to indicate progress. 8. You are prompted to bring the other CPM to Primary. Enter Y to continue. Continue with in-service upgrade to bring other CP to Primary? <Y or N>[Y]: NOTE: The new CPM will be rebooted into Primary CPM mode running the upgraded XOS software. 9. You are prompted to execute the next batch. This is the first batch to be moved that consists of the resources configured in Batch1. Continue with in-service upgrade to execute next batch? <Y or N>[Y]:
2.
237
The module in Batch1 is moved from the old CPM to the new CPM as shown in the following illustration.
Figure 20.
Batch1 Operation
NOTE: When all resources in the batch have been successfully moved to the upgraded half of the chassis, a successful completion message is displayed. The resources are now running the upgraded software, but are not processing traffic. 10. You are prompted to continue with the next batch. This is the second batch consisting of the resources configured in Batch2. The following illustration shows Batch2 moved to the new CPM.
Figure 21.
Batch2 Operation
NOTE: During this batch, the first NPMs are moved to the upgraded half of the chassis. After this transition, the interfaces are active, however, traffic is not processed by the new half of the chassis. Restart applications that do not tolerate an interface state transition after completion of this batch. 11. You are prompted to continue with the next batch. This is the third batch consisting of the resources configured in Batch3. The following illustration shows Batch3 moved to the new CPM.
In-Service Upgrade
238
Figure 22.
Batch3 Operation
NOTE: During this step, the second half of the NPMs are transitioned to the upgraded half of the chassis. This transition triggers the traffic flows to move from the old half to the new half of the chassis. All traffic is then processed by the upgraded half of the chassis and full interface redundancy is restored. 12. You are prompted to continue with the next batch. This is the fourth batch consisting of the resources configured in Batch4. The following diagram shows Batch4 moved to the new CPM.
Figure 23.
Batch4 Operation
NOTE: During this step, the remaining APMs transition to the new half of the chassis. This restores full APM capacity to the chassis. 13. Choose an option to complete or abort the In-Service Upgrade:
z z z
Option 1 aborts the In-Service Upgrade transitioning all NPMs and APMs to the old CPM and transitioning the new CPM offline. An interruption in service occurs during this transition. Option 2 transitions the old CPM into offline state, however the In-Service Upgrade remains in progress. Option 3 results in the old CPM upgrading to the new software and reloading into CPM secondary mode. This restores full redundancy to the chassis running the new software.
239
Executing a Non-Interactive In-Service Upgrade

A Non-Interactive In-Service Upgrade performs the same steps as an Interactive In-Service Upgrade. After you have confirmed the batch configuration and confirmed that you want to continue with the upgrade, there is no further interaction required. The Non-Interactive In-Service Upgrade continues automatically from that point. To execute a Non-Interactive In-Service Upgrade: 1. The shar file contains the files needed to execute the upgrade. These files must be copied to the correct location (/usr/os/rpm), unpacked, and have the correct permissions applied (chmod 777). For more information, see Backups, Upgrades, and Reinstallation on page 225. At the admin prompt on one of the CPMs, execute the following commands: [root@xxx admin]# echo isu_child_tmo=10800 >> /usr/os/etc/cbscprmd.cf [root@xxx admin]# killall -HUP cbscprmd 3. 4. Repeat the previous step for the other CPM. Begin an Interactive In-Service Upgrade as shown below: CBS # upgrade in-service CBS(in-service upgrade)# install <XOS version> non-interactive The system displays informational messages regarding verifications and checks. Correct any issues before proceeding with the In-Service Upgrade. 5. When the dependency checks are complete, you are prompted to save the system configuration. Save the configuration to ensure that the upgraded half of the chassis is synchronized with the current configuration. Do you want to save it to startup-config? <Y or N>[Y]: Enter Y to save the configuration. 6. The current batch configuration is displayed as shown in the following example: Batch1: Batch2: Batch3: Batch4: FW_1(3) np2(2) np1(1) FW_2(4)
2.
Review the batch order. Once the batch order is agreed to, it cannot be modified during the rest of the In-Service Upgrade. Enter Y to indicate that you are done with batches or enter N to reconfigure the batches. For more information, see Custom Batch List on page 236. 7. You are then prompted to continue with the upgrade, CLI sessions will be terminated, and there will be a loss of CPM redundancy if you continue at this point. Enter Y to continue.
The Non-Interactive In-Service Upgrade continues automatically.
Aborting a Non-Interactive In-Service Upgrade

To abort a Non-Interactive In-Service Upgrade, close all open CLI sessions. Then perform the following: 1. 2. 3. Start a new CLI session. Execute the following command: CBS # upgrade in-service install <XOS version> Press CTL-Y, once in the interactive session to execute the In-Service Upgrade abort.
In-Service Upgrade
240
B
RAID-Related Hard Drive Configuration and Repair
With the dual drive and RAID capabilities in the APM-8600, APM-8650, APM-9600, CPM-8600, and CPM-9600, there exists a number of possible scenarios for upgrading, repairing, replacing, swapping, and otherwise changing drive configurations in these modules. This appendix provides information and procedures that deal with the following scenarios:
z z z z
Going from a non-RAID configuration to a RAID configuration Going from one RAID configuration to a different RAID configuration Going from a RAID configuration to a non-RAID configuration Correcting RAID drive failures
NOTE: Configuring RAID 1 with only a single drive installed is not supported. This chapter contains the following major topics:
z z z z z z z z z z z z
RAID Feature Definitions on page 242 Additional Documentation on page 242 Prerequisites for RAID on page 242 Important RAID Information on page 243 Obtain Hard Drive Information on page 244 Error Situations and Error Messages on page 247 Configure RAID on an Existing Non-RAID System on page 248 Change the Existing RAID Setting on a CPM-8600 on page 250 Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group on page 251 Configuring Non-RAID on a CPM-8600 with Existing RAID 1 on page 251 Detect and Replace Failed Drives on an Existing RAID System on page 253 Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255
241
RAID Feature Definitions

There are three possible RAID settings:
z z
Non-RAID The system writes only to the primary disk (on CPM-8600s), or to the disks independently (on APM-8600s, APM-8650s, and APM-9600s). RAID 0 The APMs write to both drives in a process called striping. This arrangement produces maximum speed/efficiency of operation, but the data is not duplicated and both disks are required to access the full content of the data. RAID 1 The CPMs or APMs write the same data to each disk at the same time, providing a complete, ongoing backup in case of one drives failure. You can use the RAID 1 option in combination with the CPM SYNC feature.
Additional Documentation
This appendix only covers RAID configuration. For related information, refer to the following information:
z z
For XOS and module upgrades, refer to the upgrade sections of this configuration guide. For information about removing and reinserting CPM and APM modules and physically inserting hard drives, refer to the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide. For Install Server information, refer to the Install Server User Guide. For information on using the USB Installer, refer to the USB Installer User Guide.
z z
Prerequisites for RAID

This section provides hardware, software and other requirements to use RAID on an XOS system.
z z z z
CPM-8600s, CPM-9600s, APM-8600s, APM-8650s, and/or APM-9600s (with various RAID and non-RAID standard configurations). RAID 0 or RAID 1 requires two hard drives of the same capacity. Console connection to issue CLI and UNIX commands to modules. One of these methods of installing XOS software

Install Server Software with diskless partition that allows the CPM-8600 to boot from the server The latest XOS USB Installer, plugged into the CPM USB port.
RAID-Related Hard Drive Configuration
242
Important RAID Information

The following sections contain important general and specific RAID information and general information about RAID issues and errors. For more information about error situations and error messages, refer to Error Situations and Error Messages on page 247.
Disk Drive and RAID Information for CPM-8600s, APM-8600s, and APM-8650s
The following information will help you understand the various RAID scenarios explained in this document.
z
On CPM-8600s, APM-8600s, and APM-8650s, the two drive slots are labeled SATA1 (the inside slot) and SATA2 (the outside slot). Refer to the hard drive diagram in Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255. If you are installing only a single drive into a CPM-8600 or APM-8600, install the drive into the SATA1 position (inside slot).
Disk Drive and RAID Information for CPM-9600s and APM-9600s

The CPM-9600 comes from the factory with two disk drives configured for RAID 1. Only RAID 1 is supported on the CPM-9600. The APM-9600 comes from the factory with no disks installed and can operate without a disk, with a single disk, or with two disks configured for either RAID 0 or RAID 1. Choose the configuration based on the requirements of your application. NOTE: The APM-9600 uses serial SCSI (SAS) disks.
RAID Information for CPM-8600s Only

The following information applies to RAID configurations on a CPM-8600:
z
To install RAID, change the RAID setting to a different RAID type, or uninstall RAID, you must do an XOS reinstallation. The RAID/non-RAID setup is part of the installation process and part of the installation command. If you are converting to a RAID configuration from non-RAID, you must add a blank drive to the SATA2 drive slot. If you are converting back to non-RAID from a RAID configuration, Crossbeam recommends that you remove the SATA2 hard drive. If you do not remove the second drive, and later the non-RAID SATA1 drive fails, you must move the SATA2 bootable drive to the SATA1 position. If you replace the failed SATA1 drive with a blank drive, the CPM-8600 will mistakenly boot from the original SATA2 RAID 1 drive, or display an error message.
z z
Always use a blank drive to repair a failed RAID 1 drive situation. This ensures that the system will boot from the existing, working RAID 1 drive as intended.
RAID Information for CPM-9600s Only

The CPM-9600 comes from the factory with two disks installed in a RAID 1 configuration. Only the RAID 1 configuration is supported.
243
Important RAID Information for APM-8600s, APM-8650s, and APM-9600s

The following information applies to RAID configurations on an APM-8600:
z
On an APM-8600, APM-8650, or APM-9600, RAID (or non-RAID) is configured for an entire VAP group, not for individual APMs, and is set up using the configure vap-group command, not by XOS reinstallation. When an installed APM-8600, APM-8650, or APM-9600 becomes an active part of the VAP group, its drives, if present, are automatically set up for the RAID operation established for that VAP group.
NOTE: APM-8600s, APM-8650s, and APM-9600s cannot be configured in the same VAP group. Crossbeam recommends that all APMs in a VAP group have the same configuration.
Obtain Hard Drive Information

Use the following commands and resources to help with RAID configuration and operation.
Table 20.
RAID Configuration and Operation Commands

Used For Shows whether there is a Hard disk and a Second hard disk and if there have been errors on either disk. Shows number of drives in each module. The /proc/mdstat file provides information on the status of the disk drives and RAID. Refer to Using mdstat to Determine RAID-Related Drive Information on page 244.
Command or Tool show module status show module status disk cat /proc/mdstat
Using mdstat to Determine RAID-Related Drive Information

The file /proc/mdstat provides status on the drives in a RAID pair, and whether or not they are in sync. The following file was generated by an APM-8600 with two hard drives set up for RAID 1 operation: Some of the file entries (shown in bold below) indicate:
z z
Type of RAID set up (raid1) Names, numbers, and positions of the drives as follows: sda which is SATA1, the inside drive slot sdb which is SATA2, the outside drive slot Refer to the diagram in Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255.
Total number of drives in the RAID group ( [2/2] ) v3_2 (cpm): root$ cat /proc/mdstat Personalities : [raid0] [raid1] read_ahead 1024 sectors Event: 3 md1 : active raid1 sdb1[1] sda1[0] 93433920 blocks [2/2] [UU]
244
md5 : active raid1 sdb5[1] sda5[0] 2104448 blocks [2/2] [UU] md6 : active raid1 sdb6[1] sda6[0] 2104448 blocks [2/2] [UU] ...... unused devices: <none> NOTE: MD1 is always configured as raid1, regardless of the RAID0 or RAID1 configuration.
z
An indication that both drives are in sync: [UU] or, as in the following partial example, one drive is degraded [_U] (not in sync or failed): md1 : active raid1 sdb1[1] sda1[0] 93433920 blocks [2/1] [_U] [===>.................] recovery = 17.8% (16638196/93433920) finish=127.9
NOTE: If a RAID drive has failed, the output of the cat /proc/mdstat command includes (F) next to the failed drive information. For example: Personalities : [raid1]md0 : active raid1 sda1[0] sdb1[1](F) 71681920 blocks [2/1] [U_] In this example, the [U_] indicates that sdb1 is either degraded or has failed. The (F) indicates that it is failed, not degraded. The following example shows a CPM-8600, previously set up for RAID 1. One drive failed and was replaced (Part 1) but the drives are not synchronized. The xos-raid-add command is issued (Part 2) and the system begins the synchronization process for the two drives. In the example, md1 is 47% synchronized (Part 3). Part 1: New drive inserted but drives are not synchronized [root@mysysCPM2 admin]# cat /proc/mdstat Personalities : [raid0] [raid1] read_ahead 1024 sectors Event: 5 md1 : active raid1 sda1[0] 104320 blocks [2/1] [U_] md5 : active raid1 sda5[0] 505920 blocks [2/1] [U_] md6 : active raid1 sda6[0] 1003904 blocks [2/1] [U_] md7 : active raid1 sda7[0] 8008256 blocks [2/1] [U_] md8 : active raid1 sda8[0] 28001152 blocks [2/1] [U_] unused devices: <none> Part 2: Second drive is added to the RAID1 pair [root@mysysCPM2 admin]# /usr/os/bin/xos-raid-add Duplicating firsSCSI device sdb: 195371568 512-byte hdwr sectors (100030 MB) t drive partition Table... SCSI device sdb: 195371568 512-byte hdwr sectors (100030 MB) Hot adding secondary drive... /dev/md1 /dev/md5 /dev/md6 /dev/md7 /dev/md8
245
Part 3: Check synchronization progress, md1 shows 47% complete [root@mysysCPM2 admin]# more /proc/mdstat Personalities : [raid0] [raid1] read_ahead 1024 sectors Event: 11 md1 : active raid1 sdb1[2] sda1[0] 104320 blocks [2/1] [U_] [=========>...........] recovery = 47.0% (50096/104320)
246
Error Situations and Error Messages

If, when configuring or repairing RAID and non-RAID instances, you do not use blank disk drives as recommended, or fail to follow other recommended procedures, there are a number of error situations that can occur. Refer to the table below for explanations and remedies.
Table 21.
SATA1
RAID Error Messages

SATA2 Error Message Recovery Information
CPM-8600 Drive Mismatch Situations: Disk config mismatch RAID 1 non-RAID SATA1:RAID1 and SATA2:non-RAID SATA1:non-RAID and SATA2:RAID1 If you are configuring RAID, reverting to non-RAID, or changing to a different RAID setting, unless there is a drive you need to remove to preserve, just continue with the configuration process and the system will reconfigure the mismatched drive as needed.
non-RAID RAID 1
CPM-8600 Conversion from RAID to non-RAID without Removing the SATA2 Drive RAID 1 RAID 1 Warning Message (see text below)a If the system proceeds with a normal XOS installation, the first drive will become non-RAID and you will end up with a non-RAID/RAID 1 drive mismatch. So the system stops the reinstallation process with the warning message (detailed below the table). At this point, if you want to preserve SATA2, respond q to quit the install process, then remove the drive. Otherwise, respond y to continue the install and allow the rewrite. If the second drive was bootable, it will be made non-bootable. CPM-8600 RAID 1 Failed Drive and Missing Drive Situations Failed RAID 1 Replacing the drive with a non-blank drive will cause a mismatch error, or the CPM will boot from the wrong drive. You must replace the failed drive with a blank drive. If you place another bootable drive in the SATA1 position, the system will not boot off your good, SATA2 RAID 1 drive.
a. Warning message received when converting from RAID to non-RAID on a CPM-8600 and user fails to remove the SATA2 RAID-configured drive.
WARNING: This is a non-RAID CPM install and a second drive is present. To make the first drive bootable, we must rewrite the second drive's MBR to make it non-bootable. Respond 'y' to continue with the rewrite, respond 'n' to leave the MBR alone (you know it is non-bootable), or 'q' to quit the install and remove the drive. (y|n|q)? [y]
247
Configure RAID on an Existing Non-RAID System

This section explains how to change an existing, non-RAID, XOS V8.0 or later system, to a RAID configuration. This section assumes that you have CPM-8600s, CPM-9600s, APM-8600s, APM-9650s, and/or APM-9600s installed. NOTE: This section does not provide the instructions to physically remove the board, insert the hard drives, and reinsert the board. It also assumes that you understand how to take antistatic precautions. For these instructions, refer to the document that is included with the disk drive(s).
Adding a Second Drive to a CPM-8600 and Enabling RAID

To install RAID on an existing CPM-8600 you must perform an XOS reinstallation which requires shutting down the CPM-8600. Complete the following steps to remove a CPM-8600, insert its second drive, and perform the reinstall/RAID configuration. (The following procedure assumes you are using the Install Server.) 1. 2. 3. Log on to the console port as admin. If desired, back up the Systems Running Configuration using the CLI: CBS# copy running-config <file-name> FTP the configuration file to a workstation or server. Transfer the configuration file (stored by default at the following location) to an external storage device. /tftpboot/.private/home/admin/<file-name> 4. 5. Shut down the system from the console of the CPM-8600 using the CLI command: CBS# shutdown Remove the module and insert a blank hard drive into the SATA2 position (outside slot).
If you need physical installation instructions, refer to the more complete instructions in the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide. 6. When you reinsert the CPM-8600, the CPM-8600 boots as usual and you must stop the boot process so you can issue the install command. In this case, from the console, as soon as the Spinning Pipe prompt displays, type: <ESC><Ctrl-r> You will see the ROM> prompt. NOTE: If the CPM-8600 does not boot as usual, contact Crossbeam Customer Support. 7. At the ROM> prompt, you have two ways to install the XOS software:

Use an Install Server. See Installing XOS Using an Install Server, next. Use the Crossbeam USB Installer (USBI). See Installing the XOS Software Using the Crossbeam USB Installer (USBI) on page 249.
Installing XOS Using an Install Server

1. 2. 3. At the ROM> prompt, type boot net to start the XOS install process. Log in as root (there is no password). Install the XOS system software, as follows, where <system dirname> is the directory name containing the system software release to be installed. Enter the install command with the RAID 1 switch (-r1): # /usr/os/bin/install-xos /common/<system dirname> -r1 4. When the install-xos script completes, enter reboot at the prompt. The CPM-8600 then reboots from its own disk and prompts you for a login name and password.
248
Repeat these steps if you are installing RAID on a second CPM-8600. NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in sync with each other or to resynchronize the two CPM-8600s.
Installing the XOS Software Using the Crossbeam USB Installer (USBI)
1. 2. 3. At the ROM> prompt, type boot usb. When the menu appears, select either Install XOS Release 9.x or Later or Install XOS Release 8.x or 7.x, depending on the XOS version you want to install. The USBI boots the appropriate kernel version. Enter the login and password provided, accept the license if required, and follow the on-screen instructions:

Select the XOS version to install. Answer the hard disk configuration questions. (The USBI presents only valid hard disk options.) Install XOS. Installing from the USBI takes 10 to 20 minutes, on average.
4.
When the installation completes, select an option from the menu. Crossbeam recommends that you halt the CPM. When the system has halted, unseat the CPM, remove the USBI device, and then reseat the CPM to reboot it.
Warning: Do not remove the USBI while the CPM is booted from the device. Do not leave the USBI in the CPM during normal operation. Either action can compromise the integrity of the USBI device.
NOTE: After the system halts, you may see one or more error messages, and the Active LED on the CPM may blink. If you received the System halted message, you can ignore the errors and the blinking LED and safely reboot the CPM.
Adding Hard Drives to an APM-8600, APM-8650, or APM-9600 and Configuring RAID

Caution: Setting or changing the RAID settings on the VAP group deletes everything on the hard drives in that group.
To add drives and configure the APM-8600, APM-8650, or APM-9600 for RAID, complete the following steps: 1. 2. RAID is enabled on a VAP group-basis, not on individual APMs. Ensure that the APMs that you are going to configure for RAID are all part of a VAP group that will use the RAID feature. From the CLI, configure the desired RAID feature using one of the following settings: non-RAID Drives work independently, or device can work with only one drive. CBS# configure vap-group vapgroupname no raid RAID 0 Drives work at maximum speed and efficiency with different data written to each drive. Both working drives are required to access the data. CBS# configure vap-group vapgroupname raid 0 RAID 1 Drives work as a single drive with the same data written to both drives (data is duplicated). CBS# configure vap-group vapgroupname raid 1 3. Remove the APM and insert one or two disk drives, as required. If you need instructions for physically removing the modules and inserting the hard drives, refer to the document that is included with the disk drive(s).
249
4. 5.
After reinserting the APM, the next time that APM is used in the VAP group, the system detects the changes and reconfigures the drive(s) as needed for the new RAID setting. Continue this process until you have added the needed hard drives to each of the APMs.
Change the Existing RAID Setting on a CPM-8600

To change to a different RAID setting on a CPM-8600, follow the XOS reinstall process below. 1. 2. 3. Log on to the console port as admin. If desired, back up the Systems Running Configuration using the CLI: CBS# copy running-config <file-name> FTP the configuration file to a workstation or server. Transfer the configuration file (stored by default at the following location) to an external storage device. /tftpboot/.private/home/admin/file-name 4. 5. Reboot the system from the CPM-8600 using the CLI command: CBS# reload all You must be ready to stop the boot process. From the console, as soon as the Spinning Pipe prompt displays, type: <Esc><Ctrl-r> You will see the ROM> prompt. 6. At the ROM> prompt, you have two ways to install the XOS software:


Repeat these steps if you are installing RAID on a second CPM-8600. NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in synchronization with each other or to resynchronize the two CPM-8600s.
1. 2. 3. At the ROM> prompt, type boot usb. When the menu appears, select either Install XOS Release 9.x or Later or Install XOS Release 8.x or 7.x, depending on the XOS version you want to install. The USBI boots the appropriate kernel version. Enter the login and password provided, accept the license if required, and follow the on-screen instructions:
250
4.
NOTE: After the system halts, you may see one or more error messages, and the Active LED on the CPM may blink. If you received the System halted message, you can ignore the errors and the blinking LED and safely reboot the CPM.
Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group
Caution: Setting or changing the RAID settings on the VAP group deletes everything on the hard drives in that group.
To change the RAID setting on the APMs in a VAP group, complete the following steps: 1. From the CLI, configure the desired RAID feature setting to one of the following. non-RAID Drives work independently, or device can work with only one drive. CBS# configure vap-group <VAP_group_name> no raid RAID 0 Drives work at maximum speed and efficiency with different data written to each drive. Both working drives are required to access the data. CBS# configure vap-group <VAP_group_name> raid 0 RAID 1 Drives work as a single drive with the same data written to both drives (data is duplicated). CBS# configure vap-group <VAP_group_name> raid 1 2. (Change From one RAID Setting to Another) Reload the APMs so that the configuration takes effect. To minimize traffic interruption, reload slot-by-slot as follows: CBS# reload module <slot#> 3. To make this configuration permanent, save the configuration.
NOTE: If you convert from RAID to non-RAID, and leave the second hard drive installed, be aware that the second drive is accessed as /mnt/aplocaldrive_2. Refer to Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255.
Configuring Non-RAID on a CPM-8600 with Existing RAID 1

Use the following procedure to convert CPM-8600 modules back to non-RAID condition. You must perform these steps for each of the CPM-8600s. For APM-8600s, refer to Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group on page 251.
251
To convert from RAID to non-RAID on a CPM-8600, follow these steps: 1. 2. 3. Log on to the console port as admin. Back up the Systems Running Configuration (if desired) using the CLI: CBS# copy running-config <file-name> FTP the configuration file to a workstation or server or transfer the configuration file to an external storage device. By default, the configuration file is stored in the following location: /tftpboot/.private/home/admin/<file-name> 4. Shut down or reboot the system from the CPM-8600 using one of the following CLI commands:
Use the shutdown command if you want to remove the second hard drive from the CPM-8600. This is recommended. However, if you do leave the second drive in, the install process will make the drive non-bootable (after warning you). CBS# shutdown Remove the module and remove the second hard drive. If you need physical installation instructions, refer to the more complete instructions in the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide. document.
Use the reload all command if you choose to leave the second drive in the CPM-8600. CBS# reload all
5.
When you reinsert the CPM-8600 (or issue the reload all command) the module reboots. You must stop the boot process so you can issue the install command. From the console, as soon as the Spinning Pipe prompt displays, type: <Esc><Ctrl-r> You will see the ROM> prompt.
6.
At the ROM> prompt, you have two ways to install the XOS software:


Repeat these steps if you are installing RAID on a second CPM-8600. NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in sync with each other or to resynchronize the two CPM-8600s.
1. 2. At the ROM> prompt, type boot usb. When the menu appears, select either Install XOS Release 9.x or Later or Install XOS Release 8.x or 7.x, depending on the XOS version you want to install. The USBI boots the appropriate kernel version.
252
3.
Enter the login and password provided, accept the license if required, and follow the on-screen instructions:

4.
NOTE: After the system halts, you may see one or more error messages, and the Active LED on the CPM may blink. If you received the System halted message, you can ignore the errors and the blinking LED and safely reboot the CPM. 5. If your system has a second CPM-8600, repeat the above steps to convert it to non-RAID.
Detect and Replace Failed Drives on an Existing RAID System

The following sections explain how to restore a working RAID pair when a hard disk drive fails. IMPORTANT: To determine if an HDD is bad, and which drive is bad, refer to Using mdstat to Determine RAID-Related Drive Information on page 244. In RAID 1 configurations, you can replace only the failed drive. In RAID 0 configurations, replace both drives. Always replace a failed drive with a blank drive.
Replace Failed Drive on a CPM-8600 or CPM-9600 Configured for RAID 1

IMPORTANT: You must insert a blank replacement drive. If you insert a drive pre-configured for RAID 1, the system will not know which drive to sync to. Complete the following steps to replace a failed RAID 1 drive on a CPM-8600 and reestablish a working RAID 1 pair: 1. 2. 3. 4. Determine which drive on the CPM-8600 has failed by examining the file /proc/mdstat. Log on to the console port of the CPM-8600 with a failed HDD as admin. Back up the Systems Running Configuration (if desired) using the CLI: CBS# copy running-config <file-name> FTP the configuration file to a workstation or server. Transfer the configuration file (stored by default at the following location) to an external storage device. /tftpboot/.private/home/admin/<file-name> 5. Shut down the CPM-8600 with the failed drive using the following CLI command:
253
CBS# shutdown 6. 7. 8. 9. Remove the CPM-8600 with the bad drive from the system. Remove the failed drive from the CPM-8600 board by removing the four screws using a Phillips #1 screwdriver (or a small flat blade screwdriver if necessary). Install a blank replacement drive. Reinsert the CPM-8600 into the chassis.
10. To sync the drives on CPM-8600 with the replacement HDD, connect to the CPM-8600 using the console connection and log into XOS as root: 11. Log into XOS as root: CBS# unix su Password: [root@xxxxx admin]# 12. Issue the command: /usr/os/bin/xos-raid-add
Replace Failed Drive on an APM-8600 with RAID 1

NOTE: The good drive of the RAID 1 pair still contains your logging and other stored information. It is not necessary to back up this drive unless you are unsure which drive is the bad one. Complete the following steps to replace a failed RAID 1 drive on an APM-8600 and reestablish a working RAID 1 pair: 1. 2. Pull out the module with the bad drive. Remove the failed drive (based on the information in mdstat), insert a blank, good replacement drive, and reinsert the APM-8600. When the APM-8600 is next used in the VAP group, it will be configured according to the RAID setting for that VAP group and disk mirroring will occur.
Replace Failed Drive on an APM-9600 with RAID

The APM-9600 supports 1 or 2 disk Serial Attached SCSI (SAS) drives. To remove a disk drive, remove the four mounting screws on the reverse side of the module, then slide the drive up and away from the handle side of the module.
Identifing a Failed Disk Drive on an APM-9600 Configured for RAID

Identify which APM-9600 disk drive has failed, by performing these steps. 1. Log in to the primary CPM on your chassis and execute these commands. CBS# unix su Password: <root_password> [root@<chassis_hostname> admin]# rsh <IP_address_of_APM-9600> <VAP_group_member> (<chassis_hostname>): root# /crossbeam/bin/ cbs_scsi_disk_info.pl 2. 3. The output from the /crossbeam/bin/cbs_scsi_disk_info.pl script includes identifying information for the failed disk drive, including the serial number. Record the serial number of the failed disk drive.
254
Replacing a Failed Disk Drive on an APM-9600 Configured for RAID

To replace the failed disk drive, perform these steps. 1. 2. 3. Remove the APM-9600 module from the chassis. Use the serial number that you recorded to identify which disk drive to remove. The serial number of each disk drive is located on the label on the top of the disk drive. Replace the disk drive with the matching serial number and reinsert the APM-9600 into the chassis.
Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured

In a non-RAID configuration, if you have one or two hard drives in the APM-8600, depending on the slot they are inserted into, they are accessed according to the information in the following table.
Table 22.
Drive Access Information

Hard-coded Mounting Name /mnt/aplocaldrive /mnt/aplocaldrive_2 Reference Name in /proc/scsi/scsi scsi0 scsi3
Position on the Board (see diagram below) SATA1 (inside slot) SATA2 (outside slot)
You can determine a drives slot position by examining the /proc/scsi/scsi file as follows: 1. Log on to the CLI and use the following command to see information about the APM-8600s. CBS# show ap-vap-mapping Module Slot Status VAP IP Address AP6 8 Up 1.1.2.101 2. VAP Group v3 Index 1 Master true
Use the VAP IP address information to log in to the VAP group from the Unix prompt and access the / proc/scsi/scsi file. CBS# uni su [root@mycomputer admin]# rsh 1.1.2.101 Last login: Sat Apr 28 14:36:35 on ttyS0 v3_1 (mycomputer): root$ cat /proc/scsi/scsi Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: HTE721010G9SA00 Rev: Type: Direct-Access ANSI Host: scsi3 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: HTE721010G9SA00 Rev: Type: Direct-Access ANSI fw_1 (10PercentCPM2): root$
MCZO SCSI revision: 05 MCZO SCSI revision: 05
The example above shows the /proc/scsi/scsi file with two disks installed. The scsi0 drive refers to the drive in the SATA1 slot; scsi3 refers to SATA2, the drive in the outside slot. Refer to the diagram below.
255
Figure 24.
Disk Drive Locations on an APM-8600 or APM-8650
Module Memory Location
Module Disk-Drives
Figure 25.
Disk Drive Locations on an APM-9600
Disk Drives
256
C
Swatch Dynamic Display Tool
This chapter describes the Swatch tool, used to display dynamically-changing data related to the X-Series Platform operation. This chapter includes the following major topics:
z z z z z z z z
Overview on page 257 Understanding the Swatch Tool on page 258 Starting and Stopping the Swatch Tool on page 259 Using Single Key Commands on page 259 Swatch Tool Configuration Guidelines on page 260 Performance Considerations on page 264 Using Existing Swatch Tool Configuration Files on page 264 Troubleshooting the Swatch Tool on page 265
Overview
The XOS Open Security Appliance Swatch Dynamic Display Tool (Swatch) is a Linux tool designed to display dynamically-changing data on a terminal screen in a customized display format. The Swatch tool runs from the UNIX prompt and the XOS CLI. Originally designed to format and display device packet statistics from Linux/proc files, the Swatch tool now allows you to easily spot changes in your system status and can display any type of data that can be obtained from an ASCII file or by executing any Linux command that writes to a standard output. The Swatch tool supports the following data types:
z z z z
String 32-bit integer 64-bit integer 64-bit floating point
The Swatch tool can also perform simple arithmetic operations on data, such as rate, min, max, sum, and diff. All data can be reset (resets min, max, rate) and counter data can be zeroed out.
257
Understanding the Swatch Tool

The Swatch tool is designed to read configuration files, determine the data (known as Swatch variables) to acquire, and then determine where to display them on your screen. Configuration files are simple text files and can be created and edited using any text editor. Refer to the TCL documentation for the appropriate TCL commands and Swatch Tool Configuration Guidelines on page 260 for Swatch-specific commands. When reading the configuration file, the Swatch tool:
z z z
Parses the file using a TCL interpreter, allowing you to use any TCL commands. After parsing the configuration files, the Swatch tool displays it on the first screen. Defines one or more acquisition files to obtain data from. The Swatch tool periodically reads all fields from acquisition files into Swatch variables in memory. Defines one or more virtual screens that can be displayed. The Swatch tool periodically displays values of Swatch variables on a virtual screen.
Data from APMs can be displayed using the Swatch tool by using shell commands such as telnet or rsh. These commands allow you to log in to a remote slot and then execute a command on that slot.
Performance Monitoring with the Swatch Tool

The swatch scripts provided in XOS indicate problems at a high level. When swatch scripts indicate problems, further use of precise measurement tools such as network traffic analyzers is often necessary to determine the root cause problem. As a best practice for validating configurations, baselining a performance test, or integrating an application into an X-Series Platform, the following subset of XOS CLI commands and swatch scripts should be run. 1. 2. 3. CBS# show flow distribution Shows the distribution of traffic from the NPM(s) to the APM(s). CBS# show flow active Determines if traffic is passing through the X-Series Platform. Look for the word drop if a flow is dropped. Swatch script 16. npmdevstats Shows the Rx and Tx traffic from the NPMs. The traffic should be roughly equivalent of total Rx and Tx traffic into the APMs. Swatch script 1. apmdevstats Shows the Rx and Tx traffic for the APMs. The traffic should be roughly equivalent to the total Rx and Tx traffic into the NPMs. CBS# show cpu Displays aggregate CPU usage. CBS# show chassis Displays the status of installed modules.
4.
5. 6.
258
Starting and Stopping the Swatch Tool

To start the Swatch tool from the XOS CLI, enter swatch. When running Swatch from the CLI, you can only run existing configuration files, as described in Using Existing Swatch Tool Configuration Files on page 264. To start the Swatch Tool from the Linux shell prompt on a CPM or APM, cd to /crossbeam/bin and enter the following command: ./swatch [<options>] [ <script> ... ] Where the valid options are: -D<Varname>=[<value>,...], assigns values to the TCL variable --debug |-g <level>, set debugging level -i, show script information -g <n>, sets the debugging level to <n>. Where the value zero means display no debug information and the higher value levels display more information. -x <n>, causes the Swatch tool to dump internal program objects up to level <n>, with a higher level value displaying more information. -h | -- help, displays help for command usage and exits. -v | --version, displays the Swatch tool program version and exits. -p <interval>, log the screen data to files every <interval> seconds in background -f <filename>, log the screen data to file in foreground To stop the Swatch tool, press the q or Q key anytime while the Swatch tool is running.
Using Single Key Commands

The following single-key commands can be used anytime while the Swatch tool is running: <space-bar>, pauses/resumes screen updates. s, causes a single screen update. c (lowercase), clears all clear-able counters displayed on the current screen only. C (uppercase), clears all clear-able counters on all screens, whether or not they display in current screen. u (lowercase), restores the original values of the counters on the current screen. U (uppercase), restores the original values of the counters on all screens. r (lowercase), resets the data it on the current screen. R (uppercase), resets the data it on all screens. d, downloads the current screen to the /tmp directory, where the filename is the <screenname>.scr. +/-, moves to the next or previous screen respectively. 00-99, moves to the designated screen number from 0 to 99. h, displays a help page. l, displays a screen list page. q or Q, quits the Swatch tool program.
259
Swatch Tool Configuration Guidelines

The following are guidelines to use when creating a Swatch tool configuration file.
z
Use the first few lines of the file to set default values for TCL variables, which can be overridden by using a -D option on the Swatch command line. You can use -Dline=x to limit the output display. For example, ./swatch -Dline=10 interrupts will display the interrupt stats from line 10 onwards. Minimize the number of swacquire commands within a given configuration file. Minimize the number of lines and columns displayed on a virtual screen. Place a Title line at the top of each virtual screen. Use unique virtual screen names to allow users to easily identify the screen. Use the counter option on all swread commands to indicate if an item is a counter. Use the once option on swread commands if the item value does not change. Define only one or two different screen types per configuration file.
z z z z z z z
Creating a Swatch Tool Configuration File

To create a Swatch tool configuration file, use the following commands:
z z z z z z
swacquire swread swcalc swaggr swscreen swdisplay
Using the swacquire Command

The swacquire command specifies the file read periodically or the shell run to retrieve the command output.
swacquire Syntax
swacquire [file <filename>] [shell <shell>] [shellinit <prompt> <resp>] [shellcmd <prompt> <resp>] [period <poll-period>] [delim <delim-chars>] Where: file parameter is the existing files (including /proc files) or you can specify a file that saves the shell command output. shell parameter designates the running commands that generate output. shellinit parameter defines a one-time dialog. For example, use this parameter to specify a login-prompt/username or password-prompt/password. shellcmd parameter defines a dialog at each poll period. For example, use this parameter to specify a shell prompt/command executed in a telnet session. delim parameter specifies a set of characters used as a delimiter between fields. The following example specifies that the file /proc/net/dev is read every 2 seconds: swacquire file /proc/net/dev period 2
260
Display Position Syntax

The following information defines the position of a display or data item, when using this command. line/field/column-spec [+|-]x[@y] Where: +|- parameter specifies the new position is relative to last position. @y parameter specifies the new position is incremented by y for each variable instance. In the following example, the starting line number is +4 from the current line number, and the line number is incremented by 2 for each variable instance referenced in the command. line-spec is +4@2
Using the swread Command

The swread command defines a data item at fixed positions within the swacquire file (or shell output).
swread Syntax:
swread <varname> <line-spec> <field-spec> <fmt> [counter] [once] Where: <varname> is a Swatch tool variable name that holds values read from a file. <line-spec> and <field-spec> define the line# and field# position within a file. <fmt> defines how a data value is read. This is the same value as the scanf() function. counter parameter indicates that the data item is a counter. Note, this value can be zeroed out. once parameter indicates that a data item is read one-time only. NOTE: All swread commands are relative to the last swacquire command The following example defines an integer variable pktdrops at line 2; field 1 is read from the previous acquisition file with a format of %u. This variable is a counter. swread pktdrops 2 1 %u counter
Using the swcalc Command

The swcalc command defines the data item calculated from other data it.
swcalc Syntax
swcalc <varname> <operation> <operand> [<operand> ...] Where: <varname> is the result variable name. If this is an array, the number of array indices in the operands must match the number of array indices in the result. <operation> can be rate, min, max, add, or sub. <operand> is the name of Swatch tool variable used as a source for the operation. If this is an array, the number of array indices in the operands must match the number of array indices in the result. The following example defines an array drop rate that is calculated by computing the rate of the values in the pktdrops array. swcalc droprate(1:10) rate pktdrops(1:10)
261
Using the swaggr Command

The swaggr command defines a data item aggregated from other data.
swaggr Syntax
swaggr <varname> <operation> <operand> [<operand> ...] Where: <varname> is the result variable name. This must be a scalar value. <operation> is the sum or diff operation. <operand> is the name of the Swatch tool variable used as the source for operation. This can also be an array. The following example defines a scalar variable totdrops that is calculated by computing the sum of all values in the pktdrops array. swaggr totdrops sum pktdrops(1:10)
Using the swscreen Command

The swscreen command defines the virtual screen displayed.
swscreen Syntax
swscreen <screenname> [period <n>] Where the period parameter defines the screen refresh rate in seconds. The default value is 1 second. The following example defines a virtual screen named devpktcnt that when displayed, refreshes every 2 seconds. swscreen devpktcnt 2
Using the swdisplay Command

The swdisplay command defines the display item created at a fixed position on a virtual screen.
swdisplay Syntax
swdisplay <line-spec> <col-spec> <size> <fmt> [<varname>] [scale <scale-factor>] Where: <line-spec>, <col-spec> parameter defines the position of the display item on the virtual screen. <size> is the total width of the display item. <fmt> is the format used for writing a display item. This value is the same as the printf function. <varname> is the name of the previously defined Swatch tool data variable. scale parameter defines the scale factor applied to the data variable prior to output. NOTE: All swdisplay commands are relative to last swscreen command. The following example displays the values of the array pktdrops at column 15 on lines 4-13, with a width of 10 characters. swdisplay 4@1 15 10 %u pktdrops(1:10)
262
swdisplay Position Syntax

The following information defines the position of a display or data item, when using this command. line/field/column-spec [+|-]x[@y] Where: +|- parameter specifies the new position is relative to last position. @y parameter specifies the new position is incremented by y for each variable instance. In the following example, the starting line number is +4 from the current line number, and the line number is incremented by 2 for each variable instance referenced in the command. line-spec is +4@2
Logging Screen Data to a File

You can log screen data to files in the background and foreground for a swatch function.
Collect Screen Data in the Background

Use the flag -p with a swatch command to collect all screens data in the background. For example: swatch -p interval <script_name> The output files are written to /crossbeam/logs/ using the following filenaming scheme: scriptname_scrID-MM_DD_YY:HH:MM:SS-pid.log Use the command kill <pid> to terminate the background instance.
Collect Screen Data in the Background

Use the flag -f with a swatch command to display the logged screen in the foreground. For example: swatch script_name -f filename
Using Predefined TCL Variables in a Swatch Tool Configuration File

The following TCL variables can be referenced in a Swatch tool configuration file:
z z
SLOT_IP(1)..SLOT_IP(14), contains the IP address of slot or the value 0.0.0.0 if no slot is present. SLOT_NAME(1)SLOT_NAME(14), contains the host name of the slot or unknown if the slot is not present.
Using Swatch Tool Variables in All Commands

The following are Swatch tool variable guidelines and restrictions:
z z z z
Variable names can be any alphanumeric string. For example, pktdrops. Variables can be either scalar or arrays. Variables are only visible in current configuration file, unless they are prefixed with global tag. Array elements are specified as <varname>(<index>) where the index is a non-negative integer. The notation varname(<index1>:<index2>) is shorthand, which expands the command for each variable instance between varname(index1) and varname(index2) inclusive.
263
Using Existing Swatch Tool Configuration Files

This section describes the existing Swatch tool configuration files that you can use:
z
apmdevstats.swc - displays packet statistics per APM slot (one screen for each selected network device). Use a Ddevs value of <dev1>, <dev2>, and so on to specify the desired devices. The default value is devs=sdp0. apmdevstats_slot.swc - displays packet statistics per device slot (one screen for each selected APM slot). Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12. apmfwstats.swc - displays firewall statistics per APM slot in a single screen display. Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12. apmfwstats_slot.swc - displays firewall stats per device (one screen per selected APM slot). Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12. apmsnmpstats.swc - displays IP, ICMP, TCP, and UDP statistics and rates for the APMs. cbsinitdstats.swc - displays a list of Crossbeam daemons and their status. fabricstats.swc - displays NPM statistics and rates for the switched data path fabric on the dataplane. flowcalcstats.swc - displays information on flow calculation statistics. flowsched.swc - displays which member of a VAP Group is eligible for the next flow assignment. groupintstats.swc - displays statistics for group interface members. health_cpubsy.swc - displays CPU activity for the APMs and CPMs that are installed. health_cpumem.swc - displays CPU load, utilization, and memory information for each physical slot. health_temp.swc - displays CPU and Board minimum, maximum, and chassis current temperature information. moduleuptime.swc - displays the Linux up-time for each NPM and APM slot. netifstats.swc - displays packet statistics for all Linux devices on a current slot. This file can be run on a CPM or VAP. npmdevstats.swc - displays packet statistics for NPM physical interfaces. Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. npmfragstats.swc - displays virtual defragmentation (vdf) statistics per NPM.
z z z z z z z z z z z z z z z
Performance Considerations
The following are performance consideration to heed when using the Swatch tool.
Swatch Tool Child Processes

The Swatch tool creates many child processes on a local host. This can slow system performance. The following are examples of this:
z z
There is one child process for computing all derived or calculated data it. For example, data it defined in swcalc or swaggr commands. There is one child process for each swacquire command in the configuration files listed on startup. Note, if a swacquire command has a shell option, an additional child process is created to execute the shell command in.
NOTE: Some shell commands may cause TCP/UDP connections to remote hosts or cause new processes to be created on a remote host.
264
Single Shared-Memory Region and Semaphore

The Swatch tool uses a single shared-memory region to communicate data between processes. Currently, the region is set to 400,000 bytes. The Swatch tool also uses a single semaphore to provide atomic access to the shared memory region.
Troubleshooting the Swatch Tool

The following are some known problems that may result when using the Swatch Tool.
Data Not Being Displayed or Updated

The following bullets list some solutions when data is not being displayed or updated:
z z z
For swacquire file commands, make sure the file exists. For swacquire shell commands, make sure that the shell command can be invoked manually. Run the Swatch tool with a -g <n> option to create debug files. Note, higher debug levels give more debug information. Debug files are located in the swatch.xxxxx.log, where the xxxxx is the child process ID.
Data is Not Displayed in the Correct Location

The following bullets list some solutions when data is not displayed in correct screen location:
z z
Check the swdisplay line, column, and format options for the data item. Run the Swatch tool with a -x <n> option to download program objects. Note, higher dump levels give more dump information.
Swatch Tool is Affecting CPM Performance

The following bullets provide some solutions when the Swatch tool is affecting performance of the CPM:
z z z z
Reduce the number of Swatch tool programs being run simultaneously. Reduce the number of startup configuration files listed on Swatch tool command line. Reduce the polling rate for swacquire commands (period option). Reduce the display refresh rate for swscreen commands (period option).
Swatch Tool is Affecting Performance of Remote APMs or NPMs

The following bullets provide possible solutions when the Swatch tool is affecting performance of remote APMs or NPMs:
z z
Reduce the number of Swatch tool configurations that remotely access APMs or NPMs. Reduce the polling rate for swacquire commands that remotely access APMs or NPMs.
NOTE: Some CPM programs, such as apmdevstats and npmdevstats, query data on remote slots through the control bus.
265
266
D
Crossbeam SNMP MIB Reference
This chapter contains a reference to Crossbeam simple network management protocol (SNMP) management information base (MIB) files. The main topics in this appendix are:
Overview on page 267 New Crossbeam MIB Entries in XOS V9.5. on page 269 Greenlight Element Manager (GEM) MIB Reference on page 269 XOS Alarms and SNMP Traps on page 275 Using a MIB browser on page 268 CROSSBEAM-SYSTEMS-MIB Reference on page 280 CBS-HARDWARE-MIB Reference on page 281 CBS-MODULE-RESOURCES-MIB Reference on page 288 CBS-VAPGROUP-MIB Reference on page 293 CBS-VRRP-MIB Reference on page 294 Standard MIBs on the Crossbeam X-Series Platform on page 296
Overview
The following five Crossbeam SNMP MIB files are located in the /usr/share/snmp/mibs directory:
z z z z z
CROSSBEAM-SYSTEMS-MIB.txt CBS-HARDWARE-MIB.txt CBS-MODULE-RESOURCES-MIB.txt CBS-VAPGROUP-MIB.txt CBS-VRRP-MIB.txt
Each entry in the Crossbeam MIBs contain an object identifier (OID). The Crossbeam specific reference is 6848 as seen here: .1.3.6.1.4.1.6848 iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) crossbeamSystems(6848) Entries in the Crossbeam MIBs are organized into branches with unique OIDs. For example: CBS-HARDWARE-MIB::cbsHardware .1.3.6.1.4.1.6848.2.1 iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) crossbeamSystems(6848) cbsMgmt(2) cbsHardware(1) The information in this appendix describes a tree view into Crossbeam MIBs with a reference to the complete OID for each MIB entry. For comprehensive details, view each MIB entry in a MIB browser.
267
Using a MIB browser

There are several free MIB browsers, such as snmptranslate, that display details for the Crossbeam MIBs. The MIB browser displays the following: 1. 2. 3. 4. A tree-view of the MIBs and MIB entries The name and OID for each entry Syntax rules that include options. For example: up, down Access, status, and a text description for each entry
If you do not have access to a MIB browser, access the control processing module (CPM) from a command prompt and display the MIB files in /usr/share/snmp/mibs for details related to each MIB entry.
Using HP OpenView Network Node Manager

Crossbeam provides a Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i. The official Crossbeam integration package for this release is cbs_nnmi_v900i_integration.tar.gz. The integration package is available for download from the Crossbeam Support Portal at http://www.crossbeam.com/support/online-support/. A valid support account with username and password is required. Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i enables custom integration of Crossbeam X-Series Platforms into HPOV/NNMi v9.00. Once installed, it provides the following features:
z z z z
Ability to launch the Crossbeam Greenlight Element Manager (GEM) in a new browser window from HP OpenView. A node group representing all Crossbeam hardware making it easier to apply changes to only Crossbeam equipment. Access this node group from Inventory > Node Groups > Crossbeam Devices. Integration of proprietary Crossbeam MIBs into HP OpenView / NNMi, enabling the custom polling of Crossbeam MIBs, and provides information in addition to standard MIBs. Overall device status is indicated by icon color.
For information, see the Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i Release Notes.
New Crossbeam MIB Entries in XOS V9.5

Crossbeam added new MIB entries in the XOS V9.5 release. A summary of the new MIBs is available in Table 23 on page 269. Refer to the respective MIB reference section for specific entries, descriptions, and OID. In addition to new MIBs to expand the data available about the Crossbeam chassis, Crossbeam added MIBs for all data available in the browser based views of the Greenlight Element Manager. A detailed listing of the new MIBs for each view of the Greenlight Element Manager is available in the Greenlight Element Manager (GEM) MIB Reference on page 269. Most alarms raised on an the Crossbeam chassis also generate one or more SNMP traps. Refer to XOS Alarms and SNMP Traps on page 275 for a list of alarms and their associated SNMP traps.
268
Table 23.
New Crossbeam MIB Entries in XOS V9.5.

Description New information includes:
z z z z
Crossbeam MIB CBS-HARDWARE-MIB
Fan Tray compatibility and status Firmware revision status CP Redundancy state Minor, Major, and Critical Alarms
CBS-MODULE-RESOURCES-MIB
New information includes:

z
Flow Distribution Table
CBS-VRRP-MIB
New information includes:

z
VRRP Failover Group Status (DBHA)
Greenlight Element Manager (GEM) MIB Reference

The table below contains a list of the MIBs for all the data presented in the GEM User Interface.
Table 24.
GEM MIB Reference

MIB MIB Object
View / Component System View Hostname Chassis Type Alarms Hardware Properties Revision Serial Number Part Number Chassis Temp DBHA XOS Version Firmware Version Modules Module Name Module Status VAP Groups Name Application
MIB-II CBS-HARDWARE-MIB See Alarms View CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB See DBHA View MIB-II See Firmware View CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-VAPGROUP-MIB
sysName cbsHwSystemDescription
cbsHwSystemBackplaneRevision cbsHwSystemSerialNumber cbsHwSystemPartNumber cbsHwSystemChassisTemp
sysDescr
cbsHwModuleDescription cbsHwModuleStatus cbsVgVapGroupName
CBS-MODULE-RESOURCES cbsMgmt.cbsModuleResources.cbsM -MIB oduleApplicationTable.cbsModuleApp licationEntry.cbsModuleApplicationNa me cbsMgmt.cbsModuleResources.cbsM oduleApplicationTable.cbsModuleApp licationEntry.cbsModuleApplicationVe rsion
VAP count Max load count

CBS-VAPGROUP-MIB CBS-VAPGROUP-MIB
cbsVgVapCount cbsVgMaxLoadCount
269
View / Component Fan Trays Status
MIB CBS-HARDWARE-MIB
MIB Object cbsHwSystemUpperFanTray cbsHwSystemLowerFanTray cbsFanStatus
Compatible
CBS-HARDWARE-MIB
cbsHwSystemStatus.cbsHwSystemU pperFanTrayCompatible cbsHwSystemStatus.cbsHwSystemL owerFanTrayCompatible
Revision
CBS-HARDWARE-MIB
cbsHardware.cbsHwSystemStatus.cb sHwSystemUpperFanTrayRevision cbsHardware.cbsHwSystemStatus.cb sHwSystemLowerFanTrayRevision
Serial Number
CBS-HARDWARE-MIB
cbsHwSystemStatus.cbsHwSystemU pperFanTraySerialNumber cbsHwSystemStatus.cbsHwSystemL owerFanTraySerialNumber
Part Number
CBS-HARDWARE-MIB
cbsHwSystemStatus.cbsHwSystemU pperFanTrayPartNumber cbsHwSystemStatus.cbsHwSystemL owerFanTrayPartNumber
Power Type Power Supplies Power Supply Power Feed Firmware View Slot Module Type Board Revision Bootstrap Rev Current Required Bootloader Rev Current Required Ctrl FPGA revision Focus FPGA revision Current Required Current Required
CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB
cbsHwSystemPowerType cbsHwSystemStatus cbsHwSystemStatus
CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB
cbsHwModuleID cbsHwModuleDescription cbsHwModuleBoardRevision cbsHwModuleBootStrapRev cbsHwModuleReqBootStrapRev cbsHwModuleBootloaderRev cbsHwModuleReqBootloaderRev cbsHwModuleCtrlFpgaRev cbsHwModuleReqCtrlFpgaRev cbsHwModuleFocusFpgaRev cbsHwModuleReqFocusFpgaRev
DBHA View Failover Group ID CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpFailover GroupID crossbeamSystems.cbsMgmt.cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpFailover GroupName
270
Failover Group Name
CBS-VRRP-MIB
View / Component Status
MIB CBS-VRRP-MIB
MIB Object crossbeamSystems.cbsMgmt.cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpStatus crossbeamSystems.cbsMgmt..cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpPriority crossbeamSystems.cbsMgmt.cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpAdminPr iority crossbeamSystems.cbsMgmt.cbsVrr p.cbsVrrpFailoverGroupTable.cbsVrr pFailoverGroupEntry.cbsVrrpMasterP riority
Actual Priority
CBS-VRRP-MIB
Configured Priority
CBS-VRRP-MIB
Master Priority
CBS-VRRP-MIB
NPM View Slot Module Type Hardware Properties Revision Serial Number Part Number Status Uptime Interfaces Label State BW Data Rate(In) Data Rate(Out) APM View Slot Module Type Vap Name Hardware Properties Revision Serial Number Part Number Module Status Uptime Application CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-VAPGROUP-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB cbsHwModuleID cbsHwModuleDescription cbsVmVapName cbsHwModuleBoardRevision cbsHwModuleSerialNumber cbsHwModulePartNumber cbsHwModuleStatus CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB cbsHwModuleID cbsHwModuleDescription cbsHwModuleBoardRevision cbsHwModuleSerialNumber cbsHwModulePartNumber cbsHwModuleStatus
CBS-MODULE-RESOURCES cbsModuleUptime -MIB IF-MIB IF-MIB IF-MIB IF-MIB IF-MIB ifDescr ifOperStatus ifSpeed ifInOctets ifOutOctets
CBS-MODULE-RESOURCES cbsModuleUptime -MIB CBS-MODULE-RESOURCES cbsModuleAppName -MIB cbsModuleAppVersion
271
View / Component App Status Avg CPU Util Avg CPU Load Peak Core Util Total Memory
MIB
MIB Object
CBS-MODULE-RESOURCES cbsModuleAppNewState -MIB CBS-MODULE-RESOURCES cbsModuleCPUUtil5 -MIB CBS-MODULE-RESOURCES cbsModuleCPULoad5 -MIB CBS-MODULE-RESOURCES cbsModuleCpuCoreHiUtilPerc -MIB CBS-MODULE-RESOURCES cbsModuleMemoryTotal -MIB cbsModuleMemoryFree cbsModuleMemoryBuffer cbsModuleMemoryCached
Low Memory
CBS-MODULE-RESOURCES cbsModuleMemoryLoTotal -MIB cbsModuleMemoryLoFree
CPM View Slot Module Type Hardware Properties Revision Serial Number Part Number Module Status CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB cbsHwModuleID cbsHwModuleDescription cbsHwModuleBoardRevision cbsHwModuleSerialNumber cbsHwModulePartNumber cbsMgmt.cbsHardware.cbsCPRedun dancyState.cbsCP2RedundancyOper State or cbsCP1RedundancyOperState sysDescr
XOS Version Uptime Admin state
MIB-II
CBS-MODULE-RESOURCES cbsModuleUptime -MIB CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHar dware.cbsCPRedundancyState.cbsC PRedundancyAdminState crossbeamSystems.cbsMgmt.cbsHar dware.cbsCPRedundancyState.cbsC P1RedundancyAdminState crossbeamSystems.cbsMgmt.cbsHar dware.cbsCPRedundancyState.cbsC P2RedundancyAdminState
Sync state
CBS-HARDWARE-MIB
crossbeamSystems.cbsMgmt.cbsHar dware.cbsCPRedundancyState.cbsC PRedundancyOperState crossbeamSystems.cbsMgmt.cbsHar dware.cbsCPRedundancyState.cbsC PRedundancySync
Avg CPU Util
CBS-MODULE-RESOURCES cbsModuleCPUUtil5 -MIB

272
View / Component Avg CPU Load Peak Core Util Total Memory
MIB
MIB Object
CBS-MODULE-RESOURCES cbsModuleCPULoad5 -MIB CBS-MODULE-RESOURCES cbsModuleCpuCoreHiUtilPerc -MIB CBS-MODULE-RESOURCES cbsModuleMemoryTotal -MIB cbsModuleMemoryFree cbsModuleMemoryBuffer cbsModuleMemoryCached
Disk usage
root boot cbconfig tftpboot mgmt
CBS-MODULE-RESOURCES cbsModuleDURoot -MIB CBS-MODULE-RESOURCES cbsModuleDUBoot -MIB CBS-MODULE-RESOURCES cbsModuleDUCbconfig -MIB CBS-MODULE-RESOURCES cbsModuleDUTftpboot -MIB CBS-MODULE-RESOURCES cbsModuleDUMgmt -MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB cbsHwModuleRAIDStatus cbsHwModuleRAIDMode
RAID
Status Mode Flows View
Chassis Utilization
Flow Table Partition Threshold NPM Flow Table Critical Alarm NPM Flow Table Utilization NPM Flow Table Utilization TCP Flow Entries NPM Flow Table Utilization UDP Flow Entries NPM Flow Table UtilizationI CMP Flow Entries NPM Flow Table Utilization Other IP Flow Entries
CBS-MODULE-RESOURCES cbsFlowTablePartitionThreshold -MIB CBS-MODULE-RESOURCES cbsFlowTableCriticalAlarm -MIB CBS-MODULE-RESOURCES cbsFlowTableUtilization -MIB CBS-MODULE-RESOURCES cbsFlowTableUtilizationTcpFlowEntri -MIB es
CBS-MODULE-RESOURCES cbsFlowTableUtilizationUdpFlowEntri -MIB es
CBS-MODULE-RESOURCES cbsFlowTableUtilizationIcmpFlowEntr -MIB ies
CBS-MODULE-RESOURCES cbsFlowTableUtilizationOtherIpFlowE -MIB ntries
273
View / Component NPM Slot ID TCP Current Flows UDP Current Flows ICMP Current Flows Other IP Current Flows New Flow Rate
MIB
MIB Object
CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmSlotID CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmProtoTcpCurrentFlo ws CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmProtoTdpCurrentFl ows CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmProtoIcmpCurrentFl ows CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmProtoOtherIpCurren tFlows CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmNewFlowRate
Aged Flow Rate CBS-MODULE-RESOURCES cbsNpmFlowDistTable.cbsNpmFlow -MIB DistEntry.cbsNpmAgedFlowRate VAP VapName Current Flows New Flow Rate CBS-MODULE-RESOURCES cbsVapFlowDistTable.cbsVapFlowDi -MIB stEntry.cbsVapNmae CBS-MODULE-RESOURCES cbsVapFlowDistTable.cbsVapFlowDi -MIB stEntry.cbsVapCurrentFlows CBS-MODULE-RESOURCES cbsVapFlowDistTable.cbsVapFlowDi -MIB stEntry.cbsVapNewFlowRate
Aged Flow Rate CBS-MODULE-RESOURCES cbsVapFlowDistTable.cbsVapFlowDi -MIB stEntry.cbsVapAgedFlowRatge Alarms View ID CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmId crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmDate crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmSeverity crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmSource crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmDescription
Date
CBS-HARDWARE-MIB
Severity
CBS-HARDWARE-MIB
Source
CBS-HARDWARE-MIB
Description
CBS-HARDWARE-MIB
274
View / Component Device
MIB Object crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmDevice crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmSensor crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmIface crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmSubIface crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmDisk crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsActiveAla rmsTable.cbsActiveAlarmsEntry.cbsA ctiveAlarmPartition crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsAlarmSta ts.cbsActiveMinorNumber crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsAlarmSta ts.cbsActiveMajorNumber crossbeamSystems.cbsMgmt.cbsHar dware.cbsAlarmObjects.cbsAlarmSta ts.cbsActiveCriticalNumber
Sensor
CBS-HARDWARE-MIB
Interface
CBS-HARDWARE-MIB
Sub Interface
CBS-HARDWARE-MIB
Disk
CBS-HARDWARE-MIB
Partition
CBS-HARDWARE-MIB
Active minor alarms number
CBS-HARDWARE-MIB
Active major alarms number
CBS-HARDWARE-MIB
Active critical alarms number
CBS-HARDWARE-MIB
Overall Application IpAddr MIB-II ipAdEntAddr
XOS Alarms and SNMP Traps

XOS provides a number of alarms to help you monitor the state of the XOS system. The following table presents the alarms that can be raised by XOS and the associated SNMP traps. This information is also available in the Greenlight Element Manager (GEM) online help. Alarm Name applicationDown batteryVoltageProblem MIB CBS-HARDWARE-MIB CBS-HARDWARE-MIB SNMP Trap cbsModuleAppStateChange cbsHwBatteryVCCNormal cbsHwBatteryVCCOutOfRange
275
Alarm Name chassisTemperatureExceeded
SNMP Trap cbsHwChassisTempNormal cbsHwChassisTempExceeded
componentTemperatureExceeded
CBS-HARDWARE-MIB
cbsHwFPGATempExceeded cbsHwFPGATempNormal
cpuCoreUtilizationExceeded
CBS-MODULERESOURCES-MIB CBS-HARDWARE-MIB
cbsModuleCpuCoreUtilExceeded cbsModuleCpuCoreUtilNormal cbsModuleTempExceeded cbsModuleTempNormal cbsModuleCpu2TempExceeded cbsModuleCpu2TempNormal
cpuTempLimitExceeded
cpuUptimeLimitExceeded cpuUtilization15min cpuUtilization1min cpuUtilization5min diagsFailure diskConfigurationBad diskDriveError diskUtilizationBootExceeded CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-HARDWARE-MIB cbsModuleDUBootHigh cbsModuleDUBootNormal cbsModuleDUCbconfigHigh cbsModuleDUCbconfigNormal cbsModuleDUCbMgmtHigh cbsModuleDUCbMgmtNormal cbsModuleDUCbRootHigh cbsModuleDUCbRootNormal cbsModuleDUTftpbootHigh cbsModuleDUTftpbootNormal cbsModuleDUVarHigh cbsModuleDUVarNormal cbsDbhaLinkDown cbsDbhaLinkUp exhaustAirTemperatureExceeded CBS-HARDWARE-MIB cbsHwExhaustTempExceeded cbsHwExhaustTempNormal fanFailure CBS-HARDWARE-MIB cbsHwFanFailed cbsHwFanRecovered fanTrayIncompatible CBS-HARDWARE-MIB CBS-MODULERESOURCES-MIB cbsNPMDiagFailure cbsCpmDiskCheck
diskUtilizationCbConfigExceeded
diskUtilizationMgmtExceeded
diskUtilizationRootExceeded
diskUtilizationTftpBootExceeded
diskUtilizationVarExceeded
dualBoxHaLinkFailure
276
Alarm Name fanTrayMissing
SNMP Trap cbsHwLowerFanTrayInserted cbsHwLowerFanTrayRemoved cbsHwUpperFanTrayInserted cbsHwUpperFanTrayRemoved
firmwareMismatch flowTableFull CBS-HARDWARE-MIB cbsAFTUsageHigh cbsAFTUsageNormal flowTableMedianThresholdExcee ded CBS-MODULERESOURCES-MIB cbsNpmFTTcpMedianExceeded cbsNpmFTTcpMedianNormal cbsNpmFTUdpMedianExceeded cbsNpmFTUdpMedianNormal cbsNpmFTIcmpMedianExceeded cbsNpmFTIcmpMedianNormal cbsNpmFTOtherIpMedianExceeded cbsNpmFTOtherIpMedianNormal flowTableThresholdExceeded CBS-MODULERESOURCES-MIB cbsNpmFTTcpLimitExceeded cbsNpmFTTcpLimitNormal cbsNpmFTUdpLimitExceeded cbsNpmFTUdpLimitNormal cbsNpmFTIcmpLimitExceeded cbsNpmFTIcmpLimitNorma cbsNpmFTOtherIpLimitExceeded cbsNpmFTOtherIpLimitNormal fragPktRateExceeded CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB IF-MIB cbsModuleFRHigh cbsModuleFRNormal cbsModuleFreePageAvailableHigh cbsModuleFreePageAvailableNorm cbsApmGuestHealthCheck cbsHwInTempExceeded cbsHwInTempNormal linkDown linkUp memoryConfiguration memoryMismatch moduleFailure CBS-MODULERESOURCES-MIB cbsModuleSdramCheck
freePageUtilization
guestHealthProblem intakeAirTemperatureExceeded
linkFailure
277
Alarm Name moduleStateChange
SNMP Trap cbsHwModuleInserted cbsHwModuleRemoved cbsHwModuleStatusChanged
noRemoteBoxConfigured npmBuffersUsageHigh
CBS-VRRP-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-VRRP-MIB
cbsVrrpTrapNoRemoteBoxConfigured cbsModuleBUHigh cbsModuleBUNormal cbsModuleMUHigh cbsModuleMUNormal cbsModuleNtpdMonStateChange cbsVrrpTrapOnlyOnePathDefined
npmOutOfMemory
ntpServiceFailure onlyOneRemoteBoxPathDefined outOfService powerFeedFailure
CBS-HARDWARE-MIB
cbsHwPowerFeedFailed cbsHwPowerFeedRecovered
powerSupplyA_Bfailure
CBS-HARDWARE-MIB
cbsHwPowerSupplyFailed cbsHwPowerSupplyRecovered
powerSupplyMissing
CBS-HARDWARE-MIB
cbsHwSystemACPwrBay1Failed cbsHwSystemACPwrBay1Recovered cbsHwSystemACPwrBay2Failed cbsHwSystemACPwrBay2Recovered cbsHwSystemACPwrBay3Failed cbsHwSystemACPwrBay3Recovered cbsHwSystemACPwrBay4Failed cbsHwSystemACPwrBay4Recovered
raidStatusChange resourceProtectionFlowThreshold Exceeded rswFail systemAlarm systemRestart vapStateChange CBS-VAPGROUP-MIB cbsVapStatusChanged CBS-MODULERESOURCES-MIB CBS-MODULERESOURCES-MIB CBS-HARDWARE-MIB cbsNpmFlowTableUsageHigh cbsNpmFlowTableUsageNormal cbsModuleRSWStateChange cbsHwSystemAlarmTrap
278
Alarm Name voltageProblem
SNMP Trap cbsHw125VoltageNormal cbsHw125VoltageOutOfRange cbsHwModule1-2VNormal cbsHwModule1-2VOutOfRange cbsHwModule1-5VNormal cbsHwModule1-5VOutOfRange cbsHwModule12-0VNormal cbsHwModule12-0VOutOfRange cbsHw1-8VoltageNormal cbsHw1-8VoltageOutOfRange cbsHw2-5VoltageNormal cbsHw2-5VoltageOutOfRange cbsHw3-3VoltageNormal cbsHw3-3VoltageOutOfRange cbsHwCPUVoltageNormal cbsHwCPUVoltageOutOfRange cbsHwCPU2VoltageNormal cbsHwCPU2VoltageOutOfRange cbsHwFPGA1-8VoltageNormal cbsHwFPGA1-8VoltageOutOfRange cbsHwModuleETHSwVNormal cbsHwModuleETHSwVOutOfRange cbsHwModuleNP6EZDDRNormal cbsHwModuleNP6EZDDROutOfRange cbsHwModuleNP6EZCORENormal cbsHwModuleNP6EZCOREOutOfRange cbsHwModuleNP6EZHSTLNormacbsHw ModuleNP6EZHSTLOutOfRange cbsHwModuleNP6XBPRCNormal cbsHwModuleNP6XBPRCOutOfRange cbsHwModuleNP6OCTDDRNormal cbsHwModuleNP6OCTDDROutOfRange
voltageProblem (contd)
CBS-HARDWARE-MIB
cbsHwModulePrismVNormal cbsHwModulePrismVOutOfRange cbsHwModuleSupplyFGNormal cbsHwModuleSupplyFGOutOfRange
vrrpFailGroupPriorityChange vrrpFailGroupStatusChange
CBS-VRRP-MIB CBS-VRRP-MIB
cbsVrrpTrapFailGrPriorityChanged cbsVrrpTrapFailGrStatusChanged
279
Alarm Name vrrpRemoteBoxNoActivePath
MIB CBS-VRRP-MIB
SNMP Trap cbsVrrpTrapRemoteBoxNoActivePath cbsVrrpTrapRemoteBoxNoSecondaryPat h cbsVrrpTrapRemoteBoxNoStandbyPath cbsVrrpTrapRemoteBoxPathSharedIntf cbsVrrpTrapRemoteBoxPathStatusChang ed cbsVrrpTrapRemoteBoxRunLegacyXOS
vrrpRemoteBoxNoSecondaryPath CBS-VRRP-MIB vrrpRemoteBoxNoStandbyPath vrrpRemoteBoxPathSharedSingle Intf vrrpRemoteBoxPathStatusChang e vrrpRemoteBoxRunLegacyXOS CBS-VRRP-MIB CBS-VRRP-MIB CBS-VRRP-MIB CBS-VRRP-MIB
CROSSBEAM-SYSTEMS-MIB Reference
The CROSSBEAM-SYSTEMS-MIB includes high-level entries that describe Crossbeam C-Series and X-Series chassis and related components. This section contains a tree view of the CROSSBEAM-SYSTEMS-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CROSSBEAM-SYSTEMS-MIB.txt file on your CPM at /usr/share/snmp/mibs.
Identifying the CROSSBEAM-SYSTEMS-MIB OIDs

Identify the OID for specific MIB entries by appending the ordered list number for the entry to the CROSSBEAM-SYSTEMS-MIB OID .1.3.6.1.4.1.6848. The OID for Crossbeam specific product entries uses the prefix .1.3.6.1.4.1.6848.1 The OID for Crossbeam specific management entries uses the prefix .1.3.6.1.4.1.6848.2 The OID for Crossbeam specific MIBs use the prefix .1.3.6.1.4.1.6848.3 The OID for Crossbeam specific traps uses the trap prefix .1.3.6.1.4.1.6848.4 NOTE: Crossbeam originally created the CROSSBEAM-SYSTEMS-MIB for the X-Series X40 chassis. Therefore this MIB includes later X-Series chassis models such as the X45 and the X80 into a separate OID branch. However, all X-Series modules are only listed under the X40 chassis branch.
CROSSBEAM-SYSTEMS-MIB product entries

CROSSBEAM-SYSTEMS-MIB product OID: .1.3.6.1.4.1.6848.1 1. cbsX40 1. 2. cbsX40Chassis cbsX40Modules 1. cbsCPModules 1.cbsCPM4100 2.cbsCPM8100 3.cbsCPM8400 4.cbsCPM8600
280
5.cbsCPM9600 2. cbsAPModules 1.cbsAPM4100 2.cbsAPM4200 3.cbsAPM4250 4.cbsAPM8200 5.cbsAPM8400 6.cbsAPM8600 7.cbsAPM8650 8.cbsAPM9600 3. cbsNPModules 1.cbsNPM4100 2.cbsNPM4110 3.cbsNPM8200 4.cbsNPM8210 5.cbsNPM8600 6.cbsNPM8650 7.cbsNPM8620 2. cbsXSeries 1. cbsXChassis 1. 2. 3. 4. 5. 3. 1. cbsX45Chassis cbsX80Chassis cbsX60Chassis cbsX20Chassis cbsX30Chassis
cbsCSeries cbsCChassis 1. 2. 1. 2. 3. 4. 5. cbsC30Chassis cbsC10Chassis cbsC30iChassis cbsC2Chassis cbsC6Chassis cbsC12Chassis cbsC25Chassis
CBS-HARDWARE-MIB Reference
The CBS-HARDWARE-MIB includes entries for Crossbeam specific hardware features. The features include system status, hardware components, temperature, voltage, power, and LED status. This section contains a tree view of the CBS-HARDWARE-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-HARDWARE-MIB.txt file on your CPM at /usr/share/snmp/mibs.
281
Identifying the CBS-HARDWARE-MIB OIDs

Identify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-HARDWARE-MIB management OID entries (.1.3.6.1.4.1.6848.2.1). The OID for Crossbeam hardware-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.1) For example: The Crossbeam system serial number (cbsHwSystemSerialNumber) OID is .1.3.6.1.4.1.6848.2.1.1.3 The system status (cbsHwSystemAlarm) OID is .1.3.6.1.4.1.6848.2.1.2.6 The trap for a failed fan (cbsHwFanFailed) is .1.3.6.1.4.1.6848.4.1.1.9
CBS-HARDWARE-MIB entries
CBS-HARDWARE-MIB management OID: .1.3.6.1.4.1.6848.2.1 1. cbsHwSystem 1. 2. 3. 4. 5. 6. 2. 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsHwSystemProductID cbsHwSystemDescription cbsHwSystemSerialNumber cbsHwSystemBackplaneRevision cbsHwSystemPartNumber cbsHwSystemIdentifier cbsHwSystemRedundantPowerSupplyStatus cbsHwSystemRedundantPowerFeedStatus cbsHwSystemChassisTemp cbsHwSystemUpperFanTray cbsHwSystemLowerFanTray cbsHwSystemAlarm cbsHwSystemPowerType cbsHwSystemRedundantACPowerSupplyState cbsHwSystemRedundantACPowerFeedStatus
cbsHwSystemStatus
10. cbsHwSystemDCPowerFilterA 11. cbsHwSystemDCPowerFilterB 12. cbsHwSystemDCPowerFeedA 13. cbsHwSystemACPowerFeedB 14. cbsHwSystemACPwrBay1Presence 15. cbsHwSystemACPwrBay2Presence 16. cbsHwSystemACPwrBay3Presence 17. cbsHwSystemACPwrBay4Presence 18. cbsHwSystemACPwrBay1Status 19. cbsHwSystemACPwrBay2Status 20. cbsHwSystemACPwrBay3Status
282
21. cbsHwSystemACPwrBay4Status 22. cbsHwSystemAftStatus 23. cbsHwSystemUpperFanTrayCompatible 24. cbsHwSystemUpperFanTrayRevision 25. cbsHwSystemUpperFanTraySerialNumber 26. cbsHwSystemUpperFanTrayPartNumber 27. cbsHwSystemLowerFanTrayCompatible 28. cbsHwSystemLowerFanTrayRevision 29. cbsHwSystemLowerFanTraySerialNumber 30. cbsHwSystemLowerFanTrayPartNumber 3. cbsFanStatusTable 1. cbsFanStatusEntry 1. 2. 3. 4. 1. cbsFanGroupIndex cbsFanIndex cbsFanStatus
cbsHwModuleTable cbsHwModuleEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsHwModuleID cbsHwModuleProductID cbsHwModuleDescription cbsHwModuleBoardRevision cbsHwModuleSerialNumber cbsHwModuleBootStrapRev cbsHwModuleBootloaderRev cbsHwModuleDiagnosticsRev cbsHwModuleModuleSDRAMSize
10. cbsHwModuleRDRAMSize 11. cbsHwModuleDiskAPresent 12. cbsHwModuleDiskBPresent 13. cbsHwModuleDuartAPresent 14. cbsHwModuleDuartBPresent 15. cbsHwModuleAccelCard1Present 16. cbsHwModuleAccelCard2Present 17. cbsHwModulePartNumber 18. cbsHwModuleReqBootStrapRev 19. cbsHwModuleReqBootloaderRev 20. cbsHwModuleReqDiagnosticsRev 21. cbsHwModuleCtrlFpgaRev 22. cbsHwModuleReqCtrlFpgaRev 23. cbsHwModuleFocusFpgaRev
283
24. cbsHwModuleReqFocusFpgaRev 5. cbsHwComponentRevTable 1. cbsHwComponentRevEntry 1. 2. 3. 6. 1. cbsHwComponentIndex cbsHwComponentDescription cbsHwComponentRevision
cbsHwModuleStatusTable cbsHwModuleStatusEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsHwModuleStatus cbsHwModuleStatusActive cbsHwModuleCpuTemp cbsHwModuleFPGATemp cbsHwModuleInTemp cbsHwModuleInTempAlarm cbsHwModuleExhaustTemp cbsHwModuleExhaustTempAlarm cbsHwModuleCPUVoltage
10. cbsHwModule1-8Voltage 11. cbsHwModule3-3Voltage 12. cbsHwModule2-5Voltage 13. cbsHwModuleControlLinkA 14. cbsHwModuleControlLinkB 15. cbsHwModuleActiveLED 16. cbsHwModuleStandbyLED 17. cbsHwModuleFailedLED 18. cbsHwModuleCpu2Temp 19. cbsHwModuleCPU2Voltage 20. cbsHwModuleFPGA1-8Voltage 21. cbsHwModule125Voltage 22. cbsHwModule1-5V 23. cbsHwModuleBatteryVCCVoltage 24. cbsHwModuleSupplyFGVoltage 25. cbsHwModuleSupplyETHSwVoltage 26. cbsHwModuleSupplyPrismVoltage 27. cbsHwModule1-2V 28. cbsHwModule12-0V 29. cbsHwModuleSupplyNP6OCTDDRVolt 30. cbsHwModuleSupplyNP6EZCOREVolt 31. cbsHwModuleSupplyNP6EZDDRVolt 32. cbsHwModuleSupplyNP6EZHSTLVolt
284
33. cbsHwModuleSupplyNP6XBPRCVolt 34. cbsHwModuleRAIDStatus 35. cbsHwModuleRAIDMode 7. cbsHwSdpStatusTable 1. cbsHwSdpStatusEntry 1. 2. 3. 4. 8. 1. 2. 3. 4. 5. 6. 7. 9. 1. cbsHwSdpNpmSlot cbsHwSdpRemoteSlot cbsHwSdpNpmState cbsHwSdpRemoteState
cbsCPRedundancyState cbsCPRedundancyAdminState cbsCP1RedundancyAdminState cbsCP2RedundancyAdminState cbsCPRedundancyOperState cbsCP1RedundancyOperState cbsCP2RedundancyOperState cbsCPRedundancySync cbsActiveAlarmsTable 1. cbsActiveAlarmsEntry 1.cbsActiveAlarmIndex 2.cbsActiveAlarmId 3.cbsActiveAlarmDate 4.cbsActiveAlarmSeverity 5.cbsActiveAlarmSource 6.cbsActiveAlarmDescription 2. cbsAlarmStats 1. 2. 3. 1. cbsActiveMinorNumber cbsActiveMajorNumber cbsActiveCriticalNumber
cbsAlarmObjects
10. cbsTrapObjects cbsTrapSeverity
CBS-HARDWARE-MIB Trap Entries

CBS-HARDWARE-MIB trap OID: .1.3.6.1.4.1.6848.4.1 1. cbsHwTraps 1. 2. 3. 4. cbsHwPowerSupplyFailed cbsHwPowerSupplyRecovered cbsHwPowerFeedFailed cbsHwPowerFeedRecovered
285
5. 6. 7. 8. 9.
cbsHwUpperFanTrayInserted cbsHwUpperFanTryRemoved cbsHwLowerFAnTrayInserted cbsHwLowerFanTrayRemoved cbsHwFanFailed
10. cbsHwFanRecovered 11. cbsHwSystemAlarmTrap 12. cbsHwChassisTempExceeded 13. cbsHwChassisTempNormal 14. cbsHwModuleStatusChanged 15. cbsHwModuleInserted 16. cbsHwModuleRemoved 17. cbsHwModuleTempExceeded 18. cbsModuleTempNormal 19. cbsDbhaLinkUp 20. cbsDbhaLinkDown 21. cbsModuleCpu2TempExceeded 22. cbsModuleCpu2TempNormal 23. cbsAFTUsageHigh 24. cbsAFTUsageNormal 25. cbsHwSystemACPwrBay1Failed 26. cbsHwSystemACPwrBay1Recovered 27. cbsHwSystemACPwrBay2Failed 28. cbsHwSystemACPwrBay2Recovered 29. cbsHwSystemACPwrBay3Failed 30. cbsHwSystemACPwrBay3Recovered 31. cbsHwSystemACPwrBay4Failed 32. cbsHwSystemACPwrBay4Recovered 33. cbsNPPMDiagFailure 34. cbsNPMPrcLink13Down 35. cbsNPMPrcLink13Up 36. cbsNPMPrcLink14Down 37. cbsNPMPrcLink14Up 38. cbsHwInTempExceeded 39. cbsHwInTempNormal 40. cbsHwExhaustTempExceeded 41. cbsHwExhaustTempNormal 42. cbsHwFPGATempExceeded 43. cbsHwFPGATempNormal 44. cbsHwCPUVoltageNormal
286
45. cbsHwCPUVoltageOutOfRange 46. cbsHw1-8VoltageNormal 47. cbsHw1-8VoltageOutOfRange 48. cbsHw3-3VoltageNormal 49. cbsHw3-3VoltageOutOfRange 50. cbsHw2-5VoltageNormal 51. cbsHw2-5VoltageOutOfRange 52. cbsHwCPU2VoltageNormal 53. cbsHwCPU2VoltageOutOfRange 54. cbsHwFPGA1-8VoltageNormal 55. cbsHwFPGA1-8VoltageOutOfRange 56. cbsHw125VoltageNormal 57. cbsHw125VoltageOutOfRange 58. cbsHwBatteryVCCNormal 59. cbsHwBatteryVCCOutOfRange 60. cbsHwModule1-5VNormal 61. cbsHwModule1-5VOutOfRange 62. cbsHwModuleSupplyFGNormal 63. cbsHwModuleSupplyFGOutOfRange 64. cbsHwModule1-2VNormal 65. cbsHwModule1-2VOutOfRange 66. cbsHwModule12-0VNormal 67. cbsHwModule12-0VOutOfRange 68. cbsHwModuleNP6OCTDDROutOfRange 69. cbsHwModuleNP6OCTDDRNormal 70. cbsHwModuleNP6EZCOREOutOfRange 71. cbsHwModuleNP6EZCORENormal 72. cbsHwModuleNP6EZDDROutOfRange 73. cbsHwModuleNP6EZDDRNormal 74. cbsHwModuleNP6EZHSTLOutOfRange 75. cbsHwModuleNP6EZHSTLNormal 76. cbsHwModuleNP6XBPRCOutOfRange 77. cbsHwModuleNP6XBPRCNormal 78. cbsHwModuleETHSwVOutOfRange 79. cbsHwModuleETHSwVNormal 80. cbsHwModulePrismVOutOfRange 81. cbsHwModulePrismVNormal
287
CBS-MODULE-RESOURCES-MIB Reference
The CBS-MODULE-RESOURCE-MIB includes entries that describe module features such as CPU usage, memory usage, and other module characteristics. This section contains a tree view of the CBS-MODULE-RESOURCES-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-MODULE-RESOURCES-MIB.txt file on your CPM at /usr/share/snmp/mibs.
Identifying the CBS-MODULE-RESOURCES-MIB OIDs

Identify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-MODULE-RESOURCES-MIB management OID (.1.3.6.1.4.1.6848.2.3). The OID for Crossbeam module-resource-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.2) For example: The Crossbeam module CPU count (cbsModuleCPUCount) OID is .1.3.6.1.4.1.6848.2.3.1.1.2 The trap for module CPU load exceeded (cbsModuleCPULoadExceeded) OID is .1.3.6.1.4.1.6848.4.2.1
CBS-MODULE-RESOURCES-MIB entries
CBS-MODULE-RESOURCES-MIB management OID: .1.3.6.1.4.1.6848.2.3 1. cbsModuleCPULoadTable 1. cbsModuleCPULoadEntry 1. 2. 3. 4. 5. 6. 7. 8. 2. 1. cbsModuleCPUSpeed cbsModuleCPUCount cbsModuleCPULoad1 cbsModuleCPULoad5 cbsModuleCPULoad15 cbsModuleCPUUtil1 cbsModuleCPUUtil5 cbsModuleCPUUtil15
cbsModuleMemoryTable cbsModuleMemoryEntry 1. 2. 3. cbsModuleMemoryTotal cbsModuleMemoryUsed cbsModuleMemoryFree
3.
cbsModuleSwapTable 1. cbsModuleSwapEntry 1. 2. 3. cbsModuleSwapTotal cbsModuleSwapUsed cbsModuleSwapFree
4.
cbsModuleDUTable 1. cbsModuleDUEntry
288
1. 2. 3. 4. 5. 1.
cbsModuleDURoot cbsModuleDUBoot cbsModuleDUCbconfig cbsModuleDUTftpboot
cbsModuleFreePageTable cbsModuleFreePageEntry 1. 2. 3. 4. cbsModuleFreePageAvailable cbsModuleFreePageThreshold cbsModuleFreePageSeverity cbsModuleFreePageVapName
6.
cbsModuleCPUAvgUtilTable 1. CbsModuleCPUAvgUtilEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsModuleCPUAvgUtilCore1User cbsModuleCPUAvgUtilCore1Nice cbsModuleCPUAvgUtilCore1Syst cbsModuleCPUAvgUtilCore1Idle cbsModuleCPUAvgUtilCore1IRQ cbsModuleCPUAvgUtilCore1SoftIRQ cbsModuleCPUAvgUtilCore1IOWait cbsModuleCPUAvgUtilCore2User cbsModuleCPUAvgUtilCore2Nice
10. cbsModuleCPUAvgUtilCore2Syst 11. cbsModuleCPUAvgUtilCore2Idle 12. cbsModuleCPUAvgUtilCore2IRQ 13. cbsModuleCPUAvgUtilCore2SoftIRQ, 14. cbsModuleCPUAvgUtilCore2IOWait 15. cbsModuleCPUAvgUtilCore3User 16. cbsModuleCPUAvgUtilCore3Nice 17. cbsModuleCPUAvgUtilCore3Syst 18. cbsModuleCPUAvgUtilCore3Idle 19. cbsModuleCPUAvgUtilCore3IRQ 20. cbsModuleCPUAvgUtilCore3SoftIRQ 21. cbsModuleCPUAvgUtilCore3IOWait 22. cbsModuleCPUAvgUtilCore4User 23. cbsModuleCPUAvgUtilCore4Nice 24. cbsModuleCPUAvgUtilCore4Syst 25. cbsModuleCPUAvgUtilCore4Idle 26. cbsModuleCPUAvgUtilCore4IRQ 27. cbsModuleCPUAvgUtilCore4SoftIRQ 28. cbsModuleCPUAvgUtilCore4IOWait
289
29. cbsModuleCPUAvgUtilCore5User 30. cbsModuleCPUAvgUtilCore5Nice 31. cbsModuleCPUAvgUtilCore5Syst 32. cbsModuleCPUAvgUtilCore5Idle 33. cbsModuleCPUAvgUtilCore5IRQ 34. cbsModuleCPUAvgUtilCore5SoftIRQ 35. cbsModuleCPUAvgUtilCore5IOWait 36. cbsModuleCPUAvgUtilCore6User 37. cbsModuleCPUAvgUtilCore6Nice 38. cbsModuleCPUAvgUtilCore6Syst 39. cbsModuleCPUAvgUtilCore6Idle 40. cbsModuleCPUAvgUtilCore6IRQ 41. cbsModuleCPUAvgUtilCore6SoftIRQ 42. cbsModuleCPUAvgUtilCore6IOWait 43. cbsModuleCPUAvgUtilCore7User 44. cbsModuleCPUAvgUtilCore7Nice 45. cbsModuleCPUAvgUtilCore7Syst 46. cbsModuleCPUAvgUtilCore7Idle 47. cbsModuleCPUAvgUtilCore7IRQ 48. cbsModuleCPUAvgUtilCore7SoftIRQ 49. cbsModuleCPUAvgUtilCore7IOWait 50. cbsModuleCPUAvgUtilCore8User 51. cbsModuleCPUAvgUtilCore8Nice 52. cbsModuleCPUAvgUtilCore8Syst 53. cbsModuleCPUAvgUtilCore8Idle 54. cbsModuleCPUAvgUtilCore8IRQ 55. cbsModuleCPUAvgUtilCore8SoftIRQ 56. cbsModuleCPUAvgUtilCore8IOWait 7. CbsModuleSDPTable 1. CbsModuleSDPEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsModuleSDP0OutPkts cbsModuleSDP0OutOctets cbsModuleSDP0InPkts cbsModuleSDP0InOctets cbsModuleSDP1OutPkts cbsModuleSDP1OutOctets cbsModuleSDP1InPkts cbsModuleSDP1InOctets cbsModuleSDP2OutPkts
10. cbsModuleSDP2OutOctets
290
11. cbsModuleSDP2InPkts 12. cbsModuleSDP2InOctets 13. cbsModuleSDP3OutPkts 14. cbsModuleSDP3OutOctets 15. cbsModuleSDP3InPkts 16. cbsModuleSDP3InOctets 17. cbsModuleSDP4OutPkts 18. cbsModuleSDP4OutOctets 19. cbsModuleSDP4InPkts 20. cbsModuleSDP4InOctets 21. cbsModuleSDP5OutPkts 22. cbsModuleSDP5OutOctets 23. cbsModuleSDP5InPkts 24. cbsModuleSDP5InOctets 25. cbsModuleSDP6OutPkts 26. cbsModuleSDP6OutOctets 27. cbsModuleSDP6InPkts 28. cbsModuleSDP6InOctets 29. cbsModuleSDP7OutPkts 30. cbsModuleSDP7OutOctets 31. cbsModuleSDP7InPkts 32. cbsModuleSDP7InOctets 8. cbsModuleUptimeTable 1. 9. cbsModuleUptimeEntry 1. 1. cbsModuleUptime
cbsModuleNPMFlowCountTable CbsModuleNPMFlowCountEntry 1. 1. CbsModuleNPMFlowCount
10. CbsModuleAppMonTable CbsModuleAppMonEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsModuleAppName cbsModuleAppVersion cbsModuleAppRelease cbsModuleAppCPMHostName cbsModuleAppCPMIPAddress cbsModuleAppVAPGroupName cbsModuleAppVAPIndex cbsModuleAppOldState cbsModuleAppNewState
11. CbsModuleNtpdMonTable
291
1.
CbsModuleNtpdMonEntry 1. 2. 3. cbsModuleNtpdCPMHostName cbsModuleNtpdCPMIPAddress cbsModuleNtpdState
CBS-MODULE-RESOURCES-MIB Trap Entries

CBS-MODULE-RESOURCES-MIB trap OID: .1.3.6.1.4.1.6848.4.2 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsModuleCPULoadExceeded cbsModuleCPULoadNormal cbsModuleMemoryUsageExceeded cbsModuleMemoryUsageNormal cbsModuleCPUUtilExceeded cbsModuleCPUUtilNormal cbsModuleDURootHigh cbsModuleDURootNormal cbsModuleDUBootHigh
10. cbsModuleDUBootNormal 11. cbsModuleDUCbconfigHigh 12. cbsModuleDUCbconfigNormal 13. cbsModuleTftbootHigh 14. cbsModuleTftbootNormal 15. cbsModuleFreePageAvailableHigh 16. cbsModuleFreePageAvailableNormal 17. cbsModuleMUHigh 18. cbsModuleMUNormal 19. cbsModuleFRHigh 20. cbsModuleFRNormal 21. cbsModuleBUHigh 22. cbsModuleBUNormal 23. cbsModuleAppStateChange 24. cbsModuleNtpdMonStateChange 25. cbsModuleDUMgmtHigh 26. cbsModuleDUMgmtNormal 27. cbsModuleCpuCoreUtilExceeded 28. cbsModuleCpuCoreUtilNormal 29. cbsNpmFlowTableUsageHigh 30. cbsNpmFlowTableUsageNormal 31. cbsNpmFTUdpLimitExceeded 32. cbsNpmFTUdpLimitNormal 33. cbsNpmFTUdpMedianExceeded
292
34. cbsNpmFTUdpMedianNormal 35. cbsNpmFTTcpLimitExceeded 36. cbsNpmFTTcpLimitNormal 37. cbsNpmFTTcpMedianExceeded 38. cbsNpmFTTcpMedianNormal 39. cbsNpmFTIcmpLimitExceeded 40. cbsNpmFTIcmpLimitNormal 41. cbsNpmFTIcmpMedianExceeded 42. cbsNpmFTIcmpMedianNormal 43. cbsNpmFTOtherIpLimitExceeded 44. cbsNpmFTOtherIpLimitNormal 45. cbsNpmFTOtherIpMedianExceeded 46. cbsNpmFTOtherIpMedianNormal 47. cbsModuleSdramCheck
CBS-VAPGROUP-MIB Reference
The CBS-VAPGROUP-MIB includes entries that describe VAP features, configuration settings, and other details. This section contains a tree view of the CBS-VAPGROUP-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-VAPGROUP-MIB.txt file on your CPM at /usr/share/snmp/mibs.
Identifying the CBS-VAPGROUP-MIB OIDs

Identify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-VAPGROUP-MIB management OID (.1.3.6.1.4.1.6848.2.4). The OID for Crossbeam VAP group specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.3) For example: The VAP group name (cbsVgVapGroupName) OID is .1.3.6.1.4.1.6848.2.4.1.1.2
CBS-VAPGROUP-MIB entries
CBS-VAPGROUP-MIB management OID: .1.3.6.1.4.1.6848.2.3 1. cbsVapGroupTable 1. cbsVapGroupEntry 1. 2. 3. 4. 5. 6. cbsVgVapGroupID cbsVgVapGroupName cbsVgLoadPriority cbsVgPreemPriority cbsVgApmList cbsVgVapCount
293
7. 8. 9.
cbsVgMaxLoadCount cbsVgLBList cbsVgBackUpMode
10. cbsVgIpForward 11. cbsVgRpFilter 12. cbsVgLogMartians 13. cbsVgReloadTimer 14. cbsVgRemoteBoxBackup 15. cbsVgDelayFlow 2. cbsVapMappingTable 1. cbsVapMappingEntry 1. 2. 3. 4. 5. 6. cbsVmVapGroupID cbsVmVapGroupName cbsVmVapName cbsVmVapIndex cbsVmVapStatus cbsVmSlotNumber
CBS-VAPGROUP-MIB Trap Entries

CBS-VAPGROUP-MIB Trap OID: .1.3.6.1.4.1.6848.4.3 1. cbsVapStatusChanged
CBS-VRRP-MIB Reference
The CBS-VRRP-MIB includes entries for virtual routing redundancy protocol (VRRP). This section contains a tree view of the CBS-VAPGROUP-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-VAPGROUP-MIB.txt file on your CPM at /usr/share/snmp/mibs.
Identifying the CBS-VRRP-MIB OIDs

Identify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-VRRP-MIB management OID (.1.3.6.1.4.1.6848.2.5). The OID for Crossbeam VRRP-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.4) For example: The trap for a new master for VRRP (cbsVrrpTrapNewMaster) OID is .1.3.6.1.4.1.6848.4.4.1
CBS-VRRP-MIB entries
CBS-VRRP-MIB management OID: .1.3.6.1.4.1.6848.2.5 1. cbsVrrpOperations 1. cbsVrrpTrapNewMasterReason
294
2. 3. 4. 5. 6. 7. 8. 9.
cbsVrrpTrapNewMasterFailGrName cbsVrrpTrapNewMasterVrId cbsVrrpTrapNewMasterCirId cbsVrrpTrapNewMasterCirName cbsVrrpTrapProtoErrReason cbsVrrpTrapFailGrOldStatus cbsVrrpTrapFailGrNewStatus cbsVrrpTrapFailGrStatusChgReason
10. cbsVrrpTrapFailGrOldPriority 11. cbsVrrpTrapFailGrNewPriority 12. cbsVrrpTrapFailGrPriorityChgReason 13. cbsVrrpTrapRemoteBoxPathOldStatus 14. cbsVrrpTrapRemoteBoxPathNewStatus 15. cbsVrrpRemBoxPathAddrs 2. cbsVrrpFailGrTable 1. cbsVrrpFailGrEntry 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsVrrpFailGrID cbsVrrpFailGrName cbsVrrpFailGrStatus cbsVrrpFailGrConfigPriority cbsVrrpFailGrActualPriority cbsVrrpFailGrEnabled cbsVrrpFailGrPreemption cbsVrrpFailGrLastChangeTime cbsVrrpFailGrLastChangeReason
10. cbsVrrpFailGrMasterSysID 11. cbsVrrpFailGrMasterPriority 3. cbsVrrpFailGrCompTable 1. cbsVrrpFailGrCompEntry 1. 2. 3. 4. 5. 6. 4. 1. cbsVrrpFailGroupID cbsVrrpFailGroupName cbsVrrpFailGrCompIndex cbsVrrpFailGrCompType cbsVrrpFailGrCompDescr cbsVrrpFailGrCompDelta
cbsVrrpRemoteBoxPathTable cbsVrrpRemoteBoxPathEntry 1. 2. 3. cbsVrrpRemoteBoxID cbsVrrpRemoteBoxPathIndex cbsVrrpRemoteBoxPathAddr
295
4. 5. 6. 7. 8.
cbsVrrpRemoteBoxPathLocalIntf cbsVrrpRemoteBoxPathLocalAddr cbsVrrpRemoteBoxPathStatus cbsVrrpRemoteBoxPathLastChangeTime cbsVrrpRemoteBoxPathLinkQuality
CBS-VRRP-MIB Trap Entries

CBS-VRRP-MIB Trap OID: .1.3.6.1.4.1.6848.4.4 1. 2. 3. 4. 5. 6. 7. 8. 9. cbsVrrpTrapNewMaster cbsVrrpTrapProtoError cbsVrrpTrapFailGrStatusChanged cbsVrrpTrapFailGrPriorityChanged cbsVrrpTrapRemoteBoxPathStatusChanged cbsVrrpTrapRemoteBoxNoActivePath cbsVrrpTrapRemoteBoxNoSecondaryPath cbsVrrpTrapRemoteBoxNoStandbyPath cbsVrrpTrapRemoteBoxPathSharedIntf
10. cbsVrrpTrapNoRemoteBoxConfigured 11. cbsVrrpTrapOnlyOnePathDefined 12. cbsVrrpTrapRemoteBoxRunLegacyXOS
Standard MIBs on the Crossbeam X-Series Platform

In addition to the Crossbeam proprietary MIBs, the X-Series Platform uses the following standard MIBs:
DISMAN-EVENT-MIB IF-MIB IP-FORWARD-MIB IP-MIB IPV6-MIB NOTIFICATION-LOG-MIB RFC1213-MIB RMON-MIB SNMPv2-MIB TCP-MIB UDP-MIB
To view specific MIB entries used by XOS, use an snmpwalk tool. For information on configuring SNMP on XOS, including SNMP server setup, see Configuring Simple Network Management Protocol (SNMP) on page 163.
296
E
Maintenance Mode
Using Maintentance Mode
Maintenance mode can be applied to Application Processor Modules (APMs) and to Network Processor Modules (NPMs) for these purposes:
z z z
Upgrading the firmware on the module (APM and NPM) Applying hotfixes or patches while upgrading an application (APM) Replacing an existing APM or NPM module with a newer version
When peforming firmware upgrade or patch/hotfix operations, the process involves these steps:
z z z
Configuring the module to place it in maintenance mode Performing the operation on the module Enabling the module after the operation has been completed
When replacing an APM or NPM module with a newer version the process involves these steps:
z z z
Placing the module in maintenance mode Removing the module Inserting the newer version of APM or NPM
297
Maintenance Mode
298
Glossary
The following terms are used throughout the X-Series Platform documentation set.
3DES
Triple Data Encryption Standard. Provides a stronger form of DES encryption where the algorithm is applied three times in order to encrypt data.
ACL
Access Control List. Provides packet filtering through the permission or denial of packets based on certain IP criteria, such as IP address, port, or protocol.
APM
Application Processor Module. The XOS Application Processor system module that provides application processing, status monitoring, and standard and application specific logging. The APM contains one or more CPUs to host applications and network services while processing packets belonging to individual flows.
ARP
Address Resolution Protocol. An Internet protocol used to map an IP address to a MAC address.
BOTW
Bump-on-the-Wire. A device with two or more interfaces that are transparent to the adjacent Layer 3 devices.
cbsflowagentd
Flow Agent daemon that collects statistics and runs on each VAP.
cbsflowcalcd
Flow Calculator daemon that runs the flow scheduling chow file and executes on the CPM.
circuit
An abstract object representing a logical network interface (network service access point). A circuit can be mapped to either single or multiple logical lines. Attributes of a circuit include: a set of physical line or channel pairs, a Layer 2 encapsulation type, a Layer 2 address, and an IP address (optional).
CLI
Command Line Interface.
CM
Configuration manager/monitor.
core-intf
An interface which is attached to the core-facing networks.
299
CPM
Control Processor Module. The XOS system module that coordinates the actions of all other modules, enables management access to the platform, and supports access to a local disk containing configuration files and databases necessary to execute the applications which reside on the platform.
DES
Data Encryption Standard. A popular algorithm for encrypting data. It is a product cipher that operates on 64-bit blocks of data, using a 56-bit key.
device
OS concept representing either a physical or logical I/O port connected to the APM.
domain
A set of interconnected IP networks belonging to a unique address space. A domain is uniquely identified within the X-Series Platform by a 8-bit domain ID. IP flows must be unique within a given domain.
DSA
Digital Signature Algorithm.
ECC
Error Checking and Correcting. A collection of methods to detect errors in transmitted or stored data and a means to correct them.
edge interface
An interface that is attached to edge-facing networks (typically where subscribers are located).
edge server
A server that is physically located close to its end users designed to deliver faster, higher quality transmissions, typically in a local commercial ISP facility. The number of edge servers in a region depends on the number of users in the locale.
Element Management System

Graphical user interface for X-Series Platforms accessed via the web server.
Error Checking and Correcting (ECC)

A collection of methods for detecting and correcting errors in transmitted or stored data.
FCAPS
Faults, Configuration, Accounting, Performance, and Security. The general requirements of a network management system as defined by the International Organization for Standardization.
FIB
Forwarding Information Base. A set of IP data structures replacing a route table in Linux.
firewall
A set of software tools that protects a company's internal network from unwanted entry by unauthorized external users. The firewall works in conjunction with a router program to filter incoming network packets and reject those of unknown origin.
Glossary
300
flow
Specific stream of data traveling between two endpoints across a network. Specified by source IP, destination IP, source port, destination port and IP protocol type.
flow rule
A filter rule specifying how a packet is processed.
flow specific
A stream of data traveling between two endpoints across a network. Specified by source IP, destination IP, source port, destination port and IP protocol type.
flow table
A table maintained on the NPM that maps individual flows to their respective processors.
FPGA
Field Programmable Gate Array. A gate array where the logic network can be programmed into the device after its manufacture. An FPGA consists of an array of logic elements, either gates or lookup table RAMs, flip-flops, and programmable interconnect wiring.
FTP
File Transfer Protocol.
gateway
A Layer 3 devices with at least two logical interfaces, that uses a routing table to forward packets between interfaces. Note that a gateway may also act as a multi-homed host.
GBIC
Gigabit Interface Converter. A transceiver that converts electric currents (digital highs and lows) to optical signals, and optical signals to digital electric currents. The GBIC is typically employed in fiber optic and Ethernet systems as an interface for high-speed networking. The data transfer rate is one gigabit per second (1 Gbps) or more.
GEM
Greenlight Element Manager. A GUI that provides a view into the components and health of your X-Series Platform.
GLM
Gigabit Link Module.
GRUB
GRand Unified Bootloader.
hash
A crytographic operation where an entire message is run through a mathematical operation that results in a fixed-length string that is unique.
HTTP
Hypertext Transfer Protocol.
301
IDEA
International Data Encryption Algorithm. A conventional encryption algorithm, using block cipher, operating on 64-bit blocks with a 128 bit key.
IDS
Intrusion Detection System.
IOP
I/O Processor.
IP Address
Internet Protocol (IP). A numerical address that identifies senders and receivers of Internet data. The address accompanies data packets, identifying a particular network on the Internet and the specific device (such as a server) from which it originated.
IPS
Intrusion Prevention System.
In-Service Upgrade
ISU is an alternate method of upgrading XOS software while minimizing downtime. This feature has several requirements for successful completion of an ISU, including redundant CPMs, APMs, and NPMs. During ISU, the chassis is virtually split in two halves during which time only one half of the chassis will be responsible for forwarding traffic.
LACP
Link Aggregation Control Protocol.
load balancing
Distributing flows in real time amongst multiple APMs.
load table
A table that maps flow profiles to weighted lists of virtual processors.
logical interface
A channelized interface on a physical interface. A subdivision of a physical interface. Currently supported logical interface types are default and VLAN.
logical line
A combination of a physical line and a sub-line (channel). A logical line is uniquely identified by a physical line ID or channel ID pair.
MD5
Message Digest 5. A one-way function that takes a variable-length message and produces a fixed-length hash.
MAC address
Media Access Control (MAC). A hardware address that uniquely identifies each node of a network. In IEEE 802 networks, the Data Link Control (DLC) layer of the OSI Reference Model is divided into two sub-layers: the Logical Link Control (LLC) layer and the Media Access Control (MAC) layer. The MAC layer interfaces directly with the network media. Consequently, each different type of network media requires a different MAC layer.
Glossary
302
MIB
Management Information Base.
MS
Management Server.
multi-homed host
A Layer 3 device with at least two logical interfaces that generate packets but does not forward the packets.
NMS
Network Management System. Platform that manages one or more X-Series Platforms in a networked environment.
NPM
Network Processor Module. The XOS module responsible for network interface access (up to 10 Gb/sec full-duplex), flow classification, distribution of flows to APMs, and load balancing of the APMs.
PGP
Pretty Good Privacy. A high-security RSA public-key encryption application that enables files or messages to be exchanged with privacy and authentication.
physical interface
The physical hardware connector on the NPM or CPM representing a network interface port.
POS
Packet Over Sonet.
POST
Power On Self-Test.
PPP
Point to Point Protocol.
RAID
Redundant Array of Inexpensive/Independent Drives. A data storage scheme used to allow multiple drives to work as a single drive. RAID level 0 and level 1 are supported by Crossbeam Systems in our newer modules. RAID 0 writes data to whichever drive is currently free. This method is used for greater data speed efficiency (however, all drives in the RAID are needed to fully access all the data). RAID 1 writes identical data to all the drives in the RAID grouping. This method is used for greater data integrity.
RDRAM
Rambus Direct Random Access Memory.
RMON
Remote Network Monitoring.
RPM
Red Hat Package Manager.
303
SCP
Secure copy.
SMP
Symmetric Multi Processor.
SNMP
Simple Network Management Protocol. The Internet standard protocol developed to manage and monitor nodes on an IP network.
SSH
Secure Shell. A powerful authentication and encryption program replacing older and less secure tools like Telnet. SSH provides both authentication and encryption and is therefore the preferred method of network access. SSH allows a secure connection to be established between a client computer and a server host. The X-Series Platform provides SSH server, SSH client, and scp capability.
SSL
Secure Socket Layer.
Stateful Inspection (dynamic packet filtering)

A firewall architecture operating at the network layer. Unlike static packet filtering, which examines a packet based on the information in its header, stateful inspection examines not just the header information but also the contents of the packet up through the application layer in order to determine more about the packet than just information about its source and destination. A stateful inspection firewall also monitors the state of the connection and compiles the information in a state table. Because of this, filtering decisions are based not only on administrator-defined rules (as in static packet filtering) but also on context that has been established by prior packets that have passed through the firewall. As an added security measure against port scanning, stateful inspection firewalls close off ports until connection to the specific port is requested.
static route
A user-defined route that causes packets to move between a source and destination along a specific path.
sub-line
A multiplexed channel within a single line. Examples include: a DS0 channel within a T1/T3 serial interface, a ATM PVC, and a tagged VLAN. A sub-line is uniquely identified by a 32-bit channel ID.
SYSLOGD
System Logger Daemon.
Telnet
An administrator can enable Telnet as part of the boot dialogue, or by using a CLI command. Telnet comes disabled because traffic is not encrypted between the client and the X-Series Platform.
VAP
Virtual Application Processor. An application operating environment which can be run on an APM. A VAP consists of the OS, system software, and a set of applications which run concurrently.
VAP group
A virtual set of Application Processor Modules identically configured for load balancing and redundancy to process the same set of applications.
Glossary
304
VLAN
Virtual Local Area Network. A local area network with a definition that maps workstations on some other basis than geographic location (for example, by department, type of user, or primary application).
VND
Virtual Network Device. A Linux kernel object representing a logical network interface. A virtual network device is directly mapped to an NPM circuit.
VPN
Virtual Private Network. Consists of private lines, switching equipment and other networking equipment that are provided for the exclusive use of one customer. A VPN gives users a secure way to access resources over the Internet or other public or private networks using encryption, authentication, and tunneling.
VRRP
Virtual Router Redundancy Protocol. This protocol allows several routers on a multiaccess link to utilize the same virtual IP address. One router will be elected as a master with the other routers acting as backups in case of the failure of the master router. The protocol should also support the ability to load share traffic when both routers are up. A Virtual Router in XOS is an IP address or a set of IP addresses that can be instantiated on a circuit for a subset of the VAP groups on which the circuit is configured, and active only on one of the X-Series Platforms participating in multi-system High Availability configuration.
XML
Extensible Markup Language. The universal format for structured documents and data on the Web as defined by a set of specifications and recommendations from the W3C.
305
Glossary
306

XOS 9.5.4.0 Xos Configuration Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

XOS 9.5.4.0 Xos Configuration Guide

Uploaded by

Copyright:

Available Formats

XOS Configuration Guide

Version #: XOS 9.5.1

Part Number 03392P March 2011

Copyright and Trademark Information

Chapter 2: Initial Configuration

Chapter 3: Configuring System Settings

Chapter 4: Configuring Administration Settings

Chapter 5: Configuring VAP Groups

Chapter 6: Configuring Interfaces

Chapter 7: Configuring Management Interfaces

Chapter 8: Configuring Flow Provisioning

Chapter 9: Configuring Protocols

Chapter 10: Configuring Bridge Mode

XOS Configuration Guide

Chapter 11: Configuring CPM Redundancy

Chapter 12: Configuring Multi-System High Availability

Chapter 13: Configuring Secure Flow Processing

Chapter 14: Configuring RMON and SNMP Traps

Chapter 16: Viewing Performance, Statistics, and Status

Chapter 17: Viewing Alarms, Events, and Logs

Chapter 18: Upgrading XOS Software and Firmware

Chapter 19: Configuring Routing Protocols

XOS Configuration Guide

217 219 221 222

Chapter 20: Backups, Upgrades, and Reinstallation

Appendix A: In-Service Upgrade

Appendix B: RAID-Related Hard Drive Configuration and Repair

Appendix C: Swatch Dynamic Display Tool

Appendix D: Crossbeam SNMP MIB Reference

Appendix E: Maintenance Mode

XOS Configuration Guide

About This Guide

Typographical Conventions Used in Paragraph Text

Typographical Convention Bold

In the Version Number field, type 8.5.patch_number.

About This Guide

Typographical Conventions Used in Command-Line Text

Typographical Convention Courier

CBS# configure dns server <IP_address> [vap-group <VAP_group_name>]

CBS# cp-unknown-state {cp1|cp2}

CBS# configure vap-group <vap_group_name> CBS(config-vap-grp)# raid {0|1}

Cautions, Warnings, and Notes

IPv6 Address Notation

Standard Notation Compressed Notation

About This Guide

0000::/8 0100::/8 0200::/7 0400::/6

About This Guide

XOS Configuration Guide

Network Processor Module

Control Processor Module

Application Processor Module

Module Link Status

Use the show heartbeat command to view the heartbeat status.

Virtual Application Processor (VAP)

XOS Configuration Guide

Example of Traffic Flow

Security applications are provided separately.

Greenlight Element Manager

with Internet Explorer 7 or 8 with Firefox 3.0 or 3.5

XOS Configuration Guide

Third Party Pop-ups on GEM

Automated Workflow System

Greenlight Element Manager (GEM)

Command Line Interface