Embedded Diagnostics and Management in Cisco IOS: TCL, EEM Scripting and Service Diagnostics

Embedded Diagnostics
and Management in Cisco

IOS: TCL, EEM Scripting
and Service Diagnostics
BRKDEV-1191
BRKDEV-1191
14596_04_2008_c2 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 2
© 2006, Cisco Systems, Inc. All rights reserved. 1

Presentation_ID.scr
BRKDEV-1191
Session ID: BRKDEV-1191
Title: Embedded Diagnostic and Mgmt in IOS: TCL, EEM Scripting
and Service Diagnostics
Abstract: In this session, we discuss new embedded management
capabilities in IOS including Service Diagnostics, EEM (Embedded
Event Manager), EMM (Embedded Menu Manager), and TCL
scripting. The session presents use-cases for autonomic
diagnostic solutions using Cisco provided scripts. The session
concludes with an overview of the new EEM event detectors and
how they can be used to build your own custom embedded
solutions.
Owner:
Speakers: Rick Williams, rwill@cisco.com
Product Manager
OS Infrastructure and Services
Software Group
Cisco Systems, Inc.
Level: Introductory
BRKDEV-1191
What Do We Mean By “Embedded

Diagnostics and Management”?
Embedded = On the box (Router
or Switch)
Diagnostics and Management
Two applications for on-board
programming
The ability to develop programs
within Cisco IOS can enhance
other applications
Partners and ISVs may benefit
from on-board programming
End customers also benefit from
automation and customization
In this session we’ll introduce
Cisco IOS Embedded Event
Manager, Embedded Menu
Manager, and Service Diagnostics
BRKDEV-1191

Presentation_ID.scr
Today’s Network Management
Environment
Centralized, server-based
Data gathering, polling, event
correlation, root cause analysis
Multiple, network management
application specific systems
Trend for past 5-8 years has been
toward off the shelf products
Lately, incorporating Linux and
Open-Source tools
Managed Service Providers: Tighter
integration
Enterprises: Loosely integrated;
Issues with integration, scale,

accuracy, customization,
control, SPEED
BRKDEV-1191
The Future of Network Management

Networks are getting larger Manager of
Managers
and larger—Increased Si Si Si
dependence
Management Throughout
Context or Domain
High dependence means Network
higher availability Management
Si Si
requirements
Reaction times are shorter Onboard
Device
Management
Continuous cost pressures
More “intelligence” in the net
All driving need for automation
and distributed/cooperative
network management
Devices play an active
¼ Embedded Management
role in management
BRKDEV-1191

Presentation_ID.scr
Cisco IOS
Embedded Event
Manager (EEM)
BRKDEV-1191
EEM—What and Why

What in the world is an embedded event manager?
An extremely flexible and very powerful subsystem within Cisco
IOS Software
Using EEM you can:
Write programs that run on the router or switch
OK, so what? Why do I care?

It’s about automation—onboard automation
Automate troubleshooting
Automate commands
Program automatic actions based on events
Only limited by your imagination
Really just the tip of the iceberg—as we’ll soon see
BRKDEV-1191

Presentation_ID.scr
EEM Basic Architecture
* All within Cisco IOS
Policies (Scripts)
Applets and Tcl-based
EEM Server
The “brains” of the system
Event Detectors
“watch for events of interest”
BRKDEV-1191
EEM Event Detectors and EEM Policies

All of this is internal to Cisco IOS
ED notifies EEM
Think of a policy as Server; which
an action registered triggers interested
to an event policies
Applet-based policies Tcl-based policies

Defined via CLI Programmed in Tcl
Simpler As complex as you want
BRKDEV-1191

Presentation_ID.scr
Let’s Make an EEM Policy
Let’s start with a very simple applet policy

Let’s write a special syslog message when we see a
particular syslog message
For example:
When someone leaves config mode, this message is seen:
BRKDEV-1191
Configure an EEM Applet Policy
We can make an Applet policy to match the on the syslog

message
Uses a REGEXP match
Syslog Event
We will register our policy…
… Syslog Event Detector will notify the EEM Server when the
message string match occurs
Syslog Action
… Our policy action will be invoked
BRKDEV-1191

Presentation_ID.scr
Applet Configuration Detail
event manager applet CFGMSG
Define applet
named
CFGMSG
Event type will

be “syslog”
iin-rtr1(config-applet)#event syslog
BRKDEV-1191
Syslog REGEXP Match “Pattern”
iin-rtr1(config-applet)#event syslog pattern "%SYS-5-CONFIG_I:"
BRKDEV-1191

Presentation_ID.scr
Add the Action—Syslog Action
iin-rtr1(config-applet)#action 1.0 syslog
Label—used to
sort actions
Alphabetic sort
on the label
BRKDEV-1191
Complete Applet Policy Definition

event manager applet CFGMSG
event syslog pattern "{%SYS-5-CONFIG_I:}"
action 1.0 syslog priority warnings msg
"Configuration event occurred"
BRKDEV-1191

Presentation_ID.scr
Embedded Event Manager v2
Event Detectors *Not all available in all releases
Cisco IOS CLI—Triggers policies based on Interface Counter—Policies can be

commands entered via the CLI. triggered based on the specific interface
Cisco IOS Counter—Policies can be counter. Includes thresholds.
triggered based on a change of the Online Insertion and Removal—Triggers
designated Cisco IOS counter. policies based on hardware installation and
Cisco IOS Redundancy Facility—Provides removal activity.
for detection of hardware and software Object Tracking—Triggers policies based
failures related to the Stateful Switchover on routing protocol events.
service. This ED will trigger policies based Simple Network Management Protocol—
on the RF state change. It is also used to Triggers policies based on the associated
initiate switchovers as a result of a policy SNMP MIB variable. Includes MIB variable
action. threshold setting.
Cisco IOS Timer Services—Policies can Syslog—Triggers policies based on the
be scheduled to occur at the designated regular expression match of a local Syslog
time or interval. message.
Cisco IOS Watchdog/System Monitor— Resource Thresholding—Triggers policies
Triggers policies based on certain conditions based on certain internal resource usage
relative to a certain Cisco IOS process or and conditions. Interface to Embedded
subsystem’s activity. Resource Manager.
EEM Application Specific—Application Generic Online Diagnostics—Triggers
specific events can be detected or set by a policies based on diagnostic results
Cisco IOS subsystem or a policy script. This
provides the ability for one policy to trigger “None” ED—Triggers policies by command
another policy.
BRKDEV-1191
Embedded Event Manager

Built-in Actions
An Embedded Event Manager Policy Can:
Execute an IOS CLI command and receive the result
Send a CNS event
Increment or decrement an EEM counter
Force a switchover to the standby in a redundant configuration
Request system information
Send an e-mail
Cause another EEM policy to be executed
Publish an application specific EEM event
Reload the box
Send an SNMP trap with custom data
Log a message to Syslog
BRKDEV-1191

Presentation_ID.scr
Environment Variables
Variables that are referenced within policies that
can be set in the config
Can be used to customize policies
Example:
Router#config t
Router(config)#event manager environment _email_server
email.cisco.com
Router(config)#event manager environment _email_from
soandso@somecompany.com
Note: environment variable names that start with

the underscore character are reserved for Cisco
use only
BRKDEV-1191
Environment Variables (Built-In)

Environment Variables Available for All Events
$_event_type The event type that triggered the event.
$_event_pub_time The time at which the event type was published.
Environment Variables Available for SNMP Events
$_snmp_oid The Simple Network Management Protocol (SNMP) object ID that caused the event to be published.
$_snmp_oid_val The SNMP object ID value when the event was published.
Environment Variables Available for Syslog Events
$_syslog_msg The syslog message that caused the event to be published.
When a policy is entered…

Cisco defined read-only environment variables called built-in
variables are pre-set with the characteristics of the event that
triggered the policy to run
These environment variables can be used in ‘msg’ text
Will be replaced with the relevant text
Can be checked by policies
There are a bunch of these—refer to Cisco documentation
BRKDEV-1191

Presentation_ID.scr
Tcl Policies or Scripts
Tcl V8.3.4+ script support

This is the same support available in IOS for tclsh, ESM
(Embedded Syslog Manager), and IVR
There are several Tcl language extension dialects and some
differences among them
EEM keyword extensions as documented in the “Guide
To Writing EEM Policies” documentation implement
event specification, system information requests, and
built-in actions
EEM Tcl library support is available for some common
functions such as CLI, SMTP, and Tcl global variable
state checkpointing.
EEM uses the ::cisco::eem Tcl namespace
BRKDEV-1191
Getting Started:
Create Policy Directory
ashcroft#mkdir ABCCoTclPol
Create directory filename [ABCCoTclPol]?
Created dir disk0:ABCCoTclPol
ashcroft#dir
Directory of disk0:/
1 drw- 1 Oct 26 2003 13:37:42 +00:00 sys
6 drw- 1 Oct 30 2003 12:56:04 +00:00 ABCCoTclPol
47843328 bytes total (29356032 bytes free)
ashcroft#conf t
Enter configuration commands, one per line. End with CNTL/Z.
ashcroft(config)#event manager directory user policy disk0:/ABCCoTclPol
ashcroft(config)#^Z
ashcroft#ashcroft#sh event man dir user policy

disk0:/ABCCoTclPol
ashcroft#
BRKDEV-1191

Presentation_ID.scr
Getting Started:
Copy Tcl Policy to Router
ashcroft#copy tftp disk0:
Address or name of remote host []? 88.1.88.9
Source filename []? sl_cfgSaveRemT.tcl
Destination filename [sl_cfgSaveRemT.tcl]? ABCCoTclPol/sl_cfgSaveRemT.tcl
Accessing tftp://88.1.88.9/sl_cfgSaveRemT.tcl...!
1232 bytes copied in 0.620 secs (1987 bytes/sec)
ashcroft#dir
Directory of disk0:/
1 drw- 1 Oct 26 2003 13:37:42 +00:00 sys
6 drw- 1 Oct 30 2003 12:56:04 +00:00 ABCCoTclPol
ashcroft#cd ABCCoTclPol
ashcroft#dir
Directory of disk0:/ABCCoTclPol/
8 -rw- 1232 Oct 30 2003 14:14:58 +00:00 sl_cfgSaveRemT.tcl
ashcroft#
BRKDEV-1191
Getting Started:
Register the Policy
ashcroft#conf t
Enter configuration commands, one per line. End with CNTL/Z.
ashcroft(config)#event manager policy sl_cfgSaveRemT.tcl type user
ashcroft(config)#
ashcroft#sh event manager policy registered

No. Type Event Type Trap Time Registered Name
1 user syslog Off Thu Oct30 14:54:17 2003 sl_cfgSaveRem.tcl
occurs 1 pattern {%SYS-5-CONFIG_I: Configured}
nice 0 priority normal maxrun 90.000
BRKDEV-1191

Presentation_ID.scr
Security of the Embedded
Event Manager
Cisco scripts run in full Tcl mode

User scripts run in Safe-Tcl mode
Safe-Tcl allows Cisco to disable or customize individual
Tcl commands
Certain commands are not permitted
User scripting can be completely disabled by omitting

the “event manager directory user…” command
Remember, you don’t have to use it!
BRKDEV-1191
Security of the Embedded

Event Manager
All config commands are privileged global config mode

commands
If users modify Cisco system scripts, they need to be
run from the user directory in user mode
When EEM policies execute CLI commands, the “event
manager session cli username <username>” command
specifies the username that is sent to TACACS+ for
command authorization
BRKDEV-1191

Presentation_ID.scr
Writing Your Own EEM Tcl-Based
Policies
Start By Reading the Manual :-)
See “Writing Embedded Event
Manager Policies”
EEM v2.2 Doc:
http://www.cisco.com/en/US/docs/ios/netm
gmt/configuration/guide/12_4t/nm_12_4t_
book.html
9
EEM v2.1.5 Doc:
http://www.cisco.com/en/US/docs/ios/
12_2sx/sw_modularity/configuration
9
guide/evnt_mgr.html
BRKDEV-1191
EEM Tcl Policy Structure

Begin with EEM Event
Register keyword
Required
Next is any input variables or
required environment variables
to control the script
Names space imports
Entry criteria for the policy
Required
Body (logic of the script)
Exit status
Required
BRKDEV-1191

Presentation_ID.scr
Example of Tcl Policy Structure
Registration command
(Tcl extension)
Tcl namespace (namespace
import)
::cisco::eem
This namespace includes all Tcl
commands closely related to
::cisco::lib
This namespace includes
auxiliary library commands that
are not necessarily specific to
the Embedded Event Manager
Body
BRKDEV-1191
RegExp Tcl Command

http://www.tcl.tk/man/tcl8.3/TclCmd/regexp.htm
regexp ?switches? exp string ?matchVar? ?subMatchVar
subMatchVar ...?
remote-pe#sh int fa5/0 | inc put,
2901 packets input, 1605860 bytes
3165 packets output, 189978 bytes, 0 underruns
remote-pe#
Suppose we want the in and out ‘numbers’ from the above output
tclsh
% set t "sh int fa5/0 | inc put,
2901 packets input, 1605860 bytes
3165 packets output, 189978 bytes, 0 underruns
remote-pe#"
% regexp {(\d+) packets input.+?(\d+) bytes.+?(\d+) packets output.+?(\d+) bytes} $t match pin bin pout bout
1
% set pin % set pout
2901 3165
% set bin % set bout
1605860 189978
BRKDEV-1191

Presentation_ID.scr
Cisco IOS Embedded
Event Manager
Examples
BRKDEV-1191
Automatic Write Running Config

to Flash Disk
Simple applet to keep a copy of running-config on flash disk
whenever “write memory” is issued
event manager applet backup-config
event cli pattern "write memory" sync yes
action 1.0 cli command "enable"
action 2.0 cli command "config t"
action 3.0 cli command "file prompt quiet"
action 4.0 cli command "end"
action 5.0 cli command "copy running disk0:running-config"
action 7.0 cli command "no file prompt quiet"
Dealing with interaction:

event manager applet test2 *cli ’pattern’ option added by DDTS CSCsc96567
event none maxrun 20
action 10 cli command "enable"
action 20 cli command "clear counters" pattern "\[confirm\]"
action 25 cli command "y"
action 30 cli command "disable"
action 40 syslog msg "test2 ran"
BRKDEV-1191

Presentation_ID.scr
Automatic Write Running Config
to Flash Disk (Alternative)
Simple applet to keep a copy of running-config on flash disk
whenever “write memory” is issued
If you truly want the nvram "write memory" to come after the copy
to flash you can continue to use "sync yes" and set a value for
_exit_status that is greater than 0
event manager applet wrmem

event cli pattern "write memory" sync yes
action 1.0 cli command "enable"
action 3.0 cli command "file prompt quiet"
action 5.0 cli command "copy running disk0:running-config"
action 7.0 cli command "no file prompt quiet"
set 9 _exit_status 1
BRKDEV-1191
Using Environment Variables
Configuration
remote-pe#conf t
Enter configuration com mands, one perline. End with CNTL/Z.
remote-pe(config)#event manager environment u_cfgSave_on 1
BRKDEV-1191

Presentation_ID.scr
Cli Command Procedure
#
# Local procedure for CLI interface
# Pass a list of cli commands and it returns a list of outputs
#
proc CLICmdProc {cmds} { #
global errorInfo lappend clicmd "show clock"
if [catch {cli_open} result] { lappend clicmd "show auto discovery qos"
error $result $errorInfo set cliout [CLICmdProc $clicmd]
} else {
array set cli1 $result
#
}
if [catch {cli_exec $cli1(fd) "enable"} result] {
error $result $errorInfo
}
if [catch {cli_exec $cli1(fd) "term len 0"} result] {
}
foreach a_cmd $cmds {
if [catch {cli_exec $cli1(fd) $a_cmd} result] {
} else {
lappend cmd_output $result
}
}
if [catch {cli_close $cli1(fd) $cli1(tty_id)} result] {
}
return $cmd_output
}
BRKDEV-1191
Cli Configuration Command Procedure

proc CLICfgProc {cmds} {
global errorInfo lappend cmdlist "interface $intf"
if [catch {cli_open} result] { lappend cmdlist "no auto discovery qos"
} else {
CLICfgProc $cmdlist
array set cli1 $result
}
if [catch {cli_exec $cli1(fd) "enable"} result] {
}
if [catch {cli_exec $cli1(fd) "config t"} result] { Note “enable”
error $result $errorInfo command
}
foreach a_cmd $cmds {
if [catch {cli_exec $cli1(fd) $a_cmd} result] {
} else {
set cmd_output $result
}
}
if [catch {cli_exec $cli1(fd) "end"} result] {
}
if [catch {cli_exec $cli1(fd) "write mem"} result] {
}
if [catch {cli_close $cli1(fd) $cli1(tty_id)} result] {
}
}
BRKDEV-1191

Presentation_ID.scr
Context Save/Retrieve Example
# Globals are saved in context

global bw_counter
#
#First time through, we don't have a context, so catch that error
#
if [catch {context_retrieve ctx1 bw_counter} bw_counter] {
set bw_counter 0
}
Do some stuff
if { $bool_val } {
incr bw_counter
#
# Code for sending a syslog message would go here....
#
context_save ctx1 bw_counter
return
}
BRKDEV-1191
File I/O Example

tenet#sh run | inc ofilename
event manager environment _ofilename disk2:/eem/output.dat
tenet#
#
# write to a file
#
#
if [file exists $_ofilename] {
puts "file $_ofilename being overwritten"
}
set myfileid [open $_ofilename w+]
foreach outs $cliout {
puts $myfileid $outs
}
close $myfileid
BRKDEV-1191

Presentation_ID.scr
Application Event Example
Policy A
# Register for an application event for sub_system ID 798 and type 10

event_register_appl sub_system 798 type 10
# Query the eventinfo
array set arr_einfo [event_reqinfo]
if{$_cerrno != 0} {
set result[format "component=%s; subsys err=%s; posix err=%s;\n%s" \
$_cerr_sub_num $_cerr_sub_err $_cerr_posix_err $_cerr_str]
error $result
}
#log a message
set msg [format "application event: arg1: %s arg2 %s arg3 %s arg4 %s" \
$arr_einfo(data1) $arr_einfo(data2) $arr_einfo(data3) $arr_einfo(data4)]
action_syslog priorityinfo msg $msg
Policy B
# Register for a timer event thattriggers once every 60 secs

# publish an event with sub_system ID of 798 and type 10
event_register_timer watchdog name timer1 time 60.0
event_publish sub_system 798 type 10 arg1 “thisis 1” arg2 “thisis 2”
arg3 “thisis 3” arg4 “thisis 4”
action_syslog priorityinfo msg “event type 10 published”
BRKDEV-1191
EEM Feature/Product Support Matrix

5/5/08 8:55 AM
Legend Shipping In EFT EC Planning N/A

CISCO ACCESS ROUTERS
Cisco
EEM Cisco 800 Cisco 1800 Cisco 2800 Cisco 3800 Cisco 1700 Cisco 2600 Cisco 2691 Cisco 3600 Cisco 3700
2600XM
Version Series Series Series Series Series Series Series Series Series
Series
1.0 12.3(11)T 12.3(11)T 12.3(11)T 12.3(4)T 12.3(4)T 12.3(4)T 12.3(4)T
2.0
2.1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1 12.3(14)T1
2.1.5
2.2 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T 12.4(2)T
2.3 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T 12.4(11)T
2.4 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T 12.4(20)T
3.0 12.4(pi9)T 12.4(pi9)T 12.4(pi9)T 12.4(pi9)T Planning Planning Planning Planning Planning Planning
CISCO 5000 SERIES & UP

EEM Cisco 7200 Cisco 7600 Cisco 12000 Cisco XR Cisco Cisco 7500 Cisco 5000
Cisco 7301 Cisco 7304 Cisco 10000
Version Series Series Series 12000 CRS-1 Series Series
1.0 12.0(26)S 12.0(26)S
See IOS-XR See IOS-XR
2.0 12.2(27)SBC
Fault Mgr Fault Mgr
2.1 12.3(14)T1 12.3(14)T1 12.2(28)SB 12.2(18)SXF5 12.2(28)SB 12.4M
Fault Mgr Fault Mgr
2.1.5
Fault Mgr Fault Mgr
2.2 12.4(2)T 12.4(2)T1
Fault Mgr Fault Mgr
2.3 12.4(11)T 12.2(33)SB 12.2(33)SB 12.2(33)SRB 12.2(33)SB 12.4(11)T
Fault Mgr Fault Mgr
12.2(SR) 12.2(SR) 12.2(SR) See IOS-XR See IOS-XR
2.4 12.4(20)T 12.2SR Planning
Eagle Eagle Eagle Fault Mgr Fault Mgr
3.0 12.4(pi9)T Planning Planning Planning Planning Planning Planning Planning
CISCO CATALYST SWITCHES

EEM Cisco 3750 Cisco 4500 Cisco 6500
Version Switches Switches Switches
1.0
2.0
IOS w/o
2.1 Modularity
12.2(18)SXF5
2.1.5
w/ Modularity
12.2(18)SXF4
Includes futures, subject to
2.2
2.3 12.2(40)SE 12.2(40)SE 12.2(33)SXH
change; no commitment implied
2.4 Winter 09 12.2(44)SE 12.2(33)SXI)
3.0 Planning Planning Halfdome
BRKDEV-1191

Presentation_ID.scr
Event Detectors
1.0 2.0 2.1 2.2 2.1+ 2.4 * Modular IOS only

BRKDEV-1191
What’s Coming in EEMv2.4?
Two new EEM Event Detectors

Event detectors are key pieces of infrastructure software that
drive the event engine
EEM RPC ED
Remote procedure call event detector—allows for SOA-type
interface where EEM policies can be invoked from outside
the box
SNMP Proxy ED
Allows for an event to be generated when a specific trap is
received at the router or switch—allows for local action based
on external triggers from other equipment—A UPS system, for
example, might tell a switch it is on battery back-up and local
action could be taken by an EEM policy (shut interfaces, reroute
traffic, gracefully shutdown modules)
BRKDEV-1191

Presentation_ID.scr
What Else Is Coming in EEMv2.4?
Introduce three new capabilities:

Multiple event support
Bytecode support
Registration substitution enhancement
Introduce three new CLI commands:

Script policy refresh
A new command to display the supported event detectors
Add clear command to kill a Tcl script
And other ease of use enhancements
BRKDEV-1191
What Other Improvements Are

Coming in EEMv2.4?
ait,
Improve two Event Detectors: But w ore!
’s m
Interface Counter ED—rate based trigger There
SNMP ED enhancement—delta value
When and Where?

Cisco IOS Embedded Event Manager v2.4 will be available with
the Cisco IOS Software 12.2SXI—Whitney 2 release for the Cisco
Catalyst 6500 Series switches,
Cisco IOS Software Release 12.4(20)T for the Cisco ISRs,
7600 Series SR with ‘Eagle’ release,
Already in 3K metro Ethernet 12.2(40)SE, more lower-end switch
support to be announced (3k, 4k)
Other releases will gain support as well—publish dates to be
determined
BRKDEV-1191

Presentation_ID.scr
Multiple Event Support
New
Provides multiple event correlation
Trigger on user specified events
that happen within specified Event Correlation
time window Capabilities
Support for Boolean functions
Event set triggers action or
another unique event
Benefit
More flexibility, provides unique
troubleshooting capability
Goes beyond today’s one event per policy
BRKDEV-1191
EEM v2.4 RPC Event Detector

Define your own
Allows and outside entity to make SOA Services
an XML RPC request TO the
router or switch and INVOKE a Web Application
defined EEM policy or script services Server
Uses SOAP message format over

SSHv2 transport
Filters on interface and port for
SOAP over SSH
receipt of xml-rpc message and
parameters
Policy might handle cli, config,
anything, and return formatted
(XML) reply
Opens door to more flexible EEM EEM
solutions and automation driven
from outside the box
Make XML RPC calls to invoke
EEM script from outside the box
BRKDEV-1191

Presentation_ID.scr
Some Configuration Needed to Use
the EEM RPC Event Detector
1 Need to configure for SSH,

if you haven’t already
2
1
2 Need to prepare your EEM EEM
policy re S S
Hv2 3
4 Secu
3 Need to register the EEM External

policy for external invocation Program
Secu 1 2
4 Need to create your external re SS
Hv2
program interface EEM
3
BRKDEV-1191
Create Your External Program Interface

and Send an RPC Request
<?xml version="1.0"?> Request Syntax
Use Perl of some other <SOAP:Envelope
programming language xmlns:SOAP="http://www.cisco.com/e
to send the SOAP RPC em.xsd">
request <SOAP:Body>
<run_emscript>
OpenSSH and XML can <script_name> name of script
be used </script_name>
The request specifies <argc> argc value </argc>
the EEM policy— <arglist>
“script_name” that has <l> argv1 value </l>
<l> argv2 value </l>
been registered using
…
the RPC event detector
<l> argvn value </l>
You can pass </arglist>
arguments, too </run_Eemscript>
</SOAP:Body>
Sample programs on
</SOAP:Envelope>
the way ]]>]]> Don’t forget “end of message”
BRKDEV-1191

Presentation_ID.scr
The Response
The “output string” <?xml version="1.0"?>
Reply Syntax
comes from your ‘on- <SOAP:Envelope
the-router’ EEM policy xmlns:SOAP="http://www.cisco.com/e
em.xsd">
You define it’s contents <SOAP:Body>
and structure <run_Eemscript_response>
Your program can use <return_code> rc
</return_code>
the result for whatever
<output> output string
intent you have
</output>
</run_eemscript_response>
</SOAP:Body>
</SOAP:Envelope>
BRKDEV-1191
EEM v2.4 SNMP Proxy Event Detector

Router or switch can RECEIVE Yikes! Better shut interfaces,
an SNMP trap begin graceful shutdown procedures
EEM event upon trap receipt

Execute (trigger) EEM script to
take local action EEM EEM
Script sees varbind info
Example:
UPS on battery backup
¼ Shut non-critical POE ports
SNMP alert
to conserve power
Only 5 minutes remaining Uninterruptible
¼ Shutdown service modules Power Supply
gracefully
SNMP alert—on battery!
5 Minutes Remaining!
BRKDEV-1191

Presentation_ID.scr
Script Policy Refresh
EEM
Network/Device-based script Repo
management
Create central repositories
and let the devices update
from there EEM EEM
“Pull” model rather than

“push” model EEM EEM
Using the power of EEM, Please give
update periodically, by me any script
command, or as directed updates!
EEM
EEM
EEM
BRKDEV-1191
Cisco IOS
Embedded Event Manager v3.0
Theme: 3.0—EEM for Power Users
This development will enhance the performance, increase feature
integration, add new capabilities, and increase the flexibility so that EEM
can be used in new and exciting ways
Major new customer facing function:
Increased performance—up to 150 events per second*
New Event Detectors
Routing ED,
Flexible NetFlow ED,
IP SLA ED,
Enhanced CLI Command ED
User interface enhancements
SNMP library extensions
Get, Set and Notify for local and remote hosts
Ease of use items
… and More
* Target to Be Verified and May Be Platform Specific

BRKDEV-1191

Presentation_ID.scr
Cisco IOS
Service Diagnostics
BRKDEV-1191
Cisco IOS Service Diagnostics

Designed to help solve diagnostics
issues using an event-based embedded
network management approach
- Focused on “service” domain experience
- Scenario’s seeded by Cisco’s TAC for
OSPF, QOS , BGP & Resource
Diagnostics
Provide infrastructure to facilitate the
deployment of customized diagnostics
functionality
Provide a set of scripts addressing
specific diagnostics scenarios
Scripts available for editing and use by
customers on the Cisco Beyond –
Product Extension Community
http://cisco.com/go/ciscobeyond
BRKDEV-1191

Presentation_ID.scr
Service Diagnostics
Key Features
9 scripts for OSPF, BGP and QoS
3 scripts for monitoring CPU, Memory, Buffer
Ability to configure thresholds
Ability to run scripts proactively and in reaction to network events
A rich set of general reusable libraries
Script management made easy with utility scripts
Scripts are platform independent; can run on all Cisco devices
supporting TCL, EEM
Digitally Signed TCL scripts
Email and Syslog notification of diagnostic results
BRKDEV-1191
Example: OSPF Diagnosis

Problem: OSPF neighbor state goes from FULL to DOWN
Trigger: Syslog message indicating “OSPF neighbor state going from FULL to DOWN” triggers
OSPF diagnostic script
Action: Diagnostic script checks OSPF and other configuration to identify the root cause
Problem could be
a. Mismatch area ids
b. Incorrectly configured passive interfaces
c. Incorrectly configured “network” command
d. Incorrectly configured interfaces (link status is DOWN)
e. Blocking access lists
f.….
A detailed message with the problem root cause is sent out via email or a syslog message
How important it is to have a log of diagnostic activities in
persistent storage?
Email / Syslog
Ethernet 0/0
BRKDEV-1191
X Ethernet 0/1

Presentation_ID.scr
Service Diagnostics Benefits
Cost savings (Reduced MTTR)
Increased network uptime
Automatically identify the most common root causes for the most common
failure scenarios related to BGP, OSPF, QoS
Send automatic alerts on resource monitoring when configured thresholds
are crossed
Automatically collect additional context information that is relevant to
diagnosing a problem, to accelerate problem resolution
Infrastructure to customize and add additional diagnostics
Enhanced programmable platform capabilities of Cisco IOS software
BRKDEV-1191
Cisco Beyond—Product Extension

Community: EEM Scripting Community
Now on Cisco.com http://cisco.com/go/ciscobeyond
/go/ciscobeyond http://forums.cisco.com/eforum/servlet/EEM?page=main
Open source scripts, share,

upload, download, learn by
example
Categories include: Ntwk
mgmt, routing, QoS, High
availability, User interface, etc
Comments, ratings,
community managed forum
RSS feed notification
Script URL
BRKDEV-1191

Presentation_ID.scr
Cisco IOS Embedded
Menu Manager
BRKDEV-1191
Embedded Menu Manager (EMM)

Programmable menu interface built on top of tclsh
Uses an XML-based Menu Definition File (MDF) to generate
menus
Useful to provide a custom menu-driven application for operators
Available in Cisco IOS version 12.4(20)T
Ready-to-use menus available for Cisco IOS Service Diagnostics
scripts on the Cisco Beyond – Product Extension Community
http://cisco.com/go/ciscobeyond
BRKDEV-1191

Presentation_ID.scr
EMM Features
Menu item action can
Launch a optional wizard (one or more question/response pairs)
Run an IOS exec-mode command
Run an IOS config-mode command
Run an embedded TCL script
Launch a sub-menu
Query prompts can be static or dynamic
User input can be
Free form (with optional maximum length)
Discrete choices
Range
BRKDEV-1191
EMM Sample Menu – Resource Diagnostics

======================================================================
Resource Diagnostics
Enter ? for help or ?# for item help

----------------------------------------------------------------------
1. Install Diagnostic Scripts
2. Set Global Variables (email parameters)
3. Deploy CPU Diagnostic Script
4. Deploy Memory Diagnostic Script
5. Deploy Buffer Diagnostic Script
6. Display Diagnostic Policy Configuration
7. Remove Diagnostic Policies
8. Exit
Enter selection [8]:
BRKDEV-1191

Presentation_ID.scr
Embedded Menu Manager Benefits
XML MDF file very flexible and file based
Definitions can be centrally stored on network servers
Menu elements can be made more dynamic with Tcl
Built-in customizable context-sensitive help
Wizard mode
Steps users through menu application
Built-in input validation
Ability to record and play back menu sessions
BRKDEV-1191
Embedded Diagnostics and

Management: Summary and Benefits
Hierarchical Management
Security Performance
Right data, right analysis, right and Capacity
place Planning
Fault and
Faster reaction time Network
Correlation
Local action with notification
Multi-level actions Mgmt
Network Egress
Better (More Accurate) Data networks
Node perspective vs..

inference by symptom Onboard Event
Increased collection rates Analysis and
More scalable with distribution Intelligence
Ingress
networks
For Highly Available, High Performance Computing

BRKDEV-1191

Presentation_ID.scr
Q and A
BRKDEV-1191
Recommended Reading
Available Onsite at the Cisco Company Store

BRKDEV-1191

Presentation_ID.scr
More Recommendations
BRKDEV-1191
Complete Your Online

Session Evaluation
Give us your feedback and you could win Don’t forget to activate
fabulous prizes. Winners announced daily. your Cisco Live virtual
account for access to
Receive 20 Passport points for each session all session material
evaluation you complete. on-demand and return
for our live virtual event
Complete your session evaluation online now in October 2008.
(open a browser through our wireless network Go to the Collaboration
to access our portal) or visit one of the Internet Zone in World of
stations throughout the Convention Center. Solutions or visit
www.cisco-live.com.
BRKDEV-1191

Presentation_ID.scr
BRKDEV-1191

Presentation_ID.scr

Embedded Diagnostics and Management in Cisco IOS: TCL, EEM Scripting and Service Diagnostics

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Embedded Diagnostics and Management in Cisco IOS: TCL, EEM Scripting and Service Diagnostics

Uploaded by

Copyright:

Available Formats

Embedded Diagnostics

and Management in Cisco

© 2006, Cisco Systems, Inc. All rights reserved. 1

What Do We Mean By “Embedded

© 2006, Cisco Systems, Inc. All rights reserved. 2

 Issues with integration, scale,

The Future of Network Management

© 2006, Cisco Systems, Inc. All rights reserved. 3

EEM—What and Why

 OK, so what? Why do I care?

© 2006, Cisco Systems, Inc. All rights reserved. 4

EEM Event Detectors and EEM Policies

 Applet-based policies  Tcl-based policies

© 2006, Cisco Systems, Inc. All rights reserved. 5

 Let’s start with a very simple applet policy

Configure an EEM Applet Policy

 We can make an Applet policy to match the on the syslog

© 2006, Cisco Systems, Inc. All rights reserved. 6

 Event type will

Syslog REGEXP Match “Pattern”

iin-rtr1(config-applet)#event syslog pattern "%SYS-5-CONFIG_I:"

© 2006, Cisco Systems, Inc. All rights reserved. 7

iin-rtr1(config-applet)#action 1.0 syslog

Complete Applet Policy Definition

© 2006, Cisco Systems, Inc. All rights reserved. 8

 Cisco IOS CLI—Triggers policies based on  Interface Counter—Policies can be

Embedded Event Manager

© 2006, Cisco Systems, Inc. All rights reserved. 9

 Note: environment variable names that start with

Environment Variables (Built-In)

 When a policy is entered…

© 2006, Cisco Systems, Inc. All rights reserved. 10

 Tcl V8.3.4+ script support

ashcroft#ashcroft#sh event man dir user policy

© 2006, Cisco Systems, Inc. All rights reserved. 11

ashcroft#sh event manager policy registered

© 2006, Cisco Systems, Inc. All rights reserved. 12

 Cisco scripts run in full Tcl mode

 User scripting can be completely disabled by omitting

Security of the Embedded

 All config commands are privileged global config mode

© 2006, Cisco Systems, Inc. All rights reserved. 13

EEM Tcl Policy Structure

© 2006, Cisco Systems, Inc. All rights reserved. 14

RegExp Tcl Command

© 2006, Cisco Systems, Inc. All rights reserved. 15

Automatic Write Running Config

Dealing with interaction:

© 2006, Cisco Systems, Inc. All rights reserved. 16

event manager applet wrmem

Using Environment Variables

© 2006, Cisco Systems, Inc. All rights reserved. 17

Cli Configuration Command Procedure

© 2006, Cisco Systems, Inc. All rights reserved. 18

# Globals are saved in context

File I/O Example

© 2006, Cisco Systems, Inc. All rights reserved. 19

# Register for an application event for sub_system ID 798 and type 10

# Register for a timer event thattriggers once every 60 secs

EEM Feature/Product Support Matrix

Legend Shipping In EFT EC Planning N/A

CISCO 5000 SERIES & UP

Issues with integration, scale,

OK, so what? Why do I care?

Applet-based policies Tcl-based policies

Let’s start with a very simple applet policy

We can make an Applet policy to match the on the syslog

Event type will

Cisco IOS CLI—Triggers policies based on Interface Counter—Policies can be

Note: environment variable names that start with

When a policy is entered…

Tcl V8.3.4+ script support

Cisco scripts run in full Tcl mode

User scripting can be completely disabled by omitting

All config commands are privileged global config mode

Two new EEM Event Detectors

Introduce three new capabilities:

Introduce three new CLI commands:

Uses SOAP message format over

1 Need to configure for SSH,

3 Need to register the EEM External

EEM event upon trap receipt

“Pull” model rather than

Open source scripts, share,