3/15/2017 TipsonReadingCCMTraces| DanielPagan| Pulse| LinkedIn

Tips on Reading CCM Traces

Published on February 16, 2015

Daniel Pagan Follow 26 3 1

CCIE #25689

Note:Thefollowingisaheavilyeditedexcerptfroma40pagetechnical
documentIputtogetherdetailingmanyaspectsofCCMtraces.I've
omittedquiteanumberoftechnicaldetailsfromthisspecificsection
suchasSDLprocessdescriptions,signals,andcommonSDI/SDLfile
searchterms.Thepurposeistoprovideyouwithafewtipsonbetter
readingCCMtraces,andisbasicallytheconcludingsectionfromthe
fulltechnicaldocument.

CCM Traces || Tips & Shortcuts

Developingandpracticingyourowntracefilehabitsandshortcutsareessentialfor
increasingefficiencyandfamiliaritywithdetailedCCMSDI/SDLtraces.Likeanything
else,nothingcansubstituteknowledgeandexperience,butImhopingthatprovidingyou
withafewshortcutsandtipswillhelpyougetstarted.
Theinformationbelowwillnotalwaysbeapplicabletoyourinvestigationandarevery
dependentonmultiplevariablesandconditionsinthecallflowinquestion.

UseWinGrep

UseWinGreptosearchthetoplevelfolderofyouruncompressedtracefiles.WinGrep
willdisplayallfilesthatcontainamatchalongwiththenumberofmatchedresults.

https://www.linkedin.com/pulse/tipsreadingccmtracesdanielpagan 1/5
3/15/2017 TipsonReadingCCMTraces| DanielPagan| Pulse| LinkedIn

Clickingonthefilewilldisplaythespecificlinecontainingamatchinthepreviewpane.
UsingWinGrepisessentialwhenworkinganissuewheremultiplenodeshandledvarious
componentsofacallflow.

Simplecallflowsinaoneortwonodeenvironmenttypicallyinvolvetracesfromasingle
server.However,whenreviewingcomplexcallflows,youmightberequiredtoreview
CCMtracesacrossfour,five,orormoreCUCMnodes.Inthissituation,rememberthat
CCMregistrationiscloselytiedtowhereaspecificprocessinstanceresides.Letsusethe
followingsituationasanexample:

1.WedialdigitsandmatchaRoutePatternfromanIPPhone.Ourphoneisregisteredto
Node1.

2.TheRoutePatternisassociatedtoaRouteList.OurRouteListisregisteredtoNode
2.

3.PointofegressisaSIPTrunk.ThetrunkisntrunningonallnodestheCCMgroup
onlycontainsNode3.

Inthisscenario,weneedtoreviewCCMtracesacrossNode1,Node2,andNode3since
theprocessinstanceshandlingthevarioussectionsofthecallresideondifferentservers.

StationD,StationCdpc,and(mostlikely)LineControlwillbeonNode1

RouteListControlandRouteListCdpcwillbeonNode2.

SIPDandSIPCdpcwillberunningonNode3.

Theeventswillremainthesameaswediscussedinprevioussectionofthisdocument
butwillrequireinternodeSDLcommunication.

Withpractice,investigatingmanyifnotmostcallflowscanbeginatthedigitanalysis
level.OnceyoufindtheassociatedDAprocessforyourcall,gatheringtheinitialCI,TCP
Handle,and/orprotocolcallreferencenumbersissimplyamatterofreverseengineering.
Thisofcoursewillnotapplytoalltroubleshootingtaskswheretracefilereviewis
required.

Gettoknowsignalingprotocols.

Insituationswhereyouremissingkeyinformationaboutthecallflow,understanding
signalingprotocolswillallowyoutotakeaneducatedguess,reverseengineerandlocate
earliersegmentsofthecallflow,andpiecetogetheryourdatasimplybyknowingthe
protocolrequirements,theirstandardsofoperation,andkeyevents.ThisappliestoSIP,
MGCP,H.323,SCCP,andevenCTI.

We'lluseSIPandSCCPasanexampleWe'rereviewingasetofCCMtracesforan
outboundSIPcallbutdon'thaveinformationonourcallingdevice.Letssaywehaveno
TCPHandlefortheIPPhoneinvolvedinthecallbutneedtoreviewcallingdevice
signaling.

https://www.linkedin.com/pulse/tipsreadingccmtracesdanielpagan 2/5
3/15/2017 TipsonReadingCCMTraces| DanielPagan| Pulse| LinkedIn

Usingourknowledgeofthecallednumber,welocateanoutgoingSIPINVITE
requestusingdelayedofferandeventuallythe200finalresponse.The200containsan
SDPofferw/mediainformationaboutthegateway.

ThisSDPoffercontainsaUDPportnumberforRTPmediaonthemedialine(m=).

WealreadyknowtheStartMediaTransmissionmessageisusedtoinstructtheSCCP
devicewhereitmuststreamRTPpacketsandincludesmediainformationofthe
remoteentity.

WiththeUDPportnumberprovidedintheSDPinourSIP200responseinhand,we
canuseNotepad++tosearchfortheStartMediaTransmissioneventandlocateour
TCPHandleforthecallingSCCPdevice.WecanuseWinGrepiftheSCCPdeviceis
registeredtoanothernode.

FindingtheStartMediaTransmissionrequestcontainingthemediaUDPportnumberwill
provideuswiththeTCPHandleandMACaddressoftheSCCPdeviceinvolvedinthe
call.Fromthispoint,wecantraceallSCCPeventsfromourcallingdevice,allSIP
transactionsforourSIPtrunk,andtheinternalSDI/SDLsignalsbetweenthetwo.

Obviouslythiswon'talwaysapply.Oneperfectexamplewherethiswon'tapplywouldbe
inthecaseofusinganMTPresource.Inourscenarioabove,theSDPofferinformation
won'tbeextendedtothecallingdevice(regardlessofitssignalingprotocol)butwillbe
usedtosendmediainformationtotheMTPresourcethroughthe
MediaTerminationPointControlprocess.TheMTPresourcemediainformationwouldbe
senttothecallingdevice.

FamiliarizeyourselfwithNotepad++features:

StyleTokensEmphasizeandmarkspecificwordsorstringswithstyletokensby
highlightingtext,rightclick,andselectingastyletoken.Allmatchingstrings
throughoutthefilewillautomaticallybyhighlightedusingthecolorassociatedw/the
styletokenused.UsethisontextstringssuchasprocessIDs,TCPHandles,protocol
specificcallreferencenumbers,CIs,andanythingelseyoufeelareimportantpiecesof
information.Usethissparinglytoavoidconfusion.

CloningViewsMaybeyouneedtocompareaspecificSIPrequestbetweenanon
workingandaworkingcall,orperhapsyouneedtocompareccapidebugsfrom
CUBE.Viewandcomparefilessidebysideandwithinthesameinstanceof
Notepad++.AccessthisthroughView>Move/CloneCurrentDocument.

DisableWordWrapReadingthroughtracefilesiseasiestwhenwordwrapis
disabled.Eachlinewithinthetracewillbeanewentry.Whenitcomestotracefiles,
wordwrapisyourenemy.

BookmarkUsingbookmarksinNotepad++allowsyoutoquicklymovebetween
specificlineswithinatextfilesimplybypressingF2.Bookmarksappearasasmall
bluedottotheleftofeachmarkedlineoftext.Usebookmarksifyoure
troubleshootingaspecificaspectofacallleg.Forexample,ifyouknowanissue

https://www.linkedin.com/pulse/tipsreadingccmtracesdanielpagan 3/5
3/15/2017 TipsonReadingCCMTraces| DanielPagan| Pulse| LinkedIn

existswithaspecificSIPdialogandyouwishtoquicklybrowsethrougheachSIP
transaction,usestyletokenstohighlightyourCallIDandbookmarkthestartingline
ofeachSIPrequestandresponse.PressingF2willallowyoutoquicklyscrolltoeach
bookmarkedmessagewhichcancertainlycomeinhandyafterhavingopenedand
reviewedsignalingmessagesandprocesseventsspanningmultipleSDI/SDLfiles.

Pen&Paper

Finally,whenyoufirststartreviewingtraces,makesuretohaveapenorpencilandafew
piecesofpaperfordocumentingtimestampsandevents.Inyourownwords,describethe
eventthatoccurredalongwiththeexacttimestamp.Documentingyourfindingsonpaper
willhelpyoukeeptrackofthecallflowbeingreviewed.Youllfindthathavingnotes
willprovetobeinvaluableafterspendingminutestosometimesevenhourslookingat
tracefiles,especiallywhenyouneedtoprovideafullexplanationofwhatoccurredtoa
customer.

ProcessIDs

SDLprocessesarewritteninaformatthatprovidesyouwithalotofinformation.

NodeIDrepresentsthenodeonwhichtheprocessresides(1=Pub,2=1stsubtojoin
cluster,etc.)

AppIDrepresentstheapplicationunderwhichtheprocessresides(100=CCM,
200=CTIMgr,etc.)

ProcessIDrepresentstheprocessitself.

ProcessInstancerepresentsauniquevalueassignedtoaprocess.

ThecombinationofallfourIDfieldsrepresentaspecificprocessonaspecificnodeand
canalsoprovideyouwithsomeinformationonthingslikecallvolumeinacluster,
registrationstateandlocationofvariousentitiesinCUCM,etc.

DanielPagan[CCIE#25689]

Tagged in: signaling protocols,cisco call manager Report this

Daniel Pagan
CCIE #25689 Follow
3 articles

3 comments Newest

Leave your thoughts here

Moshfiqur Rahman 6mo

https://www.linkedin.com/pulse/tipsreadingccmtracesdanielpagan 4/5
3/15/2017 TipsonReadingCCMTraces| DanielPagan| Pulse| LinkedIn
UC Solution Architect, CCIE # 51697 (Collaboration)
Wonderful writing Daniel! Is there any reference book available in the market on trace level
troubleshooting? I have one from Cisco Press (Troubleshooting Cisco IP telephony) which is very old
(2002). I wish I had recent one. I will appreciate if you please recommend any study materials.
Like Reply

NARASA SARAT MEDAPALLI 1y

Sr.Asscociate at Cognizant Technology Solutions
This is great Daniel, I would strongly recommend TranslatorX for getting important details like calling
,called CI and details of call path which makes it easy to search and highlight the related logs in NP++ for
looking the rest of the call logs.
Like Reply 1 1

Daniel Pagan 1y
CCIE #25689
Thanks for the comment. I agree - trace parsers like TranslatorX are great for a quick view and
getting important, high-level details like you mentioned. Personally I prefer jumping into CCM SDL
traces directly as I find it faster in combination with WinGrep and NP++, but I definitely see the
value TranslatorX provides. Some of my coworkers use it while others go into traces directly as I
do.

Personal preference I guess :)

I guess it would also depend on the scenario since TranslatorX can provide a quick plain-text
description on ISDN progress codes (which can certainly helpful) and converts hexadecimal to IP
address last I recall. At the same time, it doesn't touch other service traces like LBM or CTI. Back
to its benefits, it can help simplify data collection when reviewing CCM SDL traces for a complex
call flow where related processes are being handled by multiple nodes.

Thanks again, Narasa!

Like Reply 1

There is 1 other comment.Show more.

Top stories from Editors Picks

The Mission Continues: Joining the Beating Nature at Its Own Game Maintaining the Positive Momentum of
Microso Board Bill Gates on LinkedIn the Global Economy
Reid Homan on LinkedIn Christine Lagarde on LinkedIn

Looking for more of the latest headlines on LinkedIn?

Discover more stories

HelpCenter About Careers Advertising TalentSolutions SalesSolutions SmallBusiness Mobile Language UpgradeYourAccount
LinkedInCorporation2017 UserAgreement PrivacyPolicy AdChoices CommunityGuidelines CookiePolicy CopyrightPolicy SendFeedback

https://www.linkedin.com/pulse/tipsreadingccmtracesdanielpagan 5/5