ICONIX PROCESS ROADMAPS

DOUG ROSENBERG

FINGERPRESSLTD
LONDON

Findmoregreatbooksat:
www.fingerpress.co.uk



This PDF contains Appendix A of ICONIX Process Roadmaps by Doug Rosenberg.

Find out more at:
http://software.fingerpress.co.uk/iconixprocessroadmaps.html

Or buy the book direct:
www.iconix.org/product.sc?productId=92&categoryId=18




























CopyrightDougRosenberg2011.WeencourageyoutoforwardthisPDFaroundandredistributeitfor
noncommercialuse.WejustaskthatyoudontmodifythePDF;andifyoulikeit,tobuyacopyofthe
book!


APPENDIX A
Bonus Roadmap: Design Driven Testing for Systems
ByDougRosenbergmanythankstoMichaelSieversattheJetPropulsionLaboratory
(Pasadena,CA)forhisvaluableinputs.

DesignDrivenTesting(DDT)wasfirstoutlinedinthebookUseCaseDrivenObjectModelingwith
UML:TheoryandPractice(byDougRosenbergandMattStephens),andthendescribedindetail
inDesignDrivenTesting:TestSmarter,NotHarderbythesameauthors.AlthoughDDTcanbe
successfullyusedinconjunctionwithTestDrivenDevelopment(TDD),itreallyfollowsthe
oppositedevelopmentphilosophy:startingwithadesign(andrequirementsandusecases)and
drivingtestsfromthem.
DDTisahighlymethodicalapproachtotesting,allowingyoutoknowwhenyouvefinished
i.e.whenyouvewrittenenoughteststocoverthedesignandtherequirements.Ithelpsyouto
zoominandwritealgorithmicteststocoverintensiveormissioncriticalsectionsofcode,and
toknowwhenitssafetozoomoutandwritefewertestsforboilerplateorlesscriticalcode.
ConsequentlyyoutendtoendupwithfewertestsoverallthanwithTDDbutcomparatively
moretestscenariospertestcase.
InthisAppendix,DougextendstheconceptofDDTforhardware/softwaresystems,allowing
SysMLbaseddesignstobetestedinahighlyrigorous,systematicway.ItsstillDesignDriven
Testing,butnowthedesignelementsthatneedtobetestedincludeallofthefourpillarsof
SysML,whereasDDTforsoftwarefocusesontestingbehavior.
Doug(notbeinganactualrocketscientist)wouldliketoacknowledgesomeinformalbutvery
valuableandentertainingdiscussionswithDr.MichaelSieversofNASAsJetPropulsion
Laboratory(JPL)regardinghowrealrocketscientistswouldapproachtheproblemofdetecting
potholesfromouterspaceandhowtheywouldtesttheirdesigns.



APPENDIXA

Introduction
ThisAppendixdiscussesanintegrated,modelbaseddesignapproachbasedonanextensionof
DesignDrivenTesting(DDT)54thatiscompatiblewithValidation&Verification(V&V)activitiesat
alldesignlevels.(WedefineV&VinthesidebaronPage216).Modelsarevalidatedateachstep
andthevalidatedmodelsarethenusedforverificationtests.Essentiallythedesignisthemodel
andthemodelisthedesign.Atsomelevelofdetailthemodelisphysicalhardwareand
executablesoftwareandtheV&Vprocessusesfamiliarunit,bench,andintegrationtests.
WhileDDTisrelatedtosoftwaresimulationscurrentlyusedintheindustry,itdiffersinat
leastonekeyaspect:typicallysoftwaresimulationsarebuiltbyindependentteamsfrom
independentrequirements.Thereisalwaysaquestionofhowclosethesoftwaresimulationis
totherealsystem.Inourapproach,thereisonesystemmodelwhichmaybeviewedindifferent
waysandatdifferentlevelsofdetailassuitsausersneeds.Moreover,aswillbeseeninthe
DesignDrivenTestingsection,ourapproachguidessmartselectionoftestcaseswhichsaves
costandschedule.Thisisparticularlyimportantbecausewhilesystemshavebecomemore
complex,thetimeallocatedfortestinghasnotchanged.Wemustnowtestmore,butinthe
sameamountoftimeassimplersystems.Thiscanonlybecomepracticalbyimprovingtest
efficiency.
DDTisintimatelylinkedtomodern,modelbasedsystemsengineeringusingtheSysML
paradigm.LikeSysMLwhichestablishesasingle,commonsourcefordesigntruth,DDTprovides
asingle,consistentsourceofV&Vtruthandestablishesclearlinesofresponsibility.Equally
importantisthatDDTisintegratedwiththeiterativedesignprocessassuringthateachlevelof
designisconsistentwithitsparentandchildren.

54
DesignDrivenTesting:TestSmarter,NotHarderbyMattStephensandDougRosenberg(Apress,2010).
210
 ROADMAP:DDT/SYSTEMS

ROCKET SCIENCE:
Why Use a Space Mission to Illustrate DDT/Systems?
A space mission is a system of interacting systems that implement a set of behaviors defined by
operational concepts. Operations concepts describe all phases of the mission from ground-
based testing through pre-launch, launch, separation, commissioning, nominal operations and
eventually disposal or decommissioning. Each of these use cases comprises complex actions,
alternate paths and exceptional conditions. When a typical mission costs taxpayers hundreds of
millions to billions of dollars, it is crucial that these systems are validated and verified from the
highest conceptual levels down to the detailed code that defines the functions in an FPGA. The
cost of finding and fixing problems escalates by orders of magnitude as the mission progresses
through its design and deployment stages hence, systematic and early Validation and Verification
(V&V) are central aspects of all space programs.

A Model System Design

UsingSysML,wedescribethedesignofafanciful,yetsomewhatrealisticspacecraftmissionfor
arunningexampleinthispaper.ThenextsectiondiscussesthesystemV&Vprocessasan
extensiontoDDT.WethenillustratethesystemV&Vmethodappliedtoourspacecraftdesign.
Ourmissiongoalisasystemthatimagesthecitystreetsitpassesoverwithasufficiently
largetelescopethatitcanresolvelargestreetimperfections.Wecallourspacecraft:Persistent
OpenTrafficHazardObstacles,LargeandElongated(POTHOLE)becauseallspacecraftmust
haveaclevername.POTHOLEsendsimagestothegroundwheretheyareprocessedandnewor
growingpotholescanbereportedtolocalmaintenancebureaus.
FigureA1showsaconceptualdiagramofPOTHOLEspacecraft.Majorvisiblecomponents
arethesolararrayingreen,telescopeindarkblue,sunshadeinlightblue,downlinkantennain
silverandtheBusinyellow.TheBusconsistsofthefunctionsneededtomaintainthevehicle
andcommunicatewiththeground.Thetelescopeandrelatedimagingfunctionsarecollectively
calledtheInstrument.ThemissionblockdiagraminFigure3showsthatthePOTHOLE
spacecraftisacompositionoftheBusandInstrumentsystems.Themissionalsorequiresa
GroundStationforcontrollingthespacecraftandacceptingandprocessingimages.

211
APPENDIXA

FigureA1.POTHOLESpacecraft

bdd [SysML Block Definition] Block Diagrams [POTHOLE Context]

Context
POTHOLE Mission

System of Systems
POTHOLE Spacecraft System of Systems
Ground Station

System System
Bus Instrument

FigureA2.POTHOLEMissionBlockDiagram

Wenextdefineafewkeybusinessrequirementsconstitutingthecontractbetweenthe
customerandtheimplementer.Businessrulesdefinetheconstraintsandfunctionsthatmustbe
deliveredbytheimplementerandarethebasisforthecustomersacceptancecriteria.
212
 ROADMAP:DDT/SYSTEMS

ROCKET SCIENCE: V&V Challenges

Several challenges are inherent in current V&V methodologies.
There are multiple layers of requirements (refer to Figure A-3) mission, systems,
subsystems, assemblies, units and components.
Mission requirements are often driven by capabilities. Instead of asking, What do you
want us to build? we are often asked, What do you have and how can we use it?
A design is V&V'd through tests, analysis, demonstration and inspection.
Keeping track of who does what and how lower level V&V rolls up into higher levels is not
managed very well.
Tests at higher levels wait until tests at lower levels have completed. So problems related
to validating higher level functions arent recognized until later in the development cycle
when it is more expensive and time consuming to fix them.
Models (e.g. aerothermal and reliability models) are often used for various aspects of
V&V. These models are not integrated and do not necessarily provide an easy path for
incorporating effects in one model into another.

213
APPENDIXA

Context
Mission

1..*

Level 2
Proj ect

1..

Level 3 1..* Level 3 1..* Level 3

Payload System Spacecraft Bus Ground Systems

1..* 1..*

Level 4 Level 4 Level 4

Instrument Spacecraft Ground
Subsystems Subsystems Subsystems

. 1..*

Level 5
.
.
Assemblies

.
. 1..*

Level 6 .
Boards

Level 7
Components

FigureA3.TypicalspacesystemdesignhierarchyincludesuptosevenlevelsofspecificationandV&V

214
 ROADMAP:DDT/SYSTEMS

POTHOLE Business Requirements

FigureA4showsthetenrequirements,asmodeledinEnterpriseArchitect.Mission
requirementsderivefromtheopportunitiesandlimitationsinsystemuse,environmental
conditions,functionalcriticality,andtheothersystemsthecustomeremploysinsupportofthe
system.

req Functional Requirements

The spacecraft shall store images for The telescope shall have an
at least one hour adjustable aim point

The spacecraft shall transmit images Each image pixel shall have ground
to the ground in real-time when a resolution of no more than 0.1 meters
ground station is available

The spacecraft shall retransmit stored

images when requested by ground
Images shall be transmitted to the
command
ground at 150 Mbps

Retransmitted images shall be

interleaved with real-time images
according to a ground configurable Images shall be downlinked in groups
downlink schedule of 16 using CCSDS format

The spacecraft shall store images in a

circular buffer over-writing older The field of view shall be at least 1.0
images with newer images Km x 1.0 Km

FigureA4.POTHOLEMissionRequirements

215
APPENDIXA

ROCKET SCIENCE: Validation and Verification (V&V)

We define validation as the processes performed to assure that we are building the right thing
for specified needs. If a user wants to measure magnetic fields around a planet then our design
must include a magnetometer. We should not build our spacecraft with something else unless we
have discussed it with our customers and they agree that our alternative is acceptable. In a space
flight mission, validation focuses on certifying that the system meets a set of mission objectives
documented in operations concepts and mission plans while operating in flight-like conditions.
Validation is often subjective because it asks whether the system is able to accomplish its
mission goals.
Verification is defined as the processes performed that assure we have built the system right.
Verification determines that the system has the right components to do its job and that at each
level of design those components satisfy a set of requirements. Collectively, requirements are a
formal contract between the customers and implementers of a system. Unlike validation,
verification is associated with formal, objective tests that have either a pass or fail result.
Defining comprehensive yet efficient V&V for complex systems is a difficult problem and is often
made more difficult by fuzzy requirements, incomplete definition of desired behaviors, multiple
teams, and unclear responsibilities. Inefficient V&V resulting in unnecessary overlaps or
repeated tests adds costs time and money but is far less dangerous than incomplete or missing
tests. Yet testing all combinations and permutations of a complex system is impossible both for
the size of the problem and for the lack of visibility into internal operations as systems are
integrated.
Typically, a space system is designed top-down and bottom-up. Mission planners think through
the type of data they want to collect which drives lower-level allocations and requirements. At the
same time, designers are often looking at what hardware and software they can reuse even
though they may not yet have a complete understanding of what functions they must provide.
Often bottom-up selection and top-down mission analyses and decompositions meet in a
integration and test facility where disconnects are frequently found.

TheBusandInstrumentstructuresarefurtherdetailedintheblockdefinitiondiagrams
(BDDs)showninFiguresA5andA6respectively.
TherearesixmajorsubsystemsthatformtheBus:Telecomwhichconsistsofradiosfor
groundcommunication.GuidanceandNavigationisresponsibleforknowwherethevehicleis
relativetothegroundandformaintainingpointingsothatthetelescopepointstotheEarths
surfacewhilethesolararraypointstothesun.Avionicshousestheflightcomputersonwhich
flightsoftwareexecutes.ThePropulsionsubsystemincludesthethrusters,valves,and
propulsiontanks.TemperaturesaremaintainedbytheThermalsubsystemandthePower
subsystemmanageselectricalpower.
216
 ROADMAP:DDT/SYSTEMS

Lunchtime Conversation with a Rocket Scientist

We discussed our fictitious mission requirements with a friendly rocket scientist over lunch. The
conversation went something like this:
Well, the first requirement tells us how long we have to keep images onboard. Once we know the
image capture rate and image size, we can place a lower bound on how much storage we need.
The seventh requirement tells us that our field of view (FOV) is 1.0 Km x 1.0 Km. Assuming we
want to take an image in every 1.0 Km x 1.0 Km patch, then we have to take an image every time
the ground track of the spacecraft moves 1.0 Km which can be computed from the spacecraft
orbit (inclination and altitude).
Pizza arrives
For example, at an altitude of 20,000 Km (GPS altitude), we need to image roughly every 100
milliseconds at the equator. Requirement 9 tells us that our image sensor is 10K pixels * 10K
pixels = 100M pixels. At 12 bits per pixel, each image is 1.2 Gb. Storing images for an hour
translates into around 4.3Tb of onboard storage a large number, but still within the realm of
current solid state recorder technology.
Time for a second slice
In a real design, we would be quite concerned about the practicalities of building a telescope and
optics with the required precision. However, for our purposes, we are only interested in data
collection, transport, and processing. Our back-of-the-envelope analysis gives us interface rates,
storage requirements, image sensor size, and hints of needed processing throughput. Were now
ready to define our domain model and use cases.

TheInstrumentcomprisesDataStorageforholdingimageswhengroundstationsarenotin
viewandforretransmission,aCamerathattakesimages,anInstrumentComputerthat
managestheinstrumentandimagecollectionfunctions,andtheTelescopewhichconsistsof
mirrorsandoptics.
FigureA7isanIBDthatshowsinterfacesbetweentheCDHprocessor,LocalEngineeringUnit
(LEU)usedforanalogdatasampling,NonVolatileMemory(NVM)usedtostorecriticalvariables
andsoftwareimages,andtheCommunicationsInterface(CommInterface)thatsendsand
receivesmessagesfromthetelecomradios.WeusethehighlightedNVMtodefineinterface
testsinthevalidationsectionbelow.

217
APPENDIXA

System
Bus

Subsystem Subsystem
Subsystem Subsystem Subsystem Subsystem
Telecom Av ionics
Guidance and Nav Prop Thermal
Pow er

block block
CDH FSW

block
block TelecomInterface
Processor block
EngineeringDataCollection
block
Non-VolatileMemory

FigureA5.POTHOLESpacecraftBusBDD

System
Instrument

System Subsystem Subsystem Subsystem

Instrument DataStore Camera Telescope
Computer

FigureA6.POTHOLEInstrumentBDD

218
 ROADMAP:DDT/SYSTEMS

: CDH

Board Board
Processor LEU
flowPort
flowPort CtlIn
EngrCtl AnalogIn AnalogIn

flowPort EngrIn DigitalIn DigitalIn

flowPort EngrOut

Board
flowPort
FIFOCtlOut NVM
flowPort
flowPort flowPort
FIFOCtlIn
InstrDataIn InstrDataIn

flowPort flowPort
flowPort
FIFOStatusIn FIFOStatusOut
XbandDL flowPort
XbandDL

flowPort EngrOut
Board
CommInterface
flowPort EngrDL flowPort flowPort
SbandDL SbandTlm
flowPort CmdIn

flowPort CmdUL
flowPort flowPort
SbandUL SbandCmd

FigureA7.ExamplesofCDHinterfaces

EarlierwemadeacasefortopdownandintegratedV&V,yethereweareshowing
significantdetailforourhypotheticalspacecraft.Naturalquestionsmightbe:howdidwecome
tothispartitioning,howdoweknowitisright,andcouldtherebeabetterdecompositionthat
doeswhatweneed?Thesimpleansweristhatthesearebasicfunctionsofa3axisstabilized
spacecraftthatcancaptureimagesandsendthemtotheground.Withinthisstructure
designersarefreetoallocatespecificbehaviorsandrequirements.Moreover,wearefreeto
modifythestructureifourV&Vprocessdeterminesthatitisnotadequate.
WhydistinguishbetweenBus,andInstrument,couldntthesebejustpartofaFlight
Vehicleandavoidtheintermediatepartition?TheBusandInstrumentpartitionsareneither
requirednornecessarybutreflecttherealitythatdifferentdevelopmentgroupsareusually
responsibleforprovidingBusandInstrumentfunctions.Thesegroupsindependentlydesignand
testtheirportionofthespacecraftandtheneventuallythosesystemsarebroughttogetherand
tested.ThishighlightsthepointmadeearlierthattraditionalV&Vwaitsforverificationoflower
levelfunctionsbeforetestinghigherlevelfunctions.Electricalandfunctionalinterface
documentsdefinehowthisshouldhappenandwheneverythingisproperlyspecifiedand
implementedthenallworksout.But,thatsnotalwaysthecase.
219
APPENDIXA

POTHOLE Domain and Use Case Models

WehaveagoodideaofwhatbehaviorsareneededinthePOTHOLEspacecraftforcollectingand
transmittingimagestothegroundforpotholeanalysis.Thosebehaviorsalsohelpdefineour
domainmodel.Domainmodelsandusecasesaretightlycoupledbecauseusecasesreferto
domainelementsanddomainelementsmustbepresentinatleastoneusecase.Consequently,
domainandusecasemodelinteractionsarethebasisofinitialmodelvalidationbecauseeach
modelcheckstheother.Neithermodelcanbeconsideredcompleteorcorrectunlesseachis
consistentwithandsupportiveoftheother.
Fromtherequirementsandmissiongoal,weknowthatourdomainmodelmusthavean
imageelement.Wealsoknowthatindividualimagesarepackagedonthespacecraftpriorto
downlink,soourdomainneedsaconceptofimagecollectionwhichiscomposedofimages.
Similarly,weknowthatwewillneedsomesortofimagequeueinourdomainmodelthat
feedsimagestoanalysissoftwarebecauseimagescannotbeprocessedconcurrently.
ThedomainmodelinFigureA8andthecorrespondingusecasemodelinFigureA9result
fromiteratingdomainelementsandusecasedescriptions.FigureA8populatesmodelelements
withconceptualfunctiondefinitions.Thesemayneedrevisionasmoredetailisknown,butat
thispoint,theelementsandfunctionsareconsistentwiththeusecasesinFigureA9andthe
missionrequirements.
ThedomainmodelincludestheInstrumentthatproducesanImageCollectionwhichis
createdasanaggregationofindividualImages.TheImageCollectionhasanassociationwiththe
GroundStationthroughtheTelecomsubsystem.TheGroundStationmaintainsanImageQueue
foranalyzingandannotatingtheImages.ImagesannotatedwithPotholesaresenttothe
Subscriberbythegroundstation.

220
 ROADMAP:DDT/SYSTEMS

class Domain Model

produces
Telecom Image Collection Instrument

sends images to

Ground Station Image Queue Image

sends annotated image to

analyzes

produces
Image Analyzer Annotated Image
Subscriber

detects

Pothole

FigureA8.POTHOLEImagingDomainDiagram

ThemissionlevelusecasesforourreferencefunctionshowninFigureA9complementthe
domainmodelandreflectourunderstandingoftheimagingcollectionandanalysisprocess.The
spacecraftmustachieveandmaintainthecorrectorbit(MaintainOrbit).Theinstrumentmust
beaimedatapointonthegroundpointpriortolookingforpotholes(AimInstrument).
Spacecrafthealthmustbemaintainedthroughoutthemission(MaintainSpacecraftHealth)
includingdealingwithspacecraftemergencies(enteringsafemodewhichkeepsthespacecraft
powerpositiveandincommunicationwiththeground).Whencommanded,wecaptureand
processimages(LookforPotholes)andifimagesarecorruptedintransmission,thegroundmay
requestthattheyberetransmitted(RetransmitImages).

221
APPENDIXA

Aim Instrument

precedes

Maintain Orbit Retransmit Images

precedes

precedes invokes
Ground Station

notify
Look for Potholes

Subscriber

invokes

Maintain Spacecraft
Ongoing Health

FigureA9.POTHOLEMissionLevelUseCases

Becauseourprimarymissionispotholedetection,wefocusontheLookforPotholes
scenariothatcollectsimageswithsufficientresolutiontodetectlargepotholesandsendsthe
imagestothegroundforprocessing.Moreprecisely:

Basic Course:

Ground Station sends a command to Spacecraft to collect Images.

Instrument collects Images, resulting in an Image Collection
Telecom sends the Image Collection to the Ground Station
Ground Station adds Images to Image Queue
Image Analyzer gets Image from Image Queue
Image Analyzer detects Potholes and produces Annotated Image showing Pothole
locations
Image Analyzer sends Annotated Image to Subscriber

Alternate Courses:

Ground Station sends a command to Spacecraft to retransmit Image Collection:

Invoke Retransmit Images
Spacecraft enters safemode: invoke Maintain Spacecraft Health
Ground Station sends a command to Spacecraft to stop collecting images: invoke
Maintain Spacecraft Health

222
 ROADMAP:DDT/SYSTEMS

AfewthingsareworthpointingoutinthisICONIXprocessstyleusecase55.Thescenariois
writtenpreciselyusingnouns(showninbold)thatrepresentobjectsinthedomainorcontext
models.Verbsapplyactionsthattransformonenounintoanothernounortransportanounto
anotheraction.Bolditalicizedtextreferstousecasesintheusecasediagram.
Comparingthenounsandverbsinourusecasewiththedomainmodelelementsgivesus
confidencethatourmodelsareconsistent.Butwedontyetknowthatwedefinedtheusecase
scenariocorrectlyandsowecantyetclaimthatwehavevalidatedourinitialconcepts.Ournext
stepcreatesaconceptualdesignintheformofarobustnessdiagram56thatfacilitatesadesign
walkthroughforvalidatingtheusecases.Wewillseelaterthattherobustnessdiagramisalso
anessentialtoolfordefiningtestcases.

POTHOLE Conceptual Design

FigureA10showsarobustnessdiagramfortheLookforPotholesusecase.Inarobustness
diagram,thelogicalandconceptualstructuresareanalyzedensuringthattheusecasesit
modelsaresufficientlyrobustfortheintendedusagebehaviorsandrequirements.Robustness
diagramsareannotatedwithrequirementsestablishingcompletenessandconsistencydesign.
Missing,incorrect,andincompleterequirementsbecomereadilyapparent.
Arobustnessdiagramcomprises:actors(peopleorexternalsystemsthatinteractwiththe
systemunderconsideration,boundaries(interfacesthatactorsinteractwith),controllers(the
verbsintheusecasestatements),andentities(elementsfromthedomainorcontextmodels).
Afewsimpleinteractionrulesdefinetheconstructionofrobustnessdiagrams:

Entitiesandcontrollersmayinteract.
Actorsandboundariesmayinteract.
Boundariesandcontrollersmayinteract.
Entitiescannotdirectlyinteractwithotherentities.

InFigureA10,circleswitharrowsrepresentcontrollers,underlinedcirclesareentitiesand
circlesattachedtoverticallinesareboundaries.Requirementsareassociatedwithcontrollers.
EverycontrollermusthaveatleastonerequirementsoFigureA10isnotcomplete.Moreover,

55
UseCaseDrivenObjectModelingwithUML:TheoryandPracticebyDougRosenbergandMattStephens.
56
IvarJacobson,et.al.(1992),ObjectOrientedSoftwareEngineeringAUseCaseDrivenApproach,AddisonWesley.
223
APPENDIXA

everyrequirementmustbeassociatedwithacontrollerandmissingassociationsmeaneither
omissionsintherobustnessdiagramorunnecessaryrequirements.
WecanwalkthroughtheLookforPotholesstepsandfollowtheflowintherobustness
diagram.Inconsistenciesarereconciledbyalteringrobustnesselementstoreflecttheusecases
itrepresentsand/orchangingtheusecasestomirrortherobustnessdiagram.Sinceupdatesto
usecasesmayalsocausedomainmodelrevisions,wheniterationsarecomplete,the
robustness,usecase,anddomainmodelsareselfconsistent.

The telescope shall have an Each image pixel shall have ground
adjustable aim point resolution of no more than 0.1 meters
Basic Course:

Ground Station sends a command to Spaceraft to
collect images.
Instrument collects Images, resulting in an Image
Collection
Telecom sends the Image Collection to the Ground
Station
Ground Station adds images to Image Queue
Image Analyzer gets Image from Image Queue
Image Analyzer detects Potholes and produces
Annotated Image showing pothole locations
Image Analyzer sends Annotated Image to
Subscriber
Alternate Courses:
(from Functional Requirements) (from Functional Requirements)
The spacecraft shall store images in a
circular buffer over-writing older
images with newer images
(from Functional Requirements)
Instrument
The spacecraft shall retransmit stored
images when requested by ground
command
capture images
(from Functional Requirements)

1. Ground Station sends a command to Spacecraft

to retransmit images: Invoke Retransmit Images Image Collection
2. Spacecraft enters safemode: invoke Maintain Retransmit Images
Spacecraft Health
3. Ground Station sends a command to Spacecraft
to stop collecting images: invoke Maintain
Spacecraft Health Pothole

send images

Telecom detect pothole

Subscriber

Ground Station send annotated image

Image Analyzer

annotate image
Annotated Image
add image get image

Image Queue

Image

FigureA10.POTHOLERobustnessDiagramwithafewexamplerequirements

FigureA11showsaportionofthesequencediagramassociatedwiththeLookforPotholes
usecase.Behaviorsareallocatedtothesystemelementsdefinedinthedomainmodeland
blockdiagrams.Eachbehaviormustbeunittested,asshowninFigureA25.
TheExtendedDomainModelinFigureA12showsanobjectorientedallocationofbehavior
tosystemelements.Allocationssatisfyallbehaviorsspecifiedintheusecaseasshowninthe
sequencediagram.

224
 ROADMAP:DDT/SYSTEMS

Ground Station Telecom Instrument Image Queue Image Analyzer Subscriber

CAPTURE_IMAGES_CMD()
Basic Course: CaptureImages()
loop
Ground Station sends a [16 images per collection]
command to Spaceraft to
collect images.
Instrument collects Images,
resulting in an Image
Image Collection
Collection
Telecom sends the Image
Collection to the Ground
Station SaveImageInOnboardStorage()
Ground Station adds images
to Image Queue
Image Analyzer gets Image GetImages()
from Image Queue
State Machines :
Image Analyzer detects TRANSMIT_IMAGES() Capture Images
Potholes and produces
Annotated Image showing
pothole locations
RECEIVE_IMAGES()
Image Analyzer sends
Annotated Image to AddImage()
Subscriber

Alternate Courses: loop all images in queue

1. Ground Station sends a GetImage()

command to Spacecraft
to retransmit images:
Invoke Retransmit
Images
2. Spacecraft enters Image Create()
safemode: invoke
Maintain Spacecraft DetectPothole()
Health Annotated Image
3. Ground Station sends a
command to Spacecraft
to stop collecting images:
invoke Maintain
Spacecraft Health
Pothole
CreateBoundaryOutline()

AnnotateImageWithPothole()

SendAnnotatedImage()
ReceiveAnnotatedImage()

FigureA11.Sequencediagramshowswhatthesystemmustdoandwhichelementsareresponsiblefor
eachbehavior

225
APPENDIXA

Instrument

+ CaptureImages() : void

produces

Telecom

+ CAPTURE_IMAGES_CMD() : void Image Collection

+ ENTER_SAFEMODE_CMD() : void
+ RETRANSMIT_IMAGES_CMD() : void + GetImages() : void
+ STOP_IMAGE_CAPTURE_CMD() : void + SaveImageInOnboardStorage() : void
+ TRANSMIT_IMAGES() : void

sends images to

Ground Station Image Queue

Image
+ RECEIVE_IMAGES() : void + AddImage() : void
+ GetImage() : void

sends annotated image to analyzes

Image Analyzer produces Annotated Image

Subscriber
+ DetectPothole() : void + AnnotatePothole() : void
+ ReceiveAnnotatedImage() : void + SendAnnotatedImage() : void + Create() : void

detects

Pothole

+ CreateBoundaryOutline() : void

FigureA12.ExpandedDomainModelshowingbehaviorsallocatedtosystemelements

FigureA13showsthestatediagramforinitializingtheinstrumentandcapturingimages.
Afterinitializingbuffersandthecamera,theinstrumententerstheIdlestatewhereitwaitsfora
groundcommand.ReceivingaCAPTURE_IMG_CMDcausesatransitiontotheCapturingImages
Stateinwhichthecameraismadeready.
Imagesarecapturedandsavedbythedobehaviors.Imagesarecapturedinthecircular
bufferuntiltheSTOP_IMAGE_CAPTURE_CMDisreceived.Thiscommanddisablesthecamera
afterthecurrentimagecaptureiscompleteandtheinstrumentreturnstotheIdlestate.
FigureA14showsapowerparametricmodelcomprisingcamera,datastore,telescope,and
instrumentcomputerpowermodels.Theparametricmodelsumsthesecomponentstoforma
totalpowerusage.

226
 ROADMAP:DDT/SYSTEMS

Ground Sends Stop Command

Initializing
Instrument
Initial + do / initialize [STOP_IMAGE_CAPTURE_CMD]
/CompleteImageCapture

Idle Capturing Images

[CAPTURE_IMG_CMD]
+ do / waitForCommand
[STOP_IMAGE_CAPTURE_CMD]
+ do / captureImage
+ exit / disableCamera
+ entry / initializeCamera
+ do / saveImage

FigureA13.Allstatesandtransitionsmustbetested

Instrument Power = Telescope Power + Camera Power +
Data Store Power + Instrument Computer Power

Subsystem
Camera Power
Camera

Subsystem Data Store Power

DataStore

Instrument Power
Instrument Power

Subsystem Telescope Power

Telescope

System Instrument Computer

Instrument Power
Computer

FigureA14.InstrumentPowerParametricModel

ThiscompletesourdescriptionofbasicSysMLartifactsandsetsupthetestingdiscussionthat
follows.WhileourPOTHOLEexampleissimplisticandfanciful,themodelswedescribedarethe
basisformanysystems.
ThenextsectiondescribesthebaselineDDTmethodologyandrationale.Wefollowwitha
descriptionofanextensionforsystems.

227
APPENDIXA

Design Driven Testing

StephensandRosenberg57defineDesignDrivenTesting(DDT)asalogicalextensiontoUML
basedsoftwaredesign.ThepremiseofDDTisthatthebottomupagilepracticeofdriving
designsfromunittests(testdrivendesignTDD)isinherentlybackwards.InTDD,testcasesare
developedbeforecodedesign.Codeisthenwrittentomakethetestcasespassandrecodedas
necessaryiftestcasesfail.TDDproducesalotoftestcasesbutmoreseriouslybecauseofitsad
hocnature,doesntfullyaccountforacustomersneeds.Infact,theuserneedsaredetermined
bythetestresults!
Arguably,ignoringthecostofunnecessarytestingandrecoding,TDDworksinanagile
developmentcycle.However,thetypeofcodeproducedisnotsuitableforcriticalapplications
andmostuserscantolerateoddbehavioruntilthenextupdateispushedout.Noonewould
everdevelopspacecraftoranyothercriticalapplicationsusingTDD.
TheDDTapproachfollowsatopdownmethodologyinwhichthetestingisautomatically
generatedfromsoftwaremodels.Automationassuresthattestcasesarentmissedandthelink
tosoftwaremodelsassuresthattestcasesaredrivenfromthedesign.
DDTisanappropriatestartingplacefordevelopingSystemsV&Vtestmethodsbecausemost
systembehaviorisaccomplishedinsoftware.WedefineSysMLbasedextensionstoDDT
(DDT/Systems)laterinthepaper.SincewearestartingfromDDT,wepresentahighlevel
summaryoftheapproachhere,whichwewillillustratebyexampleshortly.
DDTautomatestestcasegenerationforbothdevelopersusingunittestsandforQAtesters
usingacceptancetests.FigureA15showsacceptancetestrelationships.Missionrequirements
aredrivenbycustomerneedsandusecasescenariosdefinewhatauserdoeswiththesystem.
DDTgeneratesteststhatevaluaterequirementcomplianceandcorrectimplementationof
desiredbehavior.AsshowninFigureA16,DDTalsoappliesduringthedevelopmentcycleby
automatingcreationofcontrollertestsfromrobustnessmodels(seenextsection)andunittests.

57
DesignDrivenTesting:TestSmarter,NotHarderbyMattStephensandDougRosenberg(Apress,2010).Alsosee:
www.designdriventesting.com
228
 ROADMAP:DDT/SYSTEMS

FigureA15.AcceptanceTestingFlow

FigureA16.DeveloperTestingFlow

229
APPENDIXA

TheDDTmethodologycanbesummarizedas:

Generatetestcasesfromrequirements.
Generatethreadtestsfromscenarios.
Generateunittestsfromconceptualanddetaileddesigns.

WewillillustrateRequirement,Scenario,andUnittestgenerationbyexampleinafewpages.
However,thesetestsarenotsufficienttotestallaspectsofahardware/softwaresystem.Our
nextsectiondescribesanextensiontoDDTsuitableforembeddedsystems.
DDT/SystemsExtensions
ExtendingDDTforsystemsnecessitatescoveringadditionalelementsasshowninFigureA
17.

System Tests

DDT Software
Tests

FigureA17:DDT/SystemsincludesDDTSoftwareandadditionaltests

Theseelementsinclude:

StateMachines
ActivityDiagrams
BlockDefinitionDiagrams
InternalBlockDiagrams
ConstraintBlockDiagrams

FigureA18showsthesystemextensionstoDDT.

230
 ROADMAP:DDT/SYSTEMS

DDT/Systems

DDT

- State Machine Tests

- Activity Tests
- Interface Tests (BDD/IBD)
- Parametric Tests

FigureA18:DDTSystemsextendsDDTforsoftwarewithadditionalsystemtests

WenextapplyDDT/SystemstothePOTHOLEspacecraft.

231
APPENDIXA

Validating POTHOLE with DDT/Systems

WefocusourPOTHOLEdesignandV&Vexamplesonasinglereferencefunction:takingand
processingimagesforfindingpotholes.WewalkthroughasimplifiedV&Vprocessbyexamplein
thefiguresbelow.ThenextsectionsummarizestherealworldV&Vprocess.
Nowwecometothenextvalidationstep:checkingthecompleteness,correctness,and
consistencyofourusecasedescription.Wewanttoassureourselvesthatwehaventforgotten
somethingorspecifiedsomethingtoovaguelyorincorrectly.Anyerrorsinourusecasemustbe
fixedandasnecessaryandthedomainmodeliterated.
Wearenowreadytodefinetestcases.Wecanautomaticallygenerateatestcaseelement
foreveryrequirementonarequirementdiagram,asshowninFigureA19.Requirementtests
areanintegralpartofthetestplan.Generatingtestcasesautomaticallyfromallrequirements
ensuresthatnoneareforgotten.

232
 ROADMAP:DDT/SYSTEMS

Images shall be transmitted to the ground at TestCase

150 Mbps Images shall be transmitted to the ground at 150 Mbps

(from Functional Requirements) test scripts

Unit: : (Not Run) Default Run Scenario

The spacecraft shall transmit images to the TestCase

ground in real-time when a ground station is The spacecraft shall transmit images to the ground in
available real-time w hen a ground station is av ailable

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

The spacecraft shall store images for at least TestCase

one hour The spacecraft shall store images for at least one hour

(from Functional Requirements) test scripts

Unit: : (Not Run) Default Run Scenario

The spacecraft shall retransmit stored images TestCase

when requested by ground command The spacecraft shall retransmit stored images w hen
requested by ground command

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

Retransmitted images shall be interleaved with TestCase

real-time images according to a ground Retransmitted images shall be interleav ed w ith
configurable downlink schedule real-time images according to a ground configurable
dow nlink schedule
(from Functional Requirements)

The spacecraft shall store images in a circular
buffer over-writing older images with newer
images
test scripts
Unit: : (Not Run) Default Run Scenario
TestCase
The spacecraft shall store images in a circular buffer
ov er-w riting older images w ith new er images

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

Images shall be downlinked in groups of 16 TestCase

using CCSDS format Images shall be dow nlinked in groups of 16 using
CCSDS format

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

The field of view shall be at least 1.0 Km x 1.0 TestCase

Km The field of v iew shall be at least 1.0 Km x 1.0 Km

(from Functional Requirements) test scripts

Unit: : (Not Run) Default Run Scenario

The telescope shall have an adjustable aim TestCase

point The telescope shall hav e an adj ustable aim point

(from Functional Requirements) test scripts

Unit: : (Not Run) Default Run Scenario

Each image pixel shall have ground resolution TestCase

of no more than 0.25 meters Each image pixel shall hav e ground resolution of no
more than 0.25 meters

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

FigureA19.POTHOLEMissionRequirementTestCases

Weuseausecasethreadexpandertoautomaticallygeneratetestscriptsforallsunny
day/rainydaypermutationswithinausecase.Thepremiseofthisapproachisthatausecase
withonesunnydayscenarioandthreerainydayscenariosrequiresatleastfourtestscriptsin
233
APPENDIXA

ordertoexerciseallusagepaths.ThisprocessisillustratedinFiguresA19toA22.FigureA20
showsastructuredscenariothatspecifiesstepnumbersandjoinlogicforourusecase.

FigureA20.Expandedsunny/rainydaythreads:createastructuredscenario

FigureA21showsanautomaticallygeneratedactivitydiagramthatisusedtoverifythelogic
ofthescenariobeforetestsaregenerated.

234
 ROADMAP:DDT/SYSTEMS

FigureA21.Generateanactivitydiagramandcheckflowagainstusecases

235
APPENDIXA

FigureA22showsatestcasediagram,showingthecompleteusecasetextinanote,anda
testcaseelementwithatestscenario(threadtest)foreachCourseofActionwithintheuse
case.

FigureA22.ScenarioTesting

Usagepathsareautomaticallygeneratedforeachthread,asshowninFigureA23.These
threadsshowportionsoftheBasicCourseandportionsoftheAlternateCourses,asrelevantfor
eachthread.

236
 ROADMAP:DDT/SYSTEMS

FigureA23.Generatetestscriptsfromusecasetext

DDTfollowsausecasedrivenapproachtodesigninwhicheachusecaseperformedbythe
systemisfirstelaboratedataconceptuallevel(refinestheusecaseandvalidatesthesystem)
andthenatadetaileddesignlevel(forsystemverification).DDTgeneratestestcaseelements
fromcontrollersonconceptualdesign(robustness)diagrams,asshowninFigureA24.
Unittestsarealsogeneratedfrommessagesondesignlevelsequencediagramsasshownin
FigureA25.AftertestscenarioshavebeendetailedwithInputs,Outputs,andSuccessCriteria,
thetestcasesareautomaticallytransformedintounittestcodesuitableforregressiontesting
frameworkssuchasJUnit,NUnitandFlexUnit.58

58
See:www.junit.org,www.nunit.organdwww.flexunit.orgrespectively.
237
APPENDIXA

TestCase
Retransmit Images Test

Retransmit Images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Analyzer Test

Image Analyzer
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
capture images Test

capture images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
send images Test

send images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
add image Test

add image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
get image Test

get image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
detect pothole Test

detect pothole
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
annotate image Test

annotate image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
send annotated image Test

end annotated image

test scripts
Unit: : (Not Run) Default Run Scenario

FigureA24.POTHOLEControllerTestCases
238
 ROADMAP:DDT/SYSTEMS

TestCase
Ground Station RECEIVE_IMAGES Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Queue AddImage Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Queue GetImage Test

test scripts
Unit: : (Not Run) Default Run Scenario
ref
Look for Potholes Sequence TestCase
Image Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Annotated Image Create Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Analyzer DetectPothole Test

test scripts
Unit: : (Not Run) Default Run Scenario

FigureA25.SubsetofAutogeneratedLookforPotholesSequenceMessageTests

FigureA26showsstate,statetransitions,andbehaviortestcasesforthecaptureimage
statemachinedescribedearlier.Allbehaviors,transitionsandtriggersmustbetestedincluding
entry,do,andexitbehaviorsonstates.Notethatsimilartestsmustalsobedefinedfor
activitydiagrambehavioraldescriptions.
InterfaceControlDefinitionsmustalsobetested.Thisincludeselectricalandlogical
interfacedefinitionsasshowninFigureA27.Thisincludesport,required/providedinterfaces,
electricalandtiming,andfaultbehaviortests.

239
APPENDIXA

TestCase
Test Capturing Images

Capturing Images
test scripts
+ do / captureImage Unit: : (Not Run) Test captureImage
+ exit / disableCamera Unit: : (Not Run) Test completeImageCapture
+ entry / initializeCamera Unit: : (Not Run) Test disableCamera Behaviors,
+ do / saveImage
Unit: : (Not Run) Test initializeCamera transitions and
Unit: : (Not Run) Test Recieve Capture Images Command
Unit: : (Not Run) Test Recieve Stop Image Capture Command triggers must all
Unit: : (Not Run) Test saveImage be tested.
Unit: : (Not Run) Test Stop Image Capture

Idle TestCase
Test Idle
+ do / waitForCommand

test scripts
Unit: : (Not Run) Test waitForCommand

TestCase
Test Initializing Instrument
Initializing Instrument

+ do / initialize test scripts

Unit: : (Not Run) Test initialize

FigureA26:Examplesofstatetests

Board TestCase
flowPort Test Instrument Data Input
NVM
InstrDataIn

TestCase
Test Buffer Control flowPort
Interface FIFOCtlIn

flowPort TestCase
FIFOStatusOut flowPort Test XBand Output
XbandDL

(from Block Diagrams)

FigureA27.NeedtotestallbehaviorsandinterfacesdefinedinanInterfaceControlDocument(ICD)

Inmanycases,analyticalmodelsareneededforevaluatingnonbehavioralrequirements.
Thesemodelscouldevaluatemass,power,radiationeffects,andsoforth.FigureA28showsa
parametricdiagramthatevaluatesinstrumentpower.Thecomputedtotalpoweriscomparedto
amaximuminstrumentpowerrequirement.

240
 ROADMAP:DDT/SYSTEMS

par [SysML Parametric] Block Diagrams [Instrument Parametric Diagram]

Instrument Power = Telescope Power + Camera Power +

Data Store Power + Instrument Computer Power

Subsystem
Camera Power
Camera

Subsystem Data Store Power

DataStore

Instrument Power Max Instrument Power

Instrument Power notes

Not to exceed 500W
Subsystem Telescope Power
Telescope

System Instrument Computer

Instrument Power
Computer

TestCase
Test Instrument Pow er

test scripts
Unit: : (Not Run) Test Instrument Power does not exceed Max Instrument Power

FigureA28.Parametricevaluationofinstrumentpower

ThisconcludesourillustrationofDDT/SystemsusingthePOTHOLEexample.
Asillustratedinthissection,DDT/Systemscomprisesthefollowingsetoftests:
Requirementtests
Scenariotests
Controllertests
Sequencetests
Statemachine&Activitytests
Interfacetests
Parametrictests
ThisisthemostcomprehensiveandefficientsystemV&Vapproachweareawareof.

Conclusions
DDTautomatesgenerationoftestcasesfromUMLmodelsforsoftware
SystemsV&VrequiresadditionalmodelingandtestcasegenerationfromSysML
OurfictitiousPOTHOLEspacecraftillustratessomeoftheseextensions
WehavedescribedproposedadditiontoDDTthatencompassessystems(DDT/Systems)

241
 COMINGSOONFROM

fingerpress

HAMSTER
DRIVEN
DEVELOPMENT

(itsnotforthesqueamish)
SatirefromthecodingroomfloorfromtheauthorsofDesignDrivenTesting:TestSmarter,
NotHarderandExtremeProgrammingRefactored:TheCaseAgainstXP(featuringSongsof
theExtremos)

www.fingerpress.co.uk

DESIGN DRIVEN TESTING

ThegroundbreakingbookDesignDrivenTesting
bringssanitybacktothesoftwaredevelopment
processbyflippingaroundtheconceptofTest
DrivenDevelopment(TDD)restoringthe
conceptofusingtestingtoverifyadesigninstead
ofpretendingthatunittestsareareplacement
fordesign.
AnyonewhofeelsthatTDDisTooDamn
Difficultwillappreciatethisbook.




Visitthewebsiteformoreinfo:
www.designdriventesting.com

DesignDrivenTestingshowsthat,by
combiningaforwardthinking
developmentprocesswithcuttingedge
automation,testingcanbeafinely
targeted,businessdriven,rewarding
effort.Inotherwords,youlllearnhow
totestsmarter,notharder.

 OPEN
ENROLLMENT
TRAINING

Youvereadthebook,nowgetthetraining!

Learn how to apply the process roadmaps youve just read

about in this book from Doug at an ICONIX Open Enrollment
workshop

Developers:Learnarigorous,systematicpathfromrequirementstosourcecode
BusinessAnalysts:Accomplishbusinessprocessmodelingforrequirementselicitationand
processreengineering
SystemsEngineers:ApplyarigorousdesignandtestingmethodologyusingSysML

TheDesignDrivenTestingcourseusesasitsexampleaninteractivehotelmappingsystemwhich
youcantryoutforrealat:www.vresorts.com

ForthelatestOpenEnrollmentschedule,pleasevisit:
www.iconixsw.com/EA/PublicClasses.html