SIM Group Ltd.

, SQS Group AG, 2002

ISEB Foundation Certificate in Software Testing
Economics of Testing
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
In this session we will

Look at the cost of testing (or the cost of not testing)

Appreciate the value of good, timely testing
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
The earlier a fault is found, the cheaper it is to remedy

Faults in the requirements can lead to major re-engineering of
the entire system

Many faults can be found using reviews
Reviews of documentation and / or code

Allows testing to find more substantial faults
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
The average cost of fixing a defect increases tenfold
with every step of the development process

In code reviews you will find and fix defects in an average of
1 - 2 minutes
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
In initial testing, defect fix times will average between
10 to 20 minutes

In integration testing each defect can cost an hour or
more

In system test each defect can cost 10 to 40 or more
engineer hours

Watts Humphrey
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Early test design can prevent fault multiplication

Analysis of specification during test preparation often brings
faults in the specification to light

If faults in documentation are not found then the system may
be developed incorrectly
SIM Group Ltd., SQS Group AG, 2002
Economics of testing
An example
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in the requirements

Change requirements document
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in the specifications

Change requirements document

Change specifications
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in the code

Change requirements document

Change specifications

Change the code
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in system testing

Change requirements document

Change specifications

Change the code

Change the tests, re-test & regression test
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in UAT

Change requirements document

Change specifications

Change the code

Change the tests, re-test & regression test

Re-do UAT
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in Live

Change requirements document

Change specifications

Change the code

Change the tests, re-test & regression test

Re-do UAT
Plus...
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Finding the error in Live

User to report and log fault

Users must be advised of fix, work-around etc

The live database may need to be fixed

Deploying the new build may take hours

Loss of business
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Additionally

Faults found in coding or later require changes to the code
library

Documentation and help files will have to be changed

Every fault requires investigating and logging

Administration costs
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
The cost of not Testing

What is the cost of testing?

Which is cheaper?
Testing and finding faults before release
Not testing and finding faults once the system is live

Companies do not typically have figures for either
SIM Group Ltd., SQS Group AG, 2002
Economics of Testing
Summary

The purpose of testing is to find faults

The earlier a fault is found the cheaper (and easier) it is to fix

Starting testing early will help find faults quicker