1.

INTRODUCTION 2

1.1. WHAT IS TESTING? 2

1.2. THE TESTING PROCESS 2
1.3. TESTING TECHNIQUES 2
1.4. TOP DOWN AND BOTTOM UP INTEGRATION 3
1.5. UNIT TESTING 3
1.6. BLACK BOX TESTING 4
1.7. WHITE BOX TESTING 4

2. UNIT TESTING - IN GENERAL 5

2.1. GENERAL PROCEDURE FOR UNIT TESTING 6

2.2. GENERAL RULES FOR UNIT TESTING(RU-G) 8
2.3. GENERAL CHECKLIST FOR UNIT TESTING (CU-G) 8

3. EQUIVALENCE TESTING 9

3.1. DEFINITION 9
3.2. PROCEDURE FOR EQUIVALENCE TESTING (PU-EQ) 9
3.3. RULES FOR EQUIVALENCE CLASS IDENTIFICATION (RU-EQ) 9

4. BOUNDARY VALUE TESTING 11

4.1. DEFINITION 11
4.2. PROCEDURE FOR BOUNDARY TEST CASE IDENTIFICATION 11
4.3. RULES FOR BOUNDARY TEST CASE IDENTIFICATION : 11

5. HEURISTIC TESTING 12

5.1. DEFINITION 12
5.2. PROCEDURE FOR HEURISTIC TEST CASE IDENTIFICATION: 12
5.3. RULES FOR HEURISTIC TEST CASES IDENTIFICATION (RU-HR) 13

6. COVERAGE TESTING 13

6.1. DEFINITION 13
6.2. GENERAL PROCEDURES FOR ALL COVERAGE TEST CASES 13
6.3. GENERAL RULES FOR WHILE BOX TESTING (RW-G) 13
6.4. GENERAL CHECK LIST FOR WHITE BOX TESTING (CW-G) 13

7. GLOSSARY: 14

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :1/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
TESTING PROCESS

1. INTRODUCTION

Testing is an important stage of s/w life cycle. There are two kinds of testing possible. One is
validation and other one verification. Verification is doing right things. Validation is doing
things right. Here the testing process is to the validate the source code. This process is only for
manual testing.

1.1. What is Testing?

Testing is the process of operating a system or component under specified conditions,

observing or recording the results and making an evaluation of some aspects of the system or
component. (IEEE/ANSI, Std-10.12-1990)

Testing is an important stage in the software development process. Testing consumes a

significant human effort. Of the total software development effort, testing takes 35 to 50%.
Testing is a very important element in the QA strategy of any software development
organization.

1.2. The Testing Process

The Testing Process is generally described by the V model shown below.

Requirement Acceptance Testing

Analysis System Testing

System Design Integration Testing

Detailed Design Unit Testing

Coding

Testing is normally done with a bottom-up approach. The first step in testing is Unit Testing in
which the individual functions are tested. During integration, these functions are integrated into
a system. During Integration Testing proceeds incrementally. At the end of integration, the
system as a whole is tested. This stage is called System Testing. The testing activities are thus
done in three distinct stages. Each of these stages are important.

1.3. Testing Techniques

The Process of Testing checks the implementation using the desired specification as a reference
with an intent to find defects in the implementation. The Implementation must fully satisfy the
Requirements. While translating the requirements into implementation, the following are the
possibilities:

1. Correct Implementation :Some requirements are implemented correctly

2. Incorrect Implementation :Some requirements are implemented wrongly
3. Omissions :Some requirements are not implemented and omitted by
mistake
4. Surprises :Some implementation do not correspond to any
requirement
Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :2/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
 Requirements

Implementation
Omissions

Correct
Implementation

Surprises

Incorrect
Implementation

Different testing techniques will have to be used to handle these different possibilities.
Omissions, Surprises and Incorrect Implementation must be detected and the implementation
must be corrected.

The following matrix shows which technique is useful for what type of problem.

Ability to Catch
Testing Approach Omissions Surprises Incorrect
Implement.

Black box/Functional Testing yes yes

White box/Implementation based Testing yes
Heuristic Testing yes yes yes

The testing strategy must take this fact into account and use complementary testing techniques
to unearth all the problems. Thus effective Testing is a combination of Black box, White box
and Heuristic Testing Techniques.

1.4. Top down and Bottom Up Integration

Integration of software may be done in a top down or bottom up fashion. In Bottom up testing
all lowest level units are tested first. Then the higher level units are tested. The units already
tested are used during testing higher level modules. Stubs are not needed for bottom up testing.
In top down testing, the top most layer of Units are tested first followed by the next lower
layer of units. While testing the second layer of units, the already tested top layer units are
used. There is no need for Drivers in the top down testing.

1.5. Unit Testing

An Unit is a basic functional block in software. It is a function, procedure or a subroutine in a

programming language like C. It forms the basic Unit by which blocks, subsystems or systems
are built. Unit Testing is the first stage of testing. It immediately follows the coding stage. As
discussed before, Black box, White box and Heuristic Testing must be used to expose all the
problems in the Unit under test. Every Unit occupies an unique position in a hierarchy of units.
Before integrating the units together, it is very desirable that each unit is tested individually. The
Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :3/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
Unit Testing stage ensures that an Unit behaves correctly before starting to test the interactions
between different units by integration testing.

Some engineers follow a testing strategy which combines unit and integration testing.
Whichever strategy is followed, one objective is to look for problems within the individual units
and the second objective is to look for problems in the manner in which these units interact.
Unit Testing meets the first objective while integration testing meets the second objective.

1.6. Black box Testing

Black box or responsibility based testing views the unit or system under test as a Black box and
involves test cases which exercise the responsibilities to be met by the unit or system under test.
In the case of units, the Input output specifications of an individual unit is taken and the unit is
subjected to a set of cases that attempt to find omissions and defects in the implementation vis a
vis the requirements.

The most important Black box testing techniques include Equivalence Partitioning and
Boundary Value Analysis.
1.7. White box Testing

White box testing assumes a knowledge of the software under test. It is also called structural
testing sometimes. The structure of the software under test is used to generate test cases to
problems of surprises and incorrect implementation. The white box testing techniques typically
involve exercising the different sections of the code systematically. The structure of the
software and ways in which they may be exercised vary so much that a large number of test
cases are necessary to test “all possible combinations”. This is not necessary and instead an
increasingly comprehensive coverage criteria are defined which can be used while identifying
White box test cases. These criteria are called the coverage criteria. The well accepted
coverage criteria include:
• Statement Coverage
• Branch Coverage
• Decision Coverage
• Loop Coverage
• LCSAJ Coverage

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :4/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
2. UNIT TESTING - IN GENERAL

START

Identify Black Box Test

1 Cases
I/O Specification

Generate Black Box Test

2
cases

Document Generated
3
Test Case Definitions Test Case
Definitions in TCE
form
Write Drivers And
4 Stubs
Test Source for all
TCs

5 Execute Test Cases

Updated TCE form
with actual values
No and Coverage Details
check if desired End Report
6 coverage achieved
(exit criteria)
Yes

7 Identify uncovered
Branches

Identify Additional
8
white box test case

9 Generate White box Test

cases

Identify Black Box Test Cases

1.1 Identify Equivalence Test

cases
Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :5/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
 1.2 Identify Boundary Test
Cases

1.3 identify Heuristic

Test Case & classify them
Boundary / Equivalence
tcs etc.

Identify White Box Test cases

8.1 Identify Testcase for

Uncovered statements

8.2
Identify Testcase for
Uncovered Branches

8.3 Identify Testcase for

LCSAJ

2.1. General Procedure for Unit Testing

1. Identify Black Box Test Cases

1.1. Identify all the input and output data entry fields for the selected unit. There may
be some indirect data fields also. (ex. Additem_to_a_linked_list, or a functions
return value, setting a variable indirectly through the called function.)

1.2. For each of the identified data fields, identify the possible values that these data
fields can hold (both in the input and output space).

Note : Here, you have to use the following testing techniques in the given order, to identify
black box test cases.
Equivalence Class
Boundary values
Heuristic

1.3. Take a particular field as focus and list the possible values for that variable.
Repeate the setps 2 to 5 for all the identified data fields.

2. Generate Black Box Test Cases

2.1. Generate a test case by assigning a value from the list of possible values for the
selected variable and assigning any one of the possible values for all other data
entry fields in input and output. This is called a test case definition.
Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :6/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
3. Document the Generated Test Cases

3.1. The test case definitions should be entered into the Test Case Entry Template
(given in subsequent sections).

4. Identify Called functions

4.1. Identify called functions and decide whether they have to be called as actual
functions or they need stubbing for each test case.

4.2. If stubbing is necessary, for those function calls, select the necessary variable
and decide the necessary values for those data entry fields also.

5. Write drivers and stubs

5.1. Write driver code using the above test case definitions.

5.2. Write stubs with the above test case definitions obtained from step 2.7, and
modify the unit accordingly to support the stub calls instead of actual calls, is
necessary. These details also should be added to the test case definition.

6. Execute Test cases

6.1. Execute the test case(s).

6.2. Analyze the result for expected result after executing all the test cases.

7. Establish coverage

7.1. To establish coverage, while generating drivers and stubs itself, instrument the
test source so that it prints the branch and node it has traversed. The same thing
can also be done by debugging the test source.

7.2. While executing the test cases, note down the path taken by seeing the printed
mater on screen or by stepping through the FUT.

Note : Instead of the above two steps, you can use FasTest to just establish the coverage by
entering the identified testcases and executing them(incase of ‘c’).

8. Identify additional test case for coverage

8.1. If desired level of coverage is not achieved, identify additional paths and
branches which are left not covered by the currently available set of test case
definitions.

8.2. Identify additional test case to cover the remaining statements/ branches /paths.

8.3. This can be done by slightly modifying the test cases which has covered the
adjacent path.

9. Generate White box Test cases:

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :7/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
 9.1. Generate additional test cases by modifying existing test cases or generate new
test cases to cover entirely new path. The test case should be generated in the
given order
Coverage techniques
Statement coverage
Branch coverage
Path coverage.
Deccanet’s standard is Branch coverage unless otherwise specified for a particular project.

9.2. After this goto setp PU_G5

2.2. General Rules for Unit Testing(RU-G)

RU_G1 The unit under test should be compliable

RU_G2 The unit should have passed the normal inspection review process

RU_G3 Identify and remove duplicate test cases(keep the no. of test case less)

RU_G4 Test cases should have a reasonable chance of catching defects.

RU_G5 Possible values of each data entry fields should lie with in the acceptable range of the
underlying data type.

RU_G6 Consider all the direct and indirect data entry fields as a focus variable.

RU_G7 Consider both input and output variables / events as focus and generate test cases
around all of them.

RU_G8 Don't omit any test case generated for each focused variable’s possible values.

RU_G9 Don’t forget to handle any of the called functions.

RU_G10 Initialize all the pointers with appropriate value.

RU_G11 Don’t generate too complex or too trivial test cases.

RU_G12 Ensure that the standards are met from coverage point of view.

2.3. General checklist for Unit testing (CU-G)

CU_G1 Whether the unit under test is compliable or not.

CU_G2 Confirm that the unit under test had already been subjected to normal inspection /
review process.

CU_G3 Check whether the test case is too complex or too trivial?

CU_G4 Check whether a redundant test case exists?

CU_G5 Check whether the test cases have a reasonable change of catching defects?

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :8/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
CU_G6 Verify that the possible values of each data entry fields lie with in the acceptable range
of the underlying data type.

CU_G7 Check whether each of the direct and indirect data entry fields are considered as a
focus variable.

CU_G8 Check whether all the possible test cases are there for each of the focused variables.

CU_G9 Check whether all the called functions are handled properly by either stubbing it or
calling it as an actual function.

CU_G10 Check whether all the pointers are initialized properly?

CU_G11 Verify whether the standards are met from coverage point of view.

3. EQUIVALENCE TESTING
3.1. Definition

If you expect a no. of test cases to be testing the same behavior or the Function Under
Test(FUT), then you must treat them as equivalent. A group of tests forms an equivalence class
if you believe that :
• they all test the same thing
• if one test catches a bug, the others probably will too
• if one test doesn’t catch a bug, the others probably won’t either.
[ref. : 126 of Cem Kaner]

3.2. Procedure for Equivalence Testing (PU-EQ)

Follow the general procedure for unit testing. In addition to that, follow the procedure given
below, to identify equivalence classes:

1. Analyze the possible values that can be entered for each data fields(both in input and
output space) and group those values into possible equivalence classes.

2. To group the values into equivalence classes, decided whether the possible values of the
identified data field fall under any one of the following category
• Range of value
• List of values
• Specific value

3. As per the rules for the corresponding category define the test cases.

4. For all the data entry field (in the input and output space) repeat the steps 2 and 3 until
all the variables are subjected to this procedure.

3.3. Rules for Equivalence Class Identification (RU-EQ)

RU_EQ1 For any variable whose possible values lies in a range, then generate 3 equivalence
classes.
Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :9/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
Ex: If your possible values lies in a range(say 10 to 90) for a particular variable, then the
equivalence classes are
• <10 Invalid equivalence class
• >90 Invalid equivalence class
• 10 to 90 Valid equivalence class

RU_EQ2 For any variable, if the possible range of values can be further subdivided into ‘n’ sub
ranges, then define n+2 equivalence classes.
If the range can be further sub divided into sub ranges(say 10-30, 30-60 60-90), then consider
the following equivalence classes also

• 10 - 30 Valid Equivalence class

• 31-60 Valid equivalence class
• 61-90 Valid equivalence class
• <10 Invalid equivalence class
• >99 Invalid equivalence class

RU_EQ3 For any variable, if the possible value is a list of ‘n’ items, then ‘n+1’ equivalence
class should be defined, if the behavior of the FUT is expected to be different for each.
Ex: If the possible values lie in a list of n item, then n+1 equivalence class should be generated
with each list item as an individual valid equivalence class and an item not present in the list
should form the n+1th equivalence class, which is an invalid equivalence class.
Ex: If your possible values lie in a list of items, (say a, m, k) then the equivalence classes are
• Each of the list items is a valid equivalence class (a, m, k - 3 valid ECs )
• Items which are not there in the list forms one Invalid equivalence class

RU_EQ4 If the possible value is a specific values, then two equivalence class should be
generated one with the specific value and other class with any values other than the specific
Ex: If your possible value is a specific value (say 0 always), then the equivalence classes are
• specific value (i.e.) 0 Valid equivalence class
• all values other than 0 Invalid equivalence class

RU_EQ5 If the input specifies the number(N) of valid values, define one valid Equivalence
class and two invalid equivalence classes.
EX : if the valid name of project requires at least one letter and not more that 8 characters then
any name with 1 - 8 letters Valid equivalence class
any other combination of characters Invalid equivalence class

RU_EQ6 Don’t forget to test invalid equivalence classes for any variable.

RU_EQ7 Don’t omit sub equivalence classes within valid equivalence classes

RU_EQ8 Identify time-determined equivalence classes(ex. press the shift key just before,
during, and just after the system finishes scanning for keyword)

RU_EQ9 Don’t forget to identify indirect(hidden) equivalence classes

Ex. if the function deals with insertion of an item to linked list, then adding an item to the list
when the list contains less than maximum no. of elements becomes an equivalence class.
Adding an item to the list, when the list contains maximum no. of elements.

RU_EQ10 Don’t forget to focus on output variables. Consider all the output variables also for
generating equivalence classes. Apply all the above rules on these variables also.

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :10/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
RU_EQ11 Some times, a group of variable may gives raise to a possible value for a particular
situation. In this case also, apply all the above rule on these group of variables.

4. BOUNDARY VALUE TESTING

4.1. Definition

Boundary value testing is a variant and refinement of equivalence partitioning, with a major
difference - Rather than selecting any element in an equivalence class as being representative,
elements are selected such that each edge of the equivalence class is the subject of a test.
Boundaries are always a good place to look for defects.

4.2. Procedure for Boundary Test case Identification

In addition to the general procedures, the following should also be considered for boundary test
case generation

1. Analyze the possible values that can be entered for each fields(both in input and output
space) and group those values into possible boundaries with precise lower and upper
limits.
2. To group the values into boundary test cases, decide whether the possible values of the
identified data filed falls under any one of the following category :
• Range of Values
• List of continuous values

3. As per the rules for the corresponding category, define the test cases

4. For all the data entry fields, repeat steps 2 to 3 until all the variables( in the inputs and
output space) are subjected to this procedure.

4.3. Rules for Boundary Test case Identification :

RU_BV1 For any variable whose possible values lie in a range, then generate 3 equivalence
classes.
Ex :If your possible values lies in a range, say 10 to 90 then the boundary classes are
• 9,91 Invalid boundary values
• 10, 11, 89, 90 Valid boundary values

RU_BV2 For any variable, if the possible range of values can be further subdivided into ‘n’ sub
ranges, then ‘n+2’ equivalence classes should be defined.
Ex : If the range can be further sub divided into sub ranges(say 10-30, 30-60 60-90), then
consider the following boundary test cases also
• 9, 91 Invalid boundary test cases
• 10, 11, 29, 30 Valid boundary test cases
• 31, 32, 59,60 Valid boundary test cases
• 61,62, 89,90 Valid boundary test cases

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :11/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
RU_BV3 For any variable if the possible value lies in a continuous list of n items, then define
the boundary test cases by keeping the following consideration in mind.
Ex : If your possible values are a list of continuous items ( say m,n,o,p), then the boundary
classes are
• Each of the list items is a valid boundary class (m,n,o,p - valid boundary cases)
• One item below the lower boundary and one item above the upper boundary list
forms one Invalid equivalence class (l, q - Invalid boundary cases)

RU_BV4 Consider the no. of inputs to a program as an input data field and try defining test
cases around the limit of the no. of inputs
Ex: Some times no. of inputs to a program may gives raise to a test case. In this case, consider
the range from 1 to n as a valid range, give 0, 1,2,n-1, n and n+1 inputs for the program and
test it.

RU_BV5 Think of boundaries on hidden data fields, which are not explicit as a defined variable
in the project, and generate test cases around the boundary of possible values of that hidden
variable.
Ex: If the functionality of the function is to add an item to linked_list, then add and item when
the list is empty, when there is an item, when there is n-1 item where n is the maximum
capacity of the list). This will form 3 valid boundary cases. Add an item when the list is full.
This is an invalid boundary case.

RU_BV6 Look for boundaries around the loop variable limits( if any loop exists in the FUT).
Ex: If any loop is there in the FUT, the executed the loop for 0 times, 1 item, n-1 time and n
times, where n is the maximum no. of times the loop has to be executed.

RU_BV7 Don’t forget to test invalid boundary cases for any variable.

RU_BV8 Don’t omit any valid boundary cases also.

RU_BV9 Don’t forget to generate test cases for sub ranges within a range.

RU_BV10 Look boundaries around each valid equivalence classes

RU_BV11 Identify time-determined boundary classes.

Ex: Assume, the first digit has to be dialed within 5 sec after getting dial tone, then press the
first digit exactly at 5th sec just before and after 5 sec.

5. HEURISTIC TESTING
5.1. Definition

Heuristic Testing is an approach, based on intuition and experience, to identify tests that are
considered likely to expose errors. The basic idea is to make a list of possible errors or error-
prone situations and then develop tests based on the list. With the past experience we need to
give some inputs to the present program, so as to check whether the most common defects still
exists or not.

5.2. Procedure for Heuristic Test case identification:

Identify test case based on your judgement and previous experience

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :12/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
5.3. Rules for Heuristic Test cases identification (RU-HR)

In addition to the general rules, the following rules also should be followed to generate
heuristic test cases

RU_HR1 Don’t omit any variable which are directly or indirectly affecting the functionality of
the Function Under Test.

RU_HR2 Test the unit with some random values for all the variables, without focusing on any
particular variable.

RU_HR3 Any interesting feature, combinations, sequence which you feel may hold a surprise
should be exercised.

RU_HR4 Check for rarely executed conditions and sequences.

RU_HR5 Identity test cases by recollecting situations from past experiences

RU_HR6 Try to create possible error conditions.

RU_HR7 Identify test case which seem to be “interesting” to you.

6. COVERAGE TESTING
6.1. Definition
These are the testing techniques that takes advantages of the structure of the program under
test are grouped under white box or structural testing. Structural testing complements black
box testing
6.2. General Procedures for all coverage test cases

Execute the setps explined in the section PU_G (PU_G14 to PU_G18)

6.3. General rules for While box testing (RW-G)

RW_G1.Confirm that the identified test cases are covering expected branches / statements.

RW_G2. Don’t generate duplicate , trivial test cases which does not give an incremental
coverage

RW_G3. Ensure that the desired level coverage is achieved by meeting the organization
standards

RW_G4. Give more attention to the rarely accessed branches and statements

RW_G5. Unreachable code, if any, should be properly commented, so that the coverage
calculation are not affected
6.4. General check List for white box testing (CW-G)

CW_G1 Check for duplicate , trivial test cases

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :13/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001
CW_G2 Check whether all the branches are covered

CW_G3 Check whether identified TCs are covering the expected branches /statements or not

CW_G4 Verify whether all the rarely accessed branches and statements are given more
attention

CW_G5 Check whether if any unreachable code left uncommented

Note : Here there is no need for specific procedure /rule and checklist for each coverage
techniques and for all techniques the same procedures and rules are applicable.

7. GLOSSARY:
FUT : Function Under Test
LCSAJ : Linear Code Segment And Jump
EC : Equivalence Classes
BV : Boundary Values
TC : Test Case
Organizational Standards : 100 % Statement Coverage
100 % Branch Coverage

Doc. info. : Rules and steps for Testing Phase Doc. Class: Testing Process Page No. :14/14
Custodian: Prashant Hemchandra Manapure Version :TR-1
Date: 21st March 2001