Protocol testing techniques
Baoyu Wang* and David Hutchison t provide a survey of testing techniques
for communication protocols
This paper deals with testing techniques for communication
protocols. Several protocol testing support systems
designed for testing protocol layers of the ISO/OSl
(International Standards Organisation/Open Systems
Interconnection) Reference Model are introduced. These
systems are mostly designed for the conformance testing
of protocols produced by third parties, which is intended
not only to provide test sen/ices for both suppliers and
users but also to meet the requirements of standards.
Several test case generation methodologies are described,
and the authors' experience in using one of the methods is
presented.
Keywords: communication networks, protocol testing
conformance, third party testing
In data communications it is generally accepted that
protocols should be standardized, otherwise it will be
hard to reach the goal of open communication among
different systems. However, users must be assured that
products based on the same standard but produced by
different suppliers can communicate with each other
successfully and reliably. Protocol testing can be used to
show whether the products under test meet the standard
specification. In this situation suppliers and users may rely
on independent agencies who provide a test service for
the conformance of protocol products to standards (such
an agency is called a third party).
How are communication protocols tested? The major
effort in testing has been in two areas. The first area
involves building a support test system; this is because the
implementation of protocols should be tested by a
trusted independent assessment system. The test system
should provide a simulated environment which ensures
*Computer Centre, Nanjing Institute of Technology, Nanjing, Jiangsu,
People's Republic of China
tDepartment of Computing, University of Lancaster, UK
0140-3664/87/020079-09 $03.00 © 1987 Butterworth & Co (Publishers) Ltd
vol 10 no 2 april 1987
that the protocol is working under actual running
conditions. Traditional testbeds for testing protocol
implementations are separately designed by suppliers
and users to their own requirements. Obviously, there are
two main disadvantages. First, the cost of building
individual testers will be high. Second, as different
products choose their own reference product for
communication, standardization for protocol products
will hardly be attained. As a result of these, a third party
can be employed to test products in order to eliminate
both disadvantages.
The second area is to create or to apply a suitable
method for generating efficient test cases to test
implementation products. During one test, thousands of
test cases may be executed. How can finer test cases be
applied, and yet all the errors surely found? How can a test
system generating test cases require less human involve-
ment? These are the main research interests in this
area.
CONFORMANCE TESTING
The term 'testing' indicates that some operations are
carried out to certify the conformity of an implementation
to its specification. In data communications, protocol
testing means that the test object is a protocol implemen-
tation. It can be simply defined: executing a sequence of
coordinated operations to drive the implementation of
the protocol through a given sequence of input stimulations
and observing how the implementation responds to well
determined actions. However, testing a protocol also
involves testing the services which are provided by the
protocol. Such testing is called 'service testing', and it
verifies the interaction of the protocol implementation
with its user. Faults detected in service testing can be
evidence of a problematic protocol implementation, but
the absence of faults detected in service testing may be
insufficient to declare that the protocol implementation is
correct.
79
 In software engineering, testing can usually be designed
from either a functional or a structural point of view.
Structural testing is often called 'white-box' testing which
is particularly used by the implementer to check the
internal operation of a program. Functional testing is
commonly known as 'black-box' testing: it treats the
program or system as a black-box and assumes that the
internal structure of the implementation is not available.
For communication protocol testing both sorts of tests
are necessary. It is also necessary to have a trusted
protocol (a 'peer protocol' or reference protocol) to
communicate with. A reference protocol implementation
should be connected with the implementation under test
in order to access the test object within an operational
environment. If the protocol conforms to its specification,
the communication should be successful, without any
error. Normally the tests are done by two parties: one is
the supplier and the other the user. For communication
protocol products, third parties are active in testing
product conformance to the ISO's (International Standards
Organization's) OSI (Open Systems Interconnection)
standard network model.
The supplier, as the first party, should test the
conformance of a product to its specification and obtain
enough evidence to show how reliable the product is.
The user, as the second party, does not need to
understand different implementations of a protocol, and
is only concerned with services provided. As the service
specification is semantic-free, all correct implementations
of a protocol should provide the same services. The users
have to ensure that the products can meet their own
requirements and thus will set certain criteria to measure
whether the product is of good quality. The third party is
independent from both suppliers and users, and normally
provides several testers which are designed for individual
layers of the OSI standard. Any OSI products to be tested
are connected with the test system so that conformance
testing can be carried out in a standard communication
environment.
During the last few years, interest in third party testing
of communication protocols has been gradually increased
internationally. At present most test systems deal with the
network and transport protocols of the OSI standard.
Work is continuing on test facilities for the higher layers. A
successful tester should include an efficient test case
generator, therefore researchers are investigating method-
ologies for generating more effective test cases, in order to
achieve the goal of automatic testing protocols with very
little or no human interference.
SURVEY OF TESTING S Y S T E M S
UK's well known NPL assessment centre, will be given
below.
NPL assessment centre
The National Physical Laboratory (NPL) in the UK started
to develop appropriate testing techniques for protocol
implementation assessment in 19801, and is considered
one of the pioneers in the field of protocol testing. The
main objective of the NPL is to establish techniques
applicable to international standards. Their approach is to
test one layer at a time (layer N) by checking its
conformance to the ISO/OSI Reference Model, and all
layers below the layer under test (layer N-l, N-2, etc.)
are assumed to have been adequately tested. Early work
in the field has provided a basis for the development of
many protocols, and concentrates on the requirements
for testing OSI products by using a real network
connection 2.
The simple physical architecture of the assessment
centre consists of three components:
AT • NET ~ SUT
The AT (active tester) communicates with the SUT
(system under test) via a communication medium (NET).
At the N PL, British Telecom's Packet SwitchStream (PSS) is
used as the primary communication medium. The test
can be conducted many miles away from the system
under test. When the (N-l) service is not end-to-end, an
advanced physical architecture will be used. This
architecture inserts a transportable box called the EMU
(environment manipulation and monitoring unit) between
the communications medium and the client's system.
The AT is the key part of the assessment centre and has
been developed in two phases: the first adopts the simple
physical architecture and is restricted to the types of errors
that the SUT can be subjected to. The second uses the
advanced physical architecture which includes the test
driver, communication medium and EMU, and is closer to
the real structure of the corresponding OSI service and
protocols.
The logical architecture of assessment testing can be
described in terms of the OSI layers as indicated in Figure 1.
-'=--- Active tester Client's system
[ Test driver [= Test driver _I I
Responder protocol - I Test responder
I
(N) protocol+err J_
encoder/decoder r
(N)- protocol
Errors
_J
-J
I
Implementotion
under test (TUT)

In recent years, several protocol testing systems have

I_ (N-I) protocol I
been designed world-wide, specifically in the UK, USA, (N- I ) protocol I-- =I (N- I ) protocol
Canada, France, Italy and FRG. Most of these have already implementotion ] I implementation
exception
been put into operation, and all test systems are built for generator
particular layers of the OSI Reference Model. Underlying communicotion channel
A brief introduction to test systems and tools, as well as
comparing their major characteristics with those of the Figure 1. Logical structure of assessment centre

80 computer communications
 In its initial model, this testing system was built for
testing protocols for the network layer of the OSI model.
J Datosink J J Scenariofile I
Later on, however, it proved to be satisfactory for testing
protocols above the network layer when the underlying
service was expected to be end-to-end. The NPL is I o g ~ I Operator
COnSOle
working on adapting the active tester from layer-by-layer /
testing to multilayer testing, which will not consider
intermediate interfaces, so that the testing process will Test centre
become more manageable. Now NPL is takingthe lead in driver(TC) Datagenerator
developing an OSI standard for an OSI conformance
~ 1
testing methodology and framework.
Exception
generator T r
o nsport
protocol
implementation
NCC service
At the National Computing Centre (NCC) in the UK, a
_ I Network I
third party testing service has been built under the name --i interface I
of NCC Comms-AID (Communication Software Assess-
ment and Interactive Development) 3. 1
Networkprotocol
The NCC objectives are2:
Figure 2. NBS tester
• to establish an effective resource to assist the OSI user
community in the UK;
• to provide test facilities for as many protocols as valid events at the service interface or from the peer
practicable; protocol; that it can reject errors passed across the service
• to provide a better indication of commercial feasibility; interface and peer protocol errors, and that it can handle
• to test and develop further techniques; timeouts or multiplexed connections correctly as well 4.
• and to promote the OSI model and the concept of Although this system has been successfully used in
third party testing. transport layer protocol testing, the NBS considers that it
is unreasonable to construct one testing system for each
The NPL and NBS test systems (see below), which are
protocol. Consequently, an adaptation of this system has
separately used in testing network and transport layer
been planned for use in testing other layer protocols of
protocols were both installed at the NCC in 1983. The the OSI model.
NCC Com ms-AID service is a remote testing facility which
includes a tester located at the NCC site, the product
under test is installed on the client's own system, and the Two Canadian testing systems
connection of both sites is made via PSS. The tester can
generate test data and interpret responses as well. The Bochmann's group from the University of Montreal have
significant feature of this system is that the test can be developed some systematic methods to generate test
controlled either at the NCC or at the client's site. sequences for communication protocols, particularly for
NCC test facilities are already available for testing the transport layer protocols s. They have designed a simple
network layer and most parts of the transport layer. Higher and general test architecture as an experimental bed (see
layer testing facilities are under development. The NCC Figure 3) to try their methods for generating test cases. The
will use their system to provide a commercial testing test bed consists of an active tester, connected to a test
service. responder.
At the BNR laboratory in Ottawa, an integrated test
centre has been built for testing SL-10 (a trade mark of
NBS testing system Northern Telecom) packet network products. This centre
provides four test tools which make the network testing
The National Bureau of Standards (N BS) in the USA started more manageable for the implementer, manufacturer
to build a certification centre in 19824. The initial purpose and network operator, and also automatically allows
was to meet the requirements of testing their own
protocol implementation conformance to Federal
Information Processing Standards. Activetester L_ Testprotocol - [ Testresponder
The architecture of the tester was designed for the
transport layer of the OSI standard as shown in Figure 2. Peerunit I !I Unit undertest
Both tester and client are connected via the network
protocol during testing. The major difference from NPL's Network protocol I-- --j Network protocol

tester is that it does not require the test driver-responder

protocol. I Communication link
I
The aims of this testing centre are to ensure that the
implementation under test can respond correctly to all Figure 3. A test bed structure

vol 10 no 2 april 1987 81

terminals and host computers as well as total networks to Certificotion
centre (STQ) Vendor test
be tested for conformance and performance 6.
The function of these four tools can be simply I
Test driver ~.~ J Test protocol Test responder
described as follows: IPT (Interactive Protocol Tester) is
for checking protocol implementation against specification;
NLTS (Network Load Test System) generates simulated
'
N reference I N protocol N entity under
test I
entitt = I (N-I) service
traffic and measures the network's performance; NPM
(Network Process Monitor) examines operating software Public network
at the module level; and NTS (Network Test System)
provides an automatic or interactive test sequence for Figure 4. STQ test system
coordinating and controlling distributed network testing.
Each tool can be operated individually by using a TC SUT
standard data terminal. IPT, NLTS and NPM tools run on
SL-10 network processors and can be operated remotely
via the network or the NTS Driver. By using these tools in
Ts 1
scenorio (N-I) service
AR
(N)EUT
combination, a fully integrated and automatic operational
test centre can be achieved. CT
CUT

Figure 5. STP test system; TC = test centre, TS = testing

French test systems
France has been the most prolific source of development
of testing systems 2. Before 1983, there was the ADI group
which worked on the RHIN project. This project developed
three test systems7 for protocol implementation, namely
CERBERE, GENEPI, and STQ. Since 1984 another group
called BULL has been working on building an OSI
protocol testing environment based on a distributed
testing architecture (a remote testing system and a related
responder system) called STP8-11.
CERBERE is a tool designed to be introduced between
two pieces of equipment (implementation under test and
reference implementation) running high level protocols.
It acts as a relay in Layer 3 of the OSl model, and is a
portable tester that is connected between the subnetwork
and the system under test. Concurrently, it is able to
monitor, analyse or perturb the traffic between that
system and the tester.
GENEPI is a protocol data unit generator which was
initially used for testing Layers 4 and 5 of the OSI model,
and it can now be configured to test a single layer or a pair
of adjacent layers. It takes the same position in the
architecture as the encoder/decoder.
STQ (Test and Qualification System) consists of
software designed to run on a certification centre for
testing an (N)-layer protocol.
The main characteristic of STQ is that it allows testing of
Layers 4, 5 and 6 of the OSl model separately or
simultaneously, and also it assumes that there is testing
access directly to the lower layer interface of the
implementation under test. The architecture is shown in
Figure 4 and the system is available for testing Layer 4 of
the OSI model.
Finally, STP (System for Testing Protocols) is a system in
which the defined responder is simple, small and easy to
implement. This system allows an efficient methodology
for testing OSl protocol entities. The general architecture
of STP is shown in Figure 5.
The TS of STP system constructs (N)-protocol data
units to be sent to the (N) EUT (entity under test). The
system, SUT = system under test, AR = astride responder,
(N) EUT = N entity under test, CT = connection for
testing, CUT = connection under test
testing system (TS) is different from most other test
systems (but see also the NBS system in Figure 2) because
it employs a 'scenario' which basically generates test
cases, rather than by using a reference implementation.
PDUs are produced by the tester, and the (N-l) service is
used directly to exchange the (N) PDU with (N) EUT.
Another difference is that the (N) EUT allows messages
from the TS to gain access to either the upper interface
through the astride responder (AR) or the lower interface.
On the contrary, most other testing systems allow the
message from the active tester to enter only one
interface.
The STP system has been implemented and is used for
interactive testing of a transport entity.
CREI system
The CREI group in Italy started their research project to
design a testing system in 198312. The system is intended
to be used to test protocol implementations in terms of
both protocol and service testing. The basic architecture
of the system is as shown in Figure 6.
Test driver ~ Testingprotocol .~[ Test responder
(TD) (TR)
__I I
implementation under test
(PrR) under test (IUT)
1 L
F Underlying Ioyer
]
Figure 6. CREI test system

82 computer communications
 A special feature of this system is that its user can be the
implementer of the object under test, or the operator of
the system without making changes in the system
structure. The implementer using the system can work
locally to debug the implementation under test on line
and to perform the test with a remote reference
implementation (PIR). When the test centre operator uses
the system, a remote customer implementation (IUT) is
tested against the local reference implementation (PIR).
This system allows the TD (tester driver) to transfer
information about the service primitives to the TR (tester
responder) before the testing session starts. Therefore the
TR is required to have the memory and an interpreter for
storing and performing the TD instructions.
The major functions performed by the TD and TR are:
notification of the testing session opening and closing;
data needed for the synchronization of the TD and TR
behaviour; and indication of any failure 12.
The TD consists of five modules, namely the database,
operator-interface, control, test processing, and mapping
module. The TR consists only of the test processing and
mapping modules. All modules implemented depend on
the testing layer.
It is proposed that the system should be able to test
any layer of the OSI architecture, or a sublayer, or even the
combination of several layers of the OSI.
G M D system
At GMD of FRG, a special protocol tester for testing and
diagnosis of higher level protocols has been developed 13.
The structure of the tester is similar to Figure 4 (the French
STQ system).
The proposed testing techniques assume that tests are
driven under remote control by the client. At the
beginning, the strategy of manually driven tests was used,
and included the following: no test driver responder
protocol; no constraints for test responders; and a test
driver manually controlled by the client by using test
commands. Later, the strategy of automatic tests was
developed, based on common agreement of test protocols,
definition of test responders and an active test driver
which reads the test commands from a file.
AT GMD, the active tester can be accessed via X.25 or
X.28 and can be called as a test partner from the client's
system. It can also be inserted into the connection
between two partners for the purpose of arbitration
testing (which means to determine the cause of a
problem) and allows the testing of several parallel test
connections.
The GMD project is one of the collaborative projects
funded by the CEC (Commission of the European
Communities).
Remarks
As shown above, test systems are commonly constructed
by using an active tester connected to a test responder for
testing OSl or similar products. A protocol product under
vol 10 no 2 april 1987
test is located at the responder site and a reference
product is set at the active tester. All test systems should
have a mechanism for generating test cases so that the
operator does not need to input the test data manually.
Most systems provided by third parties are designed to
operate on the active tester's local site and to look on test
responders as customers. Only a few, like GMD and CREI,
allow the test responder to call the tester and can be
operated at both sites.
Current standardization work by ISO and CCITT is
contributing in the following areas: first, the development
of a portable test system for the OSI File Transfer Access
and Management (FTAM) (Layer 7) protocol; second,
automatic generation of test cases from formal protocol
specifications; and third, the development of standards
for conformance testing and formal protocol description
techniques.
When building a test system, designing the test case
generator is a vitally important part of the work. Therefore,
research to find better methods for generating test cases is
very valuable.
M E T H O D O L O G I E S IN PROTOCOL TESTING
How can a system be produced with effective test cases
and how can these be automatically generated without
looking at the details of the implementation? These
problems have attracted researchers to concentrate their
attention on test case design methodologies. As a result of
this, a number of techniques for generating test cases
have recently become available. For instance, Sarikaya
and Bochmann have found that various methods for
testing state machines can be applied to the selection of
test sequences for protocols specified as a finite state
machine s. They have applied three methods: transition
tour, checking sequence and W-method, in testing a
transport layer protocol. Ural and Probert TM have also
adapted the method of context-free grammars to protocol
testing. Most recently, Sabnani and Dahbura 15 have
created a new method to produce unique input-output
sequences for protocol testing.
Transition tours
The method of transition tours, like the others, assumes
that the protocol to be tested is specified as a finite state
machine (FSM). This is one of the simplest methods.
Transition tour is the name of an input sequence of the
FSM, which starts with the initial state and will cover all the
transitions in the FSM state table at least once. It detects
all operation errors (errors in the output function), but it
does not detect all the transfer errors (errors in the next
state function).
As the test sequence is based on the traversal of the
FSM graph, the test should start from the initial state, go
through all possible transitions, and then come back to
the initial state.
83
 b,d
Figure Z A finite state machine example; numbers
represent states, letters represent inputs
In Reference 5 the transport protocol has been used to
illustrate this method. Here a simple finite state machine
(see Figure 7) will be used to show the transition tour
sequence.
Assuming that State I is the initial state, a transition tour
sequence may be:
Input a b b d a c
State 1 2 1 1 1 2
The sequence starts from and terminates at State I and
will direct the machine to go through all possible
paths.
This method was first demonstrated by Naito 16 in his
paper on software testing. Later, Sarikaya and Bochmann
applied it in testing a transport layer protocol implemen-
tation, and found that the method was applicable to
incompletely specified machines if the machines were
strongly connected, contrary to the original assumption
that the machine under test should be minimal, strongly
connected and fully specified. This method is generally
applicable to all protocols specified by a finite state
machine. The limitation is that the fault detection
capability finds operation errors but not transfer errors.
W-method
The W-method involves two sets of input sequences: one
is the W-set, the other is the P-set. The W-set is a
characteristic set of the minimal FSM, and consists of
input sequences that can distinguish between the
behaviours of every pair of states. The P-set consists of all
partial paths.
The W-method gives a set of test sequences formed by
the concatenation of the W-set and P-set. Each test
sequence starts with the initial state and returns to it again
afterwards. It is also guaranteed to detect any misbehaviour
of the machine.
We also use the example in Figure 7 to describe how to
form a P.W set. As the W-set sequence should distinguish
the behaviours of every two states, input 'b' should be the
W-set for this finite state machine. The P-set can be
obtained from the tree in Figure 8.
84
b c
Figure 8. Testing tree:
P= { { } , b , a , d , a . {b,c},a.c. {d,a,c,b,g} }
PW = {b,b.b,a.b,d.b,a. {b,c}.b,a.c. {d,a,c,b,g}.b}
The W-set is constructed using aspecial method 17, and
the P-set can be formed from atestingtree ~8,which shows
d c a c b a c g
3 3 1 2 3 1 2 3 1
every transition from State i to State j on each input.
However, the limitation of using this method is that it is
not certain that every finite state machine will have a W-
set sequence, especially if it is an incompletely specified
machine. Therefore, if intending to use this method, one
should first make sure that the defined machine has a W-
set sequence. When a machine does not have a W-set, a
procedure can take place to form a completely specified
machine which has a W-set, for example, adding an 'error'
state and declaring all unspecified transitions to lead to
this state.
Checking sequences
A checking sequence is a test sequence which consists of
three parts:
• Initial sequence
• State recognition sequence
• Transition checking sequence
Users of this method should ensure that the FSM under
test will have a 'distinguishing sequence' (DS). The DS is
similar to the W-set mentioned above. It is an input
sequence, on which, for each initial state, the machine
will produce a different output sequence. One can get a
DS by constructing a successor tree 19 for the FSM.
At the start of testing, one should first use an initial
sequence to bring the FSM to the initial state. Then a state
recognition sequence is used to show the response of
each state to the DS, followed by a transition checking
sequence to check all individual transitions in the state
machine.
Each transition from a certain state using input Xi (say)
is checked by applying the sequence Xi.DS. The checking
sequence is formed by the concatenation of all Xi.DS
computer communications
sequences. The problem is that when the number of
states is large it is difficult to find a DS in a specified
machine. Normally, a 'read state' input/output facility is
added to help obtain a DS.
Context-free grammars
Grammars are often used to define languages. The
method of context-free grammar which will be discussed
here uses an attributed grammar to build a program
language for specifying and generating test sequences
from the service or protocol specifications.
The use of an attributed grammar to generate test cases
in software testing was suggested by Duncan and
Hutchison 2°. Ural and Probert have adapted this method
from its use in general software testing, and have
developed a method to generate test sequences for
transport layer service and protocol testing TM. When the
protocol is specified by using a context-free grammar, it
can easily be tested using this method.
When using this method in testing protocols, it is
necessary, first, to understand correctly the functional
requirements of the communication protocol; second, an
attributed context-free grammar is constructed and test
sequences specified from a functional requirement; third,
a language defined by the grammar is used to specify a
generator for test sequences; fourth, the grammar is
implemented as a generator program; finally, the programs
are run in a controlled fashion and a set of representative
test sequences generated randomly, systematically, or
selectively.
Although using the test sequence generator can
automatically produce test sequences, the generator
design needs careful human insight. Therefore, this is
called a semiautomatic method.
UlO sequence generation
Sabnani and Dahbura is have introduced another tech-
nique for generating protocol tests by means of a novel
procedure which can generate test sequences. The
procedure is based on the assumption that the protocol is
specified as a minimal finite state machine. The key idea
in the procedure is first to compute a unique input-
output (UIO) sequence for each state in the protocol
specification, and then to generate a test sequence to visit
all states and state transitions by using the UIO sequences.
The testing covers all the state transitions and checks the
UIO sequences for each state. It can also detect the
problems that exist in the implementation, such as an
incorrect edge label, an incorrect header state or an
incorrect tail state.
The procedure for generating test sequences consists
of two main steps:
First compute a UIO sequence for each state. Computing
the UIO sequence includes:
• For each edge label, compute the list of edges with that
label (for later use):
vol 10 no 2 april 1987
• Compute all input/output sequences of length 1 for
each state;
• Check if these are unique; if yes, a UIO sequence for
the state has been found;
• If not, compute sequences of length 2 for the state
which does not have a UIO sequence so far;
• Continue to compute length I + 1, and check unique-
ness until the UIO for every state is found.
The second step is to generate a test sequence, as
follows:
(1) Generate a test to check whether each state is
reachable from the initial state and whether it has a
UIO sequence (by using the method above);
(2) Generate a test to check whether each edge has the
correct head state, the correct tail state, and the
correct label;
(3) Concatenate the tests from (1) and (2), and remove
redundancy in order to reduce the test sequence
length.
The format of the test sequences is the edge label
sequence:ll/01 12/02 13/03 . . . ; the label of an edge
contains the input part (I) and expected output part (O).
The test sequence is able to give a clear overview of the
input and output behaviour, and it is therefore very
helpful to carry out a result comparison with it. When one
takes the input part of the labels as a test input sequence
to do the protocol testing, a sequence of results is
obtained. If there is any difference between the actual
result and the output part of the test sequence, it is
obvious that the protocol under test is faulty.
This method has also been applied to the Alternating
Bit Protocol 15 and the Proway protocol 22.
Comparison
Except for the context-free grammar method which is
mainly used when the protocol is specified by a context-
free grammar, all others are based on the finite state
machine specification. Bochmann has carried out a
comparison on the first three methods mentioned above.
For fault detection capability, the checking sequence is
best, and the W-method is better than the transition tour.
For the length of sequence, the transition tour gives the
shortest test sequence because the test sequence is
directly formed by checking the FSM diagram, and does
not need to add any extra characteristic sequence. For the
W-method and checking sequence methods, the problem
is that sometimes it is difficult to find the DS or W-set for a
finite state machine module, resulting in much more work.
All these methods test only whether every path is
reachable or not, and do not refer to further test cases
chosen.
The UIO sequence technique is an advance on the
others in that it is an approach to generating tests
automatically. The technique emphasises testing of each
state rather than testing the whole machine. Any level of
protocol which is specified by a FSM can be tested by this
technique. Those who do the testing need only input an
85
edge set which consists of head state, tail state and edge
label, together with the num ber of edges, and then a serial
test sequence will be generated for each state and each
edge. A great advantage of this technique is that it is
possible to make short test sequences because although
each state has a different UIO sequence, the majority of
UIO sequence lengths are one.
EXPERIENCE IN IMPLEMENTING A UlO
GENERATOR
As has already been described, the technique for
generating UIO test sequences is fairly straightforward
and the ideas involved are easy to understand. The range
of usage is so wide that any protocol specified by a finite
state machine can use it directly to generate the test
sequences unless it is not a minimal FSM. Nevertheless, it
is enough to add only a minimization process to those
exceptions. The authors have implemented this technique 22
and found that the overall test sequence generator
consists of four basic parts:
• The UIO sequence generator: generates a unique input
and output sequence for each state.
• The shortest path generator: generates a sequence to
find the shortest path for each state.
• Edge test sequence generator: generates a test
sequence for each edge, the sequence being formed
by the shortest path of the edge head state, plus the
edge label, plus the UIO sequence of the edge tail
state.
• State test sequence generator, generates a test sequence
for each state, and is formed by the shortest path of the
state, plus the UIO sequence of this state.
The unoptimized test sequence is the concatenation of
two state and edge test sequences. In order to remove
sequences that are completely contained in others and
thus make the test sequence shorter, another comparison
procedure is needed. This is a fifth part of the test
sequence generator.
The authors have applied the technique to generate
test sequences for testing Proway21 protocol implemen-
tations. The inputs were entirely based on the state machine
diagram of the protocol specification. After processing the
inputs, the generator produces a set of test sequences
automatically; these sequences are directly input to the
implementation under test. Therefore, test results were
easily obtained without doing any more work. When the
total results including the test sequences generated by
the procedure were obtained it was necessary to only
compare the outputs produced by the implementation
with the output part of the test sequence.
The authors' experience of implementing and using
this technique shows that it is simple and effective and
applicable for testing most protocols.
CONCLUSION
In this paper a survey of protocol testing systems has been
86
presented and some methods used in generating test
cases described.
Existing support test systems are all designed for
particular layers of the OSI model, especially for testing
network and transport layer protocols. These systems are
successfully used in fulfilling their original g o a l -
conformance testing for protocol standardization. With
such testing systems, standard protocol products will
have a large market and data communication between
different devices will be much easier than before.
Finding a new and simple method for generating test
cases is very important in the protocol testing area. As the
specification of a protocol is usually described formally in
a standards document, designing a test case generator will
not generally be too difficult. In this paper, the five
methods introduced can all be used to generate test cases
for communication protocols, and most of these aim at
automatic testing. Once the active testers are armed with
automatic test case generators, automatic testing can be
fully realized.
ACKNOWLEDGEMENTS
The authors are grateful to the two referees for their
helpful and detailed criticism of the original version of this
paper.
REFERENCES
1 Rayner, D 'A system for testing protocol implemen-
tations' in Sunshine, C (ed) Protocol specification
testing and verification North-Holland (1982)
2 Davidson, I 'Testing conformance to OSI standards'
Comput. Commun. Vol 8 No 4 (August 1985)
pp 170-1 79
3 Davidson, I 'Independent testing of protocols' Proc.
Networks '84 (June 1984)
4 Nightingale, J S 'Protocol testing using a reference
implementation' in Sunshine, C (ed) Protocol speci-
fication testing and verification North-Holland (1982)
5 Sarikaya, B and Bochmann, G V 'Some experience
with test sequence generation for protocols' in
Sunshine, C (ed) Protocol specification testing and
verification North-Holland (1982)
6 Hornbeek, M W A 'An integrated test centre for SL-10
packet networks' ACM CompuL Commun. Rev. Vol
15 No 4 (September 1985)
7 Ansart, J P 'A protocol independent system for
testing protocol implementation' in Sunshine, C (ed)
Protocol specification testing and verification
North-Holland (1982)
8 Rafiq, O and Haddad, J 'Description of protocol
testing scenarios' COMNET '85 (Conference on
'Services conveyed by computer networks' Budapest,
Hungary (1-4 October 1985)
9 Rafiq, O 'Tools and methodology for testing OSI
protocol entities' Int. Symp. Fault Tolerant Computing
IEEE, Ann Arbor, USA (19-21 June 1985)
computer communications
 - 0
10 Rafiq, O 'A good approach for testing protocol
implementations' Int Semin. CompuL Network. Perf.
EvaL Tokyo, Japan (18-20 September 1985)
11 Rafiq, O et al. 'Towards an environment for test OSI
protocol' Fifth International Workshop on Protocol
Specification, Testing and Verification Toulouse-
Moissac, France (10-13 June 1985)
12 Palazzo, Set al. 'A layer independent architecture for
a testing system of Protocol Implementations' in
Rudin, H and West, C H (eds) Protocol specification
testing and verification Elsevier Science (1983)
13 Giebler, A 'Testing and diagnosis aids for higher level
protocols' in Rudin, H and West, C H (eds) Protocol
specification testing and verification Elsevier (1983)
14 Ural, H and Probert, R 'User-Guided Test Sequence
Generation' in Rudin, H and West, C H (eds)
Protocol specification testing and verification Elsevier
(1983)
15 Sabnani, K and Dahbura, A 'A new technique for
generating protocol tests' ACM Comput. Commun.
vol 10 no 2 april 1987
- 0 0 0 0
Rev. Vol 15 No 4 (September 1985)
16 Naito, S and Tsunoyama, M 'Fault detection for
sequential machines by transition tours' Proc. IEEE
Fault Tolerant Computing Conf. (1982)
17 Gill, A Introduction to the theory of finite state
machine McGraw Hill, USA (1962)
18 Chow, T S 'Testing Software Design Modelled by
Finite State Machines' IEEE Trans. on Software
Engineering Vol SE-4 No 3 (May 1978)
19 Kohavi, Z Switching and finite automata theory
McGraw Hill, USA (1978)
20 Duncan, A G and Hutchison, J S 'Using attributed
grammars to test designs and implementations' Proc.
4th Int. Conf. Software Engineering (1981)
21 British Standards Instilution Draft--Process data
highway, type C, for distributed process control
system (1985)
22 Wang, B 'Implementation and testing of computer
network protocols' M. Phil Thesis, University of
Lancaster, UK (1986)
87