Specifying Operational Proles

for Modules
Denise M. Woit
Dept. of Computing and Information Science,
Queen's University,
Kingston, Ontario, Canada K7L 3N6
woit@qucis.queensu.ca

Abstract crucial that the operational prole specication

technique be general enough to allow realistic de-
We describe a technique for specifying opera- scriptions of the operating environment.
tional proles for modules. The technique is In this paper, we consider two problems: (1)
more general than those of the current literature how to accurately specify operational proles for
and allows more accurate specication of mod- modules, and (2) how to generate test cases from
ule usage. We also outline an algorithm for au- any such operational prole specication such
tomatically generating random test cases from that the test cases correspond to a random sam-
any such operational prole specication for a
module, such that the test cases correspond to
a random sampling of the module's input in ac-
tual operation. Operational-based statistical es-
timations, such as operational reliability, may be
pling of the module's inputs in actual operation.
In the literature, it is usually assumed that ac-
tual software usage can be described by assign-
ing unconditional probabilities to individual ele-
ments (or groups of elements) of the software's
more meaningful when our specication method input space [3, 9, 10, 11, 4, 1, 2, 7]. That is,
and generation algorithm are used, because our the operational prole is a probability distribu-
method permits more precise specications than tion that denes an unconditional probability of
do other methods in the current literature. occurrence for each element in the input space in
operational use. Although this operational pro-
1 Introduction
There is substantial literature detailing the vari-
ous ways of calculating statistical estimations for
software, based on the results of testing. These
calculations will produce operational-based sta-
tistical estimations, such as operational reliabil-
ity, when the test cases correspond to a random
le description is adequate in certain situations
(for instance, when the software is a batch pro-
gram), we do not believe that it necessarily will
be useful for modules, because in this case the
probability that an input is issued will likely de-
pend on certain factors, such as the previous in-
put, or the current state of the module.
Whittaker [12] has proposed an operational
sampling of the software's inputs in actual oper- prole specication method in which the proba-
ation; that is, when the test cases are selected bility that an input is issued is conditional upon
at random in accordance with a distribution of which input was issued immediately prior to it.
input values matching that which would occur He uses a Markov chain to describe the opera-
during actual software usage. We refer to such tional prole; the states of the chain are (groups
a distribution as an operational prole. The rel- of) elements of the input space. Although this
evance of estimations obtained in this way de- operational prole specication method is more
pends upon how accurately the operational pro- realistic than the unconditional probability dis-
le describes the software usage. Thus, it is tributions mentioned above, it is still more lim-
ited than we would like. We would like to be able
to succinctly specify cases in which the proba-
bility of issuing a module input depends on any
previous inputs, not simply the immediately pre-
vious input.
 Events:
An object's state can only be changed by
an event, which may be the external invo-
cation of an access program, or a change in
the value of an input variable. An access
If the operational prole is inaccurate, we can- program event description is denoted
not expect any statistical estimations (which are (
based upon it) to be meaningful. In this paper A if args(A) = 0
we present a means of more accurately specify- A(x1 ; x2 ; : : : xargs(A) ) if args(A) > 0,
ing operational proles for modules. Our tech-
nique is very general: in addition to being able where the xj represent the arguments of A.
to specify those operational proles of the cur- Each argument of an access program can be
rent literature (unconditional probability distri- classied as input, output, or both. We as-
butions and Markov chains,) it can be used to sume that in an access program event de-
specify proles having very intricate input depen- scription: the arguments classied as input
dencies. We present both a general overview and or both will represent actual values of the
a precise description of our operational prole appropriate type; and the arguments clas-
specication technique. We also present an algo- sied as output will represent variables of
rithm for automatically generating random test the appropriate type (which will be assigned
cases from any operational prole specication
for a module, such that the test cases constitute
a random sampling of the module's usage.
1.1 Terminology
In the following, we usually refer to objects by
values by the module). An access program
event description template is an access pro-
gram event description in which at least one
argument of type input or both represents a
variable.
An input variable event description is de-
their descriptors, for simplicity. For instance, the
event that is described by the string X will be
referred to as X .
 Module:
noted E (x1 ), where argument x1 (classied
as input) represents the new value of input
variable E . An input variable event descrip-
tion template is an input variable event de-
scription in which x1 represents a variable.
A module is dened as an information hiding
package of programs that implements ob-  Input Space:
jects of a particular type. Each object is The input space (denoted I ) of a module is
considered to be a nite state machine, the a set comprising all possible unique events
state of which is independent of other ob- as described above. (In determining unique-
jects of its type. The object communicates
with the outside world only by means of the
following mechanisms:
1. a set of input variables (IV) which the
object observes;
2. a set of access programs (APN), which
provide information to and/or receive
information from the object by means
of their arguments.
 args:
args(A) returns the number of arguments of
access program A.
ness of two events, the arguments classied
as output are ignored. Events that dier in
variable names only are considered identi-
cal.)
 Module Execution:
In order for the operational-based statisti-
cal estimations to be meaningful, it must
be the case that the module is re-initialized
from time to time; even real-time modules
that are intended to \run forever" can be
designed so that they are periodically re-
initialized, as suggested by Parnas et al. in
[5].
 We consider a module execution to refer to
the sequence of events issued to a mod-
ule, beginning with module initialization (or
re-initialization) and ending with the event
immediately prior to the next module re-
initialization. A module execution is de-
scribed by the sequence of event descrip-
tions, :E1 :E2 : : : : Eti , where \ " refers to
module initialization or re-initialization, Ej
is the j th event issued after this module ini-
tialization (or re-initialization), and Eti is
the last event issued before the next module
re-initialization (1  j  ti .)
 If there are no elements on the
stack, then some integer from [1
50] is pushed.
 If there are 1; 2; : : : 20 elements on
the stack, then there is an 80%
chance that some integer from [1
50] is pushed and a 10% chance
each that a pop or top is issued.
 If there are exactly 21 elements
on the stack then there is a 90%
chance that a pop is issued and a
10% chance that a top is issued.
 Execution Prex: An execution prex is
any subsequence of a module execution such Figure 1: Example usage
that the subsequence begins with . It is
described by the sequence of event descrip-
tions E1 :E2 : : : : Ej , 1  j  n (when j = 0 unbounded integer stack, and having access pro-
the prex is denoted \ ".) grams push(a), pop and top(a). Suppose module
usage is that outlined in Figure 1. In this case:
 Test Case: A test case is a mod- EP1  the event prexes resulting in an empty
ule execution; thus, it may be denoted stack; EP2  the event prexes resulting in from
:E1 :E2 : : : : Eti . 1 to 20 elements being on the stack; EP3  the
event prexes resulting in exactly 21 elements
2 Overview of Technique being on the stack; IC1  ftop(a)g; IC2 
fpopg; IC3  fpush(1), push(2), : : : push(50)g;
We believe that the most general means of spec- P1;1  0; P1;2  0; P1;3  1; P2;1  :1; P2;2  :1;
ifying operational proles is to allow the proba- P2;3  :8; P3;1  :1; P3;2  :9; P3;3  0.
bility of issuing an event to be conditional upon Note that any operational prole described as
the past input history (which can be represented a simple unconditional probability distribution,
by an execution prex.) We have selected in- P1 ; P2 ; : : : Pp , (as in the current literature) can
put histories as the \conditions" because they be described by our method by letting m 1,
are very general (they can describe any input letting n p, letting EP1  all event prexes,
dependency) and because they are user-intuitive letting P1;i Pi ; for 1  i  p, and letting
(the user interacts with the module by issuing ICj the input space partition corresponding
sequences of events.) to Pj . Also, any operational prole described as
In order to dene any operational prole (even a Markov chain [12] (which is described by a tran-
the ones assumed in the current literature), the sition matrix T of size t  t) can be described by
user must be able to partition the set of possible our method by letting m; n t, letting ICj
execution prexes into m categories, EPi ; (1  state j of the chain (1  j  t), letting EPi
i  m), and must be able to partition the in- the set of execution prexes that end in one of
put space into n categories, ICj ; (1  j  n), the events from state i of the chain (1  i  t),
such that for each EPi the user can determine and letting Pi;j Ti;j (1  i; j  t).
the probability of next issuing an event from each SmNotEPall: Ifexecution prexes are required in
ICj . We let Pi;j refer to the probability of issuing k=1 k P i;j = 0 then execution prex P:E
an event from ICj given that we have already is- is not required, 8P 2 EPi ; 8E 2 ICj . Also, if
sued a sequence of events from EPi . For example, execution prex A is not required, then any ex-
consider the case of a module implementing an ecution prex of the form A:E is not required,
8E 2 I . For instance, in the example usage are comments only. The comments in the
above (Figure 1), any execution prexes begin- last 3 rows of the table explicitly describe
ning with pop or top are not required because EP1 ; EP2 ; EP3 .
P1;1 and P1;2 are 0.
2. The variable \depth" counts the number of
elements on the stack; it is declared and ini-
2.1 Usage Considerations tialized in a special sub-program, init, and
The syntax of our operational prole specica- modied and referenced in the transition
tion technique is aected by the following objec- functions.
tives: it must be as simple as possible to use, 3. The labels on columns 2-4 of the table
and it must be tractable for automation (speci- describe IC1 ; IC2 ; and IC3 , respectively.
cation must be machine and (easily) human read- IC1 = ftop(i)g; IC2 = fpopg; and IC3 =
able, and test case generation program must be fpush(1), push(2), : : : push(50)g.
ecient.)
The objectives above in
uence the way in 4. P1;1 = 0; P1;2 = 0; P1;3 = 1; P2;1 = :1;
which we can describe the EPi in our method. P2;2 = :1; P2;3 = :8; P3;1 = :1; P3;2 = :9;
Describing each EPi as a trace [6] or as a con- P3;3 = 0.
junction of predicates (explicit descriptions) is
generally not as desirable (in terms of the ob- 5. Only those execution prexes that are re-
jectives above) as describing them by their tran- quired are included in the table. For in-
sition functions (see Section 2.1.1.) When tran-
sition functions are used, an explicit description
of each EPi is not necessary because one can al-
ways be determined directly from the transition
functions and the IC1 ; IC2 ; : : : ICn .
2.1.1 Transition Functions
stance, any execution prexes beginning
with 22 pushes are not required because
P3;3 = 0.
6. The transition functions and the ICj com-
pletely characterize the EPi .
7. The set, X = f1; 2; : : : 50g, is referred to in
In the operational prole specication, the user the table and is described in the Sets section
must dene at most m  n transition functions, of the specication.
fi;j such that fi;j need only be dened if Pi;j 6= 0; 8. An example of (the initial portion of) a test
for 1  i  m and 1  j  n. For each P 2 EPi case that corresponds to this operational
and for each E 2 ICj , fi;j returns integer kP such prole specication is: . push(40). push(7).
that execution prex P:E 2 EPkP (note that it
is possible for fi;j to return dierent integers for
dierent elements of EPi .)
2.1.2 Example
Before describing a precise syntax and seman-
pop. push(26). push(2). top(i). push(50).
The issue of test case termination is dis-
cussed in Section 3.
3 Detailed Syntax and Seman-
tics for our operational prole specication tech- tics
nique, it will be helpful to present an exam-
ple. The sample usage, described in Figure 1, In this section we present a detailed description
for the module implementing an unbounded in- of our operational prole specication technique.
teger stack is precisely specied in Figure 2 using This is accomplished by describing the syntax
our technique (some details have been altered for and semantics of the technique in a top-down
comprehension.) Points to note are: fashion.
We consider the syntax of our operational pro-
1. The column of the table labeled \HIS- le specication technique to refer to those prop-
TORY CLASSES" is optional; its entries erties that determine the appearance of the oper-
TABLE:
type input or both in the ith access program
in APN . Possibly a sub-program, init for
HISTORY INPUT CLASSES initialization purposes.
CLASSES Top(i) Pop Push( )::a in X
Sets Section: Descriptions of any sets referred
< a >

depth= 0 0 0 1
depth= 1 20 .1
; :::; .1 .8 to in the Table Section.
depth= 21 .1 .9 0
The Table and Functions sections are manda-
FUNCTIONS: tory. The condition functions, the init sub-
program and the Sets Section may not be re-
init: integer depth = 0 quired in every specication (see Sections 3.2 and
f13():
f21():
depth=1; return 2
return 2
3.3.)
f22(): depth=depth-1
if (depth=0) return 1 else return 2 Specication Semantics:
f23(): if (depth<20) {depth=depth+1; return 2}
elseif (depth=20) {return 3} An operational prole specication, S , described
f31(): return 3 using our technique, corresponds to a set of test
f32(): return 2
cases, TS . Given some specication, S 1, and test
SETS: case, t, of length tn , t is in TS 1 i for all execution
prexes, E1 :E2 : : : : Ei , of t, E1 :E2 : : : : Ei 1 is in
X={1,2,...50} some EPj , say EPx , of S 1, Ei is in some ICk ,
say ICy , of S 1, and Px;y is non-zero in S 1 (1 
Figure 2: Example operational prole i  tn).
3.2 Table Section:
ational prole specication; we consider the se-
mantics of our technique to refer to additional Table Syntax:
properties that determine the test cases that are The table is derived by modifying the template
described by the operational prole specication. appearing in Figure 3; the user modies the tem-
The syntactic information is necessary to deter-
mine the form of the event descriptions that com-
prise test sequences. The semantic information is
needed to determine where the event descriptions
may appear in the test sequences, and sequence
length.
plate by lling in the cells labeled ICj and Pi,j
(1  i  m and 1  j  n) with input class
descriptions and probability values, respectively
(described below.) Column 1 of the table is con-
sidered to be the column labeled IC1. The col-
umn labeled HISTORY CLASSES can be omit-
ted: if not omitted, its entries are user comments
3.1 The Specication in General (the comment in row j+1 of this column should
Specication Syntax: explicitly describe EPj , for the user's reference,
1  j  m.)
An operational prole specication is a document
that may consist of the following three parts:
Table Section: A table having m + 1 rows, n
columns and an optional comment column.
Functions Section: At most (n  m) transition
functions, fi;j , where 1  i  m and 1 
j  n: At most f condition functions
P APN, kFak ,,
where 1  k  f , and f = v + ki=1 i one argument of type input or both represents a
where v is the number of module input vari-
ables and ai is the number of arguments of
Input Class Descriptions: For simplicity, we
refer to the input class description in the cell of
the table labeled ICi as ICi , for 1  i  n. ICi
is of the form ECi :: Ci . The \ECi " is an ac-
cess program or an input variable event descrip-
tion (all arguments of type input or both represent
values) or an event description template (at least
variable.) When an argument of type input or
both represents a variable, the argument must be
 described in the Functions Section of the spec-
HISTORY INPUT CLASSES ication. The semantics of such functions are
CLASSES IC1 IC2


ICn described in Section 3.3. If ECj is an event de-
P1,1 P1,2


P1,n scription it is said to belong to ICj . If ECj is an
P2,1 P2,2


P2,n event description template, then the event de-
.. .. .. .. scriptions that can be obtained by replacing the
. . . .
Pm,1 Pm,2


Pm,n < v1 >; < v2 >; : : : < vvn > in ECj with values
selected according to the A1 ; A2 ; : : : Avn , respec-
Figure 3: Template for table tively, of Cj are said to belong to ICj . The belong
to relation is denoted 2.
Let ICGi be the set of event descriptions that
enclosed in angle brackets, i.e., <v>. Arguments belong to input class ICi . It must be the case
of type output must be represented by variables, that the ICi are formed such thatSthe ICGi par-
and are not enclosed in angle brackets. Argu- tition the input space; i.e., I  ni=1 ICGi and
ments must be separated by commas. ICGj \ ICGk =
6 for 1  i; j; k  n and j 6= k.
If ECi is an event description then \Ci " and Thus, the columns of the table can be seen as
\::" are omitted from the input class descrip- representing a partitioning of the input space.
tion. If ECi is an event description template,
then assume there are vn variable arguments of Probability Values: We assume that given
type input or both in ECi , and assume, without any execution prex, the user can determine the
loss of generality, that these variable names are probability that event E will be issued next,
v1 ; v2; : : : vvn , and that vj appears before vj +1 in 8E 2 I . Let the set of required, unique execu-
ECi for 1  j < vn. Then Ci is of the form tion prexes be referred to as MP . As before, we
A1 ^ A2 ^ : : : ^ Avn (a conjunction of condition assume the user partitions MP into S m groups,
m
EP1 ; EP2 ; : : : EPm ; i.e., MP  j =1 EPj and
components), where Aj (1  i  vn) may have
one of the following two forms: (1) \vj in S", EPk \ EPl =
6 for 1  j; k; l  m and k 6= l.
where S is a set that is dened in the Sets Sec- The EPi must be formed such that 8X 2 EPi
tion of the specication; (2) \vj = F", where F and 8E 2 ICj , the probability that, in actual
is a syntactically correct call of some condition operation, the user expects to issue event, E ,
function, F, that is dened in the Functions Sec- given that s/he has already issued the sequence of
tion of the specication.
Probability Values: A probability value is a
real number in [0,1]. For simplicity, we refer to
the probability value in the cell labeled Pi,j as
Pi;j .
events represented by the execution prex X , is
Pi;j . Because the ICPj partition the input space,
it is required that nj=1 Pi;j = 1 for each i in
[1; m].
We assume the EPi correspond to the rows of
the table; i.e., row (i+1) of the table corresponds
to EPi for 1  i  m. Thus, P(next event will
Table Semantics: belong to ICj j current execution prex belongs
to EPi ) = Pi;j in the specication table. By con-
Input Class Descriptions: The condition vention, 2 EP1 ; i.e., EP1 contains the module
components (the Aj ) of some condition, Ci , can prex corresponding to module initialization or
be of one of two forms, as described in Section re-initialization before the issuance of any events.
3.2: vi in S or vi = F . The rst form indicates As noted in Section 2.1, the EPi need not be
that a value for variable argument vi is selected described explicitly in the table; instead, they
according to the uniform distribution on the val- can be characterized by a set of transition func-
ues in set S , which is described in the Sets Sec- tions, fi;j , dened in the Functions Section of the
tion of the specication. The second form in- specication (1  i  m; 1  j  n). The tran-
dicates that a value for variable argument vi is sition functions characterize the EPi by deter-
calculated by the condition function, F, which is mining which EPi execution prex E1 :E2 : : : : Ej
belongs to, given knowledge of which EPi execu- init Sub-program: The user employs sub-
tion prex E1 :E2 : : : : Ej 1 belongs to and certain
other information about E1 :E2 : : : : Ej 1, such as
the value of j , or the values of certain arguments
in the event descriptions in E1 :E2 : : : : Ej 1. Such
other information is user-determined, and main-
tained in global data structures which the user
program init to declare and initialize all global
data structures needed in the transition and con-
dition functions.
Condition Functions: Any functions used in
any conditions in any input class descriptions
must declare and initialize in sub-program init. must be dened in the Functions Section of the
The values of these data structures are updated
and referenced in the transition functions, which
are described by the user.
3.3 Functions Section:
Functions Syntax:
Transition Functions: For each non-zero Pi;j
specications. As described in Section 3.2, one
such function, F, corresponds to a particular vari-
able argument, < vi >, in a particular input class
description, ICj . F may modify and/or reference
any data structures that the user has declared
and initialized in sub-program init. F returns a
string representing a value which is selected from
the possible values for vi (which are determined
in the table, the user must dene a function, fi;j by the user) according to some distribution on
in the functions section of the specication. fi;j
must be a syntactically correct, zero-argument,
C function, Fi-j(), of type integer.
init sub-program: Sub-program init is a
the possible values (also determined by the user),
or which is calculated as a function of the values
of some data structure that is maintained by the
user (function determined by the user.) For ex-
ample, ICj could be \push(<a>) :: a in S", or it
syntactically correct, zero-argument, C sub-
program, init().
Condition Functions: Those functions ap-
pearing in conditions in the ICi s must also be
could be \push(<a>) :: a = F". In the rst case,
a value for variable \a" is automatically selected
according to the uniform distribution on the ele-
ments of S; in the second case, it is obtained by
calling function F.
dened in this section; they are syntactically cor-
rect, zero-argument, C functions, whose names
are selected by the user.
Functions Semantics:
3.4 Sets Section:
Sets Syntax:
The Sets Section is composed of s subsections,
where s is the number of unique sets referred to
Transition Functions: Fi-j() returns one of in the table section of the specication. Without
the integers 1; 2; : : : m. 8P 2 EPi and 8E 2 ICj , loss of generality, let the unique sets referred to
Fi-j() returns k such that P:E 2 EPk . If the re- be named S1 ; S2 ; : : : Ss . Then section i of the Sets
turn value of Fi-j() diers for dierent execution Section is a syntactically valid zero-argument, C
prexes in EPi then the user is responsible for sub-program, SET-Si().
dierentiating between them in Fi-j(). The user
does this by using global data structures which Sets Semantics:
are declared and initialized in sub-program init
and which are modied and accessed in the tran-
sition functions. In the simplest case, 8P 2 EPi ,
and 8E 2 ICj , P:E 2 EPk ; i.e., Fi-j() simply re-
turns the integer k without referencing or modi-
fying any data structures.
A special, global, boolean variable, END (de-
scribed in Section 4), is available to the user for
use in the transition functions.
The purpose of sub-program SET-Si() is to de-
clare and initialize an array, A-Si [1..ni ], where ni
(determined by the user) is the number of unique
elements in the set named Si . The user must de-
termine an ordering of each Si ; in function SET-
Si (), the user must assign the j th element of Si to
array element A-Si [j], for 1  i  s; 1  j  ni .
The type of the array must be string.
 4 Test Case Generation Algo-
rithm
In Figure 4, we present a pseudo-code algo-
rithm for automatically generating random test
cases according to any operational prole speci-
ed with the technique described above.
In the algorithm, we assume the following pa-
rameters: NUM-TC: an integer indicating the
number of test sequences to be generated; ML:
an integer indicating the maximum length of any
test sequence; END: a boolean which is true only
Fi;j 0, i 2 [1; m]; j 2 [1; n] when the user considers the current test sequence
for k = 1 to NUM-TC do long enough, even though test case length is less
init than ML (test case length can vary, since length
TC of module executions can vary); Fi;j : an n by m
i 1 *assume 2 EP1 * integer array that is used to count the frequen-
j SelectColumn(i) cies of events in the test sequences (required to
E SelectEvent(j) determine if the test sequences generated by the
Fi;j Fi;j + 1 algorithm are statistically typical of the opera-
TC TC.E tional prole specication{this topic is not ad-
length 1 dressed in this paper); TC: a sequence of event
while (length < ML) & not(END) descriptions that is the test sequence currently
i fi;j being generated.
j SelectColumn(i) Two functions are used in the algorithm: Func-
E SelectEvent(j) tion SelectColumn(i) randomly selects one in-
Fi;j Fi;j + 1 put class, ICj according to the probabilities
TC TC.E Pi;1; Pi;2 ; : : : Pi;n. Function SelectEvent(j) re-
length length +1 turns a string representing one event description
end-while 2 ICj . If ICj has no condition, then event de-
write TC to le TCASES scription ECj is returned. If ICj has a condi-
end-for tion, then the event description is formed from
write all Fi;j to le FREQUENCIES
Figure 4: Algorithm to generate test sequences
event description template ECj by replacing each
< vk > in ECj with a value (in the form of a
string) selected according to condition compo-
nent Ak (see Section 3.2.)
5 Conclusions and Future Work
We have presented a technique for specifying op-
erational proles for modules, and an algorithm
for automatically generating test cases accord-
ing to one such specication. Our technique will
allow more precise operational prole specica-
tions; thus, the resulting statistical estimations
may be more meaningful. We plan to apply our
specication method to real-life cases in order to
assess its practical potential. Related work in-
volves developing a method for calculating statis- 33(6):836{48, June, 1990.
tical estimations based on the results of executing Previous versions:
and verifying the test cases; the method should Technical Report 88-220, Dept. of Comp. &
take into account the operational prole. Future
work involves developing a software tool based
on our test case generation algorithm, and devel-
oping a tool to calculate statistical estimations
based on the results of executing and verifying
the test cases [13]. This work, along with the
work of others involving automatic test case ex-
Info. Sci., Queen's University, May 1988.
Proc. Intl. Working Group on Nuclear Power
Plant Control and Instrumentation, IAEA
NPPCS Specialists' Meeting, International
Atomic Energy Agency, London, United
Kingdom, 10-12 May 1988.
Proc. Seventh Intl. Conference on Testing
ecution and verication [8] will comprise a com- Computer Software, San Francisco, June 18-
plete, automated method of obtaining statistical 21, 1990, pp. 89-117.
estimations for software modules, based on test-
ing. [6] D.L. Parnas and Y. Wang. The trace asser-
Acknowledgements: I would like to thank
David Parnas and Roman Viveros-Aquilera of
McMaster University for their encouragement
and helpful discussions regarding the work in this
paper. Thanks also to James Whittaker, of Soft-
ware Engineering Technology, Inc., and Pierre-
Jacques Courtois, of the University of Louvain
and AIB Vincotte in Belgium, for their comments
on preliminary versions of this work.
References
[1] J.R. Brown and M. Lipow. Testing for soft-
ware reliability. In Proc. Intl. Conf. Reliable
Software (Los Angeles, CA), pages 518{527,
April 1975.
tion method of module interface specica-
tion. Technical report 89-261, Queen's Uni-
versity, 1989.
[7] T.A. Thayer, M. Lipow, and E.C. Nelson.
Software Reliability. North-Holland, New
York, 1978.
[8] Y. Wang and D.L. Parnas. Trace rewriting
systems. In Proceedings, Third International
Workshop on Conditional Term Rewrit-
ing Systems (Pont-a-Mousson, France, July
1992.), pages 1{21, 1992.
[9] S.N. Weiss. What to compare when com-
paring test data adequacy criteria. Software
Engineering Notes, 14(6):42{49, Oct., 1989.
[10] S.N. Weiss and E.J. Weyuker. A generalized
[2] R.C. Cheung. A user-oriented software reli-
ability model. IEEE Trans. Software Engi-
neering, SE-6(2):118{125, March 1980.
[3] J.W. Duran and S.C. Ntafos. An evaluation
of random testing. IEEE Trans. Software
Engineering, 10(4):438{444, July, 1984.
[4] K.W. Miller, L.J. Morell, L.E. Noonan, S.K.
Park, D.M. Nicol, B.W. Murrill, and J.M.
Voag. Estimating the probability of fail-
ure when testing reveals no failures. IEEE
Trans. Software Engineering, 18(1):33{42,
January 1992.
[5] D.L. Parnas, J. van Schouwen, and S.P.
Kwan. Evaluation standards for safety crit-
ical software. Communications of the ACM,
domain-based denition of software reliabil-
ity. In Proceedings, Workshop on Software
Testing (Ban, Canada, July 15-17, 1986),
pages 98{107, Washington, DC, 1986. IEEE
Computer Society Press.
[11] S.N. Weiss and E.J. Weyuker. An ex-
tended domain-based model of software reli-
ability. IEEE Trans. Software Engineering,
14(10):1512{1524, October 1988.
[12] James Whittaker. Markov chain techniques
for software testing and reliability analysis.
PhD dissertation, University of Tennessee,
May, 1992.
[13] D.M. Woit. Operational-based test case gen-
eration and reliability estimation for mod-
ules. Ph.D. Research Proposal, Queen's Uni-
versity, 1992.