You are on page 1of 17

A Guide to Simulation

Springer Science+Business Media, LLC


Paul Bratley
Bennett L. Fox
Linus E. Schrage

A Guide to Simulation
Second Edition

With 32 Illustrations

Springer
Paul Bratley Linus E. Schrage
Departement d'informatique et de Graduate School of Business
recherche operationnelle University of Chicago
Universite de Montreal Chicago, IL 60637
Montreal, P.Q. H3C 317 U.S.A.
Canada

Bennett L. Fox
Department of Mathematics
University of Colorado
Denver, CO 80217-3364
U.S.A.

Mathematics Subject Classifications: 90CXX, 65ClO, 68U20

Library of Congress Cataloging in Publication Data


Bratley, Paul.
A guide to simulation.
Bibliography: p.
Includes indexes.
1. Digital computer simulation. I. Fox.
Bennett L., 1938- II. Schrage, Linus E.
III. Title.
QA76.9.C65B73 1987 001.4'34 86-26287
ISBN 978-1-4612-6457-6 ISBN 978-1-4419-8724-2 (eBook)
DOI 10.1007/978-1-4419-8724-2

© 1987 by Springer Science+Business Media New York


Originally published by Springer-Verlag New York Inc. in 1987
Softcover reprint of the hardcover 2nd edition 1987
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+Business Media, LLC), except for brief
excerpts in connection with reviews or scholarly analysis. Use in connection with any form of
information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed is forbidden.

Typeset by Composition House Ltd., Salisbury, England.

98765 4 3 2

SPIN 10534734
To
Ronald and Margaret Bratley
Daniel and Kate Fox
Alma and William Schrage
Preface to the Second Edition

Changes and additions are sprinkled throughout. Among the significant new
features are :
• Markov-chain simulation (Sections 1.3, 2.6, 3.6, 4.3, 5.4.5, and 5.5);
• gradient estimation (Sections 1.6, 2.5, and 4.9);
• better handling of asynchronous observations (Sections 3.3 and 3.6);
• radically updated treatment of indirect estimation (Section 3.3);
• new section on standardized time series (Section 3.8);
• better way to generate random integers (Section 6.7.1) and fractions
(Appendix L, prog ram UNIFL);
• thirty-seven new problems plus improvements of old problems.
Helpful comments by Peter Glynn, Barry Nelson, Lee Schruben, and Pierre
Trudeau stimulated several changes. Our new random integer routine
extends ideas of Aarni Perko. Our new random fraction routine implements
Pierre L'Ecuyer's recommended composite generator and provides seeds to
produce disjoint streams. We thank Springer-Verlag and its late editor,
Walter Kaufmann-Bilhler, for inviting us to update the book for its second
edition. Working with them has been a pleasure. Denise St-Michel again
contributed invaluable text-editing assistance.
Preface to the First Edition

Simulat ion means driving a model of a system with suitable inputs and
observing the corresponding outputs. It is widely applied in engineering,
in business, and in the physical and social sciences. Simulation method-
ology draws on computer science, statistics, and operations research and
is now sufficiently developed and coherent to be called a discipline in its
own right. A course in simulation is an essential part of any operations re-
search or computer science program. A large fraction of applied work in
these fields involves simulation ; the techniques of simulation, as tools, are
as fundamental as those of linear programming or compiler construction,
for example. Simulation sometimes appears deceptively easy, but perusal
of this book will reveal unexpected depths. Many simulation studies are
statistically defective and many simulation programs are inefficient, We
hope that our book will help to remedy this situation. It is intended to teach
how to simulate effectively .
A simulation project has three crucial components, each of which must
always be tackled :

(1) data gathering, model building, and validation;


(2) statistical design and estimation;
(3) programming and implementation.

Generation of random numbers (Chapters 5 and 6) pervades simulation,


but unlike the three components above, random number generators need
not be constructed from scratch for each project. Usually random number
packages are available. That is one reason why the chapters on random
numbers, which contain mainly reference material, follow the chapters deal-
ing with experimental design and output analysis.
x Preface to the First Edition

A simulation technique is not really appreciated until a feeling for how


it might be implemented in practice is acquired . We therefore give worked
examples to illustrate the theory. Mastering modeling comes only by doing
it ; but to give inexperienced readers the proper perspective the book begins
with a detailed discussion of modeling principles. The specific models given
occasionally in the book are there only to illustrate technical points. They
are simplistic and are not to be taken seriously as reasonable representations
of real-world problems. On the other hand, we point out a common error in
simulation models : excessive detail.
A simulation course should attempt to cover only the substantial amount
of material that deals directly with simulation. Time does not allow it to
serve also as a vehicle for teaching probability, statistics, computer pro-
gramming, or data structures. For understanding most of this book one
course -in each of these topics should suffice. This knowledge should be
complemented by a certain mathematical maturity, such as is normally
acquired in a rigorous undergraduate course in advanced calculus or linear
algebra. If the instructor briefly reviews a few points in class, good students
can get by with fewer formal prerequisites. The rare allusions to more
advanced material can be skimmed without loss of continuity. Chapter 7,
on simulation programming, requires almost no mathematical background.
Only Section 1.7, a few problems in Section 1.9, and Chapter 7 require
computer science background beyond elementary programming to be fully
appreciated.
The ordering of the material in the book reflects what we consider to be
the most natural logical flow. The progression is not always from easy to
hard: Chapters 2 and 3 are probably the most difficult in the book and Sec-
tion 1.9 has some of the most challenging problems. Other orderings are
certainly feasible, as the Precedence Diagram given after the Contents
indicates. It shows only the most prominent relationships among the
chapters which are all to some extent interrelated. Course structure is flexible.
For example, an instructor can structure a computer science oriented
course based on this book at the last-year undergraduate level by focusing
mainly on Chapters 1, 5, 6, and 7. Sections 2.1.1, 3.1, 3.2, and 4.1-4.8 can
also be covered without difficulty in an undergraduate course that other-
wise deals with computer science aspects of simulation. To concentrate
on simulation programming languages cover Sections 1.1-1.5 and 1.8, and
Chapter 7. This should be complemented by a detailed study, with pro-
grammed examples, of at least one of the special-purpose languages we
discuss. Chapter 7 is not intended as a substitute for the reference manuals
of the languages compared there . Rather, we aim to teach critical apprecia-
tion of general principles. To fully appreciate our critique, the student
should at least have access to supplementary documentation; ideally, he
should have the facilities to try several languages .
Although such a course may perhaps be thought sufficient for the com-
puter science student or the programmer, it is not adequate for a designer
Preface to the First Edition Xl

with overall responsibility for a simulation project. In this wider context we


believe that the student should acquire proper understanding of the theory
before he rushes to get simulation programs running on a computer. The
ordering of the chapters follows this principle. A course based on this book
emphasizing statistical aspects of simulation can normally be expected to be at
the first-year graduate level. If the students already have some computer
science background, as we would expect, much of Chapters 1, 7, and 8 can
be left as reading assignments.
Most of the book can be covered in a one-semester course. The instructor
can flesh out a two-semester course by presenting case studies and having
the students formulate and validate models, write simulation programs,
and make statistical analyses. If there is time left over, a few papers from
the recent research literature can be discussed, or a particular simulation
language can be studied in detail. The entire book is also suitable for in-
dependent study and as a reference for practicing professionals with diverse
interests and backgrounds.
The problems are often an integral part of a development. Some test
comprehension. Others contain results of independent interest, occasion-
ally used later. They vary from almost trivial to quite hard. They are placed
where we think it will do the reader the most good . All should be read before
proceeding: try most.
Simulation is unique in the opportunities it offers to reduce the variances
of performance estimators by astute experimental design. The book explores
these opportunities thoroughly. We highlight the roles of mono tonicity and
synchronization in inducing correlation of appropriate sign for variance
reduction. One consequence is that we favor inversion as a way of generating
nonuniform random numbers. This lets us choose nonuniform random
number generators to discuss quite selectively among the mushrooming
number of papers on the subject. Many of these papers are cited but not
treated in detail.
Occasionally we take viewpoints not universally shared and emphasize
certain aspects that others slight. The central role that we give to synchron-
ization and our consequent concentration on inversion is one example of
this. Probably controversial in some quarters is our position that finite-
horizon models are usually appropriate, especially in a simulation context.
For such models, multiple replications form the basis for fairly straight-
forward estimation and statistical inference. Infinite-horizon models are
artifices that may occasionally be convenient fictions. Sometimes useful,
they are too often used . For them, no method of constructing confidence
intervals for unknown parameters that is in all respects both completely
rigorous and practical has been devised. This probably explains why so many
approaches have been discussed in the simulation literature. We cover the
main ones: batch means and regenerative, spectral , and autoregressive
methods. When pungency, not blandness, is called for our tone is sometimes
frankly tendentious.
Xli Preface to the First Edition

The book treats all aspects of discrete-event digital simulation in detail.


We discuss so-called continuous and hybrid simulation briefly in Sections
1.4.2. and 1.4.3. The Contents, a summary of notation, and a precedence
diagram follow this preface. References cited are listed in a separate section
near the end of the book, ordered by author and year of publication. To avoid
cluttering the text, longer programmed examples are banished to an appendix.
We acknowledge the help received from many colleagues and friends.
Among those whose suggestions, comments, and criticism of earlier ver-
sions of this book were particularly valuable, we thank Robert Cooper,
Eric Denardo, Luc Devroye , U. Dieter, Peter Glynn, Donald Iglehart,
Donald Knuth, Patrice Marcotte, Barry Nelson , Bruce Schmeiser, and
Ludo Van der Heyden . Edward Russell supplied the example program of
Figur e X.7.5. Kevin Cunn ingham wrote the programs to draw many of the
figures. We are particularly grateful to those who do not share our opinions
but freely gave their help anyway.
Den ise St-Michel nurtured our manuscript through endless revisions
on our local text-editing system. For her care and cheer, we are grateful.
We also thank Yves Courcelles, whose software made life much easier. Noah
Webster settled many an argument.
Our work was generou sly supported by the Natural Sciences and Engin-
eering Research Council of Canada.
Contents

Notation
xix
Precedence Diagram xxi

CHAPTER 1
Introduction I
1.1. Systems, Models, and Simulation I
1.2. Verification, Approximation, and Validation 8
1.2.1. Verifying a Program 9
1.2.2. Approximation and Validation 9
1.3. States, Events, and Clocks II
1.4. Simulation-Types and Examples 14
1.4.1. Synchronous and Asynchronous Discrete-Event Simulation 17
1.4.2. Continuous Simulation 18
1.4.3. Hybrid Simulation 24
1.5. Introduction to Random Numbers 25
1.6. Perspective on Experimental Design and Estimation 26
1.7. Clock Mechanisms 30
1.8. Hints for Simulation Programming 32
1.9. Miscellaneous Problems 34

CHAPTER 2
Variance Reduction 44
2.1. Common Random Numbers 48
2.1.1. Informal Approach 48
2.1.2. Formal Development 51
2.1.3. Auxiliary Results 56
2.2. Antithetic Variates 56
2.3. Control Variates 59
xiv Contents

2.4. Stratification 63
2.5. Importance Sampl ing 65
2.6. Conditional Monte Carlo 70
2.7. Jackknifing 74

CHAPTER 3
Output Analysis 77
3.1. Introduction 77
3.1.1. Finite-Horizon Versus Steady-State Performance 78
3.1.2. Fixed Sample Size Versus Sequential Sampling 81
3.2. Analysis of Finite-Horizon Performance 82
3.2.1. Absolute Performance Estimation 82
3.2.2. Relative Performance Estimation 84
3.3. Analysis of Steady-State Performance 85
3.3.1. Batch Means 90
3.3.2. Regenerative Methods 95
3.3.3. Spectral Analysis Methods 100
3.3.4. Autoregressive Methods 103
3.3.5. Recommendations 106
3.4. Analysis of Transaction-Based Performance 107
3.5. Indirect Estimation via r = A.S 108
3.6. Problems 109
3.7. Renewal Theory Primer 113
3.8. Standardized Time Series 120
3.8.1. Steady State 121
3.8.2. Transients 123

CHAPTER 4
Rational Choice of Input Distributions 124
4.1. Addition and the Normal Distribution 126
4.2. Multiplication and the Lognormal 128
4.3. Memorylessness and the Exponential 129
4.4. Superposition, the Poisson, and the Exponential 130
4.5. Minimization and the Weibull Distribution 130
4.6. A Mixed Empirical and Exponential Distribution 131
4.7. Extreme Values and Spacings 132
4.8. When Not to Use a Theoretical Distribution 133
4.9. Nonstationary Poisson Processes 135

CHAPTER 5
Nonuniform Random Numbers 145
5.1. Introduction 145
5.2. General Methods 146
5.2.1. Inversion 147
5.2.2. Tabular Approximation of the Inverse Transform 149
5.2.3. Empirical cdf's 150
5.2.4. A Mixed Empirical and Exponential Distribution 150
5.2.5. Rejection 151
Contents
xv

5.2.6. Generalized Rejection 152


5.2.7. Composition 156
5.2.8. The Alias Method for Discrete Distributions 158
5.2.9. Functional Approximations of the Inverse Tr an sform 159
5.2.10. Other Ingenious Method s 161
5.3. Continuous Distributions
164
5.3.1. The Non standard Normal Distr ibution 164
5.3.2. The Multivariate (Dependent) Normal Distribution 164
5.3.3. Symmetric Stable Variates 165
5.3.4. The Cauch y Distribution 167
5.3.5. The Lognormal Distribution 168
5.3.6. The Exponential Distribution 168
5.3.7. The Hyperexponential Distribution 169
5.3.8. The Laplace and Exponen tial Power Distr ibutions 170
5.3.9. Erlang and Gamma Distributions 171
5.3.10. The Beta Distribution 172
5.3.11. The Chi-square Distribution 174
5.3.12. The F-Distribution 175
5.3.13. The r-Distribution 175
5.3.14. The Weibull Distribution 176
5.3.15. The Gumbel Distribution 177
5.3.16. The Logistic Distribution 178
5.3.17. The Gener alized Lambd a Distribution 178
5.3.18. Nonhomogeneous Poisson Processes 178
5.4. Discrete Distributions 181
5.4.1. The Binomial Distribution 181
5.4.2. The Poisson Distribution 182
5.4.3. Compound Poisson Distr ibut ions 183
5.4.4. The Hypergeometric Distribution 183
5.4.5. The Geometric Distribution 184
5.4.6. The Negative Binomial and Pascal Distributions 185
5.4.7. Multid imensional Poisson Distributions 186
5.5. Problems 186
5.6. Timings 189

CHAPTER 6
Uniform Random Numbers 192
6.1. Random Introductory Remarks 192
6.2. What Constitutes Randomness 192
6.3. Classes of Generators 194
6.3.1. Random Devices 195
6.3.2. Table s 195
6.3.3. The Midsquare Method 195
6.3.4. Fibonacci and Additive Congruential Generators 196
6.3.5. Linear Congruential Generators 196
6.3.6. Linear Recursion mod 2 Generators 198
6.3.7. Combinations of Generators 203
6.4. Choos ing a Good Generator Based on Theoret ical Considerations 205
6.4.1. Serial Correlation of Linear Congruential Generators 205
XVI Contents

6.4.2. Cycle Length of Linear Congruential Generators 206


6.4.3. Cycle Length for Tausworthe Generators 208
6.4.4. The Spectral Test 208
6.5. Implementation of Uniform Random Number Generators 211
6.5.1. Multiplicative Generator With Modulus 2" 211
6.5.2. Multiplicat ive Generators With Prime Modulus 212
6.5.3. Implementing the Tausworthe Generator 215
6.6. Empirical Testing of Uniform Random Number Generators 216
6.6.1. Chi-square Tests 217
6.6.2. Kolmogorov-Smirnov Tests 218
6.6.3. Tests Specifically for Uniform Random Numbe r Sequences 219
6.7. Proper Use of a Uniform Random Number Generator 220
6.7.1. Generating Random Integers Uniform Over an Arbitrary Interval 220
6.7.2. Nonrandomness in the Low-order Bits of Multiplicative Generators 221
6.7.3. Linear Congruential Generators and the Box-Muller Method 223
6.8. Exploiting Special Features of Uniform Generators 224
6.8.1. Generating Ant ithetic Variates With a Multiplicative Congruential
Generator 224
6.8.2. Generating a Random Number Stream in Reverse Order 225
6.8.3. Generating Disjoint Sequences 226

CHAPTER 7
Simulation Programming 228
7.1. Simulation With General-Purpose Languages 229
7.1.1. The Simplest Possible Clock Mechanism 230
7.1.2. Generating Random Variates 232
7.1.3. Data Structures in Fortran 233
7.1.4. A Complete Simulation in Fortran 234
7.1.5. Fortran Simulation Packages 236
7.1.6. Our Standard Example-the Naive Approach 237
7.1.7. Simulation Using Pascal 241
7.2. Simscript 243
7.2.1. Data Structures in Simscript 244
7.2.2. Simscript and Simulation 245
7.2.3. A Complete Simulation in Simscript 247
7.2.4. The Standard Example in Simscript 249
7.2.5. Processes and Resources 250
7.2.6. Summing-up 250
7.3. GPSS 251
7.3.1. The Basic Concepts 251
7.3.2. Resources in GPSS 254
7.3.3. Generating Random Variates 255
7.3.4. A Complete Simulation in GPSS 257
7.3.5. The Standard Example in GPSS 260
7.3.6. Summing-up 261
7.4. Simula 263
7.4.1. The Class Concept in Simula 263
7.4.2. Simulation Using System Classes and Procedures 266
7.4.3. A Complete Simulation in Simula 269
Contents xvii

7.4.4 . Demos 271


7.4.5 . The Standard Example in Simula With Demos 274
7.4.6 . Summing-up 275
7.5. General Considerations in Simulation Programming 276
7.5.1. Language Design 276
7.5.2. System Considerations in Simulation 278
7.5.3. Statistical Considerations 279

CHAPTER 8
Programming to Reduce the Variance 281
8.1. Choosing an Input Distribution 282
8.1.1. Using an Exact Method 282
8.1.2. Inverting a Tabulated Distribution 282
8.1.3. Using a Mixed Empirical and Exponential Distribution 284
8.1.4. Testing Robustness 285
8.2. Common Random Numbers 287
8.3. Antithetic Variates 289
8.4. Control Variates 291
8.4.1. The Simple Approach 291
8.4.2. Regression With Splitting 293
8.4.3. Regression With Jackknifing 294
8.5. Strat ified Sampling 295
8.6. Importance Sampling 296
8.7. Conditional Monte Carlo 299
8.8. Summary 300

APPENDIX A
The Shapiro-Wilk Test for Normality 303

APPENDIX L
Routines for Random Number Generation 306

APPENDIX X
Examples of Simulation Programming 348

References 375

Author Index 389

Subject Index 393


Notation

The following notations and abbreviations are used without further expla-
nation throughout the book.

Section 1.2.3 : Chapter I, Section 2, Subsection 3.


Problem 1.2.3 : Problem number 3 in Chapter I, Section 2
(likewise for example, figure, lemma, proposition, table, and theorem).

log natural logarithm


log , base 2 logarithm
rxl ceiling of x = smallest integer ~ x
LxJ floor of x = largest integer ::;; x
g(n) = O(f(n)) means that there is a finite constant M such that Ig(n) I ::;;
Mlf(n)1 for all sufficiently large n
g(n) = o(f(n)) means that g(n)lf(n) -+ 0 (as n -+ 0 or as n -+ 00 , depending
on the context)
o the empty set
AlBic queue a queue with arrival distribution A, service time distribution
B, and c servers (replacing A or B by M indicates that the
corresponding distribution is exponential)
FIFO the first-in, first-out queue discipline (sometimes called
FCFS: first-come, first-served)
LIFO the last-in, first-out queue discipline (sometimes called
LCFS).
= convergence in distribution of random variables (and weak
convergence of probability measures)
xx Notation

iid independent and identically distributed


cdf cumulative distribution function
pdf probability density function
pmf probability mass function

With a few exceptions we use capital letters to denote random variables,


and corresponding lower-case letters to denote their values. Unless the
context makes it clear otherwise, U denotes a random variable uniformly
distributed on (0,1).
Precedence Diagram

1.2 = Chapter 1, Section 2


3 ~ 4 = 3 is a prerequisite for 4
5 H 6 = 5 and 6 can be read profitably in parallel

r;7\
\:Y
A
I
I
J
I
I
I
I

.
I
0088 I
~
I
I I
I
I I
I
I I
I

rz-. cj
I
I

~ rt>.
~ \::;J \:::Y

You might also like