ISyE 3232C

YL. Chang

Stochastic Manufacturing and Service Systems

Fall 2015

Continuous Time Markov Chain

Motivation

Suppose there are total n 1 customers in a 1-server post office. (1 customer in service, n-1 in waiting).
Assume the interarrival time TA exp(2) and the service time TS exp(3).
Consider the state is the # of total customers in the system:
S = {0, 1, 2, ...}
Two events that can affect the state of the system:
A service is complete.
A new arrival.

If a service is complete before a new arrival, then the # of customers in the system will increase by one.
On the other hand, if a service is complete after a new arrival, then the # of customers in the system will
decrease by one. The time until the # of customers changes is random.
Questions:
1. Under what condition that the number of customers will increase to n + 1?
TA < Ts
2. What is the probability that the number of customers will increase to n + 1?
P (TA < Ts ) =

A
2
= 2/5
=
A + S
2+3

3. What is the probability that the number of customers will decrease to n 1?

P (TA > Ts ) =

S
3
=
= 3/5
A + S
2+3

4. When will the number of customers change? Do you know its distribution?
The time until the state of the system changes is called holding time.
min(TA , Ts ) exp(2 + 3)
5. If we model it as a DTMC, where the state is the number of customers in the system, can you draw
the transition diagram?

6. Do you know the difference between DTMC and this problem?

In DTMC, n = 1, 2, 3, ... are the times that transitions can occur.
In CTMC, transition occurs whenever an event occurs. The holding time is random.
7. Informally, what is CTMC?
DTMC + Holding time, where the holding time follows an exponential distribution.

Suppose S = {0, 1, 2, 3, 4} in post office. (max 3 waiting), TA exp(2), TS exp(3).

1. (Embedded DTMC)Draw DTMC type transition diagram; Transition matrix (roadmap matrix R)
Every continuous-time Markov chain has an associated embedded DTMC.
0
1
R= 2
3
4

0
0
3/5

0
0
0

1
1
0
3/5
0
0

2
0
2/5
0
3/5
0

3
0
0
2/5
0
1

0
0

0
2/5
0

2. Add holding time information

Time to transition:
State 0 : exp(2)
State 1 : exp(5)
State 2 : exp(5)
State 3 : exp(5)
State 4 : exp(3)
3. Rate diagram

4. Generator matrix G: contains all the rate information for the chain.
Its entries are NOT probabilities. It is the counterpart of the one-step transition probability matrix P
for DTMC. Sum of each row equals to ZERO.
0
1
G= 2
3
4

0
2
3

0
0
0

1
2
5
3
0
0

2
0
2
5
3
0

3
0
0
2
5
3

0
0

0
2
3

CTMC Definition

Markov Property: For a continuous time stochastic process {X(t : t 0)} with state space S, we say it
has the Markov property if
P (X(t) = j|X(s) = i, X(tn1 = in1 , ..., X(t + 1) = i1 )) = P (X(t) = j|X(s) = i),
where 0 t1 t2 ... tn1 s t is any nondecreasing sequence of n+1 times and i1 , i2 , ..., in1 , i, j S
are any n + 1 states in the state space, for any integer n 1.
Explanation:
(1)Given the state of the process at any set of times prior to time t, the distribution of the process at time
t depends only on the process at the most recent time prior to time t.
(2) Given the state of the process at time s, the distribution of the process at an time after s is independent
of the entire past of the process before time s. This notion is exactly analogous to the Markov property for
a discrete-time process.
Definition: A continuous-time stochastic process {X(t) : t 0} is called a continuous-time Markov chain
if it satisfies Markov property.
The Markov property is a forgetting property, suggesting memorylessness in the distribution of the time
a continuous-time Markov chain spends in any state. This is indeed the case if the process is also timehomogeneous.
Time Homogeneity: We say that a continuous-time Markov chain is time homogeneous if for any s t
and any state i, j, i S
P (X(t) = j|X(s) = i) = P (X(t s) = j|X(0) = i).
A continuous-time Markov chain needs not be time homogeneous, but in our course we will only consider
time homogeneous Markov chains.

CTMC Models

Solution Procedure:
(1) Identify the state space S.
(2) At state i S, let K be the set of all possible events that can change the state from state i.
(3) (Holding
P time information) Let
Pk be the rate of event k K. Then the holding time at state i follows
exp( kK k ). Put Gi,i = kK k .
(4) Consider the transition from state i to state j. let H be the set of all possible events that can change
the state of the system from state i to state j. (Note, H K.)
(5) Let
event h H. Then the rate that the system can go fromPstate i to state j is
P h be the rate of P

.
Put
G
=
i,j
hH h
hH h . Draw an arrow from state i to state j with rate
hH h .
(6) Complete the generator matrix G and rate diagram. Make sure sum of each row of G = 0.
(7) Construct the embedded DTMC transition matrix using rate diagram and holding time information.
G
.
Pij = |Gij
ii |

Remark: Usually we first draw the rate diagram and then write down the generator matrix
P G. In such a
way, you can use solution procedure (1), (4)-(7) and skip step (2)-(3). Finally let Gii = j6=i Gij .
Example 1: A machine goes up and down over time. When it is down, it takes di amount of time to be
repaired, where di is exponentially distributed with mean 1 hour. {di , i = 1, 2, 3, ...} is an iid sequence.
When it is up, it will stay up ui amount of time that is exponentially distributed with mean 10 hours.
{ui , i = 1, 2, 3, ...} is an iid sequence. Assume that up times and repair times are independent. Model it as
a CTMC.
State space S = {up, down}.

Generator Matrix G = up
down

 up
1/10
1

Transition Probability Matrix R = up

down

Figure 1: Rate diagram

Figure 2: Transition diagram

down
1/10
1

up
0
1

down
1
0

Example 2: We have two machines each of which having the same characteristics as Example 1 and there is
only one repairperson. Let X(t) be the number of machines that are up at time t. Model it as a CTMC.
State space S = {0, 1, 2}.
0
G= 1
2

0
1
1/10
0

1
1
11/10
1/5

2
!
0
1
1/5

Figure 3: Rate diagram

0
R= 1
2

0
0
1/11
0

1
1
0
1

2
0 !
10/11
0

Figure 4: Rate diagram

Remarks:
Rate from state 2 to state 1:
There are two events can change the state from 2 to 1: (1) Machine 1 is down, or (2) Machine 2 is down.
The time until Machine 1 is down is T1 exp(1/10) and the time until Machine 2 is down is T2 exp(1/10).
Then the time until the state changes is min(T1 , T2 ) exp(1/10 + 1/10). This is exactly step (4) - (5) in
the solution procedure on page 4.
Further, note the rate from state 2 to state 0 is zero. The probability that two machines fail at the same
time is P (T1 = T2 ) = 0 since they are continuous random variables.

Example 3: We have two operators and three phone lines. Call arrival follows a Poisson process with rate
= 2 calls/minute. Each call processing time is exponentially distributed with mean 5 minutes. Let X(t)
be the number of calls in the system at time t. We assume that calls arriving when no phone line is available
are lost.
State space S = {0, 1, 2, 3}.
0
1
2
3

0 2
2
0
0
1/5 11/5
2
0
G= 1
2 0
2/5
12/5
2
3
0
0
2/5
2/5

Figure 5: Rate diagram

0
0 0
1/11
R= 1
2 0
3
0

1
1
0
2/12
0

2
0
10/11
0
1

3
0
0
10/12
0

Figure 6: Transition diagram

Remark: At state 3, 2 calls are in process and one call is waiting. The events that can change state from
3 to 2 are call 1 is done or call 2 is done. The rate that call 1 is done is 1/5 and the rate that call 2 is
done is 1/5. Hence, the rate from state 3 to state 2 is 1/5 + 1/5.
Now assume if each customer has a patience that is exponential distributed with mean 10 minutes. When
the waiting time in the queue of a customer exceed this patience time, the customer abandons the system
without service. Then the rate diagram become:

It will only change rate from state 3 to state 2 since: at state 3, there is a customer is waiting, who may
abandon its service with rate 1/10.
7

Example 4: A small call center with 3 phone lines and two agents, Mary and John. Call arrivals follow a
Poisson process with rate = 2 calls per minute. Marys processing times are iid exponential with mean 4
minutes. Johns processing times are iid exponential with mean 6 minutes. Customers patience follows iid
exponential with mean 10 minutes. An incoming call to an empty system always goes to John.
If we assume the state space is same as Example 3 S = {0, 1, 2, 3}. Then the rate diagram is:

The difficulty is we cannot determine the rate from state 1 to state 0. It depends on who is serving the
customer. If Mary is serving the customer, then the rate is 1/4. If John is serving the customer, then the
rate is 1/6. This state space is not detailed enough to capture such information. Hence we need a more
descriptive state space to distinguish the two situations.
State space S = {0, 1M , 1J , 2, 3}.
0
1M
G = 1J
2
3

0
2
1/4

1/6

0
0

1M
0
9/4
0
1/6
0

1J
2
0
13/6
1/4
0

2
0
2
2
(1/6 + 1/4 + 2)
(1/4 + 1/6 + 1/10)

0
0

2
(1/4 + 1/6 + 1/10)

Stationary Distribution of CTMC

Definition: In CTMC, is said to be a stationary distribution if for all t 0,

where [P (t)]i,j = P (X(t) = j|X(0) = i).
1. Flow balance equation
in flow = out flow
State 0: 31 = 20
State 1: 20 + 32 = 51
State 2: 21 + 33 = 52
State 3: 22 + 34 = 53
State 4: 23 = 34
0 + 1 + 2 + 3 + 4 = 1
= (81/211, 54/211, 36/211, 24/211, 16/211).
2. Cutting Method
{0}, {1, 2, 3, 4}

20 = 31

{0, 1}, {2, 3, 4}

21 = 32

{0, 1, 2}, {3, 4}

22 = 33

{0, 1, 2, 3}, {4}

23 = 34

0 + 1 + 2 + 3 + 4 = 1
= (81/211, 54/211, 36/211, 24/211, 16/211).

iS

i = 1, P (t) = ,

3. G = 0

2
3
0
0
0

(0 1 2 3 4 )

2
5
3
0
0

0
2
5
3
0

0
0
2
5
3

0
0
0
2
3

= (0 0 0 0 0)

20 + 31 = 0

20 51 + 32 = 0

2 5 + 3 = 0
1
2
3

5
+
3
2
3
4 =0

23 34 = 0

0 + 1 + 2 + 3 + 4 = 1
= (81/211, 54/211, 36/211, 24/211, 16/211).

The jth equation in 0 = G is given by

0 = j j +

ij i ,

i6=j

which is equivalent to
j j =

i ij .

i6=j

On the left hand side, j is the long run proportion of time that the system is in state j, while j is
the rate of leaving state j when the system is in state j. Thus, the product j j is interpreted as the
long run rate of leaving state j. On the right hand side, ij is the rate of going to state j when the
system is in state i, so the product i ij is interpreted as the long run rate of going from state i to
state j. Summing over all i 6= j then gives the long run rate of going to state j. That is, the equation
X
j j =
i ij
i6=j

is interpreted as the long run rate out of state j=the long run rate into state j and for this reason
the equations 0 = G are called the Global Balance Equations or just B alance Equations, because
they express the fact that when the system is made stationary, there must be equality, or balance, in
the long run rates into and out of any state.

10

Interpretation of Stationary distribution: (1 , 2 , ...i , ...)

In long run, the probability that the system is in state i is i ;
The long-run fraction of time that the system is in state i is i .
Questions of interest for example on Page 3, assuming we know = (0 , 1 , 2 , 3 , 4 ):
(1) What is the long run fraction of time the system is empty?
0
(2) What is the long run fraction of time the server is busy?
1 0
(3) What is the probability that a customer is not accepted to the system?
4
(4) What is the expected # of customers in the system?
Lsys = 0 0 + 1 1 + 2 2 + 3 3 + 4 4 .
(5) What is the expected # of customers in the queue?
Lq = 0 0 + 0 1 + 1 2 + 2 3 + 3 4 .

Stationary distribution in CTMC: existence and uniqueness

Recall that finite, irreducible, aperiodic DTMC has a unique stationary distribution
Periodicity is not a concern in CTMC because continuous time for time to transition
Irreducibility is not a big concern because most of CTMC has only one class. CTMC in general concerns
with a continuous stochastic process.
Hence, as long as CTMC is irreducible,
CTMC with finite state space has a unique stationary distribution
CTMC with infinite state space has a unique stationary distribution if and only if it is stable.
If you can calculate unique stationary distribution, then the CTMC (either finite state space
or infinite state space) is stable

11

CTMC transition matrix P (t) and Generator matrix G

Motivation: Suppose S = {0, 1, 2, 3, 4} in post office. (max 3 waiting), TA exp(2), TS exp(3). Assume
there are two customers now, that is the probability that there will be three customers in 6.5 minutes later?
Generator matrix is

2 2
0
0
0
3 5 2
0
0

0
3
5
2
0
G=

0
0
3 5 2
0
0
0
3 3

CTMC transition matrix [P (t)]i,j = P [X(t) = j|X(0) = i]

P (t) = etG
P (s + t) = P (s)P (t)
P (as + bt) = [P (s)]a [P (t)]b

Note, e6.5G

2
e
e3
0
6=
e0
e
e0

0.3842
0.3840

P (6.5) = e6.5G =
0.3837
0.3833
0.3830

2
0
0
0
e
e
e
e
e5 e2
e0
e0

3
5
2
e
e
e
e0

e0
e3 e5 e2
e0
e0
e3 e3

0.2560
0.2560
0.2559
0.2557
0.2256

0.1705
0.1706
0.1707
0.1708
0.1708

0.1136
0.1137
0.1138
0.1140
0.1142

0.0757
0.0757

0.0759

0.0761
0.0763

In MATLAB, you could use the following commands:

G = [2 2 0 0 0 ; 3 5 2 0 0 ; 0 3 5 2 0 ; 0 0 3 5 2 ; 0 0 0 3 3];
expm ( 6 . 5 G)
In your exam, this matrix will be given and you dont need to calculate it by yourself.
Hence,
P (X(6.5) = 3|X(0) = 2) = [P (6.5)]2,3 = [e6.5G ]2,3 = 0.1138
In addition, Suppose P (2) and P (0.5) are available, then
P (6.5) = P (2 3 + 0.5 1) = [P (2)]3 [P (0.5)]1 .
Note: Matrix times Matrix.
Suppose e5G and e2G are available, find P (X(5) = 2, X(7) = 3|X(0) = 1).
P (X(5) = 2, X(7) = 3|X(0) = 1)
=P (X(7) = 3|X(5) = 2)P (X(5) = 2|X(0) = 1) by Markov property
=[e2G ]2,3 [e5G ]1,2

12

Comparison between DTMC and CTMC

DTMC

CTMC

fundamental diagram

transition diagram

rate diagram

fundamental matrix

transition matrix (P)

generator matrix (G)

stationary distribution

P = ,

Methods for calculating

P = ,

i = 1

P (t) = , t,

i = 1

G = 0,

i = 1

i = 1

cut method

cut method

flow balance

flow balance

13

CTMC and Queueing Theory

M/M/./. queuing model (Both interarrival and service times are exponentially distributed) can be analyzed
by CTMC, where the state is defined as the total number of customers in the system.
Examples: M/M/1/, M/M/1/b, M/M/k/, M/M//, etc.

M/M/1/
Arrival is P P () and service completion is P P ().
TA exp(), Ts exp().
Assume < for stability ( = / < 1).
Let X(t) be the number of customers in the system at time t. Then {X(t), t 0} is CTMC with state
space S = {0, 1, 2, ...}.
Rate diagram

Stationary distribution (cut method)

Using cut method, we have

1 = 2 , 2 = 1 = ( )2 0

2 = 3 , 3 = 2 = ( )3 0

..
.

n1 = n , n = n1 = ( )n 0

0 + 1 + 2 + = 1

0 (1 + + ( )2 + ( )3 + . . . ) = 1

1
0
=1
1
0 = 1 , 1 =

Hence,
0 = 1

, n = ( )n (1 ), n 0.

14

Questions:
(a) The long run average # of customers in the system (Lsys )?
Lsys = 0 0 + 1 1 + 2 2 + ...
= 0 (1 ) + 1 (1 ) + 2 2 (1 ) + 3 3 (1 ) + 4 4 (1 ) + ...
= (1 )[1 + 2 + 33 + 43 + ...]
= (1 )(1 + + 2 + 3 + ...)0
1 0
)
= (1 )(
1

=
= (1 )
=
=
(1 )2
1
1 /

(b) The expected time spent in the system by a customer (Wsys )?

Recall: Littles Law L = W .
Wsys =

1
1 /
1 1
E[V ]
Lsys
=
=
=
=

1
1
1
1

where V is the service time and E[V ] =

1
.

(c) The long run average # of customers in the queue (Lq )?

Lq = 0 0 + 0 1 + 1 2 + 2 3 + 3 4 + ... =

(n 1)n

n=2

(d) The expected time spent in the queue by a customer (Wq )?

)(
Recall the Kingman formula: for G/G/1/, Wq = E[V ]( 1

Ca2 +Cs2
).
2

Two ways:
(1) Wq =

Lq

(2)
Wq = Wsys E[V ] =

E[V ]

12 + 12
E[V ] = E[V ]
= E[V ]
(
)
1
1
1
2

which is exactly the Kingman approximation formula. Again, as 1, Wq .

At this point, I recommend that you review all the concepts in queueing theory and compare with the
results we have developed using CTMC. Queueing theory and CTMC are strongly related and queueing
theory is a very important application of CTMC.

15

M/M/1/b Queue
Identify the state space and model it as a CTMC.
Assume b = 2, then S = {0, 1, 2, 3}.

Setup the equations for stationary distribution.

Method 1: G = 0, where

G=
0
0
0

0
0
0
( + )

0
0

( + )

( + )
0
0

0 + 1 + 2 + 3 = 1.

Method 2: Cut method.

0 = 0

1 = 2 , 2 = 1 = 2 0

2 = 3 , 3 = 2 = 3 0

0 = 1 , 1 =

0 + 1 + 2 + 3 = 1
0 (1 + + 2 + 3 ) = 1
0 =

1
.
1 + + 2 + 3

Questions:
(a) What is the average number of customers in the system?
Lsys = 0 0 + 1 1 + 2 2 + 3 3
(b) What is the average number of customers in the queue?
Lq = 0 0 + 0 1 + 1 2 + 2 3

16

(c) What is the expected total time spent in the system for a customer (Wsys )? (Waiting + Service time)
Note, when the system is full (3 ), then any new customers will be blocked. Then the actual arrival
rate to the system will be 0. If the system is not full (1 3 ), then new customers can be accepted
into the system. Then the actual arrival rate to the system will be . Overall, the effective arrival rate
ef f = 0 3 + (1 3 ) = (1 3 ). Hence,
Wsys =

Lsys
0 0 + 1 1 + 2 2 + 3 3
=
ef f
(1 3 )

(d) What is the expected waiting time for a customer (Wq )?

Wq =

Lq
0 0 + 0 1 + 1 2 + 2 3
=
ef f
(1 3 )

(e) What is the throughput?

Throughput = min(ef f , ) = ef f since ef f < < .

17

M/M/ Queue
Identify the state space and model it as a CTMC.

Setup the equations for stationary distribution.

2
0
1 = 22 , 2 =
22
3
2 = 33 , 3 =
0
3!3
...
n
0
n1 = nn , n =
n!n
0 = 1 , 1 =

n = 1

n=0

2
3
+
+
+ ...] = 1
2
2!
3!3

1
1
0 =
= / = e .

2
3
e
1 + + 2!2 + 3!3 + . . .

0 [1 +

Hence,
n =

n
e , n 0.
n!n

Questions:
(a) What is the average number of customers in the system?
L=

X
n=0

=e

nn =

X
n=0

n1
X

X (/)
n
n

e
=
e
=
e
(
)
n!n
(n 1)!n
n=1 (n 1)!
n=0

X (/)n

( )
= e ( )e =
n=0 (n)!

(b) Capacity provisioning: Suppose you have actually 100 servers. You want to achieve a certain level of
service capacity. What is the probability that you will lose customers?
P (X() > 100) = 1 P (X() 100) = 1 [P (X() = 0) + P (X() = 1) + + P (X() = 100)]
= 1 [0 + 1 + 2 + + 100 ].
(c) Compute how many servers you will need to limit the loss probability to a certain level. Suppose you
want to limit the loss probability to 2%. How many servers do you need?
P (X() > c) = 0.02
1 [0 + 1 + ... + c ] 0.02
find the smallest c to satisfy the inequality.
18

M/M/k Queue
Identify the state space and model it as a CTMC.

Setup the equations for stationary distribution.

2
0
1 = 22 , 2 =
22
3
2 = 33 , 3 =
0
3!3
...
0 = 1 , 1 =

k
0
k!k

k
k = kk+1 , k+1 =
k =
0
k
k k!k

k
k+1 = kk+2 , k+2 =
k+1 = ( )2 k = ( )2
0
k
k
k k!k
...

k1 = kk , k =

k+n1 = ( )n k = ( )n
0
k
k
k k!k

k+n1 = kk+n , k+n =

...

n = 1

n=0

2
k1

0 +

+
0 + k +
k + ( )2 k + ( )3 k + = 1
0
2
k1
1!
2!
(k 1)!
k
k
k
2
k1

0 [1 +
+
+ +
] + k [1 +
+ ( )2 + ( )3 + . . . ] = 1
1! 2!2
(k 1)!k1
k
k
k

2
k1
1
0 [1 +
+
+ +
] + k
=1

2
1! 2!
(k 1)!k1
1 k
0 +

0 [1 +
0 =

2
k1
k
1
+
+ +
+
]=1

2
k1
1! 2!
(k 1)!
(k)!k 1 k
1

1+

1!

2
2!2

+ +

k1
(k1)!k1

k
1

(k)!k 1 k

Questions:
(a) What is the probability that a customer will have to wait when it arrives?
P (X() k) = k + k+1 + ... = k (1 +

1
1
k
+ ( ) 2 + ( ) 3 + . . . ) = k
=
0

k
k
k
1 k
1 k k!k

19

(b) What is the average number of customers in the queue?

The traffic intensity = k

.

+ 2( )2 + 3( )3 + . . . ) = k (1 + 2 + 32 + ...)
k
k
k
0

= k ( + 2 + 33 + ...)0 = k (
) = k
1
(1 )2

Lq = 1 k+1 + 2 k+2 + 3 k+3 ... = k (

(c) What is the average waiting time in the queue?

Wq =

Lq
1

k
=

(1 )2

Open Jackson Network

Open means arriving customers will eventually leave the system.

7.1

Tandem Queue

Suppose two queues are connected in tandem, which means that the queues are in series. Jobs arrives at rate
= 2 per minute. The first queues mean processing time is 1/1 = 0.45 minutes and the second queues
mean processing time is 1/2 = 0.40 minutes. Define X(t) = (X1 (t), X2 (t)) where Xi (t) is the number of
jobs at station i at time t.

State space S = {(i, j), i, j {0, 1, 2, ...}}.

Draw the transition diagram.

Calculate the stationary distribution when 1 < 1. (If 1 1, then you dont have to care about the steady
state of the system since the first station will explode in the long run.)
20

Two queues are independent because each queue does not seem to affect each other. Hence,
P (X() = (2, 3)) = 2,3 = P (X1 () = 2, X2 () = 3) = P (X1 () = 2)P (X2 () = 3) = (11 )21 (12 )32
where
1 =

, 2 =
.
1
2

i is the traffic intensity of queue i.

In general, the stationary distribution of the network is
(n1 ,n2 ) = P (X() = (n1 , n2 )) = (1 1 )n1 1 (1 2 )n2 2
You could verify that it is indeed the stationary distribution of the network using flow balance equation. For
example,
( + 1 + 2 )(2,3) = (1,3) + 1 (3,2) + 2 (2,4) .

7.2

Failure Inspection

Assume there is an inspection at the end of the second station. If a job is found to be defective, the job will
go back to the queue of station 1 and get reprocessed. The chance of not being defective is 50%.

The stationary distribution still has the form of

(i,j) = P (X1 () = i, X2 () = j) = P (X1 () = i)P (X2 () = j) = (1 1 )i1 (1 2 )j2 .
Determine 1 and 2 (traffic equation).
(
1 = 2
1 = + 0.52
1 = 2 = 2 = 2
Traffic intensity 1 =

1
1

= 0.9, 2 =

2
2

= 0.8

Remark: At first glance, the two stations seem to be quite interrelated, but in steady-state, the two are
in fact independent. It was found by Jackson at the end of 50s and published in Operations Research. In
theory, the independence holds for any number of stations and each station can have multiple
servers.
Questions:
(a) What is the average number of jobs in the system?
XX
Lsys =
(i + j)(i,j)
i

21

Or,

Recall that the average # of jobs in a M/M/1 queue is 1

. Hence, the average # of jobs in the system
= average # of jobs in station 1 + average # of jobs in station 2, i.e.,
Lsys =

1
0.9
0.8
1
+
=
+
= 13 jobs
1 1
1 1
1 0.9 1 0.8

(b) What is the average time in system per job?

Wsys =

Lsys
13
=
= 13 minutes

(c) What is the average time in system per job if the failure rate is reduced from 50% to 40%?
Traffic equations:
(
1 = 2
1 = + 0.42
1 = 2 =
Traffic intensity 1 =

1
1

= 3/4, 2 =

Wsys =

2
2

5
5
=
3
3

= 2/3

Lsys
1
1
3/4
2/3
=
+
=
+
= 5 minutes

1 1
1 1
1 3/4 1 2/3

Waiting time is reduced by more than a half (13 minutes to 5 minutes).

(d) Assume that there are two inspections at the end of station 2 and 3. Failure rates are 30% and 20%,
respectively. What is the stationary distribution?

Traffic equations:

1 = + 0.32
2 = 1 + 0.23

3 = 0.72
1 =

1
2
3
, 2 =
, 3 =
1
2
3

(n1 ,n2 ,n3 ) = (1 1 )n1 1 (1 2 )n2 2 (1 3 )n3 3

Remarks: When you analyze an open network, remember the following:
Assume the open network contains k stations, let ni be the number of jobs in station i, then the
stationary distribution of the open network is (n1 ,n2 ,...,nk ) = (1 1 )n1 1 (1 2 )n2 2 ...(1 k )nk k .
Set up the traffic intensity to solve i .
You can decompose the network to a sequence of M/M/1 queues with correct traffic intensity i (See
Question (b)).
22

Review
1. CTMC modeling: State space, rate diagram, generator matrix G, embedded DTMC transition probability, and transition diagram.
2. Stationary distribution: 3 ways for calculating stationary distribution and the meaning of
3. CTMC transition matrix P (t) and generator matrix G
4. Queueing theory using CTMC. Know how to model a queue, how to solve its stationary distribution
and how to calculate some probabilities and useful quantities.
5. Open Jackson Network.

23