You are on page 1of 49

Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-death processes
Jorge J ulvez
University of Zaragoza
1 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
2 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
3 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Example: Consider cells which reproduce according to the
following rules:
A cell present at time t has probability h +o(h) of splitting
in two in the interval (t , t + h)
This probability is independent of age
Events betweeen different cells are independent
6.1 Pure Birth Process (Yule-Furry Process)
Example. Consider cells which reproduce according to the following
rules:
i. A cell present at time t has probability h + o(h) of splitting in two
in the interval (t, t + h)
ii. This probability is independent of age.
iii. Events between different cells are independent
Time
>
231
What is the time evolution of the system?
4 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Example: Consider cells which reproduce according to the
following rules:
A cell present at time t has probability h +o(h) of splitting
in two in the interval (t , t + h)
This probability is independent of age
Events betweeen different cells are independent
6.1 Pure Birth Process (Yule-Furry Process)
Example. Consider cells which reproduce according to the following
rules:
i. A cell present at time t has probability h + o(h) of splitting in two
in the interval (t, t + h)
ii. This probability is independent of age.
iii. Events between different cells are independent
Time
>
231
What is the time evolution of the system?
4 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Non-Probabilistic Analysis
Let n(t ) =number of cells at time t
Let be the birth rate per single cell
Thus n(t )(t ) births occur in (t , t + t )
Then:
n(t + t ) = n(t ) + n(t )t
n(t + t ) n(t )
t
= n(t )
dn
dt
= n

(t ) = n(t )
The solution of this differential equation is: n(t ) = Ke
t
If n(0) = n
0
then
n(t ) = n
0
e
t
5 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Probabilistic Analysis
Notation:
N(t ) = number of cells at time t
P{N(t ) = n} = P
n
(t )
Assumptions:
A cell present at time t has probability h +o(h) of splitting
in two in the interval (t , t + h)
The probability of more than one birth occurring in time
interval (t , t + h) is o(h)
All states are transient
6 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Assumptions:
Probability of splitting in (t , t + h): h + o(h)
Probability of more than one split in (t , t + h): o(h)
The probability of birth in (t , t + h) if N(t ) = n is nh + o(h).
Then,
P
n
(t + h) = P
n
(t )(1 nh o(h)) + P
n1
(t )((n 1)h + o(h))
P
n
(t + h) P
n
(t ) = nhP
n
(t ) + P
n1
(t )(n 1)h + f (h), with f (h) o(h)
P
n
(t + h) P
n
(t )
h
= nP
n
(t ) + P
n1
(t )(n 1) +
f (h)
h
Let h 0,
P

n
(t ) = nP
n
(t ) + (n 1)P
n1
(t )
Initial condition P
n
0
(0) = P{N(0) = n
0
} = 1
7 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Assumptions:
Probability of splitting in (t , t + h): h + o(h)
Probability of more than one split in (t , t + h): o(h)
The probability of birth in (t , t + h) if N(t ) = n is nh + o(h).
Then,
P
n
(t + h) = P
n
(t )(1 nh o(h)) + P
n1
(t )((n 1)h + o(h))
P
n
(t + h) P
n
(t ) = nhP
n
(t ) + P
n1
(t )(n 1)h + f (h), with f (h) o(h)
P
n
(t + h) P
n
(t )
h
= nP
n
(t ) + P
n1
(t )(n 1) +
f (h)
h
Let h 0,
P

n
(t ) = nP
n
(t ) + (n 1)P
n1
(t )
Initial condition P
n
0
(0) = P{N(0) = n
0
} = 1
7 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Probabilities are given by a set of ordinary differential
equations.
P

n
(t ) = nP
n
(t ) + (n 1)P
n1
(t )
P
n
0
(0) = P{N(0) = n
0
} = 1
Solution
P
n
(t ) =
_
n 1
n n
0
_
e
n
0
t
(1 e
t
)
nn
0
n = n
0
, n
0
+ 1, . . .
where
_
n
k
_
=
n!
k!(n k)!
.
8 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Solution
P
n
(t ) =
_
n 1
n n
0
_
e
n
0
t
(1 e
t
)
nn
0
n = n
0
, n
0
+ 1, . . .
Observation: The solution can be seen as a negative binomial
distribution, i.e., probability of obtaining n
0
successes in n trials.
Suppose p =prob. of success and q = 1 p =prob. of failure.
Then, the probability that the rst (n 1) trials result in (n
0
1)
successes and (n n
0
) failures followed by success on the n
th
trial is:
_
n 1
n n
0
_
p
n
0
1
q
nn
0
p =
_
n 1
n n
0
_
p
n
0
q
nn
0
; n = n
0
, n
0
+1, . . .
If p = e
t
and q = 1 e
t
, both equations are the same.
9 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process (Yule-Furry Process)
Yule studied this process in
connection with the theory of
evolution, i.e., population consists of
the species within a genus and
creation of a new element is due to
mutations.
This approach neglects the
probability of species dying out and
size of species.
Furry used the same model for
radioactive transmutations.
10 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Processes. Generalization
In a Yule-Furry process, for N(t ) = n the probability of a
change during (t , t + h) depends on n.
In a Poisson process, the probability of a change during
(t , t + h) is independent of N(t ).
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 6
Example 2. Pure birth process (Poisson process)

i
=

i
= 0
i = 0, 1, 2, . . .
i
(0) =

1 i = 0
0 i > 0
birth probability per time unit is
constant
initially the population size is 0
0
l
1
l
2
l
i
l
i-1
l
. . .
l
All states are transient

d
dt

i
(t) =
i
(t) +
i1
(t) i > 0
d
dt

0
(t) =
0
(t)
0
(t) = e
t
d
dt
(e
t

i
(t)) =
i1
(t)e
t

i
(t) = e
t

t
0

i1
(t

)e
t

dt

1
(t) = e
t

t
0
e
t

e
t


1
dt

= e
t
(t)
Recursively

i
(t) =
(t)
i
i!
e
t
Number of births in interval (0, t) Poisson(t)
Generalization
Assume that for N(t ) = n the probability of a new change
to n + 1 in (t , t + h) is
n
h + o(h).
The probability of more than one change is o(h).
11 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Processes. Generalization
Generalization
Assume that for N(t ) = n the probability of a new change
to n + 1 in (t , t + h) is
n
h + o(h).
The probability of more than one change is o(h).
Then,
P
n
(t + h) = P
n
(t )(1
n
h) + P
n1
(t )
n1
h + o(h), n = 0
P
0
(t + h) = P
0
(t )(1
0
h) + o(h)
P

n
(t ) =
n
P
n
(t ) +
n1
P
n1
(t )
P

0
(t ) =
0
P
n
(t )
Equations can be solved recursively with P
0
(t ) = P
0
(0)e

0
t
12 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Birth Process. Generalization
Let the initial condition be P
n
0
(0) = 1.
The resulting equations are:
P

n
(t ) =
n
P
n
(t ) +
n1
P
n1
(t ), n > n
0
P

n
0
(t ) =
n
0
P
n
0
(t )
Yule-Furry processes assumed
n
= n
13 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
14 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Birth-Death Processes
Notation
Pure Birth process: If n transitions take place during (0, t ),
we may refer to the process as being in state E
n
.
Changes in the pure birth process:
E
n
E
n+1
E
n+2
. . .
Birth-Death Processes consider transitions E
n
E
n1
as
well as E
n
E
n+1
if n 1. If n = 0, only E
0
E
1
is
allowed.
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 4
The time-dependent solution of a BD process (continued)
0
l
0
m
1
1
l
1
m
2
2
l
2
m
3
i+1
l
i+1
m
i+2
i
l
i
m
i+1
. . .
l
i
l
i-1
m
i+1
m
i
The equations component wise

d
i
(t)
dt
= (
i
+
i
)
i
(t)

ows out
+
i1

i1
(t) +
i+1

i+1
(t)

ows in
i = 1, 2, . . .
d
0
(t)
dt
=
0

0
(t)

ow out
+
1

1
(t)

ow in
15 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Birth-Death Processes
16 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Birth-Death Processes
Assumptions
If the process at time t is in E
n
, then during (t , t + h):
Transition E
n
E
n+1
has probability
n
h + o(h)
Transition E
n
E
n1
has probability
n
h + o(h)
Probability that more than 1 change occurs = o(h).
P
n
(t + h) = P
n
(t )(1
n
h
n
h)
+ P
n1
(t )(
n1
h) + P
n+1
(t )(
n+1
h) + o(h)
Time evolution of the probabilities
P

n
(t ) = (
n
+
n
)P
n
(t ) +
n1
P
n1
(t ) +
n+1
P
n+1
(t )
17 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Birth-Death Processes
For n = 0
P
0
(t + h) = P
0
(t )(1
0
h) + P
1
(t )
1
h + o(h)
P

0
(t ) =
0
P
0
(t ) +
1
P
1
(t )
If
0
= 0, then E
0
E
1
is impossible and E
0
is an
absorbing state.
If
0
= 0, then P

0
(t ) =
1
P
1
(t ) 0 and hence P
0
(t )
increases monotonically.
Note:
lim
t
P
0
(t ) = P
0
() = Probability of being absorbed.
18 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Steady-state distribution
P

0
(t ) =
0
P
0
(t ) +
1
P
1
(t )
P

n
(t ) = (
n
+
n
)P
n
(t ) +
n1
P
n1
(t ) +
n+1
P
n+1
(t )
As t , P
n
(t ) P
n
(limit ).
Hence, P

0
(t ) 0 and P

n
(t ) 0.
Therefore,
0 =
0
P
0
+
1
P
1
P
1
=

0

1
P
0
0 = (
1
+
1
)P
1
+
0
P
0
+
2
P
2
P
2
=

0

2
P
0
P
3
=

0

2
P
0
etc
. . .
19 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Steady-state distribution
P
1
=

0

1
P
0
; P
2
=

0

2
P
0
; P
3
=

0

2
P
0
; P
4
= . . .
The dependence on the initial conditions has disappeared.
After normalizing, i.e.,

n=1
P
n
= 1:
P
0
=
1
1 +

n=1
n1

i =0

i +1
; P
n
=
n1

i =0

i +1
1 +

n=1
n1

i =0

i +1
, n 1
20 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Steady-state distribution
P
0
=
1
1 +

n=1
n1

i =0

i +1
; P
n
=
n1

i =0

i +1
1 +

n=1
n1

i =0

i +1
, n 1
Ergodicity condition
P
n
> 0, for all n 0, i.e.,:

n=1
n1

i =0

i +1
<
21 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Example. A single server system
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 7
Example 3. A single server system
0
1

~ Exp( ) m ~ Exp( ) l
0 1
l
m
- constant arrival rate (Poisson arrivals)
- stopping rate of the service (exponential distribution)
The states of the system

0 server free
1 server busy

d
dt

0
(t) =
0
(t) +
1
(t)
d
dt

1
(t) =
0
(t)
1
(t)
Q =

BY adding both sides of the equations


d
dt
(
0
(t) +
1
(t)) = 0
0
(t) +
1
(t) = constant = 1
1
(t) = 1
0
(t)
d
dt

0
(t) + ( + )
0
(t) =
d
dt
(e
(+)t

0
(t)) = e
(+)t

0
(t) =

+
+ (
0
(0)

+
)e
(+)t

1
(t) =

+
+ (
1
(0)

+
)e
(+)t

equilibrium
distribution

deviation from
the equilibrium

decays expo-
nentially
constant arrival rate (Poisson
arrivals)
stopping rate of service
(exponential distribution)
states of the system: 0 (server
free), 1 (server busy)
P

0
(t ) = P
0
(t ) + P
1
(t )
P

1
(t ) = P
0
(t ) P
1
(t )
22 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Example. A single server system
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 7
Example 3. A single server system
0
1

~ Exp( ) m ~ Exp( ) l
0 1
l
m
- constant arrival rate (Poisson arrivals)
- stopping rate of the service (exponential distribution)
The states of the system

0 server free
1 server busy

d
dt

0
(t) =
0
(t) +
1
(t)
d
dt

1
(t) =
0
(t)
1
(t)
Q =

BY adding both sides of the equations


d
dt
(
0
(t) +
1
(t)) = 0
0
(t) +
1
(t) = constant = 1
1
(t) = 1
0
(t)
d
dt

0
(t) + ( + )
0
(t) =
d
dt
(e
(+)t

0
(t)) = e
(+)t

0
(t) =

+
+ (
0
(0)

+
)e
(+)t

1
(t) =

+
+ (
1
(0)

+
)e
(+)t

equilibrium
distribution

deviation from
the equilibrium

decays expo-
nentially
P

0
(t ) = P
0
(t ) + P
1
(t )
P

1
(t ) = P
0
(t ) P
1
(t )
Given that: P
0
(t ) + P
1
(t ) = 1, P

0
(t ) + ( + )P
0
(t ) = .
P
0
(t ) =

+
+
_
P
0
(0)

+
_
e
(+)t
P
1
(t ) =

+
+
_
P
1
(0)

+
_
e
(+)t
Solution = Equilibrum distribution + Deviation from the
equilibrium with exponential decay.
23 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Poisson Process. Probabilities
Poisson Process
Birth probability per time unit is constant
The population size is initially 0
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 6
Example 2. Pure birth process (Poisson process)

i
=

i
= 0
i = 0, 1, 2, . . .
i
(0) =

1 i = 0
0 i > 0
birth probability per time unit is
constant
initially the population size is 0
0
l
1
l
2
l
i
l
i-1
l
. . .
l
All states are transient

d
dt

i
(t) =
i
(t) +
i1
(t) i > 0
d
dt

0
(t) =
0
(t)
0
(t) = e
t
d
dt
(e
t

i
(t)) =
i1
(t)e
t

i
(t) = e
t

t
0

i1
(t

)e
t

dt

1
(t) = e
t

t
0
e
t

e
t


1
dt

= e
t
(t)
Recursively

i
(t) =
(t)
i
i!
e
t
Number of births in interval (0, t) Poisson(t)
All states are transient
Equations
P

i
(t ) = P
i
(t ) + P
i 1
(t ), i > 0
P

0
(t ) = P
0
(t )
24 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Poisson Process. Probabilities
Equations
P

i
(t ) = P
i
(t ) + P
i 1
(t ), i > 0
P

0
(t ) = P
0
(t )
P
0
(t ) = e
t
d
dt
[e
t
P
i
(t )] = P
i 1
(t )e
t
P
i
(t ) = e
t

_
t
0
P
i 1
(t

)e
t

dt

P
1
(t ) = e
t

_
t
0
e
t

e
t

dt

= e
t
(t )
Recursively: P
i
(t ) =
(t )
i
i !
e
t
Number of births in interval (0, t ) Poisson(t ).
25 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Death Process. Probabilities
Pure Death Process
All the individuals have the same mortality rate
The population size is initially n
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 5
Example 1. Pure death process
_

i
= 0

i
= i
i = 0, 1, 2, . . .
i
(0) =
_

_
1 i = n
0 i = n
all individuals have the same
mortality rate
the system starts from state n
0
m
1
2m
2
3m
n n-1
. . .
(n-1) m nm
State 0 is an absorbing state,
other states are transient
_

_
d
dt

n
(t) = n
n
(t)
n
(t) = e
nt
d
dt

i
(t) = (i + 1)
i+1
(t) i
i
(t) i = 0, 1, . . . , n 1
d
dt
(e
it

i
(t)) = (i + 1)
i+1
(t)e
it

i
(t) = (i + 1)e
it

_
t
0

i+1
(t

)e
it

dt

n1
(t) = ne
(n1)t

_
t
0
e
nt

e
(n1)t

. .
e
t

dt

= n e
(n1)t
(1 e
t
)
Recursively

i
(t) =
_
_
_
n
i
_
_
_
(e
t
)
i
(1 e
t
)
ni
Binomial distribution: the survival
probability at time t is e
t
inde-
pendent of others
State 0 is an absorbing state. The rest are transient.
Equations
P

n
(t ) = nP
n
(t )
P

i
(t ) = (i + 1)P
i +1
(t ) i P
i
(t ), i = 0, . . . , n 1
26 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Pure Death Process. Probabilities
Equations
P

n
(t ) = nP
n
(t )
P

i
(t ) = (i + 1)P
i +1
(t ) i P
i
(t ), i = 0, . . . , n 1
P
n
(t ) = e
nt
d
dt
[e
i t
P
i
(t )] = (i +1)P
i +1
(t )e
i t
P
i
(t ) = (i +1)e
i t

_
t
0
P
i +1
(t

)e
i t

dt

P
n1
(t ) = ne
(n1)t

_
t
0
e
nt

e
(n1)t

dt

= ne
(n1)t
(1 e
t
)
Recursively: P
i
(t ) =
_
n
i
_
(e
t
)
i
(1 e
t
)
ni
Binomial distribution: The survival probability at time t is e
t
independent of others.
27 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
28 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Relation to CTMC
Innitesimal generator matrix:
Q =
_
_
_
_
_
_
_
_

0

0
0 . . . . . . . . . . . .

1
(
1
+
1
)
1
0 . . . . . . . . .
0
2
(
2
+
2
)
2
0 . . . . . .
.
.
. 0
3
(
3
+
3
)
3
0 . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
_
_
_
_
_
_
_
_
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 4
The time-dependent solution of a BD process (continued)
0
l
0
m
1
1
l
1
m
2
2
l
2
m
3
i+1
l
i+1
m
i+2
i
l
i
m
i+1
. . .
l
i
l
i-1
m
i+1
m
i
The equations component wise

d
i
(t)
dt
= (
i
+
i
)
i
(t)

ows out
+
i1

i1
(t) +
i+1

i+1
(t)

ows in
i = 1, 2, . . .
d
0
(t)
dt
=
0

0
(t)

ow out
+
1

1
(t)

ow in
29 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Relation to DTMC
Embedded Markov chain of the process.
For t , dene:
P(E
n+1
|E
n
) = Prob. of transition E
n
E
n+1
= Prob. of going to E
n+1
conditional on being inE
n
Dene P(E
n1
|E
n
) similarly. Then
P(E
n+1
|E
n
)
n
, P(E
n1
|E
n
)
n
P(E
n+1
|E
n
) =

n

n
+
n
, P(E
n1
|E
n
) =

n

n
+
n
The same conditional probabilities hold if it is given that a
transition will take place in (t , t + h) conditional on being in E
n
.
30 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
31 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Linear Birth-Death Processes
Linear Birth-Death Process

n
= n

n
= n
P

0
(t ) = P
1
(t )
P

n
(t ) = ( + )nP
n
(t ) + (n 1)P
n1
(t ) + (n + 1)P
n+1
(t )
Steady state behavior is characterized by:
lim
t
P

0
(t ) = 0 P
1
() = 0
Similarly as t P

n
() = 0
32 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Linear Birth-Death Processes
Steady state behavior is characterized by:
lim
t
P

0
(t ) = 0 P
1
() = 0
Similarly as t P

n
() = 0
Two cases can happen:
If P
0
() = 1 the probability of ultimate extinction is 1.
If P
0
() = P
0
< 1, the relations P
1
= P
2
= P
3
. . . = 0
imply with probability 1 P
0
that the population can
increase without bounds.
The population must either die out or increase indenitely.
33 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Mean of a Linear Birth-Death Process
P

n
(t ) = ( + )nP
n
(t ) + (n 1)P
n1
(t ) + (n + 1)P
n+1
(t )
Dene Mean by M(t ) =

n=1
nP
n
(t )
and consider M

(t ) =

n=1
nP

n
(t ), then:
M

(t ) = ( + )

n=1
n
2
P
n
(t ) +

n=1
(n 1)nP
n1
(t )
+

n=1
(n + 1)nP
n+1
(t )
Write (n 1)n = (n 1)
2
+ (n 1), (n +1)n = (n +1)
2
(n +1)
34 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Mean of a Linear Birth-Death Process
M

(t ) = ( + )

n=1
n
2
P
n
(t )
+

n=1
(n 1)
2
P
n1
(t ) +
_

n=1
(n + 1)
2
P
n+1
(t ) + P
1
(t )
_
+

n=1
(n 1)P
n1
(t )
_

n=1
(n + 1)P
n+1
(t ) + P
1
(t )
_
M

(t ) =

n=1
nP
n
(t )

n=1
nP
n
(t ) = ( )M(t )
M(t ) = n
0
e
()t
if P
n
0
(0) = 1
35 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Mean of a Linear Birth-Death Process
M(t ) = n
0
e
()t
If > then M(t )
If < then M(t ) 0
Similarly if M
2
(t ) =

n=1
n
2
P
n
(t ) one can show that:
M

2
(t ) = 2( )M
2
(t ) + ( + )M(t )
and when > , the variance is:
n
0
e
2()t
_
1 e
()t
_
+

36 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Outline
1 Birth Processes
2 Birth-Death Processes
3 Relationship to Markov Chains
4 Linear Birth-Death Processes
5 Examples
37 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Linear Birth-Death Process. Example
Let X(t ) be the number of bacteria in a colony at instant t .
Evolution of the population is described by:
the time that each of the individuals takes for division in
two (binary ssion), independently of the other bacteria
the life time of each bacterium (also independent)
Assume that:
Time for division is exponentially dist. (rate )
Life time is also exponentially dist. (rate )
M(t ) = n
0
e
()t
If > then the population tends to innity
If < then the population tends to 0
38 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
A queueing system
s servers

K places
s servers
K waiting places
arrival rate (Poisson)
Exp() holding time
(expectation 1/)
Is it a birth-death process?
39 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
A queueing system
s servers

K places
s servers
K waiting places
arrival rate (Poisson)
Exp() holding time
(expectation 1/)
Let N =number of customers in the system be the state variable.
N determines uniquely the number of customers in service
and waiting room.
After each arrival and departure the remaining service
times of the customers in service are Exp() distributed
(memoryless).
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 10
Example 1. A queueing system

l
m
m
m
m
K odotuspaikkaa
s palvelinta
K=5
s=4

s servers
K waiting places
arrival rate (Poisson)
Exp() holding time (expectation 1/)
The number of customers in system N is an appropriate state variable
- uniquely determines the number of customers in service and in waiting room
- after each arrival and departure the remaining service times
of the customers in service are Exp() distributed (memoryless)
l l l l l l l l l
2m m 3m 4m 4m 4m 4m 4m 4m
2 1 0 3 4 5 6 7 8 9
40 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Call blocking in an ATM network
An ATM network offers calls of two different types.
_

_
R
1
= 1Mbps

1
= arrival rate

1
= mean holding time
_

_
R
2
= 2Mbps

2
= arrival rate

2
= mean holding time
Assume that the capacity of the link is innite:
Is it a birth-death process?
41 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Call blocking in an ATM network
An ATM network offers calls of two different types.
_

_
R
1
= 1Mbps

1
= arrival rate

1
= mean holding time
_

_
R
2
= 2Mbps

2
= arrival rate

2
= mean holding time
Assume that the capacity of the link is innite:
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 11
Example 2. Call blocking in an ATM network
A virtual path (VP) of an ATM network is oered calls of two dierent types.

R
1
= 1Mbps

1
= arrival rate

1
= mean holding time

R
2
= 2Mbps

2
= arrival rate

2
= mean holding time
a) The capacity of the link is large (innite)
n
1
n
2
l
1
l
2
l
2
l
1
(n +1)
2 2
m
(n +1)
1 1
m n
1 1
m
n
2 2
m
The state variable of the Markov process in this example is the pair (N
1
, N
2
), where N
i
denes
the number of class-i connections in progress.
The state variable is the pair (N
1
, N
2
) where N
i
denes the number of
class-i connections in progress.
42 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Call blocking in an ATM network
An ATM network offers calls of two different types.
_

_
R
1
= 1Mbps

1
= arrival rate

1
= mean holding time
_

_
R
2
= 2Mbps

2
= arrival rate

2
= mean holding time
Assume that the capacity of the link is limited to 4.5 Mbps
Is it a birth-death process?
43 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Call blocking in an ATM network
An ATM network offers calls of two different types.
_

_
R
1
= 1Mbps

1
= arrival rate

1
= mean holding time
_

_
R
2
= 2Mbps

2
= arrival rate

2
= mean holding time
Assume that the capacity of the link is limited to 4.5 Mbps
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 12
Call blocking in an ATM network (continued)
b) The capacity of the link is 4.5 Mbps
n
1
n
2
44 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Exercise 1
Process denition
There are two transatlantic cables each of which handle
one telegraph message at a time.
The time-to-breakdown for each has the same exponential
random distribution with parameter .
The time to repair for each cable has the same exponential
random distribution with parameter .
Tasks:
Draw the corresponding birth-death process.
Write its innitesimal generator.
Write differential equations for the probabilities.
Compute the steady state distribution
45 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Exercise 2
Birth-disaster process
Consider that X
t
is a continuous-time Markov process dened
as follows:
Each individual gives a birth after an exponential random
time of parameter , independent of each other.
A disaster occurs randomly at exponential random time of
parameter .
Once a disaster occurs, it wipes out all the entire
population.
Tasks:
What is the innitesimal generator matrix of the process?
What is the time evolution of M(t ) = E[X
t
]?
46 / 47
Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples
Acknowledgments
Much of the material in the course is based on the following
courses:
Queueing Theory / Birth-death processes.
J. Vitano
Birth and Death Processes
http://www.bibalex.org/supercourse/
Performance modelling and evaluation. Birth-death
processes.
J. Campos
Discrete State Stochastic Processes
J. Baik
47 / 47