Marquette University

MSCS6020

Chapter 2: Elements of Probability

Continued
Daniel B. Rowe, Ph.D.

Department of Mathematics,
Statistics, and Computer Science

Copyright 2015 by D.B. Rowe

1

Marquette University

MSCS6020

Agenda
2.7 Chebyshevs Inequality and the law of Large
Numbers
2.8 Some Discrete Random Variables
2.9 Continuous Random Variables

D.B. Rowe
2

Marquette University

MSCS6020

2.7 Chebyshevs Inequality and the Laws of Large

Numbers
Markovs Inequality
If X takes on only nonnegative values, then for any value a>0,

E[ X ]
P{ X a}
a
Proof
a, if X a
Let Y
, because X 0 it follows that

0, if X a

X Y and E[ X ] E[Y ] aP{X a} .

D.B. Rowe
3

Marquette University

MSCS6020

2.7 Chebyshevs Inequality and the Laws of Large

Numbers
Chebyshevs Inequality
If X is an RV having mean and variance 2, then for k>0,

1
P{| X | k } 2
k

Proof: Since (X - )2/2 is a nonnegative RV with mean

( X )2 E[( X ) 2 ]
E

2
2
, then from Markovs inequality

( X )2
1 .
2
E[ X ]
P
k 2
2
P
{
X

a
}

k
a
D.B. Rowe
4

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Binomial
A random variable X has a discrete binomial probability mass
function, X~binomial(n,p) if

n!
P{ X i}
pi (1 p)ni
i !(n i)!

i 0,1..., n

where, 0 p 1 and n 1,2,3,... .

n = number of trials or times we repeat the experiment.
i = the number of successes out of n trials.
p = the probability of success on an individual trial.
D.B. Rowe
5

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Binomial
It can be shown that
n

n!
E[ X ] i
p i (1 p)ni
i 0 i !( n i )!

np
n

n!
var[ X ] (i np)
p i (1 p) ni
i !(n i )!
i 0

np(1 p)
2

D.B. Rowe
6

Marquette University

Binomial:

MSCS6020

n!
P( X i )
pi (1 p)ni
i !(n i)!

n=5; p=1/2;num=10^4;
x=binornd (n,p,num,1);
mean(x)
var(x)
hist(x,6)

3500

3000

2500

2000

1500

True

Simulated

2.5

2.4982

1.25

1.2455

Can also find and plot ECDF.

1000

500

0.5

1.5

2.5

3.5

4.5

D.B. Rowe
7

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Poisson
A random variable X has a discrete Poisson probability mass
function, X~Poisson() if

P{ X i}

i
i!

i 0,1..., n

where, 0 .
is called the intensity parameter

D.B. Rowe
8

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Poisson
It can be shown that

E[ X ]

i 0

i!

var[ X ]

(i )
i 0

i!

D.B. Rowe
9

Marquette University

Poisson:

P( X i )

MSCS6020

i e
i!

lam=5;num=10^4;
x=poissrnd(lam,num,1);
mean(x)
var(x)
hist(x,15)

1800
1600
1400
1200
1000
800

True

Simulated

5.0181

5.1069

Can also find and plot ECDF.

600
400
200
0

10

15

D.B. Rowe
10

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Geometric
A random variable X has a discrete geometric probability
mass function, X~geometric(p) if

P{X n} p(1 p)n1

n 1,2,...

where 0 p 1 and n 1,2,3,... .

X represents the first successful trial in a binomial experiment
with probability of success p.

D.B. Rowe
11

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Geometric
It can be shown that

E[ X ]

n 1
np
(1

p
)

n 1

var[ X ]

1/ p

2
n 1
(
n

1
/
p
)
p
(1

p
)

n 1

(1 p) / p 2

D.B. Rowe
12

Marquette University

Geometric:

MSCS6020

P( X n) p(1 p)n1

p=1/2;num=10^4;
x= geornd(p,num,1)+1;
mean(x)
var(x)
hist(x,12)

8000
7000
6000
5000
4000

True

Simulated

3000

1.9994

2000

2.0358

1000

Can also find and plot ECDF.

10

12

14

D.B. Rowe
13

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Negative Binomial
A random variable X has a discrete negative binomial
probability mass function, X~NB(p,r) if

n 1 r
n r
P{ X n}
p
(1

p
)
r 0,1..., n

r 1
where 0 p 1 and n 1,2,3,... .
X represents the number of trials needed to amass
a total of r successes from a binomial experiment
with probability of success p.

D.B. Rowe
14

Marquette University

MSCS6020

2.8 Some Discrete Random Variables

Negative Binomial
It can be shown that

n 1 r
nr
E[ X ] n
p
(1

p
)

1
nr

r/ p

n 1 r
n r
var[ X ] (n r / p)
p
(1

p
)

nr
r 1

r (1 p) / p 2

D.B. Rowe
15

Marquette University

MSCS6020

2.9 Continuous Random Variables

Uniform
A random variable x has a continuous uniform
probability density function, x ~uniform(a,b) if
1
f ( x)
ba

x [a, b]

1
b-a

where, a, b , a b.

D.B. Rowe
16

Marquette University

MSCS6020

2.9 Continuous RVs

Uniform

f(x)
1
b-a

The CDF of the continuous

uniform PDF is
0
x a

F ( x)
b a
1

xa

F(x)
1

x [ a, b]
xb
a

D.B. Rowe
17

Marquette University

MSCS6020

2.9 Continuous Random Variables

Uniform
It can be shown that
b

1
x
dx
ba
xa

ba
2

1
x
dx
ba
x a
2

(b a)2
12

(b a)

12
D.B. Rowe
18

Marquette University

1
f ( x)
ba

a=1;,b=2;,num=10^4;
x=a+(b-a)*rand(num,1);
mean(x)
var(x)
hist(x,15)

800

700

600

500
f(x|a,b)

Uniform:

MSCS6020

400

300

True

Simulated

1.5

1.4996

200

0.0833

0.0829

100

Can also find and plot ECDF.

1.1

1.2

1.3

1.4

1.5
x

1.6

1.7

1.8

1.9

D.B. Rowe
19

Marquette University

1
f ( x)
ba

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

ecdf(x)

F(x)

Uniform:

MSCS6020

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0
1

1.1

1.2

1.3

1.4

1.5
x

1.6

1.7

1.8

a=1;,b=2;,
y =cdf('unif',(1:.01:2),a,b)
plot((1:.01:2),y)

1.9

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

[F,xx]=ecdf(x);
stairs(xx,F,'LineWidth',2)

D.B. Rowe
20

Marquette University

MSCS6020

2.9 Continuous Random Variables

Normal
A random variable x has a continuous normal
probability density function, x ~normal(,2) if
0.4

f ( x | , )

where,

=10, =1
=11, =2
=12, =3

0.35

0.3

2 2

, x

0.25

f(x)

( x )2

2 2

and 0 .

0.2

0.15

0.1

E[ X ]

var( X )

0.05

10
x

12

14

16

18

20

D.B. Rowe
21

Marquette University

MSCS6020
0.2

0.16
0.14
0.12

f(x)

Normal:
The CDF of the continuous
normal PDF is

0.18

0.1
0.08
0.06

F ( x)

( t )2

2 2

0.04
0.02

dt ( x)

10
x

12

10
x

12

14

16

18

20

14

16

18

20

1
0.9
0.8
0.7

F(x)

0.6

No closed form analytic solution.

0.5
0.4
0.3
0.2
0.1
0

D.B. Rowe
22

Marquette University

MSCS6020

Normal:
f ( x)

2 2

1
0.9
0.8
=10, =1

0.7

=11, =2
=12, =3

0.6

F(x)

x=(0:.1:20)';
mu=[10,11,12];, sigma=[1,2,3];
figure(1)
hold on
for count=1:length(sigma)
y = normcdf(x,mu(count),sigma(count));
if count==1
plot(x,y,'r','LineWidth',2)
elseif count==2
plot(x,y,'g','LineWidth',2)
elseif count==3
plot(x,y,'b','LineWidth',2)
end
end
xlim([0 20])

( x )2

2 2

0.5
0.4
0.3
0.2
0.1
0

10
x

12

14

16

18

20

D.B. Rowe
23

Marquette University

Normal:

MSCS6020

f ( x)

( x )2
2 2

2 2

mu=11;,sigma=2;,num=10^4;
x=normrnd(mu,sigma,num,1);
mean(x)
var(x)
hist(x,35)

900

800

700

600

500

400

True

Simulated

11

11.0033

3.9321

300

200

100

Can also find and plot ECDF.

10

12

14

16

18

20

D.B. Rowe
24

Marquette University

Normal:

MSCS6020

f ( x)

( x )2
2 2

2 2

1
0.9

mu=11;,sigma=2;
y=normcdf((0:.01:25),mu,sigma);
plot((0:.01:25),y,r)
hold on
[F,xx]=ecdf(x);
stairs(xx,F,'LineWidth',2)

0.8
0.7

ecdf(x)

0.6
1

0.5
0.95

0.4
0.9

ecdf(x)

0.3

0.85

0.2
0.8

0.1
0.75

11

10

12

15

13

14
x

15

20

16

25

D.B. Rowe
25

Marquette University

MSCS6020

2.9 Continuous Random Variables

Exponential
A random variable x has a continuous exponential
probability density function, x~exponential() if
0
f ( x) x
e
E[ X ]

x0
x0

var( X )

where, 0 .
1

D.B. Rowe
26

Marquette University

x=(0:.1:15)';
lambda=[.25,.5,1,2,5];, beta=1./lambda;
figure(1)
hold on
for count=1:length(lambda)
y = exppdf(x,beta(count));
if count==1
plot(x,y,'r','LineWidth',2)
elseif count==2
plot(x,y,'g','LineWidth',2)
elseif count==3
plot(x,y,'b','LineWidth',2)
elseif count==4
plot(x,y,'m','LineWidth',2)
elseif count==5
plot(x,y,'k','LineWidth',2)
end
end
ylim([0 3]), xlim([0 15])

Matlab uses gamma dist def

f ( x) e x

3
=0.25
=0.50
=1.00
=2.00
=5.00

2.5

1.5

f(x)

Exponential:

MSCS6020

0.5

10

15

D.B. Rowe
27

Marquette University

MSCS6020

Exponential:

f ( x) e

x
0.8

The CDF of the continuous

exponential PDF is

f(x)

0.6

0.4

0.2

0
F ( x)
x
1

x0

x0

5
x

10

5
x

10

Note that there is a closed form solution!

0.8

F(x)

0.6

0.4

0.2

D.B. Rowe
28

Marquette University

MSCS6020

Exponential:
1
0.9

=.25
=.50
=1
=2
=5

0.8
0.7
0.6

F(x)

x=(0:.1:15)';
lambda=[.25,.5,1,2,5];, beta=1./lambda;
figure(1)
hold on
for count=1:length(lambda)
y = expcdf(x,beta(count));
if count==1
plot(x,y,'r','LineWidth',2)
elseif count==2
plot(x,y,'g','LineWidth',2)
elseif count==3
plot(x,y,'b','LineWidth',2)
elseif count==4
plot(x,y,'m','LineWidth',2)
elseif count==5
plot(x,y,'k','LineWidth',2)
end
end
xlim([0 15]) ,ylim([0 1])

f ( x) e x

0.5
0.4
0.3
0.2
0.1
0

10

15

D.B. Rowe
29

Marquette University

MSCS6010

Exponential: f ( x) e x
lambda=1;,num=10^4;
x=exprnd(1/lambda,num,1);
mean(x)
var(x)
hist(x,35)

True

Simulated

1.0018

1.0127

2500

2000

1500

1000

500

Can also find and plot ECDF.

5
x

10

D.B. Rowe
30

Marquette University

MSCS6010

Exponential: f ( x) e x
1
0.9
0.8

0.98

0.7
0.96

0.6
0.94

0.5

ecdf(x)

ecdf(x)

lambda=1;
y=expcdf((0:.01:15),1/lambda);
plot((0:.01:15),y, 'r')
hold on
[F,xx]=ecdf(x);
stairs(xx,F,'LineWidth',2)

0.4
0.3

0.92

0.9

0.2

0.88

0.1
x

10

15

D.B. Rowe
31

Marquette University

MSCS6020

Next Time
2.9 Continuous Random Variables
Memoryless Property
and Poisson Processes
2.10 Conditional Expectation and Variance

D.B. Rowe
32

Marquette University

MSCS6020

Homework 1:
Chapter 2: # 7, 13, 29, 31, 36, 37.

D.B. Rowe
33