Engineering Probability and

Statistics, IE6200
By: Professor Nasser Fard
Fall 2013
Lecture Two

Outline

Conditional Probability
Independent Events
Bayes Theorem
Random Variables (R.V.)
Expected Value and Variance
Discrete R.V.
Continuous R.V.

9/9/2013

Conditional Probability
We use probabilities because we are uncertain
about the exact outcome of an experiment.
New information can be available, which could
make us modify our belief (probability)
Conditional Probability, P(A|B), is the name
given to the new belief after receiving the new
information.
9/9/2013

Conditional Probability (cont.)

The probability of an event B occurring when it is known
that some event A has occurred is called a Conditional
Probability, and is denoted by P(B|A)
The conditional probability of B, given A, is defined by
P( A B)
P( B A)
P( A)
A

9/9/2013

P( A) 0

for
A

AB

Example: Rolling Two Dice

A is the event that the total number of two dice is 8,
A = {(2,6), (3,5), (4,4), (5,3), (6,2)};
P(A) = n / N = 5/36;
B is the event that two numbers on the dice are equal,
B = {(1,1), (2,2), (3,3), (4,4), (5,5), (6,6)};
P(B) = n / N = 6/36 = 1/6;
AB = {(4,4)}; P(AB) = 1/36;
P(B|A) = P(AB) / P(A) = 1/5
P(A|B) = P(AB) / P(B) = 1/6

9/9/2013

Independent Events
Two events A and B are independent if and only if
P(B|A) = P(B)
or
P(A|B) = P(A)
Otherwise, A and B are dependent.
This means additional knowledge of B does not change
the probability of A happening, and vice versa
Two events A and B are independent if and only if
P(AB) = P(A) . P(B)
9/9/2013

Independent Events

Events A, B and C are mutually independent if

and only if the following two conditions hold:
1. They are pairwise independent; that is:
P(AB)=P(A)P(B)
P(AC)=P(A)P(C)
P(CB)=P(C)P(B)

2. P(AB C)=P(A)P(B) P(C)

9/9/2013

Multiplicative Rules
If in an experiment the events A and B can both
occur, then :
P(AB) = P(A) P(B|A)

Transformation of conditional probability

definition.
P( A B) P(A) P(B | A)
P( B A)

P( A)
P( A)
9/9/2013

Multiplicative Rules (cont.)

The Multiplication rule extend to 3 events:
P(ABC) = P(AB)P(C | AB) = P(A) P(B | A) P(C | AB)
Similarly, it extends to n events:
P(A1 A2 An) = P(A1 An-1)P(An|A1 An-1)

= P(A1) P(A2|A1) P(A3|A1 A2) P(An| A1 An-1)

9/9/2013

Total Probability Theorem

S
B1

B2

A
AB1
AB2
AB3
B3

If B1, B2 and B3 constitute a partition

of the sample space S, then P(A) is:
P(A) = P(AB1) + P(AB2) +
P(AB3)
Using the multiplicative rules,
P(ABi) = P(Bi) P(A|Bi)
for i=1,2,3

Thus : P(A) = P(B1) P(A|B1) + P(B2) P(A|B2) + P(B3) P(A|B3)

9/9/2013

10

Example: Defective Product

B1 P(A|B1)=0.02

P(B1)=0.3

P(B2)=0.45

B1: the product is made by machine B1

P(A|B2)=0.03

B2

B2: the product is made by machine B2

B3: the product is made by machine B3

P(B3)=0.25
P(A|B3)=0.02

B3

A: the product is defective

P(A) = ?

P(A) = P(B1) P(A|B1) + P(B2) P(A|B2) + P(B3) P(A|B3)

= (0.3)(0.02) + (0.45)(0.03) + (0.25)(0.02)
= 0.0245
9/9/2013

11

Bayes Theorem
If the events B1, B2 , Bk constitute a partition
of the sample space S (they are exhaustive and
mutually exclusive events), then for any event A
in S that P(A) 0,
PBr A
PBr P A | Br
PBr | A
k
for r 1,2,...k
P A
PBi P A | Bi
i 1

9/9/2013

12

Example: Defective Product (cont.)

A: the product is defective
B1 P(A|B1)=0.02

P(B1)=0.3

B1: the product is made by machine B1

B2: the product is made by machine B2

P(B2)=0.45

P(A|B2)=0.03

B2
P(B3)=0.25
P(A|B3)=0.02

B3

B3: the product is made by machine B3

Given a product is defective, what is

the probability that it was made by
machine B3?

P(B3|A) = ?
PB3 | A

PB3 P A | B3
PB1 P A | B1 PB2 P A | B2 PB3 P A | B3

(0.25)(0.02)
(0.3)(0.02) (0.45)(0.03) (0.25)(0.02)
0.005 10

0.0254 49

9/9/2013

13

What is a Random Variable?

In many experiments it is required to allocate a
numerical description to the outcome.
Example :
A sample space giving detailed description when
three electronic components test results, where N
is non defective and D is defective is
S={NNN,NND,NDN,DNN,NDD,DND,DDN,DDD}
9/9/2013

14

Random Variable
Definition :
A Random Variable is a function that associates a
real number with each element in the sample space.
It is represented by capital letter, say X.

9/9/2013

15

Random Variable (cont.)

Example: In the previous example the R.V. (Random
Variable) X assumes the value 2 for all elements in the
subset E = { DDN,DND,NDD} of the sample space.

Example: Rolling a pair of dice. Let X be the random

variable corresponding to the sum of the dice on a roll. If
we think of the sample points as a pair (i, j), where
i = value rolled by the first dice
j = value rolled by the second dice,
then we can have: X(s) = i+j
9/9/2013

16

Discrete Random Variable

A random variable X is said to be discrete if the
number of possible values of X is finite, like
SX={x1,x2,,xk}

or at most, an infinite sequence of different values

that can be put in one-to-one correspondence with
positive integers, like SX={x1,x2,}

9/9/2013

17

Discrete Probability Distributions

A discrete R.V. assumes each of its values with a certain
probability.
Definition : The set of ordered pairs (x, f(x)) is a probability
function, probability mass function, or probability
distribution of the discrete random variable X if, for each
possible outcome x,
1) f (x) 0 x S;
2) f ( x) 1;
xS

3) P (X A) f ( x), where A S
x A

9/9/2013

18

Discrete Random Variable (cont.)

Example: Consider a random variable X that
takes on values 1, 2, 3, and 4 with probabilities
1/6, 1/3, 1/3, and 1/6, respectively
p(x)
0.35
0.30
0.25
0.20
0.15
0.10
0.05
9/9/2013

0.00

19

Cumulative Distribution
Definition : The cumulative distribution F(x) of a
discrete random variable with probability
distribution f(x) is

F(x) P(X x) f(t) for - x

tx

9/9/2013

20

Discrete Probability Distribution

Functions (pdf s)
X

P(X)

1/6

1/6

1/6

2/6

1/6

3/6

1/6

4/6

1/6

5/6

1/6

6/6

9/9/2013

F(X)=P(Xx)

P (x) = 1/k,
where k is the total
number of outcomes
This probability
distribution is also
known as Discrete

Uniform.

21

Mathematical Expectation
If f(x) is the p.m.f. of the random variable X of the
discrete type with space S and if the summation

u(x)f(x)

which is some times written

xS

u(x)f(x)
S

exists, then the sum is called Mathematical

Expectation or the expected value of the
function u(X), and it is denoted by E[u(X)]. That
is Eu ( X ) u(x)f(x)
xS

9/9/2013

22

Mathematical Expectation (cont.)

Mathematical expectation has the following
properties:
1. If c is a constant, E(c)=c
2. If c is a constant, and u is a function,
E[c.u(x)]=c E[u(x)]
3. If c1 and c2 are constants and u1 and u2 are
functions, then
E[c1u(x) +c2u(x)]=c1E[u(x)]+c2E[u(x)]
NOTE: Expectation is a linear operator
9/9/2013

23

Mean and Variance of a

Random Variable
Mean is a measure of the central tendency, it determines
the Center of gravity of the distribution.
The mean or expected value or average of X is defined by

EX xf(x)
xS

The variance is a measure of the dispersion or spread of a

random variable about its mean.
The variance is defined by:

2 var( X ) E[( X ) 2 ] E[( x 2 )] ( E[ X ]) 2

9/9/2013

24

 E(aX+b)=aE(X)+b=a+b
Var(aX+b)=E[(aX+b)-(a+b)]2
= E[aX-a]2
=a2 E[X- ]2 =a2 2
Mean and Variance of a Sum
If X1, X2,, Xn are r.v.s with means i and variances i2,
and Y= X1+X2++Xn , then

E[Y]= 1+ 2++n and V[Y]= 12+ 22 ++ n2

If Y= a1X1+a2X2++anXn , then

E[Y]=a11+a22++ann and
V[Y]=a12 12+a22 22 ++ an2 n2
9/9/2013

25

9/9/2013

26

Examples
1. E[5X+12]=5E[X]+12
V[5X+12]=25V[X]
2. E[X1]=1, E[X2]=-1
V[X1]=5, V[X2]=5, X1 and X2 are independent
a. E[X1-X2]= E[X1]- E[X2]= 1-(-1)=2
b. V[X1-X2]= V[X1]+V[X2]= 5+5=10
Note: V[-X2]=(-1)2 V[X2]=V[X2]
9/9/2013

27

Sample Mean
Consider we have n observation of the random variable
X, say x1,x2,,xn
Create a probability distribution by putting weight 1/n on
each of these values.
Sample mean is the arithmetic mean of the observations
and is denoted by x

1 n
x xi
n i 1

When is unknown,
9/9/2013

x is considered as an estimate of .
28

Sample Variance
The variance of this empirical distribution is
obtained as:
n

n
1
1
(x i - x) 2 ( ) ( xi x ) 2
n
n i 1
i 1

This is not called sample variance. But s 2which is

defined as below is sample variance
n
n
1

2
s2

(x
x
)

n 1 i 1
n 1

9/9/2013

29

Sample Standard Deviation

S2 is a better estimate for the unknown 2 than .
Sample Standard Deviation is denoted by:

s s 0
2

Sample standard deviation gives a measure of

how dispread the data are from the sample mean.

9/9/2013

30

Continuous Random Variables

Recall that a random variable X is simply a function from
a sample space S into the real numbers.
The random variable is continuous if the range of X is
uncountably infinite and the function has some specific
characteristics. (See next slide)
Typically an uncountably infinite range results from an X
representing a physical measuremente.g., the position,
size, time, age, flow, volume, or area of a characteristic.
9/9/2013

31

Probability Density Function (p.d.f)

Definition : The
function f (x) is a probability
1.
density function for the continuous random
variable X, defined over the set of real numbers
R, if

1. f ( x) 0 , for all x in S
2. f ( x)dx 1
S
b

3. P(a X b) f ( x)dx
a

9/9/2013

32

Basic Properties of a continuous p.d.f

The probability of X taking on a single value a is
always 0.
a

P( X a) P(a X a) f ( x)dx 0
a

and also
P(a X b) P(a X b) P(a X b) P(a X b)

Note that none of the above statements is true

for discrete random variables.
9/9/2013

33

Examples
Example 1: Define random variable X with range
[0,2] and its pdf by f(x)=1/2 for all x between 0
and 2 and f(x)=0 for all other values of x. Then
we have:

9/9/2013

1
f ( x)dx 1/ 2dx x 1 0 1
0
2 0
2

34

Examples (cont.)
Example 2: Lets X defines the life of a light bulb
and imagine its pdf has been defined as below:
1 x / 2
e
, x0
f ( x) 2

otherwise
0,

So, we can easily calculate values like:

1 3 x / 2
P(2 x 3) e
dx 0.14
2
2
9/9/2013

35

Cumulative Distribution Function (c.d.f)

Definition : The Cumulative Distribution
Function (c.d.f) of a continuous random variable
X with density function f (x) is denoted by F (x)
and is defined as:
x

F ( x) P( X x)

f (t )dt for - x

9/9/2013

36

Properties of a Cumulative Distribution Function

Example
The properties of Dist. f
1) lim F ( x) 1;
x

2) lim F ( x) 0;
x

3)The function is non-decreasing, that is if b a,

then F (b) F ( a);for continous R.V . X
P ( a X b) P (a X b) P (a X b )
P (a X b) F (b) F (a )
d
also f ( x)
F ( x)
dx
9/9/2013
37

Example
The life time of a light bulb has c.d.f:
1 x t / 2
F ( x) e dt 1 e x / 2
2 0

and also :
P(2 X 3) F (3) F (2) (1 e(3/ 2) ) (1 e1 ) 0.145

9/9/2013

38

Expectation and Variance of a Continuous

R.V.
When X is a continuous r.v, its Mean is
defined as:

E( X )

xf ( x)dx

And, it Variance is :

Var ( X ) ( x ) f ( x)dx
2

9/9/2013

39

Expectations and Variance (light bulb example)

We have mean of life of the light bulb as:

1 x / 2
x / 2
E ( X ) xe dx xe
e x / 2 dx 2
0
2 0
0

To compute variance of X, we need to compute E(X2):

1
x / 2
2 x / 2
2
E ( X ) x e dx x e
e x / 2 dx 8
0
2 0
0
2

Thus, we have the variance and standard deviation of the

bulbs life as:
V ( X ) 8 22 4
9/9/2013

V (X ) 2

40

9/9/2013

41