You are on page 1of 60

RELI ABI LI TY ENGI NEERI NG UNI T

ASST4403
Lec t ur e 6: REVI EW OF MATH AND PROBABI LI TY
1
Learning outcomes Learning outcomes
Review some key math/calculus necessary for the y y
unit
Review and articulate concepts of probability Review and articulate concepts of probability
Demonstrate ability to apply laws of probability
Demonstrate familiarity with the concept of discrete
and continuous random variables
Articulate implication of probability distributions:
density/mass vs cumulative function y
Demonstrate ability to calculate characteristics of a
random variable: mean variance standard random variable: mean, variance, standard
deviation etc 2
REVIEW OF SOME MATH AND
CALCULUS
3
The mn counting rule the first counting
rule
For an operation that can be done m ways and a second
operation that can be done n ways, the two operations p y , p
can occur, in order, mn ways. It can be extended to
three or more operations
Example:
A t i t b t i b d f O ti A customer is to buy a certain brand of new car. Options
for the car include two different engines, five different
paint colours and three interior packages Assume all the paint colours and three interior packages. Assume all the
options are available. How many different cars could the
customer choose from?
Your answer = ? 4
Sampling from a population with
replacement the second counting rule p g
Sampling (choosing) n items from a
l ti f i N ith population of size N with
replacement would provide N
n
possibilities possibilities
Example
In a lottery six numbers are drawn
from the digits 0 through 9, with
replacement (digits can be reused).
How many different groupings of six
numbers can be drawn? numbers can be drawn?
Your answer = ?
5
Combinations - the second
counting rule
Sampling (choosing) n items from a Sampling (choosing) n items from a
population of size N without replacement
would provide would provide
possibilities
Example Example
A maintenance depot has 16 bearings and
three are to be selected randomly to be three are to be selected randomly to be
used for a maintenance task. How many
different combinations of bearings could different combinations of bearings could
be selected?
Your answer = ?
6
There is no mathematics used by physicists and
engineers without tears
Laurent Schwartz (1915-2002)
7
Puzzle Starter
Your sock drawer contains 6 blue socks, 8 black socks
and 10 brown socks. The room is dark and you can not
th l Wh t i th i i b f k see the colors. What is the minimum number of socks
you must take out of the drawer in order to be sure
you have one complete pair of socks? you have one complete pair of socks?
Your answer = ? Your answer ?
8
D i i l Derivation rules
Example: what is the derivate Example: what is the derivate
of
Your answer = ?
9
Integration rules Integration rules
Example: calculate the
integrals
Yo r ans ers ? Your answers = ?
10
Probability Basics
11
The Car and Goat Problem The Car and Goat Problem
Monty Hall, the master of ceremonies for the game show, Lets
Make a Deal confronts you with three closed doors one of which Make a Deal, confronts you with three closed doors, one of which
hides the car car of your dreams. Behind each of the other two doors,
however, is standing a smelly goat goat. You must choose a door to
win whatever is behind it. win whatever is behind it.
You decide on a door, and announce your choice.
Your host then opens one of the other two doors and reveals a Your host then opens one of the other two doors and reveals a
goat.
He then ask you whether you would like to switch your choice to
the unopened door that you did not originally choose.
Is it in your advantage to switch??????
Monty Halls game show:
Lets Make a
Deal
12
What is Probability? y
At the bottom of the theory of probabilities is
only common sense expressed in numbers only common sense expressed in numbers.
-- P.S. Laplace (1814)
Probability model: a mathematical tool for
describing random phenomena or quantifying
uncertainty uncertainty
Toss a fair coin
Roll a well-balanced dice
Draw a card from a well-shuffled deck
Lotto draw
13
B i C T i C i Basic Concepts: Tossing a Coin
Experiment: a situation or planned study Experiment: a situation or planned study
involving chance or uncertainty
Tossing a coin Tossing a coin
Outcome: the result of a single trial of an
experiment experiment
Heads, Tails
Sample space (usually denoted by S): the set of all Sample space (usually denoted by S): the set of all
possible outcomes of interests
Toss a coin once: S={H,T} { }
Toss a coin three times, record results in order :
S={TTT, TTH, THT, THH , HTT, HTH, HHT, HHH}
Toss a coin three times, record number of heads: ,
S={0, 1, 2, 3}
14
Basic Concepts: Event
Event
A subset of sample space S A subset of sample space S
A particular outcome or set of outcomes
Toss a coin once: {H}, {T}
T i th ti d Toss a coin three times and
Exactly 2 heads are observed: Event A={HHT, HTH, THH}
At least 2 tails: Event B={TTT, TTH, THT, HTT} { }
Probability of an event
P(A) = 3/8 ( )
P(B) =1/2
15
Basic Concepts: Probability
Classical definition of probability (equiprobable)
The number of ways event A can occur The number of ways event A can occur
P(A)=
The total number of outcomes in sample space S
n
A
S
n
n
=
( ) A ( )
P(A)=
n A
n
Relative frequency definition of probability
Can probability be a negative number?
16
Solution to the Car and Goat Solution to the Car and Goat
Problem (cont)
The NO-SWITCH sample space is:
S ={(1 2 1 W) (1 3 1 W) (2 3 2 L) (3 2 3 L)}
( )
Sno-switch={(1,2,1,W), (1,3,1,W), (2,3,2,L),(3,2,3,L)}
Similarly,
Event You win ={(1 2 1 W) (1 3 1 W)} Event You win {(1,2,1,W), (1,3,1,W)}
Probability(You win)=1/3
{(2 3 2 ) (3 2 3 )} Event You lose ={(2,3,2,L),(3,2,3,L)}
Probability(You lose)=1/3+1/3=2/3
Conclusion: SWITCH is Better!
17
http://www.remote.org/frederik/projects/ziege/empirie.ht
ml
http://www.math.uni-
konstanz.de/~kohlmann/ftp/applets/MontyHallGame.html
18
http://math ucsd edu/ crypto/cgi http://math.ucsd.edu/~crypto/cgi-
bin/MontyKnows/monty2?1+9130
19
Birthday problem
What do you think the probability that at least two
l h th bi thd ? M people among us have the same birthday? More
than 50%, less than 50%?
How many people do you think it needs before the
probability exceeds 50%?
20
Laws of probability
21
Class question:
Must we use probability?
Do we have any choice? Do we have any choice?
22
Where the cannon ball hit is
f a matter of accuracy
Where the roulette ball stops
i f d is a matter of randomness
23
Axioms of Probability
1. For any event A: 0P(A)1
2 P(S) = 1 2. P(S) = 1
3. Addition rule: if events A, B mutually exclusive,
P(A U B) = P(A) + P(B)
Complement rule: P(A

)=1-P(A) Complement rule: P(A) 1 P(A)


Proof: P(A U A) = P(S)
P(A) + P(A) =1 P(A) P(A ) 1
General addition rule:
P(A U B) = P(A) + P(B) P(AB) P(A U B) = P(A) + P(B) P(AB)
Proof: A U B = A U (BA)
B = (AB) U (BA)
C diti l P b bilit Conditional Probability
P(A) = P(A|S)
Conditional probability:
( )
( | )
( )
P B A
P B A
P A
=

Conditional probability:
( )
S
A
B
25
Conditional Probability
P(A), P(B) and P(AB) are absolute terms with respect
to S
P(B|A) is relative to the reduced sample space A
S
( )
( | )
( )
P B A
P B A
P A
=

A
B
A
B
26
Conditional Probability
( )
( | )
( )
P A B
P A B
P B
=

( | ) ( | ) P A B P B A =
S
( )
( | )
P A B
P A B

( ) P B
( | ) ( )
( | ) ( )
P A B P A B
P B A P A B
>
>

S
( )
( | )
( )
P A B
P B
=

( | ) ( ) P B A P A B >
A
B
27
Conditional Probability: Example 1
Someone tosses a dice, covers it up and tells you that , p y
the number is greater than 3.
Now what is the probability that the number is even? Now what is the probability that the number is even?
A={the number is great than 3}
B={the number is even} B {the number is even}
P(A B) ({4, 6}) 2
P(B|A)=
P
P(B|A)=
P(A) ({4, 5, 6}) 3 P
= =
28
Conditional Probability: Example 2
Two cards drawn (without replacement) from a deck: Two cards drawn (without replacement) from a deck:
A={first card drawn is a Jack}
B={second card drawn is a Jack} B={second card drawn is a Jack}
Conditional probability: P(B|A)=
Unconditional probability: P(B) =
3/ 51
4/52=1/13
( | ) ( ) P B A P B =
Indicating that B depends on A
What if P(B|A) = P(B)?
29
Independent Events
Events A and B are independent if and only if Events A and B are independent if and only if
P(B|A) = P(B) and
P(A|B) = P(A) ( | ) ( )
Example: Two cards drawn (with replacement) from an
ordinary deck:
A={first card drawn is a diamond}
B={second card drawn is a club}
P(B|A)=13/52=1/4 P(B|A) 13/52 1/4
P(B) = 13/52=1/4
30
Multiplicative Rule
( )
( | )
P B A
P B A =

( )
( | )
P A B
P A B =

M lti li ti l
( | )
( )
P B A
P A
=
( | )
( )
P A B
P B
=
Multiplicative rule:
P(AB)=P(A|B)P(B)=P(B|A)P(A)
If P(B|A) = P(B), P(A|B) = P(A), the above equation
becomes P(AB) = P(A)P(B)
Events A and B are independent if and only if P(AB) =
P(A)P(B)
31
Multiplicative Rule: Example Multiplicative Rule: Example
Suppose 5 defective light bulbs were pp g
inadvertently packed in a box with 20 good
ones. Someone randomly selected 2 light
bulbs from the box (without replacement) bulbs from the box (without replacement).
What is the probability that both of them are
defective? defective?
A={First light bulb removed is defective}
B={Second light bulb removed is defective} B {Second light bulb removed is defective}
P(A) = 5/25=1/5
P(B|A) = 4/24=1/6
P(AB) P(B|A)P(A) 1/6 1/5 1/30 P(AB) = P(B|A)P(A) = 1/6 x 1/5=1/30
32
P(B)=? Problem Revisited
Two cards drawn (without replacement) from a deck: Two cards drawn (without replacement) from a deck:
A={first card drawn is a Jack}
B={second card drawn is a Jack} B={second card drawn is a Jack}
P(B|A)=3/51
P(B) = ?
A
B
P(B|A) A
B
P(B|A)
P(A)
P(B|A)P(A)
P(A)
A
B
P(B|A)
P(B|A)P(A)
( )
P(B|A )P(A )
33
Unconditional Probability P(B)=?
P(B) = P(BA) + P(BA)
P(B) = P(B|A)P(A) + P(B|A)P(A) ( ) ( | ) ( ) ( | ) ( )
P(B) = 3/51 x 4/52 + 4/51 x 48/52=1/13
A
B
P(B|A)
P(B|A)P(A)
A
A
P(A)
P(B|A)P(A)
A
P(A)
A
B
P(B|A)
P(B|A)P(A)
B
34
The Law of Total Probability (LTP) The Law of Total Probability (LTP)
P(B) = P(B|A)P(A) + P(B|A)P(A)
Example: Australian Open 2007 Final
Federer through to the final g
Semi-final Baghdatis vs. Hewitt
P{Federer beats Baghdatis}=2/3
P{Federer beats Hewitt}=1/2 P{Federer beats Hewitt}=1/2
P{Baghdatis beats Hewitt}=3/7
What is the probability that Federer wins the final? p y
35
LTP: Example 1
B={Federer wins final}
A= {Baghdatis beats Hewitt in semi}
A= {Hewitt beats Baghdatis in semi} A = {Hewitt beats Baghdatis in semi}
P(A)=3/7, P(A)=4/7
P(B|A)=2/3, P(B|A)=1/2
B th L f T t l P b bilit (LTP) By the Law of Total Probability (LTP):
P(B) = P(B|A)P(A) + P(B|A)P(A)
=2/3
.
3/7 +
.
4/7 = 4/7
36
The Law of Total Probability (LTP)
If A
1
, A
2
, ...A
n
are mutually exclusive and exhaustive
t i S th f th t B i S events in S, then for any other event B in S,
( ) ( | ) ( ) ( | ) ( ) P B P B A P A P B A P A = + +
1 1
( ) ( | ) ( ) ( | ) ( )
( | ) ( )
n n
n
P B P B A P A P B A P A
P B A P A
= + +
=

1
( | ) ( )
i i
i
P B A P A
=
=

A
1
A
n
B
AA
2
A
3
37
LTP: Example 2 LTP: Example 2
An electrical appliance retailer sells three brands of TVs,
d th i k t h ti l 50% 30% and their market shares are respectively 50%, 30%,
20%. It is estimated that 2.5% of brand 1s, 2% of brand
2s and 1% of brand 3s TV will be returned by 2 s, and 1% of brand 3 s TV will be returned by
customers for repair while under the one-year warranty
offered by manufacturers. y
Q1: What is the probability that a random customer
purchases a brand 1s TV and has to return it to the purchases a brand 1 s TV and has to return it to the
retailer for repair covered by the one-year warranty?
Q2: What is the probability that a TV sold by the retailer Q2: What is the probability that a TV sold by the retailer
will be returned for warranty repair work?
Q3: If a TV is returned for warranty repair work, what is
the probability that it is brand 1, brand 2, brand 3?
38
LTP E l LTP: Example 2
A {TV sold is brand i} i 1 2 3 A
i
={TV sold is brand i}, i=1,2,3
B={TV requires warranty repair work}
P(A
1
)=0.5, P(A
2
)=0.3, P(A
3
)=0.2
P(B|A ) 0 025 P(B|A ) 0 02 P(B|A ) 0 01 P(B|A
1
)=0.025, P(B|A
2
)=0.02, P(B|A
3
)=0.01
Q1: P(BA
1
)= P(B|A
1
) P(A
1
)=0.0125
Q2: P(B)= P(B|A
1
) P(A
1
) + P(B|A
2
) P(A
2
) +
P(B|A ) P(B|A
3
) P(A
3
)
=0.0125 + 0.02x0.3 + 0.01x0.2
=0.0205
Q3: P(A
1
|B)= P(BA
1
)/ P(B)=0.61
39
Bayes theorem
Thomas Bayes (1702-1761)
40
Bayes Theorem
If A
1
, A
2
, ...A
n
constitute a partition of S with P(A
i
)>0, for
i=1 n then for any other event B in S with P(B)>0, i 1,n, then for any other event B in S with P(B)>0,
( ) ( | ) ( ) P A B P B A P A ( ) ( | ) ( )
( | )
( )
( | ) ( )
k k k
k
n
P A B P B A P A
P A B
P B
P B A P A
= =

1
( )
( | ) ( )
1
i i
i
P B A P A
k
=

1, , k n =
41
E l i Example question:
A = test person has disease 0 3% A = test person has disease. 0.3%
of the population has the disease,
i.e., P(A)=0.003 i.e., P(A) 0.003
B = test result is positive
P(B|A) = 0.95
P(B|A) = 0 05 where A = person P(B|A ) 0.05, where A person
has no disease
What is P(A|B)? That is given that What is P(A|B)? That is, given that
a person tests positive, what is the
probability that he/she has the probability that he/she has the
disease
42
Answer:
Using the formula with A
1
=A, B = B, we have
P(A|B)=P(B|A)P(A)/[P(B|A)P(A)+P(B|A)P(A)]
= 0 95*0 003/(0 95*0 003+0 05*0 997) = 0 05 = 0.95*0.003/(0.95*0.003+0.05*0.997) = 0.05
Do you notice the difference between P(A) and
P(A|B)? ( | )
P(A)=0.003 while P(A|B) = 0.05, more than 10 times!
Why?
43
Random variables and probability Random variables and probability
distributions
44
Random Variable (r.v.)
A random variable (r.v.) associates a number with each
outcome in a given sample space. g p p
Example: tossing a coin
S={H,T}
Assign 0 for H, 1 for T, then X=0, or X=1
P(X=0)=0 5 P(X=1)=0 5 P(X=0)=0.5, P(X=1)=0.5
Example: tossing a coin twice
S {HH HT TH TT} S={HH,HT,TH,TT}
Count the number of Hs, then X=0,1,2
P(X=0)=0 25 P(X=1)=0 5 P(X=2)=0 25 P(X=0)=0.25, P(X=1)=0.5, P(X=2)=0.25
45
Discrete r.v.
Can assume a finite number of values; or Can assume a finite number of values; or
A countable infinity of values
Example: Toss a coin until a T is obtained, then
S={T, HT, HHT, HHHT, }
Define X=the number of throwing, then
X=1, 2, 3, 4,
d t d b l tt X Y r.v. denoted by upper case letters: X,Y,
A particular value it can take on denoted by
corresponding lower case letters: x,y,
46
Continuous r.v.
A r v is continuous if its set of possible values is an A r.v. is continuous if its set of possible values is an
entire interval of numbers.
T i l l Typical examples:
A persons height and weight
Lif ti f t f t hi l Lifetimes of a component of a motor vehicle
Quantities measured in units of mass, length, time,
temperature and so on p
47
Discrete Probability Distribution y
The probability distribution or probability mass function
(p m f ) of a discrete r v X is defined by ( ) P(X ) for (p.m.f.) of a discrete r.v. X is defined by p(x)=P(X=x) for
every possible value x.
(1) 0p(x)1
(2)
x
p(x) = 1
p(x)
( )
x
p( )
(3) P(X=x)=p(x)
Example: toss a coin
twice and count Hs
x
t ce a d cou t s
x 0 1 2
x
0 1 2
x 0 1 2
p(x) 0.25 0.5 0.25
48
Cumulative Distribution Function
(CDF)
The cumulative distribution function (c d f ) F(x) of a The cumulative distribution function (c.d.f.) F(x) of a
discrete rv X with pmf p(x) is defined for every number x
bby
F(x)=P(Xx)=
y:yx
p(y)
p(x)
F(x)
1 0
( )
0.75
1.0
0.25
x
x
0 1 2
0 1 2
49
Exercise
Toss two dice. Define rv X as the sum of numbers of the
two dice. Find p(x) and F(x).
x p(x)=P(X=x)
F(x)=P(Xx)=
y:yx
p(y)
1 P(X1)=0 0
2 P(X=2)=1/61/6=1/36 1/36
3 P(X=3)=2/36=1/18 1/12
4 P(X=4)=3/36=1/12 1/6 4 P(X 4) 3/36 1/12 1/6
5 P(X=5)=4/36=1/9 5/18
6 P(X=6)=5/36 5/12
7 P(X=7)=6/36=1/6 7/12
8 P(X=8)=5/36 13/18
9 P(X=9)=4/36=1/9 5/6 9 P(X=9)=4/36=1/9 5/6
10 P(X=10)=3/36=1/12 11/12
11 P(X=11)=2/36=1/18 35/36
12 P(X=12)=1/36 1
13 P(X13)=0 1
50
Exercise: pmf & cdf
0.18
1
0.12
0.14
0.16
0.8
0.06
0.08
0.1
0.4
0.6
0.02
0.04
0.2
0 2 4 6 8 10 12 14
0
0 2 4 6 8 10 12 14
0
51
Measures of Location and Dispersion
Measure of location (typical value)
M Mean
Median
Measure of dispersion (spread about the typical value) Measure of dispersion ( spread about the typical value)
Variance
Standard deviation Standard deviation
52
53
Discrete distribution
The mean is (p(x) is the probability mass function)

= ) (x xp
The standard deviation

) ( p
| |

= ) ( ) (
2
x p x o | |

= ) ( ) ( x p x o
54
Continuous r.v.
A r v is continuous if its set of possible values is an A r.v. is continuous if its set of possible values is an
entire interval of numbers.
T i l l Typical examples:
A persons height and weight
Lif ti f t f t hi l Lifetimes of a component of a motor vehicle
Quantities measured in units of mass, length, time,
temperature and so on p
55
Probability Density Function (PDF) y y ( )
A probability distribution or probability density A probability distribution or probability density
function (p.d.f.) of a continuous r.v. X is a
function f
X
(x) such that for any number ab the function f
X
(x) such that for any number ab the
probability that X assumes a value in the interval
[a, b] is given by [a, b] is given by
f
X
(x)
{ } ( )
b
b f d
}
{ } ( )
X
a
P a X b f x dx s s =
}
a b x
56
Properties of PDF Properties of PDF
1.
0 ( ) 1
X
f x s s
2.
( ) 1
X
f x dx

=
}
3.
{ } ( )
b
X
a
P a X b f x dx s s =
}
f
X
(x)
{ } 0 P X x = =
{ } { } P a X b P a X b s s = < s = { } { }
{ } { }
P a X b P a X b
P a X b P a X b
s s < s
s < = < <
{ } ( )
X
P X x f x = =
a b x
57
Cumulative Distribution
Function (CDF)
Th l ti di t ib ti f ti (CDF) F( ) f The cumulative distribution function (CDF) F(x) of a
continuous rv X with pdf f
X
(x) is given by
( ) { } ( )
x
X
F x P X x f t dt

= s =
}
0.8
0.9
1
F(x)
f(x)
f(x)
0.4
0.5
0.6
0.7
0
0.1
0.2
0.3
F(x)
x
F(x)
0
x
x
x
58
P i f CDF Properties of CDF
( ) dF x ( )
(1) ( )
X
dF x
f x
dx
=
(2) lim ( ) 0 and lim ( ) 1
x x
F x F x
+
= =
(3) If then ( ) ( ) x x F x F x < s
f
X
(x)
1 2 1 2
(3) If then ( ) ( ) x x F x F x < s
1 2 2 1
(4) ( ) ( ) ( ) P x X x F x F x < s =
f (x)
F(x
2
)-F(x
1
)
f
X
(x)
F( )
F(x
1
)
F(x
2
)
x
0
x
1
x
2
59
Continuous distribution Continuous distribution
For a continuous rv X whose pdf is defined by f
X
(x), its
mean is given by g y
( )
X X
xf x dx

=
}
Its variance var(X) or
X
2
is given by
( )
X X
xf x dx

}
Its variance, var(X) or
X
is given by
2 2
( ) ( ) ( ) X f d

}
2 2
var( ) ( ) ( )
X X X
X x f x dx o

= =
}
60

You might also like