Probability and Counting:

Sampling Table (n objects, k draws)

Bayes Rule

Inclusion-Exclusion

Binomial Coefficient

Laws of Independence

LoTP

Events A and B are conditionally independent given C if:

Vandermondes Identity (group of k with j men and k-j women)
k
m+n =
m n

k
k j
j=0 j
Discrete Random Variables:
PMF
CDF

( ) ( )( )

Expected Values

Indicators

LOTUS

1.

E ( g ( X ) )= g ( x ) P ( X =x)
x

2.

I A k =I A

I A =1I A
c

3. I A B=I A I B
4. I A B =I A + I BI A I B
Variance

Linearity

Fundamental Bridge

PDF

Var ( X +c )=Var ( X)
Var ( cX )=c 2 Var ( X )
Var ( X ) 0
Continuous Random Variables:
CDF

LOTUS

Geometric
n
n+1
1x
k=0 1x

Series
Taylor

n
nx ! =e x , for all x
n=0

Binomial Theorem
n
n
( x+ y ) = n x k y nk
k=0 k

()

Bernoulli (Bern) Distribution

One trial that can result in success or failure where probability of success is p.
Binomial (Bin) Distribution
n independent Bernoulli trials with probability of success p on every trial. Counting
number of successes. Equivalent to sum of independent Bernoulli trials with the same p.
Geometric (Geom) Distribution
Independent trials with probability of success p, counting the number of failures until the
first success (not counting the first success). A memoryless distribution (at any point, we
have the same expected wait time for a success).
First Success (FS) Distribution
Independent trials with probability of success p, counting the failures until the first
success AND the first success. Equivalent to Geom ( p )+ 1 .
Negative Binomial (NBinom) Distribution
The number of failures before the rth success, where each trial has probability p of
success. NBinom(1, p) = Geom(p).
Hypergeometric (HGeom) Distribution
The number of successes in n draws from a population of b undesired objects and w
desired objects (picking w is parked as a success, b as a failure) without replacement.
Poisson (Pois) Distribution
There are many possible events A1, A2, An, each with a small pj of occurring (note the
n

events dont have to be independent since p is small). Define parameter

= p j . Can
j=1

be used to approximate the number of events that occur.

Uniform (Unif) Distribution
Generates a completely random variable in segment between parameters a and b.
Normal (N) Distribution
CDF:
Birthday Problem
k people, each persons birthday is equally likely to be any of 365 days, peoples
birthdays are independent. Find P(2 or more people have the same birthday).
365364(365k + 1)
P(no birthday match) =
365k
365364(365k +1)
P(at least 1 match) = 1 P(no match) = 1
365 k
Matching Problem
Shuffled deck of n cards labeled 1 to n. Flip cards saying 1 to n as you go. You win if you
say the same number as you draw. Find P(Win).
Ai = event that ith card has number i. We want P( A1 A 2 A n)
1
1
1
P ( Ai A j A k )=
P ( A i )= , P ( A i A j ) =
,
n
n (n1)
n(n1)(n2)

 i=1
1
1
=Taylor Series=1
n!
e
n +1
1
1 1
1
=1 + +
n!
2! 3 !
n
n
2
3
n
P ( n Ai ) =
+
+
n n ( n1 ) n ( n1 ) ( n2 )
1 n +1

()

()

Random Coin Example

One fair coin and one biased coin that lands on Heads with probability 3/4. Pick one coin
at random and flip 3 times for 3 Heads. Find P(picked fair coin). Let A = event that 3
Heads were flipped. Let F = event that you picked the fair coin.
1 3 1
2 2
P ( A|F ) P ( F )
P ( F| A ) =
=
=0.23
3
3
c
c
P ( A|F ) P ( F ) + P ( A|F ) P ( F ) 1 1 + 3 1
2 2
4 2
Winter Girl Problem
Family has 2 children. Find P(both girls|at least one girl born in winter). Gender and
seasons all equally likely, and independent.
at least one winter girl
P
P(both girls at least one winter girl )
P ( bothgirls|at least one winter girl )=

1
1
P ( both girls at least 1 winter girl )=P ( both girls at least 1 winter child )= P ( at least 1 winter child )= ( 1
4
4
Testing for a Disease (Sensitivity and Specificity)
Disease affects 1% of the population. Patient tests positive. Let D be the event patient has
the disease, and T be the event that he tests positive. Test is 95% accurate. Find P(D|T)
Sensitivity=P ( T|D )=0.95 ,
Specificity=P ( T c|Dc ) =0.95
P ( T|D ) P( D)
P ( T| D ) P(D)
0.950.01
P ( D|T )=
=
=
=0.16
c
c
P(T )
P ( T|D ) P ( D )+ P ( T |D ) P(D ) 0.950.01+0.050.99
Monty Hall
3 doors: 2 have goats, 1 a car. Choose a door, Monty Hall opens another one and reveals a
goat. Do you switch doors?
Let contestant pick door 1. Let Ci be the event that the car is behind door i.
( get car|C 1)1 ( get car|C 2 )1 ( get car|C 3 )1
P ( get car )=P
+P
+P
3
3
3
01 11 11 2
+
+
=
With switch: P ( get car )=
3
3
3
3
11 01 01 1
+
+
=
Without switch: P ( get car )=
3
3
3
3
Elk Problem (Capture-Recapture)

( )( )
( )( ) ( )( )

Forest has N elk. m elk are captured, tagged, and released. Later, n elk are recaptured at
random. Previously captured elk are as likely to be recaptured. This corresponds to a
hypergeometric distribution: HGeom( m, Nm, n) .
Fisher Exact Test
Random sample of n women and m men gathered and tested (accurately) for disease.
Number of women with disease is X Bin (n , p 1) , number of men with disease is
Y Bin(m, p2) . Test if p1= p2 . Let X +Y =r .
Women
Men
Total
x
rx
r
Disease
nx
mr + x
n+mr
No Disease
n
m
n+m
Total
P ( X +Y =r|X= x ) P( X =x) P(X =rx )P( X=x )
P ( X=x|X +Y =r )=
=
P(X +Y =r )
P(X +Y =r )
Since X and Y are independent, X +Y Bin (n+m , p)
1 p n x

1 p n+ mr

n+ m p r
r
1 p mr + x n p x
x
r
x
m p
rx
P ( X=x|X +Y =r )=
Gamblers Ruin
Gamblers A and B make series of $1 bets. In each bet, A has probability p of winning, B
has probability q = 1 p of winning. Gambler A starts with i dollars, B starts with N i
dollar. Find P(W) = probability A wins the game.
pi=P ( W | A starts at i , wins round 1 )p + P ( W | A starts at i ,loses round 1 )q=P ( W | A starts at i+ 1 )p+ P ( W |

( )

()

( )

q i
1
p
1
, if p
N
2
pi= 1 q
p
i
1
,if p=
N
2

()
()

Simpsons Paradox
Dr. H and Dr. N perform 2 types of surgery: hard and easy. Each surgery is either a
success or failure. H has a higher success rate for both surgeries individually, but N has
an overall success rate higher when we aggregate both categories. Let A = event of
successful surgery, B = event that N is surgeon, C = event that surgery is hard type.
and
P ( A|B C ) < P( AB c C)
c
c
c
P ( A|B C ) < P( AB C )

But: P ( A|B )> P ( ABc )

This is because N is more likely to perform easy surgeries with a higher success rate.
St. Petersburg Paradox
Flip a coin and until it lands Heads. You receive $ 2n for the n rounds before Heads.
What is the expected payoff? Let X = winnings from game X =2N
1
1
1
E ( X )= 2+ 4 + 8+ = ,
but N ~ FS(1/2), so E(N) = 2
2
4
8
So E ( 2N )=2 E (N )=22=4
Chicken-Egg Story
Chicken lays random number of eggs N ~ Pois( ). Each egg independently hatches
with probability p and fails to hatch with probability q = 1 p. Let X = number of eggs
that hatch, Y = number of eggs that dont hatch, so X+Y = N. Find joing PMF of X and Y.
X |N=n Bin ( n , p ) ,Y | N =n Bin (n , q)

P ( X=i Y = j )= P ( X=i Y = j|N=n ) P(N =n) , since i+j has to be n:

n=0

P ( X=i Y = j )=P ( X=i Y = j| N=i+ j ) P ( N =i+ j ) , since Y=j is redundant:

p i

q j

q
e
ep
q je i + j
P ( X=i Y = j )=P ( X=i|N=i+ j ) P ( N=i+ j ) = i+ j p i
=
i
( i+ j ) !
Proves: (1) X and Y are independent, and (2) X ~ Pois( p ) and Y ~ Pois( q )

( )