123 views

Uploaded by Anup Kulkarni

- BSC Stat 1.pdf
- Stat230 S2010 Course Notes
- Diaz-pet03 - 14 Hal
- SDA 3E Chapter 3 (1)
- BIOSTATchapter4-IPS14e
- iet 603 700 forum 5
- dec 4.docx
- lesson5-a-sup-160127203931.pptx
- Mohammad Modarres, Mark P. Kaminskiy, Vasiliy Krivtsov - Reliability Engineering and Risk Analysis _ a Practical Guide-CRC Press (2017)
- Item Analysis HUMMS a Ok
- HW_5
- Extreme Value and Related Models With Applications in Engineering and Science
- Untitled
- tables
- Numerical Calculation
- Lehrer Etal 2011 Measures of Distribution
- A New Study on Reliability-based Design Optimization for Fatigue Life
- HW01-3
- EC 2024_PS3
- Full Text

You are on page 1of 24

A Concise Course in

Statistical Inference

By Larry Wasserman

Toby Xu

UW Madison

05/29/07

Chapter 1 Overview

Sample Space

Disjoint or Mutually Exclusive

Probability Axioms/Basic Properties

Finite sample spaces

Independent Events

Conditional Probability

Baye’s Theorem

Sample Space

Sample Space Ω is the set of possible outcomes.

Ex: Toss fair coin twice Ω={HH,HT,TH,TT}

Points ω in Ω are called sample outcomes, realizations,

elements.

Subsets of Ω are called events.

Ex: Event that first toss is head is A={HH,HT}

Ac = complement of A (or NOT A)

∅ = complement of Ω

A∪ B = {ω ∈ Ω : ω ∈ A or ω ∈ B or ω ∈ both}

A∩ B = {ω ∈ Ω : ω ∈ A and ω ∈ B}

Disjoint or Mutually Exclusive

Two events (A and B) are mutually exclusive iff A∩ B = ∅

Indicator function of A

Monotone increasing if A1 ⊂ A 2 ⊂ A3 ⋯

Monotone decreasing if A1 ⊃ A2 ⊃ A3 ⋯

Intro to Probability

P(A) = Probability Distribution or Probability Measure

Axioms:

1) P(A)≥0 for every A

2) P(Ω) = 1

If A1,A2…are disjoint then

∞ ∞

P (∪ A i ) = ∑ P ( A i )

i =1 i =1

Statistical Interference:

Frequentist

Bayesian Schools

Basic Properties of Probability

P(∅) = 0

A ⊂ B ⇒ P ( A) ≤ P (B )

0 ≤ P ( A) ≤ 1

A∩ B = ∅ ⇒ P ( A∪ B ) = P ( A) + P ( B ) For Disjoint Probabilities only

P(Ac)=1-P(A)

P ( A∪ B ) = P ( A) + P ( B ) − P ( AB )

A or B = (ABc) or (AB) or (AcB)

Probability on Finite Sample

Spaces

P(A)=A/Ω n n

= = 1

n n! 0 n

= N choose k n n

k k !( n − k )! =

k n − k

• N choose K is counting how many ways can we get a

k-subset out from a set with n elements.

•Ex: There’s 10 people in the Book Club; We want

groups of 2 people. How many possible

combinations?

10 10!

= = 45

2 2!(10 − 2)!

Independent Events

Two events are independent (does not directly affect the probability

of the other from happening) iff: P(AB) = P(A)P(B)

A set of events {Ai : i ε I} is independent if P (∩ A i ) = ∏ P ( A i )

i∈ J i∈ J

Assume A & B are disjoint events, each with +ve probability. Can

they be independent?

Ans: No because:

Independent = P(AB) = P(A)P(B)

But here P(AB)=φ=0 and P(A)P(B)>0

Ex: Toss a fair die A={2,4,6} B{1,2,3,4} A B = {2, 4}

P(AB)=2/6=P(A)P(B)=1/2*2/3

∩

Conditional Probability

Assuming P(B)>0…P(A|B)=P(AB)/P(B)

If A & B are independent then P(A|B)=P(A)

Ex: Draw Ace of clubs (A) and then Queen of Diamonds (B)

P(AB)=P(A)P(B|A)=1/52*1/51=1/2652

P(.|B) satisfies the axioms, for fixed B

P(A|.) does not satisfy the axioms of probability, for fixed A

In general P(A|B) ≠ P(B|A)

Bayes’ Theorem

k

The Law of Total Probability P ( B ) = ∑ P ( B | Ai ) P ( Ai )

i =1

Ω

Where Cj = BAj C1…k are disjoint

each i. If P(B)>0 then, for each i=1,…,k

P( B | Ai ) P( Ai )

P( Ai / B) =

∑ P( B | Aj ) P( Aj )

Additional Examples

Disease Test with + or – outcomes D Dc

probability of one having disease? Ans: not - .001 .891

90%...actually 8% P (+∩ D ) .009

P(D | +) = = ≈ 0.08

P(+) .009 + .099

3 Catergories of Email: A1= “spam”, A2= “low priority”,

A3= “high priority”

P(A1)=.7,P(A2)=.2,P(A3)=.1 Let B be event that email

contains the word “Free”

P(B|A1)=.9, P(B|A2)=.01, P(B|A3)=.01

Q: receive an email with word “free”, what’s the

probability that it is spam?

.9 × .7

P ( A1 | B) = = .995

(.9 × .7) + (.01× .2) + (.01× .1)

Chapter 2 Overview

Random Variable

Cumulative Distribution Function (CDF)

Discrete Vs. Continuous probability functions (Probability Density

Function PDF)

Discrete: Point Mass, Discrete Uniform, Bernoulli, Binomial, Geometric, Poisson

Distribution

Continuous: Uniform, Normal (Gaussian), Exponential, Gamma, Beta, t and Cauchy,

Χ2

Bivariate Distribution

Marginal Distribution

Independent Random Variables

Conditional Distribution

Multivariate Distributions: Multinomial, Multivariate Normal

Transformations of Random Variables.

Random Variable

A random variable is a mapping ω P(ω) X(ω)

X: Ω R TT ¼ 0

That assigns a → real number TH ¼ 1

X(ω) to each outcome ω

Ex: Flip coin twice and let X be HT ¼ 1

number of heads. HH ¼ 2

x P(X=x)

0 ¼

1 ½

2 ¼

Cumulative Distribution

Function

CDF = FX: R→[0,1] Theorem: let X and Y have

CDF F and G. If F(x) = G(x)

FX(x)=P(X≤x) for all x, then P(X∈ A) = P(Y ∈

Ex: Flip a fair coin twice A)

Theorem: CDF needs to

and let X be number of

satisfy:

heads F is non-decreasing: x1<x2,

also F(x1)≤F(x2)

0 x<0

1/4 F is normalized:

0 ≤ x<1

Fx ( x) = lim F ( x) = 0

3/4 1 ≤ x<2 x →−∞

x →+∞

F is right-continuous:

F(x) = F(x+) for all,

where F(x+)= lim F ( y )

y→x

y>x

Discrete Vs. Continuous

Distributions

Def: X is discrete if it take countably many values. We define the

probability function or probability mass function for X by

fX(x)=P(X=x)

Def: X is continuous if there exists a function fX such that fX(x)≥0 for

all x,+∞ and for every a≤b

∫ f ( x )dx = 1

−∞ b

P ( a < X < b ) = ∫ f ( x )dx

a

x

Fx ( x ) = ∫

−∞

fx(t )dt

Note: mention Lemmas (pg 25)

Some Important Discrete

Functions

Point Mass Distribution: if P(X=a)=1 otherwise 0

Discrete Uniform

f(x)= 1

for x=1,...,k

k

0 otherwise

f(x)=Px(1-p)1-x for x ε {0,1}

Binomial (flip coins n time and let X be number of heads)

f(x)=P(X=x)={ n for x=0,…,n

P ^ x(1 − p )^ ( n − x)

x

{0 otherwise

Geometric p ε (0,1) (Think X as the No. of flips needed to get the 1st head)

P(X=k)=p(1-P)k-1, k=1,2,3,…

Poisson (Counting rare events like radioactive decay or traffic accidents)

λ^ x x≥0

f ( x )= e ^ ( − λ )

x!

Some Important Continuous

Distributions

Uniform Distribution

f(x)={1/(b-a) for x ∈ [a,b]

{0 otherwise

Normal (Gaussian)

1 1

f ( x) = exp{ − ( x − µ )^ 2}

σ 2π 2σ ^ 2

if X ~ N(µ, σ2), then Z=(X- µ)/ σ N(0,1)

Exponential (Model lifetimes of electronic components/ waiting

times between rare events)

1

f ( x) = e ^ ( − x / β ), x > 0

β

t distribution is like Normal except with thinker tails

Cauchy is special case of t distribution where ν=1

X2 distribution deals with various degrees of freedom

Bivariate Distribution

Discrete random variables Y=0 Y=1

X=0 1/9 2/9 1/3

function by f(x,y)=P(X=x

and Y=y) X=1 2/9 4/9 2/3

F(1,1)=P(X=1,Y=1)=4/9

1/3 2/3 1

In continuous case, we use

f(x,y) a PDF for the random

variables (X,Y)

(i) f(x,y) ≥ 0 for all (x,y)

(ii) ∞ ∞

∫∫

−∞ −∞

f ( x , y ) dxdy = 1 and ,

A) =

∫∫ f ( x, y )dxdy

Marginal Distributions

Def: if (X,Y) have joint distribution with mass function fX,Y,

then the marginal mass function for X is defined by

fX ( x ) = P ( X = x ) = ∑ P ( X = x , Y = y ) = ∑ f ( x , y )

y y

fY ( x ) = P ( X = x ) = ∑ P ( X = x , Y = y ) = ∑ f ( x , y )

x x

For continuous random variables, the marginal densities are

fX ( x) = ∫ f ( x, y )dy

Y=0 Y=1

fY ( y ) = ∫ f ( x, y )dx X=0 1/10 2/10 3/10

X=1 3/10 4/10 7/10

4/10 6/10

Independent Random

Variables

Def: Two random variables X & Y=0 Y=1

Y are independent iff, for every

X=0 ½ 0 ½

A & B,

X=1 0 ½ ½

P( X ∈ A, Y ∈ B) = P( X ∈ A) P(Y ∈ B) ½ ½ 1

Y=0 Y=1

X=0 ¼ ¼ ½

These are not independent

because

X=1 ¼ ¼ ½

fX(0)fY(1)=(1/2)(1/2)=1/4

½ ½ 1

yet f(0,1)=0

fX(0)fY(0)=f(0,0)

fX(0)fY(1)=f(0,1)

fX(1)fY(0)=f(1,0)

fX(1)fY(1)=f(1,1)

Conditional Distributions

Def:

f(x|y)=P(X=x|Y=y)=P(X=x,Y=y)/P(Y=y)=f(x,y)/f(y)

Assuming that f(y)>0

Def: for continuous random variables,

f(x|y)= f(x,y)/f(y)

Assuming that f(y)>0. Then,

P ( X ∈ A / Y = y ) = ∫ f ( x | y ) dx

A

Multivariate Distribution and IID

Samples

If X1,…,Xn are independent and each has the same marginal

distribution with CDF F, we say that X1,…,Xn are independent

and identically distributed and we write

X1,…Xn ~ F

Random sample size n from F

2 Important Multivariable

Distributions

Multinomial: multivariate version of a Binomial

X ~ Multinomial(n,p)

n

f ( x ) = P1^ ( x1)... Pk ^ ( xk )

x1 ... xk

Where

n n!

=

x1 ... xk x1!...xk !

Transformations of Random

Variables

Three steps for Transformations

1. For each y, find the set Ay={x:r(x)≤y)

2.Find CDF

F(y)=P(Y ≤y)=P(r(X) ≤y)

=P({x;r(x) ≤y})

=

∫ fx ( x ) dx

Ay

3. The PDF is f(y)=F’(y)

- BSC Stat 1.pdfUploaded byaliahad84
- Stat230 S2010 Course NotesUploaded byThierry Lusetti
- Diaz-pet03 - 14 HalUploaded byUrahara Jef
- SDA 3E Chapter 3 (1)Uploaded byxinearpinger
- BIOSTATchapter4-IPS14eUploaded bySamantha Cruz Blanco
- iet 603 700 forum 5Uploaded byapi-285292212
- dec 4.docxUploaded byDulce Amor Ermita
- lesson5-a-sup-160127203931.pptxUploaded byAbdiasis Abdallah Jama
- Mohammad Modarres, Mark P. Kaminskiy, Vasiliy Krivtsov - Reliability Engineering and Risk Analysis _ a Practical Guide-CRC Press (2017)Uploaded byHenrique de Oliveira
- Item Analysis HUMMS a OkUploaded byjun del rosario
- HW_5Uploaded byShaggypatel
- Extreme Value and Related Models With Applications in Engineering and ScienceUploaded byAamir Mahmood
- UntitledUploaded byapi-232747878
- tablesUploaded byJeremiah
- Numerical CalculationUploaded byJorge Yarasca
- Lehrer Etal 2011 Measures of DistributionUploaded byDiana Marcela Tello Monsalve
- A New Study on Reliability-based Design Optimization for Fatigue LifeUploaded bySijun Liu
- HW01-3Uploaded byXue Adams
- EC 2024_PS3Uploaded byJIM BENNY
- Full TextUploaded byMichiel Fenaux
- Auckland - 325bookUploaded bykhoantd
- problemset2sUploaded byAnirban Banerjee
- MATH540 Week 3 Assignment_Jet Copies_Karmon_Jeneka_in Word(2)-3Uploaded byKarmon Jeneka
- C1 Lesson 1- Exploring Random VariablesUploaded byTeresa Navarrete
- bernoulli.pdfUploaded bympedraza-1
- intro,methodologyUploaded byNorbert Lim
- Midterm ReviewUploaded byArthur Chow
- bm_aktienkurse_e.pdfUploaded byupendra1616
- 5.Network TheoryUploaded byIftekhar Ali
- MikUploaded byMika De Chavez

- Dynamics_Ch11_print.pdfUploaded byKim Taehwan
- Separation of Variables in Laplace's Equation in Cylindrical CoordinatesUploaded byOBStaff
- 29741728 Matrix Methods of AnalysisUploaded byArifsalim
- A1.2 - Perms and CombosUploaded byamolkhadse
- Math Challenge 15Uploaded byHermann Brital Acebido-Basmayor Dejero-Lozano
- Polarizable Vacuum (PV) and the Schwarzschild SolutionUploaded bydgE
- A15 201-NYC Linear AlgebraUploaded byNicole Wong
- M2 R08 NovDec 11Uploaded byBala Sekar
- Alevel FP2newUploaded bynuddin123
- Path Integrals LinetskyUploaded byStephen Goodman
- SlipUploaded byCharles Odada
- A Brief Look on Gaussian Integrals Quantum Field Theory William O. StraubUploaded byMartinAlfons
- Csir Net_22 Jun. 2014Uploaded byssnainamohammed2312
- Sheet 6 (2015-2016)Uploaded byMenna Elsaoudy
- Models.woptics.dielectric Slab WaveguideUploaded byneomindx
- Leibniz - Fundamental Theorem of CalculusUploaded byJohn Dodoe
- NEw COursework.docxUploaded bypvaibhav08
- M12-5-IF0334Uploaded byVictor Daniel Waas
- Worked Problems on Limits, Continuity and Branches of Complex FunctionsUploaded byPaul Benedict
- DYNAMICS OF MACHINERYUploaded byJuan Gonzalez Leon
- Gravitational SingularityUploaded byAbdul Khaliq Muhammad
- ext12Uploaded byasdawanwuwi
- Unit 1Uploaded bykchhang
- Chebyshev Polynomials - WikipediaUploaded byAnonymous c1USJCV
- Roman Domination Number of a GraphUploaded byJASH MATHEW
- Laplace TI 89 TITANIUMUploaded byDavid Bol
- Numerical Techniques for Optimization Problems with PDE ConstraintsUploaded bylelix
- clc7eap1503Uploaded bymohil
- Fourier Series in MathsUploaded byCh Ashish Dahiya
- Transportation ProblemUploaded byHarjeet Kaur