1.

April 1st, 2016 - Lecture:

Covariance and Chebyshev's
Inequality
1. Topics
Variance of a sum of dependent random variables and the definition of covariance.

Properties of covariance. Independence implies correlation but the converse does not hold.

Statement of Chebyshev's inequality and a basic application.

2. Introduction
If X and Y are indepedant, thenV(X+Y)=V(X)+V(Y)

If not, then equality may or may not hold.

Example: A case where equality does not hold.

o Take P(X=1)=12, P(X=0)=12, x=1Y=0, x=1,Y=1.V(Y)=14

o SoV(X)+V(Y)=14+14=12

o V(X+Y) equals 0? X+Y will always be 1.V(X+Y)<V(X)+V(Y)

3. Variance
"Swinging together"V(X+Y)<V(X)+V(Y)

"Swings unrelated"V(X+Y)=V(X)+V(Y)

"Swings are opposite each other"V(X+Y)>V(X)+V(Y)

We want to measure "how much they're together or opposite."

The covariance of X and Y is defined to be:Cov(x,y)=E[E(xE[x])(yE[y])]

 Intuition

o "Swing together"Cov(X,Y)>0

o "Swings unrelated"Cov(X,Y)=0

o "Swings opposite"Cov(X,Y)<0

Example (non rigourous pictures)

o Examples, x is height and y is shoe sizeCov(X,Y)>0

o v | .

o a | . .

o l | . . .

o u | . . .

o e | . . .

o | . . .

o y | . .

o | .

o 0 +-----------------

o 0 value of X

o Examples, x is temp and y is sales of hot chocolateCov(X,Y)<0

o v | .

o a | . .

o l | . . .

o u | . . .
o e | . . .

o | . . .

o y | . .

o | .

o 0 +-----------------

o 0 value of X

Formula:Cov(X,Y)=E(XY)E(X)E(Y)

Claim: If X,Y are indepedant, Cov(X,Y)=0

o ProofCov(X,Y)=E(XY)E(X)E(Y)

=E(X)E(Y)E(X)E(Y)

=0

4. Tchbycheff's Inequility
Recall Markov's give tail bound base only on E(X)

Chebyshev's give tail bound using E(X) and V(X)

Chebyshev's Theorem: Let X be a random variable with E(X)=

o Then for any >0.

Example: Roll a fair die 100 times and let Z be the sum and X1, ..., X100 be
the outcomes.

o What's the probability that Z are within 50 of its mean?