CS 70 Discrete Mathematics and Probability Theory Lecture 24: Intro to Continuous Probability

Terry Filiba August 1, 2012

Review
A random experiment. The sample space (set of possible outcomes). The probability of each possible outcome of the experiment.

Recall that a probability space consists of the following:

So far all of our examples of sample spaces have been discrete Like ipping coins, rolling dice, sending n bit signals, polling people Even if there were an innite number of elements in the set, they were still discrete We leaned how to do probability over these discrete sample spaces

Continuous probability

What happens when the sample space is continuous? Example: Spin an arrow (sharp arrow spinning on an axis) so it traces out a circle of circumference L When it stops spinning, the tip is equally likely to be pointing at any point 1

along the circle. In other words the probability space S is a circle of circumference L. What is the probability that the tip points at a point in the right half of the circle when it stops (i.e. between 12 and 6 oclock, if the circle were a clock)? At a point in the upper right or lower left (i.e. either between 12 and 3 oclock or between 6 and 9 oclock)? At the point exactly at 12 oclock? Could we treat the points on the circle the same way as we have done so far, assigning each a positive probability? There are innitely many points so if they all had positive probability the total space would sum to innity If they all had 0 probability, summing them would yield 0 Let us think of the circumference of the circle as the interval of numbers from 0 to L, that is [0,L]. To be careful, the points 0 and L are the same, since it is a circle, but that wont change any answers we compute. Then it is natural to say that the chance that the arrow stops at a point in an interval [a,b] lying inside [0,L] is: P ([a, b]) = (b a)/L This has the natural properties that P ([0, L]) = L/L = 1 ... probability that you land somewhere is 1 If [c,d] is shorter than [a,b], so b1-a1 b-a, then P ([c, d]) < P ([a, b]), i.e. it is less likely to land in the shorter segment P ([0, L/2]) = P ([L/2, L]) = 1/2 ... probability that you land in left half ... = probability that you land in right half = 1/2 One important property of probability theory that we have used many times so far is that if A and B are disjoint events, then: P (A B) = P (A) + P (B) In our example, suppose A = [a1,a2] and B=[b1,b2] are disjoint subintervals of [0,L]. Then it is natural to also insist that P (A B) = (a2 a1)/L + (b2 b1)/L = length(AU B)/L More generally, given any nite union of intervals C, its length is well dened, and we say P (C) = length(C)/L 2

We would use the same denition if C were a union of innitely many intervals; how to do this carefully is a subject called measure theory, and beyond the scope of this course. Suce it to say that we need to exclude some weird sets whose length is hard to dene. If you spin the wheel of fortune, what is the probability that when the arrow stops, it is between 8 and 9 oclock, or between 1 and 1:30? Answer: There are 12*60=720 minutes in the whole circle, and the length of the 2 intervals is 90 minutes, so the probability is 90/720 = 1/8. Example: Throw a dart at a square (sides of length 1) with a circle (diameter 1) inside it. What is the probability the dart lands inside the circle? We are basically picking a random point in the square and checking if it falls in the circle How do we dene this random point? We could spin a wheel where L = 1 once to get the x coordinate and again to get the y coordinate Thus x and y are independent for the same reason that ipping a coin twice gives you independent results. What is the probability that (x,y) falls in the left half of the square? P (xin[0, 1/2])) = 1/2 = area of left half of (unit) square What is probability that (x,y) falls in the upper half of the square? P (yin[1/2, 1])) = 1/2 = area of top half of (unit) square What is probability that (x,y) falls in the upper left quarter of the square? This is the same as P (xin[0, 1/2]andyin[1/2, 1])) = P (xin[0, 1/2])P (yin[1/2, 1])) ... by independence = 1/2 1/2 = 1/4 = area of upper left quarter of the square More generally, we see that for any rectangle A, P(A) = area(A), and by the same rule as above P (A B) = P (A) + P (B) if A and B are disjoint we see that P(A) = area(A) for any union of rectangles. Since (many!) regions can be approximated as closely as you like by a union of disjoint rectangles, it is natural to dene P(A) = area(A)/area(unit square) = area(A) 3

Again, care is needed to dene this carefully (outside the scope of CS70). Finally, we can answer the question: What is the chance that the randomly thrown dart lands in the unit circle? P (circle) = area(circle) = pi/4 .785. Example: (Buons Needle Problem): Drop a needle of length L so it lands anywhere at random on a board covered with parallel lines a distance L apart. What is the probability that the needle intersects one of these lines? Now the sample space S is dened by two parameters: y = distance of the middle of the needle to the closest parallel line, so 0 <= y <= L/2 = angle of needle with respect to a line perpendicular to all the parallel lines, so /2 <= <= /2 y and and uniformly distributed along their intervals. Basically, we are picking a random point in a rectangle [0, L/2]x[/2, /2] Some points interest a line, and others do not. For some points (y, ) the needle will intersect the closest line, By elementary trigonometry the needle intersects the closest line exactly when y <= (L/2) cos(), so the desired area is
/2 /2

(L/2) cos()d = (L/2) sin()|/2 = (L/2) sin(pi/2) (L/2) sin(pi/2) =L

/2

The total area of the rectangle is (L/2) , so the desired probability of intersection is L/((L/2) ) = 2/ .637 This gives us a (very slow) algorithm to estimate : Drop n needles on the board, one at a time, where n is very large Count the number m that intersect a line Estimate 2 n/m

Continuous random variables

Now we want to generalize the idea of a random variable and apply it continuous probability spaces. In the previous examples, we had random variables with uniform distributions What if the arrow gets stuck on one side of the circle? Or when you throw cars you aim for the center? We need a way to express this distribution, cant just look that the size of the interval anymore Denition: A probability density function f(x) for a random variable X satises: b P (a < X b) = a f (x)dx for all a b. Ex: Suppose we go back to the arrow experiment and say X is where it lands This is uniformly distributed in [0,L]. Then we let f (x) = 1/L if a < x b, 0 otherwise So if [a, b] is inside [0, L]: b P (a < X b) = a 1/Ldx = (b a)/L as expected. And if [a,b] is not inside [0,L], we get the right answer too: 1 P (X < 1) = 0dx = 0
2L

P (L/2 < X <= 2 L) =

L/2 L

f (x)dx
2L

=
L/2

1/Ldx +
L

0dx

= 1/2 Properties of the PDF 1. f (x) 0. Otherwise we might have some interval that has negative probability 2. f (x)dx = 1. The random variable must have some value, with probability one. This is the analogue of the requirement that xS P (x) = 1 for a discrete probability function. 5

The point of probability density functions is to permit non-uniform distributions: Example: f (x) = (2/L2 ) x for 0 x L, and 0 otherwise. To see that f (x) is a legimate probability distribution, we need to check 1. f (x) is nonnegative 2.
(2/L2 )

xdx = (1/L2 ) x2 |L = 1 0

Now P (X < L/2) = 1/4 is not the same as P (X L/2) = 3/4 Note that if L is small, say L = .1, then f (L) = 2/L > 1. So unlike a probability itself, f(x) does not have to be less than or equal to 1, just its integral. Sometimes it is easier to work with the integral of f(x) directly instead of f(x) itself: Denition: The Cumulative Distribution Function (CDF) F(x) of random variable X is F (a) = P (X a) Note that we can use the CDF to compute the probability over any interval we like. Suppose a < b P (a < X b) = P (X B) P (X A) = F (b) F (a) Thus a F (a) = f (x)dx or, by the Fundamental Theorem of Calculus (remember!) dF (x)/dx = f (x)

Continuous expectation and variance

Now we naturally want to know the expectation and variance of the random variable Denition: If X is a random variable with density function f (x), its expectation is dened as E(X) = x f (x)dx

This is analogous to the denition for discrete probability E(X) = r r P (X = r) And has similar properties E(X + Y ) = E(X) + E(Y ) E(X + c) = E(X) + c E(a X) = a E(X) Example: Spin the wheel of fortune, let X = stopping position. Then L E(X) = 0 x (1/L)dx = x2 /(2 L)|L = L/2, 0 as expected. Example: Throw the dart at the circular target, let X = distance from center. Then L E(X) = 0 x (2 x/L2 )dx = (2/3) x3 /L2 |L = (2/3) L 0 Denition: If X is a random variable with density function f(x), its variance is: V (X) = E(X 2 ) (E(X))2 = x2 f (x) (E(X))2 Example: Spin the wheel of fortune, let X = stopping position. Then L V (X) = 0 x2 (1/L)dx (L/2)2 = (1/3 1/4) L2 = (1/12) L2 To see the similarity to picking a random integer Y from 1 to L, we recall E(Y ) = L i (1/L) = (L + 1)/2 i=1 V (Y ) = L i2 (1/L) ((L + 1)/2)2 = (L2 1)/12 i=1