Chapter 3 Conditional Probability and Independence

Lectures 8 -12 In this chapter, we introduce the concepts of conditional probability and independence. Suppose we know that an event already occurred. Then a natural question asked is ``What is the effect of this on the probabilities of other events?'' This leads to the notion of conditional probability.

Definition 3.1 Let

conditional probability of an event

be a probability space and given denoted by

be such that is defined as

The

Define

Then

is a

-field of subsets of

(see Exercise 3.2). , if

A collection (i) (ii) 's are pairwise disjoint

of events is said to be a partition of

Here

may be

. If

, then partition is said to be finite partition and if

, it is called

a countable partition.

Theorem 3.0.8 Define

on

as follows.

Then

is a probability space.

Proof is an exercise, see Exercise 3.3.

Theorem 3.0.9 (Law of total probability-discrete form)

be a partition of such that

Let for all

be a probability space and . Then for ,

Proof.

Theorem 3.0.10 (Bayes Theorem) Let

and . Then

be a probability space and

such that

Proof.

Now use Law of total probability to complete the proof.

Definition 3.2 (Independence ) Two events

are said to be independent if

Remark 3.0.3 If

, then

and

are independent iff

This confirms

the intuition behind the notion of independence, ``the occurrence one event doesn't have any effect on the occurrence of the other''.

Example 3.0.17 Define the probability space

as follows.

and

Consider the events are independent and and are dependent.

and

. Then

and

Example 3.0.18 Consider the probability space defined by

and

given by (1.0.6). Consider the events are independent and are dependent.

. Then

The notion of independence defined above can be extended to independence of three or more events in the following manner.

Definition 3.3 (Independence of three events). The events

(i) (ii) If the events satisfies only (i), then ; and are independent and

are independent (mutually) if

are said to be pairwise independent.

Example 3.0.19 Define

and

Consider the events

Then

are pairwise independent but not independent.

Example 3.0.20 Define

and

as follows .

Let

Then

are independent.

Definition 3.4 The events

from

are said to be independent if for any distinct

Using the notion of independence of events, one can define the independence of

-fields as follows.

Definition 3.5 Two

-fields

of subsets of

are said to be independent if for any

Definition 3.6 The

-fields

of subsets of

are said to be independent if . -fields as follows.

are independent whenever One can go a step further to define independence of any family of

Definition 3.7 A family of

finite subset

-fields , the -fields

, where

is an index set, are independent if for any are independent.

Finally one can introduce the notion of independence of random variables through the corresponding field generated. It is natural to define independence of random variables using the corresponding since the -field contains all information about the random variable.

-fields,

Definition 3.8 Two random variables

independent.

and

are independent if

and

are

Definition 3.9 A family of random variables

independent.

are independent if

are

Example 3.0.21 Let

the random variables Proof. Let and and

are independent events iff are independent.

and

are independent

-fields iff

are independent. Consider

Hence and independent. Now Since

are independent. Changing the roles of and we have and are independent implies that and are independent.

and

are

it follows that

and

are independent. Converse statement is obvious.

The second part follows from

Example 3.0.22 The trivial

-field

is independent of any

-field of subsets of

Example 3.0.23

Define a probability space

as follows:

and

For

set

Then

are independent.

This can be seen from the following.

Note that

Hence

Also

Example 3.0.24 Let

is given by as

and

Define two random variables

as

Then

and

are independent. Here note that

and

We conclude this chapter with the celebrated Borel-Cantelli Lemma. To this end, we need the notion of limsup and liminf of events.

Definition 3.10

of sets) For

subsets of

, define

Similarly

In the following theorem, we list some useful properties of limsup and liminf. This will make the objects and of sets much clearer.

Theorem 3.0.11 1.Let

Then

2.Let

Then

3.The following inclusion holds.

4.The following identity holds.

5.If

, then

6.If

then

Proof. 1.Consider

This proves (1).

2.Consider

Thus (2).

3.Proof of (3) follows from (1) and (2).

4.Proof of (4) follows from De Morgen's laws.

5.Consider

Similarly

This proves (5)

6.The proof of (6) follows from (4) and (5).

Remark 3.0.4 The properties (1)-(6) are analogous to the corresponding properties of
of real numbers.

and

Remark 3.0.5 Analogous to the notion of limit of sequence of numbers, one can say that
exists if

Hence from Theorem 3.0.11 (5), if and is

is an increasing sequence of sets, then

exists

. Similarly using Theorem 3.0.11 (6), if exists and is

is an decreasing sequence of sets, then

. Now student can see why property (6) in Theorem 1.0.1 is called

continuity property of probability.

Theorem 3.0.12 (Borel - Cantelli Lemma) Let

. (i) If

be a probability space and

then

(ii) If

and

are independent, then

Proof: (i) Consider

Note that the r.h.s

as

, since

Therefore

(ii) Note that

Therefore

Now

The last equality follows from

Using

, we get

Since

we get

Therefore

Thus

Therefore

This completes the proof.

Example 3.0.25 Define a probability space

as follows.

Set

where

Let

Then

is a field, see Exercise 3.5. Define

For

, define

For

, there exists

which are pairwise disjoint such that

Define

as follows.

Now extend Let

to

using the extension theorem.

One can calculate

using Borel - Cantelli Lemma as follows. Define

Then

are independent and

Hence

Also note that

Therefore using Borel - Cantelli Lemma