CHAPTER 4.

AVERAGES

92

regardless of when it occurs. Stationary random processes will play a central, but not exclusive, role
in the ergodic theorems.
Observe that (4.29) implies that
m(T n F ) = m(F ), all eventsF, all n = 0, 1, 2, . . .

(4.30)

The following lemma follows immediately from (4.27), (4.28), and (4.30).
Lemma 4.7.1 If m is stationary with respect to T and f is integrable with respect to m, then
Em f = Em f T n , n = 0, 1, 2, . . .
Lemma 4.7.2 If f is in L1 (m) and m is stationary with respect to T , then the sequences {f T n ; n =
0, 1, 2, . . .} and {< f >n ; n = 0, 1, 2, . . .}, where
< f >n = n1

n1
X

f T i,

i=0

are uniformly integrable. If also f L2 (m), then in addition the sequence {f f T n ; n = 0, 1, 2, . . .}

and its arithmetic mean sequence {wn ; n = 0, 1, . . .} defined by
wn (x) = n1

n1
X

f (x)f (T i x)

(4.31)

i=0

are uniformly integrable.

Proof: First observe that if G = {x : |f (x)| r}, then T i G = {x : f (T i x) r} since x T i G
if and only if T i x G if and only if f (T i x) > r. Thus by the change of variables formula and the
assumed stationarity
Z
Z
T i G

and therefore

|f T i | dm =

Z
|f T | dm =
i

|f T i |r

|f | dm, independent of i,
G

|f |r

|f | dm r 0 uniformly in i.

This proves that the f T i are uniformly integrable. Corollary 4.4.6 implies that the arithmetic mean
sequence is then also uniformly integrable.
2

Exercises
1. A dynamical system is said to be N -stationary for an integer N if it is stationary with respect
to T N ; that is,
m(T N F ) = m(F ), all F B.
Show that this implies that for any event F m(T nN F ) = m(F ) and that m(T n F ) is a
periodic function in n with period N . Does Lemma 4.7.2 hold for N -stationary processes?
Define the measure mN by
N 1
1 X
m(T i F )
mN (F ) =
N i=0

4.7. STATIONARY AVERAGES

93

for all events F , a definition that we abbreviate by

mN =

N 1
1 X
mT i .
N i=0

Show that mN is stationary and that for any measurement f

EmN f =

N 1
1 X
Em f T i .
N i=1

Hint: As usual, first consider discrete measurements and then take limits.
2. A dynamical system with measure m is said to be asymptotically mean stationary if there is a
stationary measure m for which
n1
1X
m(T i F ) = m(F ); all F B.
n n
i=0

lim

Show that both stationary and N -stationary systems are also asymptotically mean stationary.
Show that if f is a bounded measurement, that is, |f | K for some finite K with probability
one, then
n1
1X
Em (f T i ).
Em f = lim
n n
i=0

94

CHAPTER 4. AVERAGES

Chapter 5

Conditional Probability and

Expectation
5.1

Introduction

We begin the chapter by exploring some relations between measurements, that is, measurable functions, and events, that is, members of a -field. In particular, we explore the relation between
knowledge of the value of a particular measurement or class of measurements and knowledge of an
outcome of a particular event or class of events. Mathematically these are relations between classes
of functions and -fields. Such relations will be useful in developing properties of certain special
functions such as limiting sample averages arising in the study of ergodic properties of information
sources. In addition, they are fundamental to the development and interpretation of conditional
probability and conditional expectation, that is, probabilities and expectations when we are given
partial knowledge about the outcome of an experiment.
Although conditional probability and conditional expectation are both common topics in an
advanced probability course, the fact that we are living in standard spaces will result in additional
properties not present in the most general situation. In particular, we will find that the conditional
probabilities and expectations have more structure that enables them to be interpreted and often
treated much like ordinary unconditional probabilities. In technical terms, there will always be
regular versions of conditional probability and we will be able to define conditional expectations
constructively as expectations with respect to conditional probability measures.

5.2

Measurements and Events

Say we have a measurable spaces (, B) and (A, B(A)) and a function f : A. Recall from
Chapter 1 that f is a random variable or measurable function or a measurement if f 1 (F ) B for
all F B(A). Clearly, being told the outcome of the measurement f provides information about
certain events and likely provides no information about others. In particular, if f assumes a value
in F B(A), then we know that f 1 (F ) occurred. We might call such an event in B an f -event
since it is specified by f taking a value in B(A). Consider the class G = f 1 (B(A)) = {all sets of
the form f 1 (F ), F B(A)}. Since f is measurable, all sets in G are also in B. Furthermore, it is
easy to see that G is a -field of subsets of . Since G B and it is a -field, it is referred to as a
sub--field.
95

96

CHAPTER 5. CONDITIONAL PROBABILITY AND EXPECTATION

Intuitively G can be thought of as the - field or event space consisting of those events whose
outcomes are determinable by observing the output of f . Hence we define the -field induced by f
as (f ) = f 1 (B(A)).
Next observe that by construction f 1 (F ) (f ) for all F B(A); that is, the inverse images
of all output events may be found in a sub--field. That is, f is measurable with respect to the
smaller -field (f ) as well as with respect to the larger -field B. As we will often have more
than one -field for a given space, we emphasize this notion: Say we are given measurable spaces
(, B) and (A, B(A)) and a sub--field G of B (and hence (, G) is also a measurable space). Then
a function f : A is said to be measurable with respect to G or G-measurable or, in more detail,
G/B(A)-measurable if f 1 (F ) G for all F B(A). Since G B a G-measurable function is clearly
also B-measurable. Usually we shall refer to B-measurable functions as simply measurable functions
or as measurements.
We have seen that if (f ) is the -field induced by a measurement f and that f is measurable
with respect to (f ). Observe that if G is any other -field with respect to which f is measurable,
then G must contain all sets of the form f 1 (F ), F B(A), and hence must contain (f ). Thus
(f ) is the smallest -field with respect to which f is measurable.
Having begun with a measurement and found the smallest -field with respect to which it is measurable, we can next reverse direction and begin with a -field and study the class of measurements
that are measurable with respect to that -field. The short term goal will be to characterize those
functions that are measurable with respect to (f ). Given a -field G let M(G) denote the class of
all measurements that are measurable with respect to G. We shall refer to M(G) as the measurement
class induced by G. Roughly speaking, this is the class of measurements whose output events can
be determined from events in G. Thus, for example, M((f )) is the collection of all measurements
whose output events could be determined by events in (f ) and hence by output events of f . We
shall shortly make this intuition precise, but first a comment is in order. The basic thrust here
is that if we are given information about the outcome of a measurement f , in fact we then have
information about the outcomes of many other measurements, e.g., f 2 , |f |, and other functions of
f . The class of functions M((f )) will prove to be exactly that class containing functions whose
outcome is determinable from knowledge of the outcome of the measurement f .
The following lemma shows that if we are given a measurement f : A, then a real-valued
function g is in M((f )), that is, is measurable with respect to (f ), if and only if g() = h(f ())
for some measurement h : A <, that is, if g depends on the underlying sample points only through
the value of f .
Lemma 5.2.1 Given a measurable space (, B) and a measurement f : A, then a measurement
g : < is in M((f )), that is, is measurable with respect to (f ), if and only if there is a
B(A)/B(R)-measurable function h : A < such that g() = h(f ()), all .
Proof: If there exists such an h, then for F B(<), then
g 1 (F ) = { : h(f ()) F } = { : f () h1 (F )} = f 1 (h1 (F )) B
since h1 (F ) is in B(A) from the measurability of h and hence its inverse image under f is in (f ) by
definition. Thus any g of the given form is in M((f )). Conversely, suppose that g is in M((f )).
Let qn : A R be the sequence of quantizers of Lemma 4.3.1. Then gn defined by gn () = qn (f ())
are (f )-measurable simple functions that converge to g. Thus we can write gn as
gn () =

M
X
i=1

ri 1F (i) (),