6.7.

ERGODICITY

147

Conversely, if (6.42) holds every indicator function of an invariant event is either 1 or 0 with
probability 1, and hence every simple invariant function is equal to a constant with probability one.
Since every invariant measurement is the pointwise limit of invariant simple measurements (combine
Lemma 4.3.1 and Lemma 6.3.3), every invariant measurement is also a constant with probability one
(the ordinary limit of the constants equaling the invariant simple functions). Thus every invariant
measurement will be constant with probability one if and only if (6.42) holds.
Suppose next that a system possesses ergodic properties with respect to all indicator functions
or, equivalently, with respect to all bounded measurements. Since the limiting sample averages
of all indicator functions of invariant events (in particular) are constants with probability one, the
indicator functions themselves are constant with probability one. Then, as previously, (6.42) follows.
Thus we have shown that if a dynamical system possesses ergodic properties with respect to a
class of functions containing the indicator functions, then the limiting sample averages are constant
a.e. if and only if (6.42) holds. This motivates the following definition: A dynamical system (or the
associated random process) is said to be ergodic with respect to a transformation T or T -ergodic or,
if T is understood, simply ergodic, if every invariant event has probability 1 or 0. Another name for
ergodic that is occasionally found in the mathematical literature is metrically transitive. .
Given the definition, we have proved the following result.
Lemma 6.7.1 A dynamical system is ergodic if and only if all invariant measurements are constants
with probability one. A necessary condition for a system to have ergodic properties with limiting
sample averages being a.e. constant is that the system be ergodic.
Lemma 6.7.2 If a dynamical system (, B, m, T ) is AMS with stationary mean m, then (, B, m, T )
is ergodic if and only if (, B, m, T ) is.
Proof: The proof follows from the definition and Lemma 6.3.1.
2
Coupling the definition and properties of ergodic systems with the previously developed ergodic
properties of dynamical systems yields the following results.
Lemma 6.7.3 If a dynamical system has ergodic properties with respect to a bounded measurement
f and if the system is ergodic, then with probability one
< f >n n Em f = lim n1
n

n1
X

Em f T i .

i=0

If more generally the < f >n are uniformly integrable but not necessarily bounded (and hence the
convergence is also in L1 (m)), then the preceding still holds. If the system is AMS and either the
< f >n are uniformly integrable with respect to m or f is m-integrable, then also with probability
one
< f >n n Em f.
In particular, the limiting relative frequencies are given by
n1

n1
X

1F T i n m(F ), all F B.

i=0

In addition,
lim n1

n1
XZ
i=0

f T i dm = (Em f )m(G), all G B,

CHAPTER 6. ERGODIC PROPERTIES

148

where again the result holds for measurements for which the < f >n are uniformly integrable or f
is m integrable. Letting f be the indicator function of an event F , then
lim n1

n1
X

m(T i F G) = m(F )m(G).

i=0

Proof: If the limiting sample average is constant, then it must equal its own expectation. The
remaining results then follow from the results of this chapter.
2
The final result of the previous lemma yields a useful test for determining whether an AMS
system is ergodic.
Lemma 6.7.4 Suppose that a dynamical system (, B, m, T ) has ergodic properties with respect to
the indicator functions and hence is AMS with stationary mean m and suppose that B is generated
by a field F, then m is ergodic if and only if
lim n1

n1
X

m(T i F F ) = m(F )m(F ), all F F.

i=0

Proof: Necessity follows from the previous result. To show that the preceding formula is sufficient
for ergodicity, first assume that the formula holds for all events F and let F be invariant. Then
the left-hand side is simply m(F ) = m(F ) and the right-hand side is m(F )m(F ) = m(F )2 . But
these two quantities can be equal only if m(F ) is 0 or 1, i.e., if it (and hence also m) is ergodic. We
will be done if we can show that the preceding relation holds for all events given that it holds on a
generating field. Toward this end fix > 0. From Corollary 1.5.3 we can choose a field event F0 such
that m(F F0 ) and m(F F0 ) . To see that we can choose a field event F0 that provides a
good approximation to F simultaneously for both measures m and m, apply Corollary 1.5.3 to the
mixture measure p = (m + m)/2 to obtain an F0 for which p(F F0 ) /2. This implies that both
m(F F0 ) and m(F F0 )) must be less than .
From the triangle inequality
|

n1
1X
m(T i F F ) m(F )m(F )|
n i=0

n1
n1
1X
1X
m(T i F F )
m(T i F0 F0 )|
n i=0
n i=0

+|

n1
1X
m(T i F0 F0 ) m(F0 )m(F0 )| + |m(F0 )m(F0 ) m(F )m(F )|.
n i=0

The middle term on the right goes to 0 as n by assumption. The rightmost term is bound
above by 2. The leftmost term is bound above by
n1
1X
|m(T i F F ) m(T i F0 F0 )|.
n i=0

Since for any events D, C

|m(D) m(C)| m(DC)

(6.43)

6.7. ERGODICITY

149

each term in the preceding sum is bound above by m((T i F F )(T i F0 F0 )). Since for any
events D, C, H m(DC) m(DH) + m(HC), each of the terms in the sum is bound above by
m((T i F F )(T i F0 F )) + m((T i F0 F )(T i F0 F0 )) m(T i (F F0 )) + m(F F0 ).
Thus the remaining sum term is bound above by
n1
1X
m(T i (F F0 )) + m(F F0 ) n m(F F0 ) + m(F F0 ) 2,
n i=0

which completes the proof.

Examples
We next consider a few examples of ergodic processes and systems. Suppose that we have a random process {Xn } with distribution m and alphabet A. The process is said to be memoryless or
independent and identically distributed (IID) if there is a measure q on (A, BA ) such that for any
rectangle G = iJ Gi , Gi BA , j J , J I a finite collection of distinct indices,
Y
q(Gj );
m(jJ Gj ) =
jJ

that is, the random variables Xn are mutually independent. Clearly an IID process has the property
that if G and F are two finite-dimensional rectangles, then there is an integer M such that for
N M
m(G T n F ) = m(G)m(F );
that is, shifting rectangles sufficiently far apart ensures their independence. A generalization of this
idea is obtained by having a process satisfy this behavior asymptotically as n . A measure m
is said to be mixing or strongly mixing with respect to a transformation T if for all F, G B
lim |m(T n F G) m(T n F )m(G)| = 0

(6.44)

and weakly mixing if for all F, G B

|m(T i F G) m(T i F )m(G)| = 0.

(6.45)

If a measure is mixing, then it is also weakly mixing. We have not proved, however, that an IID
process is mixing, since the mixing conditions must be proved for all events, not just the rectangles
considered in the discussion of the IID process. The following lemma shows that proving that (6.44)
or (6.45) holds on a generating field is sufficient in certain cases.
Lemma 6.7.5 If a measure m is AMS and if (6.45) holds for all sets in a generating field, then m
is weakly mixing. If a measure m is asymptotically stationary in the sense that limn m(T n F )
exists for all events F and if (6.44) holds for all sets in a generating field, then m is strongly mixing.
Proof: The first result is proved by using the same approximation techniques of the proof of Lemma
6.7.4, that is, approximate arbitrary events F and G, by events in the generating field for both m
and its stationary mean m. In the second case, as in the theory of asymptotically mean stationary
processes, the Vitali-Hahn-Saks theorem implies that there must exist a measure m such that
lim m(T n F ) = m(F ),

CHAPTER 6. ERGODIC PROPERTIES

150

and hence arbitrary events can again be approximated by field events under both measures and the
result follows.
2
An immediate corollary of the lemma is the fact that IID processes are strongly mixing since
(6.44) is satisfied for all rectangles and rectangles generate the entire event space.
If the measure m is AMS, then either of the preceding mixing properties implies that the condition
of Lemma 6.7.4 is satisfied since
1

|n

n1
X

m(T i F F ) m(F )m(F )|

i=0

|n1

n1
X

(m(T i F F ) m(T i F )m(F )| + |n1

i=0

n1

n1
X

n1
X

m(T i F )m(F )) m(F )m(F )|

i=0

|m(T i F F ) m(T i F )m(F )| + m(F )|n1

i=0

n1
X

m(T i F ) m(F )| n 0.

i=0

Thus AMS weakly mixing systems are ergodic, and hence AMS strongly mixing systems and IID
processes are also ergodic. A simpler proof for the case of mixing measures follows from the observation that if F is invariant, then mixing implies that m(F F ) = m(F )2 , which in turn implies
that m(F ) must be 0 or 1 and hence that m is ergodic.

Exercises
1. Show that if m and p are two stationary and ergodic processes with ergodic properties with
respect to all indicator functions, then either they are identical or they are singular in the
sense that there is an event G such that p(G) = 1 and m(G) = 0.
2. Suppose that mi are distinct stationary and ergodic sources with ergodic properties with
respect to all indicator functions. Show that the mixture
X
i mi (F ),
m(F ) =
i

where

i = 1

is stationary but not ergodic. Show that more generally if the mi are AMS, then so is m.
3. A random process is N -ergodic if it is ergodic with respect to T N , that is, if T N F = F for all
F implies that m(F ) = 1. Is an N -ergodic process necessarily ergodic? Is an ergodic process
necessarily N -ergodic? If m is N -ergodic, is the mixture
m(F ) =

N 1
1 X
m(T i F )
N i=0

ergodic? Suppose that a process is N -ergodic and N -stationary and has the ergodic property
with respect to f . Show that
N 1
1 X
E(f T i ).
Ef =
N i=0

6.7. ERGODICITY

151

4. If a process is stationary and ergodic and T is invertible, show that m(F ) > 0 implies that
m(

T i F ) = 1.

i=

Show also that if m(F ) > 0 and m(G) > 0, then for some i m(F T i G) > 0. Show that, in
fact, m(F T i G) > 0 infinitely often.
5. Suppose that (, B, m) is a directly given stationary and ergodic source with shift T and that
f : is a measurable mapping of sequences in into another sequence space with shift
S. Show that the induced process (with distribution mf 1 ) is also stationary and ergodic.
6. Show that if m has ergodic properties with respect to all indicator functions and it is AMS
with stationary mean m, then
n1 n1
n1
1X
1 XX
m(T i F F ) = lim 2
m(T i F T j F ).
n n
n n
i=0
i=0 j=0

lim

7. Show that a process m is stationary and ergodic if and only if for all f , g L2 (m) we have
n1
1X
E(f gT i ) = (Ef )(Eg).
n n
i=0

lim

8. Show that the process is AMS with stationary mean m and ergodic if and only if for all bounded
measurements f and g
n1
1X
Em (f gT i ) = (Em f )(Em g).
lim
n n
i=0
9. Suppose that m is a process and f L2 (m). Assume that the following hold for all integers k
and j:
E m f T k = Em f

Em (f T k )(f T j ) = Em (f )(f T |kj| )

lim Em (f f T |k| ) = (Em f )2 .

Prove that < f >n Em f in L2 (m).

10. Prove that if a stationary process m is weakly mixing, then there is a subsequence f atJ =
{j1 < j2 < . . .} of the positive integers that has density 0 in the sense that
n1
1X
1J (i) = 0
n n
i=0

lim

and that has the property that

lim

n,n6J

|m(T n F G) m(F )m(G)| = 0

for all appropriate events F and G.