Introduction to Hilbert Spaces

Herman J. Bierens Pennsylvania State University (June 24, 2007)

1.

Vector spaces

The notion of a vector space should be known from linear algebra:

Definition 1. Let V be a set endowed with two operations, the operation "addition", denoted by "+", which maps each pair (x,y) in V V into V, and the operation "scalar multiplication", denoted by a dot (.), which maps each pair (c,x) in V [or V] into V. Thus, a scalar is a real or complex number. The set V is called a real [complex] vector space if the addition and multiplication operations involved satisfy the following rules, for all x, y and z in V, and all scalars c, c1 and c2 in []: : (a) (b) (c) (d) (e) (f) (g) (h) x + y = y + x; x + (y + z) = (x + y) + z; There is a unique zero vector 0 in V such that x + 0 = x; For each x there exists a unique vector !x in V such that x + (!x) = 0;1 1.x = x; (c1c2).x = c1.(c2.x); c.(x + y) = c.x + c.y; (c1 + c2).x = c1.x + c2.x. It is trivial to verify that the Euclidean space n is a real vector space. However, the notion of a vector space is much more general. For example, let V be the space of all continuous functions on , with pointwise addition and scalar multiplication defined the same way as for real numbers. Then it is easy to verify that this space is a real vector space.

In the sequel, x + (!y) will be denoted by x ! y. 1

Another (but weird) example of a vector space is the space V of positive real numbers endowed with the "addition" operation x + y = x.y and the "scalar multiplication" c.x = xc. In this case the null vector 0 is the number 1, and !x = 1/x.

Definition 2. A subspace V0 of a vector space V is a non-empty subset of V which satisfies the following two requirements: (a) (b) For any pair x, y in V0, x + y is in V0; For any x in V0 and any scalar c, c.x is in V0.

Thus, a subspace V0 of a vector space is closed under linear combinations: any linear combination of elements in V0 is an element of V0. It is not hard to verify that a subspace of a vector space is a vector space itself, because the rules (a) through (h) in Definition 1 are inherited from the "host" vector space V. In particular, any subspace contains the null vector 0, as follows from part (b) of Definition 2 with c = 0. Definition 3. An inner product on a real vector space V is a real function <x,y>: VV 6 such that for all x, y, z in V and all c in , (1) (2) (3) (4) <x,y> = <y,x> <cx,y> = c<x,y> <x+y,z> = <x,z> + <y,z> <x,x> > 0 when x 0.

An inner product on a complex vector space is defined similarly. The inner product is then complex-valued, <x,y>: VV 6 . Condition (1) then becomes (1*) <x,y> ' <y,x> , 2

and (2) now holds for all complex numbers c. Note that in both cases, <x,x> is real valued. Finally, the norm of x in V is defined as ||x|| ' <x,x> .

The bar denotes the complex conjugate: for z ' a % i.b, z ' a & i.b. 2

For example, in the space C[0,1] of continuous real functions on [0,1], the integral <f,g> = f(t)g(t)dt is an inner product. Moreover, in the vector space of zero-mean random variables m0 with finite second moments the expectation <X,Y> ' E[X.Y] is an inner product.
1

As is well-known from linear algebra, for vectors x , y 0 n, |x Ty| # ||x||.||y||, which is known as the Cauchy-Schwarz inequality. This inequality carries over to general inner products: Theorem 1. (Cauchy-Schwarz inequality) |< x,y>| # ||x||.||y||. Proof: Let the vector space involved be real. Then for any real 8, 0 # <x%y , x%y> ' <x , x%y> % <y , x%y> ' <x , x> % 2<x , y> % 2< y , y> ' ||x||2 % 2<x , y> % 2||y||2 . Minimizing the latter to 8 yields the result. The complex case is similar. Q.E.D. Given the norm ||x|| ' <x,x>, the following properties hold: ||x|| > 0 if x 0; ||c.x|| ' |c|.||x||; ||x%y|| # ||x|| % ||y||. [Triangular inequality] (1) (2) (3)

The properties (1) and (2) follow trivially from Definition 3. In the case of a real vector space the triangular inequality (3) follows from ||x%y||2 ' <x%y,x%y> ' <x,x%y> % <y,x%y> ' <x,x> % <x,y> % <y,x> % <y,y> ' ||x||2 %2<x,y> % ||y||2 # ||x||2 % 2|<x,y>| % ||y||2 # ||x||2 % 2||x||.||y|| % ||y||2 ' ||x|| % ||y|| 2 where the last inequality is due to Theorem 1. A norm can also be defined directly: Definition 4. A norm on a vector space V is a mapping ||.||: V 6 [0,4) such that for all x and y in V and all scalars c, (1), (2) and (3) hold. A vector space endowed with a norm is called a normed

space.

A norm ||.|| defines a metric d(x,y) ' ||x&y|| on V, i.e., a function that measures the distance between two elements x and y of V, for which (trivially) the following four properties hold. For all x, y and z in V, d(x,y) ' d(y,x) ; d(x,y) > 0 if x y ; d(x,x) ' 0 ; d(x,z) # d(x,y) % d(y,z) [Triangular inequality] . More generally, Definition 5. A metric on a space V is a mapping d(.,.): VV 6 [0,4) satisfying the properties (4) through (7) for all x , y and z in V. A space endowed with a metric is called a metric space. (4) (5) (6) (7)

In this definition the space V is not necessarily a vector space: Any space endowed with a metric is a metric space. Moreover, inner products and norms may not be defined on metric spaces.

Definition 6. A sequence of elements xn of a metric space with metric d(.,.) is called a Cauchy sequence if for every g > 0 there exists an n0 such that for all k,m $ n0, d(xk,xm) < g.

The notion of a Cauchy sequence plays a critical role in defining Hilbert spaces. See the next section. Theorem 2. Every Cauchy sequence in R or R , R < 4 , has a limit in the space involved. Proof: Consider first the case . Let x ' limsupn64xn , where xn is a Cauchy sequence. I will show first that x < 4 . There exists a subsequence nk such that x ' limk64xn . Note that xn is
k k

also a Cauchy sequence. For arbitrary g > 0 there exists an index k0 such that |xn & xn | < g if
k m

k,m $ k0. Keeping k fixed and letting m 6 4 it follows that |xn & x| < g, hence x < 4 . Similarly,
k

x ' liminfn64xn > &4 . Now we can find an index k0 and subsequences nk and nm such that for k,m $ k0, |xn & x| < g, |xn & x| < g, and |xn & xn | < g, hence |x & x| < 3g. Since g is
k m k m

arbitrary, we must have x ' x ' limn64xn . Applying this argument to the real and imaginary parts of a complex Cauchy sequence the result for the case follows, and applying the argument to each component of a (complex) vector valued Cauchy sequence the result for the cases R and R follow. Q.E.D.

2.

Hilbert spaces x Tx ' <x,x> and associated metric ||x ! y||, such that every Cauchy sequence takes a limit

A Euclidean space n is a vector space endowed with the inner product <x,y> ' x Ty , norm ||x|| = in n . This makes n a Hilbert space:

Definition 7. A Hilbert space H is a vector space endowed with an inner product and associated norm and metric, such that every Cauchy sequence in H has a limit in H.

A Hilbert space is also a Banach space:

Definition 8. A Banach space B is a normed space with associated metric d(x,y) = ||x&y|| such that every Cauchy sequence in B has a limit in B.

The difference between a Banach space and a Hilbert space is the source of the norm. In the Hilbert space case the norm is defined via the inner product, ||x|| ' <x,x> , whereas in the Banach space case the norm is defined directly, by Definition 4. Thus, a Hilbert space is a Banach space, but the other way around may not be true, because in some cases the norm cannot be associated with an inner product. An example of a Hilbert space is the space L2(a,b):

Definition 9. The space L2(a,b) is the collection of Borel measurable real or complex valued square integrable functions f on (a,b), i.e., <f,g> ' ma |f(t)|2dt < 4 , endowed with inner product ma b f(t)g(t)dt , and associated norm and metric
b

|f(t)|2dt , d(f,g) ' ||f&g|| ' |f(t)&g(t)|2dt , ma ma respectively, where the integrals involved are Lebesgue integrals. ||f || '
b b

Note that in this case f and g are interpreted as being equal if they differ on (a,b) only on a set with Lebesgue measure zero. The proof that L2(a,b) is a Hilbert space will be given in Section 6 below. Recall that vectors x and y in n are orthogonal if x Ty ' 0. More generally,

Definition 10. Elements x, y of a Hilbert space are orthogonal if <x,y> = 0, also denoted by x z y , and orthonormal if in addition ||x|| = 1 and ||y|| = 1.

Note that in a Banach space orthogonality is a non-existing property, because an inner product is not defined on a Banach space. This is the main reason for working with Hilbert spaces.

Definition 11. An orthonormal sequence en, n =1,2,3,... in a Hilbert space H is complete if the only member of H which is orthogonal to all en is the zero vector.

for every x in H, x ' 'n'1< x,en>en and ||x||2 ' 'n'1|< x,en>|2 .
4 4

Theorem 3. Let en, n =1,2,3,... be a complete orthonormal sequence in a Hilbert space H. Then

Proof: Observe that 0 # ||x & 'n'1< x,en>en||2 ' x & 'n'1< x,en>en , x & 'n'1< x,en>en
m m m

' x , x & 'n'1< x,en>en & 'n'1< x,en> en , x & 'n'1< x,en>en
m m m

(8)

' ||x||2 & 'n'1|< x,en>|2

m

hence, letting m 6 4, we have 'n'1|< x,en>|2 # ||x||2 < 4 . Therefore,

4

Let ym '

m 'n'1< x,en>en

limm64'n'm|< x,en>|2 ' 0

4 m

(9)

and note that

||x & ym||2 ' min ,...., ||x & 'n'1nen||2.

1 m

(10)

Thus, we can write

x ' ym % zm ' 'n'1< x,en>en % z m, where <zm , en> ' 0 for all n # m ,
m

(11)

hence <ym,z m> ' 0 and therefore ||x||2 ' ||ym||2 % ||zm||2 . Moreover, ||ym & yk||2 ' 'n'min(k,m)|< x,en>|2 # 'n'min(k,m)|< x,en>|2 6 0
max(k,m) 4

(12)

(13)

for min(k,m) 6 4 , hence it follows from (9) that ym is a Cauchy sequence. Therefore, it follows from the properties of a Hilbert space that yn converges to a limit y 0 H : limm64||y & ym|| ' limm64||y & 'n'1< x,en>en|| ' 0.
m m

(14)

Thus, we can write y as

y ' 'n'1< x,en>en % u m ' ym % u m, where limm64||um|| ' 0. y ' 'n'1< x,en>en .
4

(15)

This result gives rise to the notation

(16)

However, keep in mind that the actual definition of y is given by (15). On the other hand, without loss of generality we may treat y as the right-hand side of (16), as will be demonstrated below. {ek} that z = 0, hence x = y. To prove this, note that by (15), z ' x & 'n'1< x,en>en & um for
m

Let z ' x & y. If <z , ek> = 0 for all k then it follows from the completeness of

all m. Then for all m > k, <z , ek> ' <x , ek> & 'n'1<x , en><ek , en> & <ek , u m>
m

' <x , ek> & <x , ek><ek , ek> & <ek , um> ' <x , ek> & <x , ek> & <ek , u m> ' & <ek , u m> hence by Theorem 1 and (15), |<z,ek>| ' limm64|<ek,um>| # limm64||um||.||ek|| ' limm64||um|| ' 0. Thus, x = y. It follows now from (11) and (15) that zm ' u m, hence 7

(17)

(18)

x ' 'n'1< x,en>en % u m, where <en , u m> ' 0 for all n # m, and limm64||um|| ' 0.
m

(19)

Again, this result is denoted by

x ' 'n'1< x,en>en.

4

(20)

This proves the first part of the theorem. The second part follows now easily from (8). Q.E.D. right-hand side of (16), because then z ' x & 'n'1< x,en>en , hence
4 4

Note that the result (18) could have been obtained more directly by treating y as the

<z , ek> ' <x , ek> & 'n'1<x , en><ek , en> ' <x , ek>&<x , ek><ek , ek> ' <x , ek>&<x , ek> ' 0. <x,v> ' limm64'n'1< x,en><en,v> % limm64<um,v>
m

Similarly, (19) implies that for any v 0 H,

'

4 'n'1< x,en><en,v>

% limm64<um,v> '

4 'n'1< x,en><en,v>

(21)

where the last equality follows from the fact that by Theorem 1 and (19), |limm64<u m,v>| # limm64||um||.||v| ' 0, whereas the result (21) would have followed trivially if we has used (20). Definition 12. The coefficients < x,en> in the series representation x = 'n'1< x,en>en are called
4

the Fourier coefficients of x.

Definition 13. Let A be a set of vectors in a Hilbert space H. Then Lin(A) is the intersection of all sub-spaces of H which contain A, and Clin(A) is the closure of Lin(A).

In other words, Lin(A) is the set of all linear combinations of the elements of A. If A is a finite then Clin(A) = Lin(A). The same applies if A d n, because there are only a finite number of linear independent vectors in A, which can be made orthonormal..

Theorem 4. Let en, n =1,2,3,... be an orthonormal sequence in a Hilbert space H. Then the following statements are equivalent. (1) (2) en is complete; H ' Clin({en, n$1}) . 8

(3)

For all x in H, ||x||2 ' 'n'1|<x,en>|2 .

4

Proof: (1) Y (2) and (1) Y (3) has already been proved in Theorem 3. Proof of (3) Y (1): Suppose that the sequence en is not complete. Then there exists a z 0 in H
4

such that <z, en> = 0 for all n. But then by part (3) , ||z||2 ' 'n'1|< z,en>|2 ' 0 , which contradicts z 0. Proof of (2) Y (1): Let <z, en> = 0 for all n, and define E ' {x 0 H: <x,z> ' 0} . Then E contains all en , and for fixed z, E is a subspace, so that Lin({en, n$1}) d E. Moreover, it is not hard to verify from the continuity of the inner product that E is closed, hence H = Clin({en, n$1}) d E . But z 0 H, and thus z 0 E, and consequently, <z,z> = 0. This proves (1). Q.E.D.

3.

Fourier analysis

The following theorem implies that for every Borel measurable real or complex valued function f in L2(!,) has the series expansion f(x) ' (2)&1'n'&4cnexp(i.n.x) , where cn '
4

m&

f(y)exp(&i.n.y)dy .

(22)

Recall from the proof of Theorem 3 that (22) should be interpreted as: For all m > 0, f(x) ' (2)&1'n'&mcnexp(i.n.x) % gm(x), where
m

limm64 |gm(x)|2dx ' 0. m&

(23)

Theorem 5. The complex functions en(x) ' (2)&exp(i.n.x) , n ' 0 , 1 , 2 ,.... form a complete orthonormal sequence in L2(!,).

The orthonormality of the sequence {en(x)} follows from the fact that

<en,em> ' (2)&1

m&

exp(i.(n&m).x)dx ' (2)&1

m&

cos((n&m).x)dx % i.(2)&1

m&

sin((n&m).x)dx

' (2)&1

m&

cos((n&m).x)dx '

sin((n&m)) ' 0 if n m , (n&m) 1 if n ' m.

The rest of the proof of Theorem 5 is too long and is therefore given in the Appendix, Section A.1. Since every function g in L2(a,b) can be converted to a function f in L2(!,), namely f(x) ' g a%(b&a)(x%)/(2) and vice versa, g(x) ' f &2(b&x)/(b&a) , it follows from (22) that g(x) ' (2)&1/2'n'&4cnexp i.n. &2(b&x)/(b&a)
4

(24)

where cn ' (2)&1/2 m&

f(y)exp(&i.n.y)dy ' (2)&1/2 '

m&

g a%(b&a)(y%)/(2) exp(&i.n.y)dy (25)

2 b g(x)exp &i.n. &2(b&x)/(b&a) dx b&a ma

Therefore, Theorem 6. Every function g in L2(a,b), &4 < a < b < 4 , can be written as where n ' (b&a)&1 ma g(x) ' 'n'&4 nexp 2n.i.x/(b&a)
4 b

g(x)exp &2 n.i.x/(b&a) dx .

If g is real valued, then n ' (b&a)&1 ' n&in , ma

b

g(x) cos 2 n.i.x/(b&a) dx & i(b&a)&1

ma

g(x) sin 2 n.i.x/(b&a) dx (26)

say, so that the result of Theorem 6 reads g(x) ' 0 % 'n'1 nexp 2n.i.x/(b&a) % 'n'1 nexp &2n.i.x/(b&a)
4 4

' 0 % 2'n'1 ncos 2n.x/(b&a) % 2'n'1 nsin 2n.x/(b&a) a.e. on (a,b) .

(27)

10

4.
b1 b2

Functions of two or more variables |g(x1,x2)|2dx1dx2 < 4, the set N2 ' {x2 0 (a2 , b2) :
b1

Let g(x1,x2) be a function in L 2 (a1 , b1)(a2 , b2 , &4 < aj < b j < 4 , j ' 1,2 . Since

|g(x1 , x2)|2dx1 ' 4} has Lebesgue ma1 ma2 ma1 measure zero. Consequently, for every fixed x2 0 (a2 , b2) \N2 the function g(x1,x2) is a member of L 2(a1 ,b1) . Applying Theorem 6 then yields
4

g(x1 , x2) ' 'n'&4 n(x2)exp 2n.i.x1/(b1&a1) a.e. on (a1,b1) , n(x2) ' (b1&a1)&1
4 b1

(28)

where g(x1 , x2)exp &2 n.i.x1/(b1&a1) dx1 . ma1 But (29) is a member of L 2(a2 ,b2) , so that by Theorem 6, where n,m ' (b1&a1)&1(b2&a2)&1 1 2g(x1 , x2)exp &2 n.i.x1/(b1&a1) exp &2 m.i.x2/(b2&a2) dx1dx2 . ma1 ma2 Substituting (30) in (28) yields
b b

(29) (30)

n(x2) ' 'm'&4 n,mexp 2m.i.x2/(b2&a2) a.e. on (a2,b2) ,

(31)

g(x1 , x2) ' 'n'&4 'm'&4 n,mexp 2n.i.x1/(b1&a1) exp 2m.i.x2/(b2&a2) .

4 4

(32)

This result can be extended to functions of more than two variables: Theorem 7. Let C ' j'1(aj , bj) , &4 < aj < b j < 4 , let D be the diagonal matrix with diagonal elements b1&a1 , .... , bk&a k , and denote by k the set of k-dimensional vectors with where for n 0 k, (n) ' det[D &1] mC g(y)exp[&2.i.n TD &1y]dy . integer components. Then for every function g(x) in L 2(C) , g(x) ' 'n0k (n)exp[2.i.n TD &1x]
k

Again, we can write (n) ' (n) & i.(n) , where (n) ' det[D &1] (n) ' det[D &1] so that if g is real valued, mC g(y)cos[2n TD &1y]dy , (33) g(y)sin[2n TD &1y]dy ,

mC

11

g(x) ' 'n0k ((n)&i.(n)) cos[2.n TD &1x] % i.sin[2.n TD &1x]

' (0) % 'n0k\{0} (n)cos[2.n TD &1x] % 'n0k\{0} (n)sin[2.n TD &1x].

(34)

5.

Hilbert spaces of random variables

As mentioned before, for zero-mean random variables X and Y with finite second moments the expectation E[X.Y] can be interpreted as an inner product:

Theorem 8. Let H be the space of zero-mean random variables with finite second moments defined on a common probability space {,,P}, endowed with the inner product <X,Y> = E[X.Y], norm ||X|| = | E[X 2] and metric ||X-Y||. Then H is a Hilbert space.

Proof: It is trivial to verify that the space of these random variables is a vector space. Therefore, we only need to show that every Cauchy sequence Xn ,n $ 1, has a limit in H, as follows. Since by Chebishevs inequality, P[|Xn&Xm| > g] # E[(Xn&Xm)2]/g2 ' ||Xn&Xm||2/g2 6 0 as n,m 6 4 for every g > 0, it follows that |Xn&Xm| 6 0 in probability as n,m 6 4. Therefore, there exists a subsequence nk such that |Xn &Xn | 6 0 a.s. as n,m 6 4. 3 The latter implies that there exists a
k m

null set N such that for every 0 \N , Xn () is a Cauchy sequence in , hence

k

limk64Xn () ' X() exists for every 0 \N . Now for every fixed m,
k

(Xn &Xm)2 6 (X&Xm)2 a.s. as k 6 4. By Fatous lemma (see the Appendix, Section A.2) and the
k

Cauchy property the latter implies that ||X&Xm||2 ' E[(X&Xm)2] # liminfk64E[(Xn &Xm)2] 6 0 as
k

m 6 4. Moreover, it is easy to verify that E[X] ' 0 and E[X ] < 4 . Q.E.D.
2

Note that the result of Theorem 8 can be translated in the following way:

This follows from the fact that if Xn 6 0 in probability then every subsequence nk contains a further subsequence nk(m) such that Xn (m) 6 0 a.s. as m 6 4.
3
k

12

Corollary 1. Let {,,P} be a probability space, and let L0 (P) be the space of measurable functions4 X(): 6 satisfying X()2dP() < 4 , X()dP() ' 0, endowed with the m m inner product <X , Y> = X()Y()dP() and associated norm ||X|| ' <X , X> and metric m 2 ||X & Y||. Then L0 (P) is a Hilbert space. Moreover, if we take = , = probability measure on

B and P = , where B is the Euclidean Borel field and is a

B , then Corollary 1 reads:

2

Corollary 2. The space L0 () of Borel measurable real functions on satisfying f(x)2d(x) < m 4 and f(x)d(x) ' 0 , endowed with the inner product < f , g > = f(x)g(x)d(x) and associated m m norm ||f || ' < f , f > and metric ||f & g||, is a Hilbert space. The space L 2(a,b)
2

6.

The results in Section 5 can be used to prove that the space L 2(0,1) is a Hilbert space, which then implies that L 2(a,b) is a Hilbert space. First, consider the subspace L0 (0,1) of Borel f(x)dx ' 0 . It follows straightforwardly from m0 2 Corollary 2 that L0 (0,1) is a Hilbert space. Now let fn be a Cauchy sequence in L 2(0,1). Then measurable functions f in L 2(0,1) satisfying m0 limn64||g&gn|| = 0 because gn(x) = fn(x) &
1 1

f n(u)du is a Cauchy sequence in L0 (0,1): there exists a g in L0 (0,1) such that

/0m0

f k(u)du &

m0

fm(u)du # |f (u) & fm(u)|du # /0 m0 k

1

m0

|f k(u) & fm(u)|2du ' ||f k & fm||, m0

1

(35)

hence ||gk & gm|| # 2||fk & f m||. But inequality (35) also implies that sequence in the Hilbert space , and therefore
1

fn(u)du is a Cauchy

' limn64 fn(u)du 0 . m0 2 Next, let f = g + . Then f 0 L 2(0,1) because g 0 L0 (0,1) d L 2(0,1) and the constant function is a member of L 2(0,1) . Moreover,

Recall that X(): 6 is measurable if for all Borel sets B the sets { 0 : X() 0 B} are members of .
4

13

||fn & f|| # ||g n & g|| % as n 6 4. Thus, L 2(0,1) is a Hilbert space.

/0m0

fn(u)du & 6 0 /0

Along the same lines it is easy to show that: Theorem 9. The space L 2(a,b) is a Hilbert space,

and more generally that: Theorem 10. The space L 2() of Borel measurable real functions on satisfying f(x)2d(x) < m 4, endowed with the inner product < f , g > = f(x)g(x)d(x) and associated norm ||f || ' < f , f > m and metric ||f & g||, is a Hilbert space.

Appendix
A.1 Proof of Theorem 5 The proof of Theorem 5 involves the following two steps: (1) Show that any continuous 2-periodic real function f on [!,], confined to (!,), is

contained in Clin({ en , n ' 0 , 1 , 2 ,.... }). Then the same applies to complex continuous 2periodic functions on [!,] , by applying the argument to the real and imaginary parts of f. (2) Show that the subset CP [!,] of continuous 2-periodic functions on [!,], confined to

(!,), is a dense subset of L2(!,), i.e., L2(!,) is the closure of L2(!,)1CP [!,]. Then the result follows from Theorem 4. Step 1. Let f be a continuous real function on [!,], and let fm ' 'n'&m<f,en>en and
m

Fm ' (m%1)&1'n'0 fn .
m

Clearly, Fm 0 Lin({ en , n ' 0 , 1 , 2 ,.... }). Therefore, f 0 Clin({ en , n ' 0 , 1 , 2 ,.... }) if Fm 6 f, so I will show that the latter is true. Observe that < f , en> ' < en , f > ' (2)& f(x)exp(&i.n.x)dx , hence m& m &1 f m(y) ' (2) f(x)'n'&mexp(i.n.(y&x))dx m& and thus

14

Fm(y) ' (2)&1 say, where

m&

f(x) (m%1)&1'j'0'n'&jexp(i.n.(y&x)) dx ' (2)&1

m j

m&

f(x)Km(y&x)dx,

K m(t) ' (m%1)&1'j'0'n'&jexp(i.n.t) , t 0 .

m j

The latter function is known as the Fejer kernel. Note that for t such that exp(i.t) = 1, Km(t) ' (m%1)&1'j'0(1%2j) ' 1 % m ,
m

whereas for all t such that exp(i.t) 1, K m(t) ' (m%1)&1'j'0 1 % 'n'1exp(i.n.t) % 'n'1exp(&i.n.t)
m j j

' (m%1)&1'j'0 'n'0 exp(i.t)

m j

% 'n'0 exp(&i.t)
j

& 1

' (m%1)&1'j'0
m

1&exp(i.t(j%1)) 1&exp(&i.t(j%1)) % & 1 1&exp(i.t) 1&exp(&i.t) 1 1 % & 1 1&exp(i.t) 1&exp(&i.t)

' & (m%1)&1 ' &(m%1)&1

exp(i.t) m exp(&i.t) m 'j'0exp(i.t.j) & (m%1)&1 'j'0exp(&i.t.j) 1&exp(i.t) 1&exp(&i.t)

exp(i.t) 1&exp(i.t.(m%1)) exp(&i.t) 1&exp(&i.t.(m%1)) & (m%1)&1 1&exp(i.t) 1&exp(i.t.) 1&exp(&i.t) 1&exp(&i.t.) ' (m%1)&1 ' (m%1)&1 2&exp(i.t.(m%1))&exp(&i.t.(m%1)) (1&exp(i.t))(1&exp(&i.t.)) sin((m%1)t/2) sin(t/2)
2

2&2cos(t.(m%1)) 1 ' 2&2cos(.t) m%1

Thus, Km(t) $ 0 and Km(t) = Km(!t). Moreover, it is not hard to show that and that for any fixed 0 (0 , ) , limm64 m&
&

K m(t)dt ' 2 ,

(36) ' 0.

m& m Note that (36) implies, replacing t by y - x, that K m(t)dt % my&

y%

Km(t)dt

(37)

K m(y&x)dx ' 2 ,
y%

hence for y 0 [!,],

f(y) ' (2)&1

my&

f(y)Km(y&x)dx. 15

(38)

Since f and Km are both 2-periodic, it follows for each y, f(x)Km(y&x) is 2-periodic. Since a 2periodic function has the same integral over [y&,y%] as over [&,] , we have m& my& Hence, it follows from (38) and (39) that for any 0 (0,), Fm(y) ' (2)&1

f(x)K m(y&x)dx ' (2)&1

y%

f(x)Km(x&y)dx

(39)

*Fm(y)& f(y)* # (2)&1 ' (2)&1 my&

y&

my&

y%

*f(x)&f(y)*K m(x&y)dx my&

y%

*f(x)&f(y)*Km(x&y)dx % (2)&1 % (2)&1 my%

y%

*f(x)&f(y)*K m(x&y)dx (40)

*f(x)&f(y)*K m(x&y)dx

' &1sup&#x#|f(x)| Km(t)dt % &1sup&#x#|f(x)| K m(t)dt m& m

&

% (2)&1

my&

y%

*f(x)&f(y)*K m(x&y)dx

Since f is bounded, it follows from (37) that the first and second integral at the right hand side of (40) converge to zero as m 6 4, whereas the third term can be bounded by (2)&1 my&
y%

*f(x)&f(y)*K m(x&y)dx # sup|x&y|#|f(x)&f(y)|(2)&1 ' sup|x&y|#|f(x)&f(y)|,

m&

Km(t)dt (41)

where the equality follows from (36). Since continuous functions on a compact interval are uniformly continuous, it follows that f is uniformly continuous on [!,], and since f is 2periodic it follows that f is uniformly continuous on . It follows therefore from (40) and (41) that limm64sup&#y#*Fm(y)& f(y)* ' 0 . Note that (42) implies limm64||Fm & f || ' 0 , which completes step 1 of the proof. Step 2. First, note that every continuous real function on [!,] can be written as a limit of 2periodic functions on [!,]. In particular, define for a continuous real function f on [!,], (42)

16

f(x) f n(x) ' n.f(&n &1)(&x)

if &%n &1 # x # &n &1 , if &n &1 < x # .

n.f(&%n &1)(x%) if 0 # x < &%n &1,.

Then fn is continuous on [!,], and since fn(&) ' f n() (' 0) , it can be extended to the real line as a 2-periodic function. Moreover, ||fn & f ||2 ' m&
&%1/n

(f n(x)&f(x))2dx %

m&1/n

(fn(x)&f(x))2dx #

2 sup&#x#f(x)2 6 0 n

as n 64. A similar result holds for complex-valued continuous functions. Consequently, the closure of the space CP [!,] of continuous 2-periodic functions on [!,] contains all the continuous functions on [!,]. Next, let B be an arbitrary Borel subset of [!,], and let for n =1,2,3,.., fn(x) ' exp &n &1infy0B|x&y| & exp &n &1infy0B\B|x&y| , x 0 [& , ] , where B is the closure of B. Note that fn(x) is continuous on [!,] because infy0B|x&y| is a continuous function on [!,], and so is infy0B\B|x&y| . To see this, observe that for any pair x1 , x2 0 [&,] , infy0B|x1&y| # |x1&x2| % infy0B|x2&y| and infy0B|x2&y| # |x1&x2| % infy0B|x1&y| , hence infy0B|x1&y| & infy0B|x2&y| # |x1&x2|, and similarly, infy0B\B|x1&y| & infy0B\B|x2&y| # |x1&x2|. Since limn64 fn(x) ' I(x 0 B) , where I(.) is the indicator function, and therefore by bounded convergence, limn64 ||fn(x) & I(x 0 B) || ' 0, the function I(x 0 B) is contained in the closure of the space CP [!,], and consequently, all real simple functions on [!,] are included as well. Finally, since a Borel measurable function is a limit of a sequence of simple functions, it is easy to verify from the dominated convergence theorem that all Borel measurable functions f |f(t)|2dt < 4 are included in the closure of the space CP [& ,] . This completes step m& 2 of the proof. Q.E.D. satisfying

17

A.2

Fatous Lemma

Fatous lemma states: For a sequence Xn ,n $ 1, of non-negative random variables, E[liminfn64Xn] # liminfn64E[Xn] . Proof: Put X ' liminfn64Xn and let n(x) be a simple function satisfying 0 # n(x) # x. Moreover, put Y n ' min(n(X) , Xn). Then Y n 6p n(X) because for arbitrary g > 0, P[|Y n & n(X)| > g] ' P[Xn < n(X)&g] # P[Xn < X&g] 6 0 . Since E[n(X)] < 4 because n is a simple function, and Y n # n(X) , it follows from Y n 6p n(X) and the dominated convergence theorem that E[n(X)] ' limn64E[Y n] ' liminfn64E[Y n] # liminfn64E[Xn] . (43) and the definition of E[X] that E[X] # liminfn64E[Xn] . Finally, note that in the case E[liminfn64Xn] = 4 Fatous lemma states that then also liminfn64E[Xn] = 4. Q.E.D. (43) Taking the supremum over all simple functions n satisfying 0 # n(x) # x it follows now from

18