FUNCTIONAL ANALYSIS LECTURE NOTES

CHAPTER 1. HILBERT SPACES

CHRISTOPHER HEIL

1. Elementary Properties of Hilbert Spaces

Notation 1.1. Throughout, F will denote either the real line R or the complex plane C.
All vector spaces are assumed to be over the field F.

Definition 1.2 (Semi-Inner Product, Inner Product). If X is a vector space over the field F,
then a semi-inner product on X is a function h·, ·i : X × X → F such that
(a) hx, xi ≥ 0 for all x ∈ X,
(b) hx, yi = hy, xi for all x, y ∈ X, and
(b) Linearity in the first variable: hαx + βy, zi = αhx, zi + βhy, zi for all x, y, z ∈ X and
α, β ∈ F.

Exercise 1.3. Immediate consequences are:

(a) Anti-linearity in the second variable: hx, αy + βzi = ᾱhx, yi + β̄hx, zi.
(b) hx, 0i = 0 = h0, yi.
(c) h0, 0i = 0.

Definition 1.4. If a semi-inner product h·, ·i satisfies:

hx, xi = 0 =⇒ x = 0,
then it is called an inner product, and X is called an inner product space or a pre-Hilbert
space.

Notation 1.5. There are many different standard notations for a semi-inner product, e.g.,
hx, yi = [x, y] = (x, y) = hx|yi,
to name a few. We shall prefer the notation hx, yi.

Date: February 20, 2006.

These notes closely follow and expand on the text by John B. Conway, “A Course in Functional Analysis,”
Second Edition, Springer, 1990.
1
2 CHRISTOPHER HEIL

Exercise 1.6. Prove the following.

(a) The dot product x · y = x1 ȳ1 + · · · + xn ȳn is an inner product on Cn .
(b) If w1 , . . . , wn ≥ 0 are fixed scalars, then the weighted dot product hx, yi = x1 ȳ1 w1 +
· · · + xn ȳn wn is a semi-inner product on Cn . It is an inner product if wi > 0 for each i.
(c) If A is an n × n positive semi-definite matrix (Ax · x ≥ 0 for all x ∈ Cn ), then
hx, yi = Ax · y is a semi-inner product on Cn , and it is an inner product if A is positive
definite (meaning Ax · x > 0 for all x 6= 0). The weighted dot product is just the special case
where A is diagonal.
(d) Show that if h·, ·i is an arbitrary inner product on Cn , then there exists a positive
definite matrix A such that hx, yi = Ax · y. Hint: Let e1 , . . . , en be the standard basis. Then
hx, yi = ȳ1 hx, e1 i + · · · + ȳn hx, en i. For each k, the mapping x 7→ hx, ek i is linear.

Exercise 1.7. Let I be a countable index set (e.g., I = N or Z). Let w : I → [0, ∞). Let
`2w = `2w (I) be the weighted `2 space defined by
n X o
2 2 2
`w = `w (I) = x = (xi )i∈I : |xi | w(i) < ∞ .
i∈I

Show that X
hx, yi = xi ȳi w(i)
i∈I

defines a semi-inner product on `2w . If w(i) > 0 for all i then it is an inner product.
If w(i) = 1 for all i, then we simply call this space `2 .
The series defining hx, yi converges because of the Cauchy–Schwarz inequality.

Example 1.8. Let (X, Ω, µ) be a measure space (X is a set, Ω is a σ-algebra of subsets of

of X, and µ : Ω → [0, ∞] is a countably additive measure). Define
n Z o
2
L (X) = f : X → F : |f (x)|2 dµ(x) < ∞ ,
X

where we identify functions that are equal almost everywhere, i.e.,

f = g a.e. ⇐⇒ µ{x ∈ X : f (x) 6= g(x)} = 0.
Then Z
hf, gi = f (x) g(x) dµ(x)
X
defines an inner product on L2 (X).
Other notations for L2 (X) are L2 (µ), L2 (X, µ), L2 (dµ), L2 (X, dµ), etc.
The space `2w (I) is a special case of L2 (X), where X = I and µ is a weighted counting
measure on I.
 CHAPTER 1. HILBERT SPACES 3

Exercise 1.9. Every subspace of an inner product space is itself an inner product space
(using the same inner product).
Hence every subspace of `2w or L2 (X) is also an inner product space. For example,
V = L2 (Rn ) ∩ C(Rn ) = {f ∈ L2 (Rn ) : f is continuous}
is a subspace of L2 (Rn ) and hence is also an inner product space. List some other subspaces
of some particular `2w or L2 (X) spaces.

Notation 1.10 (Associated Semi-Norm or Norm). If h·, ·i is a semi-inner product on X,

then we will write
kxk = hx, xi1/2 .
We will see later that k · k defines a semi-norm on X, and that it is a norm on X if h·, ·i is
an inner product on X.

Lemma 1.11 (Polar Identity). If h·, ·i is a semi-inner product on X, then for all x, y ∈ X
we have
kx + yk2 = kxk2 + 2 Re hx, yi + kyk2 .
Proof. Using the fact that z + z̄ = 2Re z, we have
kx + yk2 = hx + y, x + yi = kxk2 + hx, yi + hy, xi + kyk2
= kxk2 + 2 Re hx, yi + kyk2 . 

Theorem 1.12 (Cauchy-Bunyakowski-Schwarz Inequality). If h·, ·i is a semi-inner product

on X, then
∀ x, y ∈ X, |hx, yi| ≤ kxk kyk.
Moreover, equality holds if and only if there exist scalars α, β ∈ F, not both zero, such that
kαx + βyk = 0.
Proof. If x, y ∈ X and α ∈ F, then
0 ≤ kx − αyk2 = hx − αy, x − αyi
= hx, xi − αhy, xi − ᾱhx, yi + αᾱhy, yi
= kxk2 − αhy, xi − ᾱhx, yi + |α|2 kyk2 .
Write hy, xi = beiθ where b ≥ 0 and θ ∈ R (θ = 0 if F = R). Set α = e−iθ t where t ∈ R.
Then for this α we have
0 ≤ kxk2 − e−iθ tbeiθ − eiθ tbe−iθ + t2 kyk2 = kxk2 − 2bt + kyk2 t2 .
This is a quadratic polynomial in t. In order for this polynomial to be everywhere nonneg-
ative, it can have at most one real root, which means that the discriminant must be ≤ 0,
i.e.,
(−2b)2 − 4 kyk2 kxk2 ≤ 0.
4 CHRISTOPHER HEIL

Hence
|hx, yi|2 = |hy, xi|2 = b2 ≤ kxk2 kyk2 .
Exercise: Supply the proof for the case of equality. 

Corollary 1.13. If h·, ·i is a semi-inner product on X, then kxk = hx, xi1/2 defines a semi-
norm on X, which means that:
(a) kxk ≥ 0 for all x ∈ X,
(b) kαxk = |α| kxk for all x ∈ X and α ∈ F,
(c) Triangle Inequality: kx + yk ≤ kxk + kyk for all x, y ∈ X.

If h·, ·i is an inner product on X then k · k is a norm on X, which means that in addition

to (a)–(c) above, we also have:

(d) kxk = 0 =⇒ x = 0.
Proof. (a), (b), (d) Exercises.
(c) Using the Polar Identity, we have
kx + yk2 = hx + y, x + yi = kxk2 + 2 Re hx, yi + kyk2
≤ kxk2 + 2 |hx, yi| + kyk2
≤ kxk2 + 2 kxk kyk + kyk2

2
= kxk + kyk . 

Definition 1.14 (Distance). Let h·, ·i be an inner product on X. Then the distance from x
to y in X is
d(x, y) = kx − yk.

Exercise 1.15. d(·, ·) defines a metric on X.

Definition 1.16. Many of the results in this chapter are valid not only for inner product
spaces, but for any space which possesses a norm. Assume that X is a vector space. A
semi-norm on X is a function k · k : X → [0, ∞) such that statements (a)-(c) above hold.
If, in addition, statement (d) holds, then k · k is a norm, and X is called a normed space,
normed linear space, or normed vector space.

Definition 1.17 (Convergence). Let X be a normed linear space (such as an inner product
space), and let {fn }∞
n=1 be a sequence of elements of X.
 CHAPTER 1. HILBERT SPACES 5

(a) We say that {fn } converges to f ∈ X, and write fn → f , if

lim kf − fn k = 0,
n→∞

i.e.,
∀ ε > 0, ∃ N > 0 such that n > N =⇒ kf − fn k < ε.

(b) We say that {fn } is Cauchy if

∀ ε > 0, ∃ N > 0 such that m, n > N =⇒ kfm − fn k < ε.

Exercise 1.18. Let X be a normed linear space. Prove the following.

 
(a) Reverse Triangle Inequality: kxk − kyk ≤ kx − yk.
(b) Continuity of the norm: xn → x =⇒ kxn k → kxk.
(c) Continuity of the inner product: If X is an inner product space then
xn → x, yn → y =⇒ hxn , yn i → hx, yi.

(d) All convergent sequences are bounded, and the limit of a convergent sequence is unique.
(e) Cauchy sequences are bounded.
(f) Every convergent sequence is Cauchy.
(g) There exist inner product spaces for which not every Cauchy sequence is convergent.

Definition 1.19 (Hilbert Space). An inner product space H is called a Hilbert space if it is
complete, i.e., if every Cauchy sequence is convergent. That is,
{fn }∞
n=1 is Cauchy in H =⇒ ∃ f ∈ H such that fn → f.
The letter H will always denote a Hilbert space.

Definition 1.20 (Banach Space). A normed linear space X is called a Banach space if it is
complete, i.e., if every Cauchy sequence is convergent. We make no assumptions about the
meaning of the symbol X, i.e., it need not denote a Banach space. A Hilbert space is thus
a Banach space whose norm is associated with an inner product.

Example 1.21. Cn , `2w (w strictly positive), and L2 (X) are all Hilbert spaces (using the
default inner products).

Exercise 1.22. Prove that Cn and `2w are Hilbert spaces.

6 CHRISTOPHER HEIL

∞
Definition 1.23 (Convergent Series).
P∞ Let {fn }n=1 be a sequence of elements of a normed
linear space X. Then the series n=1 fn converges and equals f ∈ X if the partial sums
sN = N
P
n=1 fn converge to f . That is,
N
X
kf − sN k = f − fn → 0 as N → ∞.

n=1

P∞
Exercise 1.24. Let X be an inner product space, and suppose that the series n=1 fn
converges in X. Show that if g ∈ X, then
DX∞ E ∞
X
fn , g = hfn , gi.
n=1 n=1

Note that this is NOT an immediate consequence of the definition of inner product. You must
use both the fact that the inner product is linear in the first variable AND the continuity of
the inner product (Exercise 1.18) to prove this result.

Exercise 1.25 (Absolutely Convergent Series). LetP∞X be a Banach space and let P {f n }∞
n=1
∞
be a sequence of elements of X. Prove that if n=1 kfn k < ∞ then the series n=1 fn
converges in X. We say that such a series is absolutely convergent.
Hint: You must show that the sequence of partial sums {sN } converges. Since X is a
Banach space, you just have to show that this sequence is Cauchy.

Exercise 1.26 (Unconditionally Convergent Series). Let P∞X be a Banach space and let
∞
{fn }n=1 be a sequence of elements of X. The series f = n=1 fn is said to
P converge uncon-
ditionally if every rearrangement of the series converges. That is, f = ∞ n=1 fn converges
unconditionally if for each bijection σ : N → N the series
∞
X
fσ(n)
n=1

converges. It can be shown that in this case, the series will converge to f , i.e., f = ∞
P
n=1 fσ(n)
for every permutation σ. P
Prove that if a series f = ∞n=1 fn converges absolutely, then it converges unconditionally.
In finite dimensions the converse is true, but we will see later that the converse fails in infinite
dimensions (see Exercise 4.16).

Proposition 1.27. Let X P∞ be a Banach space and let {fn }∞

n=1 be a sequence of elements of
X. Then the series f = n=1 fn converges unconditionally if and only if it converges with
respect to the net of finite subsets of N, i.e., if
X
∀ ε > 0, ∃ finite F0 ⊆ N such that ∀ finite F ⊇ F0 , f − fn < ε.

n∈F
 CHAPTER 1. HILBERT SPACES 7

Definition 1.28 (Topology). Let X be a normed linear space.

(a) The open ball in X centered at x ∈ X with radius r > 0 is
B(x, r) = Br (x) = {y ∈ X : kx − yk < r}.

(b) A subset U ⊆ X is open if

∀ x ∈ U, ∃ r > 0 such that B(x, r) ⊆ U.

(c) A subset F ⊆ X is closed if X \ F is open.

Definition 1.29 (Limit Points, Closure, Density). Let X be a normed linear space and let
A ⊆ X.
(a) A point f ∈ A is called a limit point of A if there exist fn ∈ A with fn 6= f such that
fn → f .
(b) The closure of A is the smallest closed set Ā such that A ⊆ Ā. Specifically,
T
Ā = {F ⊆ X : F is closed and F ⊇ A}.

(c) We say that A is dense in X if Ā = X.

Exercise 1.30. (a) The closure of A equals the union of A and all limit points of A:
Ā = A ∪ {x ∈ X : x is a limit point of A} = {z ∈ X : ∃ yn ∈ A such that yn → z}.

(b) If X is a normed linear space, then the closure of an open ball B(x, r) is the closed
ball B(x, r) = {y ∈ X : kx − yk ≤ r}.
(c) Prove that A is dense if and only if
∀ x ∈ X, ∀ ε > 0, ∃ y ∈ A such that kx − yk < ε.

Example 1.31. The set of rationals Q is dense in the real line R.

Proposition 1.32. Let X be a normed linear space and let F ⊆ X. Then

F is closed ⇐⇒ F contains all its limit points.
Proof. ⇒. Suppose that F is closed but that there exists a limit point f that does not belong
to F . By definition, there must exist fn ∈ F such that fn → f . However, f ∈ X \ F , which
is open, so there exists some r > 0 such that B(f, r) ⊆ X \ F . Yet there must exist some f n
with kf − fn k < r, so this fn will belong to X \ F , which is a contradiction.
⇐. Exercise. 
8 CHRISTOPHER HEIL

Exercise 1.33. In finite dimensions, all subspaces are closed sets. This is not true in infinite
dimensions.
(a) Prove that
c00 = {x = (x1 , . . . , xN , 0, 0, . . . ) : N > 0, x1 , . . . , xN ∈ F}
is a subspace of `2 (N) that is not closed. Prove that c00 is dense in `2 (N).
(b) Prove that
c0 = {x = (xk )∞
k=1 : lim xk = 0}
k→∞
2
is a dense subspace of ` (N).
(c) Prove that Cc (R), the space of continuous, compactly supported functions on R, is a
subspace of L2 (R) that is dense and not closed. The support of a function f : R → C is
the closure in R of the set {x ∈ R : f (x) 6= 0}. Thus a function is compactly supported if
nonzero only within a bounded subset of R.
(d) Let E be a (Lebesgue) measurable subset of Rn . Let M = {f ∈ L2 (Rn ) : supp(f ) ⊆
E}. Prove that M is a closed subspace of L2 (Rn ).

Proposition 1.34. Let H be a Hilbert space and let M be a subspace of H. Then M is

itself a Hilbert space (using the inner product from H) if and only if M is closed.
Let X be a Banach space and let M be a subspace of X. Then M is itself a Banach space
(using the norm from X) if and only if M is closed.
Proof. ⇒. Suppose that M is a Hilbert space. Let f be any limit point of M , i.e., suppose
fn ∈ M and fn → f . Then {fn } is a convergent sequence in H, and hence is Cauchy in
H. Since each fn belongs to M , it is also Cauchy in M . Since M is a Hilbert space, {fn }
must therefore converge in M , i.e., fn → g for some g ∈ M . However, limits are unique, so
f = g ∈ M . Therefore M contains all its limit points and hence is closed.
⇐. Exercise. 

Exercise 1.35. Find examples of inner product spaces that are not Hilbert spaces.

Remark 1.36. In constructing the examples for the preceding exercise, you will probably
look for examples of inner product spaces X which are subspaces of a larger Hilbert space
H. Is every inner product space a subspace of a larger Hilbert space? The answer is yes, it
is also possible to construct a Hilbert space H ⊇ X, called the completion of X.

Theorem 1.37 (Parallelogram Law). If X is an inner product space, then

∀ f, g ∈ X, kf + gk2 + kf − gk2 = 2 kf k2 + kgk2 .

Proof. Exercise. 
 CHAPTER 1. HILBERT SPACES 9

Exercise 1.38. Show that `p for p 6= 2 and Lp (R) for p 6= 2 are not inner product spaces
under the default norms (show that the Parallelogram Law fails).

Exercise 1.39. Suppose that X is a Banach space over the complex field C, and the norm
k · k of X satisfies the Parallelogram Law. Prove that
1
kf + gk2 − kf − gk2 + ikf + igk2 − ikf − igk2


hf, gi =
4
is an inner product on X, and that kf k2 = hf, f i.

2. Orthogonality
Definition 2.1. Let H be a Hilbert space.
(a) Two vectors f , g ∈ H are orthogonal (written f ⊥ g) if hf, gi = 0.
(b) A sequence of vectors {fi }i∈I is an orthogonal sequence if hfi , fj i = 0 whenever i 6= j.
(c) A sequence of vectors {fi }i∈I is an orthonormal sequence if it is orthogonal and each
vector is a unit vector, i.e.,
(
1, i = j,
hfi , fj i = δij =
0, i 6= j.

Exercise 2.2. (a) If {fi }i∈I is an orthogonal sequence of nonzero vectors, then it is finitely
linearly independent, i.e., every finite subset is linearly independent.
(b) Let I = N. Define en = (δmn )∞m=1 , i.e., en is the sequence having a 1 in the nth
component and zeros elsewhere. Show that {en }∞ 2
n=1 is an orthonormal sequence in ` (called
the standard basis of `2 ).
(c) Let X = [0, 1] and let µ be Lebesgue measure. Define en (x) = e2πinx . Show that
{en }n∈Z is an orthonormal sequence in L2 [0, 1].

Theorem 2.3 (Pythagorean Theorem). If f1 , . . . , fn ∈ H are orthogonal, then

X n 2 X n
fk = kfk k2 .


k=1 k=1

Proof. Exercise. 

Definition 2.4 (Convex Set). Let X is a vector space and K ⊆ X, then K is convex if
x, y ∈ K, 0 ≤ t ≤ 1 =⇒ tx + (1 − t)y ∈ K.
Thus, the entire line segment between x and y is contained in K (including the midpoint
1
2
x + 21 y in particular).
10 CHRISTOPHER HEIL

Exercise 2.5. (a) Every subspace of a vector space X is convex.

(b) If X is a normed linear space, then open and closed balls in X are convex.

Definition 2.6. Let X be a normed linear space and let A ⊆ H. The distance from a point
x ∈ H to the set A is
dist(x, A) = inf{kx − yk : y ∈ A}.

Theorem 2.7 (Closest Point Property). If H is a Hilbert space and K is a nonempty closed,
convex subset of H, then given any h ∈ H there exists a unique point k0 ∈ K that is closest
to h. That is, there is a unique point k0 ∈ K such that
kh − k0 k = dist(h, K) = inf{kh − kk : k ∈ K}.
Proof. Let
d = dist(h, K) = inf{kh − kk : k ∈ K}.
By definition, there exist kn ∈ K such that kh − kn k → d, and furthermore we have d ≤
kh − kn k for each n. Therefore, if we fix any ε > 0 then we can find an N such that
n>N =⇒ d2 ≤ kh − kn k2 ≤ d2 + ε2 .
By the Parallelogram Law,
k(h − kn ) − (h − km )k2 + k(h − kn ) + (h − km )k2 = 2 kh − kn k2 + kh − km k2 .

Hence,
k − k 2
m n 1 2 kh − kn k2 kh − km k2 km + k n
2
= k(h − k ) − (h − k )k = + − h − .

n m
2 4 2 2 2

However, km +k2
n
∈ K since K is convex, so kh − km +k 2
n
k ≥ d and therefore
km + k n
2
− h − ≤ −d2 .

2
Also, if m, n > N then
kh − kn k2 , kh − km k2 ≤ d2 + ε2 .
Therefore,
k − k 2
m n d2 + ε 2 d2 + ε 2
≤ + − d2 = ε2 .
2 2 2

So, kkm − kn k ≤ 2ε for all m, n > N , which says that the sequence {kn } is Cauchy. Since
H is complete, this sequence must converge, i.e., kn → k0 for some k0 ∈ H. But kn ∈ K for
all n and K is closed, so we must have k0 ∈ K.
Since h − kn → h − k0 , we have
kh − k0 k = lim kh − kn k = d,
n→∞

and thus kh − k0 k ≤ kh − kk for every k ∈ K.

Exercise: Prove that k0 is the unique point closest to h by assuming that there exists
another point k00 that satisfies kh−k00 k ≤ kh−kk for every k ∈ K, and derive a contradiction.
k +k 0
Draw a picture and think about the midpoint 0 2 0 . 
 CHAPTER 1. HILBERT SPACES 11

Notation 2.8 (Notation for Closed Subspaces). Since we will often deal with closed sub-
spaces of a Hilbert space, we declare that the notation
M ≤ H
means that M is a closed subspace of the Hilbert space H. The letter H will always denote
a Hilbert space.

Theorem 2.9. Let M ≤ H, and fix h ∈ H. Then the following statements are equivalent.
(a) h = p + e where p is the (unique) point in M closest to h.
(b) h = p + e where p ∈ M and e ⊥ M (i.e., e ⊥ f for every f ∈ M ).
Proof. (a) ⇒ (b). Let p be the (unique) point in M closest to h, and let e = p − h. Choose
any f ∈ M . We must show that hf, ei = 0.
Since M is a subspace, p + λf ∈ M for any scalar λ ∈ F. Hence,
kh − pk2 ≤ kh − (p + λf )k2 = k(h − p) − λf k2
= kh − pk2 − 2 Re hλf, h − pi + |λ|2 kf k2
= kh − pk2 − 2 Re λhf, ei + |λ|2 kf k2 .
Therefore,
∀ λ ∈ F, 2 Re λhf, ei ≤ |λ|2 kf k2 .
If we consider λ = t > 0, then we can divide through by t to get
∀ t > 0, 2 Re hf, ei ≤ t kf k2 .
Letting t → 0+ , we conclude that Re hf, ei ≤ 0. Similarly, taking λ = t < 0 and letting
t → 0− , we obtain Re hf, ei ≥ 0.
If F = R then we are done. If F = C, then we take λ = it with t > 0 and λ = it with
t < 0 to show that Im hf, ei = 0 as well.
(b) ⇒ (a). Suppose that h = p + e where p ∈ M and e ⊥ M . Choose any f ∈ M . Then
p − f ∈ M , so h − p = e ⊥ p − f . Therefore, by the Pythagorean Theorem,
kh − f k2 = k(h − p) + (p − f )k2 = kh − pk2 + kp − f k2 ≥ kh − pk2 .
Hence p is the point in M that is closest to h. 

Definition 2.10 (Orthogonal Projection). Let M ≤ H.

(a) For each h ∈ H, the point p ∈ M closest to H is called the orthogonal projection of
h onto M .
(b) Define P : H → H by P h = p, the orthogonal projection of h onto M . Then the
operator P is called the orthogonal projection of H onto M . Note that by definition
and Theorem 2.9, h = P h + e where e = h − P h ⊥ M .
12 CHRISTOPHER HEIL

Definition 2.11 (Orthogonal Complement). Let A be a subset (not necessarily a subspace)

of a Hilbert space H. The orthogonal complement of A is
A⊥ = {x ∈ H : x ⊥ A} = {x ∈ H : hx, yi = 0 ∀ y ∈ A}.

Exercise 2.12. (a) If A ⊆ H, then A⊥ is a closed subspace of H (even if A itself is not).

(b) {0}⊥ = H and H ⊥ = {0}.

Definition 2.13 (Notation for Operators). Let X, Y be normed linear spaces. Let T : X →
Y be a function (= operator = transformation). We write either T (x) or T x to denote the
image of an element x ∈ X.
(a) T is linear if T (αx + βy) = αT (x) + βT (y) for every x, y ∈ X and α, β ∈ F.
(b) T is injective if T (x) = T (y) implies x = y.
(c) The kernel or nullspace of T is ker(T ) = {x ∈ X : T (x) = 0}.
(c) The range of T is range(T ) = {T (x) : x ∈ X}.
(d) The rank of T is the dimension of its range, i.e., rank(T ) = dim(range(T )). In
particular, T is finite-rank if range(T ) is finite-dimensional.
(d) T is surjective if range(T ) = Y .
(e) T is a bijection if it is both injective and surjective.
(f) We use the notation I or IH to denote the identity map of a Hilbert space H onto
itself.

Exercise 2.14. Show that if T : X → Y is linear and continuous, then ker(T ) is a closed
subspace of X and that range(T ) is a subspace of Y . Must range(T ) be a closed subspace?

Theorem 2.15 (Properties of Orthogonal Projections). Let M ≤ H, and let P be the

orthogonal projection of H onto M . Then the following statements hold.
(a) h − P h ⊥ M for every h ∈ H.
(b) h − P h ∈ M ⊥ for every h ∈ H.
(c) kh − P hk = dist(h, M ) for every h ∈ H.
(d) P is a linear transformation of H into itself.
(e) kP hk ≤ khk for every h ∈ H.
(f) P is idempotent, i.e., P 2 = P .
(g) ker(P ) = M ⊥ .
(h) range(P ) = M .
(i) I − P is the orthogonal projection of H onto M ⊥ .
 CHAPTER 1. HILBERT SPACES 13

Proof. (a) Follows from the definition of orthogonal projection and Theorem 2.9.
(b) Follows from (a) and the definition of M ⊥ .
(e) Choose any h ∈ H. Then h = P h + e with P h ∈ M and e ∈ M ⊥ . Hence P h ⊥ e, so
by the Pythagorean Theorem, khk2 = kP hk2 + kek2 ≥ kP hk2 .
(g) Suppose that h ∈ ker(P ), i.e., P h = 0. Then h = h − P h ⊥ M , so h ∈ M ⊥ .
Conversely, suppose that h ∈ M ⊥ . Then h = 0 + h with 0 ∈ M and h ∈ M ⊥ , so we must
have P h = 0 and hence h ∈ ker(P ).
The remaining parts are left as exercises. 

Corollary 2.16 (Double Complements). If M ≤ H, then (M ⊥ )⊥ = M .

Proof. Choose any x ∈ M . Then hx, yi = 0 for every y ∈ M ⊥ . Thus x ⊥ M ⊥ , so x ∈ (M ⊥ )⊥ .
Conversely, suppose that x ∈ (M ⊥ )⊥ . Since M is closed, we can write x = p + e with
p ∈ M and e ∈ M ⊥ . Since x ∈ (M ⊥ )⊥ we have hx, ei = 0. But we also have p ∈ M ⊆ (M ⊥ )⊥ ,
so he, pi = 0. Therefore kek2 = he, ei = hx − p, ei = hx, ei − hp, ei = 0. Hence e = 0, so
x = p ∈ M. 

Definition 2.17 (Span, Closed Span). Let A be a subset of a normed linear space X.
(a) The finite span (or linear span or just the span) of A, denoted span(A), is the set of
all finite linear combinations of elements of A:
X n 
span(A) = αk xk : n > 0, xk ∈ A, αk ∈ F .
k=1

In particular, if A is a countable sequence, say A = {xk }∞k=1 , then

Xn 
∞
span({xk }k=1 ) = αk xk : n > 0, αk ∈ F .
k=1

(b) The closed finite span (or closed linear span or closed span) of A, denoted span(A)
or ∨A, is the closure of the set of all finite linear combinations of elements of A:
∨A = span(A) = span(A) = {z ∈ X : ∃ yn ∈ span(A) such that yn → z}.

Beware: This does NOT say that

X ∞ 
span(A) = αk xk : xk ∈ A, αk ∈ F .
k=1

It does not even imply that every element of span(A) can be written x = ∞
P
k=1 αk xk
for some xk ∈ A, αk ∈ F. What does it say about span(A)? If we consider the case
of a countable sequence A = {xk }∞k=1 , then we have
n Xn o
∞
span({xk }k=1 ) = z ∈ X : ∃ αk,n ∈ F such that αk,n xk → z as n → ∞ .
k=1
14 CHRISTOPHER HEIL

That is, an element z lies in the closed span if there exist αk,n ∈ F such that
n
X
αk,n xk → z as n → ∞.
k=1
P∞
In contrast, to say that x = k=1 αk xk means that
n
X
αk xk → x as n → ∞,
k=1

i.e., the scalars αk must be independent of n.

(c) If X is a Banach space then we say that a subset A ⊆ X is complete if span(A) is

dense in X, or equivalently, if span(A) = X.

Beware of the two different meanings that we have assigned to the word complete!

Exercise 2.18. Let {en }∞ 2 ∞

n=1 be the standard basis for ` (N). Prove that c00 = span({en }n=1 )
∞ 2
and conclude that {en }n=1 is complete in ` (N).

Exercise 2.19. Let A be any subset of a Hilbert space H.

(a) (A⊥ )⊥ = span(A).
(b) A is complete if and only if A⊥ = {0}.

Note that, by definition, A⊥ = {0} is equivalent to the statement that x ⊥ A =⇒ x = 0.

Thus, this exercise says that A is complete if and only if the only element orthogonal to every
element of A is 0.

3. The Riesz Representation Theorem

Definition 3.1 (Continuous and Bounded Operators). Let X, Y be normed linear spaces,
and let L : X → Y be a linear operator.
(a) L is continuous at a point x ∈ X if xn → x in X implies Lxn → Lx in Y .
(b) L is continuous if it is continuous at every point, i.e., if xn → x in X implies
Lxn → Lx in Y for every x.
(c) L is bounded if there exists a finite K ≥ 0 such that
∀ x ∈ X, kLxk ≤ K kxk.
Note that kLxk is the norm of Lx in Y , while kxk is the norm of x in X.
(d) The operator norm of L is
kLk = sup kLxk.
kxk=1
 CHAPTER 1. HILBERT SPACES 15

(e) We let B(X, Y ) denote the set of all bounded linear operators mapping X into Y ,
i.e.,
B(X, Y ) = {L : X → Y : L is bounded and linear}.
If X = Y = X then we write B(X) = B(X, X).
(f) If Y = F then we say that L is a functional. The set of all bounded linear functionals
on X is the dual space of X, and is denoted
X 0 = B(X, F) = {L : X → F : L is bounded and linear}.

Exercise 3.2. Let X, X, Y be normed linear spaces. Let L : X → Y be a linear operator.

(a) L is injective if and only if ker L = {0}.

(b) L is bounded if and only if kLk < ∞.
(c) If L is bounded then kLxk ≤ kLk kxk for every x ∈ X (note that three different
meanings of the symbol k · k appear in this statement!).
(d) If L is bounded then kLk is the smallest value of K such that kLxk ≤ Kkxk holds
for all x ∈ X.
kLxk
(e) kLk = sup kLxk = sup .
kxk≤1 x6=0 kxk

(f) B(X, Y ) is a subspace of the vector space V containing ALL functions A : X → Y .

Moreover, the operator norm is a norm on the space B(X, Y ), i.e.,
i. kLk ≥ 0 for all L ∈ B(X, Y ).
ii. kLk = 0 if and only if L = 0 (the zero operator that sends every element of X
to the zero vector in Y ).
iii. kαLk = |α| kLk for every L ∈ B(X, Y ) and every α ∈ F.
iv. kL + Kk ≤ kLk + kKk for every L, K ∈ B(X, Y ).
(g) Part (f) shows that B(X, Y ) is itself a normed linear space, and we will see later that
it is a Banach space if Y is a Banach space. However, in addition to vector addition
and scalar multiplication operations, there is a third operation that we can perform
with functions: composition. Prove that the operator norm is submultiplicative, i.e.,
if A ∈ B(X, Y ) and B ∈ B(Y, Z), then BA ∈ B(X, Z), and furthermore,
kBAk ≤ kBk kAk. (3.1)
In particular, when X = Y = Z, we see that B(X) is closed under composition. The
space B(X) is an example of an algebra.

(h) If L ∈ X 0 = B(X, F) then kLk = sup |Lx|.

kxk=1

Exercise 3.3. Let M ≤ H, and let P be the operator of orthogonal projection onto M .
Find kP k.
16 CHRISTOPHER HEIL

Exercise 3.4. Let X, Y be normed linear spaces. Prove that if L : X → Y is linear and
X is finite-dimensional (take Y = Cn or Rn if you like), then L is bounded. Hint: If
x = (x1 , . . . , xn ) ∈ Cn then x = x1 e1 + · · · + xn en where {e1 , . . . , en } is the standard basis
for Cn . Use the Triangle Inequality. If X is any finite-dimensional vector space, do the same
but with an arbitrary basis for X, and use the fact (that we will prove later) that all norms
on a finite-dimensional space are equivalent.

The next lemma is a standard fact about continuous functions. L−1 (U ) denotes the inverse
image of U ⊆ Y , i.e., L−1 (U ) = {x ∈ X : Lx ∈ U }.

Lemma 3.5. Let X, Y be normed linear spaces. Let L : X → Y be linear. Then

 
−1
L is continuous ⇐⇒ U open in Y =⇒ L (U ) open in X .
Proof. ⇒. Suppose that L is continuous and that U is an open subset of Y . We will show
that X \ L−1 (U ) is closed by showing that it contains all its limit points.
Suppose that x is a limit point of X \ L−1 (U ). Then there exist xn ∈ X \ L−1 (U ) such that
xn → x. Since L is continuous, this implies Lxn → Lx. However, xn ∈ / L−1 (U ), so Lxn ∈/ U,
i.e., Lxn ∈ Y \ U , which is a closed set. Therefore Lx ∈ Y \ U , and hence x ∈ X \ L−1 (U ).
Thus X \ L−1 (U ) is closed, so L−1 (U ) is open.
⇐. Exercise. 

Theorem 3.6 (Equivalence of Bounded and Continuous Linear Operators). Let X, Y be

normed linear spaces, and let L : X → Y be a linear mapping. Then the following statements
are equivalent.
(a) L is continuous at some x ∈ X.
(b) L is continuous at x = 0.
(c) L is continuous.
(d) L is bounded.
Proof. (c) ⇒ (d). Suppose that L is continuous but unbounded. Then kLk = ∞, so
there must exist xn ∈ X with kxn k = 1 such that kLxn k ≥ n. Set yn = xn /n. Then
kyn − 0k = kyn k = kxn k/n → 0, so yn → 0. Since L is continuous and linear, this implies
Lyn → L0 = 0, and therefore kLyn k → k0k = 0. But
1 1
kLyn k = kLxn k ≥ ·n = 1
n n
for all n, which is a contradiction. Hence L must be bounded.
(d) ⇒ (c). Suppose that L is bounded, so kLk < ∞. Suppose that x ∈ X and that
xn → x. Then kxn − xk → 0, so
kLxn − Lxk = kL(xn − x)k ≤ kLk kxn − xk → 0,
i.e., Lxn → Lx. Thus L is continuous.
The remaining implications are left as exercises. 
 CHAPTER 1. HILBERT SPACES 17

Exercise 3.7. Let X, Y be normed linear spaces. Suppose that L : X → Y is linear and
satisfies kLxk = kxk for all x ∈ X. Such an operator is said to be an isometry or norm-
preserving. Prove that L is injective and find kLk. Find an example of an isometry that is
not surjective. Contrast this with the fact that if A : Cn → Cn is linear, then A is injective
if and only if it is surjective.

Exercise 3.8. (a) Define L : `2 (N) → `2 (N) by L(x) = (x2 , x3 , . . . ) for x = (x1 , x2 , . . . ) ∈
`2 (N). Prove that this left-shift operator is bounded, linear, surjective, not injective, and is
not an isometry. Find kLk.
(b) Define R : `2 (N) → `2 (N) by R(x) = (0, x1 , x2 , x3 , . . . ) for x = (x1 , x2 , . . . ) ∈ `2 (N).
Prove that this right-shift operator is bounded, linear, injective, not surjective, and is an
isometry. Find kRk.
(c) Compute LR and RL. Contrast this computation with the fact that in finite dimen-
sions, if A, B : Cn → Cn are linear maps (hence correspond to multiplication by n × n
matrices), then AB = I implies BA = I and conversely.

Exercise 3.9. Let X be a Banach space and Y a normed linear space. Suppose that
L : X → Y is bounded and linear. Prove that if there exists c > 0 such that kLxk ≥ ckxk
for all x ∈ X, then L is injective and range(L) is closed.

Exercise 3.10. For each h ∈ H, define a linear functional Lh : H → F by

Lh (x) = hx, hi, x ∈ H.
Prove the following.
(a) Lh is linear.
(b) ker(Lh ) = {h}⊥ .
(c) kLh k = khk, so Lh is a bounded linear functional on H. Therefore Lh ∈ H 0 =
B(H, F).

Thus, each element h of H determines an element Lh of H 0 . In the Riesz Representation

Theorem we will prove that these are all the elements of H 0 .

Example 3.11 (Dual Space of Cn ). Consider H = Cn (with F = C). Let h = (h1 , . . . , hn )T ∈

H = Cn (think of h as a column vector). Then Lh (x) = x · h = x1 h̄1 + · · · + xn h̄n defines a
bounded linear functional on Cn .
Suppose that L : Cn → C is any linear functional on Cn (it will necessarily be bounded
since Cn is finite-dimensional, see Exercise 3.4). Any linear mapping from Cn to Cm is given
by multiplicationby an m × n matrix.
 So in this case there must be a 1 × m matrix (i.e., a
row vector) A = a1 · · · an such that
x1
 

Lx = Ax = a1 · · · an  ...  = x1 a1 + · · · + xn an = x · h = Lh (x)
 

xn
18 CHRISTOPHER HEIL

where h = (ā1 , . . . , ān )T .

Thus we see that every element of (Cn )0 is an Lh for some h ∈ Cn .

Exercise 3.12. Define T : Cn → (Cn )0 by

T (h) = Lh , h ∈ Cn .
Prove directly that T is anti-linear, i.e.,
T (αh + βk) = ᾱT (h) + β̄T (k),
and that T is an isometry, i.e., kT (h)k = khk for each h. All isometries are injective, and
in the preceding example we showed that T is surjective, so T is an anti-linear isometric
bijection of Cn onto (Cn )0 .
If we define S : Cn → (Cn )0 by S(h) = Lh̄ , then S a linear isometric bijection of Cn onto
(Cn )0 . Such an S is called an isomorphism, and thus Cn and (Cn )0 are isomorphic.

Theorem 3.13 (Riesz Representation Theorem). For each h ∈ H, define Lh : H → F by

Lh (x) = hx, hi, x ∈ H.

(a) For each h ∈ H, we have Lh ∈ H 0 and kLh k = khk.

(b) If L ∈ H 0 , then there exists a unique h ∈ H such that L = Lh .
(c) The mapping T : H → H 0 defined by T (h) = Lh is an anti-linear isometric bijection
of H onto H 0 .
Proof. (a) See Exercise 3.10.
(b) Choose any L ∈ H 0 . If L = 0 then h = 0 is the required vector, so assume that L 6= 0
(i.e., L is not the zero operator). Then L does not map every vector to zero, so the kernel
of L is a closed subspace of H that is not all of H.
Choose any z ∈ / ker(L), and write
z = p + e, p ∈ ker(L), e ∈ ker(L)⊥ .
Note that L(p) = 0 by definition, and therefore we must have L(e) 6= 0 (since L(z) 6= 0).
Set u = e/L(e), and note that u ∈ ker(L)⊥ and L(u) = 1.
Given x ∈ H, since L is linear and L(u) = 1 we have


L x − L(x) u = L(x) − L(x) L(u) = 0.
Therefore x − L(x) u ∈ ker(L). However, u ⊥ ker(L), so
0 = hx − L(x) u, ui = hx, ui − hL(x) u, ui = hx, ui − L(x) kuk2 .
Hence,
1 D u E
L(x) = hx, ui = x, = hx, hi = Lh (x)
kuk2 kuk2
where
u
h = .
kuk2
 CHAPTER 1. HILBERT SPACES 19

Thus L = Lh , and from part (a), we have that kLk = kLh k = khk.
It remains only to show that h is unique. Suppose that we also had L = Lh0 . Then for
every x ∈ H we have
hx, h − h0 i = hx, hi − hx, h0 i = Lh (x) − Lh0 (x) = L(x) − L(x) = 0.
Consequently, h − h0 = 0.
(c) Parts (a) and (b) show that T is surjective and that T is norm-preserving. Therefore,
we just have to show that T is anti-linear.
Let h ∈ H and let c ∈ F. We must show that T (ch) = c̄T (h), i.e., that Lch = c̄Lh . This
follows immediately from the fact that for each x ∈ H, we have
Lch (x) = hx, chi = c̄ hx, hi = c̄ Lh (x).
The proof that Lh+k = T (h + k) = T (h) + T (k) = Lh + Lk is left as an exercise. 

Corollary 3.14. If L : `2 (I) → F is a bounded linear functional, then there exists a unique
h = (hi )i∈I ∈ `2 (I) such that
X
L(x) = xi h̄i , x = (xk )i∈I ∈ `2 (I).
i∈I

Corollary 3.15. If (X, Ω, µ) is a measure space and L : L2 (X) → F is a bounded linear

functional, then there exists a unique h ∈ L2 (X) such that
Z
L(f ) = f (x) h(x) dµ(x), f ∈ L2 (X).
X

4. Orthonormal Sets of Vectors and Bases

Definition 4.1 (Hamel Basis). Let V be a vector space. We say that {fi }i∈I is a basis for
V if it is both finitely linearly independent and its finite span is all of V . That is, every
vector in V equals a unique finite linear combination of the fi (aside from trivial zero terms),
i.e., every f 6= 0 can be written f = N
P
k=1 ck fik for a unique choice of indices i1 , . . . , iN and
nonzero scalars c1 , . . . , cN . Because the word “basis” is heavily overused, we shall refer to
such a basis as a Hamel basis or a vector space basis. For most vector spaces, Hamel bases
are only known to exist because of the Axiom of Choice; in fact, the statement “Every vector
space has a Hamel basis” is equivalent to the Axiom of Choice.
As in finite dimensions, it can be shown that any two Hamel bases for a vector space V
must have the same cardinality. This cardinality is called the (vector space) dimension of V .

Example 4.2 (Existence of Unbounded Linear Functionals). We can use the Axiom of
Choice to show that there exist unbounded linear functionals L : X → F whenever X is an
infinite-dimensional normed linear space.
Let {fi }i∈I be a Hamel basis for an infinite-dimensional normed linear space X, normalized
so that kfi k = 1 for every i ∈ I. Let J0 = {j1 , j2 , . . . } be any countable subsequence of I.
20 CHRISTOPHER HEIL

Define L : X → C by setting L(fjn ) = n for n ∈ N and L(fi ) = 0 for i ∈ I\J0 . Then extend
L linearly to all of X: if f = N
P
k=1 c k fik is the unique representation of f using nonzero
scalars c1 , . . . , cN , then define L(f ) = N
P
k=1 ck L(fik ). This L is a linear functional on X, but
since kfjn k = 1 yet |L(fjn )| = n, we have kLk = supkf k=1 |L(f )| = ∞.
Thus, if L : X → Y and Y is finite-dimensional, we cannot conclude that L must necessar-
ily be bounded. Contrast this with the fact that if L : X → Y and X is finite-dimensional,
then L must be bounded (see Exercise 3.4).

Remark 4.3. Aside from the preceding result, Hamel bases are not going to be much use
to us. For example, consider the space `2 = `2 (N), and let {en }n∈N be the standard basis
for `2 . Note that {en }n∈N is NOT a Hamel basis for `2 ! Its finite linear span is the proper
subspace c00 . Thus {en }n∈N is a Hamel basis for c00 , but not for `2 . In fact, a Hamel basis
for an infinite-dimensional Hilbert or Banach space must be uncountable.
The main point is that since we have a norm, there is no reason to restrict to only finite
linear combinations. Given a sequence of vectors {fi }i∈N and scalars (ci )i∈N , we can consider
“infinite linear combinations”
X∞
ci f i .
i=1
HOWEVER, we must be very careful about convergence issues: the series above will not
converge for EVERY choice of scalars ci . On the other hand, if our sequence {fi } is orthog-
onal, or even better yet, orthonormal, then we will see that the convergence issues become
easy to deal with, and we can make sense of what it means to have a “basis” which allows
“infinite linear combinations.”
HOWEVER, there still remains an issue of how many vectors we need in this “basis”—for
some Hilbert spaces (the separable Hilbert spaces) we will be able to use a basis that requires
only countably many vectors, as was implicitly assumed in the discussion in the preceding
paragraph. But for others, a “basis” may need uncountably many vectors, even allowing
“infinite (but still countable) linear combinations.” In some fields (e.g., the geometry of
Banach spaces literature), it is customary to only use the word basis for the case of countable
bases. Conway does not follow this custom, which is fine as long as you realize that other
authors use the term “basis” differently.
Conway’s definition of a basis for a Hilbert space follows. The definition does not seem to
say anything about linear combinations, infinite or otherwise—we shall see the connection
later. To emphasize that Conway’s definition is not universally accepted, I will refer to his
definition of basis as a Conway basis.

Definition 4.4 (Conway Basis). Let H be a Hilbert space. Then a Conway basis for H is
a maximal orthonormal set. That is, a subset E ⊆ H is a Conway basis if
(a) E is orthonormal, and
(b) there does not exist an orthonormal set E 0 ⊆ H such that E ⊆ E 0 .
 CHAPTER 1. HILBERT SPACES 21

Exercise 4.5. Zorn’s Lemma is an equivalent formulation of the Axiom of Choice. Learn
what Zorn’s Lemma says, and use it to prove that every Hilbert space has a Conway basis.
Then ask all your friends the classic riddle: What is yellow and equivalent to the Axiom of
Choice?1 That one is nearly as good as the classic: What is purple and commutes?2

Exercise 4.6. Show that {en }n∈N is a Conway basis for `2 (N).

Exercise 4.7. Let I be any set. Define `2 (I) to be the set of all functions x : I → F such
that x(i) 6= 0 for at most a countable number of i and such that
X
kxk2 = |x(i)|2 < ∞
i∈I

(there are at most countably many nonzero terms in the above sum, so this series with a
possibly uncountable index set just means to sum the countably many nonzero terms). Then
`2 (I) is a Hilbert space with respect to the inner product
X
hx, yi = x(i) y(i).
i∈I

For each i ∈ I, define ei : I → F by

(
1, i = j,
ei (j) = δij =
0, i =
6 j.
Then {ei }i∈I is called the standard basis for `2 (I). Prove that it is a Conway basis for `2 (I).

Instead of “Conway basis,” the following gives us a terminology that does not require us
to use the word basis.

Lemma 4.8. Let {fi }i∈I be a sequence of elements of H. Then:

{fi }i∈I is orthonormal and complete ⇐⇒ {fi }i∈I is a Conway basis.
Proof. ⇒. Suppose that {fi }i∈I is orthonormal and complete. Then by Exercise 2.19, there
is no nonzero g ∈ H that is orthogonal to every fi . Therefore there can be no orthonormal
set that properly contains {fi }i∈I .
⇐. Exercise. 

Definition 4.9 (Schauder basis). Now we give the “correct” definition of a basis.
Let X be a Banach space. Then a (countable!) sequence {fi }i∈N is a Schauder basis, or
just a basis, for X if for each f ∈ X there exist unique scalars {ci }i∈N such that
X∞
f = ci f i .
i=1

1Zorn’s Lemon!
2An abelian grape.
22 CHRISTOPHER HEIL

Note in particular that this means that N

P
i=1 ci fi → f as N → ∞.
To be a little more precise, the definition of Schauder basisP really includes the requirement
that if we let ci (f ) denote the unique scalars such that f = ∞ i=1 ci (f )fi , then the mapping
f 7→ ci (f ) must be a continuous linear functional on X for each i. However, the Strong
Basis Theorem proves that this is automatically true when X is a Banach space, so we have
omitted that requirement in our definition.

Exercise 4.10. The standard basis {en }n∈N is an orthonormal Schauder basis for `2 (N).

Exercise 4.11. (See also Exercise 10 on p. 98 of Conway.) Let {fi }i∈N be a Schauder basis
for a Banach space X.
(a) Prove that {fi }i∈N is complete in X, i.e., that the set of finite linear combinations is
dense in X. Note: The converse is false, i.e., a complete sequence need not be a Schauder
basis (see Example 4.18). On the other hand, we will see that in a separable Hilbert space,
a complete orthonormal sequence IS a Schauder basis.
(b) Let
N
X 
S = ri fi : N > 0, Re (ri ), Im (ri ) ∈ Q .
i=1
Prove that S is a countable, dense subset of X. A Banach space which has a countable,
dense subset is said to be separable.

Remark 4.12. Thus, if a Banach space has a Schauder basis, then it must be separable.
Does every separable Banach space have a Schauder basis? This was a longstanding open
problem in Banach space theory, called the Basis Problem. It was settled by Enflo in 1973:
there exist separable, reflexive Banach spaces which do not possess any Schauder bases.

Exercise 4.13. Let X be a Banach space. Suppose there exists an uncountable sequence
{xi }i∈I of elements of X such that kxi − xj k ≥ ε > 0 for all i 6= j ∈ I. Prove that X is not
separable.
Exercise 4.14. Use the preceding exercise to prove the following.
(a) Prove that `∞ (N) is not separable.
(b) Prove that L∞ (R) is not separable.
(c) Prove that if I is uncountable then `2 (I) is not separable. More generally, any Hilbert
space which contains an uncountable orthonormal sequence is not separable.

We will show that, in a separable Hilbert space, any complete orthonormal sequence is a
Schauder basis for H. If H is nonseparable, something very similar happens; we just have to
be careful with what we mean by infinite series with uncountably many terms (only countably
many terms can be nonzero). First, however, we will present the results for separable Hilbert
spaces.
 CHAPTER 1. HILBERT SPACES 23

Theorem 4.15. Let {en }n∈N be any orthonormal sequence in a Hilbert space H. Then the
following statements hold.
∞
X
(a) Bessel’s Inequality: |hf, en i|2 ≤ kf k2 .
n=1
∞
X
(b) If f = cn en converges, then cn = hf, en i.
n=1
∞
X ∞
X
(c) cn en converges ⇐⇒ |cn |2 < ∞.
n=1 n=1

∞
X
(d) If cn en converges then it converges unconditionally, i.e., it converges regardless of
n=1
the ordering of the indices.
∞
X
(e) f ∈ span({en }n∈N ) ⇐⇒ f = hf, en i en .
n=1

(f) If f ∈ H, then
∞
X
Pf = hf, en i en
n=1
is the orthogonal projection of f onto span({en }n∈N ).
Proof. (a) Choose f ∈ H. For each N define
N
X
fN = f − hf, en i en .
n=1

Show that fN ⊥ e1 , . . . , eN (exercise). Therefore, by the Pythagorean Theorem, we have

XN 2 XN N
X
2 2 2
kf k = fN + hf, en i en = kfN k + khf, en i en k ≥ |hf, en i|2 .

n=1 n=1 n=1
Since this is true for each finite n, Bessel’s inequality follows.
(b) Exercise.
P∞
(c) ⇐. Suppose that n=1 |cn |2 < ∞. Set
N
X N
X
sN = cn en , tN = |cn |2 .
n=1 n=1

We know that {tN } is a convergent (hence Cauchy) sequence of scalars, and we must show
that {sN } is a convergent sequence of vectors. We have for N > M that
X N 2 XN XN
2 2
ksN − sM k = cn en = kcn en k = |cn |2 = |tN − tM |.

n=M +1 n=M +1 n=M +1
24 CHRISTOPHER HEIL

Since {tN } is Cauchy, we conclude that {sN } is Cauchy and hence converges.
⇒. Exercise.
(d), (e) Exercise.
(f) By Bessel’s inequality and part (c), we know that the series defining P f converges, so
we just have to show that it is the orthogonal projection of f onto span({en }). Check that
hf − P f, en i = 0 for every n (exercise). By linearity, conclude that f − P f ⊥ span({en }).
By continuity of the inner product, conclude that f − P f ⊥ span({en }) (exercise). 

P
Exercise 4.16. Use Theorem 4.15 to construct an example of a series cn en in a Hilbert
space that converges unconditionally but not absolutely (compare Exercise 1.26).

The next theorem shows that any countable orthonormal sequence that is complete must
be a Schauder basis, and vice versa. Such an orthonormal Schauder basis is usually just
called an orthonormal basis (or just ONB ) for H.

Theorem 4.17. Let {en }n∈N be any orthonormal sequence in a Hilbert space H. Then the
following statements are equivalent.
(a) {en }n∈N is a Schauder P
basis for H, i.e., for each f ∈ H there exist unique scalars
(cn )n∈N such that f = ∞ n=1 cn en .
∞
X
(b) For each f ∈ H, f = hf, en i en .
n=1

(c) {en }n∈N is complete (i.e, it is a Conway basis for H).

∞
X
2
(d) Plancherel/Parseval Equality: For each f ∈ H, kf k = |hf, en i|2 .
n=1

∞
X
(e) Parseval/Plancherel Equality: For each f , g ∈ H, hf, gi = hf, en i hen , gi.
n=1

Proof. (b) ⇒ (e). Suppose that (b) holds, and choose any f , g ∈ H. Then
∞
DX E ∞
X ∞
X


hf, gi = hf, en i en , g = hf, en i en , g = hf, en i hen , gi,
n=1 n=1 n=1

where we have used Exercise 1.24 to move the infinite series outside of the inner product.
(c) ⇒ (b).
P∞If {en } is complete, then its closed span is all of H, so by Theorem 4.15(e) we
have f = n=1 hf, en i en for every f ∈ H.
 CHAPTER 1. HILBERT SPACES 25
P∞
(d) ⇒ (b). Suppose that kf k2 = 2
n=1 |hf, en i| for every f ∈ H. Fix f , and define
PN
sN = n=1 hf, en i en . Then, by direct calculation and the by the Pythagorean Theorem,
kf − sN k2 = kf k2 − hf, sN i − hsN , f i + ksN k2
N
X N
X N
X
2 2 2
= kf k − |hf, en i| − |hf, en i| + |hf, en i|2
n=1 n=1 n=1
N
X
= kf k2 − |hf, en i|2 → 0 as N → ∞.
n=1
P∞
Hence f = n=1 hf, en i en .
The remaining implications are exercises. 

Example 4.18. Let {en }n∈N be an orthonormal basis for a separable Hilbert space H. Define
fn = e1 + en /n. Then {fn }n∈N is complete (exercise) but it is not a Schauder basis for H
(harder).

Exercise 4.19. Let H be a Hilbert space. Show that H is separable if and only if there exists
a countable sequence {xn }n∈N that is an orthonormal basis for H. Hint: By Exercise 4.5 we
know that there exists a complete orthonormal sequence for H.

We state without proof the facts about complete orthonormal systems in nonseparable
Hilbert spaces.

Theorem 4.20. Let {ei }i∈I be an orthonormal sequence in a Hilbert space H (note that I
might be uncountable). Then the following statements hold.
(a) If f ∈ H then hf, ei i 6= 0 for at most countably many i.
X
(b) For each f ∈ H, |hf, ei i|2 ≤ kf k2 .
i∈I
X
(c) For each f ∈ H, hf, ei i ei converges with respect to the net of finite subsets of I
i∈I
(see Proposition 1.27 for the meaning of net of finite subsets).

Theorem 4.21. Let {ei }i∈I be an orthonormal sequence in a Hilbert space H. Then the
following statements are equivalent.
(a) {ei }i∈I is complete (i.e., is a Conway basis for H).
X
(b) For each f ∈ H, f = hf, ei i ei with respect to the net of finite subsets of I.
i∈I
X
(c) For each f ∈ H, kf k2 = |hf, ei i|2 (only countably many terms are nonzero).
i∈I
26 CHRISTOPHER HEIL

The (Hilbert space) dimension of H is the cardinality of a complete orthonormal sequence.

It can be shown that any two complete orthonormal sequences must have the same cardinal-
ity. Further, H is separable if and only if it has a countable complete orthonormal sequence
(which by Theorem 4.17 will be an orthonormal Schauder basis for H).

5. Isomorphic Hilbert Spaces and the Fourier Transform for the Circle
Definition 5.1. If H1 , H2 are Hilbert spaces, then L : H1 → H2 is an isomorphism if L is
a linear bijection that satisfies
∀ f, g ∈ H1 , hLf, Lgi = hf, gi.
In this case we say that H1 and H2 are isomorphic, and write H1 ∼
= H2 .
If H1 = H2 = H and L : H → H is an isomorphism, then we say that L is unitary.

Proposition 5.2. Let L : H1 → H2 be a linear mapping. Then the following statements are
equivalent.
(a) L is inner-product-preserving, i.e., hLf, Lgi = hf, gi for all f , g ∈ H1 .
(b) L is norm-preserving (an isometry), i.e., kLf k = kf k for all f ∈ H1 .
Proof. We need only show that (b) implies (a). Assume that L is an isometry, and fix f ,
g ∈ H. Then for any scalar λ ∈ F we have by the Polar Identity and the fact that L is
isometric that
kf k2 + 2 Re λ̄ hf, gi + |λ|2 kgk2 = kf + λgk2
= kLf + λLgk2
= kLf k2 + 2 Re λ̄ hLf, Lgi + |λ|2 kLgk2
= kf k2 + 2 Re λ̄ hLf, Lgi + |λ|2 kgk2 .
Thus Re λ̄ hLf, Lgi = Re λ̄ hf, gi for every λ ∈ F. This implies (take λ = 1 and λ = i) that
hLf, Lgi = hf, gi. 

In particular, every isomorphism is an isometry and hence is automatically injective.

Corollary 5.3. Let L : H1 → H2 be given. Then L is an isomorphism if and only if it is a

linear surjective isometry.

Theorem 5.4. All separable infinite-dimensional Hilbert spaces are isomorphic.

Proof. Let H1 and H2 be separable infinite-dimensional Hilbert spaces. Then H1 has a
countable orthonormal basis {en }n∈N , and H2 has P∞a countable orthonormal basis {hn }n∈N .
2
P ∞ 2
If f ∈ H1 then by Theorem 4.17 we have f = n=1 hf, en i eP n and kf k = n=1 |hf, en i| .
Since thisPis finite and since {hn } is an ONB, the series Lf = ∞n=1 hf, en i hn converges, and
kLf k = ∞
2
n=1 |hf, e n i| 2
= kf k 2
.
Exercise: Finish the proof by showing that L is a linear surjective isometry. 
 CHAPTER 1. HILBERT SPACES 27

In particular, every separable Hilbert space is isomorphic to `2 (N).

The proof can be extended to show that any two Hilbert spaces which have the same
dimension are isomorphic.

Example 5.5 (Fourier Series). Now we give one of the most important examples of an
orthonormal basis.
We saw in Exercise 2.2 that if we define en (x) = e2πinx , then {en }n∈Z is an orthonormal
sequence in L2 [0, 1] (the space of square-integrable complex-valued functions on the domain
[0, 1], or we can consider these functions to be 1-periodic functions on the domain R).
It is a fact that {en }n∈Z is actually complete in L2 [0, 1] and hence is an orthonormal basis
for L2 [0, 1]. The Fourier coefficients of f ∈ L2 [0, 1] are
Z 1
ˆ
f (n) = hf, en i = f (x) e−2πinx dx, n ∈ Z.
0

By Theorem 4.17, we have

fˆ(n) en
X X
f = hf, en i en = (5.1)
n∈Z n∈Z

(the series converges unconditionally, so it does not matter what ordering of the index set Z
that we use to sum with). Equation 5.1 is called the Fourier series of f .
However, note that the series in 5.1 converges in the norm of the Hilbert space, i.e., in
L -norm. That is, the partial sums converge in L2 -norm, i.e.,
2

N
ˆ
X
lim f − f (n) en = 0,

N →∞ 2
n=−N
or,
Z 1
 N 2
ˆ(n) e
X
2πinx
 
lim f (x) − f  dx = 0.
N →∞ 0
 
n=−N

We cannot conclude from this that the equality

fˆ(n) e2πinx
X
f (x) =
n∈Z

holds pointwise. In fact, one of the deepest results of Fourier series is the Carleson–Hunt
Theorem, which proves the conjecture of Lusin that if f ∈ Lp [0, 1] where 1 < p ≤ ∞, then
the Fourier series of f converges to f a.e.

Exercise 5.6. Show that the mapping F : L2 [0, 1] → `2 (Z) given by F (f ) = fˆ = {fˆ(n)}n∈Z
is exactly the isomorphism constructed by Theorem 5.4 for the case H1 = L2 [0, 1] and
H2 = `2 (Z). The operator F is the Fourier transform for the circle (thinking of functions in
L2 [0, 1] as being 1-periodic, the domain [0, 1] is topologically a circle).
28 CHRISTOPHER HEIL

Exercise 5.7. Prove the following (easy) special case of the Riemann–Lebesgue Lemma: If
f ∈ L2 [0, 1] then fˆ(n) → 0 as |n| → ∞. The full Riemann–Lebesgue Lemma for the circle
states the same conclusion holds if we only assume that f ∈ L1 [0, 1].

Definition 5.8. In honor of Fourier series, if {en }n∈I is any orthonormal basis of a separable
Hilbert space
P H, then {hf, en i}n∈I is the sequence of (generalized) Fourier coefficients of f
and f = n∈I hf, en i en is the (generalized) Fourier series of f .

Exercise 5.9. The Plancherel formula with respect to the orthonormal basis {en }n∈Z for
L2 [0, 1] is
|fˆ(n)|2 .
X
kf k22 =
n∈Z
Use the Plancherel formula to derive a formula for π by applying it to the function f =
χ[0,1/2) − χ[1/2,1) .