Professional Documents
Culture Documents
Orthonormal Sets
Definition 5: Let V be an ips. A set S V is called an
orthonormal set if S is orthogonal and kxk = 1 for each x S. In
addition, if an orthonormal set S is also a basis for V , then S is
called an orthonormal basis.
Example 12:
(a) The standard basis of Rn is an orthonormal basis of it.
(b) The set of functions {cos mt : m N} in the real ips C [0, 2]
R 2
with inner product as hf , g i = 0 f (t)g (t) dt is an orthogonal set.
R 2
But 0 cos2 t dt 6= 1. Hence, it is not an orthonormal set.
A Result
Proposition 4: Let S = {u1 , u2 , . . . , un } be an orthonormal set in
an ips V . Let x span(S). Then
P
P
x = nj=1 hx, uj iuj and kxk2 = nj=1 |hx, uj i|2 .
Proof. x = 1 u1 + 2 u2 + + n un . Then hx, uj i = j .
P
P
Next, kxk2 = h j j uj , i i ui i
P P
= j i j i huj , ui i
P
= j j j
P
= nj=1 |hx, uj i|2 .
Some Corollaries
Proposition 5: (Fourior Expansion and Parsevals Identity)
Let {v1 , v2 , . . . , vn } be an orthonormal basis for an ips V . Let
x V . Then P
P
x = nj=1 hx, vj ivj and kxk2 = nj=1 |hx, vj i|2 .
Proposition 6: (Bessels Inequality) Let {u1 , u2 , . . . , un } be an
orthonormal set in an P
ips V . Let x V . Then
n
2
2
j=1 |hx, uj i| kxk .
P
Proof. Let y = nj=1 hx, uj i uj Then hx, ui i = hy , ui i.
That is, x y ui , for each i. So, x y y .
By Pythagoras theorem, kxk2 = kx y k2 + ky k2 ky k2 . As
y span{u
P 1 , . . . , un }, by Parsevals identitty,
ky k2 = nj=1 |hx, uj i|2 .
Projection
Given two linearly independent vectors u1 , u2 on the plane how do
we construct two orthogonal vectors?
Keep v1 = u1 .
Take out the projection of u2 on u1 to get u2 .
Now, u2 u1 .
What is the projection of u2 on u1 ? Its length is hu2 , u1 i. Its
direction is that of u1 , i.e., u1 /ku1 k. Thus
v1 = u1 , v2 = u2
hu2 , v1 i
v1 .
hv1 , v1 i
Gram-Schmidt Orthogonalization
Let V = span{u1 , u2 , . . . , un }. Define
v1 = u1 .
2 ,v1 i
v2 = u2 hu
hv1 ,v1 i v1
..
.
,vn i
,v1 i
v1 huhvn+1
vn .
vn+1 = un+1 huhvn+1
n ,vn i
1 ,v1 i
Proposition 7: If {u1 , u2 , . . . , un } is linearly independet, then
{v1 , v2 , . . . , vn } is orthogonal and
span{v1 , v2 , . . . , vn } = span{u1 , u2 , . . . , un }.
Proof. hv2 , v1 i = hu2
hu2 ,v1 i
hv1 ,v1 i
v1 , v1 i
hu2 ,v1 i
hv1 ,v1 i hv1 , v1 i
= hu2 , v1 i
= 0.
Proof follows by induction.
Examples
Example 14:
The vectors u1 = (1, 1, 0), u2 = (0, 1, 1), u3 = (1, 0, 1) form a basis
for F3 . Apply Gram-Schmidt Orthogonalization:
v1 = (1, 1, 0).
2 ,v1 i
v2 = u2 hu
hv1 ,v1 i v1
(0,1,1)(1,1,0)
(1,1,0)(1,1,0) (1, 1, 0)
= (0, 1, 1) 21 (1, 1, 0) = ( 12 , 12 , 1).
hu3 ,v2 i
3 ,v1 i
v3 = u3 hu
hv1 ,v1 i v1 hv2 ,v2 i v2
= (0, 1, 1)
Example 15:
The vectors u1 = 1, u2 = t, u3 = t 2 form a linearly independent set
in the ips of all polynomials considered as functions from [1, 1] to
R1
R; inner rpoduct is hp(t), q(t)i = 1 p(t)q(t) dt. Applying
Gram-Schmidt Process, we see:
v1 = u1 = 1.
v2 = u2
hu2 ,v1 i
hv1 ,v1 i
R1
v1 = t
R1
1
1
t dt
dt
hu3 ,v2 i
hu3 ,v1 i
hv1 ,v1 i v1 hv2 ,v2 i v2
R1 3
R1 2
t dt
1 t dt
2
t R1
1 R1
1
2
dt
1
1 t dt
1 = t.
v3 = u3
=
t = t 2 1/3.
Best Approximation
What is the best point on a line that approximates a point not on
the line? What is the best point on a circle that approximates a
point not on the circle? What is the best point on a surface that
approximates a point not on the surface?
Definition 7: Let U be a subspace of an ips V . Let v V . A
vector u U is a best approximation of v if kv uk kv xk for
each x U.
Proposition 11: Let U be a subspace of an ips V . A vector u U
is a best approximation of v V iff v u U. Moreover, a best
approximation is unique.
Proof. (a) Suppose v u U. Let x U. Now,
kv xk2 = k(v u) + (u x)k2
= kv uk2 + ku xk2 kv uk2 .
as u x U and Pythagoras.
Hence, kv uk kv xk for each x U.
Best Approximation
Proposition 12: Let {u1 , . . . , un } be an orthonormal basis for
U S V , an ips.
v V . The unique best approximation of v
PLet
n
from U is u = i=1 hv , ui iui .
P
Proof. Let x U. Then, x = nj=1 hx, uj iuj . Now,
P
P
hv u, xi = hv ni=1 hv , ui iui , nj=1 hx, uj iuj i
P
P P
= nj=1 hx, uj ihv , uj i ni=1 nj=1 hv , ui ihx, uj ihui , uj i
P
P
= nj=1 hx, uj ihv , uj i ni=1 hv , ui ihx, ui i = 0.
That is, v u U.
Pn
Examples
Example 16: What is the best approximation of v = (1, 0) R2
from U = {(a, a) : a R}?
Find (, ) so that (1, 0) (, ) (, ) for all .
Find (, ) so that (1 , ) (1, 1) = 0.
So, = 1/2. The best approximation here is (1/2, 1/2).
R1
Example 17: In C [0, 1] over R, with hf , g i = 0 f (t)g (t) dt, what
is the best approximation of t 2 from P1 ?
Determine , so that t 2 ( + t) 1 and t 2 ( + t) t.
R1 2
R1 3
2
(t
t)
dt
=
0
=
0
0 (t t t ) dt.
1/3 /2 = 0 = 1/4 /2 /3.
The best approximation is 1/6 + t.