Some Important Matrix Factorizations

LU Decomposition:
A square matrix A can be factored as A = LU, where L is the lower trianular matrix with !"s
on the main diaonal and U is the upper trianular matrix#
Example1: Find the LU decomposition of A =
$ % &
' ( !
% ( )
_




,
Solution: Let A = LU
$ % &
' ( !
% ( )
_




,
=
! * *
! *
!
a
b c
_



,

*
* *
d e f
g h
i
_



,
=
d e f
ad ae g af h
bd be cg bf ch i
_

+ +


+ + +
,
+quatin the correspondin elements, we et
d = $, e = %, f = ,&, h = !', a = ), = ,%, c = %, b = $, i = ,&-
.herefore L =
! * *
) ! *
$ % !
_



,
and U =
$ % &
* % !'
* * &-
_

,
Example2: Find the LU decomposition of A =
% ( !$
( $* $*
!$ $* %!
_



,
without pi/otin#
Solution: A =
% ( !$
( $* $*
!$ $* %!
_



,
=
% ( !$
* % %
!$ $* %!
_

,
0
$
0
$
1 $ 0
!
=
% ( !$
* % %
* % &
_

,
0
)
0
)
1 ) 0
!
=
% ( !$
* % %
* * !
_

,
0
)
0
)
2 0
$
.hus U =
% ( !$
* % %
* * !
_

,
.o determine L, the steps of elimination are represented in terms of matrix
multiplication# .he first modified matrix can be represented as
% ( !$
* % %
!$ $* %!
_

,
=
! * * % ( !$
$ ! * ( $* $*
* * ! !$ $* %!
_ _

, ,
= +
!
A
.he next step can be represented as
% ( !$
* % %
* % &
_

,
=
! * *
* ! *
) * !
_

,

% ( !$
* % %
!$ $* %!
_

,
= +
$
+
!
A
.he final step is
U =
% ( !$
* % %
* * !
_

,
=
! * *
* ! *
* ! !
_



,

% ( !$
* % %
* % &
_

,
= +
)
+
$
+
!
A
.hus U = +
)
+
$
+
!
A
and A =
! ! !
! $ )
E E E U

= LU, the in/erse of elementar3 matrices are
obtained b3 simpl3 chanin the sin of the off diaonal element
! ! !
! $ )
E E E U

=
! * * ! * * ! * *
$ ! * * ! * * ! *
* * ! ) * ! * ! !
_ _ _

, , ,
=
! * *
$ ! *
) ! !
_

,
= L
The Cholesky acto!i"ation:
Let A be a positi/e definite m m 4ermitian matrix# .hen A can be factored as
5i6 A = L L
4
5ii6 A = U
4
7U where
L = lower trianular matrix
7 = dia ( )
$ $ $
!! $$
, , ###,
mm
l l l
!! $$
5 , , ####, 6
mm
D diag l l l
Let L
4
=
D
U, where U is upper trianular which has !"s in the diaonal#
A = L L
4
=
( ) ( )
#
DU DU
= U
4
D

D
U
= U
4
7 U
8ote!9 If A is real, the choles:3 factorization becomes
5i6 A = L L
.
5ii6 A = U
.
7U
8ote$9 It can be used to sol/e the s3stem of equations A; = < where A is hermitian
and positi/e definite#
LL
.
; = <
L = = < >#5!6 where = = L
.
; >5$6
Sol/in 5!6 and then 5$6, we et the solution of A; = <#
Example$9 ?btain the choles:3 decomposition of A =
@ ' !$
' !) !!
!$ !! $'
_



,
and hence
Sol/e the s3stem A; = <, where ; = 5 x 3 z 6
.
and < = 5 !-% $)' )*( 6
.
#
Solution: Let A = L L
.
@ ' !$ * *
' !) !! * *
!$ !! $' * *
a a b d
b c c e
d e f f
_ _ _

, , ,
Multipl3in A equatin the elements, we et, a =), b = $, c = ), d = %, e = !, f = )
.herefore L =
) * *
$ ) *
% ! )
_



,

<ut L
.
=
D
U, where U is upper trianular which has !"s in the diaonal

!! $$ ))
5 , , 6 D diag l l l
) $ % ) * * !
* ) ! * ) * * !
* * ) * * ) * * !
a b
c
_ _ _

, , ,
Multipl3in and equatin, we et, a = $B), b = %B), c = !B)#
.herefore U =
$ %
!
) )
!
* !
)
* * !
_







,
<ut U
.
7 U = A
U
.
7 U =
! * *
@ * *
$
! * * @ *
)
* * @
% !
!
) )
_

_







,


,

$ %
!
) )
!
* !
)
* * !
_







,
=
@ * *
' @ *
!$ ) @
_



,

$ %
!
) )
!
* !
)
* * !
_







,
=
@ ' !$
' !) !!
!$ !! $'
_



,
.o sol/e the s3stem A; = <
Consider
@ ' !$
' !) !!
!$ !! $'
_



,
!-%
$)'
)*(
x
y
"
_ _

, ,
) * *
$ ) *
% ! )
_



,

) $ %
* ) !
* * )
_



,
!-%
$)'
)*(
x
y
"
_ _

, ,
,,,,,,,5!6 since A = L L
.

) $ %
* ) !
* * )
_



,

!
$
)
x y
y y
" y
_ _

, ,
,,,,,,,,,5$6
.herefore 5!6 becomes
) * *
$ ) *
% ! )
_



,

!
$
)
y
y
y
_



,
=
!-%
$)'
)*(
_



,
.his i/es 3
!
= &(, 3
$
= %*, 3
)
= !$
Usin this in 5$6, we et
) $ %
* ) !
* * )
_



,

x
y
"
_



,
=
&(
%*
!$
_



,
.his i/es x = ', 3 = !$, z = %#
7efinition9 .he conDuate of the transpose of a i/en matrix A is called the conDuate
transpose of A and is denoted b3 A

# .hus A

=
( )
T
T
% %
7efinition 9 A square matrix A of order n is said to be unitar3 if A

A = AA

=I
n
Definition: Let A be a non sinular matrix# .hen there exists a unitar3 matrix E and an
upper trianular matrix 0 such that A = E0#
&' facto!i"ation by (!am Schmidt )!ocess
Let A =
!! !$ !
! $
####
### ### ###
####
n
n n nn
a a a
a a a
_



,
Step!9 Let F
!
, F
$,
># F
n
be the columns of A
i#e# F
!
=
!! !$ !
$! $$ $
$
! $
, , ####,
### ### ###
n
n
n
n n nn
a a a
a a a
* *
a a a
_ _ _






, , ,
Step$9 Find the orthonormal basis G H
!,
H
$
, >, H
n
I b3 Jram Schmidt process# Hhere
H
!
=
!
!
*
* P P
U
$
= F
$
K H
!
, F
$
L H
!

H
$
=
$
$
U
U P P
U
)
= F
)
K H
!
, F
)
L H
!
K H
$
, F
)
L H
$
H
)
=
)
)
U
U P P
And in the same wa3 U
n
= F
n
K H
!
,F
n
LH
!
>## K H
n!
, F
n
L H
n!
n
n
n
U
+
U

P P
Step)9 Form E = 5 H
!,
H
$
, >, H
n
6
Step%9 Hrite the /ectors F
!
, F
$,
># F
n
in terms of H
!,
H
$
, >, H
n

Step&9 Hrite 0 from the coefficients in the expression of F
!
, F
$,
># F
n
in step%#
Step'9 E 0 = A
Example,: Find E0 factorization for the matrix A =
) $
% &
_

,
b3 Jram Schmidt process#
Solution: Let A =
) $
% &
_

,
Let F
!
,

F
$
be the columns of A# i#e# F
!
=
)
%
_

,
, F
$
=
$
&
_

,
.he orthonormal basis is 5 H
!
, H
$
6
H
!
=
!
!
*
* P P
=
$ $
)
)
!
&
% %
) %
&
_

_


+ ,

,
U
$
= F
$
K H
!
, F
$
L H
!

=
) )
$ $
& &
,
& % & %
& &
_ _

_ _


, ,

, ,
=
)
$
' $*
&
& % & &
&
_

_

+
' ;
,

,
=
$(
$&
$!
$&
_

,
H
$
=
$
$
U
U P P
=
%
&
)
&
_

,
E = 5 H
!
, H
$
6
=
) %
& &
% )
& &
_

,
0ewrite the /ectors F
!
, F
$
in terms of H
!
and H
$
4ere F
!
= &H
!
F
!
= &H
!
2 *H
$
Let F
$
= H
!
2 H
$
) %
$
& &
& % )
& &

_ _

_
+

,

, ,
) % = !*
% 2 ) = $&
sol/in this, we et =
$' -
& &
and
F
$
=
! $
$' -
& &
+ + +
.herefore 0 =
$'
&
&
-
*
&
_




,
8ow E0 =
) %
& &
% )
& &
_

,

$'
&
&
-
*
&
_




,
=
) $
% &
_

,
= A
Example-: ?btain the E0 decomposition of A =
* * &
* % !
! ! !
_



,
Solution: Let F
!,
F
$
, F
)
be the columns of A# i#e# F
!
=
$ )
* * &
* , % , !
! ! !
* *
_ _ _




, , ,
.he orthonormal basis is 5 H
!
, H
$
, H
)
6 where
H
!
=
!
!
*
*
!
*
*
_

,
P P
U
$
= F
$
K H
!
, F
$
L H
!
=
*
%
*
_



,
H
$
=
$
$
U
U P P
=
*
!
*
_



,
U
)
= F
)
K H
!
, F
)
L H
!
K H
$
, F
)
L H
$
=
&
*
*
_



,
H
)
=
)
)
U
U P P
=
!
*
*
_



,
8ow E = 5 H
!
H
$
H
)
6
=
* * !
* ! *
! * *
_



,
Hrite the /ectors F
!,
F
$,
F
)
in terms of H
!,
H
$ ,
H
)
Clearl3, F
!
= H
!
= H
!
2 *H
$
2 *H
)
Let F
$
= H
!
2 H
$
2 H
)
i#e# F
$
= H
!
2 %H
$
2 *H
)
Also F
)
= H
!
2 H
$
2 H
)
i#e# F
)
= H
!
2 H
$
2 &H
)
.herefore 0 =
! ! !
* % !
* * &
_



,
E0 =
* * ! ! ! !
* ! * * % !
! * * * * &
_ _

, ,
=
* * &
* % !
! ! !
_



,
= A
Example.: Find the sinular /alue decomposition of A =
$ !
$ !
% $
_

,
Solution: Find A
.
A#
A
.
A =
$% !$
!$ '
_

,
.he eien /alues and eien /ectors of A
.
A are = )*, * and
$ !
,
! $
_ _

, ,

respecti/el3#
.he orthonormal eien /ectors are ;
!
=
$
&
!
&
_

,
and ;
$
=
!
&
$
&
_




,
4ence onl3 one non zero sinular /alue 5square root of non zero eien /alues6 is )*
.he diaonal matrix 7 with non zero sinular /alue as diaonal is 7 = 5 )* 6
!!
Find the matrix
)$
with 7 at the top left and the other /alues are zero#
=
)* *
* *
* *
_



,
8ow the unitar3 matrix F with orthonormal eien /ectors of A
.
A

as columns is
F =
$ !
& &
! $
& &
_

,
Consider the matrix AA
.
AA
.
=
& & !*
& & !*
!* !* $*
_

,
.he eien /alues of AA
.
are = )*, * , * and its correspondin eien /ectors are
! ! !
! , ! , !
$ * !
_ _ _

, , ,
.he unitar3 matrix U with orthonormal eien /ectors of AA
.
as its column is
U =
! ! !
' $ )
! ! !
' $ )
$ !
*
' )
_

,
Claim9 = U
.
A F
7efinition9 Let A be an mn matrix with complex entries# .he eneralized in/erse of
A 5pseudo in/erse6 of A is the unique nm matrix A
2
satisfies
5i6 AA
2
is 4ermitian i#e# AA
2
= 5
%%
+
6
.
5ii6 A
2
A is 4ermitian
5iii6 AA
2
A = A 5i/6 A
2
AA
2
= A
2
Example/: Find the pseudo in/erse A
2
of A =
$ !
$ !
% $
_

,
Solution: From the pre/ious example, we ha/e =
)* *
* *
* *
_



,
and
!
=
!
* *
)*
* * *
_



,
Also F =
$ !
& &
! $
& &
_

,
and U =
! ! !
' $ )
! ! !
' $ )
$ !
*
' )
_

,
8ow the eneralized in/erse A
2
is i/en b3 A
2
= F
!
U
.
=
! ! $
!& !& !&
! ! !
)* )* !&
_

,
8ote9 Usin pseudo in/erse, sol/e the s3stem A; = <
AA
2
A ; = < 5 usin the definition6
; = A
2
AA
2
<
= A
2
<
Example0: 7etermine E0 decomposition b3 4ouseholder transformation of
A =
* ) &*
) & $&
% * $&
_



,
Solution9 Let x =
*
)
%
_



,
, the first column of A#
.hen m =
$ $ $
* ) % & + +
$q
$
= m
$
2 mx
!
sn5x
!
6 = $& 2 & 5*6 526 = $&
U =
! !
$
)
sn5 6 &
)
%
x m x
x
x
_ + _

,
,
8ow M
!
= I
)
( )
$
! * * &
!
* ! * ) & ) %
$ $&
* * ! %
T
UU
1
_ _




, ,
U
!
= M
!
=

* !& $*
!
!& !' !$
$&
$* !$ @
_





,


U
!
A =
* !& $*
!
!& !' !$
$&
$* !$ @
_





,

* ) &*
) & $&
% * $&
_



,
=
& ) )&
-
* $'
&
$%
* %)
&
_

,
Let x =
-
&
$%
&
_

,
m =
$ $
- $%
&
& &
_ _
+

, ,

$q
$
= m
$
2 mx
!
sn5x
!
6 = $& 2 & 5-B&6 526 = )$
U =
! !
$
sn5 6 x m x
x
_ +

,
=
- )$
&5 6
& &
$% $%
& &
_ _
+ +





, ,
M
$
= I
$

$
)$
! *
! )$ $%
&
* ! $% $ )$ & &
&
T
UU
1
_

_
_


, ,

,
=
- $%
!
$% - $&
_

,
U
$
=
$
! *
* )
_

,
=
$& * *
!
* - $%
$&
* $% -
_

,
U
$
U
!
A =
$& * *
!
* - $%
$&
* $% -
_

,

& ) )&
-
* $'
&
$%
* %)
&
_

,
=
& ) )&
* & )%
* * )-
_



,
.a:e 0 = U
$
U
!
A
and E = U
!
U
$
=
* !& $*
!
!& !' !$
$&
$* !$ @
_





,


$& * *
!
* - $%
$&
* $% -
_

,
=
* !& $*
!
!& !' !$
$&
$* !$ @
_





,
Claim9 A = E 0
Example2: Sol/e the s3stem A; = < where A =
* ) &* &)
) & $& , )' ,
% * $& ))
x
3 4 y
"
_ _ _




, , ,
Solution: 4ere NAN *
.herefore 05A6 = ) and hence 05A,<6 = )# Also the number of /ariable = )#
4ence the s3stem has unique solution#
Let x =
*
)
%
_



,

m =
$ $ $
* ) % & + +

$q
$
= m
$
2 mx
!
sn5x
!
6 = $& 2 & 5*6 526 = $&
U =
! !
$
)
sn5 6 &
)
%
x m x
x
x
_ + _

,
,
8ow M
!
= I
)
( )
$
! * * &
!
* ! * ) & ) %
$ $&
* * ! %
T
UU
1
_ _




, ,
U
!
= M
!
=

* !& $*
!
!& !' !$
$&
$* !$ @
_





,


U
!
5A,<6 =
* !& $*
!
!& !' !$
$&
$* !$ @
_





,

* ) &* &)
) & $& )'
% * $& ))
_



,

=
& ) )& %(
* - B & $' !$)B &
* $%B & %) $)@B &
_





,
Let x =
-
&
$%
&
_

,
m =
$ $
- $%
&
& &
_ _
+

, ,

$q
$
= m
$
2 mx
!
sn5x
!
6 = $& 2 & 5-B&6 526 = )$
U =
! !
$
sn5 6 x m x
x
_ +

,
=
- )$
&5 6
& &
$% $%
& &
_ _
+ +





, ,
M
$
= I
$

$
)$
! *
! )$ $%
&
* ! $% $ )$ & &
&
T
UU
1
_

_
_


, ,

,
=
- $%
!
$% - $&
_

,
U
$
=
$
! *
* )
_

,
=
$& * *
!
* - $%
$&
* $% -
_

,
U
$
U
!
5A,<6 =
$& * *
!
* - $%
$&
* $% -
_

,

& ) )& %(
* - B & $' !$)B &
* $%B & %) $)@B &
_





,
=
& ) )& %(
* & )% )@
* * )- )-
_





,
8ow 0 =
& ) )&
* & )%
* * )-
_

,
and a =
%(
)@
)-
_

,
Sol/in 0; = a, we ha/e &x )3 )&z = %(
&3 )%z = )@
)-z = )-
i#e# x = $, 3 = !, z = !#
7efinition9 A nn matrix is said to be .oeplitz matrix if a
ii
are equal and a
iD
= a
i2! D2!

for all iKn, DKn#
+xample9 A =
! ) & -
$ ! ) &
% $ ! )
' % $ !
_





,

8ote9 All entries of this matrix are defined once the first column and first row ha/e been
specified#
7efinition9 A tridiaonal matrix is a band matrix of width )# 8on zero elements appear onl3
on the main diaonal, super diaonal and sub diaonal# All other diaonal contain onl3 *
elements#
+xample9 A =
! $ * *
) ! $ *
* ) ! $
* * ) !
_





,
.he LU decomposition for a .ridiaonal matrix#
Step!9 If a
!!
= *, the decomposition is not possible#
Let l
!! =
a
!!
Set sub diaonal of L = .he sub diaonal of A
Set each diaonal of U = !
Set all other elements of L and U are *#
Step$9 u
i! i
= a
i! i
B l
i! i!
and l
ii
= a
ii
l
i i!
# u
i!

i
If i=n stop
If l
ii
= *, factorization is not possible# ?therwise increase b3 ! and o to step$#
.his factorization will produce an L matrix ha/in non zero elements onl3 on its diaonal
and its sub diaonal and U ha/e non zero element onl3 on diaonal and super diaonal#
Example: Find the LU decomposition of A =
$ ! * * * *
! $ ! * * *
* ! $ ! * *
* * ! $ ! *
* * * ! $ !
* * * * ! $
_

,
4ere 7iaonal = $, Sub diaonal = !, Superdiaonal = !, n = '# Let l
!!
= a
!!
= $
L =
* * * * * * ! * * * * *
! * * * * * * ! * * * *
* ! * * * * * * ! * * *
* * ! * * * * * * ! * *
* * * ! * * * * * * ! *
* * * * ! * * * * * * !
and U
_ _

, ,
8ow u
i! i
= a
i! i
B l
i! i!
and l
ii
= a
ii
l
i i!
# u
i!

i
Hhen i = $ , u
!$
= a
!$
B l
!!
=
!
$
and l
$$
= a
$$
l
$!
u
!$
=
&
$
Hhen i = ) , u
$)
= a
$)
B l
$$
=
$
&
and l
))
= a
))
l
)$
u
$)
=
!$
&
Hhen i = % , u
)%
= a
)%
B l
))
=
&
$
and l
%%
= a
%%
l
%)
u
)%
=
$@
!$
Hhen i = & , u
%&
= a
%&
B l
%%
=
!$
$@
and l
&&
= a
&&
l
&%
u
%&
=
-*
$@
Hhen i = ' , u
&'
= a
&'
B l
&&
=
$@
-*
and l
''
= a
''
l
'&
u
&'
=
!'@
-*
8ow i = n = '# 4ence stop the procedure
% L U
=
$ * * * * * ! !B $ * * * *
! &B $ * * * * * ! $B & * * *
* ! !$B & * * * * * ! &B!$ * *
* * ! $@B!$ * * * * * ! !$B @ *
* * * ! -*B $@ * * * * * ! $@B -*
* * * * ! !'@B -* * * * * * !
_ _

, ,
Example: Find LU decomposition of .ridiaonal .oeplitz matrix#
A =
! $ * * *
! $ $ * *
* $ ! ! *
* * ) ! !
* * * ) $
_

,
4ere n = &, d
!!
= a
!!
= !#
L =
* * * * *
! * * * *
* $ * * *
* * ) * *
* * * ) *
_

,
and U =
! * * * *
* ! * * *
* * ! * *
* * * ! *
* * * * !
_






,
8ow u
i! i
= a
i! i
B l
i! i!
and l
ii
= a
ii
l
i i!
# u
i!

i
Hhen i = $ , u
!$
= a
!$
B l
!!
= $ and l
$$
= a
$$
l
$!
u
!$
= %
Hhen i = ) , u
$)
= a
$)
B l
$$
=
!
$
and l
))
= a
))
l
)$
u
$)
= $
Hhen i = % , u
)%
= a
)%
B l
))
=
!
$
and l
%%
= a
%%
l
%)
u
)%
=
&
$

Hhen i = & , u
%&
= a
%&
B l
%%
=
$
&
and l
&&
= a
&&
l
&%
u
%&
=
%
&
A = L U
=
! * * * * ! $ * * *
! % * * * * ! !B $ * *
* $ $ * * * * ! !B $ *
* * ) &B $ * * * * ! $B &
* * * ) %B & * * * * !
_ _

, ,
&' facto!i"ation 5 %nothe! 6ethod7
Let A be a i/en matrix and F
!
, F
$
, >##, F
n
be the columns of A#
7efine r
::
= NNF
::
NN
E
:
=
!
kk
*
!
, r
::
*
= choose an3 E
i
normalized to E
!
, E
$
, >#, E
:!
Calculate r
:D
= K E
:
# F
D
L where U
D
= F
D
r
:D
E
:

I Iteration9
r
!!
= NN F
!
NN and E
!
=
!
!!
*
!
r
!$
= K E
!
# F
$
L , r
!)
= K E
!
# F
)
L , >>>##, r
!n
= K E
!
# F
n
L
U
$
= F
$
r
!$
E
!
, U
)
= F
)
r
!)
E
!
, >>, U
n
= F
n
r
!n
E
!
II Iteration9
r
$$
= NN U
$
NN and E
$
=
$
$$
U
!
r
$)
= K E
$
# U
)
L , r
$%
= K E
$
# U
%
L , >>>##, r
$n
= K E
$
# U
n
L
H
)
= U
)
r
$)
E
$
, H
%
= U
%
r
$%
E
$
, >>, H
n
= U
n
r
$n
E
$
III Iteration9
r
))
= NN H
)
NN and E
)
=
)
))
+
!
r
)%
= K E
)
# H
%
L , r
)&
= K E
)
# H
&
L , >>>##, r
)n
= K E
)
# H
n
L
;
%
= H
%
r
)%
E
)
, ;
&
= H
&
r
)&
E
)
, >>>, ;
n
= H
n
r
)n
E
)
IF Iteration9
r
%%
= NN ;
%
NN and E
%
=
%
%%
4
!
and continue in this manner# Finall3 E = 5 E
!
E
$
E
)
>>## E
n
6
0 =
!! !$ !)
$$ $)
))
%%
###
* ###
* * ###
* * *
* * ## ##
! ! !
! !
!
!
_






,
Example: Find the E0 factorization of A =
* ! ! !
! * ! !
! ! * !
! ! ! *
_





,
Solution: Let F
!
=
$ ) %
* ! ! !
! * ! !
, , ,
! ! * !
! ! ! *
* * *
_ _ _ _






, , , ,
r
!!
= NN F
!
NN = ) and E
!
=
!
!!
*
!
=
*
!
!
!
)
!
_





,
r
!$
= K E
!
# F
$
L =
*
!
!
!
)
!
_





,
#
!
*
!
!
_





,
=
$
)

similarl3 r
!)
= K E
!
# F
)
L =
$
)
, and r
!%
= K E
!
# F
%
L =
$
)
U
$
= F
$
r
!$
E
!
=
!
*
!
!
_





,

$
)

*
!
!
!
)
!
_





,
=
!
$B )
!B )
!B )
_

,

U
)
= F
)
r
!)
E
!
=
!
!
*
!
_





,

$
)

*
!
!
!
)
!
_





,
=
!
!B )
$B )
!B )
_

,
Similarl3, U
%
= F
%
r
!%
E
!
=
!
!B )
!B )
$B )
_

,
II Iteration9
r
$$
= NN U
$
NN =
!&
)
and E
$
=
$
$$
U
!
=
)
!&

!
$B )
!B )
!B )
_

,
=
!
!&
)
$
!
!
_

,
r
$)
= E
$
# U
)
=
!
!&
)
$
!
!
_

,
#
!
!B )
$B )
!B )
_

,
=
$
!&

r
$%
= E
$
# U
%
=
!
!&
)
$
!
!
_

,
#
!
!B )
!B )
$B )
_

,
=
$
!&
H
)
= U
)
r
$)
E
$
=
!
!B )
$B )
!B )
_

,

$
!&

!
!&
)
$
!
!
_

,
=
)B &
)B &
%B &
!B &
_

,
H
%
= U
%
r
$%
E
$
=
!
!B )
!B )
$B )
_

,

$
!&

!
!&
)
$
!
!
_

,
=
)B &
)B &
!B &
%B &
_

,
III Iteration9
r
))
= NN H
)
NN =
)&
&
and E
)
=
)
))
+
!
=
&
)&
)B &
)B &
%B &
!B &
_

,
r
)%
= E
)
# H
%

&
)&
)B &
)B &
%B &
!B &
_

,
#
)B &
)B &
!B &
%B &
_

,
=
$
)&

;
%
= H
%
r
)%
E
)
=
)B &
)B &
!B &
%B &
_

,

$
)&
&
)&
)B &
)B &
%B &
!B &
_

,
=
)B -
)B -
)B -
'B -
_

,

IF Iteration9
r
%%
= NN ;
%
NN =
')
-
and E
%
=
%
%%
4
!
=
)
)
!
)
')
'
_

,

8ow E and 0 are defined b3 the procedure i/en abo/e#
+xample9 Find the E0 decomposition of A =
% % $
% % !
$ ! *
_

,
Solution9 Let F
!
=
%
%
$
_



,
, F
$
=
%
%
!
_

,
, F
)
=
$
!
*
_



,
r
!!
= NN F
!
NN = ' and E
!
=
!
!!
*
!
=
$
!
$
)
!
_



,
r
!$
= K E
!
# F
$
L = &
r
!)
= K E
!
# F
)
L =
$
)

U
$
= F
$
r
!$
E
!
=
$
!
$
)
(
_

,

U
)
= F
)
r
!)
E
!
=
!%
!
!)
@
$
_



,
II Iteration9
r
$$
= NN U
$
NN = $
$
and E
$
=
$
$$
U
!
=
!
!
!
!(
%
_

,
r
$)
= E
$
# U
)
=
!
!(

H
)
= U
)
r
$)
E
$
=
)
!
)
$
*
_



,
III Iteration9
r
))
= NN H
)
NN =
!(
$
and E
)
=
)
))
+
!
=
!
!
!
$
*
_



,
8ow E = 5 E
!
E
$
E
)
6 and 0 =
!! !$ !)
$$ $)
))
*
* *
! ! !
! !
!
_



,
Method of Least Square
Least Square Mroblem9 Ji/en an inconsistent s3stem of equations %4 3 , we want to find
a /ector
4
, so that the error
3 %4
is the smallest possible error# .he /ector
4
is called
the least
square solution#
Ji/en a s3stem of equations %4 3 , a least square solution to the s3stem denoted b3
4
,
will also be a solution to the associated normal s3stem A
.
A;=A
.
<# Further if A has linearl3
independent columns then there is a unique least square solution i/en b3
( )
!
T T
4 % % % 3

+xample !%9 Use least squares to find the equation of the line that will best approximate the
points
5),-*6, 5!,$!6, 5-,!!*6 and 5&,)&6
Solution9 He ha/e to find the /alues of x and 3 for which the equation 3 = ax 2 b passes
throuh the
i/en points# Substitutin these points, we et a s3stem of equations
)x 2 3 = -*, x 2 3 = $!, -x 23 = !!*, &x 2 3 = )&# It can be expressed as A; = <
where
) !
! !
- !
& !
%
_

,
,
x
4
y
_

,
,
-*
$!
!!*
)&
3
_

,
.he normal s3stem that we need to sol/e is
T T
% %4 % 3

) ! -*
) ! - & ! ! ) ! - & $!
! ! ! ! - ! ! ! ! ! !!*
& ! )&
x
y

1 1
1 1
_ _ _
1 1

1 1
, , ,
1 1

] ]
(% % !!)%
% % !''
x
y
_ _ _

, , ,
(% % !!)% A % % !'' x y x y +
Sol/in this, we et
!$#! $@#% a and b
4ence the line of best fit is
!$#! $@#% y x +
+xample !&9 Compute the error for the solution from the abo/e example#
Solution9 .he line of best approximate fit is
!$#! $@#% y x +
He compute the errors for each of the points b3 substitutin the i/en x /alue in this
line
and then ta:in the difference of the result from the equation and the :nown 3 /alue#
!
-* O !$#!5 )6 $@#%P %#) +
$
$! O !$#!5!6 $@#%P )#- +
)
!!* O !$#!5 -6 $@#%P %#! +
%
)& O !$#!5&6 $@#%P )#@ +
4ence the error is
$ $
! %
### '%#$ (#*!$& + +
+xample !'9 Find the least squares solution to the s3stem of equations
$ ! ! %
! & $ $
) ! % &
! ! ! !
x
y
"
1 1
1
1 1

1
1 1

1
1 1
1
1 1
]

] ]
Solution9 .he i/en s3stem is of the form A; = <# the s3stem is
T T
% %4 % 3
!& !! !- $$
!! $( !' *
!- !' $$ $!
x
y
"
_ _ _





, , ,
Sol/in the s3stem we et,
!( !&! !*-
, ,
- $!* $!*
x y "
.heorem9 Suppose that A has linearl3 independent columns# .hen the normal s3stem
associated
with A; = < can be written as
T
'4 & 3
He :now that
% &'
# Usin this in the i/en s3stem
( ) 5 6 5 6
T
T
&' &' 4 &' 3
T T T T
' & &'4 ' & 3
<ut
T
&& 8 and
T
'
is in/ertible# 4ence premultipl3 b3
( )
!
T
'

, we et
T
'4 & 3
+xample !-9 Find the least square solution to the s3stem
$ ! ! %
! & $ $
) ! % &
! ! ! !
x
y
"
1 1
1
1 1

1
1 1

1
1 1
1
1 1
]

] ]
b3
usin
E0 decomposition#
Solution9 Ji/en s3stem is of the form A; = < where
$ ! ! %
! & $ $
, ,
) ! % &
! ! ! !
x
% 4 y 3
"
1 1
1
1 1

1
1 1

1
1 1
1
1 1
]

] ]
4ere the columns of A are linearl3 independent#
<ut
% &'
where
$ - !!(
!& %%(& $*@)*
! '% !!
$*@)*
!& %%(&
) !( (!
!& %%(& $*@)*
! % !(
!& %%(& $*@)*
&
_

,
,
!! !-
!&
!& !&
$@@ &)
*
%%(& %%(&
$!*
* *
$*@)*
'
_

,
8ow the normal s3stem
T
'4 & 3 becomes
!! !- $$
!&
!& !& !&
$@@ &) $%$
*
%%(& %%(& %%(&
$!* !*-
* *
$*@)* $*@)*
x
y
"
_ 1

1
1
_
1


1


1
,
1
1
, ]
.his corresponds to the s3stem of equations
$!* !*- !*-
$!*
$*@)* $*@)*
" "
$@@ &) $%$ !&!
$!* %%(& %%(& %%(&
y " y
!! !- $$ !(
!&
- !& !& !&
x y " x +
MA.0I; .4+?0=
Uni/ersit3 Euestions
QC: 81565 SC: MA 9217 June 2012
!# Under what condition Choles:3 decomposition method is used#
$# Ji/e two applications on .oeplitz matrices#
)# Find the E0 decomposition of the matrix
$ ! )
! * -
* ! !
%
_





,
%# Mro/e if E is an nm matrix with n m then the columns of E are an orthonormal
set of /ectors in 0
n
with the standard +uclidean inner product if and onl3 if E
.
q = I
m
#
&# Find the least squares solution to the s3stem of equations
$ ! ! %
! & $ $
) ! % &
! ! ! !
x
y
"
1 1
1
1 1

1
1 1

1
1 1
1
1 1
]

] ]
'# Suppose that A has linearl3 independent columns# .hen the normal s3stem
associated with %x b can be written as,
T
'x & b
QC: 91692 SC: MA 9217 January 2012
!# State an3 two properties of sinular /alue decomposition#
$# 7efine band matrix# Ji/e an example#
)# Find the least square solution and least square error of the equation
$ ! !
! $ !
! ! !
x
y
_ _
_

,

, ,
%# In/ert the .oeplitz matrix
$ * !
* $ *
! * !
_



,
&# Find the sinular /alue decomposition of
! $
$ !
! )
_



,
QC: 33563 SC: 281108 December 2011
!# Hhat are the necessar3 conditions for Choles:3 decomposition of a matrixQ
$# State principles of least squares#
)# Find the eneralized in/erse of
* * ! $
! $ $ )
%
_

,
%# 0epresent
! $ )
$ ) %
* % &
%
_

,
in the E,0 form#
QC: 9773 SC MA 9217 December 2011
!# Hrite the different t3pes of matrix factorization#
$# Ji/en
% * $
!
* $ , * ,
$
! ! !!
% b x
1 1
1
1 1

1
1 1
]
1 1
] ]
, determine the least square error in the least
square matrix Ax=b#
)# Appl3 the E0 factorization of
! ! !
! * * ! ! ! !
5 6 , 5 6 , 5 6
! * $ ! * * !
! ! !
i % ii % iii %
1
1
1 1
1

1 1
1
] ]
1
]
%# Find a sinular /alue decomposition of
! !
$ $
$ $
%
1
1

1
1
]
QC: 31222 SC: 281108 June 2011
!# 7efine sinular /alue decomposition#
$# +xplain E0 factorization#
)# Find the Choles:3 factorization of
! % &
% $* )$
& )$ -*
_



,
%# Find the sinular /alue decomposition of A =
! !
$ $
$ $
_

,
&# Find the E0 factorization of the matrix A =
! $
! *
! $
! *
_





,
'# Find the lease square solution of A;=< for A =
! ! * *
! ! * *
! * ! *
! * ! *
! * * !
! * * !
_








,
and < =
)
!
*
$
&
!
_

,
QC: 77772 SC: MA 9217 May 2011
!# 7efine Mseudo in/erse of a full ran: matrix A#
$# 7efine .oeplitz matrix and i/e an example#
)# Compute the sinular /alues and sinular /alue decomposition of A =
! !
! !
* *
_



,
%# Find Choles:3 decomposition LL
.
for the matrix
@ )
) $
_

,
&# ?btain E0 decomposition of
* * &
* % !
! ! !
_



,
QC: 20140 SC: 281108 January 2011
!# 7efine9 .oeplitz matrix#
$# State the condition that
4
is a least square solution of A; = b and also i/e the least
square error#
)# Find the least square solution for A; = b b3 usin E0 factorization A =
$ $ '
! % )
$ % @
_

,
R b =
!
!
%
_

,
%# Find the sinular /alue decomposition for A =
! ' %
$ ' $
$ ) %
_

,
QC: 97857 SC: MA 9217 December 2010
!# List an3 two properties of pseudo in/erse of a matrix#
$# +xplain least square approximations#
)# Find the E0 factorization of A =
! ! !
! * !
! ! !
_

,
%# Find the sinular /alue decomposition of
$ ! !
$ ! !
_


,
&# Find the pseudo in/erse of
! !
* !
! *
_



,
'# Sol/e the followin equations in the lease square method9 x
!
2x
$
= !, x
!
2x
$
2x
)
2x
%
=*
QC: J7710 SC: MA 9217 June 2010
!# Find .he E0 factorization for the matrix A =
) $
% &
_

,
$# Hrite down the obDecti/e of least square method#
)# Find the Choles:3 decomposition for the matrix A =
% $ $
$ !* $
$ $ &
_

,
%# ?btain the sinular /alue decomposition of A =
$ !
$ !
% $
_

,
QC: W7669 SC: MA 9217 January 2010
!# Ji/e a positi/e definite 4ermitian matrixA, Show how the decomposition
A = U
4
7F where U is the upper trianular and 7 is diaonal follows from the
Choles:3 decomposition of A#
$# 7efine .oeplitz matrix and i/e an example#
)# ?btain the Choles:3 decomposition of A =
@ ' !$
' !) !!
!$ !! $'
_



,
and hence sol/e the
s3stem A; = b where b = O !-% $)' )*( P
.

%# ?btain the sinular /alue decomposition of A =
) *
% &
_

,
&# ?btain the E0 decomposition of
* * &
* % !
! ! !
_



,
'# ?btain the pseudo in/erse of A =
! !
! $
! )
_



,
and hence sol/e the s3stem A; = b
where ; = O x 3 P
.
and b = O * % $ P
.