L in e a r a lg e b ra e x p la in e d in fo u r p a g e s

E x c e rp t fro m th e N O B U L L S H IT G U ID E T O L IN E A R A L G E B R A b y Iv a n S a v o v
A b W e w d is c u to o ls m a th p a th s tr a ill ss b o f e m a o f u c t T h is d o c u m e n t w ill r le a r n a b o u t m a tr ic e s , m o th th e th e o r e tic a l a n d lin e a r a lg e b r a o p e n th tic s . A lo t o f k n o w le d g e n d e r s ta n d in g , in s te a d o f e v ie w th e fu n d a tr ix o p e r a tio c o m p u ta tio n a l e g a te w a y to b u z z a w a its y tr y in g to m e m a m n s, a s th o u o r e n t lin p e c e s if iz e a l e a ts tu y o a id e a s o f lin e a r a lg e b r a . r tr a n s fo r m a tio n s a n d o f lin e a r a lg e b r a . T h e d y o f m o r e a d v a n c e d u c h o o s e to fo llo w th e b u n c h o f fo r m u la s .

B . M a tr ix o p e r a tio n s T h e m a th e m a tic a l o p e ra tio n s d e n e d fo r m a tric e s a re th e fo llo w in g : a d d itio n (d e n o te d + ) C

A +

i j

i j

i j

.
n

I. I L in e a r a n a rra y a n d d e sc c o n s id e r A a lg e b ra o f n re a rib e s th e a v e c to r is th l n u ty p th a t v = m a trix A R m F o r e x a m p le , a 3 A T h e p u rp o se o o p e ra tio n s th a t fe e l o f th e p o w a n d te c h n o lo g y im p o rta n t th a t P re re q u is ite s =

n

N T R O D U C T IO N

e m m b e o h a

a th o f v e c e rs. T h e n f m a th e m a s th re e c o m

to rs a n o ta tio n tic a l o p o n e n

d m a tric e s . A v e c to r v R is S is re a d e le m e n t o f S b je c t in q u e s tio n . F o r e x a m p le , ts : R
3

s u b tra c tio n (th e in v e rs e o f a d d itio n ) m a trix p ro d u c t. T h e p ro d u c t o f m a tric e s A R m m is a n o th e r m a trix C R g iv e n b y th e f o r m u la C 2

a n d B R

= b b

A B
1 2 2 2

c 2 =
1 T

i j

= + + + a a a

( v 1 , v 2 , v 3 )

(R , R , R ) m b e rs w ith m e th is : 2 3 R R 4 R R 5 R R

a 4 a a

1 1 2 1 3 1

a a a

1 2 2 2 3 2

f th is w e c a e r o f c a n b y o u u n

is a ta b le o f n u 2 m a trix lo o k s lik 2 3 a 1 1 a 1 2 4 a 2 1 a 2 2 5 a 3 1 a 3 2 d o c u m e n t is n p e rfo rm o n lin e a r a lg e b ra e d e s c rib e d in d e rs ta n d h o w

ro w s a n d n c o lu m n s .

3 5

k = 1

b b

1 1 2 1

a 4 a a ) ):

1 1 2 1 3 1

b b b

a b b b
2 1 2 1 2 1

i k

b a a a

k j

, b b b
1 2 1 2 1 2

1 1 1 1 1 1

1 2 2 2 3 2

1 1 2 1 3 1

+ + + a a

1 2 2 2 3 2

b b b

2 2 2 2 2 2

3 5

3 2

. a th e m a tic g iv e y o u e , b u s in e s ic e s s o it a l a s, is

m a trix in v e rs e (d e n o te d A m a trix tra n s p o s e (d e n o te d

1 1 2 2

3 3

to in v e c to . M a te rm to w

tro d u c e y o u rs a n d m a tric n y p ro b le m s s o f v e c to rs o rk w ith th e s

to th e m e s a n d to in s c ie n c a n d m a tr e .

P n m a trix tra c e : T r[A ] i = 1 a i i d e te rm in a n t (d e n o te d d e t( A ) o r |A |) N o te th a t th e m a tr ix p r o d u c t is n o t a c o m m u ta tiv e o p e r a tio n : A B

2 4
1 2 3

1 2 3

5 .

B A .

C . M a tr ix -v e c to r p ro d u c t ria l is a b a s ic u n d e rs ta n d in g o f h ig h s c h o o l a ria b le s , e q u a tio n s , a n d th e fu n d a m e n ta l m b e rs : a d d itio n (d e n o te d + ), s u b tra c tio n o te d im p lic itly ) , a n d d iv is io n ( f r a c tio n s ) . w ith fu n c tio n s th a t ta k e re a l n u m b e rs a s s o u tp u ts , f : R R . R e c a ll th a t, b y f 1 u n d o e s th e e ffe c t o f f . If y o u a re d x , y o u c a n u s e th e in v e rs e fu n c tio n a s a m p le , th e fu n c tio n f ( x ) = ln ( x ) h a s th e v e rse o f g (x ) = x is g 1 ( x ) = x 2 .
E F IN IT IO N S

T h e o n ly p re re q u is ite fo r th is tu to m a th c o n c e p ts 1 lik e n u m b e rs , v a rith m e tic o p e ra tio n s o n re a l n u (d e n o te d ), m u ltip lic a tio n (d e n Y o u s h o u ld a ls o b e fa m ilia r in p u ts a n d g iv e r e a l n u m b e r s a d e n itio n , th e in v e r s e fu n c tio n g iv e n f ( x ) a n d y o u w a n t to n fo llo w s : f 1 ( f ( x ) ) = x . F o r e x in v e rs e f 1 ( x ) = e x , a n d th e in II. D A . V e c to r o p e r a tio n s

T h e m a trix -v e c to r p ro d u c t is a n im p o rta n t s p e c ia l c a s e o f th e m a trix m a trix p ro d u c t. T h e p ro d u c t o f a 3 2 m a trix A a n d th e 2 1 c o lu m n v e c to r x r e s u lts in a 3 1 v e c to r y g iv e n b y : y = A x , 3 2 3 y 1 a 1 1 a 1 2 4 y 2 5 = 4 a 2 1 a 2 2 5 y 3 a 3 1 a 3 2 2 3 a 1 1 = x 14 a 2 1 5 + x a 3 1 2 (a 1 1 , a 1 2 ) = 4 (a 2 1 , a 2 2 ) (a 3 1 , a 3 2 ) 2 x

1

x
2

4 a a 3

1 2 2 2 3 2

= 4 a 3 5 a

1 1 2 1 3 1

x
1

+
1

a + a + a

1 2 2 2 3 2

x
2

x x
1

x
2

x
2

3 5 (C )

W e n o w d e n e th e m a th o p e ra tio n s fo r v e c to rs . T h e o p e ra tio n s w e c a n p e rfo rm o n v e c to rs u = ( u 1 , u 2 , u 3 ) a n d v = ( v 1 , v 2 , v 3 ) a re : a d d itio n , s u b tra c tio n , s c a lin g , n o rm (le n g th ), d o t p ro d u c t, a n d c ro s s p ro d u c t: u + u v = v = u = ( q u

1 2

( u
1 1

+ u
1 2 1

v 1 , u
2 2

+ u
3

v 2 , u
3 3

v 3 ) v 3 )

( u

v 1 , u u
2

v 2 , u )
2 3

,
2 2 2

T h e d o t p ro d u c t a n d th e c ro s s p ro d u c t o in te rm s o f th e a n g le b e tw e e n th e tw p ro d u c t o f th e v e c to rs is u v = u v v a re o r th o g o n a l if th e a n g le b e tw e e n o rth o g o n a l v e c to rs is z e ro : u v = u T h e n o r m o f th e c r o s s p r o d u c t is g iv e c r o s s p r o d u c t is n o t c o m m u ta tiv e : u v
1

u v =

||u || =

u v
1

+ +
3

u u u

+ v
2 3

u +

u
3 3

v
3 1

T h e r e a r e tw o 2 f u n d a m e n ta lly d iff e r e n t y e t e q u iv a le n t w a y s to in te r p r e t m a trix -v e c to r p ro d u c t. In th e c o lu m n p ic tu re , (C ), th e m u ltip lic a tio n o f m a trix A b y th e v e c to r x p ro d u c e s a lin e a r c o m b in a tio n o f th e c o lu m o f t h e m a t r ix : y = A x = x 1 A [:,1 ] + x 2 A [:,2 ], w h e r e A [:,1 ] a n d A [:,2 ] th e rs t a n d s e c o n d c o lu m n s o f th e m a trix A . In th e ro w p ic tu re , (R ), m u ltip lic a tio n o f th e m a trix A b y th e v e c to p ro d u c e s a c o lu m n v e c to r w ith c o e f c ie n ts e q u a l to th e d o t p r o d u c ts r o w s o f th e m a tr ix w ith th e v e c to r x . D . L in e a r tr a n s fo r m a tio n s T h e m tr a n s fo r m a lg e b ra . c o m p u tin p ro d u c e s a trix -v a tio n , M u ltip g a lin m -v e c e c w lic e a to to r p h ic h a tio n r tr a rs a s ro d u c is o n b y a n s fo r m o u tp u T
A 2

x 5 . x

(R ) th th n a r e e s e

r x o f

v = (u

v 2 , u

f tw o v e c to rs o v e c to rs . T h c o s . W e s th e m is 9 0 v c o s (9 0 ) n b y u v = v u , in

u
1

v 3 , u
1

v
2

c a n a ls o b e fo rm u la a y tw o v e . T h e d o t = 0 . = u v fa c t u v

u
2

v 1 ) e d e s c rib e d fo r th e d o t c to rs u a n d p ro d u c t o f s in = . T h e v u . 1

t is e o f m a a tio ts :

u s e d to d e n e th e th e k e y n o tio n s in trix A R m n c a n T A th a t ta k e s n -v : R
n

n o tio th e s n b e e c to rs

n tu th a

o f d y o u g s in

a o f h t p u

lin e a r lin e a r o f a s ts a n d

g o o d te x tb o o k to (re )le a rn h ig h s c h o o l m a th is m i n i r e f e r e n c e . c o m

F o r m o re in fo s e e th e v id e o o f P ro f. S tra n g s M IT le c tu re : b i t . l y / 1 0 v m K c L

In s te a d o f th e v e c to r T A to th e c o lu m n v e Y o u c a n th e ir p ro p e

w ritin g y = T A ( x ) fo r th e lin e a r tra n s fo rm a tio x , w e s im p ly w rite y = A x . A p p ly in g th e lin e a v e c to r x c o rre s p o n d s to th e p ro d u c t o f th e m c to r x . W e s a y T A is re p re s e n te d b y th e m a trix th in k o f lin e a r tra n s fo rm a tio n s a s v e c to r fu n c tio rtie s in a n a lo g y w ith th e re g u la r fu n c tio n s y o u a R R T
B

n T A a p p lie r tra n s fo rm a a trix A a n d A . n s a n d d e sc re fa m ilia r w R

m

d to tio n th e rib e ith :

III. C

O M P U T A T IO N A L L IN E A R

A L G E B R A

O k a y , I h e a r w h a t y o u a re s a y in g D u d e , e n o u g h w ith th e th e o ry ta lk , le ts s e e s o m e c a l c u l a t i o n s . I n t h i s s e c t i o n w e l l l o o k a t o n e o f t h e f u n d a m e n t a l a lg o rith m s o f lin e a r a lg e b ra c a lle d G a u s s J o rd a n e lim in a tio n . A . S o lv in g s y s te m s o f e q u a tio n s S u p p o s e w e re a s k e d to s o lv e th e fo llo w in g s y s te m 1 x

1 1

fu n c tio n f : R in p u t x g

lin e a r tra n s fo rm a tio n T

A

: R R
m

o f e q u a tio n s : (1 )

in p u t x R
A

+ +

2 x
2 2

= =

5 , 2 1 .

o u tp u t f ( x ) f = g (f (x ))
1

o u tp u t T (T
A

(x ) =

A x B A x
1

3 x

9 x

(x )) =

fu n c tio n in v e rs e f

m a trix in v e rs e A N (A ) C (A )

z e ro s o f f ra n g e o f f

n u ll s p a c e o f A c o lu m n s p a c e o f A = ra n g e o f T
A

N o te th a t th e c o m b in e d e ffe c t o f a p p ly in g th e tra n s fo rm a tio n T A fo llo w e d b y T B o n th e in p u t v e c to r x is e q u iv a le n t to th e m a tr ix p r o d u c t B A x . E . F u n d a m e n ta l v e c to r s p a c e s A v e c to r s p a c e c o n s is ts o f a s e t o f th e s e v e c to rs . F o r e x a m p le th e v e c to a ll v e c to rs o f th e fo rm v = v 1 + W e n o w d e n e th re e fu n d a m e n ta l v e c T h e c o lu m n s p a c e o f a m a trix A is a s lin e a r c o m b in a tio n s o f th e c o lu m n C (A ) { y R

m

v e c r sp v 2 , to r th e s o

to rs a c e w h e sp a c se t o f th e

a n d a ll lin e a r S = sp a n { v 1 re a n d a e s a s s o c ia te d f v e c to rs th a t m a trix A : R
n

c o m , v 2 } re re w ith c a n } .

b in a tio n s c o n s is ts a l n u m b e a m a trix b e p ro d u c

o f o f rs. A . e d

W ith o u t a k n o w le d g e o f lin e a r a lg e b ra , w e c o u ld tio n , o r s u b tra c tio n to n d th e v a lu e s o f th e tw o G a u s s J o rd a n e lim in a tio n is a s y s te m a tic p ro c o f e q u a tio n s b a s e d th e fo llo w in g ro w o p e r a tio n s ) A d d in g a m u ltip le o f o n e ro w to a n o th e r ro ) S w a p p in g tw o ro w s ) M u ltip ly in g a ro w b y a c o n s ta n t T h e s e ro w o p e ra tio n s a llo w u s to s im p lify th e s y c h a n g in g th e ir s o lu tio n . T o illu s tra te th e G a u s s J o rd a n e lim in a tio n p ro c s e q u e n c e o f ro w o p e ra tio n s re q u ire d to s o lv e th e d e s c rib e d a b o v e . W e s ta rt b y c o n s tru c tin g a n a u g 1 2 5 . 3 9 2 1

u s e s u b s titu tio n , e lim in a u n k n o w n s x 1 a n d x 2 . e d u re fo r s o lv in g s y s te m s : w

s te m

o f e q u a tio n s w ith o u t

e d u re , w e ll n o w s h o w th e s y s te m o f lin e a r e q u a tio n s m e n te d m a tr ix a s fo llo w s :

| y =

A x fo r so m e x

T h e c o lu m n s p a c e is th e r a n g e o f th o f p o s s ib le o u tp u ts ). Y o u c a n c o n v in c th e d e n itio n o f th e m a trix -v e c to r p ro v e c to r A x c o n ta in s x 1 tim e s th e 1 st c o o f A , e tc . V a ry in g o v e r a ll p o s s ib le in c o m b in a tio n s o f th e c o lu m n s o f A , h e T h e n u ll s p a c e N ( A ) o f a m a trix A th a t th e m a trix A s e n d s to th e z e ro v e N (A ) x R T h e W e o n ly T c o m c o m v n u ll R

n

v e c to rs in th e n u ll s p a c e a re o r th o g o n a l to a ll th e ro w s o f th e m a trix . c a n s e e th is fro m th e ro w p ic tu re (R ): th e o u tp u t v e c to rs is 0 if a n d if th e in p u t v e c to r x is o rth o g o n a l to a ll th e ro w s o f A . h e ro w s p a c e o f a m a trix A , d e n o te d R ( A ) , is th e s e t o f lin e a r b in a tio n s o f th e ro w s o f A . T h e ro w s p a c e R ( A ) is th e o rth o g o n a l p le m e n t o f th e n u ll s p a c e N ( A ) . T h is m e a n s th a t fo r a ll v e c to rs R ( A ) a n d a ll v e c to rs w N ( A ) , w e h a v e v w = 0 . T o g e th e r, th e s p a c e a n d th e ro w s p a c e fo rm th e d o m a in o f th e tra n s fo rm a tio n T A , = N (A ) R (A ), w h e re s ta n d s fo r o r th o g o n a l d ire c t s u m .

e lin e a r tra n s fo rm a e y o u rs e lf o f th is f d u c t in th e c o lu m n lu m n o f A , x 2 tim e p u ts x , w e o b ta in a n c e th e n a m e c o lu R m n c o n s is ts c to r: n | A x = 0 .

tio n T A (th e s e t a c t b y re v ie w in g p ic tu re (C ). T h e s th e 2 nd c o lu m n ll p o s s ib le lin e a r m n s p a c e . o f a ll th e v e c to rs

T h e rs t c o lu m n in th e a u g m e n te d m a trix c o rre s p o n d s to th e c o e f c ie n ts o f th e v a ria b le x 1 , th e s e c o n d c o lu m n c o rre s p o n d s to th e c o e f c ie n ts o f x 2 , a n d th e th ird c o lu m n c o n ta in s th e c o n s ta n ts fro m th e rig h t-h a n d s id e . T h e G a u s s -J o rd a n e lim in a tio n p ro c e d u re c o n s is ts o f tw o p h a s e s . D u rin g th e rs t p h a s e , w e p ro c e e d le ft-to -rig h t b y c h o o s in g a ro w w ith a le a d in g o n e in th e le ftm o s t c o lu m n (c a lle d a p iv o t) a n d s y s te m a tic a lly s u b tra c tin g th a t ro w fro m a ll ro w s b e lo w it to g e t z e ro s b e lo w in th e e n tire c o lu m n . In th e s e c o n d p h a s e , w e s ta r t w ith th e r ig h tm o s t p iv o t a n d u s e it to e lim in a te a ll th e n u m b e rs a b o v e it in th e s a m e c o lu m n . L e ts s e e th is in a c tio n . 1 ) T h e r s t s te p is to u s e th e p iv o t in th e r s t c o lu m n to e lim in a te th e v a ria b le x 1 in th e s e c o n d ro w . W e d o th is b y s u b tra c tin g th re e tim e s th e rs t ro w fro m th e s e c o n d ro w , d e n o te d R 2 R 2 3 R 1 , 1 2 5 . 0 3 6 2 ) N e x t, w e c r e a te a p iv o t in th e s e c o n d r o w u s in g R 1 2 5 . 0 1 2 3 ) W e n o w s ta rt th e b a c k w a rd p h a fro m th e rs t ro w . W e d o th is ro w fro m th e rs t ro w R 1 R 1 0
2 3 1

R
2

F . M a tr ix in v e r s e B y d e n itio n , th e in v e rs e m a trix A T h e c u m u la tiv e e ff e c t o f a p p ly in g A A

1 1 1

T h e id e n tity m a trix (o n e s o n th e d ia g o n a l a n d z e ro s e v e ry w h e re e ls e ) c o rre s p o n d s to th e id e n tity tra n s fo rm a tio n : T ( x ) = x = x , fo r a ll x . T h e m a trix in v e rs e is u s e fu l fo r s o lv in g m a trix e q u a tio n s . W h e n e v e r w e w a n t to g e t rid o f th e m a trix A in s o m e m a trix e q u a tio n , w e c a n h it A w ith its in v e rs e A 1 to m a k e it d is a p p e a r. F o r e x a m p le , to s o lv e fo r th e m a trix X in th e e q u a tio n X A = B , m u ltip ly b o th s id e s o f th e e q u a tio n b y A 1 fro m th e rig h t: X = B A 1 . T o s o lv e fo r X in A B C X D = E , m u ltip ly b o th s id e s o f th e e q u a tio n b y D 1 o n th e rig h t a n d b y A 1 , B 1 a n d C 1 (in th a t o rd e r) fro m th e le ft: X = C 1 B 1 A 1 E D 1 .

u n d o e s th e e ffe c ts o f th e m a trix A . a fte r A is th e id e n tity m a trix : 2 3 1 0 .. 4 5 . . 0 1

T h e m a trix is n o w in re d u c e d ro w e c h e lo n fo r m (R R E F ), w h ic h is its s im p le s t fo rm it c o u ld b e in . T h e s o lu tio n s a re : x 1 = 1 , x 2 = 2 .

s e a n d e lim in a te th e s e c o n d v a ria b le b y s u b tra c tin g tw o tim e s th e s e c o n d 2 R 2 : 1 0 1 . 1 2

B . S y s te m s o f e q u a tio n s a s m a tr ix e q u a tio n s W e w ill n o w d is c u s s a n o th e e q u a tio n s . U s in g th e d e n itio n o f th is s y s te m o f e q u a tio n s (1 ) a s a 1 2 3 9 T h is m a trix e q u a tio n h a d th is th e v e c to r o f u n k n o w n s , a x b y m u ltip ly in g b o th s id e s x 1 A 1 A x = x = x 2 B u t h o w d id w e k n o w r a p p ro a c h fo r s o lv in g th e s y s te m o f th e m a trix -v e c to r p ro d u c t, w e c a n e x p re s s m a trix e q u a tio n : x 1 5 = . x 2 2 1 = b , w h e re A e c to r o f c o n s ta a tio n b y th e m 3 23 = 1 1 3
1

w h a t th e in v e rs e m a trix A

e fo rm A x n d b is a v o f th e e q u = A 1 b

is ?

is a 2 2 n ts . W e c a n a trix in v e rs 5 1 = 2 1 2

m a trix , x s o lv e fo r e A 1 : .

IV . In th is s e c tio n th e in v e rs e o f a m m e th o d w e u s e to A . U s in g O n e a p e lim in a tio o f th e m a ro w o p ro a c n p ro trix A h

O M P U T IN G

T H E IN V E R S E O F A

M A T R IX

w e ll lo o k a t s e v e ra l d iffe re n t a p p ro a c h e s fo r c o m p u tin g a trix . T h e m a trix in v e rs e is u n iq u e s o n o m a tte r w h ic h n d th e in v e rs e , w e ll a lw a y s o b ta in th e s a m e a n s w e r. tio n r c o re . th e s m p u tin g th e in v e rs e is to u s e th e G a u s s J o rd a n S ta rt b y c re a tin g a n a rra y c o n ta in in g th e e n trie s le ft s id e a n d th e id e n tity m a trix o n th e rig h t s id e : 1 2 1 0 . 3 9 0 1 p ro c e d u re o n th is a rra y . e tim e s th e rs t ro w fro m ta in : . b y 3 : R
2 3 1

C . U s in g a c o m p u te r T h e la s t (a n d m o s t p ra c tic a l) a p p ro a c h fo r n d in g th e in v e rs e o f a m a trix is to u s e a c o m p u te r a lg e b ra s y s te m lik e th e o n e a t l i v e . s y m p y . o r g .

> > > A = [ 1 , [ 3 , [ M a t r i x ( 2 ] 9 ] [ [ 1 , 2 ] , [ 3 , 9 ] ] ) # d e f i n e A

p e ra fo c e d u o n

> > >

A . i n v ( ) 3 , - 2 / 3 ] [ - 1 , 1 / 3 ]

c a l l s

t h e

i n v

m e t h o d

o n

Y o u c a n u s e s y m p y to c h e c k y o u r a n s w e rs o n h o m e w o rk p ro b le m s . V . O W e ll n o w A . B a s is I n tu itiv e ly , a b a s is fo r a v e c to r s p a c e . th e x y -p la n e th a t is y -a x is . A v e c to r v re s p e c t to th e s e a x e a n d (0 , 1 ) a re r e s p e c tiv e ly . H o w e v is s e t o f Y o u a re m a d e u c a n b e s, o r e q u u n it v e e r, o th e r v e c to rs th a t c a n b e u s e d c e rta in ly fa m ilia r w ith p o f tw o o rth o g o n a l a x d e s c rib e d a s a c o o rd in i v a l e n t l y a s v = v x + c to rs th a t p o in t a lo n g c o o rd in a te s y s te m s a re a s a c o o rd in a te th e s ta n d a rd b e s : th e x -a x is a te p a ir ( v x , v v y , w h e r e th e x -a x is a n d a ls o p o s s ib le . s y s te m a s is fo r a n d th e y ) w i t h (1 , 0 ) y -a x is

T H E R T O P IC S

N o w w e p e rfo rm th e G a u s s -J o rd a n e lim in a tio n 1 ) T h e rs t ro w o p e ra tio n is to s u b tra c t th re th e s e c o n d ro w : R 2 R 2 3 R 1 . W e o b 1 2 1 0 0 3 3 1 2 ) T h e se c o n d ro w

d is c u s s a n u m b e r o f o th e r im p o rta n t to p ic s o f lin e a r a lg e b ra .

3 ) T h e th ird ro w

T h e a rra y is n o w in re d u c e d ro w e c h e lo n fo rm (R R E F ). T h e in v e rs e m a trix a p p e a rs o n th e rig h t s id e o f th e a rra y . O b s e rv e th a t th e s e q u e n c e o f ro w o p e ra tio n s w e u s e d to s o lv e th e s p e c i c s y s te m o f e q u a tio n s in A x = b in th e p re v io u s s e c tio n a re th e s a m e a s th e ro w o p e ra tio n s w e u s e d in th is s e c tio n to n d th e in v e rs e m a trix . In d e e d , in b o th c a s e s th e c o m b in e d e ffe c t o f th e th re e ro w o p e ra tio n s is to u n d o th e e ffe c ts o f A . T h e rig h t s id e o f th e 2 4 a rra y is s im p ly a c o n v e n ie n t w a y to re c o rd th is s e q u e n c e o f o p e ra tio n s a n d th u s o b ta in A 1 . B . U s E v e m u ltip m a tric in g ry lic e s e le m ro w a tio n in c o R R R L e ts re v is re p re s e n tin 1 ) T h e c a tio E
1

o p e ra tio n is R 1 1 0 0 1

o p e r a tio n is d iv id e th e s e c o n d r o w 1 2 1 0 . 1 0 1 1 3 R
1

R
2

3 1

2 R
1 3

2 2 3

D e n itio n (B a s is ). A b a s is fo r a n -d im e n s io n a l v e c to r s p a c e S o f n lin e a r ly in d e p e n d e n t v e c to r s th a t a re p a r t o f S . A n y s e t o f t w o l i n e a r l y i n d e p e n d e n t v e c t o r s { e 1 , e 2 } c a n fo r R 2 . W e c a n w rite a n y v e c to r v R 2 a s a lin e a r c o m b b a s i s v e c t o r s v = v 1 e 1 + v 2 e 2 . N o te th e s a m e v e c to r v c o rre s p o n d s to d iffe re n t c o o rd in a in g o n th e b a s is u s e d : v = ( v x , v y ) in th e s ta n d a rd b a s is B v = ( v 1 , v 2 ) in th e b a s is B e { e 1 , e 2 } . T h e r e f o r e , i t i s i m in m in d th e b a s is w ith re s p e c t to w h ic h th e c o e f c ie n ts a n e c e s s a r y s p e c if y th e b a s is a s a s u b s c r ip t, e .g ., ( v x , v y ) B s C o n v e rtin g a c o o rd in a te v e c to r fro m th e b a s is B e to p e rfo rm e d a s a m u ltip lic a tio n b y a c h a n g e o f b a s is m a trix v x e 1 v = v = v e 1 y B B B B
s s e e

is a n y s e t

s e rv e a s a b a s is in a tio n o f th e s e

e n o p b y rre

ta r y m a e ra tio n a n e le m sp o n d e n : R
1

tr ic e s w e p e rfo rm o n a m a trix is e n ta r y m a tr ix . T h e re a re th re c e w ith th e th re e ty p e s o f ro R 1 + m R 2 E = R

2

: R
1

E
1

= E =

: R
1

m R

it g n A

th e e a c rst r b y =

2 ) T h e se c o n d ro w E
2

ro w h ro o w th e

w o e 1

3 ) T h e n a l s te p , R

o p e ra tio n R 2 R 2 c o rre s p o n d s to a m a trix E 3 1 0 1 2 1 2 (E 1 A ) = = . 1 0 0 3 0 1 3

1 2 1

o p e ra tio n s o p e ra tio n p e ra tio n R le m e n ta ry 0 1 1 3

w e u s a s a n R 2 m a trix 2 = 9

e d to e le m 2 E 1 : 1 0

n d A 1 in th e a b o v e s e c tio n e n ta ry m a trix m u ltip lic a tio n . 3 R 1 c o rre s p o n d s to a m u ltip li2 3 . :

e q u iv a le n t to a le f te ty p e s o f e le m e n ta ry w o p e ra tio n s : 1 m 0 1 0 1 1 0 m 0 0 1

N o te th e v e c to r v c o o rd in a is a c c o m

c h a n g e re m a in s te s y s te m p lis h e d p la R n n sf f th R

o f b u n c . T u s in

a s is m a trix h a n g e d it h e c h a n g e o g th e in v e rs

is a c tu a lly a n id e n tity tra n s fo rm a tio n . T h e is s im p ly e x p re s s e d w ith re s p e c t to a n e w f b a s is fro m th e B s -b a s is to th e B e -b a s is e m a trix : B e [ ]B s = ( B s [ ]B e ) 1 . th e rib e c ie n e in p ta tio re p re s e n ta th e m a trix t to k n o w u t sp a c e . F n c o rre sp o 3 | T ( )5 R | tio n o f lin e th a t c o rre s th e e ffe c ts o r a lin e a r n d s to
2 2

te p a irs d e p e n d { , } , a n d s p o rta n t to k e e p re ta k e n , a n d if o r (v 1 , v 2 )B e . th e b a s is B s is : e 2 v 1 . e 2 v 2

B . M a tr ix re p re s e n ta tio n s o f lin e a r tr a n s fo r m a tio n s B a se s tio n s T : lin e a r tra v e c to rs o T : R 2 y a n im p o rta n t ro le R m . T o fu lly d e o rm a tio n T , it is s u e s ta n d a rd b a s is fo r 2 , th e m a trix re p re s 2 M
T

in sc f th e n

a r p o o f tra

tra n s f n d s to T to n sfo rm

o rm a so m e th e n a tio n

1 2

E
3

(E

E
1

N o te th a t E
3

E
2

E
1 3

A
2

= E
1

T h e e le m e n ta ry m a trix a p p ro a c h te a c h e s u s th a t e v e ry in v e rtib le m a trix c a n b e d e c o m p o s e d a s th e p ro d u c t o f e le m e n ta ry m a tric e s . S in c e w e k n o w A 1 = E 3 E 2 E 1 t h e n A = ( A 1 ) 1 = ( E 3 E 2 E 1 ) 1 = E 1 1 E 2 1 E 3 1 .

, s o th e p 1 2 = 0 1

, c o rre s p o n d s to th e m a trix E 3 : 1 2 1 2 1 0 A ) = = . 0 1 0 1 0 1 2 R
2

A s a rs t e x a m p le , c o n s id e r th e tra n s fo rm a tio n x w h i c h p r o j e c t s v e c t o r s o n to th e x -a x is . F o r a n y v e c to r v = ( v x , v y ) , w e h a v e (v x , 0 ). x ( v ) = T h e m a trix re p re s e n ta tio n o f x is 1 0 1 0 M = = . x x x 0 1 0 0 : A s a s e c o n d e x a m p le , c o u n te rc lo c k w is e ro ta tio M R = R le ts n d n b y th e a 1 R 0 th e m a trix re p re s e n ta tio n o f R n g le : 0 c o s s in = . 1 s in c o s , th e

| 4 T ( ) |

ro d u c t E 3 E 2 E 1 m u s t b e e q u a l to A 1 0 1 0 3 23 = . 1 1 0 3 1 1 3 3

T h e rs t c o lu m n o f M R s h o w s th a t R m a p s t h e v e c t o r 1 0 to th e v e c to r 1 = ( c o s , s in ) T . T h e s e c o n d c o lu m n s h o w s th a t R m a p s th e v e c to r = 1 2 to th e v e c to r 1 ( 2 + ) = ( s in , c o s ) T .

C . D im e n s io n a n d b a s e s fo r v e c to r s p a c e s T h e d im e n s io n o f a v e c to r s p a c e is d e n e d a s th e n u m b e r o f v e c to rs a b a s is fo r th a t v e c to r s p a c e . C o n s id e r th e fo llo w in g v e c to r s p a c e S = s p a n { ( 1 , 0 , 0 ) , ( 0 , 1 , 0 ) , ( 1 , 1 , 0 ) } . S e e in g th a t th e s p a c e is d e s c rib e d b y th re e v e c to rs , w e m ig h t th in k th a t S is 3 -d im e n s io n a l. T h is is n o t th e c a s e , h o w e v e r, s in c e th e th re e v e c to rs a re n o t lin e a rly in d e p e n d e n t s o th e y d o n t fo rm a b a s is fo r S . T w o v e c to rs a re s u f c ie n t to d e s c rib e a n y v e c to r in S ; w e c a n w rite S = s p a n { ( 1 , 0 , 0 ) , ( 0 , 1 , 0 ) } , a n d w e s e e th e s e tw o v e c to rs a re lin e a rly in d e p e n d e n t s o th e y fo rm a b a s is a n d d im ( S ) = 2 . T h e re is a g e n e ra l p ro c e d u re fo r n d in g a b a s is fo r a v e c to r s p a c e . S u p p o s e y o u a r e g iv e n a d e s c r ip tio n o f a v e c to r s p a c e in te r m s o f m v e c to r s V = s p a n { v 1 , v 2 , . . . , v m } a n d y o u a re a s k e d to n d a b a s is fo r V a n d th e d im e n s io n o f V . T o n d a b a s is fo r V , y o u m u s t n d a s e t o f lin e a rly in d e p e n d e n t v e c to rs th a t s p a n V . W e c a n u s e th e G a u s s J o rd a n e lim in a tio n p ro c e d u re to a c c o m p lis h th is ta s k . W rite th e v e c to rs v i a s th e ro w s o f a m a trix M . T h e v e c to r s p a c e V c o rre s p o n d s to th e ro w s p a c e o f th e m a trix M . N e x t, u s e ro w o p e ra tio n s to n d th e re d u c e d ro w e c h e lo n fo rm (R R E F ) o f th e m a trix M . S in c e ro w o p e ra tio n s d o n o t c h a n g e th e ro w s p a c e o f th e m a trix , th e ro w s p a c e o f re d u c e d ro w e c h e lo n fo rm o f th e m a trix M is th e s a m e a s th e ro w s p a c e o f th e o rig in a l s e t o f v e c to rs . T h e n o n z e ro ro w s in th e R R E F o f th e m a trix fo r a b a s is fo r v e c to r s p a c e V a n d th e n u m b e rs o f n o n z e ro ro w s is th e d im e n s io n o f V . in D . R o w S e sp c e o f e w p m s p a c e , c o lu m n s s p a c e , a n d r a n k o f a m a tr ix

E . In v e r tib le m a tr ix th e o re m T h e re is a n im p o rta n t d is tin c tio n b e tw e e n m a tric e s th a t a re in v e rtib le a n d th o s e th a t a re n o t a s fo rm a liz e d b y th e fo llo w in g th e o re m . T h e o 1 ) 2 ) 3 ) 4 ) 5 ) 6 ) 7 ) r e A T T T T A T m . F o r a n n is in v e r tib le h e R R E F o f A h e r a n k o f th e h e ro w sp a c e h e c o lu m n s p a d o e sn t h a v e h e d e te r m in a n n m a tr ix A , th e fo llo w in g s ta te m e n ts a re e q u iv a le n t: is th e m a tr ix o f A is c e o f A a n u ll t o f A n is R n is sp a is n n R
n

n id e n tity m a tr ix

c e (o n ly th e z e ro v e c to r N ( A ) = o n z e ro d e t( A ) = 0

{ 0 } )

F o r a g iv e n m a tr ix A , th e a b o v e s ta te m e n ts a r e e ith e r a ll tr u e o r a ll fa ls e . A n in v e rtib le m a trix A c o rre s p o n d s to a lin e a r tra n s fo rm a tio n T A w h ic h m a p s th e n -d im e n s io n a l in p u t v e c to r s p a c e R n to th e n -d im e n s io n a l o u tp u t v e c t o r s p a c e R n s u c h t h a t t h e r e e x i s t s a n i n v e r s e t r a n s f o r m a t i o n T A 1 t h a t c a n fa ith fu lly u n d o th e e ffe c ts o f T A . O n th e o th e r h a n d , a n n n m a trix B th a t is n o t in v e rtib le m a p s th e in p u t v e c to r s p a c e R n to a s u b s p a c e C ( B ) R n a n d h a s a n o n e m p ty n u ll s p a c e . O n c e T B s e n d s a v e c to r w N ( B ) to th e z e ro v e c to r, th e re is n o T B 1 t h a t c a n u n d o t h i s o p e r a t i o n . F . D T c o m o r n e te r m in h e d e te r b in e th e o t. T h e a 1 1 a 2 1 a a a
1 2 2 2 3 2

T h e r e d u c e d r o w e c h e lo n f o r m o f th e m a tr ix A c o n ta in s th r e e p iv o ts . T h e lo c a tio n s o f th e p iv o ts w ill p la y a n im p o r ta n t r o le in th e f o llo w in g s te p s . T h e v e c to rs { ( 1 , 3 , 0 , 0 ) , ( 0 , 0 , 1 , 0 ) , ( 0 , 0 , 0 , 1 ) } fo rm a b a s is fo r R ( A ) . T o n d a b a s is fo r th e c o lu m n s p a c e C ( A ) o f th e m a trix A w e n e e d to n d w h ic h o f th e c o lu m n s o f A a re lin e a rly in d e p e n d e n t. W e c a n d o th is b y id e n tify in g th e c o lu m n s w h ic h c o n ta in th e le a d in g o n e s in rre f( A ) . T h e c o rre s p o n d in g c o lu m n s in th e o rig in a l m a trix fo rm a b a s is fo r th e c o lu m n s p a c e o f A . L o o k in g a t rre f( A ) w e s e e th e rs t, th ird , a n d fo u rth c o lu m n s o f th e m a trix a re lin e a rly in d e p e n d e n t s o th e v e c to rs { ( 1 , 2 , 3 ) T , ( 3 , 7 , 9 ) T , ( 3 , 6 , 1 0 ) T } fo rm a b a s is fo r C ( A ) . N o w le ts n d a b a s is fo r th e n u ll s p a c e , N ( A ) { x R 4 | A x = 0 } . T h e s e c o n d c o lu m n d o e s n o t c o n ta in a p iv o t, th e r e f o r e it c o r r e s p o n d s to a fre e v a r ia b le , w h ic h w e w ill d e n o te s . W e a re lo o k in g fo r a v e c to r w ith th re e u n k n o w n s a n d o n e fre e v a ria b le ( x 1 , s , x 3 , x 4 ) T th a t o b e y s th e c o n d itio n s :
2 1 4 0 0 0 0 3 1 0 1 0 0 5 64 x x 0 3 2 6 x
1 3 4

R e c a ll th e fu n d a m e n ta l v e c to r s p a c e s fo r m a tric e s th a t w e d e n e d in c tio n II-E : th e c o lu m n s p a c e C ( A ) , th e n u ll s p a c e N ( A ) , a n d th e ro w a c e R ( A ) . A s ta n d a r d lin e a r a lg e b r a e x a m q u e s tio n is to g iv e y o u a rta in m a trix A a n d a s k y o u to n d th e d im e n s io n a n d a b a s is fo r e a c h its fu n d a m e n ta l s p a c e s . In th e p re v io u s s e c tio n w e d e s c rib e d a p ro c e d u re b a s e d o n G a u s s J o rd a n lim in a tio n w h ic h c a n b e u s e d d is till a s e t o f lin e a rly in d e p e n d e n t v e c to rs h ic h fo rm a b a s is fo r th e ro w s p a c e R ( A ) . W e w ill n o w illu s tra te th is ro c e d u re w ith a n e x a m p le , a n d a ls o s h o w h o w to u s e th e R R E F o f th e a trix A to n d b a s e s fo r C ( A ) a n d N ( A ) . C o n s id e r th e fo llo w in g m a trix a n d its re d u c e d ro w e c h e lo n fo rm : 2 3 2 3 1 3 3 3 1 3 0 0 6 7 6 5 0 1 0 5 . A = 4 2 rre f(A ) = 4 0 3 9 9 1 0 0 0 0 1

If th e |A | =

a a a

1 1 2 1 3 1

a n ts m in a n t o f a m a trix e n trie s o f a m a trix d e te rm in a n t fo rm u a 1 2 = a 1 1 a 2 2 a 2 2 a 1 3 a 2 2 a 2 3 = a 1 1 a 3 2 a 3 3 0 th e n A

, d e n o te d d e t ( A ) o r |A |, is a s p e c ia l w a y to th a t s e rv e s to c h e c k if a m a trix is in v e rtib le la s fo r 2 2 a n d 3 3 m a tric e s a re a a a

1 2

2 1

, a

a n d a a
2 1 3 1

2 3 3 3

is n o t in v e rtib le . If |A | =

1 2

a a

2 3 3 3

0 th e n A se s th h t o f in im p le e o u tp e c o n s

1 3

is in v e rtib le . p u t v e c to s c a lin g . u t is th e ta n t is

a a

2 1 3 1

a a

2 2 3 2

. r n e d

G . E ig e n v a lu e s a n d e ig e n v e c to r s T h e s e t o f e ig e n v e c to rs o f a m a trix is a s p e c ia l w h ic h th e a c tio n o f th e m a trix is d e s c rib e d a s a a m a trix is m u ltip lie d b y o n e o f its e ig e n v e c to rs e ig e n v e c to r m u ltip lie d b y a c o n s ta n t A e = e . T a n e ig e n v a lu e o f A . T o n d th e e ig e n v a lu e o f a m a trix w e s ta rt fro m A e = e , in s e rt th e id e n tity , a n d re w rite it a s A e = e s o lu tio n w h 1 , 2 , . . . , |. T to rs in th e e w ritte n e n s id e r th e g o n a l, a n d m n s: 2 6 Q
= 6 6 6 6 6 6 4

rs fo W h e sa m c a lle

th e e ig e n v a lu e e q u a tio n a n u ll-s p a c e p ro b le m : ) e = 0 .

T h is e q u a tio n w ill h a v e a o f A R n n , d e n o te d { p o ly n o m ia l p ( ) = |A e ig e n v a lu e i a re th e v e c C e rta in m a tric e s c a n b a n d th e ir e ig e n v a lu e s . C o th e m a trix A o n th e d ia e ig e n v e c to rs o f A a s c o lu =

6 6 6 6 4 2 6 6 6

e n e v e r |A n } , a r e t h e h e e ig e n v e c n u ll s p a c e o n tire ly in te m a trix th th e m a trix | e |

n 7 7 5 7 7 7 7 3 7 ,

(A

| = 0 . T h e e ig e n v a lu e s ro o ts o f th e c h a r a c te r is tic to r s a s s o c ia te d w ith th e f th e m a trix ( A i ) . rm s o f th e ir e ig e n v e c to rs a t h a s th e e ig e n v a lu e s o f Q c o n s tru c te d fro m th e

.. . 0

.. . 0

0 0
n
7 7 5 7 7 7 7

3 7 ,

s 77 5 =

0 4 0 5 0

1 x
1

+ 1 x 1 x

3 s
3 4

= = = 0 0

L e ts e x p re s s th e u W e im m e d ia te ly s e T h e re fo re , a n y v e c n u ll s p a c e o f A . W O b s e rv e th a t th e r a n k o f th e m a trix w h ic h is th e d im e n

n k e to e

o w n s x a t x 3 = o f th e rite N d im ( C ( A A . A ls o , s io n o f th e

n th r w

, x 3 , 0 a n d fo rm ( (A ) = )) = d d im ( R in p u t
1

a n d x 4 in te rm s o f th e fre e v a x 4 = 0 , a n d w e c a n w rite x 1 3 s , s , 0 , 0 ), fo r a n y s R , sp a n { ( 3 , 1 , 0 , 0 )T } . im ( R ( A ) ) = 3 , th is is k n o w ( A ) ) + d im ( N ( A ) ) = 3 + s p a c e o f th e lin e a r tra n s fo rm a

ria b le s . = 3 s . is in th e n a s th e 1 = 4 , tio n T A .

M a tric e s th a t c a n b e T h e d e c o m p o s itio g iv e s v a lu a b le in s ig h P a g e R a n k a lg o rith fo rm a liz e d a s a n e ig

w ritte n n o f a ts in to m fo r e n v e c to

th is w m a trix th e p ro ra n k in r c a lc u

th e n A

a y a re c a lle d d ia g o n in to its e ig e n v a lu e p e rtie s o f th e m a trix g w e b p a g e s b y im la tio n o n th e m a trix

P L U G

a liz a b s a n d . G o o p o rta o f w e

le . e ig e n v e c to rs g le s o rig in a l n c e c a n b e b h y p e rlin k s .

V I. T

E X T B O O K

If y o u re in te re s te d in le a rn in g m o re a b o u t lin e a r a lg e b ra , y o u c a n c h e c k o u t m y n e w b o o k , th e N O B U L L S H IT G U ID E T O L IN E A R A L G E B R A . S ig n u p to b e n o ti e d w h e n th e b o o k is re le a s e d : b i t . l y / 1 8 x P b s 9