Matrix Inversion Lemma

Lecture #14 EEE 574 Dr. Dan Tylavsky

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Sometimes we have a solution for Ax=b, and we want a solution for Ax=b, where the difference between A and A is a low rank change. Two approaches:
Partial Matrix refactorization
Find the factor path of the nodes that have changed. Refactorize all rows in the factor path Used when A will be used many times, or rank is relatively large.

Matrix Inversions Lemma

The following Lemma can be proven: Lemma: Given conformable matrices P & Q:

( I  PQ ) 1 P ! P ( I  QP ) 1

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Derive the matrix inversion lemma:

( I  P ) 1 ( I  P ) ! I Provided (I+P) is non-singular ( I  P)

1

! I  ( I  P) P

1

Consider (A+UBV), where: Nonsingular ANXN, original matrix for which we have: A=LU, A-1b UBVNXN, UNXQ, BQXQ, VQXN, Want: (A+UBV)-1b

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

( A  UBV ) ! A( I  A 1UBV ) Provided A is invertible

( A  UBV )

1

! ( I  A UBV ) A

1

1

1

! A 1 ( I  UBVA 1 ) 1 By Lemma 1:

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Setting: P ! A 1UBV ( I  P ) 1 ! I  ( I  P ) 1 P & Substituting into: 1 1 1 1 (  ) ! (I  )

! I  (I  ! ! ! *
1 1 1

1 1

) (I  (
1

1

1 1 1 1

A
)
1

1 1 1 1

 (I   
1 1

) 

1

1

on sin gular 1 1 Rank inverse needed is: ( BQvQ  VQv N AN v N U QvQ ) QvQ

(Never calculate large inverses explicitly.)

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

( A  UBV ) 1 b ! A 1b  A 1U ( B 1  VA 1U ) 1VA 1b
Mat. Vect. SubsM ult. F/B Subs Mat. Vect. Mult. F/B Subs

Original Solution

F/B Subs OR Inverse Calc w/ Mat./Vect Mult..

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Example: Show the operations needed in finding E for YE=I if the LU factors of the Y matrix for the network in Fig. 1 are available and the E values of the modified network of Fig. 2 are desired.
1 2 Fig 1 3 4 1 2 Fig 2 3 g 4

x x Y !

x x x x x x

x x

x x Y !

x xg x g x x x

g x x  g

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Assume we have:
x x Y ! LU ! E ! U 1 L1 I x x x x 1 x 1 x 1 x 1 x

We want to find the solution when the Y matrix is:

Y ! (Y  UBV )

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Consider nonsymmetric change. Use following matrices:
g1 B! , g1 ! g 2 ! g g2

x x !

x x  g1 x x x x

 g1

 g2 x x  g2

0 0 1  1 Creates row U! 0 0 2 of UBV. Creates row  1 1 4 of UBV.

Inserts g1 in col. 2.

0 1 0 0 V ! 0 0 0 1 Inserts g2 col. 4.

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

0 0 1  1 g1 UBV ! 0 0 1 1 0 0 1  1 0 UBV ! 0 0 0 1 1 0 0 0 1 0 0 1  1 0 ! 0 0 0 g 2 0 0 0 1 1 1 g1 0 0 0 0 0 0 0 g 1 ! 0 0 g2 0  g 1 0 0  g2 0 0 0 g2 0 g1 0 0 0 0 g2

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

If g1=g2=g:
0 0 1 1 ? g 0  g A ?g A?0 1 0  1A! 0 UBV ! 0 0  1  1 0 0 0 0 0 1 0 g 0  g UBV ! ? g 0  g A! 0 0 0 0 0 0  1 0 g 0 g

!g is a scalar! (i.e., this is a rank 1 update.)

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Using the matrix inversion lemma to solve the modified problem:
E ! Y 1 I ! (Y  UBV ) 1 I

! Y 1 I  Y 1U ( B 1  VY 1U ) 1VY 1 I ! E  Y 1U ( B 1  VY 1U ) 1VE where e1 e E ! 2 e3 e4

Step 1: Solve YE=I

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Step 2: Multiply E by V to get E.

E ! E  Y 1U ( B 1  VY 1U ) 1VE e1 e VE ! ? 1 0  1A 2 ! ? 2  e4 A! ed e 0 e3 Step 3: Calculate (B-1 + VY-1 U)-1 e. 1 e4 0 1 1 1 1 1 1 e' Y ( B  VY U ) e' ! g  ? 1 0  1A 0 0  1 (Use FF since only need elements in 2nd and ! g ' 4th positions and FullBwill need full Y-1U.)

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Now we have to solve:.

E ! E  g d1U Y
Step 3: Solve Y-1 U= E using fast forward & full backward substitution.

0 1 d Y 1U ! Y 1 ! E d 0  1 Step 4: Find: E ! E  g dd Ed

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Individually: Given the matrix, its LU factors and the solution to YE=I:
2  1 2 1 0 0 1  1 2  1 0  0.5 1 1.5  1 ! Y ! 0  1 2  1  .667 1 1.333  1  .75 1 0 0 1 2 1.25
1 .8 0 .6 Y 1 I ! Y 1 ! 0 .4 0 .2

Use the matrix inversion lemma, find the solution to:

3  1  1 0 1  1 2  1 0 0 Y 1 I !  1  1 3  1 0 0 0  1 2 0

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

The following results may be useful:
1 0.4 0  0. 2 Y 1 !  1  0.8 0  0. 4

0.073 1 0 0.4  0.036 ! Y 1 *  1 2.2  0.145  0.073 0

The End

 Copyright 1999 Daniel Tylavsky

Matrix Inversion Lemma

Solution is:
0.72727 0.63636 E! 0.54545 0.27273