Professional Documents
Culture Documents
09
Adequacy of Solutions
Example 1
Is this system of equations well-conditioned?
1 2 x 4
2 3.999 y 7.999
Solution
The solution to the above set of equations is
x 2
y 1
Make a small change in the right hand side vector of the equations
04.09.1
04.09.2 Chapter 04.09
1 2 x 4.001
2 3.999 y 7.998
gives
x 3.999
y 4.000
Make a small change in the coefficient matrix of the equations
1.001 2.001 x 4
2.001 3.998 y 7.999
gives
x 3.994
y 0.001388
This last systems of equation “looks” ill-conditioned because a small change in the
coefficient matrix or the right hand side resulted in a large change in the solution vector.
Example 2
Is this system of equations well-conditioned?
1 2 x 4
2 3 y 7
Solution
The solution to the above equations is
x 2
y 1
Make a small change in the right hand side vector of the equations.
1 2 x 4.001
2 3 y 7.001
gives
x 1.999
y 1.001
Make a small change in the coefficient matrix of the equations.
1.001 2.001 x 4
2.001 3.001 y 7
gives
Adequacy of Solution 04.09.3
x 2.003
y 0.997
This system of equation “looks” well conditioned because small changes in the
coefficient matrix or the right hand side resulted in small changes in the solution vector.
Later we will see how much (quantifiable terms) we can trust the solution in a system of
equations. Every invertible square matrix has a condition number and coupled with the
machine epsilon, we can quantify how many significant digits one can trust in the
solution.
Example 3
Find the row sum norm of the following matrix [A].
10 7 0
A 3 2.099 6
5 1 5
04.09.4 Chapter 04.09
Solution
max 3
A
1i 3
a
j 1
ij
max10 7 0 , 3 2.099 6 , 5 1 5
max10 7 0 , 3 2.099 6 , 5 1 5
max17,11.099,11
17.
C
4.001 4
7.998 7.999
0.001
0.001
and
3.999 2
X
4.000 1
5.999
3.000
then
C
0.001
X
5.999
The relative change in the norm of the solution vector is
X 5.999
X 2
2.9995
The relative change in the norm of the right hand side vector is
C 0.001
C 7.999
1.250 10 4
See the small relative change of 1.250 10 4 in the right hand side vector results in a
large relative change in the solution vector as 2.9995.
In fact, the ratio between the relative change in the norm of the solution vector and the
relative change in the norm of the right hand side vector is
X / X 2.9995
C / C 1.250 10 4
23993
Let us now go back to the well-conditioned system of equations.
1 2 x 4
2 3 y 7
gives
04.09.6 Chapter 04.09
x 2
y 1
Denoting the system of equations by
AX C
X 2
C
7
Making a small change in the right hand side vector
1 2 x 4.001
2 3 y 7.001
gives
x 1.999
y 1.001
Denoting the above set of equations by
AX ' C '
the change in the right hand side vector is then found by
C C ' C
and the change in the solution vector is
X X ' X
then
C
4.001 4
7.001 7
0.001
0.001
and
X
1.999 2
1.001 1
0.001
0.001
then
C 0.001
X
0.001
The relative change in the norm of solution vector is
Adequacy of Solution 04.09.7
X 0.001
X
2
5 10 4
The relative change in the norm of the right hand side vector is
C 0.001
C 7
1.429 10 4
See the small relative change the right hand side vector of 1.429 10 4 results in the
small relative change in the solution vector of 5 10 4 .
In fact, the ratio between the relative change in the norm of the solution vector and the
relative change in the norm of the right hand side vector is
X
/X 5 10 4
C
/C 1.429 10 4
3 .5
If there is such a relationship, will it help us quantify the conditioning of the matrix?
That is, will it tell us how many significant digits we could trust in the solution of a
system of simultaneous linear equations?
X C
and
X C
and between
X A
and
X A
These relationships are
X C
A A 1
X X C
and
X A
A A1
X A
The above two inequalities show that the relative change in the norm of the right hand
side vector or the coefficient matrix can be amplified by as much as A A1 .
This number A A1 is called the condition number of the matrix and coupled with the
machine epsilon, we can quantify the accuracy of the solution of [ A][ X ] [C ] .
Prove for
[ A][ X ] [C ]
that
X A
A A 1 .
X X A
Proof
Let
AX C (1)
Then if [ A] is changed to A', the [X ] will change to X ' , such
that
A'X ' C (2)
From Equations (1) and (2),
AX A'X '
Denoting change in [ A] and [X ] matrices as A and X , respectively
A A' A
X X ' X
Adequacy of Solution 04.09.9
then
AX A AX X
Expanding the above expression
AX AX AX AX AX
[0] AX A X X
AX AX X
X A1 AX X
Applying the theorem of norms, that the norm of multiplied matrices is less than the
multiplication of the individual norms of the matrices,
X A1 A X X
Multiplying both sides by A
A X A A1 A X X
X A
A A1
X X A
How do I use the above theorems to find how many significant digits are correct in
my solution vector?
The relative error in a solution vector is Cond (A) relative error in right hand side.
The possible relative error in the solution vector is Cond (A) mach
Hence Cond (A) mach should give us the number of significant digits, m at least correct
in our solution by comparing it with 0.5 10 m .
Example 4
How many significant digits can I trust in the solution of the following system of
equations?
1 2 x 2
2 3.999 y 4
Solution
For
A
1 2
2 3.999
it can be shown
04.09.10 Chapter 04.09
3999 2000
A1
2000 1000
A 5.999
A 1 5999
Cond A A A1
5.999 5999.4
35990
Assuming single precision with 24 bits used in the mantissa for real numbers, the
machine epsilon is
mach 21 24
0.119209 10 6
Cond ( A) mach 35990 0.119209 10 6
0.4290 10 2
Comparing it with 0.5 10 m
0.5 10 m 0.4290 10 2
m 2
So two significant digits are at least correct in the solution vector.
Example 5
How many significant digits can I trust in the solution of the following system of
equations?
1 2 x 4
2
3 y 7
Solution
For
A
1 2
2 3
it can be shown
3 2
A1
2 1
Then
A 5,
Adequacy of Solution 04.09.11
A1 5.
Cond (A) A
A 1
55
25
Assuming single precision with 24 bits used in the mantissa for real numbers, the
machine epsilon
mach 21 24
0.119209 10 6
Cond ( A) mach 25 0.119209 10 6
0.2980 10 5
Comparing it with 0.5 10 m
0.5 10 m 0.2980 10 5
m5
Key Terms:
Ill-Conditioned matrix
Well-Conditioned matrix
Norm
Condition Number
Machine Epsilon
Significant Digits