1

Iterative solution methods:

Starts with an initial approximation for the solution vector (x0) At each iteration updates the x vector by using the sytem Ax=b During the iterations A, matrix is not changed so sparcity is preserved Each iteration involves a matrix-vector product If A is sparse this product is efficiently done

Write the system Ax=b in an equivalent form x=Ex+f (like x=g(x) for fixed-point iteration) Starting with x0, generate a sequence of approximations {xk} iteratively by xk+1=Exk+f Representation of E and f depends on the type of the method used But for every method E and f are obtained from A and b, but in a different way

Different iteration types:

Jacobi Iteration Gauss-Seidel Iteration Successive Over Relaxation (S.O.R) SOR is a method used to accelerate the convergence Gauss-Seidel Iteration is a special case of SOR method

Biography of Carl Jacobi:

Carl Jacobi came from a Jewish family but he was given the French style name Jacques Simon at birth. His father, Simon Jacobi, was a banker and his family were prosperous. Carl was the second son of the family, the eldest being Moritz Jacobi who eventually became a famous physicist. Moritz Jacobi has an entry in his own right in There was a sister, Therese Jacobi, and a third brother, Eduard Jacobi, who was younger than Carl. Eduard did not pursue an academic career, but followed instead his father's profession as a banker.

Jacobi's early education was given by an uncle on his mother's side, and then, just before his twelfth birthday, Jacobi entered the Gymnasium in Potsdam. He had been well taught by his uncle and he had remarkable talents so in 1817, while still in his first year of schooling, he was put into the final year class. This meant that by the end of the academic year 1816-17 he was still only 12 years old yet he had reached the necessary standard to enter university. The University of Berlin, however, did not accept students below the age of 16, so Jacobi had to remain in the same class at the Gymnasium in Potsdam until the spring of 1821. Of course, Jacobi pressed on with his academic studies despite remaining in the same class at school. He received the highest awards for Latin, Greek and history but it was the study of mathematics which he took furthest. By the time Jacobi left school he had read advanced mathematics texts such as Euler's Introductio in analysin infinitorum and had been undertaking research on his own attempting to solvequintic equations by radicals. Jacobi entered the University of Berlin in 1821 still unsure which topic he would concentrate on. He attended courses in philosophy, classics and mathematics for two years before realising that he had to make a definite decision between these subjects. He chose mathematics, but this did not mean that he could attend high level courses in mathematics for at this time the standard of university education in mathematics in Germany was rather poor. As he had done at the Gymnasium, Jacobi had to study on his own reading the works of Lagrange and other leading mathematicians. By the end of academic year 1823-24 Jacobi had passed the examinations necessary for him to be able to teach mathematics, Greek, and Latin in secondary schools. Of course, one might have expected him to have problems obtaining a teaching position since, as we noted at the beginning of this article, he was Jewish. His brilliance appears to have been sufficient to allow this hurdle to be overcome for, in 1825, he was offered a teaching post at the Joachimsthalsche Gymnasium, one of the leading schools in Berlin. He had submitted his doctoral dissertation to the University of Berlin even before he received the offer of the teaching post, and he was allowed to move quickly to work on his habilitation thesis. Jacobi presented a paper concerning iterated functions to the Berlin Academy of Sciences in 1825. However, the referees did not consider the results worth publishing and indeed the paper was not published by the Berlin Academy. The paper was published eventually, for in 1961 it was published with a commentary in . Biermann, the author of , quotes the opinions of the original referees and criticises

them strongly. Although this was not the best start for the young Jacobi, it did not hold him back for long and his publication record over the following years would be quite remarkable for both the number and quality of the works

Jacobis method:
There are other methods that can be used to solve a set of linear equations that are based on iteration. In these cases, an initial estimate of the parameters is estimated and then the equations are solved, yielding an updated version of the parameters. These new values are then inserted back into the equations and the process continues until the desired solution is reached. The one iterative methods discussed here are the Jacobi method . The Jacobi method is a method of solving a matrix equation on a matrix that has no zeros along its main diagonal (Bronshtein and Semendyayev 1997). Each diagonal element is solved for, and an approximate value plugged in. The process is then iterated until it converges. This algorithm is a stripped-down version of the Jacobi transformation method of matrix diagonalization. The Jacobi method is easily derived by examining each of the equations in the linear system of equations in isolation. If, in the th equation

solve for the value of gives

while assuming the other entries of

remain fixed. This

which is the Jacobi method. Jacobis method is valid only if all the aijs are non-zero or if the equations can be suitably re-arranged to make this so This can always be done if A is invertible. Faster convergence can be achieved if we re-arrange the rows so that the diagonal elements have magnitude as large as

possible relative to the magnitudes of other co-efficients in the same row. If this is not done sometimes jaccobis method may not converge. To solve the system of equations by Jaccobis method, the following steps are used a) Choose initial guesses: x1(0) = x2(0) = . = xn(0) = 0 if no better initial guesses asre available. b) Set r = 0 c) For each i=1,2,3,n, compute

Assuming all aij 0; i=1,2,3,..,n; r 0. d) If solution vector x(r) is sufficiently accurate, then go to the last step. If x (r) is not sufficiently accurate, then add 1 or r and go to step c. e) Termination of the process: The following two possible stopping criteria are used: i. Use a fixed number of iterations. ii. Use pre-assigned accuracy as:x(r+1) x(r) . We can combine both of them as well Jacobis method is also known as the method of simultaneous displacements because each of the equations is simultaneously changed, by using the most recent set of x-values available

Practice :

Q: Solve the following systems of equations using jacobis method: 10x +y + z = 24 -x + 20 y + z = 21 - x + y + 100 z = 300

Solution:
10x +y + z = 24 -x + 20 y + z = 21 __________ (1) __________ (2)

- x + y + 100 z = 300 __________ (3) Converting the system into the Jacobian iterative form: from (1), we get, x(r+1) =2.4-0.1(y(r) +z(r)) from (2),we get, y(r+1) =1.05+0.05(x(r) -z(r)) from (3),we get, z(r+1) =3+0.01x(r) +0.02y(r)

initial approximation:

x(0)= y(0)= z(0)=0. First iteration: Set r=0. x(0+1) =2.4-0.1(y(0) +z(0)) =2.4-0.1(0+0)
x(1) =2.4

y(0+1) =1.05+0.05(x(0) -z(0)) =1.05+0.05(0-0)

y(1)=1.05

z(0+1) =3+0.01x(0) +0.02y(0) =3+0+0

z(1) =3

Second iteration: Set r=1. x(1+1) =2.4-0.1(y(1) +z(1)) x(2)=2.4-0.1(1.05+3)

x(2) =1.9935

y(1+1) =1.05+0.05(x(1) -z(1)) y(2)=1.05+0.05(2.4-3)

y(2) =0.9976

z(1+1) =3+0.01x(1) +0.02y(1) z(2)=3+0.01*2.4+0.02*1.05

z(2) =3.0403

Third iteration: Set r=2. x(2+1) =2.4-0.1(y(2) +z(2)) x(3)=2.4-0.1(1.02+3.045)

x(3) =1.9935

y(2+1) =1.05+0.05(x(2) -z(2)) y(3)=1.05+0.05(1.995-3.045)

y(3) =0.9976

z(2+1) =3+0.01x(2) +0.02y(2) z(3)=3+0.01(1.995)+0.02(1.02)

z(3) =3.0403

Fourth iteration: Set r=3. x(3+1) =2.4-0.1(y(3) +z(3)) x(4)=2.4-0.1(0.9974+3.0403)

x(4) =1.9962

y(3+1) =1.05+0.05(x(3) -z(3)) y(4)=1.05+0.05(1.9935-3.0403)

y(4) =0.9978

z(3+1) =3+0.01x(3) +0.02y(3) z(4)=3+0.01(1.9935)+0.02(0.9976)

z(4) =3.0403

Fifth iteration: Set r=4. x(4+1) =2.4-0.1(y(4) +z(4)) x(5)=2.4-0.1(0.9978+3.0399)

x(5) =1.9962

y(4+1) =1.05+0.05(x(4) -z(4)) y(5)=1.05+0.05(1.9962-3.0399)

y(5) =0.9978

z(4+1) =3+0.01x(4) +0.02y(4) z(5)=3+0.01(1.9962)+0.02(0.9978)

z(5) =3.0399

after five iterations, the result correct to 4 decimal places is, x=1.9962 y=0.9978 z=3.039

______________________________________
Q. 2: Solve the following systems of equations using jacobis method: 8x1-3x2+2x3=20eq(1) 4x1+11x2-x3=33eq(2) 6x1+3x2+12x3=36..eq(3) Solution:
From eq1, eq2&eq3 we get, X1=2.5+0.375x2-0.25x3 X2=3-0.36x1-0.0909x3 X3=3-0.5x1-0.25x2 Rewriting the system in Jacobis iterative form: X1(r+1)=2.5+0.375x2(r)-0.25x3(r) X2(r+1)=3-0.36x1(r)-0.0909x3(r)

10

X3(r+1)=3-0.5x2(r)-0.25x3(r) Initial approximation: X1(0)=x2(0)=x3(0)=0 First iteration: Set r=0 X1(0+1)=2.5+0-0 X1(1)=2.5 X2(0+1)=3-0+0 X2(1)=3 X3(0+1)=3-0-0 X3(1)=3 Second iteration: Set r=1 X1(2)=2.875 X2(2)=2.3727 X3(2)=1

Third iteration: Set r=2 X1(3)=3.1398

11

X2(3)=2.0559 X3(3)=0.96933 Fourth iteration: Set r=3 X1(4)=3.02 X2(4)=1.95 X3(4)=0.916 Fifth iteration: Set r=4 X1(5)=3.0022 X2(5)=1.8296 X3(5)=1.0025 After fifth iteration, by rounding off X1=3 X2=2 X3=1

______________________________________