MathematicalTechniques

ConvergenceTolerance
Asolutiontof (x)=0isconsideredconverged
atiterationkif:
t it ti k if
f ( xk )

:tolerance thathasbeenspecified

Dr.YehiaElShazly

SuccessiveSubstitution
Simplestalgorithmforasinglevariablenonlinearalgebraic
equation
f(x)=0
x=g(x)
Ourguess forxatiterationk+1isbasedonthe
evaluationofg(x)atiterationk

xk +1 = g ( xk )
If
Ifformulatedproperlyitconverges
f
l d
l i
toasolution(withina
l i ( i hi
desiredtolerance)
Ifnotformulatedproperly,itmaydiverge orconvergeto
physicallyunrealisticsolutions
3

NewtonRaphson Method
Mostcommonmethodforsolvingnonlinear
algebraic equations
algebraicequations
DerivedbyperformingaTaylorseriesexpansionof
f(x):
f ( x + x) = f ( x) + f ' ( x)x +

where
f ' ( x) =

f " ( x)
f " ' ( x)
(x) 2 +
(x)3 + ..... = 0
2
6

f ( x)
2 f ( x)
, f " ( x) =
,
x
x 2

Dr.YehiaElShazly

 Neglectingthesecondorderandhigher
derivativetermsandsolvingforf(x+x) =0,
weobtain:
x =

f ( x)
f ' ( x)

xk +1 xk =

f ( xk )
f ' ( xk )

xk + 1 = xk

f ( xk )
f ' ( xk )
5

Noticethatwecanobtain
thefollowinggraphical
representationfor
Newtonssmethod.Starting
Newton
method Starting
fromtheinitialguessof,
wefindthatisthe
intersectionofwiththex
axis.Evaluateanddrawa
linewithslopetothexaxis
tofind.Thisprocedureis
continueduntil
convergence.

Dr.YehiaElShazly

Example1 AReactorwithSecondOrder
Kinetics
Thedynamicmodelforanisothermal,constantvolume,
chemicalreactorwithasinglesecondorderreactionis:
dC A F
F
= C Af C A kC 2A
dt
V
V
Findthesteadystateconcentrationforthefollowinginputs
andparameters:

F / V = 1 min 1 , C Af = 1g
gmol / liter , k = 1liter /(( g
gmol min))

Dr.YehiaElShazly

 Atsteadystate,dCA/dt =0andsubstitutingthe
parameterandinputvalues,wefind:
2
1 C As C As
=0
Letx=CAs :

f ( x) = x 2 x + 1 = 0
x=1.618and0.618

Aconcentrationcannotbethenegative,sotheonly
physicallymeaningfulsolutionisx=0.618.

Tousethedirectsubstitutionmethod,wecan
rewritethefunctionintwodifferentways:
(i)x2 =x+1and(ii)x=x2 +1

x2 = x + 1

x = x + 1 = g ( x)

Or,usingsubscriptk toindicatethekth iteration

xk +1 = xk + 1 = g ( xk )

forafirstguessofx0=0.5,wefindthefollowing
sequence x = 0.5 + 1 = 0.7071
1

x 2 = 0.7071 + 1 = 0.5412
x 3 = 05412 + 1 = 0.6774
x 4 = 0.6774 + 1 = 0.5680

10

Dr.YehiaElShazly

 Thissequenceslowlyconvergesto0.618

11

Noticethataninitialguessofx0 =0or1oscillates
between0and1,neverconvergingordiverging

12

Dr.YehiaElShazly

MATLABRoutinesforSolvingFunctionsof
aSingleVariable
roots
fzero

polynomial
generalnonlinearequation

13

SYSTEMSOFLINEARALGEBRAIC
EQUATIONS
Considerthefollowingsetoflinearequations
containing three unknowns:
containingthreeunknowns:
x1 +2x2 3x3 =4
13
2xl +2x2 +3x3 =
1
x1 2x2 +2x3 =
Theobjectiveistodeterminethevaluesx1,x2,andx3
thatsimultaneouslysatisfyallthreeequations.

14

Dr.YehiaElShazly

 Thegeneralproblemofnlinearequationscontaining
nunknowns(xi's):
=
b1
a11 x1 +a12 x2 ++aln xn
a21 x1 +a22 x2 ++a2n xn
=
b2
.........................
......................
anl x1 +an2 x2 ++ann xn

bn

15

a11
a 21
a31
.
.
an1

a12 a13
a 22 a 23
a32 a33
.
.
.
.
an 2 an3

.
.
.
.
.
.

.
.
.
.
.
.

a1n x1
a 2n x 2
a3n x3
. .
. .
ann x
n

b1
b2
b3
.
.
bn

Ax = b
n

a x =b
i=1

ki i

k=1,2,...,n

16

Dr.YehiaElShazly

 Onlycaseswherethenumberofequationsequals
thenumberofunknowns(squarecases).
Eachequationmustbeanindependentequationin
orderforthesystemofequationstobesolvable.

17

GaussianElimination
Systematiceliminationofunknownsfromtheproblem
untilonlyoneequationandoneunknownareleft.
Thevalueofthisunknownisdeterminedandthe
h
l
f hi
k
i d
i d d h
remainingunknownsarethencalculatedinturn.
Considerasystemofthreelinearequationsandthree
unknowns:
3x1 2x2 +3x
+ 3x3
=8
8
(1a)
(1
a)
4x1 +2x2 2x3
=2
(1b)
=9
(1c)
x1 +x2 +2x3
18

Dr.YehiaElShazly

 First,Equation1awillbeusedtoeliminatex1 from
both1band1c:
x1 2/3x2 + x3
=8/3
(2a)
14/3 x2
14/3x
6 x3 =26/3
6x
26/3
(2b)
(2
b)
5/3x2 +
x3 =19/3
(2c)
Next,Equation2bisdividedby14/3,resultingin:
x2 9/7x3 =13/7
Atthispoint,thetheeequationsaregivenas:
this point the thee equations are given as :
At
x1 2/3x2 +x3 =8/3 (3a)
9/7x3 =13/7
(3b)
x2
=19/3
(3c)
5/3x2 +x3
19

Continuing,Equation3bwillbeusedtoeliminatex2
fromEquation3c.
=
198/21
or
66/21x3
x3 =3
3
Finallythethreeequationsare:
(4a)
x1 2/3x2 +x3 =8/3
(4b)
x2 9/7x3 =13/7
(4c)
x3 =3
x1 = 1
x2 = 2
x3 = 3
20

Dr.YehiaElShazly

10

InMatlab
In MATLAB we solve Ax = b with the single
command,,

x = A\b

21

A=[111;213;316];
b=[4;7;2];
x=A\b
x=
19.0000
7.0000
8.0000
22

Dr.YehiaElShazly

11

NonlinearSystems
Asystemofn nonlinearequationsinn unknowns:

f1(x
( 1,x2,x3,..,xn)=0
) 0
f2(x1,x2,x3,..,xn)=0
f3(x1,x2,x3,..,xn)=0
.
.
fn(x1,xx2,xx3,..,xxn)=0
) = 0

23

Solvethefollowingsystemofnonlinearequations:

f1 ( x1x2 ) = x1 4 x12 x1 x2 = 0
f 2 ( x1 x2 ) = 2 x2 x22 + 3x1 x2 = 0
Fromthefirstequationwecansolveforx2 intermsofx1:

x2 = 1 4 x1

Substitutingintothesecondequation:
2

1 + 3x1 28x1 = 0

whichhasthetwosolutionsforx
hi h h th t
l ti
f
(f
th
d ti f
l )
1 (fromthequadraticformula)

x1 =0.25orx1 = 0.1429
Thecorrespondingvaluesofx2 are

x2 =0.0andx2 =1.5714
24

Dr.YehiaElShazly

12

Or,writingthesesolutionsinvectorfrom:

solution1 is0.25 whilesolution2 is 0.149

1.5714

Arewecertainthatthereareonlytwosolutions?

25

Wecanalsoseebyinspectionthattheorigin
0
x=
0

,isalsoasolution(trivial solution),
Anothersolutionthatisslightlylessobviousis
0
x = ,
2

satisfiestheEquations.
26

Dr.YehiaElShazly

13

 SolvethesecondEquationforx1 intermsofx2,then
substitutethisresultintothefirstEquationtosolve
forx2:
x22 4 x2 + 4 = 0

andusingthequadraticformula
x2 =

4
16 16

=2
2
2

which gives us the solution

whichgivesusthesolution

0
x = .
2

Wearestillmissingthetrivialsolution x = 00

27

Themistake :whenwesolvedthefirstEquationfor
x2 intermsofx1.ThefirstEquationisquadraticinx1,
thereforetherearetwosolutionsforx1 intermsof
x2.Thisisclearerifwewriteitas:
4x12 + x1 (1 x 2 ) = 0

Solveforx1 : x1 =0
or x1 =1/4(1 x2).
SubstitutingthesevaluesintothesecondEquation
willleadtothefoursolutions:
x = 0, 0, 0.25, and 0.1429.
1.5714
0 2 0

28

Dr.YehiaElShazly

14

NewtonsMethodforMultivariableProblems
Forthegeneralsetofequations
f1 ( x1 x2 ,...xn ) 0
f ( x x ,...x )
n
2 1 2
0
.
= .

.
.
f n ( x1 x2 ,...xn ) 0

UseaTaylorseriesexpansionforeachfi:
fi
x j +higher order
j =1 x j
n

f i ( x + x ) = f i ( x ) +

29

Neglectingtheordertermsandwritinginmatrix
form:

f (x + x) = f (x) + Jx
whereJ
h
J isknownasthe
i k
th Jacobian
f1
x
1
.
J =
.
f n

x1

f1
x2
.
.
f n
x2

. .
. .
. .
. .

f1
xn

.
f n

xn

Now,sincewewishtosolveforx suchthatf(x+x)=0,
then:f(x)+Jx=0
30

Dr.YehiaElShazly

15

 Solvingfor x atiterationk:
1

x k = J k f ( x k )

(1)
x:changeinthex vectorfromthepreviousiteration:
x k = x k +1 x k
(2)
SubstitutingEquation2intoEquation1
1
(3)
x k +1 = x k J k f ( x)
xk : vectorofvaluesatiterationk.
Forasingleequation,Equation3is:

x k + 1 = xk

f ( xk )
f ' ( xk )
31

f1 ( x1 x2 ) = x1 4 x12 x1 x2 = 0

f 2 ( x1x2 ) = 2 x2 x22 + 3x1x2 = 0

x1 4x12 x1x 2 = 0
2x 2 x 22 + 3x1x 2 0

TheJacobian elementsare
f
f
J11 = x = 1 8x x
J12 = x
J21 = xf = 3x
J22 = xf
1

= x1

= 2 2 x 2 + 3x 1

x1
1 8x1 x 2

3x 2
2 2x 2 + 3x1

J=

32

Dr.YehiaElShazly

16

Considerinitialguessofx1 =1andx2 =1.

Letx(0)representsthevectorforthisinitialguess
x (0)=
x 1 (0) 1
x 2 (0)

ThevalueoftheJacobian atthisinitialguessis
J(x(0))= 10 1

3 1

TheinverseoftheJacobian
The
inverse of the Jacobian is
J1 (x(0))= 1 1
13
3

13

13
10

13

33

Thevalueofthefunctionvectorfortheinitialguessis

f(x(0))=

1 4 1 = 6
2(1) 1 + 3 0

Theguessforx(1),wherex(1)representsthe
vectoratiteration1:
x(1)=x(0) J1 (x(0))f(x(0))
x(1)=
x(1) = 1 1 1 6 x(1)=
x(1) = 0.5385
13 13
1 3 10 0

13 13

0.3846

34

Dr.YehiaElShazly

17

Continuingwithiterations2through7wefindthe
followingresults
Iteration

x1

x2

-1

-1

-0.5385

0.3846

-0.3104

1.0688

-0.2016

1.3952

-0.1561

1.5317

-0.1439

1.5683

-0.1429

1.5714

-0.1429

1.5714
35

Thesequenceofiterationsforaninitialguess
ofx1 =1andx2 =1
Iteration

x1

x2

0.6190

0.0476

o.3893

0.0081

0.2870

0.0008

0.2542

0.0000

0.2501

0.0000

0.2500

0.0000

36

Dr.YehiaElShazly

18

MATLABRoutinesforSystemsof
NonlinearAlgebraicEquations
fsolve :usedtosolvesetsofnonlinear
algebraicequations,usingaquasiNewtons
l b i
ti
i
iN t
method.

37

ReconsideredUsingMATLAB
ThemfileusedtoimplementtheExampleusingfsolve
is:
Openanmfile
function F = nle (x)
functionF=nle
F(1)=x(1) 4*x(1)*x(1) x(1)*x(2);
F(2)=2*x(2) x(2)*x(2)+3*x(1)*x(2);
Savethemfileunderthenamenle
Theinitialguessisenteredinthecommandwindow
>>x0=[11];
Andweobtainthesolutionbyentering
>>x=fsolve('nle',x0)
Whichgivesustheexpectedresults:
>>x=[0.25000.0000]
38

Dr.YehiaElShazly

19

ODE
OrdinaryDifferentialEquations
Linear:therateofchangeofthedependentvariable
isalinearfunctionofthedependentvariable.
l
f
f h d
d
bl

dy
= y
dx

dy
= e x y
dx

Nonlinear:therateofchangeofthedependent
variable is a nonlinear function of the dependent
variableisanonlinearfunctionofthedependent
variable.

dy
= y 2
dx
39

Solving requiresasetofconditions:
All conditions given at one
value of x
Initial Value Problem
(IVP ).

Conditions given at
different values of x
Boundary Value
Problem (BVP )

40

Dr.YehiaElShazly

20

Matlab
dsolve

41

EulerMethod
Simplest method to solve a first order ODE.
Consider
C id the
h first
fi order
d IVP:
IVP

y' = dy = f(x,y);
dx

y(0) = yo

The initial condition (xo,yo) is used to calculate the

p of y(x)
y( ) at x = 0:
slope

dy
dx x=xo = f(xo,yo)
42

Dr.YehiaElShazly

21

 Assuming that the slope (dy/dx) remains constant for

a small distance, x, the value of y(xo + x) can be
estimated as:

y(xo + x) = y(xo) + x f(xo,yo)

Next these values of x and y [i.e., xo + x and y(xo +
x)] are used to estimate the slope at the new point.
Once again,
again y(xo + 2x) can be calculated as

y(xo + 2x) = y(xo + x) + x f[xo + x ,y(xo + x)]

43

y(xi + x) = y(xi) + x f(xi,yi)

yi+1 = yi + x f(xi,yi)
44

Dr.YehiaElShazly

22

ADVANTAGES
Method is straightforward
Easy to program.
program

DISADVANTAGES
Small step size must be used for accuracy.
If too
t small
ll off a step
t size
i is
i used,
d numerical
i l
truncation problems may result.

45

Example
UsingtheEulermethod,calculatethevalueofyatx
= 1 if:
=1if:

dy 2
dx =x y ;wherey=1atx=0

46

Dr.YehiaElShazly

23

 Solution

yi+1 =yi +xxi2 yi

Choosingx=0.1forxo =0andyo =1,
y1 =1+0.1*02*1=1.0000
Thenforx1 =0.1andy1 =1,
y2 =1+0.1*0.12*1=1.0010
Then
y3 =1.0010+0.1
= 1 0010 + 0 1 *0.2
0 22*1.0010=1.0050
1 0010 = 1 0050
and
y4 =1.0050+0.1*0.32*1.0050=1.0140
47

x
0.1

0.05

0.02

0.01

Analytical
value

1.0000

1.0000

1.0000

1.0000

1.0000

0.1

1 0000
1.0000

1 0001
1.0001

1 0002
1.0002

1 0003
1.0003

1 0003
1.0003

0.2

1.0010

1.0018

1.0023

1.0025

1.0027

0.3

1.0050

1.0069

1.0081

1.0086

1.0090

0.4

1.0140

1.0176

1.0199

1.0207

1.0216

0.5

1.0303

1.0361

1.0400

1.0412

1.0425

0.6

1.0560

1.0650

1.0707

1.0727

1.0747

0.7
0

1 0940
1.0940

1 1070
1.1070

1 1154
1.1154

1 1182
1.1182

1 1211
1.1211

0.8

1.1819

1.1861

1.1476

1.1661

1.1718

0.9

1.2211

1.2468

1.2635

1.2692

1.2751

1.0

1.3200

1.3559

1.3792

1.3873

1.3956

48

Dr.YehiaElShazly

24

1.4

1.3

y;0.1

1.2

y;0.05
y;0.01
y;exact
1.1

1
0

0.2

0.4

0.6

0.8

49

ModifiedEuler
Consider the first order IVP:
d
dy
y' = dx
= f(x,y);
y(0) = yo

y`i+1=yi+h f(xi,yi)
yi+1=yi+h/2 (f(x
(f( i,yi)+f(x
)+f( i+1,y``i+1))

50

Dr.YehiaElShazly

25

1.4

1.3

1.2

y;0.1Euler
y;0.1ModifiedEuler
y;exact

1.1

1
0

0.2

0.4

0.6

0.8

51

Runge Kutta
Consider the first order IVP:
yy' = dyy = f(
f(x,y);
,y);
y(x0) = yo
y(
dx

yi+1=yi+h/6[k1+2k2+2k3+k4]
xi+1=xi +h
k1=f(xi,yyi)
k2=f(xi+h/2,yi+hk1/2)
k3=f(xi+h/2,yi+hk2/2)
k4=f(xi+h,yi+hk3)

Dr.YehiaElShazly

52

26

 k1 is the slope at the beginning of the interval;

k2 is the slope at the midpoint of the interval, using
slope k1 to determine the value of y at the point tn + h
/ 2 using Euler's
Euler s method
k3 is again the slope at the midpoint, but now using
the slope k2 to determine the y-value;
k4 is the slope at the end of the interval, with its yvalue determined using k3.
In
I averaging
i the
h four
f
slopes,
l
greater weight
i h is
i given
i
to the slopes at the midpoint:

53

k1

x+h/2

y+hk1/2

0.05

k2

y+hk2/2

k3

0.0025 1.000125 0.0025

x+h

y+hk3

k4

0.1

1.00025 0.010003

0.1

1.000333385 0.010003

0.15

1.000834 0.022519 1.001459 0.022533

0.2

1.002587 0.040103

0.2

1.002670218 0.040107

0.25

1.004676 0.062792 1.00581 0.062863

0.3

1.008957 0.090806

0.3

1.009040611 0.090814

0.35

1.013581 0.124164 1.015249 0.124368

0.4

1.021477 0.163436

0.4

1.021562501

0.16345

0.45

1.029735 0.208521 1.031989 0.208978

0.5

1.04246 0.260615

0.5

1.042546887 0.260637

0.55

1.055579 0.319313 1.058513 0.3202

0.6

1.074567 0.386844

0.6

1.07465532

0.386876

0.65

1.093999 0.462215 1.097766 0.463806

0.7

1.121036 0.549308

07
0.7

1 121125738 0.549352
1.121125738
0 549352

0 75
0.75

1 148593 0.646084
1.148593
0 646084 1.15343
1 15343 0.648804
0 648804

08
0.8

1 186006 0.759044
1.186006
0 759044

0.8

1.186095267 0.759101

0.85

1.22405 0.884376 1.230314 0.888902

0.9

1.274985 1.032738

0.9

1.275068529 1.032806

0.95

1.326709 1.197355 1.334936 1.20478

1.395547 1.395547

1.395612219 1.395612

1.05

1.465393 1.615596 1.476392 1.627722

1.1

1.558384 1.885645

54

Dr.YehiaElShazly

27

1.4

1.3

y; RK4
y;RK4
1.2

y;0.1ModifiedEuler
y;exact

1.1

1
0

0.2

0.4

0.6

0.8

55

y; Euler

difi d
y; Modified
Euler

y; RK4

y; exact

1.000000

1.320015696

1.386016923

1.395612219

1.395612425

Error

0.075596729

0.009595502

0.000000206

Percent
Deviation

5.416742326

0.687547749

0.000014769

56

Dr.YehiaElShazly

28

Matlab
ode45
>>ff=inline(x^2*y,x,y)
inline( x 2 y , x , y )
>>ode45(f,[01],1)
1.4

1.35
1.3
1.25
1.2
1.15
1.1
1.05
1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

57

>>ode45(f,0:0.01:1,1)
1.4
1.35
1.3
1.25
1.2
1 15
1.15
1.1
1.05
1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

58

Dr.YehiaElShazly

29

0
0.025
0.05
0.075
0.1
0.125
0.15
0.175
0.2
0.225
0.25
0.275
0.3
0.325
0.35
0.375
0.4
0.425
0.45
0.475
0.5
0.525
0.55
0.575
0.6
0.625
0.65
0.675
0.7
0.725
0.75
0.775
0.8
0.825
0.85
0.875
0.9
0.925
0.95
0.975
1

>>[ab]=ode45(f,[01],1)
>>[ab]=ode45(f,0:0.01,1,1)

1
1.000005
1.000042
1.000141
1.000333
1.000651
1.001126
1.001788
1.00267
1.003804
1.005222
1.006956
1.009041
1.011508
1.014394
1.017734
1.021563
1.025919
1.030841
1.03637
1.042547
1.049417
1.057025
1.065421
1.074655
1.084783
1.095862
.09586
1.107955
1.121126
1.135447
1.150993
1.167846
1.186095
1.205835
1.227167
1.250205
1.275069
1.301891
1.330815
1.361998
1.395612

59

>>ode45(f,[01],1:0.2:3)
4.5

3.5

2.5

1.5

1
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

60

Dr.YehiaElShazly

30