Chapter 19 Numerical Differentiation

Estimate the derivatives (slope, curvature, etc.)

of a function by using the function values at
only a set of discrete points
Ordinary differential equation (ODE)
Partial differential equation (PDE)
Represent the function by Taylor polynomials
or Lagrange interpolation
Evaluate the derivatives of the interpolation
polynomial at selected (unevenly distributed)
nodal points
0

( ) ( )
Differentiation lim
( )
( ) ( )
Difference
( )
A smaller step (or h) results in a smaller error
x
y f(x)
dy f x x f x
dx x x x
y f x x f x dy
x x x x dx
x
A
=
+ A
=
+ A
A + A
= ~
A + A
A
x
i-3
x
i-2
x
i-1
x
i
x
i+1
x
i+2
x
i+3

h x = A
Evenly distributed points along the x-axis

x
1
x
2
x
3

Unevenly distributed points along the x-axis
Distance between two neighboring points is the same, i.e. h.
Forward difference
Backward difference
Centered difference
Numerical Differentiation
Forward difference
x
i1
x
i
x
i+1
x
h
Backward difference
x
i1
x
i
x
i+1
x
h
Centered difference
x
i1
x
i
x
i+1
x
2h
First Derivatives
Forward difference

Backward difference

Central difference
) x ( f
'
i-2 i-1 i i+1 i+2
1 i 1 i
1 i 1 i
1 i 1 i
1 i 1 i
1 i i
1 i i
1 i i
1 i i
i 1 i
i 1 i
i 1 i
i 1 i
x x
y y
x x
) x ( f ) x ( f
) x ( f
x x
y y
x x
) x ( f ) x ( f
) x ( f
x x
y y
x x
) x ( f ) x ( f
) x ( f
+
+
+
+

+
+
+
+

~ '

~ '

~ '
x
y
Truncation Errors
Uniform grid spacing

+
' ' '

' '
+
'
= =
+ ' ' ' + ' ' + ' + = + =

) x ( f
! 3
h
) x ( f
! 2
h
) x ( f h ) x ( f ) h x ( f ) x ( f
) x ( f
! 3
h
) x ( f
! 2
h
) x ( f h ) x ( f ) h x ( f ) x ( f
i
3
i
2
i i i 1 i
i
3
i
2
i i i 1 i

' ' '

= '
' '
+

=
'
' '

= '
+

+
) ) (
) ( ) (
) ( :
) (
) ( ) (
) ( :
) (
) ( ) (
) ( :
2
3
2
1 i 1 i
i
2
1 i i
i
1
i 1 i
i
O(h f
6
h
h 2
x f x f
x f central
O(h) f
2
h
h
x f x f
x f backward
O(h) f
2
h
h
x f x f
x f forward

Example: First Derivatives

Use forward and backward difference approximations
to estimate the first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
Forward Difference



Backward Difference
2 . 1 x 25 . 0 x 5 . 0 x 15 . 0 x 1 . 0 ) x ( f
2 3 4
+ =

= =

= ' =
= =

= ' =
% . , .
.
. .
. .
) . ( ) . (
) . ( , .
% . , .
.
. .
.
) . ( ) (
) . ( , .
5 26 155 1
25 0
925 0 63632813 0
5 0 75 0
5 0 f 75 0 f
5 0 f 25 0 h
9 58 45 1
5 0
925 0 2 0
5 0 1
5 0 f 1 f
5 0 f 5 0 h
t
t
c
c

= =

= ' =
= =

= ' =
% . , .
.
. .
. .
) . ( ) . (
) . ( , .
% . , .
.
. .
.
) ( ) . (
) . ( , .
7 21 714 0
25 0
10351563 1 925 0
25 0 5 0
25 0 f 5 0 f
5 0 f 25 0 h
7 39 55 0
5 0
2 1 925 0
0 5 0
0 f 5 0 f
5 0 f 5 0 h
t
t
c
c
Example: First Derivative
Use central difference approximation to estimate the
first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
Central Difference




2 . 1 x 25 . 0 x 5 . 0 x 15 . 0 x 1 . 0 ) x ( f
2 3 4
+ =
% 4 . 2 , 934 . 0
5 . 0
10351563 . 1 63632813 . 0
25 . 0 75 . 0
) 25 . 0 ( f ) 75 . 0 ( f
) 5 . 0 ( f , 25 . 0 h
% 6 . 9 , 0 . 1
1
2 . 1 2 . 0
0 1
) 0 ( f ) 1 ( f
) 5 . 0 ( f , 5 . 0 h
= =

=
'
=
= =

=
'
=
t
t



c
c
Second-Derivatives
Taylor-series expansion
Uniform grid spacing





Second-order accurate O(h
2
)
(

+ ' ' ' ' + ' ' + = +

+ ' ' ' ' + ' ' ' ' ' + ' =
+ ' ' ' ' + ' ' ' + ' ' + ' + =
+

) x ( f
! 4
h
) x ( f
! 2
h
) x ( f 2 ) x ( f ) x ( f
) x ( f
! 4
h
) x ( f
! 3
h
) x ( f
! 2
h
) x ( f h ) x ( f ) x ( f
) x ( f
! 4
h
) x ( f
! 3
h
) x ( f
! 2
h
) x ( f h ) x ( f ) x ( f
i
4
i
2
i 1 i 1 i
i
4
i
3
i
2
i i 1 i
i
4
i
3
i
2
i i 1 i
) (
!
) ( ) ( ) (
) ( f
4
h
h
x f x f 2 x f
x f
2
2
1 i i 1 i
i
' ' ' '

+
=
' '
+
Centered Finite-Divided Differences
Forward Finite-divided differences
Backward finite-divided differences
First Derivatives
3 -point Forward difference


3 -point Backward difference

) x ( f
'
i-2 i-1 i i+1 i+2
2 i i
2 i 1 i i
i 2 i
i 1 i 2 i
x x
x f 3 x f 4 x f 3
x f
x x
x f 3 x f 4 x f
x f

+
+ +

+
~ '

+
~ '
) ( ) ( ) (
) (
) ( ) ( ) (
) (
Parabolic curve
Example: First Derivatives
Use forward and backward difference approximations
of O(h
2
) to estimate the first derivative of

at x = 0.5 with h = 0.25 (exact sol. = -0.9125)
Forward Difference



Backward Difference
2 . 1 x 25 . 0 x 5 . 0 x 15 . 0 x 1 . 0 ) x ( f
2 3 4
+ =
% 82 . 5 , 859375 . 0
5 . 0
) 925 . 0 ( 3 ) 6363281 . 0 ( 4 2 . 0
) 25 . 0 ( 2
) 5 . 0 ( f 3 ) 75 . 0 ( f 4 ) 1 ( f
) 5 . 0 ( f
= =
+
=
+
= '
t


c
% 77 . 3 , 878125 . 0
5 . 0
2 . 1 ) 035156 . 1 ( 4 ) 925 . 0 ( 3
) 25 . 0 ( 2
) 0 ( f ) 25 . 0 ( f 4 ) 5 . 0 ( f 3
) 5 . 0 ( f
= =
+
=
+
= '
t
c
Higher Derivatives
All second-order accurate O(h
2
)








More nodal points are needed for higher derivatives
Higher order formula may be derived
4
2 i 1 i i 1 i 2 i
i
3
2 i 1 i 1 i 2 i
i
2
1 i i 1 i
i
1 i 1 i
i
h
) x ( f ) x ( f 4 ) x ( f 6 ) x ( f 4 ) x ( f
) x ( f
h
) x ( f ) x ( f 2 ) x ( f 2 ) x ( f
) x ( f
h
) x ( f ) x ( f 2 ) x ( f
) x ( f
h 2
) x ( f ) x ( f
) x ( f
+ +
+ +
+
+
+ +
=
' ' ' '
+
=
' ' '
+
=
' '

=
'
19.3 Richardson Extrapolation
2 4 2
1 2 1 4 1 1 1 1
( ) ........ ( ) ( ) (1) D D h a h a h D h E h = + + + = +
D is the true value but unknown and D(h
1
) is an approximation
based on the step size h
1
. Reducing the step size to half, h
2
=h
1
/2,
we obtained another approximation D(h
2
).
By properly combining the two approximations, D(h
1
) & D(h
2
),
the error is reduced to O(h
4
).
2 4 2
2 2 2 4 2 2 2 2
2
4
1 1 2 1
3 1
2
2 2
( ) ( ) ( ) (2)
( ) 4 ( ) ( )
4 Hence 4*(2) - (1), + ( )
3
( )
D D h a h a h D h E h
E h D h D h
D E h
E h
= + + + = +

~ =
Example of using Richardson Extrapolation
Central Difference Scheme

By combining the two approximations, D(h/2) & D(h), the error
of f(h) is reduced to O(h
4
).
( )
( )
2
4
2
4
1
'( ) [ ( ) ( )] '''( ) ( ) (1)
2 6
1
'( ) / 2 [ ( / 2) ( / 2)] '''( ) ( ) (2)
24
h
f x D h f x h f x h f x O h
h
h
f x D h f x h f x h f x O h
h
= = + +
= = + +
4
4 ( / 2) ( )
4*(2) - (1), ' ( )
3
D h D h
f O h

= +
Ex19.2: Richardson Extrapolation
Use central difference approximation to estimate the
first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)



2 . 1 x 25 . 0 x 5 . 0 x 15 . 0 x 1 . 0 ) x ( f
2 3 4
+ =
1 t
2 t
2 1
t
(1) (0)
( 0.5) 1.0, 9.6%
1 0
(0.75) (0.25)
( 0.25) 0.934, 2.4%
0.75 0.25
4
0.9125, 0%
3
f f
D h
f f
D h
D D
D
c
c
c

= = = =

= = = =

= = =
General Three-Point Formula
Lagrange interpolation polynomial for
unequally spaced data





) (
) )( (
) )( (
) (
) )( (
) )( (
) (
) )( (
) )( (
) ( ) ( ) ( ) ( ) ( ) ( ) (
1 i
i 1 i 1 i 1 i
i 1 i
i
1 i i 1 i i
1 i 1 i
1 i
1 i 1 i i 1 i
1 i i
1 i 1 i i i 1 i 1 i
x f
x x x x
x x x x
x f
x x x x
x x x x
x f
x x x x
x x x x
x f x L x f x L x f x L x f
+
+ +

+
+

+
+
+ +


+


+


=
+ + =
Lagrange Interpolation
1st-order Lagrange polynomial


Second-order Lagrange polynomial

) ( ) ( ) ( ) ( ) ( ) (
1
0 1
0
0
1 0
1
1 1 0 0 1
x f
x x
x x
x f
x x
x x
x f x L x f L x f

= + =
) x ( f
) x x )( x x (
) x x )( x x (
) x ( f
) x x )( x x (
) x x )( x x (
) x ( f
) x x )( x x (
) x x )( x x (
) x ( f
2
1 2 0 2
1 0
1
2 1 0 1
2 0
0
2 0 1 0
2 1
2


+


+


=
Lagrange Interpolation
Third-order Lagrange polynomial



) x ( f
) x x )( x x )( x x (
) x x )( x x )( x x (
) x ( f
) x x )( x x )( x x (
) x x )( x x )( x x (
) x ( f
) x x )( x x )( x x (
) x x )( x x )( x x (
) x ( f
) x x )( x x )( x x (
) x x )( x x )( x x (
) x ( f
3
2 3 1 3 0 3
2 1 0
2
3 2 1 2 0 2
3 1 0
1
3 1 2 1 0 1
3 2 0
0
3 0 2 0 1 0
3 2 1
3


+


+


+


=
Lagrange Interpolation
L
1
(x)f(x
1
)
L
2
(x)f(x
2
)
L
0
(x)f(x
0
)
x
0
x
1
x
2
General Three-Point Formula
Lagrange interpolation polynomial for
unequally spaced data





First derivative
) )( (
) )( (
) (
) )( (
) )( (
) (
) )( (
) )( (
) (
) ( ) ( ) ( ) ( ) ( ) ( ) (
i 1 i 1 i 1 i
i 1 i
1 i
1 i i 1 i i
1 i 1 i
i
1 i 1 i i 1 i
1 i i
1 i
1 i 1 i i i 1 i 1 i
x x x x
x x x x
x f
x x x x
x x x x
x f
x x x x
x x x x
x f
x f x L x f x L x f x L x f


+


+


=
+ + =
+ +

+
+
+
+
+

+ +
) )( (
) (
) )( (
) (
) )( (
) ( ) (
i 1 i 1 i 1 i
i 1 i
1 i
1 i i 1 i i
1 i 1 i
i
1 i 1 i i 1 i
1 i i
1 i
x x x x
x x x 2
x f
x x x x
x x x 2
x f
x x x x
x x x 2
x f x f


+


+


= '
+ +

+
+
+
+
+

Second Derivative
First Derivative for unequally spaced data




Second Derivative for unequally spaced data
) )( (
) (
) )( (
) (
) )( (
) ( ) (
i 1 i 1 i 1 i
i 1 i
1 i
1 i i 1 i i
1 i 1 i
i
1 i 1 i i 1 i
1 i i
1 i
x x x x
x x x 2
x f
x x x x
x x x 2
x f
x x x x
x x x 2
x f x f


+


+


= '
+ +

+
+
+
+
+

) )( (
) (
) )( (
) (
) )( (
) ( ) (
i 1 i 1 i 1 i
1 i
1 i i 1 i i
i
1 i 1 i i 1 i
1 i
x x x x
2
x f
x x x x
2
x f
x x x x
2
x f x f

+

+

= ' '
+ +
+
+ +

Differentiation of Noisy Data

MATLABs Methods
Derivatives are sensitive to the noise
Use least square fit before taking derivatives

p = polyfit(x, y, n) - coefficients of P
n
(x)
polyfit(p, x) - evaluation of P
n
(x)
polyder(p) - differentiation

| |
| |

+

= =
=
=
difference backward 1) x(i at dy/dx
difference forward x(i) at dy/dx
iff(x) diff(y)./d dy
) 1 n ( x ) n ( x , ), 2 ( x ) 3 ( x ), 1 ( x ) 2 ( x ) x ( diff
) n ( x , ), 2 ( x ), 1 ( x x