EEE 305 Lecture 8: Numerical Differentiation

Numerical Differentiation
The mathematical definition of a derivative begins with a difference approximation:
∆𝑦 𝑓 𝑥𝑖 + ∆𝑥 − 𝑓 𝑥𝑖
=
∆𝑥 ∆𝑥
and as Δx is allowed to approach zero, the difference becomes a derivative:
𝑑𝑦 𝑓 𝑥𝑖 + ∆𝑥 − 𝑓 𝑥𝑖
= lim
𝑑𝑥 ∆𝑥→0 ∆𝑥
This can be shown as

Figure 1
Taylor series expansion can be used to generate high-accuracy formulas for derivatives by using
linear algebra to combine the expansion around several points.
Three categories for the formula include forward finite-difference, backward finite-difference, and
centered finite-difference.
Forward finite-difference: Let’s recall that using Taylor’s Series

If we truncate the series after first derivative we get:

The equation can be solved for

This equation is given a formal label in numerical methods—it is called a finite divided difference. It
can be represented generally as:

Or

where Δ fi is referred to as the first forward difference and h is called the step size, that is, the length
of the interval over which the approximation is made. It is termed a “forward” difference because it
1

utilizes data at i and i + 1 to estimate the derivative. The entire term Δf/h is referred to as a first
Page

finite divided difference.

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

Backward Difference Approximation of the First Derivative: The Taylor series can be expanded
backward to calculate a previous value on the basis of a present value, as in

Truncating this equation after first derivative we get:

where the error is O(h), and ∇ fi is referred to as the first backward difference.
Centered Difference Approximation of the First Derivative: A third way to approximate the first

derivative is to subtract Eq. from the forward

Taylor series expansion: to yield

Which can be solved for:

or

Figure below shows the phical depiction of (a) forward, (b) backward, and (c) centered finite-divided-
differenceapproximations of the first derivative.

2
Page

Figure 2 Graphical depiction of (a) forward, (b) backward, and (c) centered finite-divided-differenceapproximations of
the first derivative.

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

3
Page

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

Forward Finite-Difference

Backward Finite-Difference

4
Page

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

Centered Finite-Difference

Unequally spaced data

One way to calculated derivatives of unequally spaced data is to determine a polynomial fit and take
its derivative at a point.
As an example, using a second-order Lagrange polynomial to fit three points and taking its derivative
yields:
2 x  x1  x2 2 x  x0  x2 2 x  x0  x1
f x   f x0   f x1   f x2 
x0  x1 x0  x2  x1  x0 x1  x2  x2  x0 x2  x1 

Derivatives and Integrals for Data with Errors

A shortcoming of numerical differentiation is that it tends to amplify errors in data, whereas
integration tends to smooth data errors.
One approach for taking derivatives of data with errors is to fit a smooth, differentiable function to
the data and take the derivative of the function.
5
Page

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

Figure 3 Illustration of how small data errors are amplified by numerical differentiation: (a) data with no error, (b) data
modified slightly, (c) the resulting numerical differentiation of curve (a), and (d) the resulting differentiation of curve (b)
manifesting increased variability. In contrast, the reverse operation of integration [moving from (d) to (b) by taking the
area under (d)] tends to attenuate or smooth data errors.

The table below shows some functions to implement numerical differentiation and integration in
matlab
Table 1 MATLAB functions to implement (a) integration and (b) differentiation.

• diff(y)./diff(x)
– Returns the difference between adjacent values in y divided by the corresponding
difference in adjacent values of x
6

• fx = gradient(f, h)
Page

Determines the derivative of the data in f at each of the points. The program uses forward

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 8: Numerical Differentiation

difference for the first point, backward difference for the last point, and centered difference
for the interior points. h is the spacing between points; if omitted h=1.
• The major advantage of gradient over diff is gradient’s result is the same size as the original
data.
• Gradient can also be used to find partial derivatives for matrices:
[fx, fy] = gradient(f, h)
Visualization
MATLAB can generate contour plots of functions as well as vector fields. Assuming x and y represent
a meshgrid of x and y values and z represents a function of x and y,
– contour(x, y, z) can be used to generate a contour plot
– [fx, fy]=gradient(z,h) can be used to generate partial derivatives and
– quiver(x, y, fx, fy) can be used to generate vector fields

1. Chapra chapter 23 exercise 23.1, 23.2, 23.3, 23.8

2. Chapra example 23.1

7
Page

Prepared BY
Shahadat Hussain Parvez