Numerical Methods for Engineers Lecture 4

K. Veroy-Grepl Aachen Institute for Advanced Study in Computational Engineering Science Winter Semester 2010/11

Overview
Finding Roots of Equations
IV. Closed Domain or Bracketing Methods IV. Open Domain Methods A Simple Fixed-Point Iteration B Newton-Raphson Method C Secant Method D Mllers Method E Brents Method V. Roots of Polynomials VI. Systems of Nonlinear Equations

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

1/31

IV.C. Secant Method

Secant vs. Newton-Raphson
Similar to Newton-Raphson in that a new estimate is predicted by extrapolating a tangent to the x-axis. Unlike Newton-Raphson, derivative is approximated For certain functions, derivatives may be dicult or inconvenient to evaluate exactly Derivative is approximated by a backward nite dierence f (xi1 ) f (xi ) f (xi ) = xi1 xi where xi1 = previous or old guess and xi = current guess

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

2/31

IV.C. Secant Method

Newton-Raphson Secant

(Chapra & Canale)

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

3/31

IV.C. Secant Method

Secant vs. Newton-Raphson
Recall the Newton-Raphson formula: xi+1 = xi f (xi ) f (xi )

The formula for the secant method is then: xi+1 = xi f (xi )(xi1 xi ) f (xi1 ) f (xi )

Note: The approach requires two initial estimates, but the estimates are not required to bracket the root

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

4/31

IV.C. Secant Method

Secant vs. False Position
Recall the false position formula: c=b f (b)(b a) f (b) f (a)

Compare with the secant formula: xi+1 = xi Formulas are identical Both use two initial estimates to compute an approximation of the slope of the function Function is used to project to the x-axis for a new estimate
Veroy-Grepl Numerical Methods for Engineers - WS 2010/11 5/31

f (xi )(xi1 xi ) f (xi1 ) f (xi )

IV.C. Secant Method

Secant vs. False Position
Dierence is in how one of the initial values is replaced by the new estimate False Position New estimate replaces initial value whose function value has the same sign Root is always bracketed Method always converges Secant Replaces the values in sequence: xi+1 replaces xi , xi replaces xi1 Root may not be bracketed May sometimes diverge

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

6/31

IV.C. Secant Method

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

7/31

IV.C. Secant Method

Secant vs. False Position

When the secant method does converge, it does so at a quicker rate than the false-position method In false-position, one end may stay xed to maintain the bracketing of the root. Advantage of false-position is that it prevents divergence. Disadvantage of false-position is that convergence may be slowed.

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

8/31

IV.C. Secant Method

Modied Secant Method
Rather than use two (arbitrary) values to estimate the derivative, use a fractional perturbation: f (xi + xi ) f (xi ) f (xi ) = xi where = small perturbation fraction (say, 0.01) The new estimate is then: xi+1 = xi f (xi )xi f (xi + xi ) f (xi )

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

9/31

IV.C. Secant Method

Modied Secant Method

Choice of a proper value of not automatic If is too small, method may be swamped by round-o error due to subtractive cancellation in the denominator f (xi + xi ) f (xi ) If is too big, technique can become inecient and even divergent.

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

10/31

IV.D. Mllers Method

Similar to the secant method, which uses linear interpolation Main idea: Use three points to determine a quadratic function

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

11/31

IV.D. Mllers Method

Similar to the secant method, which uses linear interpolation Main idea: Use three points to determine a quadratic function

Would conceivably yield a better estimate than linear interpolation

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

11/31

IV.D. Mllers Method

(DERIVATION)

New estimate is given by: x3 = x2 + 2c b b2 4ac

Problem: Method yields two roots Sign is chosen to agree with the sign of b Results in largest denominator, giving the root estimate closest to x2 Given x3 , which point is discarded? Choose two original points that are nearest to x3 (usually for real roots) Use sequential approach (usually for real and complex roots)

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

12/31

IV.E. Brents Method

A hybrid approach Combines the reliability of bracketing with the speed of open methods Applies a fast open method wherever possible, but reverts to a reliable bracketing method if necessary Developed by R. Brent (1973) based on an earlier algorithm of T. Dekker (1969) Method used by Matlabs fzero function Bracketing method: bisection method Open methods: secant method and inverse quadratic interpolation

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

13/31

IV.E. Brents Method

Inverse Quadratic Interpolation Similar to Mllers method, which uses quadratic interpolation Main idea: Use three points to determine a quadratic function Note: Parabola might not intersect the x-axis!

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

14/31

IV.E. Brents Method

Inverse Quadratic Interpolation Solution: Use an inverse or sideways parabola (PICTURE) Instead of taking y = x2 , take x = y 2 Let x = g (y ) where g (y ) is a quadratic function passing through the given points (xa , ya ), (xb , yb ), (xc , yc ). How do we determine such a function? (DERIVATION)

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

15/31

IV.E. Brents Method

Inverse Quadratic Interpolation Lagrange polynomial g (y ) = (y yb )(y yc ) (ya yb )(ya yc ) + + xa xb xc

(y ya )(y yc ) (yb ya )(yb yc ) (y ya )(y yb ) (yc ya )(yc yb )

Note that g (ya ) = xa , g (yb ) = xb , g (yc ) = xc

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

16/31

IV.E. Brents Method

Inverse Quadratic Interpolation The root then corresponds to y = 0 which yields: xi+1 = g (0) = yb yc (ya yb )(ya yc ) + + ya yc (yb ya )(yb yc ) ya yb (yc ya )(yc yb ) xa xb xc

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

17/31

IV.E. Brents Method

Inverse Quadratic Interpolation There is at most one quadratic polynomial that interpolates three given points (3 equations, 3 unknowns) Generally exhibits superlinear convergence When does it fail? If the three y values are not distinct. (PICTURE) If this happens, the algorithm reverts to the less ecient secant method to generate a root using two of the points.

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

18/31

Overview
Finding Roots of Equations
IV. Closed Domain or Bracketing Methods IV. Open Domain Methods V. Roots of Polynomials A Multiple Roots B Newtons Method C Complex Roots D Other Methods VI. Systems of Nonlinear Equations

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

19/31

V. Roots of Polynomials
Polynomials:
A common form of nonlinear equation Applications in which the roots of polynomials are sought: Characteristic equation in eigenproblems Curve-tting of data Characteristic equation of higher-order ordinary dierential equations ... One can generally use any of the root-nding methods But there are some features or techniques special to polynomials

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

20/31

V. Roots of Polynomials
Properties of Polynomials:
General form of an nth-degree polynomial: Pn (x) = a0 + a1 x + a2 x2 + an xn where a0 to an are constant coecients, and may be real or complex An nth-degree polynomial has exactly n roots, which may be real or complex If the coecients are real, then complex roots always occur in conjugate pairs Roots may be single (or simple), or multiple (or repeated)

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

21/31

V. Roots of Polynomials
Properties of Polynomials:
The roots of a linear polynomial can be determined directly (analytically) The roots of a second-degree polynomial can also be determined directly. Exact formulas for the roots of third- and fourth-degree polynomials also exist, but are quite complicated and rarely used Iterative methods are generally used to nd the roots of higher-degree polynomials High-degree polynomials can be ill-conditioned sensitive to small changes in the values of the coecients

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

22/31

V. Roots of Polynomials
Example: Consider the factored fth-degree polynomial f (x) = (x 1)(x 2)(x 3)(x 4)(x 5) which has ve positive real roots, 1, 2, 3, 4, and 5. Expanding the polynomial yields f (x) = 120 + 274x 225x2 + 85x3 15x4 + x5 Consider a new fth-degree polynomial g (x) = 120 + 274x 226x2 + 85x3 15x4 + x5 . The ve roots are now: 1.0514, 1.6191, 5.5075, 3.4110 + i1.0793, and 3.4110 i1.0793

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

23/31

V.A Multiple Roots

A multiple root corresponds to a point where not only the function value is zero, but also the higher derivatives A double root occurs if the rst derivative is also zero. A triple root occurs if the rst and second derivative are also zero. ... Odd roots cross the x-axis, even roots do not.

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

24/31

IV.A. Multiple Roots

Multiple roots pose some diculties for root-nding methods: Function does not change sign at even multiple roots, precluding the use of bracketing methods, limiting one to open methods The fact that f (x) is also zero poses problems for the Newton-Raphson and secant methods. Ralston and Rabinowitz (1978) Demonstrates that f (x) will reach zero before f (x) Shows that Newton-Raphson and secant methods are linearly, rather than quadratically, convergent for multiple roots Proposes alternative forms of the Newton-Raphson method to deal with multiple roots

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

25/31

V.B. Newtons Method

V.B.1. NM for Simple Roots
Standard method can be applied directly to nd simple roots of polynomials An extremely rapid procedure as long as a reasonable initial approximation is available. Once a root has been found, how can we avoid convergence to the same root? How can we nd the other roots systematically?

V.B.2. Polynomial Deation

Deate the original polynomial by removing the known root (Note: This can of course also be applied with other open methods.)

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

26/31

V.B. Newtons Method

V.B.2. Polynomial Deation
Example: Consider the third-order polynomial f3 (x) = x3 6x2 + 11x 6 Assume that one root, x = 1, is known or has been found. Dividing the original polynomial by the linear factor corresponding to the known root yields a second-order polynomial g2 (x) = f3 (x) x1 = x2 5x + 6

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

27/31

V.B. Newtons Method

V.B.2. Polynomial Deation
Solve for the unknown roots of the deated polynomial by repeated application of the deation procedure (until the deated polynomial is quadratic).

V.B.3. NM for Multiple Roots

Recall: RR (1978) shows that a nonlinear function f (x) apporaches zero faster than the derivative f (x) Standard Newtons method can be used, but care must be taken to discontinue the iterations as f (x) approaches zero Rate of convergence drops to rst-order for a multiple root Variations or modications of the standard Newtons method restore quadratic convergence

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

28/31

V.B. Newtons Method

V.B.4. NM for Multiple Roots: Alternative #1 [RR]

Includes the multiplicity m in the formula for the new estimate xi+1 = xi m f (xi ) f (xi )

This slight change in the Newton formulation returns it to quadratic convergence (DERIVATION) Requires foreknowledge of the multiplicity of the root

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

29/31

V.B. Newtons Method

V.B.5. NM for Multiple Roots: Alternative #2
Apply Newtons method to a new function u(x) dened as: u(x) = f (x ) f (x)

[RR]

Also restores quadratic convergence, since u(x) has a single root (DERIVATION) Disadvantages: Requires additional eort, especially for calculation of f (xi ) Round-o errors may be introduced due to denominator Less ecient than standard Newtons method for simple roots

Veroy-Grepl

Numerical Methods for Engineers - WS 2010/11

30/31

V.C. Complex Roots

Recall that roots may be complex even if the polynomial coecients are real (in this case the complex roots occur in conjugate pairs) Newtons, secant, or Mllers method may be used to nd complex roots of polynomials Complex arithmetic must be used and reasonably good complex initial guesses must be specied. Some methods for extracting complex roots of polynomials with real coecients which do not require complex arithmetic: Bairstows method QD (quotient-dierence) method Graees method For polynomials with complex coecients, Newtons method or the secant method using complex arithmetic and complex initial guesses are the methods of choice
Veroy-Grepl Numerical Methods for Engineers - WS 2010/11 31/31