EE 616

Computer Aided Analysis of Electronic Networks

Lecture 5

Instructor: Dr. J. A. Starzyk, Professor

School of EECS
Ohio University
Athens, OH, 45701

09/19/2005
Note: materials in this lecture are from
the notes of EE219A UC-berkeley
http://www-
1 cad.eecs.berkeley.edu/~nardi/EE219A/contents.html
 Outline

Nonlinear problems
Iterative Methods
Newtons Method
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
Multidimensonal Newton Method
Basic Algorithm
Quadratic convergence
Application to circuits

2
 DC Analysis of Nonlinear Circuits - Example

1
Vd
I1 Ir I I d I s (e Vt
1) 0
d
0

Need to Solve

I r I d I1 0
e1
1
e1 I s (e 1) I1 0
Vt
g (e1 ) I1
R
3
 Nonlinear Equations

Given g(V)=I
It can be expressed as: f(V)=g(V)-I

Solve g(V)=I equivalent to solve f(V)=0

Hard to find analytical solution for f(x)=0

Solve iteratively

4
 Nonlinear Equations Iterative Methods

Start from an initial value x0

Generate a sequence of iterate xn-1, xn, xn+1
which hopefully converges to the solution x*
Iterates are generated according to an
iteration function F: xn+1=F(xn)

Ask
When does it converge to correct solution ?
What is the convergence rate ?
5
 Newton-Raphson (NR) Method

Consists of linearizing the system.

Want to solve f(x)=0 Replace f(x) with its linearized version
and solve.
df *
f ( x) f ( x ) ( x )( x x* )
*
Taylor Ser ies
dx
k 1 df k k 1 k
f ( x ) f ( x ) ( x )( x x )
k

dx
1
k 1 df k
x x
k
( x ) f (xk ) Iteration function
dx
Note: at each step need to evaluate f and f
6
 Newton-Raphson Method Graphical View

7
 Newton-Raphson Method Algorithm

Define iteration
Do k = 0 to .?
1
k 1 df k
x x k
( x ) f (xk )
dx
until convergence

How about convergence?

An iteration {x(k)} is said to converge with order q if there exists a
vector norm such that for each k N:

k 1 q
x x x x
k
8
 Newton-Raphson Method Convergence

2
df d f
0 f ( x ) f ( x ) ( x )( x x ) 2 ( x%
* k k * k
)( x* x k ) 2
dx dx
some x% [ x k , x* ]

Mean Value theorem

truncates Taylor series
But
df k k 1 k by Newton
0 f ( x ) ( x )( x x )
k

dx definition
9
 Newton-Raphson Method Convergence

Subtracting
df k k 1 * d2 f
( x )( x x ) 2 ( x% )( x k x* ) 2
dx d x
2
df d f
Dividing k 1 * k 1
( x x ) [ ( x )] 2
( %
x )( x k
x * 2
)
dx d x
df k 1 d 2 f
Let [ ( x )] 2
( x) K
% k

dx d x
k 1 * 2
then x x K x x
* k k

Convergence is quadratic
10
 Newton-Raphson Method Convergence

Local Convergence Theorem

If
df
a) bounded away from ze ro
dx
2 K is bounded
d f
b) bounded
dx 2

Then Newtons method converges given a

sufficiently close initial guess
(and convergence is quadratic)
11
 Newton-Raphson Method Convergence

x 0 = Initial Guess, k 0
Repeat {
f x k
x
x k 1
xk f xk

k k 1
} Until ?

x k 1
x threshold ?
k
f x k 1
threshold ?

12
 Newton-Raphson Method Convergence Check

False convergence since increment in f small but with large increment in X

Need a "delta-x" check to avoid false convergence

f(x)
x k 1 x k xa xr x k 1

k 1 k
x x x *
X

f x k 1 fa

13
 Newton-Raphson Method Convergence Check

False convergence since increment in x small but with large increment in f

Also need an "f x " check to avoid false convergence

f(x)

f x k 1 fa

x*
x k 1 x k X

x k 1 x k xa xr x k 1
14
 Newton-Raphson Method Convergence

15
 Newton-Raphson Method Local Convergence

Convergence Depends on a Good Initial Guess

f(x)

x1 x1
2 0 0 X
x x x

16
 Newton-Raphson Method Local Convergence

Convergence Depends on a Good Initial Guess

17
 Nonlinear Problems Multidimensional Example

v1 i2 + v2b - v2
i1 i3 Nodal Analysis
+ + At Node 1: i1 i2 0
v1b v3b
g v1 g v1 v2 0
- -
At Node 2: i3 i2 0
g v3 g v1 v2 0
Nonlinear
Resistors Two coupled
i g v nonlinear equations
in two unknowns
18
 Multidimensional Newton Method

Problem: Find x such that F x 0

*
*

F ( x) F ( x* ) J ( x* )( x x* ) Taylor Ser ies

F1 ( x) F1 ( x)
x
x
1 N

J ( x) Jacobian Matrix
FN ( x) FN ( x)
x1 x N

k 1 k 1
x x J (x ) F (x )
k k
Iteration function
19
 Multidimensional Newton Method Computational Aspects

Iteration : x k 1 x k J ( x k ) 1 F ( x k )
k 1
Do not compute J ( x ) (it is not sparse).
k 1
Instead solve : k
J ( x )( x x ) F (x )
k k

Each iteration requires:

1. Evaluation of F(xk)
2. Computation of J(xk)
3. Solution of a linear system of algebraic
equations whose coefficient matrix is J(xk) and
whose RHS is -F(xk)
20
 Multidimensional Newton Method Algorithm

x 0 = Initial Guess, k 0
Repeat {

Compute F x k , J F x k

Solve J F x x x F x
k k 1 k k
for x k 1
k k 1

} Until x k 1 x k ,
f x k 1 small enough

21
 Multidimensional Newton Method Convergence

Local Convergence Theorem

If
a)
J F1 x k Inverse is bounded
b) JF x JF y l x y Derivative is Lipschitz Cont

Then Newtons method converges given a

sufficiently close initial guess
(and convergence is quadratic)
22
 Application of NR to Circuit Equations
Companion Network

Applying NR to the system of equations we find

that at iteration k+1:
all the coefficients of KCL, KVL and of BCE of the linear
elements remain unchanged with respect to iteration k
Nonlinear elements are represented by a linearization
of BCE around iteration k
This system of equations can be interpreted as
the STA of a linear circuit (companion network)
whose elements are specified by the linearized
BCE.

23
 Application of NR to Circuit Equations
Companion Network

General procedure: the NR method applied to

a nonlinear circuit (whose eqns are
formulated in the STA form) produces at each
iteration the STA eqns of a linear resistive
circuit obtained by linearizing the BCE of the
nonlinear elements and leaving all the other
BCE unmodified
After the linear circuit is produced, there is no
need to stick to STA, but other methods (such
as MNA) may be used to assemble the circuit
eqns
24
 Application of NR to Circuit Equations
Companion Network MNA templates

Note: G0 and Id depend on the iteration count k

G0=G0(k) and Id=Id(k)
25
 Application of NR to Circuit Equations
Companion Network MNA templates

26
 Modeling a MOSFET
(MOS Level 1, linear regime)

27
 Modeling a MOSFET
(MOS Level 1, linear regime)

gds

28
 DC Analysis Flow Diagram
For each state variable in the system

29
 Implications

Device model equations must be continuous with

continuous derivatives and derivative calculation must be
accurate derivative of function
Watch out for floating nodes (If a node becomes
disconnected, then J(x) is singular)
Give good initial guess for x(0)
Most model computations produce errors in function
values and derivatives.
Want to have convergence criteria || x(k+1) - x(k) || < such that >
than model errors.

30
 Summary

Nonlinear problems
Iterative Methods
Newtons Method
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
Multidimensional Newton Method
Basic Algorithm
Quadratic convergence
Application to circuits

31
 Improving convergence

Improve Models (80% of problems)

Improve Algorithms (20% of problems)

Focus on new algorithms:

Limiting Schemes
Continuations Schemes

32
 Outline

Limiting Schemes
Direction Corrupting
Non corrupting (Damped Newton)
Globally Convergent if Jacobian is Nonsingular
Difficulty with Singular Jacobians

Continuation Schemes
Source stepping
More General Continuation Scheme
Improving Efficiency
Better first guess for each continuation step

33
 Multidimensional Newton Method
Convergence Problems Local Minimum

Local
Minimum

f
At a local minimum, 0
x
34 Multidimensional Case: J F x is singular
 Multidimensional Newton Method
Convergence Problems Nearly singular

f(x)

x1
0
X
x

35 Must Somehow Limit the changes in X

 Multidimensional Newton Method
Convergence Problems - Overflow
f(x)

0
X
x x1

36 Must Somehow Limit the changes in f(x)

 Newton Method with Limiting

Newton Algorithm for Solving F x 0

x 0 = Initial Guess, k 0
Repeat {

Compute F x k , J F x k

Solve J F x x F x
k k 1 k
for x k 1
x k 1 x k limited x k 1

k k 1
} Until
x k 1 , F x k 1 small enough
37
 Newton Method with Limiting
Limiting Methods
Direction Corrupting
limited x k 1 x k 1
xik 1 if xik 1

limited x k 1
sign xik 1 otherwise
i

NonCorrupting Scales differently

different dimensions

limited x k 1 x k 1
x k 1
limited x k 1

Scales proportionally
In all dimensions
min 1, k 1
x

Heuristics, No Guarantee of Global Convergence

38
 Newton Method with Limiting
Damped Newton Scheme
General Damping Scheme

Solve J F x k x k 1 F x k for x k 1

x k 1 x k k x k 1

Key Idea: Line Search

2
Pick to minimize F x x
k k k k 1
2

Fx
2
k 1 T
F x x
k k k 1
F x x
k k k
k x k 1
2

Method Performs a one-dimensional search in

Newton Direction
39
 Newton Method with Limiting
Damped Newton Convergence Theorem

If
a)
J F1 x k Inverse is bounded
b) JF x JF y l x y Derivative is Lipschitz Cont
Then
There exists a set of k ' s 0,1 such that

F x k 1 F x k k x k 1
F xk with <1

Every Step reduces F-- Global Convergence!

40
 Newton Method with Limiting
Damped Newton Singular Jacobian Problem

x2
1 0 X
x 1 xD x

Damped Newton Methods push iterates to local minimums

41 Finds the points where Jacobian is Singular
 Newton with Continuation schemes
Basic Concepts - General setting

Newton converges given a close initial guess

Idea: Generate a sequence of problems, such that a
problem is a good initial guess for the following one

F (x) 0 Solve F% x , 0 where:

a) F% x 0 , 0 0 is easy to solve Starts the continuation

b) F% x 1 ,1 F x Ends the continuation

c) x is sufficiently smooth Hard to insure!
42
 Newton with Continuation schemes
Basic Concepts Template Algorithm
Solve F% x 0 , 0 , x prev x 0

While 1 {
x 0 x prev
Try to Solve F% x , 0 with Newton
If Newton Converged
x prev x , , 2
Else
1
, prev
} 2
43
 Newton with Continuation schemes
Basic Concepts Source Stepping Example

44
 Newton with Continuation schemes
Basic Concepts Source Stepping Example

R
v 1
f v , idiode v v Vs 0
%
R

Vs Diode
+
-

f% v, idiode v 1
Not dependent!
v v R

Source Stepping Does Not Alter Jacobian

45
 Newton with Continuation schemes
Jacobian Altering Scheme

F% x , F x 1 x

Observations
=0 F% x 0 , 0 x 0 0 Problem is easy to solve and
Jacobian definitely nonsingular.
F% x 0 , 0
I
x
=1 F% x 1 ,1 F x 1
Back to the original problem
F% x (1),1)
0 , 0 F x 1 and original Jacobian

46 x x
 Summary
Newtons Method works fine:
given a close enough initial guess
In case Newton does not converge:
Limiting Schemes
DirectionCorrupting
Non corrupting (Damped Newton)
Globally Convergent if Jacobian is Nonsingular
Difficulty with Singular Jacobians

Continuation Schemes
Source stepping
More General Continuation Scheme
Improving Efficiency
Better first guess for each continuation step
47