ECE1647 NonlinearControl

Introduction to Nonlinear Control Systems
Manfredi Maggiore
Systems Control Group
Dept. of Electrical and Computer Engineering
University of Toronto
Fall 2012
ii
iii
Preface
These notes are intended for rst-year graduate students in engineering. They provide a mathematical
introduction to the vast subject of nonlinear control theory from a dynamical systems perspective. Four
main subjects are covered. Chapter 2 introduces the general concept of a dynamical system through
the notion of the phase ow. It is shown that when the state space of a dynamical system is a nite-
dimensional vector space, the phase ow can always be represented by a vector eld associated to
an ordinary differential equation (ODE). The proof of this equivalence will take the reader through a
journey through existence and uniqueness of solutions of ODEs. Chapter 3 presents the basic language
of dynamics allowing one to characterize the steady-state behaviour of nonlinear systems, the qualitative
behaviour of solutions near equilibria, and the stability of closed orbits. Chapter 4 presents Lyapunovs
theory of stability of equilibria and the LaSalle invariance principle. Finally, Chapter 5 provides an
introduction to the theory of stabilization of equilibria for nonlinear control systems. The chapter rst
presents conditions for existence of stabilizing feedbacks, and then focuses on a class of systems, passive
systems, presenting results on passivity-based stabilization. In this setting interesting analogies with
mechanics are made.
Acknowledgements
The author wishes to thank Mireille Broucke, Lacra Pavel, and Bruce Francis for teaching various editions
of this course and providing valuable feedback on these notes. The author is also grateful to the many
students who took ECE1647 since 2006 for suggesting improvements.
Toronto, September 10, 2012
iv
v
Contents
Preface iii
Contents v
Notation ix
1 Mathematical Preliminaries 1
1.1 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Euclidean space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Vector norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Topology of R
n
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.7 Continuity on R
n
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.8 Limits of functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.9 Differentiability on R
n
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.10 Uniform convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Dynamical Systems, Flows, and Vector Fields 19
2.1 Deterministic dynamical systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Finite dimensional phase ows and vector elds . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.1 Phase ows of nite-dimensional dynamical systems . . . . . . . . . . . . . . . . . . 24
2.2.2 Vector Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.3 Existence and uniqueness of solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
vi
2.2.4 Maximal solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.5 Continuity and differentiability with respect to time and initial conditions . . . . . 37
2.2.6 The (local) phase ow generated by a vector eld . . . . . . . . . . . . . . . . . . . . 40
3 Introduction to Dynamics 43
3.1 Equilibria, limit sets, and invariant sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Nagumos invariance condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3 The Poincar-Bendixson Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4 Stability of closed orbits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.5 Linearization of Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.5.1 Stable, Centre, and Unstable Subspaces of Linear Time-Invariant Systems . . . . . . 65
3.5.2 Stable and Unstable Manifolds of Nonlinear Systems . . . . . . . . . . . . . . . . . . 68
3.5.3 The Hartman-Grobman Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4 Stability Theory 75
4.1 Stability of equilibria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.2 Lyapunovs main theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.3 Domain of attraction and its estimation using Lyapunov functions . . . . . . . . . . . . . . 83
4.4 Global asymptotic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.5 LaSalles Invariance Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.6 Stability of LTI systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.7 Linearization and exponential stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.8 Converse stability theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5 Introduction to Nonlinear Stabilization 101
5.1 Regular Feedbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2 Control-Lyapunov functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.3 Artstein-Sontag Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.4 Brocketts necessary conditions for continuous stabilizability . . . . . . . . . . . . . . . . . 108
5.5 Passive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.6 Passivity-Based Stabilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.7 Application to Hamiltonian control systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.8 Damping Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
vii
Bibliography 125
viii
ix
Notation
The empty set
Z The set of integers
N The set of natural numbers, 1, 2, . . .
Z
+
The set of nonnegative integers, 0, 1, 2, . . .
R The set of real numbers
R
+
The set of nonnegative real numbers
R
n
The set of all ordered n-tuples of real numbers
R
nm
The set of all n m matrices
n The index set 1, . . . , n
ij
Kronecker delta:
ij
= 1 if i = j,
ij
= 0 if i ,= j
a b Statement A is equivalent to statement B
a = b Statement A implies statement B
diag(a
1
, . . . , a
n
) Diagonal matrix whose diagonal elements are a
1
, . . . , a
n
min
(Q) The smallest eigenvalue of the symmetric matrix Q
max
(Q) The largest eigenvalue of the symmetric matrix Q
I
n
The n n identity matrix
(Q)
ij
The ij-th element of matrix Q
v, v
i
If v is a vector, v
i
denotes its i-th component
|x| The norm of the vector x 3
|x|
p
The p-norm of the vector x 3
|A| The (induced) norm of the matrix A 4
|A|
p
The induced p-norm of the matrix A 5
Q(x) The quadratic form Q(x) = x
Qx 5
B
n
(p) Open ball of radius centred at p R

n
8
B
n
(p) Closed ball of radius centred at p R

n
8
x S x is an element of the set S
S x The set S contains the element x
S
1
S
2
Set S
1
is a subset of set S
2
S
1
S
2
Set S
2
is a subset of set S
1
S
1
S
2
Intersection of sets S
1
and S
2
x
S
1
S
2
Union of sets S
1
and S
2
S
c
The complement of the set S 8
S The closure of the set S 8

S The boundary of the set S 8
supS The supremum of the set S R 12
inf S The inmum of the set S R 12
T
(x) The Bouligand tangent cone to at x 50

f g(x) f (g(x))
f [
The restriction of the function f to a set 2

f () The image of the set under f 2
lim
xx
0
f (x) The limit of f (x) as x tends to x
0
12
limsup
tt
0
f (t) The upper limit of f (t) as t tends to t
0
13
liminf
tt
0
f (t) The lower limit of f (t) as t tends to t
0
13
df
x
0
The derivative map or differential of the function f at x
0
13
d f
x
0
The matrix representation of df
x
0
13
L
v
f (x) If v is a vector and f (x) is a differentiable function, this is the
directional derivative of f at x in the direction of v.
13
L
f
(x) If f (x) is a vector eld and (x) is a C
1
function, this is the
Lie derivative of along f .
51
(t, x) The state transition function of a dynamical system 22
t
(x) The t-advance mapping of a dynamical system 22
T
x
0
The domain of denition of the integral curve through x
0
25.
Alternatively, the maximal interval of existence of a solution
with initial condition x
0
35
T
+
x
0
[T
x
0
] The positive [negative] portion of T
x
0
25, 35
T The union of all sets T
x
0
25
O(x
0
) The phase curve (orbit) through x
0
25
O
+
(x
0
) [O
(x
0
)] The positive [negative] semiorbit through x
0
25
L
+
(x
0
) [L
(x
0
)] The positive [negative] limit set of x
0
45
Logic notation and set notation
Refer to Section 3.1 in the notes by Bruce Francis [Fra03]. Here we briey present some sample logic
statements and their explanation.
Logic statement Explanation
(x R)(y R) y < x. For all x R, there exists y R such that y < x. The symbol is
the existential quantier, while is the universal quantier. After the
declaration of the existential quantier (here, y R), always append
the words such that
xi
(c > 0)(x R) f (x) < c. There exists c > 0 such that for all x R, f (x) < c
(x R)(c > 0) f (x) < c. For all x R, there exists c > 0 such that f (x) < c. Compare this
statement to the previous one. Changing the order of the existential
and universal quantiers radically changes the meaning of the state-
ment: the previous statement means that f is everywhere bounded
below c, while this statement simply means that for every x R f (x)
takes non-innite values
(x R) x > 0 = f (x) <
0
For all x R, x > 0 implies that f (x) < 0. Equivalently, for all x R
with x > 0, f (x) < 0.
Next we present two denitions of sets and their meaning.
Set denition Explanation
x R : [x[ > 2, e
x
< 1 The set of points in R such that [x[ > 2 and e
x
< 1. Here : means
such that and , means and
k Z : (j Z) e
k
= j The set of all integers k such that there exists an integer j such that
e
k
= j. Equivalently, the set of all integers whose exponential is also
an integer.
xii
1
Chapter 1
Mathematical Preliminaries
This chapter provides an overview of the basic mathematical notions needed in these notes. All deni-
tions and theorems are standard; their proof are omitted. Consult [Fra03], [KF70], [Roy88], and [Rud76]
for more details.
1.1 Functions
Consider two sets A and B. A function f with domain A and codomain B is a unique assignment, to
each element x in A, of an element f (x) in B. Thus, three objects are needed to dene a function: its
domain, its codomain, and the assignment f . The relationship just described between these three objects
is denoted by f : A B. There are several ways to specify the assignment f . The two most commonly
used are:
(i) Write f (x) = expression
(ii) Write x expression.
Thus, to dene the function that given a real number computes its square value, one may write
f : R R, f (x) = x
2
,
or, equivalently,
f : R R, x x
2
.
One can dene a function without using a label for it by writing, for instance, the function R R, x
x
2
. The x y notation is useful when dealing with functions of several variables; use it to concisely
dene a function of a subset of variables, keeping the rest of the variables as parameters. For example,
suppose we want to use f : R
2
R, f (x, y) = x + y to dene the function g(x) obtained by viewing y
as a parameter in f . We can do that by writing
g : R R, x f (x, y).
This version: December 7, 2012
2 CHAPTER 1. MATHEMATICAL PRELIMINARIES
If f : A B is a function, and is a subset of A, then f [
denotes the restriction of f to , so we have

a new function f [
: B.
Denition 1.1. Consider a function f : A B and a subset A. The image of under f is the set
f () dened as follows
f () = y B : (x ) f (x) = y.
The set f (A) is the image of f .
Denition 1.2. The graph of a function f : A B is the set graph( f ) A B dened as graph( f ) :=
(x, y) : x A, y = f (x).
Denition 1.3. A function f : A B is injective (one-to-one) if, given a, b A,
a ,= b = f (a) ,= f (b).
A function f : A B is surjective (onto) if f (A) = B. A function f : A B which is both injective and
surjective is bijective, or a one-to-one correspondence. If A and B are vector spaces, and f : A B is
bijective and linear, then f is called an isomorphism.
Denition 1.4. Let f : A B, then g : B A is a left-inverse for f if
(a A) g f (a) = a.
Further, h : B A is a right-inverse for f if
(b B) f h(b) = b.
Lemma 1.5. Let f : A B, with A ,= , then f has a left inverse [or right inverse] if and only if f is injective
[or surjective].
Corollary 1.6. Let f : A B. If and only if there exists a left inverse g : B A and a right inverse h : B A
for f , then f is bijective and g = h = f
1
.
1.2 Euclidean space
Given a positive integer n, let R
n
denote the set of all ordered n-tuples x = (x
1
, . . . , x
n
), where x
1
, . . . , x
n
are real numbers. We write 0 R
n
to indicate the vector of zeros. The set R
n
is given a vector space
1.3 VECTOR NORMS 3
structure over the eld R as follows. Given x, y R
n
and R, dene
x + y = (x
1
+ y
1
, . . . , x
n
+ y
n
) R
n
x = (x
1
, . . . , x
n
) R
n
.
We give R
n
the inner product operation
x, y =
n
i=1
x
i
y
i
.
Representing x and y as column vectors we have x, y = x
y. The vector space R

n
with this inner
product operation is called Euclidean n-space. The distance between two points x and y in R
n
is |x
y| =
_
x y, x y.
1.3 Vector norms
Denition 1.7. The norm of a vector x R
n
is a function | | : R
n
R satisfying the three properties
(i) (x R
n
) |x| 0 and |x| = 0 x = 0 (positivity)
(ii) (x, y R
n
) |x + y| |x| +|y| (triangle inequality)
(iii) (x R
n
)( R) |x| = [[|x| (homogeneity).
Using the triangle inequality, one can show that the following useful inequality holds
(x, y R
n
)|x + y| [|x| |y|[.
Denition 1.7 can be used in much greater generality to introduce the concept of norm on much more
general sets than R
n
. We will not need this greater level of generality in these notes.
The distance function on the Euclidean n-space, |x| =
_
x, x, is in fact a norm. It is called the
Euclidean norm. Another example of a vector norm is the p-norm, with 1 p . For 1 p < , it is
dened as
|x|
p
= ([x
1
[
p
+ +[x
n
[
p
)
1/p
.
For p = , it is dened as
|x|
= max
i
[x
i
[.
According to the denition above, |x|
2
coincides with the Euclidean norm.
Exercise 1.1. Show that all p-norms are, in fact, norms.
Exercise 1.2. Draw the level sets
x R
2
: |x|
1
= 1, x R
2
: |x|
2
= 1, x R
2
: |x|
= 1.
It can be shown that if p and q are two positive integers such that p q then
|x|
q
|x|
p
.
Exercise 1.3. Show that, if p q, then |x|
q
|x|
p
. Use this fact to show that, if p q, then
x R
n
: |x|
p
< 1 x R
n
: |x|
q
< 1.
The next inequality, known as Holders inequality, is often useful. For any integers p, q 1 such that
1
p
+
1
q
= 1, and for any x, y R
n
,
[x
y[ |x|
p
|y|
q
.
In particular, when p = q = 2, we obtain Cauchy-Schwarzs inequality
[x
y[ |x|
2
|y|
2
.
Exercise 1.4. Show that Holders inequality is valid even when p = and q = 1.
Exercise 1.5. Show that, for any c > 0 and any x, y R
n
,
x
y
1
2c
|x|
2
2
+
c
2
|y|
2
2
.
Hint: Start by observing that (x y)
(x y) 0. Then, use the fact that x
y = (1/
c x
)(
c y).
The next theorem states that all vector norms are equivalent. It has important implications for the
topological properties of R
n
and in the denitions of continuity and differentiability seen later.
Theorem 1.8. Let | | and

| | be two vector norms on R
n
. Then there exist positive constants a and b such that
(x R
n
) a|x|

|x| b|x|. (1.1)
Exercise 1.6. Understand geometrically Theorem 1.8: show that inequality (1.1) can be equivalently restated as follows
(c > 0)
_
x R
n
: |x|
c
b
_

_
x R
n
:

|x| c
_

_
x R
n
: |x|
c
a
_
.
1.4 Matrix norms
Denition 1.9. The norm of a square matrix A R
nn
is a function | | : R
nn
R satisfying the four
properties
(i) (A R
nn
) |A| 0 and |A| = 0 A = 0 (positivity)
(ii) (A, B R
nn
) |A + B| |A| +|B| (triangle inequality)
(iii) (A R
nn
)( R) |A| = [[|A| (homogeneity)
(iv) (A, B R
nn
) |AB| |A| |B| (sub-multiplicativity).
1.5 QUADRATIC FORMS 5
A matrix norm can be viewed as a vector norm on the vector space of real n n matrices. It follows that
Theorem 1.8 holds for matrix norms as well.
Property (iv) is not strictly needed to dene a matrix norm, but all norms we discuss in this section are
sub-multiplicative.
The class of matrix norms that is most often used is that of induced norms, also called operator norms.
Denition 1.10. Given a vector norm on

| | on R
n
, the induced norm | | on R
nn
is dened as
follows
|A| = max
_
|Ax| : x R
n
,

|x| 1
_
.
Lemma 1.11. The induced norm can be equivalently expressed as
1
|A| = max
_
|Ax| : x R
n
,

|x| = 1
_
= sup
_
|Ax|
|x|
: x R
n
, x ,= 0
_
.
Exercise 1.7. Show that, for all x R
n
,
|Ax| |A|
|x|. Use this result to prove that any induced norm is sub-multiplicative.
When one chooses the vector norm on R
n
to be the p-norm, then the corresponding induced norm is
called the induced p-norm and it is denoted by | |
p
. Thus, if A R
nn
,
|A|
p
= max
_
|Ax|
p
: x R
n
, |x|
p
1
_
.
Of particular interest are the cases p = 1, 2, and ,
|A|
1
= max
j
n
i=1
[a
ij
[
|A|
2
=
_
max
(A
A)
|A|
= max
i
n
j=1
[a
ij
[.
Exercise 1.8. Using Denition 1.10 and Lemma 1.11, prove the three identities above.
1.5 Quadratic forms
Denition 1.12. A quadratic form on R
n
is a map Q : R
n
R dened as
Q(x) = x, Qx = x
Qx,
where Q R
nn
is a symmetric matrix.
1
If S is a subset of R, the notation supS stands for the supremum of the set S, dened in Section 1.8.
Equivalently, a quadratic form on R
n
is a homogeneous polynomial of degree two in n variables. For
instance, a quadratic form on R
2
is a function
Q(x
1
, x
2
) =
_
x
1
x
2
_
_
a b/2
b/2 c
_
. .
Q
_
x
1
x
2
_
= ax
2
1
+ bx
1
x
2
+ cx
2
2
.
Denition 1.13. A quadratic form Q(x) = x
Qx or the associated symmetric matrix Q are said to be

(i) positive denite if (x R
n
) Q(x) 0 and Q(x) = 0 x = 0.
(ii) positive semidenite if (x R
n
) Q(x) 0.
(iii) negative denite if Q(x) is positive denite.
(iv) negative semidenite if Q(x) is positive semidenite.
(v) indenite if Q(x) is neither positive semidenite nor negative semidenite.
We recall two properties of real symmetric matrices:
They are diagonalizable and all their eigenvalues are real.
Their eigenvectors can be chosen to be orthonormal, i.e., if v
1
, . . . , v
n
are the eigenvectors, then
v
i
, v
j
=
ij
for all i, j n.
The following theorem provides a test to check whether a quadratic form (or the associated symmetric
matrix Q) is positive denite.
Theorem 1.14. The following statements are equivalent
(i) Q(x) is positive denite.
(ii) All eigenvalues of Q are positive.
(iii) The principal leading minors M
1
, . . . , M
n
2
of Q are positive.
Example 1.15. Consider the matrix
Q =
_
_
3 0 0
0 2 1
0 1 3
_
_
The principal leading minors of Q are
M
1
= 3, M
2
= det
_
3 0
0 2
_
= 6, M
3
= det Q = 15.
2
The principal leading minor M
k
of Q is the determinant of the rst k rows and columns of Q.
1.5 QUADRATIC FORMS 7
They are all positive so Q is positive denite. Equivalently, we look at the eigenvalues of Q,
3,
5
5
2
.
As expected, they are all positive.
Exercise 1.9. Suppose that all principal leading minors of Q are negative. Does that imply that Q is negative denite?
We now make basic geometric considerations about positive denite quadratic forms on R
n
. The eigen-
values and eigenvectors of the matrix Q associated with a quadratic form Q completely determine the
geometric properties of the level set
Q
1
= x R
n
: Q(x) = 1.
This is most easily seen by using a special coordinate transformation. Let V R
nn
be the matrix
whose columns are the eigenvectors of Q. Since Q is symmetric, its eigenvectors can be chosen to be
orthonormal and so V is an orthonormal matrix, i.e., V
V = I
n
. This way, we have
V
QV = ,
where = diag(
1
, . . . ,
n
) and
i
, i n, are the eigenvalues of Q which we assume to be positive.
Now consider the linear transformation
z = V
x.
It inverse is x = Vz. In z-coordinates, Q takes the normal form
Q(Vz) = z
QVz = z
z =
1
z
2
1
+ +
n
z
2
n
. (1.2)
For i n, let l
i
=
_
1
i
. The linear transformation x z maps the level set Q
1
to a set Q
1
in z coordinates
given by
Q
1
= z R
n
: Q(Vz) = 1 =
_
z R
n
:
_
z
1
l
1
_
2
+ +
_
z
n
l
n
_
2
= 1
_
.
It is clear that Q
1
is an ellipsoid whose i-th semi-axis lies on the z
i
axis and has length l
i
. Moreover,
the smallest hypersphere circumscribing Q
1
has radius R = max
i
l
i
= 1/
_
min
(Q), while the largest
hypersphere inscribed in Q
1
has radius r = min
i
l
i
= 1/
_
max
(Q). An equivalent way of saying this
is to write the inequalities
min
(Q)|z|
2
2
Q(Vz)
max
(Q)|z|
2
2
. Since V is an orthonormal matrix, we
have that |x|
2
2
= x
x = z
Vz = z
z = |z|
2
2
, and so we conclude that
min
(Q)|x|
2
2
Q(x)
max
(Q)|x|
2
2
. (1.3)
Recall that the underlying assumption in the development above is that Q(x) is positive denite. The
inequalities in (1.3) will be used in Chapter 4.
Exercise 1.10. We know that Q
1
is an ellipsoid whose i-th semi-axis lies on the z
i
axis and has length l
i
. Show that Q
1
is also
an ellipsoid, that its i-th axis lies on the subspace spanv
i
(where v
i
denotes the i-th eigenvector of Q), and that its length is
l
i
.
Exercise 1.11. Let
Q =
_
3
2
3
2
3
2
5
2
_
.
Draw the level set Q
4
= x R
2
: Q(x) = 4.
Exercise 1.12. Suppose that a quadratic form Q(x), not necessarily positive denite, has the property that, for all x R
n
,
Q(x) = 0. Prove that Q must be the zero matrix.
1.6 Topology of R
n
Denition 1.16. Consider R
n
and a vector norm | |. Let A be a subset of R
n
.
(i) Let p be an arbitrary point in R
n
. For > 0, the set B
n
(p) = x R
n
: |x p| < is called an
open ball of radius centred at p or a neighborhood of p. The set

B
n
(p) = x R
n
: |x p|
is called a closed ball of radius centred at p.
(ii) A is open if it contains a neighborhood of each of its points, i.e., if
(p A)( > 0) B
n
(p) A.
(iii) A point p A is an interior point of A if there is a neighborhood of p which is contained in A.
(iv) A point p R
n
is a limit point of A if every neighborhood of p contains at least a point q ,= p
which is contained in A, i.e.,
( > 0)(q B
n
(p), q ,= p) q A.
(v) A point p R
n
is a boundary point of A if every neighborhood of p contains at least one point in
A and one point not in A. The set of boundary points is called the boundary of A, denoted by A.
(vi) A is closed if it contains all its limit points. The closure of A is the set

A = A
all limit points of A.
(vii) A is bounded if there exists M > 0 such that A B
n
M
(0).
(viii) A is compact if it is closed and bounded.
(ix) Two sets A, B R
n
are separated if both A

B and

A B are empty.
(x) A is connected if it is not a union of two nonempty separated sets. If A is not connected, the
connected separated subsets of A whose union is A are called the connected components of A.
(xi) A is a domain of R
n
if it is open and connected.
Remark 1.17. One should not confuse the notion of domain of R
n
in Denition 1.16 with the domain of
a function presented in Section 1.1. The domain of a function is the set where the function is dened.
Such set is not necessarily a domain of R
n
in the sense of Denition 1.16, in that it is not necessarily
open and connected. For instance, the domain of the function x

x is [0, ). This set is not a domain
of R because it is not open. Whenever no confusion arises from the context in which the word domain
is used, we will write domain instead of domain of R
n
.
Theorem 1.18 (Closed Set Theorem). A set A R
n
is closed if and only if every convergent sequence x
n
with x
n
A has its limit in A.
1.7 CONTINUITY ON R
n
9
Theorem 1.19. If A R
n
is bounded, then

A is bounded as well.
Exercise 1.13. Show that R
n
and the empty set are both open and closed.
Exercise 1.14. Show that A is closed if and only if the complement A
c
= x R
n
: x , A is open.
Exercise 1.15. Show that A is closed if and only if A contains all its boundary points.
Exercise 1.16. Consider the following subsets of R
2
:
1. Z
2. x R
2
: x
1
= 1/n, n N, x
2
= 0
3. x R
2
: 1 x
1
2
4. x R
2
: 1 < x
1
< 2
5. x R
2
: 1 |x|
2
2.
For each of the sets, answer this questions. Is the set open? Is the set closed? What are the interior points, limit points, and
boundary points of the set?
Exercise 1.17. Is the set A = x R
2
: [x
2
[ < [x
1
[ connected? What about the set A (0, 0)? Find the closure of A.
Exercise 1.18. Consider the subset of the Euclidean plane R
2
, A = x R
2
: 0 < |x|
2
< 1 (2, 2). Find all limit points
and all boundary points of A.
It would seem that the concepts introduced in Denition 1.16 depend upon the choice of a vector norm
| | on R
n
because different vector norms on R
n
give different neighborhoods B
n
(p). Remarkably, this

is not the case. So, for instance, if a set A R
n
is open using the Euclidean norm | |
2
in the denition
of a neighborhood, then it is also open using any p-norm | |
p
for any p N. The reason for the
independence of Denition 1.16 from the choice of vector norm is the result in Theorem 1.8. Recall
that, given any two vector norms | | and

| | on R
n
, Theorem 1.8 states that (see Exercise 1.6) for any
neighborhood in R
n
dened using

| |, call it

B
n
(p), there exist two neighborhoods dened using | |,

B
n
1
(p) and B
n
2
(p), such that
B
n
1
(p)

B
n
(p) B
n
2
(p).
Exercise 1.19. Using the statement of Theorem 1.8 just given, prove that parts (ii)-(vii) and (ix)-(x) in Denition 1.16 are
indepedent of the choice of | |.
1.7 Continuity on R
n
Denition 1.20. Consider a function f : A B, where A and B are domains of R
n
and R
m
, and consider
two vector norms on R
n
and R
m
. For notational simplicity, we denote both norms by | |, even though
generally they may be different. Let x
0
be an arbitrary point in A.
(i) f is continuous at x
0
if
( > 0)( > 0)(x A) x B
n
(x
0
) = f (x) B
m
( f (x
0
)).
If f is continuous at all points of A, then f is continuous.
(ii) f is uniformly continuous if
( > 0)( > 0)(x, x
0
A) x B
n
(x
0
) = f (x) B
m
( f (x
0
)).
(iii) f is Lipschitz continuous at x
0
if
(, L > 0)(x, y B
n
(x
0
)) |f (x) f (y)| L|x y|.
We call L a Lipschitz constant at x
0
. If f is Lipschitz continuous at all points of A then f is said to
be locally Lipschitz on A.
(iv) If D A is a domain of R
n
, f is Lipschitz on D if
(L > 0)(x, y D) |f (x) f (y)| L|x y|.
We call L a Lipschitz constant on D. If D = A, then f is globally Lipschitz.
(v) f is a homeomorphism if it is bijective, continuous, and its inverse, f
1
: B A, is continuous as
well.
Remark 1.21. By virtue of Theorem 1.8, all continuity notions in Denition 1.20 are independent of the
choice of norm (see exercise 1.6). Thus, if a function is found to be continuous using certain vector norms
for its domain and codomain, then the function is continuous when choosing any other vector norm.
A function f is continuous if small variations in x cause small variations in f (x). Note that, according to
the denition in part (i) of a continuous function, the number depends on both x
0
and . On the other
hand, the denition in part (ii) of a uniformly continuous function states that, given > 0, one can use
the same for any x
0
.
Lipschitz continuity at x
0
requires that f stretches or compresses the radius of small neighborhoods
of x
0
by at most a nite amount L. One expects that this notion be related to some property of the
derivative of f , and indeed we will see in Theorem 1.43 of Section 1.9 that this is the case, although
differentiability is not required for Lipschitz continuity. It is easy to see that the notion of Lipschitz
continuity at x
0
is stronger than that of continuity at x
0
, i.e., if f is Lipschitz continuous at x
0
, then it is
also continuous at x
0
. The converse, however, is not true (see Exercise 1.22).
According to part (iii) of Denition 1.20, if f is a locally Lipschitz function on A, then the Lipschitz
constant L depends on x
0
A. On the other hand, if f is Lipschitz on a domain D A (or if it is
globally Lipschitz), then L does not depend on x
0
: f stretches any neighborhood of x
0
in D by the same
amount L. Clearly, this form of continuity is much stronger than local Lipschitz continuity. For instance,
any polynomial function is locally Lipschitz on R, but if it has degree > 1 it is not globally Lipschitz!
Exercise 1.20. Show that f : R R, f (x) = x
2
is locally Lipschitz on R but it is not globally Lipschitz. Show that, however, f
is Lipschitz on the compact interval [0, 1].
Similar considerations can be made when comparing uniform continuity to continuity. The next example
1.7 CONTINUITY ON R
n
11
illustrates the difference between these two concepts.
Example 1.22. Consider the function f : R R, f (x) = x
2
. We pick the absolute value as the norm
on R. We now use part (i) of Denition 1.20 to show that f is a continuous function. Let x
0
be an
arbitrary point in R and pick an arbitrary > 0. We need to nd a > 0 such that if [x x
0
[ < , then
[x
2
x
2
0
[ < . Choose small enough that ( + 2[x
0
[) < . Then,
[x
2
x
2
0
[ = [x x
0
[ [x + x
0
[ = [x x
0
[ [x x
0
+ 2x
0
[ [x x
0
[([x x
0
[ + 2[x
0
[) ( + 2[x
0
[) < .
Now we show that f (x) is not uniformly continuous. Referring to Denition 1.20(ii) we need to show
that
( > 0)( > 0)(x, x
0
A) x B
n
(x
0
) and f (x) , B
m
( f (x
0
)).
In other words, we need to nd an > 0 such that, no matter how small we pick , there exist x and x
0
in
A such that [x x
0
[ < and [x
2
x
2
0
[ . Let = 1 and let > 0 be arbitrary. Let x = x
0
+ /2, so that
[x x
0
[ = /2 < and [x
2
x
2
0
[ = x
0
+
2
/4. Now choosing x
0
sufciently large that x
0
+
2
/4 > 1
we see that f (x) is not uniformly continuous.
Exercise 1.21. Show that f : (0, ) R, f (x) = 1/x, is continuous but not uniformly continuous.
Exercise 1.22. Starting from Denition 1.20, write the denition that a function f : A B is not Lipschitz continuous at x
0
A.
Using this latter denition, prove that f : R R, f (x) = x
1/3
is not Lipschitz continuous at x
0
= 0, despite the fact that it is
continuous at x
0
= 0. The intuition here is that the slope of f is innite at 0.
Exercise 1.23. Is the function f : R R, f (x) = x
3
a homeomorphism?
The next result concerns the special situation when continuity and local Lipschitz continuity can be used
to infer the stronger properties of uniform continuity and Lipschitz continuity.
Theorem 1.23. Consider a function f : A B with A and B domains of R
n
and R
m
. Let be a compact subset
of A. Then,
(i) f continuous = f [
uniformly continuous
(ii) f locally Lipschitz on = f Lipschitz on .
The next result states that continuous real-valued functions admit minima and maxima over a compact
set.
Theorem 1.24. Let f : A B, with A R
n
and B R domains, be a continuous real-valued function. Let
be a compact subset of A. Then f is bounded on , i.e., there exists M > 0 such that [ f [ M. Moreover, there
exist x
min
and x
max
in such that
f (x
min
) = min
x
f (x), f (x
max
) = max
x
f (x).
We conclude this section with a result concerning uniformly continuous functions. It states that a
uniformly continuous function dened on a domain of R
n
has a unique continuous extension dened
on the closure of the domain.
Theorem 1.25. Let f : A B, where A and B are domains of R
n
and R
m
, be a uniformly continuous function.
Then there exists a unique function g :

A B such that g(x) = f (x) for all x A. Moreover, g is uniformly
continuous.
1.8 Limits of functions
Denition 1.26. (i) Consider a function f : A B, where A and B are domains of R
n
and R
m
, and
let x
0
be a point in

A. f is said to have a limit y
0
as x tends to x
0
, written lim
xx
0
f (x) = y
0
, if there
exists y
0
B such that
( > 0)( > 0)(x A) 0 < |x x
0
| < = f (x) B
m
(y
0
).
(ii) Given a function f : R R
m
, we write lim
t+
f (t) = y
0
if there exists y
0
R
m
such that
( > 0)(M > 0)(t R) t > M = f (t) B
m
(y
0
).
(iii) Given a function f : R R
m
, we write lim
t+
f (t) = if
(K > 0)(M > 0)(t R) t > M = f (t) , B
m
K
(0).
(iv) Given a function f : A R
m
, where A is a domain of R
n
, and given a point x
0

A, we write
lim
xx
0
f (x) = if
(K > 0)( > 0)(x R
n
) x B
n
(x
0
) = f (x) , B
m
K
(0).
Remark 1.27. By comparing Denition 1.20(i) and Denition 1.26(i) it is clear that f : A B is continuous
at x
0
if and only if x
0
A and lim
xx
0
f (x) = f (x
0
).
Theorem 1.28. If lim
xx
0
f (x) exists, then it is unique.
Theorem 1.29. Let f : R R be a continuous and nonincreasing function. Assume that f (t) 0 for all t R.
Then, there exists c 0 such that lim
t
f (t) = c.
We now dene the limsup and liminf operations on real-valued functions. We rst require the deni-
tions of sup and inf.
Denition 1.30. Let S be a subset of R. The supremum of S, denoted supS, is the least upper bound
of S, i.e., it is such that (x S) x supS, and (M R)(M < supS = (x S) M < x). The
inmum of S, denoted inf S, is the greatest lower boound of S, dened in an analogous manner.
It is a fact that for any S R, supS and inf S always exist, but they may be equal to + and ,
respectively.
1.9 DIFFERENTIABILITY ON R
n
13
Denition 1.31. Let f : R R be a function, then the upper and lower limits of f are dened as
follows:
limsup
tt
0
f (t) := lim
0
_
supf () : B
1
(t
0
), ,= t
0
_
liminf
tt
0
f (t) := lim
0
_
inff () : B
1
(t
0
), ,= t
0
_
limsup
t
f (t) := lim
t
(supf () : t)
liminf
t
f (t) := lim
t
(inff () : t) .
It is a fact that while the limit of a function may not exist, its upper and lower limits always exist,
although they may be innite. To illustrate, lim
t
sin t is undened, but liminf
t
sin(t) = 1 and
limsup
t
sin(t) = 1. Similarly, lim
t0
[ sin(1/t)[ is undened, but liminf
t0
[ sin(1/t)[ = 0 and
limsup
t0
[ sin(1/t)[ = 1.
1.9 Differentiability on R
n
We begin by recalling the notion of differentiability for real-valued functions on R.
Denition 1.32. A function f : R R is differentiable at x
0
R if
lim
h0
f (x
0
+ h) f (x
0
)
h
= L.
The limit L (if it exists) is called the derivative of f at x
0
and is denoted by f
(x
0
).
In order to generalize this denition to the case of vector-valued functions on R
n
, we rewrite the deni-
tion of derivative in the following equivalent way
lim
h0
[ f (x
0
+ h) f (x
0
) Lh[
[h[
= 0. (1.4)
Denition 1.32 can be rephrased as follows: there exists a linear transformation L : R R, h Lh,
such that (1.4) holds. The generalization is then made by replacing the absolute values by norms and by
letting the derivative be a linear transformation between vector spaces.
Denition 1.33. Consider a function f : A B, where A and B are domains of R
n
and R
m
. Let x
0
be
an arbitrary point in A.
(i) f is differentiable at x
0
if there exists a linear transformation df
x
0
: R
n
R
m
such that
lim
h0
|f (x
0
+ h) f (x
0
) df
x
0
(h)|
|h|
= 0.
The transformation df
x
0
(h) is called the derivative map, or the differential of f . If f is differ-
entiable at every x
0
A, then f is differentiable. The m n matrix representation of df
x
0
(h)
obtained using the natural bases for R
n
and R
m
is the Jacobian of f , denoted by d f
x
0
. Hence,
df
x
0
(h) = d f
x
0
h.
(ii) f is continuously differentiable (C
1
) if it is differentiable and the function A R
mn
, x d f
x
is
continuous.
(iii) Denote by f
i
, i m, the i-th component of f and denote the natural basis of R
n
by e
1
, . . . , e
n
. For
i m and j n, dene
f
i
x
j
: A R as
f
i
x
j
(x
0
) = lim
t0
f
i
(x
0
+ te
j
) f
i
(x
0
)
t
.
If the limit exists, then
f
i
x
j
(x
0
) is the j-th partial derivative of f
i
at x
0
.
(iv) For v R
n
, dene L
v
f : A R
m
as
L
v
f (x
0
) = lim
t0
f (x
0
+ tv) f (x
0
)
t
.
If the limit exists, then L
v
f (x
0
) is the directional derivative of f at x
0
in the direction of v.
(v) f is a diffeomorphism if it is bijective, continuously differentiable, and its inverse, f
1
: B A, is
continuously differentiable as well.
Remark 1.34. In order to check continuity of the map R
n
R
mn
, x d f
x
, one needs to use a vector
norm for R
n
and a matrix norm for R
mn
.
Exercise 1.24. Let A R
nn
and consider the linear transformation A : R
n
R
n
, x Ax. Using the denition, show that
this function is continuously differentiable and its differential is
dA
x
0
(h) = Ah.
Therefore, the matrix representation of the differential of Ax is A.
Exercise 1.25. Using the denition, show that a quadratic form Q(x) : R
n
R is differentiable and that its differential is
dQ
x
0
(h) = 2x
0
Qh,
so that the matrix representation of the differential is dQ
x
0
= 2x
0
Q. Show that Q(x) is in fact C
1
.
Now a series of useful results. First, differentiability implies continuity.
Theorem 1.35. If f : A B is differentiable at x
0
A, then f is continuous at x
0
.
1.9 DIFFERENTIABILITY ON R
n
15
If f is differentiable, then all partial derivatives exist and they characterize the Jacobian of f .
Theorem 1.36. Suppose f : A B (A R
n
and B R
m
domains) is differentiable at x
0
A. Then the partial
derivatives
f
i
x
j
: R
n
R, i m, j n, exist and
(d f
x
0
)
ij
=
f
i
x
j
(x
0
).
Example 1.37. Consider f : R
2
R
2
dened as
f (x
1
, x
2
) =
_
x
2
1
e
x
2
x
1
sin x
2
_
and the point x
0
= (1, 0). It can be shown (Theorem 1.41) that f is differentiable. Its Jacobian is given by
d f
x
0
=
_
f
1
x
1
(x
0
)
f
1
x
2
(x
0
)
f
2
x
1
(x
0
)
f
2
x
2
(x
0
)
_
=
_
2x
1
e
x
2

x
2
1
e
x
2
sin x
2
x
1
cos x
2
_
x
1
=1,x
2
=0
=
_
2 1
0 1
_
.
The familiar chain rule applies to vector functions.
Theorem 1.38 (Chain rule). Consider f : B C and g : A B, where A R
n
, B R
m
, C R
p
are
domains. Suppose that f and g are differentiable. Then, f g : A C is differentiable and
d(f g)
x
0
(h) = df
g(x
0
)
(dg
x
0
(h)).
In terms of matrix representations:
d( f g)
x
0
= d f
g(x
0
)
dg
x
0
.
Example 1.39. Let f : R
2
R
2
and g : R R
2
be dened as follows
f (x
1
, x
2
) =
_
x
1
+ x
2
x
2
2
_
, g(t) =
_
sin t
cos t
_
,
and let t
0
= /2. The composition of f with g is
f g(t) =
_
sin t + cos t
cos
2
t
_
,
and its Jacobian at t
0
is
d( f g)
t
0
=
_
cos t sin t
2 cos t sin t
_
t=/2
=
_
1
0
_
.
Lets verify that we get the same result using the chain rule:
d( f g)
t
0
= d f
g(t
0
)
dg
t
0
=
_
1 1
0 2x
2
_
x
1
=1,x
2
=0
_
cos t
sin t
_
t=/2
=
_
1 1
0 0
_ _
0
1
_
=
_
1
0
_
,
as expected.
When f is differentiable, the Jacobian can be used to compute directional derivatives.
Theorem 1.40. If f is differentiable at x
0
, then L
v
f (x
0
) = df
x
0
(v) = d f
x
0
v.
A function is C
1
if and only if all its partial derivatives exist and are continuous.
Theorem 1.41. A function f : A B, where A and B are domains of R
n
and R
m
, is C
1
if and only if the partial
derivatives
f
i
x
j
: R
n
R, i m, j n, exist and are continuous.
In light of Theorem 1.36, it is tempting to speculate that if the partial derivatives of f exist then f is
differentiable. This conjecture is in general false, as the next example illustrates.
Example 1.42. Consider the function f : R
2
R dened as
f (x
1
, x
2
) =
_
_
x
3
1
x
2
1
+ x
2
2
(x
1
, x
2
) ,= 0
0 (x
1
, x
2
) = 0.
Using the Euclidean norm for R
2
, we next show that f is continuous at 0 (it is obviously continuous
elsewhere). Pick any > 0 and set = . Letting |x 0|
2
< , we have
[ f (x
1
, x
2
) f (0, 0)[ =
x
3
1
x
2
1
+ x
2
2
= [x
1
[
x
2
1
x
2
1
+ x
2
2
[x
1
[
x
2
1
+ x
2
2
x
2
1
+ x
2
2
|x|
2
< .
Exercise 1.26. The partial derivatives of f exist for all x ,= 0, compute them using standard differentiation rules. Show, using
Denition 1.33(iii), that the partial derivatives also exist at x = 0 and they are given by
f
x
1
(0) = 1,
f
x
2
(0) = 0. Verify that the
partial derivatives are not continuous at x = 0.
We now show that despite the fact that f is continuous and its partial derivatives exist, f is not differen-
tiable at x = 0.
Exercise 1.27. Using Denition 1.33(iv), show that, for any v R
2
, L
v
f (0) exists and L
v
f (0) = v
3
1
/(v
2
1
+ v
2
2
).
If f were differentiable at x = 0, then by Theorem 1.40 we would have that, for any v R
2
, L
v
f (0) =
d f
0
v. By virtue of Theorem 1.36, d f
0
= [1 0]. Choose v = [1 1]
. We should have
L
v
f (0) =
1
2
= d f
0
v = 1.
Since the equality is false, f is not differentiable at 0. The problem here is that while the partial deriva-
tives of f exist at x = 0, they are not continuous.
1.10 UNIFORM CONVERGENCE 17
The next two theorems provide an effective means to check for (local) Lipschitz continuity.
Theorem 1.43. Suppose f : A B, where A and B are domains of R
n
and R
m
, is C
1
. Then, f is locally Lipschitz
on A.
Theorem 1.44. Suppose f : A B, where A and B are domains of R
n
and R
m
, is differentiable. Let D be a
convex
3
subset of A, and assume that there exists M > 0 such that |d f
x
| M for all x D (i.e., |d f
x
| is
uniformly bounded on D). Then f is Lipschitz on D.
Below we summarize the logical dependencies among various continuity and differentiability concepts.
As usual, we consider a function f : A B. We assume that A is a convex open set in R
n
and B is a
domain of R
m
.
f is continuous f is locally Lipschitz on A f is Lipschitz on A
f is differentiable f is C
1
f is C
1
and |d f
x
| is uniformly bounded on A
Finally, a generalization of the mean value theorem of calculus to real-valued functions of several vari-
ables.
Theorem 1.45 (Mean Value Theorem). Suppose f : A R (A R
n
domain) is C
1
. Let x, y A be such
that the line segment L connecting x and y is contained in A (i.e., for all [0, 1], (1 )x + y A). Then,
there exists a point z L such that
f (x) f (y) = d f
z
(x y) =
n
i=1
f
x
i
(z)(x
i
y
i
).
1.10 Uniform convergence
In this section we consider sequences of functions f
i
, with f
i
: A B. Here A is any set, while B is a
subset of R
m
.
Denition 1.46. The sequence of functions f
i
iN
converges uniformly on A (notation, f
i
f uni-
formly on A) if
( > 0)(I N)(x A)(i > I) |f
i
(x) f (x)| < .
The convergence is called uniform because, given , the integer I works for all x A. Next, we present
the Cauchy criterion for uniform convergence.
3
D is convex of for any x, y D the segment joining x and y is contained in D. More precisely, (x, y D), ( [0, 1]),
x + (1 )y D.
Theorem 1.47. The sequence of functions f
i
iN
converges uniformly on A if and only if
( > 0)(I N)(j, l > I)(x A) |f
j
(x) f
l
(x)| < .
The limit of uniformly convergent sequences of functions is a continuous function.
Theorem 1.48. Let f
i
iN
be a sequence of continuous functions A B, where A and B are domains of R
n
and R
m
, and suppose that f
i
f uniformly on A. Then, f is continuous on A.
When dealing with uniformly convergent sequences of functions, one can exchange the operations of
limit and integration.
Theorem 1.49. Let f
i
iN
be a sequence of integrable functions on an interval [a, b] R, and suppose that
f
i
f uniformly on [a, b]. Then f is integrable on [a, b] and
_
b
a
f (x)dx = lim
i
_
b
a
f
i
(x)dx.
19
Chapter 2
Dynamical Systems, Flows, and Vector Fields
This chapter introduces the notion of dynamical system as a deterministic process that evolves in time.
The concept of deterministic evolution is captured by the so-called phase ow. The general denition of
dynamical system is later specialized to the nite-dimensional setting. In this context, it is shown that,
under minor technical conditions, a nite-dimensional dynamical system is equivalent to a vector eld.
The approach taken in this chapter is inspired by V.I. Arnold [Arn73]. Additional references for this
material are [Hal80], [Har02] (this is rather advanced), [HS74], [Kha02], and [Pet66]. In particular, the
material in Sections 2.2.3-2.2.6 is adapted from [HS74].
2.1 Deterministic dynamical systems
A deterministic dynamical system is a process that evolves in time in a predictable way. Before giving
formal denitions for the type of dynamical systems presented in these notes, in the next three examples
we build up some intuition.
Example 2.1. Consider a thin rod of length L made of an homogeneous material. Assume its sides are
thermally insulated, while its ends are kept at constant temperatures T
1
and T
2
.
0 L
T
1
T
2
Let u
0
(x) : [0, L] R be a differentiable function representing the temperature distribution in the rod
at time 0. This is the initial condition. Then, the evolution of the temperature distribution as time
progresses is described by the partial differential equation
u
t
= k
2
u
x
2
, k > 0.
This is the one-dimensional heat equation. The unique solution of this PDE is the differentiable function
20 CHAPTER 2. DYNAMICAL SYSTEMS, FLOWS, AND VECTOR FIELDS
u(t, x) : R[0, L] R such that, for all x [0, L] and all t R,
u(0, x) = u
0
(x), u(t, 0) = T
1
, u(t, L) = T
2
.
Here we are not interested in the explicit solution of the PDE, but rather in some of its general properties
deduced by our physical intuition.
The process is deterministic. In order to unequivocally determine the temperature distribution at
any time t > 0, one needs to specify the initial condition at time 0, u
0
(x), and the time t that has
passed since the process was initialized. Furthermore, given an initial condition u
0
(x) at time 0,
one can unequivocally determine the temperature distribution at any time in the past, i.e., at any
time t < 0.
At each time t
1
> 0, the function x u(t
1
, x) contains all the information concerning the evolution
of the system up to time t
1
. Indeed, let u
1
(x) = u(t
1
, x) and view u
1
(x) as a new initial condition.
Then, the temperature distribution any time t
2
> t
1
depends entirely on the initial distribution
u
1
(x) and the time difference t
2
t
1
. So to determine u(t
2
, x) we can either specify the pair
(t
2
, u
0
(x)) or the pair (t
2
t
1
, u
1
(x)).
The two properties in the example above are not peculiar characteristics of the heat equation. They
are shared by all processes that evolve deterministically in time, including mechanical systems obeying
Newtons laws, as the next example illustrates.
Example 2.2. Consider a frictionless pendulum of length l, mass M, and moment of inertia I.
Mg
l
M
Let denote the angle that the pendulum forms with the vertical axis. Newtons equation for this system
is
I

= Mgl sin .
This is an ordinary differential equation. Let (
0
,

0
) be the initial condition: the angle and velocity of
the pendulum at time 0. Then, the differential equation above has a unique solution (t) representing
the evolution of the pendulum as a function of time. As in the case of the heat propagation process, the
evolution of the pendulum from the initial condition (
0
,

0
) is deterministic. For instance, if
0
=

0
= 0,
then it must be that (t) = 0 for all t R, i.e., the pendulum is at equilibrium. If, on the other hand,
0
= /4 and

0
= 0, then the pendulum will oscillate periodically between /4 and /4, and
2.1 DETERMINISTIC DYNAMICAL SYSTEMS 21
has done so in the past as well. Another similarity with the heat propagation process is that there is
a quantity, the vector ((t),

(t)), which summarizes the evolution of the pendulum up to time t. The
difference is that the analogous quantity in the heat propagation example is not a vector, but rather a
differentiable function [0, L] R.
The previous examples are dynamical systems arising from physical processes. The next example
presents a dynamical system, the so-called curve-shortening ow, which does not arise from physics,
but it has remarkable applications in image processing.
Example 2.3. In this example we present the so-called curve-shortening ow for planar curves. Consider
a closed curve on the plane, represented as the image of a T-periodic function x(s), x : R R
2
.
x(s)
x(R)
s
If x(s) is twice continuously differentiable, then the signed curvature of the curve at a point x(s) is a
number k(s) indicating how much a bead traversing the curve turns left or right. More precisely, the
magnitude [k(s)[ is the inverse of the radius of the osculating circle at x(s), i.e., the circle that best
approximates the curve at x(s). The sign of k(s) tells us whether the curve turns left or right: if k(s) > 0
then the curve is convex near x(s), while if k(s) < 0 then that the curve is concave near x(s). We assume
that the curve is regular, i.e., x
(s) ,= 0 for all s R, so that it does not have cusps. At each point x(s),
we attach a normal vector N(s). Noting that the vector x
(s) is tangent to the curve at x(s), N(s) is the

unit vector obtained from x
(s)/|x
(s)| by counter-clockwise rotation by /2.

Now suppose that we evolve the curve x(R) in time, so as to create a family of curves (t, x(s)). The
meaning of the function is this. For a xed t, the function s (t, x(s)) parametrizes the curve
resulting from the evolution of x(R) after t units of time; if we x s, then the function t (t, x(s))
describes the evolution in time of the point x(s). The particular evolution process we consider here is
this: at each time t, the point (t, x(s)) moves with velocity k(s, t)N(s, t), where k(s, t) and N(s, t) are
the signed curvature and normal vector of the curve s (t, x(s)). The evolution process is depicted
below.
(t, x(s))
k(s, t)N(s, t)
Thus, each point of the curve is pulled along the direction of its normal vector, with a speed equal to
the curvature. In particular, points where the curve is convex are pulled in, whereas points where the
curve is concave are pushed out. Moreover, points where the magnitude of the curvature is larger evolve
faster in time. The function (t, x(s)) arising from the evolution process just described is called the
curve-shortening ow. Since the velocity at time t of the point (t, x(s)) is (/t)((t, x(s))), it follows
that (t, x(s)) is the solution at time t of the equation
t
u(s, t) = k(s, t)N(s, t).
with initial condition u(s, 0) = x(s). The above is a PDE because N(s, t) depends on u(s, t). It has
been shown [Gra87] that if x(R) is a regular closed curve without self-intersections, then under the
curve-shortening ow these properties are preserved, the curve tends to a circle, and the circle shrinks
to a point in nite time. Variations of the curve-shortening ow give rise to a family of algorithms
used in computer graphics for curve smoothing and shape recognition [DVM02]. In control theory, the
curve-shortening ow has been used for the control of formations of point-mass vehicles [SBF07].
Examples 2.1, 2.2, and 2.3 share some fundamental properties. All examples describe a process (the
propagation of heat, the motion of a pendulum, the deformation of a closed curve) which evolves in time.
In all examples there is a key quantity summarizing the status of the system at any given time t R
(the function x u(t, x) describing the heat distribution at time t, the vector ((t),

(t)) indicating the
position and velocity of the pendulum at time t, and the function s (t, x(s)) parametrizing the curve
at time t). We call this quantity the state of the system at time t. The set where the state evolves as time
progresses is the state space of the system. Thus, the state space of the heat propagation process is the
set of differentiable functions on the interval [0, L]; for the pendulum, it is
1
R
2
; for the curve-shortening
process, it is the set of closed curves parametrized by twice continuously differentiable functions x(s)
that are T-periodic and such that x
(s) ,= 0. The evolution of the three processes is deterministic in that

the state reached after t units of time is uniquely determined by the initial condition at time 0. We are
now ready for a formal denition of a dynamical system.
Denition 2.4. A deterministic continuous-time autonomous dynamical system (in short, a dynamical
system) is a pair (A, ), where:
(i) A is a set called the state space
(ii) : RA A is the phase ow of (A, ), and enjoys the following properties:
(a) (x
0
A) (0, x
0
) = x
0
(consistency)
(b) (t, s R)(x
0
A) (t, (s, x
0
)) = (t + s, x
0
) (semigroup property).
1
Actually, it is more appropriate to say that the state space of the pendulum is the set (,

) : R(mod2),

R. In
other words, and + 2 are considered to be the same angle. One way of saying that and + 2 are the same object is to
associate with the unit circle, S
1
. Think of as the angle of a point on the circle. Then, the state space of the pendulum is
S
1
R, a cylinder.
2.1 DETERMINISTIC DYNAMICAL SYSTEMS 23
The phase ow embodies the evolution of the process. Given a time t R and an initial condition
x
0
A, (t, x
0
) is the state of the system resulting after evolution for t units of time. Sometimes, we will
replace x
0
by x and write (t, x) to highlight the fact that x is an independent variable ranging over the
set A. The consistency and semigroup properties of make the process deterministic.
Exercise 2.1. Show that, for each t R, the mapping x (t, x) is bijective; nd its inverse. Thus, given (t, x), one can
uniquely recover the initial condition x.
Consistency guarantees that if time doesnt progress, the process doesnt evolve. The semigroup prop-
erty states that the evolution of the process for time t + s from the initial condition x
0
is equivalent
to the evolution of the system for time t from the initial condition x
1
= (s, x
0
). The consistency and
semigroup properties together rule out the situation depicted below.
x
1
x
0
x
2
Here, the system evolves from x
0
to x
1
and then revisits x
1
at a later time before moving on to x
2
.
Initializing the system at x
1
, we see that the evolution is non-deterministic in that we cant say for sure
whether the system will loop around or move on to state x
2
.
Exercise 2.2. Suppose, by way of contradiction, that there exist T > 0 and x
1
A such that x
1
= (T, x
1
). Show that, for all
t R, (t + T, x
1
) = (t, x
1
), so the system keeps looping around without possibly moving on to x
2
.
Similarly, the situation depicted below cannot occur.
x
1
x
2
Exercise 2.3. Suppose that there exist two times t
1
, t
2
R and two initial conditions x
1
, x
2
A, x
1
,= x
2
, such that (t
1
, x
1
) =
(t
2
, x
2
). First show that it cannot be that t
1
= t
2
. Now consider the case t
1
,= t
2
, lets say t
1
> t
2
. Show that (t
1
t
2
, x
1
) = x
2
so the system evolves from x
1
to x
2
in time t
1
t
2
, ruling out the situation depicted above.
The semigroup property makes the process autonomous (or stationary), meaning that the function of
the system doesnt change with time. Initializing the system at time t
1
or at time t
2
> t
1
produces the
same evolution shifted in time by the amount t
2
t
1
.
2.2 Finite dimensional phase flows and vector fields
The state spaces of the heat propagation and curve-shortening processes in Examples 2.1 and 2.3 are
function spaces, while the state space of the pendulum in Example 2.2 is a nite-dimensional vector
space. In this section, and throughout the rest of these notes, we focus our attention on the nite-
dimensional setting when the phase ow is a C
1
function. In this case, one can always associate to the
phase ow a vector eld which turns out to be its innitesimal generator, in the following sense. The
vector eld denes an ordinary differential equation whose solutions, under certain technical assump-
tions, are shown to regenerate the phase ow which was used to dene the vector eld in the rst
place. The exploration of the equivalence of nite-dimensional phase ows and vector elds will require
us to investigate the topic of existence and uniqueness of solutions of ordinary differential equations.
2.2.1 Phase flows of finite-dimensional dynamical systems
Denition 2.5. A C
1
nite-dimensional autonomous dynamical system (or just a dynamical system) is
a pair (A, ), where:
(i) A is an open subset of R
n
, for some n N, and W RA is open.
(ii) The function (t, x
0
) (t, x
0
), W A is C
1
and it satises the consistency property (0, x
0
) = x
0
and semigroup property (t, (s, x
0
)) = (t +s, x
0
), as long as either side of the identity is dened.
The function is called the local phase ow of (A, ). In the special case W = R A, is called
the phase ow of (A, ).
(iii) (0, x
0
) : x
0
A W.
(iv) For each x
0
A, the set T
x
0
= t R : (t, x
0
) W is an interval containing 0. Dene T
+
x
0
=
T
x
0
[0, +) and T
x
0
= T
x
0
(, 0].
(v) If either one of the endpoints of T
x
0
is nite, then for any compact set K A, there exists t T
x
0
such that (t, x
0
) , K.
Aside from the requirement that A R
n
and that be C
1
, the main difference between this denition
and Denition 2.4 is that here the phase ow is only dened on a subset W of R A. This relaxation
is motivated by the fact that many physical processes are characterized by unstable behavior and their
evolution isnt dened for all initial conditions and all times. The denition places restrictions on W.
Requirement (iii) guarantees that (0, x
0
) is dened for all x
0
A, and it implies that the consistency
property (0, x
0
) = x
0
holds on all of A. Requirement (iv) rules out the meaningless situation when the
evolution (t, x
0
) is dened on two or more time intervals: time jumps cannot occur in a deterministic
dynamical system. Therefore, for each x
0
A, the evolution of the process is well-dened on the time
2.2 FINITE DIMENSIONAL PHASE FLOWS AND VECTOR FIELDS 25
interval T
x
0
. The time intervals T
+
x
0
and T
x
0
are the portions of T
x
0
corresponding to positive and negative
times, respectively. Requirement (v) ensures that, for each x
0
, the domain of (t, x
0
) is maximal, i.e., it
cannot be made larger. Specically, if T
x
0
= (, a) (or T
x
0
= (a, +)), where a is a real number, then
(t, x
0
) must approach the boundary of A as t a. In particular, if A = R
n
, then (t, x
0
) must go off
to innity. The properties of W are illustrated in the gure below.
0
t
t
A
W
x
0
x
0
T
x
0
T
x
0
T
+
x
0
t
A
W
x
0
W violates requirement (v)
Remark 2.6. Denition 2.5 can be generalized by allowing A to be a smooth nite-dimensional manifold,
i.e., a surface of nite dimension. For instance, as mentioned in the footnote on page 22, the state space
of the pendulum is a cylinder. Throughout these notes we will limit ourselves to the less general setting
of Denition 2.5.
Denition 2.7. Consider a C
1
nite-dimensional dynamical system (A, ), with : W RA A.
(i) For each x
0
A, the phase curve through x
0
(or the orbit through x
0
) is the set O(x
0
) = (T
x
0
, x
0
).
The positive [or negative] semiorbit through x
0
is the set O
+
(x
0
) = (T
+
x
0
, x
0
) [or O
(x
0
) =
(T
x
0
, x
0
)].
(ii) The integral curve through x
0
(or the trajectory through x
0
) is the graph of the function t
(t, x
0
).
Remark 2.8. A function and its graph contain the same information, and therefore it is customary to
identify them. One can equivalently dene the integral curve through x
0
(the trajectory through x
0
) as
the function t (t, x
0
). Whenever convenient, in these notes we will adopt this equivalent denition.
The orbit of a dynamical system through x
0
describes the evolution of the system in the state space, but
it does not contain any information about the time parameterization. On the other hand, the integral
curve through x
0
retains the information about the time parameterization.
Example 2.9. Consider the dynamical system (R
2
, ), with : RR
2
dened as
(t, x
0
) =
_
cos t sin t
sin t cos t
_
x
0
.
For each t R, x
0
(t, x
0
) performs the rotation of x
0
by the angle t. Note that (0, x
0
) = x
0
and
(t, (s, x
0
)) is the composition of two rotations, rst by angle s and then by angle t; this corresponds to
the rotation of x
0
by angle t +s, (t +s, x
0
). Since the domain of is all of RR
2
, is a phase ow; it is
the collection of all rotations in R
2
. For each x
0
R
2
, the phase curve through x
0
is the circle centred at
the origin passing through x
0
, while the integral curve is a helix. Observe the difference between phase
and integral curves: phase curves do not contain any information about their time parametrization,
while integral curves do.
x
1
x
2
x
0
(/2, x
0
)
(/2, x
0
)
x
1
x
2
t
Exercise 2.4. Verify that each of the following is the phase ow of a dynamical system (R
2
, ). For each system draw a few
phase curves and integral curves.
1. (t, x) =
_
e
t
0
0 e
2t
_
x 2. (t, x) =
_
e
t
0
0 e
2t
_
x 3. (t, x) =
_
e
t
0
0 e
t
_
x
4. (t, x) = e
t
_
cos t sin t
sin t cos t
_
x 5. (t, x) = e
t
_
cos t sin t
sin t cos t
_
x 6. (t, x) = e
t
_
cos 2t sin2t
sin2t cos 2t
_
x
Exercise 2.5. Which of the following maps RR R is a phase ow?
1. (t, x) = x 2. (t, x) = tx 3. (t, x) = (t + 1)x
4. (t, x) = e
t+1
x 5. (t, x) = e
t
x.
We next present an example of a nite-dimensional dynamical system which generates a local phase
ow.
Example 2.10. Consider the dynamical system (R, ) with
(t, x
0
) =
x
0
1 x
0
t
.
Note that (0, x
0
) = x
0
and, as long as 1 x
0
t, 1 x
0
s, and 1 x
0
(t + s) are not equal to zero,
(t, (s, x)) =
x
1xs
1
x
1xs
t
=
x
1 x(t + s)
= (t + s, x).
The function is dened on the set (t, x) R
2
: xt ,= 1. Parts of this set must be discarded in order
for to give rise to a meaningful dynamical system. Specically, when x
0
> 0, the integral curve should
be dened on T
x
0
= (, 1/x
0
), rather than (, 1/x
0
) (1/x
0
, +), as it wouldnt make sense to
have the dynamical system incur a singularity as t 1/x
0
and then continue evolving for t > 1/x
0
(this
is the reason behind requirement (iv) in Denition 2.5). Similarly, if x
0
< 0, the integral curve should
be dened on T
x
0
= (1/x
0
, +). Finally, if x
0
= 0, then the integral curve is dened on all of R. In
conclusion, the domain of is the set W in this gure. A few integral curves are also depicted on the
right-hand side.
t
W
x
0
x
0
1/x
0
t
x
Observe how the integral curves blow up in nite time, either in the future (positive time) or in the
past (negative time), depending on the initial condition.
2.2.2 Vector Fields
To the (local) phase ow of an autonomous dynamical system one naturally associates a vector eld.
This is the eld of velocities of points moving along phase curves of the system. Recall that the phase
curve through a point x A is the image of the map t (t, x). The velocity at (t, x) of a point
moving along this curve is the vector d(t, x)/dt. Since (0, x) = x, the velocity at x is
f (x) =
d(t, x)
dt
t=0
.
Since is C
1
, the function f (x) : A R
n
R
n
is continuous.
Denition 2.11. Given a C
1
nite-dimensional autonomous dynamical system (A, ), the vector eld
associated to (A, ) is the map f : A R
n
dened as
f (x) =
d(t, x)
dt
t=0
.
Thus a vector eld is an assignment, to each point x in A, of a vector f (x) in R
n
attached at x. To
highlight the fact that it arises from an autonomous dynamical system, f (x) is sometimes referred to
as an autonomous vector eld. One could also have non-autonomous vector elds of the form f (t, x),
arising from non-autonomous dynamical systems.
Example 2.12. We return to Example 2.9 and associate a vector eld to the phase ow. Compute:
f (x) =
d(t, x)
dt
t=0
=
_
sin t cos t
cos t sin t
_
t=0
x =
_
x
2
x
1
_
Now to each point (x
1
, x
2
) R
2
we associate the vector (x
2
, x
1
).
6 4 2 0 2 4 6
6
4
2
0
2
4
6
x
1
x
2
Example 2.13. The vector eld associated to the local phase ow in Example 2.10 is f : R R R
dened as
f (x) =
d
dt
x
1 xt
t=0
= x
2
.
Exercise 2.6. Find the vector eld associated to each dynamical system in Exercise 2.4.
The vector eld associated to a phase ow contains local information (i.e., the direction of the tangent
vectors) about the shape of the phase curves of the ow. It seems that one should be able, from this local
information, to recover the entire set of phase curves. Additionally, the vector eld contains velocity
information (i.e., the length of the tangent vectors) which is directly related to the time parametrization
of the phase curves. Hence, it seems reasonable to expect that, besides recovering the phase curves,
one should be able to recover the entire set of integral curves. If this were the case, given a vector eld
one should be able, at least in principle, to recover the phase ow that generated it. We will see in the
next section that, to a great extent, this intuition is correct. The next theorem contains the germ of our
subsequent discussion.
Theorem 2.14. Let f : A R
n
be the vector eld associated to a C
1
autonomous dynamical system (A, ).
Then, for each x
0
A, the map t (t, x
0
) satises
d(t, x
0
)
dt
t=t
0
= f ((t
0
, x
0
)).
Proof.
d(t, x
0
)
dt
t=t
0
=
d( + t
0
, x
0
)
d
=0
=
d(, (t
0
, x
0
))
d
=0
= f ((t
0
, x
0
)).
2.2.3 Existence and uniqueness of solutions
We have seen in the previous section that to a given (local) phase ow generated by a C
1
nite-
dimensional dynamical system, one may associate a continuous vector eld x f (x). Theorem 2.14
asserts that, for each x
0
A, the map t (t, x
0
) satises the ordinary differential equation
x = f (x), x A R
n
. (2.1)
Motivated by this result, we now focus on the fundamental properties of this differential equation.
Denition 2.15. Given x
0
A, a function x : T
x
0
A, where T
x
0
R is an interval containing 0, is a
solution of (2.1) with initial condition x
0
if
(i) x(0) = x
0
(ii) x(t) is differentiable and (t T
x
0
)
dx(t)
dt
= f (x(t)).
Remark 2.16. It is readily seen that if x(t) is a solution satisfying x(0) = x
0
, then x(t + t
0
) is a solution
satisfying x(t
0
) = x
0
.
Continuity of f (x) is sufcient to guarantee that (2.1) has at least one solution for each x
0
A.
Theorem 2.17 (Peano). If f : A R
n
is continuous, then for each x
0
A there exists at least one solution
of (2.1) with initial condition x
0
.
We will not prove this theorem (see [Har02, Section II.2]). We rather present a few examples showing
that, while continuity of f is sufcient to guarantee existence of solutions, it does not guarantee that
solutions are unique.
Example 2.18. Consider the differential equation x = x
1/3
with initial condition x(0) = 0. The function
x x
1/3
is continuous. The two functions
x(t) 0, x(t) =
_
_
_
(2/3t)
3/2
t 0
0 t < 0
,
are solutions to the differential equation and satisfy the initial condition x(0) = 0.
The next example illustrates that a continuous differential equation may have innite solutions.
Example 2.19. Consider the equation x =
_
[x[ (check that this is continuous) with initial condition
x(0) = 0. For each C 0, dene
x
C
(t) =
_
_
_
_
Ct
2
_
2
t > C
0 t C
The function x
C
(t) enjoys the following properties:
x
C
(0) = 0
x
C
(t) is continuously differentiable
x
C
(t) is a solution to the differential equation.
A few solutions in the family x
C
(t) : C 0 are depicted below.
-2 0 2 4 6 8 10
-20
-18
-16
-14
-12
-10
-8
-6
-4
-2
0
t
x
Exercise 2.7. Does the family of functions x
C
(t) : C 0 exhaust all possible solutions to the differential equation in
Example 2.19?
The examples show that continuity of f is too weak a property to guarantee uniqueness of solutions. In
both examples, the right-hand side of the differential equation is continuous but not Lipschitz continuous
at 0. Lipschitz continuity of f is strong enough to guarantee uniqueness, as seen in the next result.
Theorem 2.20 (Picard-Lindelf; Local existence and uniqueness). If f : A R
n
is locally Lipschitz on A,
then for each x
0
A there exists a > 0 and a unique solution x : [a, a] A of the differential equation (2.1)
0
.
Proof. (Existence) Let x
0
be an arbitrary point in A. Since A R
n
is open, there exists > 0 such that
B
n
(x
0
) A. Moreover, the set

B
n
(x
0
) is compact. Then, by Theorem 1.23, f is Lipschitz on

B
n
(x
0
) with
some Lipschitz constant L. Since f is continuous, |f | is also a continuous function and, by Theorem 1.24,
|f | is bounded on the compact set

B
n
(x
0
). Let M be the maximum of |f | on

B
n
(x
0
).
A function x : I A, where I is an interval, is a solution to
x = f (x)
x(0) = x
0
(2.2)
if and only if
x(t) = x
0
+
_
t
0
f (x())d (2.3)
for all t I. For, if x(t) is a solution to (2.2), then integration of both sides of (2.2) from 0 to t yields (2.3).
If, on the other hand, x(t) satises (2.3), then differentiation of x(t) yields (2.2).
Choose a > 0 such that a < min/M, 1/L. We now dene a sequence of functions dened on the
interval [a, a] and show that this sequence converges uniformly to a solution of (2.3). Let
x
0
(t) x
0
x
1
(t) = x
0
+
_
t
0
f (x
0
())d
x
i+1
(t) = x
0
+
_
t
0
f (x
i
())d, i N.
It is easy to see inductively that all functions x
i
are continuous (x
i+1
(t) is continuous because f (x
i
(t)) is
continuous). Further, we claim that for all t [a, a] and for all i Z
+
, x
i
(t)

B
n
(x
0
). The statement
is true for i = 0. Assume that x
i
(t)

B
n
(x
0
), then
|x
i+1
(t) x
0
| =
_
_
_
_
_
t
0
f (x
i
())d
_
_
_
_

_
t
0
|f (x
i
())|d
_
t
0
Md
Ma < ,
and so x
i+1
(t)

B
n
(x
0
), proving the claim.
Using the Lipschitz continuity of f on

B
n
(x
0
), we now show that x
i
(t)
iZ
+
converges uniformly on
[a, a]. First, we look at
max
t[a,a]
|x
2
(t) x
1
(t)| = max
t[a,a]
_
_
_
_
_
t
0
f (x
1
()) f (x
0
())d
_
_
_
_
max
t[a,a]
_
t
0
|f (x
1
()) f (x
0
())|d
max
t[a,a]
_
t
0
L|x
1
() x
0
()|d = max
t[a,a]
_
t
0
L|x
1
() x
0
|d
max
t[a,a]
_
t
0
Ld
La.
Next, we show by induction that
max
t[a,a]
|x
i+1
(t) x
i
(t)| (La)
i
, i Z
+
. (2.4)
Suppose that, for some i 2, max
t[a,a]
|x
i
(t) x
i1
(t)| (La)
i1
. Then,
max
t[a,a]
|x
i+1
(t) x
i
(t)| max
t[a,a]
_
t
0
|f (x
i
()) f (x
i1
())|d max
t[a,a]
L
_
t
0
|x
i
() x
i1
()|d
La(La)
i1
= (La)
i
.
Let I be a positive integer. Then, for all j, l > I, with j > l,
|x
j
(t) x
l
(t)| |x
j
(t) x
j1
(t)| + +|x
l+1
(t) x
l
(t)|
k=I
|x
k+1
(t) x
k
(t)|
k=I
(La)
k
.
By our choice of a, La < 1, and so the geometric series
k=0
(La)
k
converges. It is then clear that for all
> 0, there exists I > 0 such that
k=I
(La)
k
< . By Theorem 1.47, x
i
x uniformly on [a, a] and,
since all x
i
s are continuous, by Theorem 1.48 the limit x is a continuous function on [a, a].
By denition,
x
i+1
(t) = x
0
+
_
t
0
f (x
i
())d.
Taking the limit as i of both sides of the equation we obtain
x(t) = x
0
+ lim
i
_
t
0
f (x
i
())d
=
(by Theorem 1.49)
x
0
+
_
t
0
lim
i
f (x
i
())d
= x
0
+
_
t
0
f (x())d.
In other words, x(t) satises (2.3).
(Uniqueness) Suppose that there exists another solution y(t) of (2.2) dened on [a, a]. We show that
x(t) = y(t) for all t [a, a]. Indeed,
max
t[a,a]
|x(t) y(t)| = max
t[a,a]
_
_
_
_
_
t
0
x() y()d
_
_
_
_
max
t[a,a]
_
t
0
|f (x()) f (y())|d
L max
t[a,a]
_
t
0
|x() y()|d (2.5)
La max
t[a,a]
|x(t) y(t)|.
Hence,
(1 La) max
t[a,a]
|x(t) y(t)| 0.
Since La < 1, the only way this is possible is that max
t[a,a]
|x(t) y(t)| = 0, and thus x(t) y(t).
Remark 2.21. The sequence of functions x
i
(t) in the proof above is called Picards iteration. It is useful
to numerically approximate the solution of a differential equation.
Exercise 2.8. What justies inequality (2.5)?
Exercise 2.9. Apply Picards iteration to
x = cx, x R
x(0) = 1,
where c is a real number not equal to zero. Verify analytically that x
i
(t) e
ct
.
Exercise 2.10. Generalization of Exercise 2.9. Let A be an n n matrix. Apply Picards iteration to
x = Ax, x R
n
x(0) = x
0
.
Show that x
i
(t) = x
i1
(t) +
A
i
t
i
i!
x
0
. Verify that x
i
(t) e
At
x
0
.
Exercise 2.11. Can x(t) depicted below be a solution of a scalar differential equation with locally Lipschitz autonomous vector
eld?
t
x
When f is globally Lipschitz the next result states that there exists a unique global (i.e., dened for all
t R) solution of (2.1).
Theorem 2.22 (Global existence and uniqueness). If A = R
n
and f : A R
n
is globally Lipschitz, then
for each x
0
A there is a unique solution x : R A of the differential equation (2.1) with initial condition
x(0) = x
0
.
Proof. Let L be the Lipschitz constant of f over A, and pick a (0, 1/L).
Claim. For any x
0
in A, there exists a unique solution of (2.1) dened on [a, a].
Proof of the claim. Since f is globally Lipschitz, dening the sequence of functions x
n
(t)
nZ
+
as in the
proof of Theorem 2.20, we see that inequality (2.4) holds for any x
0
A with the same value of L. Since
a < 1/L, x
n
(t)
nZ
+
is a Cauchy sequence and so x
n
(t) x(t). Following the same argument as in the
proof of Theorem 2.20 we conclude that x(t) is the unique solution of (2.1) over the interval [a, a].
We now use this claim to show that for any T > 0 there is a unique solution x
T
(t) on [0, T]. The idea is
to divide the interval [0, T] into segments of length a and to apply the Claim to each of the segments.
Let h be a positive integer such that ha T, and dene x
T
(t) : [0, ha] A as follows:
(i 0, . . . , h 1)(t [ia, (i + 1)a]) x
T
(t) = x
i
(t ia),
where x
0
(t) = x(t) and x
i
(t) is the solution of
x
i
= f (x
i
)
x
i
(0) = x
i1
(a).
(2.6)
Thus, x
T
(t) is obtained by a process of gluing together h segments end-to-end. Each segment is the
solution of the differential equation initialized at the nal value of the previous segment. We start from
the solution x
0
(t) dened on [0, a]. The nal value x
0
(a) becomes the initial condition for the segment
x
1
(t). The segment is then shifted in time and glued on to x
0
(t) to dene x
T
(t) on the interval [0, 2a].
This procedure is repeated h times. By the Claim, x
i
(t) is the unique solution of (2.6) on [a, a]. This
implies, as discussed in Remark 2.16, that the segment x
T
: [ia, (i + 1)a] A is the unique solution
satisfying x
T
(ia) = x
i1
(a).
The function x
T
(t) is clearly differentiable on each interval (ia, (i + 1)a). It is also differentiable at the
points t = ia, i h, since the functions x
i1
(t (i 1)a) and x
i
(t ia) overlap on the interval [(i 1)a, ia]
and by uniqueness of solutions they must coincide over this interval. By construction, x
T
is a solution
of (2.1) and it is in fact the unique solution on the interval [0, ha].
We have thus shown that for each T > 0, (2.1) has a unique solution dened on [0, T]. Since T is
arbitrary, this implies that there is a unique solution dened on [0, ). The same argument applies,
mutatis mutandis for negative T, proving that (2.1) has a unique solution on R.
Exercise 2.12. Clarify the statement, in the proof above: It is also differentiable at the points t = ia, i h, since the functions
x
i1
(t + (i 1)a) and x
i
(t + ia) overlap on the interval [(i 1)a, ia] and by uniqueness of solutions they must coincide over
this interval.
Example 2.23. Let A R
nn
and consider the linear differential equation
x = Ax.
The vector eld x Ax is globally Lipschitz. This can be seen in at least two ways. First, using the
fact (see Exercise 1.24) that the map x Ax is C
1
and its Jacobian is A. Since the norm of the Jacobian
is constant, and thus uniformly bounded on R
n
, Theorem 1.44 assures that the vector eld is globally
Lipschitz. Another way to see this is directly using Denition 1.20.
Theorem 2.22 guarantees that for each x
0
R
n
there exists a unique solution dened on the entire real
line. We have already found it in Exercise 2.10: its x(t) = e
At
x
0
.
Example 2.24. In Example 2.10 we investigated a dynamical system with local phase ow
(t, x)
x
1 xt
.
In Example 2.13 we found that the associated vector eld is f (x) = x
2
. Since f (x) is locally Lipschitz but
not globally Lipschitz (see Exercise 1.20), we know that for each x
0
R there exists a unique solution
of x = x
2
0
dened on some interval around 0. We cant say a priori whether
the solution is dened for all t R because Theorem 2.22 cannot be applied. Lets use separation of
variables to solve this differential equation. Write
_
x
x
0
1
2
d =
_
t
0
dt.
Integration yields
1
x
+
1
x
0
= t.
Solving for x,
x =
x
0
1 x
0
t
.
The solution is only dened on [0, 1/x
0
) if x
0
is positive. Note that we have recovered the phase ow
(t, x
0
) (t, x
0
)! Is it a coincidence?
The preceding example may lead one to believe that it is necessary for the right-hand side of a differential
equation to be globally Lipschitz for solutions to be dened for all t 0. This is false, see the next
exercise.
Exercise 2.13. Consider the differential equation x = x
3
. Show that x x
3
is not globally Lipschitz. Using separation of
variables, nd all solutions of the differential equation and verify that they are dened for all t 0.
2.2.4 Maximal solutions
The idea we used in the proof of the global existence and uniqueness theorem was to extend each local
solution step by step. Starting at t = 0, there exists a unique solution on [0, a]. Putting a new initial
condition at t = a, we are guaranteed the existence of another piece of the solution dened on [a, 2a].
Continuing this process indenitely, one obtains a solution dened on the entire real line. What makes
this process possible is this: when f is globally Lipschitz the constant a can be chosen independently of
the initial condition. When f is not globally Lipschitz, this isnt any longer true: in general, the interval
of denition will depend on the initial condition and so the argument in the proof of Theorem 2.22
fails. Lets see how. Starting at t = 0, we get the unique solution x(t) on [0, a(x
0
)]. Let a
1
= a(x
0
)
and x
1
= x(a
1
). At time a
1
, we put the initial condition x
1
and get the unique solution on the interval
[a
1
, a
1
+ a(x
1
)]. Let a
2
= a
1
+ a(x
1
), and continue this process indenitely. We obtain an increasing
sequence a
i
iN
and a unique solution x(t) dened on the union of all time intervals [0, a
i
],

iN
[0, a
i
].
This is just the interval
_
0, lim
i
a
i
_
. If the limit is nite, then the procedure gives a solution dened on
a bounded time interval, only! It should be clear from the construction that there is no hope to extend
it in positive time. Example 2.24 is a manifestation of this phenomenon. In light of this discussion, we
introduce the notion of maximal solution, i.e., the solution with the largest domain.
Denition 2.25. Let f : A R
n
be locally Lipschitz on A. A function x(t) : T
x
0
A, where T
x
0
is an
open interval around 0, is the maximal solution of (2.1) with initial condition x
0
if, for any solution
y : (a, b) A with 0 (a, b) and y(0) = x
0
, one has that (a, b) T
x
0
and y x[
(a,b)
. The set T
x
0
is the
maximal interval of existence for x
0
. The set T
+
x
0
= T
x
0
[0, +) [or T
x
0
= (, 0] T
x
0
] is the maximal
positive [or negative] interval of existence for x
0
.
We will show that for each x
0
A there exists a unique maximal solution with initial condition x
0
. We
begin with the next lemma.
Lemma 2.26. Let f : A R
n
be locally Lipschitz on A, and let x(t), y(t) be two solutions of (2.1) dened on
an interval I R. Suppose that, for some t
0
I, x(t
0
) = y(t
0
). Then, x(t) = y(t) for all t I.
Proof. By Theorem 2.20 we know that x(t) = y(t) on some interval around t
0
. Let I
denote the largest

open interval around t
0
on which x(t) = y(t) (i.e., the union of all open intervals around t
0
on which
x(t) = y(t)). We claim that I
= I. If not, then there exists t

1
I which is an endpoint of I
, say the
right-hand endpoint. Since x and y are continuous, x(t
1
) = y(t
1
). By Theorem 2.20, there exists a unique
solution of x = f (x) with initial condition x(t
1
) and hence x(t) = y(t) on an interval around t
1
. This
implies that t
1
cannot be an endpoint of I
.
With this lemma, we can now prove the following.
Theorem 2.27 (Existence of a unique maximal solution). Let f : A R
n
be locally Lipschitz on A. Then,
for any x
0
A, there exists a unique maximal solution with initial condition x
0
.
Proof. Let T
x
0
be the union of all open intervals around 0 on which there is a solution (unique, by
Theorem 2.20) with initial condition x
0
. If I
1
, I
2
are any two such intervals, and x
I
1
(t) and x
I
2
(t) are the
two corresponding solutions, by Lemma 2.26 we have x
I
1
(t) = x
I
2
(t) on I
1
I
2
. We can therefore dene
a function x
I
1
I
2
(t) dened over the interval I
1
I
2
as follows
x
I
1
I
2
(t) =
_
_
_
x
I
1
(t), t I
1
x
I
2
(t), t I
2
.
Since x
I
1
(t) and x
I
2
(t) are solutions through x
0
, x
I
1
I
2
is also a solution through x
0
, and in fact it is
the unique solution over the interval I
1
I
2
. Using this procedure, we dene the unique solution x(t)
through x
0
dened on the whole interval T
x
0
by piecing together all solutions dened on the subintervals
of T
x
0
. By the way T
x
0
was dened, x(t) is maximal.
Intuitively, if the maximal solution is not dened for all t R, something must go wrong. The next result
states that x(t) blows up, in that it approaches the boundary of A as time approaches the boundary
of T
x
0
(if A = R
n
, it approaches innity). Compare this result to requirement (v) in Denition 2.5.
Theorem 2.28 (Finite escape time). Let f : A R
n
be locally Lipschitz on A and x(t) be the maximal
solution of (2.1) with initial condition x
0
. Let T
x
0
= (a, b) R be the domain of denition of x(t), and suppose
that b < . Then, for any compact set K A, there exists t T
+
x
0
:= T
x
0
[0, ) such that x(t) , K. An
analogous results holds when a > .
Proof. Suppose, by way of contradiction, that there exists a compact set K A such that x(t) K for all
t T
+
x
0
. Since |f | is continuous, by Theorem 1.24 it has a maximum M on K. We now show that x(t) is
uniformly continuous. For any > 0, let = /M. Then, for any t
1
, t
2
T
x
0
such that [t
2
t
1
[ < , we
have
|x(t
1
) x(t
2
)| =
_
_
_
_
_
t
2
t
1
x()d
_
_
_
_
M[t
2
t
1
[ < .
Having shown uniform continuity, by Theorem 1.25 there exists a unique continuous function y(t) :
T
x
0
= [a, b] A such that y(t) = x(t) for all t T
x
0
. We claim that this function is a solution of (2.1).
For all t (a, b), we have
y(t) = x
0
+
_
t
0
f (y())d. (2.7)
By continuity of y,
y(b) = lim
tb
x(t) = lim
tb
_
x
0
+
_
t
0
x()d
_
= x
0
+ lim
tb
_
t
0
f (x())d
= x
0
+
_
b
0
f (y())d.
The same result holds for y(a). Therefore, y(t) satises the integral equation (2.7) for all t [a, b],
and thus it is the solution of (2.1) through x
0
dened on [a, b]. Setting y(b) as a new initial condition,
Theorem 2.20 implies that the domain of denition of y can be extended to an interval [a, b
], where
b
> b. Hence, the interval of existence (a, b) cannot be maximal, giving a contradiction.
This result has a useful corollary which is often used in stability theory.
Corollary 2.29. Let K A be a compact set and let x
0
K. Suppose that the maximal solution x(t) with initial
condition x
0
is such that x(t) K for all t T
+
x
0
[or all t T
x
0
]. Then, T
+
x
0
= [0, +) [or T
x
0
= (, 0]], i.e.,
the solution is dened for all t 0 [or all t 0].
Exercise 2.14. Prove this corollary.
2.2.5 Continuity and differentiability with respect to time and initial
conditions
Consider once again the differential equation
x = f (x), x A R
n
. (2.8)
We have determined in Theorem 2.27 that if f is locally Lipschitz on A, then for each x
0
A there exists a
unique maximal solution of (2.8) with initial condition x
0
. To highlight the fact that the maximal solution
depends on x
0
, let us denote it by (t, x
0
). The notation here is not accidental, for we will see in the next
section that (t, x
0
) is a phase ow. The domain of the map (t, x
0
) is W = (t, x
0
) R A : t T
x
0
.
In this section we show that the set W is open and that the map (t, x
0
) (t, x
0
) is continuous. If f is
C
1
, then is C
1
as well.
First, we need this lemma.
Lemma 2.30 (Gronwalls inequality). Let C
1
, C
2
0 and suppose a continuous and nonnegative function
f : [t
0
, t
1
] R satises
f (t) C
1
+ C
2
_
t
t
0
f ()d.
Then, for all t [t
0
, t
1
],
f (t) C
1
e
C
2
(tt
0
)
.
Proof. Suppose rst that C
1
> 0. Let x(t) = C
1
+ C
2
_
t
1
t
0
f ()d. Differentiating x we get
x(t) = C
2
f (t) C
2
x(t),
and hence
x(t)
x(t)
=
C
2
f (t)
x(t)
C
2
.
Note that
d log x(t)
dt
C
2
.
Integrating from t
0
to t,
log x(t) log x(t
0
) + C
2
(t t
0
).
Exponentiation gives
x(t) x(t
0
)e
C
2
(tt
0
)
= C
1
e
C
2
(tt
0
)
.
Now consider the case C
1
= 0. We need to show f (t) 0. Let C
i
be a sequence of positive numbers
such that C
i
0. Consider the sequence of functions f
i
(t), with f
i
(t) = f (t) + C
i
. Then, f
i
f
uniformly and
f
i
(t) C
i
+ C
2
_
t
t
0
f ()d
C
i
+ C
2
_
t
t
0
f
i
()d.
By the preceding argument, f
i
(t) C
i
e
C
2
(tt
0
)
. Taking the limit as i , we nd f (t) 0.
Exercise 2.15. Why, in the rst part of the proof of Gronwalls inequality, we need C
1
> 0?
We use Gronwalls inequality to show that if two solutions of (2.8) start close to each other, then they
remain close to each other on a closed time interval.
n
be locally Lipschitz on A. Let x(t) be a solution of (2.8) dened on [t
0
, t
1
], with
x(t
0
) = x
0
. Then, there is a neighborhood B
n
(x
0
) A and a constant L > 0 such that, for all y
0
B
n
(x
0
), there
is a unique solution y(t) also dened on [t
0
, t
1
] with y(t
0
) = y
0
and
|x(t) y(t)| |x
0
y
0
|e
L(tt
0
)
for all t [t
0
, t
1
].
Remark 2.32. The theorem above implies that for every > 0, there exists > 0 such that for all y
0

B
n
(x
0
) and all t [t
0
, t
1
], (t, y
0
) B
n
((t, x
0
)). In other words, all phase curves originating in B
n
(x
0
)
remain inside a tube of radius around the solution (t, x
0
). This follows by setting = e
L(t
1
t
0
)
in
the theorem statement.
Proof. Since [t
0
, t
1
] is compact and x(t) is continuous, by Theorem 1.24 x(t) is bounded on [t
0
, t
1
]. There-
fore, for any > 0, the -neighborhood of the curve x(t)
U = y R
n
: (t [t
0
, t
1
]) |y x(t)|
is compact. Choose small enough that U A. The compactness of U and the local Lipschitz continuity
of f (x) imply, by Theorem 1.23, that f (x) is Lipschitz on U with some Lipschitz constant L. Take
y
0
B
n
(x
0
), with . By Theorem 2.27, there exists a unique maximal solution y(t) of (2.8), dened
on T
x
0
= (a, b) t
0
, with initial condition y(t
0
) = y
0
. We now show that if is small enough, then b > t
1
and y(t) does not leave U. Suppose, by way of contradiction, that b t
1
. On the interval [t
0
, b), we have
|x(t) y(t)| =
_
_
_x
0
y
0
+
_
b
t
0
f (x()) f (y())d
_
_
_
|x
0
y
0
| +
_
b
t
0
L|x() y()|d
+ L
_
b
t
0
|x() y()|d.
Now we apply Gronwalls inequality to |x(t) y(t)| with C
1
= and C
2
= L. We obtain
|x(t) y(t)| e
L(tt
0
)
for all t [t
0
, b). Choosing e
L(t
1
t
0
)
, we see that |x(t) y(t)| for all t [t
0
, b), i.e., y(t) U
for all t [t
0
, b). Since U is compact, by Theorem 2.28 this result contradicts the fact that (a, b) is the
maximal interval of existence of y(t). We have thus established that y(t) is dened on [t
0
, t
1
]. We have
also established the inequality
|x(t) y(t)| |x
0
y
0
|e
L(tt
0
)
,
for all t [t
0
, t
1
].
Now the main result.
Theorem 2.33 (Continuity of ). Let f : A R
n
be locally Lipschitz on A; dene W = (t, x
0
) R A :
t T
x
0
, where T
x
0
is the maximal interval of existence for x
0
. Consider the map : W A, where for each
x
0
A, (t, x
0
) is the unique maximal solution of (2.8) with initial condition x
0
. Then, W is open and is
continuous.
Proof. Let (t
0
, x
0
) be an arbitrary point in W and assume, without loss of generality, that t
0
0. By the
denition of W, t
0
T
x
0
and hence the maximal solution t (t, x
0
) of (2.8) with initial condition x
0
is well-dened on the closed interval [0, t
0
]. By Theorem 2.20, the solution can be extended to a slightly
larger interval [a, t
0
+ a]. By Theorem 2.31, there exists a neighborhood B
x
0
(x
0
) A such that, for all
y
0
B
x
0
(x
0
), there exists a unique solution t (t, y
0
) also dened on [a, t
0
+ a]. This means that the
open set (a, t
0
+ a) B
x
0
(x
0
) (t
0
, x
0
) is contained in W, proving that W is open.
We now turn to the continuity of . We already have that t (t, x
0
) is continuous. Let (t
0
, x
0
), a, and
x
0
be as above and let > 0 be arbitrary. Theorem 2.31 implies that there exists L > 0 such that, for any
t [a, t
0
+ a] and any x B
x
0
(x
0
), (t, x) is well-dened and
|(t, x) (t, x
0
)| |x x
0
|e
Lt
<
x
0
e
L(t
0
+a)
.
Choose
x
0
small enough that
x
0
e
L(t
0
+a)
< /2. Since t (t, x) is continuous, we have
(
t
0
> 0)(t [a, t
0
+ a]) ([t t
0
[ <
t
0
) = (|(t, x
0
) (t
0
, x
0
)| < /2).
If necessary, make
t
0
smaller so that
t
0
a. This guarantees that if [t t
0
[ <
t
0
, then t [t
0
a, t
0
+a]
[a, t
0
+ a]. Now consider the inequality
|(t, x) (t
0
, x
0
)| |(t, x) (t, x
0
)| +|(t, x
0
) (t
0
, x
0
)|.
For all (t, x) such that [t t
0
[ <
t
0
and |x x
0
| <
x
0
, we have |(t, x) (t
0
, x
0
)| < /2 + /2 = .
Therefore, (t, x) (t, x) is continuous at (t
0
, x
0
).
Exercise 2.16. Consider the differential equation
x = f (x, ),
where R
m
is a vector of constant parameters. Suppose that the map (x, ) f (x, ) is locally Lipschitz on A R
m
. For
each x
0
and each , let (t, x
0
, ) denote the maximal solution with initial condition x
0
. Prove that (t, x
0
, ) (t, x
0
, ) is
continuous. Thus, small variations in initial conditions and parameters generate small variations in the solutions.
Exercise 2.17. Consider the differential equation
x = f (t, x).
Suppose that the map (t, x) f (t, x) is locally Lipschitz on RA. Show that for each (t
0
, x
0
) RA there exists a unique
maximal solution (t, t
0
, x
0
) such that (t
0
, t
0
, x
0
) = x
0
, and that the function (t, t
0
, x
0
) (t, t
0
, x
0
) is continuous.
When f , besides being locally Lipschitz, is C
1
, then the partial derivatives of with respect to t and x
are continuous.
Theorem 2.34 (Differentiability of ). Let f : A R
n
be C
1
. Then : W A is C
1
as well.
For the proof of this theorem, see [Har02, Section V.3] or [Kha02, Section 3.3].
2.2.6 The (local) phase flow generated by a vector field
We have collected all the ingredients that make it possible to understand the relationship between a
phase ow and the associated vector eld. The objective of this section is to show that:
Under certain regularity assumptions, a vector eld generates a (local) phase ow (Theorem 2.35).
This ow coincides with that which generated the vector eld in the rst place (Theorem 2.38).
Thus, a vector eld and a phase ow contain essentially the same information and one may, in principle,
choose any of the two to represent a C
1
nite-dimensional dynamical system. In practice, however, the
vector eld representation is far more convenient.
n
be C
1
; dene W = (t, x
0
) R A : t T
x
0
, where T
x
0
is the maximal
interval of existence for x
0
, and for each x
0
A let (t, x
0
) be the unique maximal solution of (2.8) with initial
condition x
0
. Then, (A, ) is a C
1
nite-dimensional dynamical system with local phase ow : W A.
Proof. We need to show that
1. (x
0
A) (0, x
0
) = x
0
.
2. (t, (s, x
0
)) = (t + s, x
0
) as long as one side of the equation is dened.
3. W is open and : W A is C
1
.
4. W satises requirements (iii)-(v) in Denition 2.5
Requirement 1 is obviously met. The domain of is W = (t, x
0
) RA : t T
x
0
. By Theorem 2.27,
T
x
0
is an interval containing 0. This property and Theorem 2.28 together imply that requirement 4 is
met. If is C
1
, then it is also locally Lipschitz (Theorem 1.43) and by Theorems 2.33 and 2.34, property 3
holds. We are left to prove that enjoys the semigroup property. This is left as an exercise.
Exercise 2.18. Prove that satises the semigroup property.
Corollary 2.36. If A = R
n
and f : A R
n
is C
1
and globally Lipschitz, then (R
n
, ) is a C
1
nite-dimensional
dynamical system with phase ow : RR
n
A.
Denition 2.37. Let f : A R
n
be C
1
. The map : W A dened in Theorem 2.35 is the (local)
phase ow generated by the vector eld f (x).
Lets summarize the major developments so far. We have associated to a C
1
nite-dimensional dynamical
system (A, ) a vector eld f (x) which was used to dene the differential equation x = f (x). We found
that when f is C
1
, the maximal solutions of the differential equation can be used to dene a dynamical
system (A,
) and a local phase ow. The next result shows that the two dynamical systems (A, ) and
(A,
) are in fact the same.

Theorem 2.38. Let (A, ) be the C
1
nite-dimensional dynamical system with local phase ow : W A.
Suppose that the associated vector eld f (x) is C
1
and denote the local phase ow it generates by
: W
A.
Then, W = W
and =
.
Proof. By Theorem 2.14, for each x
0
A, the function T
x
0
A, t (t, x
0
) is a solution of x = f (x)
0
. Let T
x
0
= (a, b). Let
(t, x
0
) denote the maximal solution with initial condition
x
0
dened on the maximal interval T
x
0
= (a
, b
). By existence and uniqueness (Theorem 2.20), (t, x

0
) =
(t, x
0
) for all t T
x
0
T
x
0
. We claim that T
x
0
= T
x
0
(and hence W = W
, =
). Since (t, x
0
) is a
solution, necessarily its domain of denition must be contained in the maximal interval of existence, that
is, T
x
0
T
x
0
. On the other hand, T
x
0
T
x
0
. Suppose not, then either a
< a or b
> b. Consider the case

b
> b (the other case is similar). Take

t (b, b
). Since the interval [0,

t] is compact and t
(t, x
0
) is
continuous, |
(t, x
0
)| has a maximum M on [0,

t]. Recall that T
x
0
satises property (v) in Denition 2.4,
that is, for any compact set K A, there exists t T
x
0
such that (t, x
0
) , K. We use this property
with K =

B
M
(x
0
) to assert that there exists t T
x
0
(i.e., t <

t) such that |(t, x
0
)| > M, and since
(t, x
0
) =
(t, x
0
), |
(t, x
0
)| > M as well. However, since t [0,

t], we also have |
(t, x
0
)| M,
which gives a contradiction.
The diagram below gives a global view of the relationships between a C
1
nite-dimensional dynamical
system and its vector eld.
Phase ow : W A Dynamical system (A, ) x = f (x)
if f is C
1
Vector eld x f (x)
The diagram shows that talking about a dynamical system (or its phase ow) is equivalent to talking
about an ordinary differential equation, provided the vector eld is C
1
. From now on we will assume
that vector elds are C
1
and henceforth we will not distinguish between a differential equation, a vector
eld, or a dynamical system. We will apply the terminology of Denition 2.7 to differential equations
and vector elds. Thus, we will speak of the phase curve (or the integral curve) through x
0
of a vector
eld or a differential equation. We will write (t, x
0
) for the maximal solution with initial condition x
0
.
Exercise 2.19. Consider a C
1
nite-dimensional dynamical system (R
n
, ). Show that the phase ow is a linear function of
the initial condition x
0
, that is, (t, x
0
) = M(t)x
0
, if and only if it is generated by a linear time-invariant differential equation
x = Ax.
43
Chapter 3
Introduction to Dynamics
The eld of dynamics is concerned with the study of properties of dynamical systems. The language of
dynamics is used extensively in control theory because many of the underlying concepts in the two elds
are closely related. In this chapter the basic concepts and introductory results in the eld of dynamics
are presented. References for this material are [Arn73], [GH83], and [Kha02].
3.1 Equilibria, limit sets, and invariant sets
Consider the dynamical system
x = f (x), x A (3.1)
where A R
n
is open and f is C
1
.
Denition 3.1. A point x
A is an equilibrium point of (3.1) (or an equilibrium) if x(0) = x
implies
x(t) = x
for all t R.
Thus, an equilibrium is a constant solution. It is clear from the denition that x
is an equilibrium
point if and only if f (x
) = 0. Equilibria are classied, according to their stability properties, as stable,

(positive semiorbits through points near the equilibrium remain close to the equilibrium) asymptotically
stable, (in addition to being stable, positive semiorbits through points near the equilibrium approach the
equilibrium), or unstable (i.e., not stable). We will discuss in detail this classication in a later chapter.
In the case of two-dimensional linear time-invariant (LTI) systems x = Ax, with x R
2
, a ner classi-
44 CHAPTER 3. INTRODUCTION TO DYNAMICS
cation is possible, and is summarized below in terms of the eigenvalues of A,
1
and
2
.
Equilibrium type Eigenvalues
stable node
1
,
2
R,
1
< 0,
2
< 0
unstable node
1
,
2
R,
1
> 0,
2
> 0
saddle point
1
,
2
R,
1
2
< 0
stable focus
1,2
= i, > 0, ,= 0
unstable focus
1,2
= i, > 0, ,= 0
centre
1,2
= i, ,= 0
When one or both eigenvalues are zero, then the LTI system has a continuum of equilibria, Ker(A).
Refer to [Bro06] for a detailed discussion on this classication, and for gures illustrating the differences
between the equilibrium classes.
Exercise 3.1. For each of the following matrices A, consider the associated differential equation x = Ax. Find the eigenvalues
and eigenvectors of A; use them to determine the equilibrium type and to draw the phase portrait in x coordinates.
1. A =
_
1 0
2 1
_
2.
_

7
2

3
2
3
2

3
2
_
Next, we turn our attention to periodic integral curves.
Example 3.2. Consider the vector eld in Example 2.12
x
1
= x
2
x
2
= x
1
.
We already know, from Example 2.9, that its integral curves are periodic,
(t, x
0
) =
_
cos t sin t
sin t cos t
_
x
0
,
while its phase curves are closed (circles centred at the origin). This system is the harmonic oscillator.
Notice that the amplitude of the oscillations in the integral curves (the radius of the circles representing
the phase curves) depends on the initial conditions.
x
1
x
2
3.1 EQUILIBRIA, LIMIT SETS, AND INVARIANT SETS 45
Now consider the dynamical system
x
1
= x
2
x
2
= x
1
+ (1 x
2
1
)x
2
,
(3.2)
where is a positive parameter. This equation arises from an LC circuit in series with a nonlinear
element with a cubic characteristic. It is known as the Van der Pol oscillator. A few phase curves are
plotted below.
-4 - 3 -2 -1 0 1 2 3 4
-4
-3
-2
-1
0
1
2
3
4
x
1
x
2
One of the phase curves of this system is closed, while the ones nearby arent. This is a limit cycle.
Notice that all phase curves (except the equilibrium solution at the origin) approach the limit cycle (i.e.,
the limit cycle is stable); hence, the amplitude of oscillation in steady-state does not depend on the initial
condition. If we reverse the direction of the phase curves by changing the sign of the vector eld, then
the resulting system still has a limit cycle, but nearby phase curves diverge from it (i.e., the limit cycle is
unstable).
Now the denition of a limit cycle, due to Jules Henri Poincar, 1854-1912.
Denition 3.3. A limit cycle is a nontrivial closed isolated phase curve. Nontrivial and closed means
that the corresponding integral curve is periodic and not constant. Isolated means that in a sufciently
small neighborhood of the limit cycle there is no other closed phase curve.
Exercise 3.2. Show that a linear system x = Ax cannot exhibit limit cycles.
[Hint: you need to show that any closed phase curve of a linear system cannot be isolated.]
Another important concept in dynamics is that of a limit set.
Denition 3.4. Let x
0
A and let T
x
0
= (a, b). A point p A is a positive [or negative] limit point of
x
0
for (3.1) if there is a sequence t
i
iN
T
x
0
with t
i
b [or t
i
a] such that (t
i
, x
0
) p. The set of
all positive [or negative] limit points of x
0
for (3.1) is the positive [or negative] limit set of x
0
, denoted
L
+
(x
0
) [or L
(x
0
)].
This denition was rst given by George D. Birkhoff in [Bir27]. He used the terminology -limit set
and -limit set rather than positive and negative limit set. Both Poincar and Birkhoff gave fundamental
contributions to the theory of dynamical systems and their investigations (like those of many others)
were motivated by problems in Celestial Mechanics, chief among them the three-body problem.
To illustrate the notion of limit set, consider a stable limit cycle. Let x
0
be a point in a small neighborhood
of the cycle, as in the gure below. We now show that L
+
(x
0
) is the limit cycle. Following the denition,
let p be an arbitrary point on the limit cycle. Draw the straight line through p orthogonal to the limit
cycle. Let x
1
be the rst point where the positive semiorbit through x
0
meets the line, and let t
1
be the
corresponding time; let x
2
be the second intersection at time t
2
, and so on.
x
1
x
2
p
x
i
We obtain a sequence x
i
iN
= (t
i
, x
0
)
iN
with t
i
and x
i
p. Thus p is a limit point of x
0
.
Repeating this argument for any point on the limit cycle, we obtain that the limit cycle is contained in
the positive limit set of x
0
. Actually, the limit cycle is the positive limit set of x
0
. Why?
Exercise 3.3. Consider the Van der Pol oscillator in Example 3.2. Find L
+
(x
0
) and L
(x
0
) in the two cases: x
0
in the domain
outside the limit cycle; x
0
in the domain inside the limit cycle.
Exercise 3.4. Show that the positive limit set and negative limit sets of any point on a limit cycle are the limit cycle itself.
Example 3.5. Consider the LTI system
x
1
= x
1
x
2
= x
2
.
The origin is a saddle point; it is the positive limit set of any point on the x
2
axis and the negative limit
set of any point on the x
1
axis. If x
0
is neither on the x
1
nor the x
2
axis, then L
+
(x
0
) = and L
(x
0
) = .
Denition 3.6. A set A is invariant for (3.1) if
x
0
= (t T
x
0
) (t, x
0
) .
A set A is positively invariant for (3.1) if
x
0
= (t T
+
x
0
) (t, x
0
) .
is negatively invariant for (3.1) if
x
0
= (t T
x
0
) (t, x
0
) .
Thus, an invariant set has the property that the any initial condition on yields a solution that doesnt
leave in positive or negative time. In other words, any phase curve through a point in is entirely
contained in . Equilibria and limit cycles are invariant sets. A positively invariant set enjoys a weaker
property: any solution with initial condition in does not leave in positive time, but it may leave it in
negative time. That is, solutions may enter the set, but not exit it. Similarly, solutions may exit negatively
invariant sets, but may not enter it. An invariant set is both positively invariant and negatively invariant.
Example 3.7. Lets see some examples of invariant, positively invariant, and negatively invariant set. We
begin with the Van der Pol oscillator in Example 3.2. The domain inside the limit cycle is an invariant
set, and so is the domain outside the limit cycle.
Now consider the LTI system
x
1
= x
1
x
2
= 2x
2
.
The origin is a stable node. The set depicted below is positively invariant.
The complement of ,
c
, is negatively invariant. The same set is negatively invariant for the unstable
node
x
1
= x
1
x
2
= 2x
2
,
while
c
is positively invariant.
Limit sets are key in understanding the steady-state behavior of dynamical systems. The following
fundamental result, due to Birkhoff [Bir27], outlines some of their important features. Given a set
S R
n
and a point x R
n
, we denote by |x|
S
the point-to-set distance,
|x|
S
= inf
yS
|x y|.
Theorem 3.8 (Birkhoffs Theorem). Positive [or negative] limit sets of (3.1) are closed and invariant. Moreover,
if the positive semiorbit [or negative semiorbit] of (3.1) through x
0
A is a bounded set, then L
+
(x
0
) [or L
(x
0
)]
is nonempty, compact, connected, invariant, and |(t, x
0
)|
L
+
(x
0
)
0 as t + [or |(t, x
0
)|
L
(x
0
)
0 as
t ].
Theorem 3.8 is a useful tool for the analysis of stability of dynamical systems. Well see how it is used
to derive LaSalles invariance principle in Chapter 4. It is also used in the analysis of stability properties
of passive systems.
When O
+
(x
0
) is unbounded, then the limits sets of x
0
may be empty or, even if they are not empty, they
may not be connected.
Proof. (Closed) To show that L
+
(x
0
) is closed, we use the Closed Set Theorem (Theorem 1.18). Let
x
i
iN
be an arbitrary convergent sequence in L
+
(x
0
) with limit p. We need to show that p L
+
(x
0
).
Let x(t) = (t, x
0
). Since each x
i
belongs to L
+
(x
0
), there exists a sequence of times t
i
k
kN
T
x
0
=
(a, b), with t
i
k
b as k , such that x(t
i
k
) x
i
. Pick a time

t
i
from each sequence t
i
k
kN
such that
t
i
> b 1/i and |x(
t
i
) x
i
| < 1/i. Now consider the sequence x(
t
i
)
iN
. We show that it converges
to p. For,
|x(
t
i
) p| |x(
t
i
) x
i
| +|x
i
p|
1
i
+|x
i
p|.
Taking the limit as i , we see that x(
t
i
) p. Therefore p L
+
(x
0
), as required.
(Invariant) Let p L
+
(x
0
). We need to show that (t, p) L
+
(x
0
) for all t T
p
. By denition,
there exists a sequence of times t
i
T
x
0
with t
i
b, such that (t
i
, x
0
) p. By continuity of ,
(t, (t
i
, x
0
)) (t, p). By Theorem 2.35, satises the semigroup property; hence, (t + t
i
, x
0
)
(t, p) and (t, p) L
+
(x
0
).
(Nonempty) Now suppose that O
+
(x
0
) is a bounded set, that is, the solution (t, x
0
) remains inside
closed ball for all t T
x
0
. Then, by Corollary 2.29, T
x
0
= R. Let t
i
R be any sequence of times such
that t
i
+. Since the sequence (t
i
, x
0
) is bounded, it has a convergent subsequence
1
(t
i
k
, x
0
)
with t
i
k
as k . The limit of this convergent subsequence is a limit point of x
0
, and thus L
+
(x
0
)
is not empty.
(Compact) We already know that the limit set is closed. By the denition of limit point, it follows that
L
+
(x
0
) is a subset of

O
+
(x
0
), the closure of the positive semiorbit through x
0
. Since O
+
(x
0
) is bounded,
its closure is bounded as well (Theorem 1.19). Thus, L
+
(x
0
) is bounded.
1
This is a result from Calculus known as the Bolzano-Weierstrass theorem: every bounded sequence x
i
iN
in R
n
has a
convergent subsequence x
i
k
kN
, with i
k
< i
k+1
for all k 1.
(Convergence) We will prove this by way of contradiction. Suppose |(t, x
0
)|
L
+
(x
0
)
, 0 as t . In
other words, referring to Denition 1.26,
( > 0)(M > 0)(t > M)|(t, x
0
)|
L
+
(x
0
)
> .
Pick a sequence M
i
iN
R
+
, with M
i
. By the above, there exists a sequence t
i
iN
with
t
i
> M
i
, such that |(t
i
, x
0
)|
L
+
(x
0
)
> . By construction, t
i
as i . Since O
+
(x
0
) is bounded,
the sequence (t
i
, x
0
) is bounded, so it has a convergent subsequence (t
i
k
, x
0
), with t
i
k
and
(t
i
k
, x
0
) p as k . We thus have that p L
+
(x
0
). However, by construction |(t
i
k
, x
0
)|
L
+
(x
0
)
> ,
which implies that the limit p satises |p|
L
+
(x
0
)
> 0, a contradiction.
(Connected) Suppose, by contradiction, that there exist two separated subsets A and B of L
+
(x
0
), that
is, A

B = and

A B = . Since L
+
(x
0
) is compact, it follows A and B are also compact and the
conditions reduce to A B = . The function
d : A A R
+
, (x, y) |x y|.
is continuous. Since A and B are compact, so is AB, and hence d has a minimum over AB achieved
at some point in AB. Since A B = , this minimum must be positive; denote it . Then, > 0 is the
distance between the sets A and B. Let p and q be points in A and B, respectively and let t
2i+1
, t
2i
be diverging time sequences such that t

2i
< t
2i+1
, (t
2i+1
, x
0
) p, and (t
2i
, x
0
) q. For sufciently
large i, |(t
2i+1
, x
0
)|
A
/4 and |(t
2i
, x
0
)|
B
/4. These inequalities imply
|(t
2i+1
, x
0
)|
B

3
4
, |(t
2i
, x
0
)|
A

3
4
.
By the continuity of , there exists

t
i
(t
2i
, t
2i+1
) such that
|(
t
i
, x
0
)|
A

2
and |(
t
i
, x
0
)|
B

2
.
Now we have a sequence (for sufciently large i) (
t
i
) which is bounded because O
+
(x
0
) is bounded.
Hence, there exists a convergent subsequence. The limit, x
, is in L
+
(x
0
). On the other hand,
|x
|
A

2
and |x
|
B

2
.
Therefore x
does not belong to either A or B, a contradiction.

Exercise 3.5. There is a gap in the proof of invariance in Theorem 3.8. Find it and ll it.
[Hint: Use Theorem 2.31.]
Corollary 3.9. Let be a compact positively invariant set for (3.1). Then, the positive limit set of any x
0
is
nonempty, compact, connected, invariant, and |(t, x
0
)|
L
+
(x
0
)
0 as t +.
Proof. If x
0
, then Corollary 2.29 implies that T
+
x
0
= [0, +). The boundedness of implies that
O
+
(x
0
) is bounded. Apply Theorem 3.8.
3.2 Nagumos invariance condition
Theorem 3.8 states that each bounded solution of a dynamical system asymptotically converges to an
invariant set. The characterization of the invariance property is therefore crucial in understanding of
the steady-state behavior of dynamical systems. In this section we provide a simple geometric criterion
to conclusively determine whether or not a set is invariant. This criterion is the basis for the Lyapunov
theory presented in Chapter 4.
Denition 3.10. Let be a closed subset of R
n
, and let x . The Bouligand tangent cone to at x is
dened as
T
(x) :=
_
v R
n
: liminf
t0
+
|x + tv|
t
= 0
_
.
If x is an interior point of , then T
(x) = R
n
because |x + tv|
0 for all small t. If x , , then

T
(x) = because |x + tv|
> > 0 for all small t. So T
(x) is nontrivial only when x . In this

case, T
(x) is the collection of all vectors that are pointing to the interior of, or at tangent to at x. More
precisely, if x , then |x|
= 0, and the quantity liminf

t0
+ |x +tv|
/t is the lower Dini derivative

of t |x + tv|
at t = 0. This derivative generalizes the directional derivative of Denition 1.33

t
|x + tv|
to non-differentiable functions, and it is always well-dened. Roughly

speaking, the lower Dini derivative gives the minimum slope of the line
that best approximates the function t |x + tv|
at t = 0. For in-
stance, for the graph of t |x +tv|
illustrated on the right-hand side,

liminf
t0
+ |x + tv|
/t = 0. To apply the denition of T
(x), given
v R
n
, we draw the half line x + tv : t 0, we compute the distance
of each point of this line to , we graph it, and we check whether the
minimum slope of the graph at t = 0 is zero. The results of this procedure on a few examples are
illustrated below.
x
x x
x
(x)
T
(x)
T
(x)
T
(x)
In particular, if is a differentiable surface and x , then T
(x) is a subspace of R
n
, precisely the
tangent plane to at x.
Theorem 3.11 (Nagumo [Nag42]). A closed set R
n
is positively invariant for (3.1) if and only if (x
) f (x) T
(x).
In other words, for positive invariance of it is necessary and sufcient that on the boundary of , the
3.2 NAGUMOS INVARIANCE CONDITION 51
vector eld points inside or is tangent to . We will only prove necessity. The proof of sufciency can
be found in [Bre70].
Proof. ( =) Suppose is positively invariant for (3.1). Let x be arbitrary. Then, (t, x) for all
t 0. By denition of point-to-set distance we have that
|x + t f (x)|
= inf
y
|x + t f (x) y|
|x + t f (x) (t, x)| (because (t, x) )
= |(t, x) x t f (x)|.
Now divide both sides by t and take the lower limit as t 0
+
,
liminf
t0
+
|x + t f (x)|
t
liminf
t0
+
_
_
_
_
(t, x) x
t
f (x)
_
_
_
_
= |f (x) f (x)|
= 0.
Exercise 3.6. Give necessary and sufcient conditions for a closed set to be negatively invariant for (3.1). Give necessary and
sufcient conditions for a closed set to be invariant for (3.1).
We now specialize Theorem 3.11 to smooth surfaces described as level sets of functions (such as the unit
circle in R
2
described by the relation x
2
1
+ x
2
2
1 = 0). First off, the tangent cone coincides with the
tangent plane.
Lemma 3.12. Let : A R
m
, m n, be C
1
and let = x A : (x) = 0. Suppose for all x
rank d
x
= m. Then, for all x , T
(x) = Ker d
x
.
The proof of this lemma is omitted. The condition rank d
x
0
= m in Lemma 3.12 guarantees that the
surface is differentiable and it has a well-dened tangent plane at each point.
Denition 3.13. Let : A R
m
be a C
1
function. The Lie derivative of along the vector eld f is
the function
x L
f
(x) := L
f (x)
(x) = d
x
f (x).
If L
f
(x) = 0 on the set = x A : (x) = 0, then f (x) Ker d
x
for all x , and hence
the Nagumo invariance condition is satised. Thus, the combination of Theorem 3.11 and Lemma 3.12
yields the following result.
Theorem 3.14. Let : A R
m
, m n, be C
1
and let = x A : (x) = 0. Suppose for all x
rank d
x
= m. Then, is invariant if and only if L
f
(x) = 0 for all x .
The next gure illustrates the invariance condition
f (x
0
)
L
f
(x
0
) = 0
d
x
0
(x) = 0
x
0
Proof. By Lemma 3.12, L
f
(x) = 0 for all x if and only if f (x) T
(x) for all x , if and

only if (Theorem 3.11) is positively invariant for (3.1). Since L
f
(x) = 0 if and only if L
f
(x) = 0,
the condition L
f
(x) = 0 for all x is equivalent to the positive invariance of for the system
x = f (x), which in turn is equivalent to the negative invariance of for (3.1).
Remark 3.15. Note, in the statement of Theorem 3.14, the assumption that d
x
0
be of full rank m for
all x
0
which, as mentioned earlier, guarantees that the set is a surface with a well-dened
tangent plane. Application of Theorem 3.14 when this assumption doesnt hold may lead to erroneous
conclusions. This is shown in Example 3.18 below.
Now another special case of Nagumos theorem. Let be the sublevel set of a C
1
function A R,
= x A : (x) 0.
Note the difference with the earlier situation: if = x
2
1
+ x
2
2
1, then is the unit disk, rather than the
unit circle. In this case, the tangent cone to at a point on its boundary is a half-plane.
Lemma 3.16. Let : A R be C
1
, and let = x A : (x) 0. Suppose for all x R
n
such that
(x) = 0 it holds that d
x
,= 0. Then, for all x R
n
such that (x) = 0, T
(x) = v R
n
: L
v
(x) 0.
The proof of this lemma is omitted. Theorem 3.11 and Lemma 3.16 give the following.
Theorem 3.17. Let : A R be C
1
and let = x A : (x) 0. Suppose for all x R
n
such that
(x) = 0, d
x
,= 0. Then, is positively invariant for (3.1) if and only if
(x R
n
)(x) = 0 = L
f
(x) 0.
f (x
0
)
d
x
0
x
0
L
f
(x
0
) 0
(x) 0
3.2 NAGUMOS INVARIANCE CONDITION 53
The proof of the above is obvious in light of Lemma 3.16. Note, in the statement of Theorem 3.17, the
essential requirement that d
x
0
,= 0 whenever (x
0
) = 0.
Example 3.18. Consider the system
x
1
= x
3
2
x
2
= x
3
1
+ x
1
x
2
.
We claim that the set = (x
1
, x
2
) : x
1
= x
2
is invariant. Letting (x
1
, x
2
) = x
1
x
2
, we use
Theorem 3.14 to verify the claim. We need to compute L
f
(x), where
f (x) =
_
x
3
2
x
3
1
+ x
1
x
2
_
.
We have
L
f
(x) = d
x
f (x) = [1 1]
_
x
3
2
x
3
1
+ x
1
x
2
_
= x
3
2
+ x
3
1
x
1
+ x
2
.
For all (x
1
, x
2
) , i.e., x
1
= x
2
, the expression above is zero and hence is invariant. This is conrmed
when plotting a few phase curves. Note how phase curves through points on slide along .
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
x
1
x
2
Now consider the dynamical system
x
1
= 1
x
2
= 1
and the function (x
1
, x
2
) = x
2
1
+ x
2
2
. The zero level set of is the origin, i.e., = (0, 0). Clearly, is
not an equilibrium, and hence not an invariant set, and yet
L
f
(0) = [2x
1
2x
2
]
_
1
1
_
x
1
=x
2
=0
= 0.
The problem is that the assumption that d
x
be full rank on is violated since d
0
= [0 0].
Next, consider again the linear system in Example 3.7,
x
1
= x
1
x
2
= 2x
2
.
Let be the following region delimited by an ellipse,
= (x
1
, x
2
) : x
2
1
+ 4x
2
2
4.
(This is the same ellipse shown in the gure of Example 3.7. Lets show that is positively invariant.
Let (x
1
, x
2
) = x
2
1
+ 4x
2
2
4 and use Theorem 3.17. We have
L
f
(x) = [2x
1
8x
2
]
_
x
1
2x
2
_
= x
2
1
8x
2
2
.
This expression is always 0, so of course it is 0 on the boundary of . We have thus obtained a
stronger result: any ellipse
c
= (x
1
, x
2
) : x
2
1
+ 4x
2
2
c,
with c > 0, is a positively invariant set for the system.
Lets apply Nagumos theorem to LTI systems. We ask the question: what are necessary and sufcient
conditions for a subspace 1 of R
n
to be an invariant set for x = Ax? Write 1 = Ker(M), where M is
a matrix of suitable dimension with full column rank. Setting (x) = Mx, we can use Theorem 3.14 to
conclude that 1 is invariant if and only if
(x 1) MAx = 0,
or,
(x 1) Ax Ker(M),
or,
(x 1) Ax 1.
Concisely, 1 is an invariant set for x = Ax if and only if
A(1) 1,
where A : R
n
R
n
is the linear transformation x Ax. The condition we found coincides with the
denition of an A-invariant subspace, see [Fra03, Section 3.6]. Thus, an A-invariant subspace is a special
kind of invariant set of an LTI system. Notice that invariant subspaces do not exhaust all the possible
invariant sets of an LTI system! As a matter of fact, by existence and uniqueness, any phase curve of a
dynamical system is an invariant set.
Exercise 3.7. Consider a mass-spring system. Suppose that the spring characteristic is nonlinear, F
spring
= f (y), where y is
the displacement of the spring from equilibrium. The function f is assumed to be locally Lipschitz on R and such that f (0) = 0,
and for all y ,= 0, y f (y) > 0. The model of the system is M y = f (y). The energy of the system is E(y, y) =
_
y
0
f ()d +
1
2
M y
2
.
Show that every level set of the energy is invariant.
Exercise 3.8. Consider the mass-spring system of Exercise 3.7, but now suppose that the system is subject to a friction force
F
friction
= g( y), where g is locally Lipschitz on R and such that yg( y) > 0 for all y ,= 0.
1. Show that any sublevel set of the energy is positively invariant.
2. Let f (y) = y
3
and g( y) = y. Show that any initial condition (y
0
, y
0
) has a nonempty and compact positive limit set.
It turns out that the positive limit set of any initial condition is the equilibrium state; therefore all phase curves approach
the equilibrium state, as expected. We dont yet have the tools to prove this claim, so we will postpone its proof until the
presentation of the LaSalle invariance principle.
3.3 THE POINCAR-BENDIXSON THEOREM 55
3.3 The Poincar-Bendixson Theorem
Now we focus our attention to dynamical systems on the plane
x
1
= f
1
(x
1
, x
2
)
x
2
= f
2
(x
1
, x
2
),
(3.3)
where (x
1
, x
2
) R
2
and the vector eld ( f
1
, f
2
) is C
1
.
Theorem 3.19 (Poincar-Bendixson). A nonempty compact positive or negative limit set of (3.3) which contains
no equilibrium point is a closed orbit.
The proof is at times informal. See [HS74] and [Har02] for a detailed, although essentially equivalent,
proof. First, two denitions.
Denition 3.20. A local section of f at a point p is a straight-line segment
S R
2
such that for all x S, f (x) , T
S
(x).
f (x)
S
x
In other words, the vector f (x) is not tangent to S and not zero either. This implies that, on S, f points
on one side of S. We say that S is transverse to f .
Lemma 3.21. If p R
2
is such that f (p) ,= 0, then there exists a local section of f at p.
Proof. Let v = [v
1
v
2
]
be a unit vector orthogonal to f (p). For some > 0, let S = x R

2
: x =
p +vt, t [, ]. This is a straight-line segment of length 2, centred at p, and lying on the line spanned
by the vector v. We claim that there exists > 0 such that S is a local section of f at p. Consider the
function : R
2
R, (x) = [v
2
v
1
] f (x). This function is continuous because f is such. Moreover,
(p) ,= 0 because, by construction, f (p) is not parallel to v. By continuity of (x), there exists > 0
such that (x) ,= 0 for all x B
2
(p), implying that f is transversal to v on B

2
(p). Picking (0, ) we

have that S B
2
(p), implying that S is a local section of f at p.

Denition 3.22. A owbox of f at p is a curvilinear rectangular closed
set V R
2
whose boundary is formed by two orbits of f and two parallel
local sections of f . p
V
S
1
S
2
Lemma 3.23. If p R
2
is such that f (p) ,= 0, then there exists a owbox of f at p.
Proof. By Lemma 3.21 there exists a local section S of f at p. Let p
1
and p
2
be the endpoints of S.
Following the argument in the proof of Lemma 3.21, there exists > 0 such that all parallel translations
of S contained in B
2
(p) are local sections of f at p. Dene the closed set V B

2
(p) whose boundary is

formed by sufciently small arcs of the orbits O(p
1
) and O(p
2
) (these cannot intersect by uniqueness of
solutions) and two parallel translations of S in B
2
(p). By construction, V is a owbox at p.

Proof of Theorem 3.19. The proof is divided into ve parts.
(Part 1) Let V be a owbox at p R
2
, and S be a local section of f at p. Let S
1
and S
2
be the two local
sections of f on the boundary of V. If V is sufciently small, then by continuity
of f we have that g is close to a constant vector eld, implying that the arcs of
orbits of f in V are close to parallel straight lines. Each such arc will have a unique
intersection with S and will traverse the entire owbox entering V at S
1
and exiting
it at S
2
, or viceversa. From this property we infer that for sufciently small V there
exists > 0 such that for any q V, there exists a unique [, ] such that
(, q) S. Now suppose that there exist times t
1
< t
2
, with t
2
t
1
> 2, such
V
S
S
1
S
2
p
q
that q
i
:= (t
i
, x
0
) V, i = 1, 2. Then, there exist unique
1
,
2
[, ] such that (t
i
+
i
, x
0
) S,
i = 1, 2. Moreover, t
1
+
1
t
1
+ < t
2
t
2
+
2
. In conclusion, if O
+
(x
0
) enters V at two
consecutive times t
1
< t
2
such that t
2
t
1
> 2, then O
+
(x
0
) hits S at consecutive times t
1
+
1
< t
2
+
2
.
(Part 2) Let S is a local section of f and suppose that a trajectory (t, x
0
) intersects S at times t
1
< t
2
<
. Then, x
i
= (t
i
, x
0
) are monotone along S. That is, the point x
i
is between x
i1
and x
i+1
.
x
i1
x
i
x
i+1
S
To prove this claim, consider the closed curve formed by the arc between x
1
and x
2
and the line segment
on S joining x
1
and x
2
. We can have one of the two situations depicted below.
x
i
x
i+1
S
x
i
x
i+1
S
The shaded set on the left-hand side is positively invariant, implying that the next intersection point,
x
i+2
, must lie in the set. Thus, x
i+1
lies between x
i
and x
i+2
. On the other hand, the shaded set on the
right-hand side is negatively invariant, implying that the previous intersection point, x
i1
, must lie in
the set. In this case, x
i
lies between x
i1
and x
i+1
.
(Part 3) If S is a local section of f , we claim that L
+
(x
0
) S is either empty or one point. Suppose that
L
+
(x
0
) S is not empty, and let p
1
, p
2
L
+
(x
0
) S.
We will show that p
1
= p
2
. Since by assumption
there are no equilibria in L
+
(x
0
), by Lemma 3.23
there exist owboxes V
i
at p
i
, i = 1, 2. Since p
i

L
+
(x
0
), and since L
+
(x
0
) is compact by assumption,
there exist sequences t
i
n
nN
(0, ), i = 1, 2,
such that t
i
n
and (t
i
n
, x
0
) p
i
as n .
Without loss of generality, we can select elements of
the two sequences such that
t
1
n
< t
2
n
< t
1
n+1
< t
2
n+1
, n N,
and the difference between consecutive times is >
2, with > 0 dened in part 1. Also, since
S
p
1
p
2
(t
1
k
, x
0
)
(t
1
k+1
, x
0
)
(t
2
k
, x
0
)
(t
2
k+1
, x
0
)
x
1
k
x
1
k+1
x
2
k
x
2
k+1
V
1
V
2
(t
i
n
, x
0
) p
i
V
i
, we can pick large enough times such that (t
i
n
, x
0
) V
i
for all n Nand i = 1, 2. By
the result in part 1, for each n and i, there exists a unique time
i
n
[, ] such that x
i
n
:= (t
i
n
+
i
n
, x
0
)
S and, moreover,
t
1
n
+
1
n
< t
2
n
+
2
n
< t
1
n+1
+
1
n+1
< t
2
n+1
+
2
n+1
, n N.
Since the above time sequence and the sequences t
1
n
nN
, t
2
n
nN
are monotonic, by part 2 the se-
quences x
1
n
nN
, x
2
n
nN
, and x
1
1
, x
2
1
, . . . , x
1
n
, x
2
n
, . . . are monotone along S. Moreover, since (t
i
n
, x
0
)
p
i
as n , and p
i
V
i
, we have that
i
n
0 as n . By continuity of (t, x
0
), we deduce that
x
i
n
= (t
i
n
+
i
n
, x
0
) p
i
as n . The monotonicity of the three sequences x
1
n
nN
, x
2
n
nN
, and
x
1
1
, x
2
1
, . . . , x
1
n
, x
2
n
, . . . implies that they must have the same limit, and hence p
1
= p
2
.
(Part 4) Pick a point x
1
in L
+
(x
0
). We claim that the integral curve through x
1
is periodic. That is,
O
+
(x
1
) is a closed curve. Since L
+
(x
0
) is invariant, O
+
(x
1
) L
+
(x
0
). This fact and the fact that L
+
(x
0
)
is closed imply that L
+
(x
1
) L
+
(x
0
). Additionally, the compactness of L
+
(x
0
) implies that O
+
(x
1
) is
bounded, so that L
+
(x
1
) is nonempty. Let x
2
L
+
(x
1
) and consider a local section S of f at x
2
. Its
existence follows from Lemma 3.21 because L
+
(x
1
) L
+
(x
0
), and by assumption there are no equilibria
in L
+
(x
0
). Now consider a sequence t
i
iN
, t
i
< t
i+1
, t
i
, such that (t
i
, x
1
) x
2
. We can
assume that t
i+1
t
1
> 2. By part 1, there exist unique times
i
[, ] such that (t
i
+
i
, x
1
) S,
and t
i
+
i
< t
i+1
+
i+1
. Let s
i
= t
i
+
i
. Then, (s
i
, x
1
) S. Since O
+
(x
1
) is contained in L
+
(x
0
),
(s
i
, x
1
)
iN
L
+
(x
0
) S. By part 3, L
+
(x
0
) S must consist of only one point. Therefore, for all i,
(s
i
, x
1
) = (s
i+1
, x
1
). By the semigroup property, (s
i+1
s
i
, x
1
) = x
1
. Since s
i+1
s
i
> 0, the latter
equality can hold in only two cases: either x
1
is an equilibrium, or the integral curve through x
1
is
periodic. The former case is ruled out by the assumption that L
+
(x
0
) contains no equilibrium point.
Hence the phase curve through x
1
is closed.
(Part 5) In part 4 we have shown that there exists a closed phase curve in L
+
(x
0
). Denote the closed
phase curve , and let T be a period of the corresponding integral curve. We claim that L
+
(x
0
) = . If
we show that |(t, x
0
)|
0, then L
+
(x
0
), which implies that = L
+
(x
0
). Pick a point p and
let S be a local section of f at p. Let V be a owbox at p. S and V are guaranteed to exist by Lemmas 3.21
and 3.23. Since p is a positive limit point of x
0
, using an argument analogous to what done in part 4,
we infer the existence of times s
i
iN
, s
i
< s
i+1
such that x
i
:= (s
i
, x
0
) S, x
i
p, and the sequence
x
i
is monotone along S. Since (T, p) = p, and since x
i
p, by continuity of with respect to initial
conditions we have that for i sufciently large, (T, x
i
) V. By part 1, there exists a unique
i
[, ]
such that x
i+1
= (T +
i
, x
i
) S. Therefore, (s
i+1
, x
0
) = (T +
i
+s
i
, x
0
), or s
i+1
s
i
= T +
i
T +.
By Theorem 2.31, there exists > 0 and a constant L > 0 such that if |x
i
p| < , then
|(t, x
i
) (t, p)| |x
i
p|e
Lt
for all t [0, T + ]. Now consider the function
[s
i
, s
i+1
] R, t |(t, x
0
)|
.
For large enough i, |x
i
p| < and
|(t, x
0
)|
|(t, x
0
) (t s
i
, p)| (since (t s
i
, p) )
|(t s
i
, (s
i
, x
0
)) (t s
i
, p)|
|(t s
i
, x
i
) (t s
i
, p)|
|x
i
p|e
L(ts
i
)
|x
i
p|e
L(T+)
for all t [s
i
, s
i+1
] [s
i
, s
i
+ T + ]. Taking the limit as i of the above inequality corresponds to
taking the limit as t , and it yields |(t, x
0
)|
0.
Remark 3.24. Part 2 of the proof tacitly uses the following result: a closed planar curve with no self-
intersections separates the plane into two connected regions. This fundamental fact is known as the
Jordan curve theorem. Its proof can be found in a book on topology.
The Poincar-Bendixson theorem has a number of important consequences.
Theorem 3.25. Let be a compact positively invariant set for (3.3). Then, contains either an equilibrium or a
closed orbit (or both).
Proof. Since is compact and positively invariant, the positive semiorbit through any point in is a
bounded set. The result then follows from Theorem 3.8 and Poincar-Bendixsons theorem.
Example 3.26. The system
x
1
= x
1
+ x
2
x
1
(x
2
1
+ x
2
2
)
x
2
= 2x
1
+ x
2
x
2
(x
2
1
+ x
2
2
)
has a unique equilibrium at the origin. Consider the annulus = (x
1
, x
2
) : 1/2 x
2
1
+ x
2
2
3/2.
Using Theorem 3.17 we show that is positively invariant. Let (x
1
, x
2
) = 1/2 x
2
1
x
2
2
. Then,
L
v
(x
1
, x
2
) = 2(x
2
1
+ x
2
2
)(1 x
2
1
x
2
2
) + 2x
1
x
2
.
We need to show that, when = 0, L
v
0. Recall that (see Exercise 1.5) x
1
x
2
x
2
1
/2 + x
2
2
/2. Hence,
L
v
[
x
2
1
+x
2
2
=1/2
2
1
2

1
2
+
1
2
= 0,
and therefore the set (x
1
, x
2
) : x
2
1
+ x
2
2
1/2 is positively invariant. Next, let (x
1
, x
2
) = x
2
1
+ x
2
2
3/2.
Then,
L
v
(x
1
, x
2
)[
x
2
1
+x
2
2
=3/2
=
_
2(x
2
1
+ x
2
2
)(1 x
2
1
x
2
2
) 2x
1
x
2
x
2
1
+x
2
2
=3/2
0.
So the set (x
1
, x
2
) : x
2
1
+ x
2
2
3/2 is positively invariant. In conclusion, is positively invariant and
by Theorem 3.25 it contains a closed orbit. The gure below conrms our prediction.
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
-1.5
-1
-0.5
0
0.5
1
1.5
2
x
1
x
2
Exercise 3.9. In this problem you will investigate a property of certain conservative systems with one degree of freedom.
Consider a second-order system
x
1
= x
2
x
2
= f (x
1
),
where f is locally Lipschitz on R and such that f (0) = 0, and x
1
f (x
1
) > 0 for all x
1
,= 0. Notice that the mass-spring system
in Exercise 3.7 ts this model.
1. Show that the energy E(x
1
, x
2
) =
_
x
1
0
f ()d +
1
2
x
2
2
is a nonnegative continuous function.
2. For any > 0, show that there exists E
> 0 such that, for all c (0, E
), the set E
c
= (x
1
, x
2
)

B
2
(0) : E(x
1
, x
2
) = c
is contained in B
2
(0).
[Hint: dene E
= min
|x|=
E(x
1
, x
2
).]
3. Show that E
c
, c (0, E
), is closed (and hence, by part 2, compact) and invariant.

4. Using the result in part 3, show that all orbits through points sufciently near the origin are closed curves.
Thus, for small initial conditions, all solutions are periodic oscillations.
Theorem 3.27. Let be a closed orbit of (3.1) and let U be the region delimited by . Then, U contains an
equilibrium point.
The proof of this result can be found in [HS74]. This theorem is useful because it provides a simple
necessary condition for the existence of closed orbits. Taking the logical negation of the statement, we
obtain a sufcient condition for the non-existence of closed-orbits: If there are no equilibria in a planar
simply connected domain D
2
, then there cannot exist a closed orbit entirely contained in D.
2
Roughly speaking, a simply connected domain is a connected domain with no holes. More precisely, the region delimited
by any closed curve in the domain is a subset of the domain.
The following result provides another sufcient condition for the non-existence of closed orbits.
Theorem 3.28 (Bendixsons Criterion). If, on a simply connected domain D R
2
, the function
f
1
x
1
+
f
2
x
2
is not identically zero and does not change sign, then (3.3) has no closed orbits lying entirely in D.
Proof. Suppose that there exists a periodic integral curve (x
1
(t), x
2
(t)) with period T such that the cor-
responding closed orbit is entirely contained in D. Consider the following trivial identity,
_
T
0
f
2
(x
1
(t), x
2
(t)) x
1
+ f
1
(x
1
(t), x
2
(t)) x
2
dt = 0,
and note that the left-hand side is the line integral of the vector function (f
2
, f
1
) along :
_
f
2
(x
1
, x
2
)dx
1
+ f
1
(x
1
, x
2
)dx
2
=
_
T
0
f
2
(x
1
(t), x
2
(t)) x
1
+ f
1
(x
1
(t), x
2
(t)) x
2
dt = 0.
Since the vector eld ( f
1
, f
2
) is continuously differentiable, so is the vector eld (f
2
, f
1
). Then, letting
R be the domain delimited by , Greens theorem gives
__
R
_
f
1
x
1
+
f
2
x
2
_
dx
1
dx
2
=
_
f
2
(x
1
, x
2
)dx
1
+ f
1
(x
1
, x
2
)dx
2
= 0.
If
f
1
x
1
+
f
2
x
2
is not identically zero and does not change sign, the expression above cannot be zero.
Exercise 3.10. Consider the system of Exercise 3.9 and suppose the system has dissipation,
x
1
= x
2
x
2
= f (x
1
) g(x
2
),
where g is locally Lipschitz on R and such that g(0) = 0 and g
(x
2
) > 0 for all x
2
,= 0. Show that the system cannot have any
closed orbits.
3.4 Stability of closed orbits
This section presents the classical theory of stability of closed orbits developed by Poincar. The reference
for this material is [HS74]. Consider once again the dynamical system
x = f (x), x A, (3.4)
where A R
n
is open and f is C
1
. Suppose that the system has a nontrivial closed orbit through a
point p A. Let T be the minimum period of the integral curve (t, p).
Denition 3.29. The closed orbit is asymptotically stable (or orbitally asymptotically stable) if
( > 0)( > 0)(x
0
A) |x
0
|
< = (t 0) |(t, x
0
)|
< ,
and |(t, x
0
)|
0 as t +.
3.4 STABILITY OF CLOSED ORBITS 61
In other words, if the initial condition x
0
is sufciently close to , then the corresponding phase curve
remains close to for all future time and approaches as t . The limit cycle in Van der Pols
oscillator is an example of asymptotically stable closed orbit.
In this section we present sufcient conditions for the asymptotic stability of . Unfortunately, the
conditions are difcult to check in practice because they rely on the knowledge of the ow of (3.4).
However, they can be checked approximately using numerical methods. Most importantly, they are very
important in understanding the nature of the stability problem.
Poincars idea is described in the gure below. It relies on the the notion of local section already seen
in Poincar-Bendixsons theorem. This notion is generalized from the plane to R
n
in a straightforward
manner. Let S be an open subset
3
of an n 1 dimensional hyperplane through p transverse to f . That
is, S is such that, for each x in S, the vector f (x) is not tangent to S and not zero either.
S
S
0
p
x
0
x
1

Picking an initial condition x
0
in S sufciently close to p, the phase curve through x
0
intersects S again at
a point x
1
. If x
1
is sufciently close to p, then the same phase curve will intersect S again at some point
x
2
. If the sequence of points x
i
thus obtained converges to p, it is clear that the phase curve through x
0
approaches the closed orbit . If the same is true for any initial condition x
0
in a small neighborhood
of p, then is asymptotically stable. Thus, the stability problem is reduced to the study of the properties
of a discrete-time system x
k+1
= g(x
k
). More precisely, for any x
0
in a sufciently small neighborhood
W of p, there is a time (x
0
) > 0 at which the phase curve intersects S, i.e., such that ((x
0
), x
0
) S.
As a matter of fact, there is a unique map x
0
(x
0
) which is C
1
and has the properties
(x
0
W) ((x
0
), x
0
) S, (p) = T.
The proof of this fact relies on the implicit function theorem and is found in [HS74, p.243]. Let S
0
= SW
and dene g : S
0
S as
g(x) = ((x), x).
Given x S
0
, g(x) represents the next intersection of the phase curve through x with the section S.
The function g is called a Poincar map. Note that g is C
1
because and are C
1
. Now consider the
discrete-time system
x
k+1
= g(x
k
), x
k
S
0
, k Z
+
(3.5)
0
. The state space S
0
of the system is an open subset of an n 1-dimensional
vector space. Note that g(p) = ((p), p) = (T, p) = p. Therefore, if x
0
= p then x
k
= p for all k N.
3
By an open subset of an hyperplane is meant the intersection of the hyperplane with an open set in R
n
.
The point p is said to be an equilibrium of the discrete-time system (3.5).
Denition 3.30. The equilibrium p of (3.5) is asymptotically stable if
( > 0)( > 0)(x
0
A) x
0
B
n
(p) S
0
= (k Z
+
) x
k
B
n
(p) S
0
,
and x
k
p.
In other words, if one picks x
0
S
0
close to p, then the sequence x
k
S
0
remains close to p and
it asymptotically approaches p. It turns out that asymptotic stability of p is equivalent to asymptotic
stability of .
Theorem 3.31. The closed orbit is asymptotically stable if and only if the equilibrium p of (3.5) is asymptotically
stable.
Proof. ( =) Suppose that is asymptotically stable. We need to show that p is asymptotically stable.
For any > 0, let
> 0 be such that

x A : |x|
<
S
0
B
n
(p) S
0
.
In other words, the tube of radius
around is small enough that its intersection with S

0
is contained
in the ball of radius in S
0
centred at p. The stability of implies that there exists
> 0 such
that, for all x
0
A with |x
0
|
<
, |(t, x
0
)|
<
for all t 0. Pick > 0 small enough that

B
n
(p) x A : |x|
<
. Then, x
0
B
n
(p) S
0
implies |x
0
|
<
, which gives |(t, x

0
)|
<
for all t 0. Hence, in particular,

x
k
x A : |x|
<
S
0
B
n
(p) S
0
for all k Z
+
. It remains to be shown that x
k
p. This is an obvious consequence of |(t, x
0
)|
0.
( =) Next, suppose that p is asymptotically stable. We need to show that is asymptotically stable. For
any > 0, let W be an open set containing such that if x
0
W, then |(t, x
0
)|
< for all t [0, 2T)

(recall that T is the period of ). Since the function (x) is continuous and (p) = T, there exists
> 0
such that for all x B
n
(p) S
0
, (x) < 2T. If necessary,
can be made smaller in order to guarantee

that B
n
(p) W. Since p is asymptotically stable, there exists
> 0 such that if x

0
is in B
n
(p) S
0
, then
x
k
B
n
(p) S
0
for all k 0. Dene U = (t, x
0
) : x
0
B
n
(p) S
0
, t 0. The set U is positively
invariant because it is a collection of positive semiorbits. By denition, any x U can be expressed
as x = (t, x
0
) for some x
0
B
n
(p) S
0
and some t 0. Actually, t can be always chosen such that
t [0, 2T). For, letting x
k
be the sequence generated by (3.5) with initial condition x
0
, x is a point
between some x
k
and x
k+1
. Then, there exists t
0 such that (t, x

0
) = (t
, x
k
). Since x
0
B
n
(p) S
0
,
it follows that x
k
B
n
(p) S
0
. Since B
n
(p) S
0
W, we have (x
k
) < 2T, and thus t
[0, 2T). So
the pair (t, x
0
) can be replaced by (t
, x
k
). We have thus reached the conclusion that any x U can be
expressed as x = (t
, x
k
), with x
k
B
n
(p) S
0
W and t
[0, 2T). By the way W was dened, the

above readily implies that U x : |x|
< . Now let > 0 be small enough that x : |x|
< U.
3.4 STABILITY OF CLOSED ORBITS 63
If |x
0
|
< , then x
0
U, and thus (t, x
0
) U for all t 0, implying that |(t, x
0
)|
< for all t 0.

We are left to show that |(t, x
0
)|
0 as t +. This result follows from the continuity of , the

fact that x
k
p, and (x
k
) [0, 2T).
Exercise 3.11. What guarantees the existence of the set W and the positive number in the ( =) part of the proof of
Theorem 3.31?
Because of Theorem 3.31, we focus our attention on the stability of the equilibrium p of the discrete-time
system (3.5). The next result gives a sufcient condition for asymptotic stability. See [HS74] for its proof.
Well see an analogous result for continuous-time systems in Section 3.5.2.
Theorem 3.32. If all eigenvalues of dg
p
have magnitude less than one, then p is an asymptotically stable equilib-
rium. If at least one of the eigenvalues of dg
p
has magnitude greater than one, then is unstable in the sense that
there exist initial conditions arbitrary close to yielding phase curves that diverge from .
The theorem above provides a conceptually simple stability criterion: pick a point p and a local
section of f at p; form the Poincar map g; compute its Jacobian at p, and check its eigenvalues. If all
eigenvalues of dg
p
have magnitude less than or equal to one, and some of them have magnitude one,
then the theorem does not allow one to draw any conclusion about the stability of .
Despite its conceptual simplicity, the criterion is difcult to apply in practice because it is impossible, in
many cases, to compute the function g analytically. As a matter of fact, notice that in order to nd g one
has to explicitly know the ow , which in most cases is not available. Nevertheless, it is not difcult to
nd approximations to dg
p
by means of numerical integration of the nonlinear system (3.4).
A remark about the computation of dg
p
. Since the domain and codomain of g are open subsets of an
n 1 dimensional vector space, by expressing g in the coordinates of this vector space one writes
g(x) = (g
1
(x
1
, . . . , x
n1
), . . . , g
n1
(x
1
, . . . , x
n1
)) .
Thus, dg
p
is an n 1 n 1 matrix. The next example, illustrating the Poincar methodology, makes
this clear.
x
1
= x
1
x
2
x
1
(x
2
1
+ x
2
2
)
x
2
= x
1
+ x
2
x
2
(x
2
1
+ x
2
2
).
The set S = (x
1
, x
2
) : x
1
> 0, x
2
= 0 is a local section of the vector eld because, on S, x
2
> 0 and hence
the vector eld is transverse to S. Transform the system to polar coordinates (r, ), with r =
_
x
2
1
+ x
2
2
,
= arg(x
1
+ i x
2
).
r = r(1 r
2
)
= 1.
r > 0, R(mod2).
The system in polar coordinates is composed of two decoupled subsystems. For any r
0
> 0, the so-
lution r(t) of the rst subsystem approaches 1 asymptotically. That means that the system in (x
1
, x
2
)
coordinates has a stable limit cycle which is a circle of radius 1.
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
S
x
2
x
1
Lets verify this using the result in Theorem 3.32. The section in polar coordinates is S = (r, ) : r >
0, = 0. The ow of the system in polar coordinates is
(t, (r
0
,
0
)) =
_
_
1 +
1 r
2
0
r
2
0
e
2t
_
1/2
, t +
0
_
.
We use the knowledge of the phase ow to determine a Poincar map g on S. A point in (r, ) in S
is entirely specied by its rst component, hence well express g as a function of r. Given an initial
condition (r
0
, 0) in S, the time it takes for the phase curve to cross S again is (r
0
) = 2. The value of r
after time (r
0
) is
_
1 +
1r
2
0
r
2
0
e
4
_
1/2
. Thus, g : S S is dened as
g(r) =
_
1 +
1 r
2
r
2
e
4
_
1/2
.
We nd the equilibria of g by solving the equation
g(r) = r,
which has only one positive solution, r = 1. As expected, there is only one closed orbit passing through
the point (r, ) = (1, 0). The derivative of g at 1 is
dg
1
= g
(1) = e
4
< 1.
Thus, the unit circle in (x
1
, x
2
) coordinates is asymptotically stable.
Exercise 3.12. In this exercise youll develop a nave numerical method to compute the Jacobian of the Poincar map in a
simple two-dimensional example. Consider the system in Example 3.26, for which we showed that there is at least one closed
orbit in the annulus . Let S =
_
(x
1
, x
2
) :

1/2 < x
1
<
3/2, x
2
= 0
_
.
1. Show that S is a local section for the vector eld.
2. Using a numerical software, take an initial condition x
0
on S and compute the solution of the system for long enough
time that the solution is approximately on the limit cycle. Save the nal state x
f
of your simulation.
[If you are using Matlab, use the function ode45.]
3.5 LINEARIZATION OF NONLINEAR SYSTEMS 65
3. Numerically solve the differential equation with initial condition x
f
until the solution crosses S for the rst time. The
crossing point, being on S and lying approximately on the limit cycle, can be considered the equilibrium p of the
Poincar map.
[If you are using Matlab, look at the Events option in odeset.]
4. Write a Matlab function that approximately implements the Poincar map. This should be a function g(x
1
) : S S.
5. Using the Matlab function, use a nite difference approximation of the derivative to numerically compute dg
p
. Verify
that [dg
p
[ < 1.
3.5 Linearization of Nonlinear Systems
x = f (x), x A, (3.6)
where A R
n
is open and f is C
1
. Suppose that the origin x = 0 is an equilibrium point of (3.6). There
is no loss of generality in assuming that the equilibrium is at the origin. For, if x
,= 0 is an equilibrium,
let y = x x
, then
y = x 0 = f (x) = f (y + x
).
Letting

f (y) = f (y + x
), we have that y = 0 is an equilibrium of y =

f (y).
The linearization of (3.6) at 0 is the linear time-invariant system
z = (d f
0
)z. (3.7)
In this section we answer this question: what properties of (3.6) can be inferred from its lineariza-
tion (3.7)?
3.5.1 Stable, Centre, and Unstable Subspaces of Linear Time-Invariant
Systems
We review in this section some structural properties of the linear time-invariant system
x = Ax, x R
n
.
Let
1
, . . . ,
r
be the set of real eigenvalues of A, and
1
,
1
, . . . ,
s
,
s
be the set of complex eigen-
values of A. We have r + 2s = n.
For simplicity of exposition, our discussion is conned to the situation when A is diagonalizable. The
general situation is handled by replacing eigenvectors by generalized eigenvectors and essentially re-
peating the same discussion. The diagonalizability of A implies that there are n linearly independent
eigenvectors. Partition the eigenvectors in two sets v
1
, . . . , v
r
and w
1
, w
1
, . . . , w
s
, w
s
, with Av
i
=
i
v
i
and Aw
j
=
j
w
j
. For i = 1, . . . r, the real eigenspaces 1
i
= spanv
i
are A-invariant because if x 1
i
,
i.e., x = cv
i
with c R, then Ax = c
i
v
i
is in 1
i
as well. The complex eigenspaces can also be
used to dene A-invariant real subspaces. Indeed, for each i = 1, . . . , s, the two dimensional subspace
J
i
= spanRe(w
i
), Im(w
i
) is A-invariant.
Exercise 3.13. Prove this statement.
[Hint: begin by observing that spanw
i
, w
i
is A-invariant.]
Let E
s
be the sum of all subspaces 1
i
, J
j
associated to eigenvalues with negative real part. This is
called the stable subspace. Similarly, dene the centre subspace E
c
to be the sum of all subspaces 1
i
,
J
j
associated to eigenvalues with zero real parts, and the unstable subspace E
u
to be the sum of all
subspaces 1
i
, J
j
associated to eigenvalues with positive real part.
Lemma 3.34. The subspaces E
s
, E
c
, and E
u
are A-invariant. Moreover, they are independent and R
n
= E
s
E
c
E
u
.
Proof. In order to show that E
s
, E
c
, and E
u
are A-invariant, we need to show that the sum of A-invariant
subspaces is A-invariant. Let 1
1
and 1
2
be two A-invariant subspaces and let x = v
1
+ v
2
, with v
i
1
i
for i = 1, 2. Then, Ax = Av
1
+ Av
2
. The A-invariance of 1
1
and 1
2
gives Av
i
1
i
, so Ax 1
1
+1
2
.
In order to show E
s
, E
c
, and E
u
are independent, we need to show that all subspaces 1
i
, J
j
are in-
dependent. Since A is diagonalizable, all eigenvectors are linearly independent over the eld C and
thus
C
n
= spanv
1
spanv
r
spanw
1
, w
1
spanw
s
, w
s
. (3.8)
We use this fact to show that v
1
, . . . , v
r
, Re(w
1
), Im(w
1
), . . . , Re(w
s
), Im(w
s
) is a linearly independent
set over the eld R. Suppose that, for some real numbers c
1
, . . . , c
r
, d
1
, . . . , d
2r
,
c
1
v
1
+ + c
r
v
r
+ d
1
Re(w
1
) + d
2
Im(w
1
) + + d
2r1
Re(w
r
) + d
2r
Im(w
r
) = 0.
Using the fact that Re(w
i
) =
1
2
(w
i
+ w
i
) and Im(w
i
) =
1
2i
(w
i
w
i
) we rewrite the equation above as
c
1
v
1
+ + c
r
v
r
+
d
1
i d
2
2
w
1
+
d
1
+ i d
2
2
w
1
+ +
d
2r1
i d
2r
2
w
r
+
d
2r1
+ i d
2r
2
w
r
= 0.
By (3.8), the identity above can only hold if
c
1
= = c
r
= 0, d
1
id
2
= 0, . . . , d
2r1
d
2r
= 0,
or, equivalently, c
i
= 0 for i = 1, . . . , r and d
j
= 0 for j = 1, . . . , 2r. This proves that
R
n
= spanv
1
spanv
r
spanRe(w
1
), Im(w
1
) spanRe(w
s
), Im(w
s
),
which in turn implies that E
s
, E
c
, and E
u
are independent and R
n
= E
s
E
c
E
u
.
The Lemma above allows us to decompose the ow of a linear time-invariant system into three de-
coupled parts. We now discuss this decomposition in detail. Let n
s
= dim(E
s
), n
c
= dim(E
c
), and
n
u
= dim(E
u
). By the Lemma, n
s
+ n
c
+ n
u
= n. Consider the linear coordinate transformation y = Tx,
where the rst n
s
columns of T
1
form a basis for E
s
, the next n
c
columns of T
1
form a basis for E
c
,
and the last n
u
columns of T
1
form a basis for E
u
. The A-invariance of E
s
, E
c
, and E
u
implies that, in y
coordinates, the differential equation reads
y =
_
_
A
s
0 0
0 A
c
0
0 0 A
u
_
_y,
where A
s
R
n
s
n
s
, A
c
R
n
c
n
c
, and A
u
R
n
u
n
u
are matrices whose eigenvalues have negative, zero,
and positive real parts, respectively.
Exercise 3.14. Justify the block diagonal structure of the differential equation in y-coordinates.
We have just determined that in y coordinates the system is decomposed into three decoupled subsys-
tems
y
1
= A
s
y
1
y
2
= A
c
y
2
y
3
= A
u
y
3
.
All nonzero integral curves of the y
1
subsystem approach the origin with exponential decay, while those
of the y
3
subsystem grow unbounded at an exponential rate. On the other hand, integral curves of the
y
2
subsystem may be constant or periodic
4
.
Lets return to x coordinates and use the decomposition above to draw conclusions about the phase
ow. The subspaces E
s
, E
c
, and E
u
are invariant sets for x = Ax. The motion on E
s
is an exponential
decay to the origin, while that on E
u
is characterized by exponential growth. The motion on E
c
is either
constant or periodic. The motion off of these three subspaces is a composition of the three types just
discussed. If, for instance, E
c
= 0, then all phase curves through initial conditions not in E
s
approach E
u
asymptotically and in doing so they diverge from the origin.
Example 3.35. Let A =
_
_
1 3 0
3 1 0
0 0 1
_
_. The eigenvalues are

1
= 1,
1
= 1 + 3i, and
1
= 1 3i.
The eigenvectors are v
1
= [0 0 1]
, w
1
= [1 i 0]
, and w
1
= [1 i 0]
. We have J
1
= E
s
=
span[1 0 0]
, [0 1 0]
, 1
1
= E
u
= span[0 0 1]
, and E
c
= 0. Integral curves on E
s
spiral
toward the origin. Therefore, integral curves through initial conditions not in E
s
spiral toward E
u
while
diverging from the origin.
-1 ?1
-0.5
0
0.5
1
?0.5
0
-0.5
-1
-25
-20
-15
-10
-5
0
5
10
15
20
25
x
2
x
1
x
3
E
s
E
u
4
If A is not diagonalizable and there are repeated eigenvalues with zero real part whose associated Jordan block has
dimension > 1, then some integral curves of the y
2
subsystem grow unbounded.
3.5.2 Stable and Unstable Manifolds of Nonlinear Systems
We return in this section to the nonlinear system (3.6). Given a neighborhood V of 0, we dene two sets
W
s
V
= x
0
V : O
+
(x
0
) V, (t, x
0
) 0 as t +
W
u
V
= x
0
V : O
(x
0
) V, (t, x
0
) 0 as t .
Note that W
s
V
and W
u
V
are subsets of V. The set W
s
V
is positively invariant, while W
u
V
is negatively
invariant. By denition, positive semiorbits through initial conditions in W
s
V
are contained in V and
approach the origin asymptotically. In contrast, phase curves through initial conditions in W
u
V
diverge
from the origin. The sets W
s
V
and W
u
V
are nonlinear generalizations of stable and unstable subspaces of
linear time-invariant systems.
Before introducing the next fundamental result, well present an informal denition of a manifold which
is incomplete and not entirely precise, but is sufcient for the scope of these notes.
A set M R
n
is a k-dimensional C
1
manifold if for any point p in M, there exists a neighborhood V of
p and a C
1
injective function : W R
n
, where W R
k
is an open set, such that (W) = V M.
R
k
M
V M
W
V
Roughly speaking, a C
1
manifold is a surface without corners. Each function is a parametrization of
a patch V M of the surface. Spheres and cylinders in R
3
are examples of two dimensional manifolds.
A one dimensional manifold in R
n
is just a regular curve without self-intersections. Notice that an
n-dimensional manifold is an open subset of R
n
.
If h : R
k
R
nk
is a C
1
function, then its graph M = (x
1
, x
2
) R
k
R
nk
: x
2
= h(x
1
) is a k-
dimensional C
1
manifold in R
n
. As a matter of fact, in this case one can take V = R
n
, and parametrize
the entire set with one function : R
k
R
n
dened as : x
1
(x
1
, h(x
1
)). Since h is C
1
, is also C
1
.
Moreover, is clearly injective ((x
1
) = (x
2
) = x
1
= x
2
) and its image is precisely M.
The next result states that in a sufciently small neighborhood V of an equilibrium point (which, without
loss of generality, we take to be the origin) the sets W
s
V
and W
u
V
are graphs of functions, and hence
manifolds. Whats more, at the equilibrium these graphs are tangent to the stable and unstable subspaces
of the linearization. Before stating the result, given a subspace E R
n
, we denote by E
the orthogonal
complement of E in R
n
, dened as
E
= v R
n
: (w E) v, w = 0,
where is the Euclidean inner product. By using coordinates x
1
and x
2
for E and E
, respectively, we
can represent a point in R
n
as a pair (x
1
, x
2
).
Theorem 3.36 (Stable Manifold Theorem). Let 0 be an equilibrium of (3.6) and suppose that d f
0
has n
s
eigenvalues with negative real part, n
u
eigenvalues with positive real part, and no eigenvalues with zero real part.
Let E
s
and E
u
be the stable and unstable subspaces of the linearization (3.7). Then, there exists a neighborhood V
of 0 and C
1
functions h
s
: E
s
V (E
s
)
, h
u
: E
u
V (E
u
)
such that
W
s
V
= (x
1
, x
2
) (E
s
V) (E
s
)
: x
2
= h
s
(x
1
)
W
u
V
= (y
1
, y
2
) (E
u
V) (E
u
)
: y
2
= h
u
(y
1
),
and h
s
(0) = 0, h
u
(0) = 0, dh
s
0
= 0, dh
u
0
= 0. Moreover, solutions on W
s
V
[W
u
V
] approach 0 as t +
[t ] at an exponential rate
5
The theorem, illustrated in the gure below, states that if V is sufciently small the sets W
s
V
and W
u
V
can
be represented as graphs of C
1
functions and hence they are C
1
manifolds, commonly referred to as the
stable and unstable manifolds of the equilibrium. Their dimensions are n
s
and n
u
, respectively. Further,
the theorem states that the stable and unstable manifolds contain the origin (h
s
(0) = 0, h
u
(0) = 0), and at
the origin they are tangent to the stable and unstable subspaces of the linearization (dh
s
0
= 0, dh
u
0
= 0). If
the equilibrium of interest is a point x
,= 0, then the origin of the subspaces E

s
, E
u
, and their orthogonal
complements would no longer be the point x = 0, but rather the point x
, and the gure below would

be centred at x
.
0
x
1
V
W
s
V
W
u
V
y
1
h
s
(x
1
)
h
u
(y
1
)
E
s
E
u
E
s
V
E
u
V
Example 3.37. Consider the nonlinear system
x
1
= x
1
x
2
= x
2
+ x
2
1
.
The origin is the only equilibrium. The linearization at 0 is
z
1
= z
1
z
2
= z
2
.
Hence, n
s
= 1 and n
u
= 1. The stable and unstable subspaces are E
s
= span[0 1]
and E
u
=
span[1 0]
. We expect to have one dimensional stable and unstable manifolds, i.e, curves. The
5
More precisely, ( > 0) (x
0
W
s
V
) [(x
0
W
u
V
)] (c > 0) (t 0) [(t 0)] (|(t, x
0
)| ce
t
).
nonlinear system is integrable. For, writing
dx
1
= x
1
dt
dx
2
= x
2
dt + x
2
1
dt,
and eliminating dt from the second equation and rearranging terms we obtain
(x
2
x
2
1
)dx
1
+ x
1
dx
2
= 0.
The LHS of this equation is an exact differential. That is, it is the differential of the real-valued function
(x
1
, x
2
) = x
1
x
2
x
3
1
/3 + C, where C is an arbitrary real number. This fact implies (by Theorem 3.14)
that all level sets
(x
1
, x
2
) R
2
: x
1
x
2
x
3
1
/3 = K, K R
are invariant sets of the nonlinear system. We are interested in the invariant set through the origin, i.e.,
the level set
(x
1
, x
2
) R
2
: x
1
x
2
x
3
1
/3 = 0 = (x
1
, x
2
) R
2
: x
1
= 0 (x
1
, x
2
) R
2
: x
2
= x
2
1
/3.
The set x
1
= 0 is itself an invariant set. On it, the x
2
dynamics reads x
2
= x
2
, and hence solutions
converge to the origin exponentially. The set x
2
= x
2
1
/3 is also invariant (check, using Theorem 3.14).
On it, the x
2
dynamics reads x
2
= 2x
2
, and hence solutions diverge from the origin exponentially. We
conclude that
W
s
R
2
= (x
1
, x
2
) : x
1
= 0
W
u
R
2
= (x
1
, x
2
) : x
2
= x
2
1
/3.
-2 -1.5 -1 - 0.5 0 0.5 1 1.5 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
E
u
E
s
= W
s
R
2
W
u
R
2
The stable manifold theorem can be strengthened as follows. If the linearization has some eigenvalues
on the imaginary axis, then the sets M
s
= (x
1
, x
2
) (E
s
V) (E
s
)
: x
2
= h
s
(x
1
) and M
u
=
(y
1
, y
2
) (E
u
V) (E
u
)
: y
2
= h
u
(y
1
) in Theorem 3.36 are contained in W
s
V
and W
u
V
, respectively.
Moreover, all phase curves through initial conditions that are in W
s
V
but are not in M
s
converge to the
origin asymptotically but not exponentially. An analogous statement holds for W
u
V
and M
u
. It is important
to notice that while the sets M
s
and M
u
are still C
1
manifolds, the sets W
s
V
and W
u
V
may not be manifolds.
Example 3.38. Consider the nonlinear system
x
1
= x
1
x
2
= x
2
2
,
with unique equilibrium at 0 and linearization
z
1
= z
1
z
2
= 0.
The linearization has eigenvalues
1
= 0,
2
= 1 and E
s
= span[1 0]
, E
c
= span[0 1]
. Every
orbit in the closed lower-half plane converges to the origin, but only orbits on the x
1
axis converge to 0
exponentially.
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
W
s
V
M
s
Here, therefore,
M
s
= E
s
,
and
W
s
R
2
= (x
1
, x
2
) : x
2
0.
The set W
s
R
2
is not a manifold.
The strengthened version of the stable manifold theorem has two important implications:
1. If n
s
= n, that is, if the linearization (3.7) has all eigenvalues with negative real part, then W
s
V
is
an n-dimensional C
1
manifold. Since an n-dimensional manifold in R
n
is an open set, it follows
that all solutions in some neighborhood of the origin stay in the neighborhood and converge to the
origin exponentially. Later on well refer to this property as exponential stability of the origin.
2. If n
u
> 0, that is, if the linearization (3.7) has at least one eigenvalue with positive real part, then
W
u
V
is a C
1
manifold of dimension 1. Thus, in any neighborhood of the origin there are solutions
diverging from the origin exponentially. Thus, the origin in an unstable equilibrium.
We have already encountered analogous statements concerning the stability of the equilibrium of a
discrete-time system in Theorem 3.32.
On the other hand, if n
u
= 0 but n
c
> 0, that is, is the linearization has all eigenvalues with real part
0 and at least one eigenvalue with zero real part, then the theorem does not allow one to conclude
anything about stability of the origin of the nonlinear system. As a matter of fact, nothing about stability
of the equilibrium of (3.6) can be inferred from the linearization, as the next example illustrates.
Example 3.39. The system
x = ax
3
, a R
has a unique equilibrium at the origin and the linearization is
z = 0.
For any a R, the linearization has one eigenvalue at 0. For all a > 0 all nontrivial solutions diverge
from the origin, while for all a < 0 all solutions converge to the origin. However, the linearization does
not depend on a.
Exercise 3.15. Consider the frictionless pendulum
x
1
= x
2
x
2
=
Mgl
I
sin x
1
.
1. Linearize the pendulum about the equilibrium (, 0) and nd the stable and unstable subspaces of the linearization.
2. Find the stable and unstable manifolds of (, 0).
[Hint: use arguments similar to those in Example 3.37 or consider the level sets of the energy.]
3. Verify that the manifolds are tangent to E
s
and E
u
at (, 0).
3.5.3 The Hartman-Grobman Theorem
Theorem 3.36 indicates that when the linearization of (3.6) at an equilibrium has no eigenvalues on the
imaginary axis (such an equilibrium is called hyperbolic), then the stability of the equilibrium is entirely
characterized by the properties of the linearization. Thus, the stable manifold theorem goes a long way in
establishing a relationship between the phase ow of a nonlinear system near an hyperbolic equilibrium
and the phase ow of its linearization. It turns out that for hyperbolic equilibria this relationship can be
pushed even further.
Theorem 3.40 (Hartman-Grobman). Suppose that 0 is an hyperbolic equilibrium of (3.6). Let
1
(t, x
0
) be
the ow of (3.6) and
2
(t, x
0
) = e
d f
0
t
x
0
be the ow of (3.7). Then, there exists a neighborhood V of 0 and a
homeomorphism (a continuous bijection whose inverse is continuous) h : V

V such that h(0) = 0 and, if
1
(t, x
0
) V, then
h(
1
(t, x
0
)) =
2
(t, h(x
0
)). (3.9)
0
x
0
V
W
s
V
W
u
V
V
E
s
E
u
h(x
0
)
The theorem states that in a sufciently small neighborhood of an hyperbolic equilibrium any solution
of the nonlinear system is bijectively mapped onto a solution of its linearization with appropriate initial
condition. Dynamicists refer to property (3.9) by speaking of topological equivalence of the two ows
1
and
2
or by saying that
1
and
2
are C
0
-conjugate. In general, it is impossible to nd a differentiable
map h satisfying (3.9), unless special additional assumptions are satised.
While nding the homeomorphism h is in practice difcult, the Hartman-Grobman theorem has an
important practical implication. The behavior of solutions near an hyperbolic equilibrium is analogous
to the behavior of solutions of its linearization. If, for instance, the linearization of a second-order system
is a stable focus, then solutions of the nonlinear system will spiral toward the equilibrium.
Exercise 3.16. Find all equilibria of the following systems and classify, if possible, each equilibrium by linearizing the system
about it. Specically, nd eigenvectors and eigenvalues, and use them to sketch the ow in the vicinity of the equilibrium.
Compare your prediction to computer plots (if you use Matlab, use the function streamslice).
1.
x
1
= x
2
1
x
2
2
x
2
= (x
2
+ 1)(x
2
1)
2.
x
1
= x
2
x
2
= sin x
1
3.
x
1
= x
1
+ x
2
1
x
2
= x
1
+ x
2
.
75
Chapter 4
Stability Theory
One of the important properties of dynamical systems of interest to control theorists is that of stabil-
ity. This chapter is devoted to the exposition of Lyapunovs theory of stability for nite dimensional
dynamical systems. The reference for this material is [Kha02].
4.1 Stability of equilibria
x = f (x), x A (4.1)
where A R
n
is open and f is either locally Lipschitz on A or C
1
. We assume throughout this chapter
that 0 A.
Denition 4.1. Suppose that x = 0 is an equilibrium point of (4.1).
(i) x = 0 is stable if
( > 0)( > 0)(x
0
B
n
(0))(t 0) (t, x
0
) B
n
(0).
(ii) x = 0 is attractive if
( > 0)(x
0
B
n
(0)) (t, x
0
) 0 as t .
It is globally attractive if for all x
0
A, (t, x
0
) 0 as t .
(iii) x = 0 is asymptotically stable if it is stable and attractive, and globally asymptotically stable if it
is stable and globally attractive.
(iv) x = 0 is exponentially stable if
(, c, d > 0)(x
0
B
n
(0)) |(t, x
0
)| c|x
0
|e
dt
.
It is globally exponentially stable if
(c, d > 0)(x
0
A) |(t, x
0
)| c|x
0
|e
dt
.
76 CHAPTER 4. STABILITY THEORY
(v) x = 0 is unstable if it is not stable.
The attractivity notion in part (ii) of the denition requires that all integral curves originating in a
neighborhood of the origin be well-dened for all t 0. The same requirement is found in the stability
denition (part (i)), but since in this case positive semiorbits originating in B
n
(0) are contained in B

n
(0),
by Corollary 2.29 it automatically holds that T
+
x
0
= [0, +) for all x
0
B
n
(0).
As discussed in Section 3.5, there is no loss of generality in the assumption that x = 0 is an equilibrium
of (4.1) because, if the equilibrium of interest is x
,= 0, then letting x = x x
, the resulting dynamical

system with state x has an equilibrium at the origin.
The notion of stability is illustrated below. The gure to the right depicts integral curves, while the
gure to the left depicts the corresponding phase curves.
x
1
x
1
x
2
x
2
t
B
n
(0)
B
n
(0)
Example 4.2. Consider a frictionless mass-spring system with unit mass and unit spring constant,
x
1
= x
2
x
2
= x
1
.
This is just the harmonic oscillator of Example 2.9. Phase curves are concentric circles centred at the
origin. The origin is a stable equilibrium, but it is not asymptotically stable. In this case, the requirement
of Denition 4.1(i) is met with = .
Next, suppose that the mass-spring system is affected by viscous friction,
x
1
= x
2
x
2
= x
1
2x
2
.
The phase curves now spiral to the origin, and the origin is a globally asymptotically stable equilibrium.
We can say even more. The phase ow is given by
(t, x(0)) =
_
(1 + t)e
t
te
t
te
t
e
t
(1 t)
_
x(0),
4.1 STABILITY OF EQUILIBRIA 77
and thus, for all t 0,
|(t, x
0
)|

_
_
_
_
_
_
(1 + t)e
t
te
t
te
t
e
t
(1 t)
__
_
_
_
_
|x(0)|
e
t
(1 + 2t)|x(0)|
2e
1/2t
|x(0)|
,
where we have used the fact that, for all t 0, 1 + 2t 2e
1/2t
. Thus, the origin of the mass-spring
system with friction is a globally exponentially stable equilibrium.
The notions of attractivity and stability are decoupled, in the sense that an equilibrium can be attractive
but unstable, as illustrated in the next example.
Example 4.3. Consider the system in polar coordinates (r, )
r = r(r 1)
= sin
2
(/2).
The equation for r has two equilibria r = 0 and r = 1. If r > 1 then r < 0, so solutions with initial
conditions outside of the unit disk remain outside of the disk and tend toward the unit circle. If 0 < r <
1, then r > 0, so solutions with nonzero initial conditions inside the unit disk asymptotically approach
the unit circle. All solutions with r(0) = 1 remain on the unit circle. The equation for has equilibria
at 0 modulo 2 and satises

0, implying that if the angle is initialized at 0 modulo 2, then it
remains constant; otherwise, the angle increases and asymptotically approaches 0 modulo 2. Putting
everything together, we obtain the following phase portrait.
1 0.5 0 0.5 1 1.5
1
0.5
0
0.5
1
x
1
x
2
x
The equilibrium x
= (1, 0) is attractive, since for any nonzero initial condition the corresponding phase
curve asymptotically approaches x
. However, x
is unstable. To see this, take an arbitrarily small ball

B
2
(x
). Pick r(0) = 1 and (0) > 0 small enough that the point cos((0), sin((0))) is inside B
2
(x
). The
phase curve through (r(0), (0)) is an arc of the unit circle, moving counterclockwise, and exists the ball
B
2
(x
), proving the instability of x
.
4.2 Lyapunovs main theorem
In this section we present the basics of Lyapunovs stability theory. Aleksandr Mikhailovich Lyapunov
(1857-1918) was a Russian mathematician and physicist. He developed his celebrated theory at the
turn of the century while investigating stability problems arising in celestial mechanics. It took thirty
years for researchers to fully appreciate the importance of Lyapunovs work and its relevance to control
theory, until N.G. Chetaev and I.G. Malkin used it in the 1930s to investigate problems in aeronautical
engineering such as the stabilization of spin in rockets. Lyapunov theory characterizes the stability
properties of Denition 4.1 in terms of the sign of the Lie derivative of a function along the vector eld
f . The intuition behind this theory is contained in this example.
Example 4.4. Return to the mass-spring system with friction of Example 4.2,
x
1
= x
2
x
2
= x
1
2x
2
,
and consider the function
V(x
1
, x
2
) =
1
2
_
3x
2
1
+ 2x
1
x
2
+ x
2
2
_
=
_
x
1
x
2
_
_
3
2
1
2
1
2
1
2
_
. .
P
_
x
1
x
2
_
,
which is a positive denite quadratic form whose level sets are ellipses. The Lie derivative of V along
the vector eld f = [x
2
x
1
2x
2
]
L
f
V(x) = dV
x
f (x) = 2x
Pf (x) = 2
_
x
1
x
2
_
P
_
x
2
x
1
2x
2
_
= x
2
1
x
2
2
is a negative denite quadratic form. Since L
f
V is always negative away from the origin, by Theorem 3.17
we deduce that the phase curves of the system enter all nonzero level sets of V. As illustrated in the
gure below, this indicates two things: rst, all phase curves approach the origin, and hence 0 is a
globally attractive equilibrium. Second, since all sublevel sets of V are positively invariant, it follows
that phase curves originating close to the origin must remain close to the origin, indicating that the
origin is a stable equilibrium.
2 1.5 1 0.5 0 0.5 1 1.5 2
2
1.5
1
0.5
0
0.5
1
1.5
2
x
1
x
2
4.2 LYAPUNOVS MAIN THEOREM 79
In conclusion, using the function V we are able to predict that the origin is a globally asymptotically
stable equilibrium of the mass-spring system with friction. It is useful to think of V as an energy-like
function, although V in this example does not represent the physical energy of the mass-spring system.
The fact that L
f
V is negative denite indicates that the energy is dissipated along solutions as long
as the system is not at equilibrium. In dissipating energy, the solutions asymptotically approach the
origin.
Before stating Lyapunovs main theorem, we need to extend the denitions of deniteness of a quadratic
form to more general functions.
Denition 4.5. Let V : D R be a function dened on a domain D A, and let p D.
(i) V is positive denite at p (abbreviated p.d. at p) if V(p) = 0 and V(x) > 0 for all x ,= p.
(ii) V is negative denite at p (abbreviated n.d. at p) if V is p.d. at p.
(iii) V is positive semi-denite at p (abbreviated p.s.d. at p) if V(p) = 0 and V(x) 0 for all x ,= p.
(iv) V is negative semi-denite at p (abbreviated n.s.d. at p) if V is p.s.d. at p.
Theorem 4.6 (Lyapunovs direct method). Let x = 0 be an equilibrium of (4.1), and suppose there exists a C
1
function V : D R, with D A a domain containing 0, such that
(i) V is p.d. at 0,
(ii) L
f
V is n.s.d. at 0.
Then, x = 0 is a stable equilibrium of (4.1). If instead
(ii) L
f
V is n.d. at 0,
then 0 is an asymptotically stable equilibrium of (4.1).
A function V satisfying assumptions (i) and (ii) (or (ii)) of the theorem above is called a Lyapunov
function. The idea of the proof is that L
f
V 0 gives positive invariance of the level sets of V, which
together with the positive deniteness of V is enough to guarantee stability. The stronger condition
L
f
V < 0 in (ii) further implies that all sublevel sets of V are entered by the positive semiorbits of f ,
yielding attractivity.
Proof. (Stability) We need to show that
( > 0)( > 0)(x
0
B
n
(0))(t 0) (t, x
0
) B
n
(0).
The various constructions that follow are illustrated in the gure below. Let > 0 be arbitrary. Without
loss of generality, we can assume that is small enough that B
n
(0) D. Next, let c
= min
|x|=
V(x). By
Theorem 1.24, since V is continuous and the set |x| = is compact, the minimum c
exists. Moreover,
since V is positive denite and the set |x| = does not contain 0, c
> 0. Pick c (0, c
) and dene
c
= x

B
n
(0) : V(x) c.
We claim that
c
B
n
(0) = , so that
c
= x B
n
(0) : V(x) c. If not, then there would exist

p D such that V(p) c and |p| = , in which case we would have that min
|x|=
V(x) c < c
,
contradicting the denition of c
. Using the claim and the fact that, if A, B R

n
, then (A B) =
(A B) (A B), we have that
c
= x B
n
(0) : V(x) = c B
n
(0). (4.2)
D
B
n
(0)
B
n
(0)
c
Next, we show that
c
is positively invariant. By assumption (ii) and the fact that
d
dt
[V((t, x
0
))] = dV
(t,x
0
)
d
dt
[(t, x
0
)] = dV
(t,x
0
)
f ((t, x
0
)) = L
f
V((t, x
0
)) 0,
we have that, for all x
0

c
, t V((t, x
0
)) is nonincreasing, so that for all t T
+
x
0
,
V((t, x
0
)) V((0, x
0
)) = V(x
0
) c.
By (4.2), if x
0

c
the only way that (t, x
0
) can exit
c
is that at some time

t T
+
x
0
, V((
t, x
0
)) > c,
which is ruled out by the inequality above, proving the positive invariance of
c
. Since
c
is bounded
(because
c
B
n
(0)) and positively invariant, by Corollary 2.29 any solution with x

0

c
is dened
for all t 0, or T
+
x
0
= [0, +).
Having established the positive invariance of
c
, we use the continuity of V to show that there exists a
ball B
n
(0) contained in
c
. Use the denition of continuity of V
(c > 0)( > 0)(x B
n
(0)) [V(x) V(0)[ < c

and apply it using the value of c chosen earlier. Since V is p.d. at 0 we have
( > 0)(x B
n
(0)) V(x) < c.

By picking (0, ), we have
x B
n
(0) = x x B
n
(0) : V(x) c =
c
.
4.2 LYAPUNOVS MAIN THEOREM 81
Therefore,
( (0, )) B
n
(0)
c
,
as required. Picking (0, ) satisfying the inclusion above, we have now all ingredients to prove
stability:
x
0
B
n
(0) = x
0

c
= (t 0) (t, x
0
)
c
= (t 0) (t, x
0
) B
n
(0).
(Attractivity) Let , c, and be as above and let x
0
B
n
(0) be arbitrary. The function t V((t, x

0
))
is nonincreasing (this was shown above), nonnegative (because V is positive denite), and continuous
(because it is the composition of two continuous functions). Therefore, by Theorem 1.29 there exists
d [0, c] such that lim
t
V((t, x
0
)) = d. We need to show that d = 0. Assume by contradiction that
d > 0, and let
(0, ) be small enough that B

n
(0)
d
(the existence of
is guaranteed by the same

reasoning used to assert the existence of ). Let
= max
|x|
L
f
V(x).
Since V is C
1
, L
f
V is continuous and, by assumption (ii), it is negative denite. The set x :
|x|
is compact and does not include the origin. Therefore, exists and > 0. For all t 0, we have
V((t, x
0
)) = V(x
0
) +
_
t
0
d
d
[V((, x
0
))]d = V(x
0
) +
_
t
0
L
f
V((, x
0
))d
V(x
0
) t. (4.3)
We deduce that for all t > (V(x
0
) d/2)/, V((t, x
0
)) < d/2 and therefore lim
t
V((t, x
0
)) < d, which
gives a contradiction. We thus have that lim
t
V((t, x
0
)) = 0. In order to show that (t, x
0
) 0, we
need to prove that
(b > 0)(T > 0)(t T) (t, x
0
) B
n
b
(0).
For all b > 0, let a (0, c) be such that a < min
b|x|
V(x). Using the same reasoning that led to (4.2),
we deduce that
a
= x

B
n
(0) : V(x) a B
n
b
(0).
Since V((t, x
0
)) 0, there exists T > 0 such that, for all t T, V((t, x
0
)) < a. Thus, for all t T,
(t, x
0
)
a
B
n
b
(0). This proves that (t, x
0
) 0 and hence the attractivity of the origin.
Exercise 4.1. Where and how, in the proof of stability in Theorem 4.6, is the assumption that V is p.d. used? Through an
explicit counterexample, show that the statement of the theorem is wrong if L
f
V is n.s.d but V is not p.d. at 0.
Exercise 4.2. In the proof of attractivity in Theorem 4.6 it was argued that, for all t > (V(x
0
) d/2)/, V((t, x
0
)) < d/2.
Show that, in the context of that proof, (V(x
0
) d/2)/ is nonnegative.
Exercise 4.3. State precisely what justies inequality (4.3).
Example 4.7. Consider the frictionless pendulum of Example 2.2
x
1
= x
2
x
2
=
Mgl
I
sin x
1
.
We want to show that the equilibrium (x
1
, x
2
) = (0, 0) is stable. Consider the mechanical energy of the
pendulum
V(x
1
, x
2
) = Mgl(1 cos x
1
) +
1
2
Ix
2
2
,
where the constant Mgl has been added to V so that the minimum of the energy is at (x
1
, x
2
) = (0, 0).
Let D = (x
1
, x
2
) R
2
: [x
1
[ < . Then, V : D R is p.d. at (0, 0) and, along pendulum trajectories,
V = 0. Therefore, by Theorem 4.6 the origin is a stable equilibrium. Note that calculating the time
derivative of V along solutions of the system is the same as calculating the Lie derivative of V along the
system vector eld.
Now suppose that the pendulum is affected by viscous friction
x
1
= x
2
x
2
=
Mgl
I
sin x
1
K
I
x
2
.
We expect that the origin is asymptotically stable. The intuition is that friction dissipates energy and
that V(t) 0 implies x(t) 0. The time derivative of the energy is

V = Kx
2
2
, and as expected

V 0,
meaning that energy is always dissipated. However,

V is only n.s.d., not n.d., because any point (x
1
, 0)
on the x
1
axis gives

V = 0. In this case, Theorem 4.6 does not allow us to conclude asymptotic stability.
We will see later that a renement of Lyapunovs direct method, the LaSalle invariance principle, allows
one to conclude asymptotic stability in this situation. For now, in order to prove asymptotic stability of
the origin we need to nd another Lyapunov function.
Exercise 4.4. Consider the pendulum with friction of Example 4.7, and assume that Mgl/I = 1, K = 2. Consider the Lyapunov
function candidate
V(x
1
, x
2
) =
1
2
_
3x
2
1
+ 2x
1
x
2
+ x
2
2
_
.
Find a domain D containing the origin such that V : D R satises assumptions (i) and (ii) of Theorem 4.6, thus proving
that (x
1
, x
2
) = (0, 0) is asymptotically stable.
[Hint: use the fact that [x
1
sin x
1
[ < [x
1
[/2 for all [x
1
[ < /2.]
We conclude this section with another result, which we wont prove, providing a sufcient condition for
instability of an equilibrium.
Theorem 4.8 (Chetaevs instability criterion). Let x = 0 be an equilibrium of (4.1), and suppose there exists
a C
1
function V : D R, with D A a domain containing 0, such that V(0) = 0 and
( > 0)(x B
n
(0)) V(x) > 0.

Let r > 0 be such that B
n
r
(0) D and dene U = x B
n
r
(0) : V(x) > 0. If L
f
V > 0 on U, then x = 0 is
unstable.
Note that the function V in the theorem is not required to be p.d. at 0. It must have the weaker property
that in any neighborhood of 0 there must be points at which V is positive. If, at any such point, L
f
V is
also positive, then the equilibrium is unstable.
4.3 DOMAIN OF ATTRACTION AND ITS ESTIMATION USING LYAPUNOV FUNCTIONS 83
Example 4.9. Consider the Van der Pol oscillator of Example 3.2, with = 1,
x
1
= x
2
x
2
= x
1
+ (1 x
2
1
)x
2
.
The origin is the unique equilibrium point. Let
V =
3
2
x
2
1
x
1
x
2
+ x
2
2
,
which is a p.d. quadratic form. We have
L
f
V =

V = x
2
1
+ x
2
2
+ x
2
1
x
2
(x
1
2x
2
).
Let D = (x
1
, x
2
) : [x
1
2x
2
[ < 1, [x
2
[ < 1.
1
1
1
1
x
1
x
2
D
For all x D, x ,= 0, we have
L
f
V x
2
1
+ x
2
2
x
2
1
[x
2
[[x
1
2x
2
[ > x
2
2
0,
and hence L
f
V : D R is p.d. at 0. Using V : D R in Theorem 4.8, we conclude that the origin is
unstable. One can arrive at the same result more easily by computing the linearization at the origin and
verifying that both eigenvalues have positive real part.
4.3 Domain of attraction and its estimation using Lyapunov functions
Denition 4.10. Let x
A be an equilibrium point of (4.1). The domain of attraction of x
is the set
T
x
= x
0
A : (t, x
0
) x
as t .
We could use the notion of domain of attraction to restate the notion of attractivity: x
is attractive if the
interior of T
x
contains x
. The next result characterizes the topological properties of T

x
.
Theorem 4.11. If x
is an attractive equilibrium of (4.1), then T

x
is open, connected, and invariant.
Proof. (Invariant) If x
0
T
x
, then by denition the maximal interval of existence T
x
0
has the form
T
x
0
= [a, +), and for all t [a, +) we have
(, (t, x
0
)) = (t + , x
0
) x
as ,
Therefore, for all t T
x
0
, (t, x
0
) T
x
.
(Connected) This part is left for an exercise, see below.
(Open) The construction that follows is illustrated in the gure below. Since x
is attractive, there exists

> 0 such that, for all x
0
B
n
(x
), (t, x
0
) x
as t . Therefore, B
n
(x
) T
x
. Pick an arbitrary
x
0
T
x
. Since lim
t
(t, x
0
) = x
, there exists T > 0 such that (t, x

0
) B
n
/2
(x
) for all t T, i.e., the

solution reaches the ball B
n
/2
(x
) in at most time T.
T
x
x
0
y
0
B
n
(x
)
B
n
/2
(x
)
after time T
B
n
(x
0
)
Now we use continuity with respect to initial conditions (Theorem 2.31) to assert the existence of > 0
such that, for all y
0
B
n
(x
0
) and all t [0, T], |(t, y
0
) (t, x
0
)| < /2. In particular, (T, y
0
)
B
n
/2
((T, x
0
)) B
n
(x
) T
x
, so that (T, y
0
) T
x
. By invariance of T
x
, y
0
T
x
as well. We have
shown that, for all x
0
T
x
there exists > 0 such that B
n
(x
0
) T
x
. Therefore, T
x
is open.
Exercise 4.5. Prove that the domain of attraction of an attractive equilibrium is connected.
[Hint: Referring to Denition 1.16(x), argue by contradiction. One way to do it is to show that every connected component of
A must contain a neighborhood of the equilibrium, contradicting the fact that the connected components of A are separated
subsets. There are other ways.]
In practice, it is very difcult to analytically determine the domain of attraction of an equilibrium point.
Yet, the problem of determining T
x
is a rather important one in practice. Suppose, for instance, that
one has designed a controller making x
asymptotically stable. The question then arises of what is a safe

domain of operation in which solutions remain bounded and converge to x
. Lyapunov theory provides

a useful and straightforward way to answer this question by nding an estimate of T
x
as follows.
Procedure to nd an estimate of T
x
1. Find a C
1
function V : D R, where D is a domain containing x
, such that V is p.d. at x
and
L
f
V is n.d. at x
.
2. If D = R
n
, set
1
c
= . Otherwise, let c
= inf
xD
V(x).
3. For any c (0, c
), if the set
c
:= x D : V(x) c is compact, then
c
T
x
.
1
Note that D = R
n
implies A = R
n
. In this case, D = .
4.3 DOMAIN OF ATTRACTION AND ITS ESTIMATION USING LYAPUNOV FUNCTIONS 85
Proposition 4.12. Let V : D R be a C
1
function, where D A is a domain of R
n
containing an equilibrium
x
. Suppose V is p.d. at x
and L
f
V is n.d. at x
. Let c
be dened as in the procedure above. For any c (0, c
),
if
c
= x D : V(x) c is compact set, then
c
T
x
.
Proof. We follow the proof of attractivity in Theorem 4.6. Let c (0, c
) be such that
c
= x D :
V(x) c is a compact set. We claim that
c
= x D : V(x) = c. Indeed,
c
= (D x
D : V(x) c) (x D : V(x) = c), and the set D x D : V(x) c is empty, because
on D we have V(x) c
> c. For any x

0
D, t V((t, x
0
)) is nonincreasing because L
f
V 0.
Thus, orbits originating in
c
cannot cross its boundary
c
= x D : V(x) = c, and therefore
c
is positively invariant. Now let x
0

c
be arbitrary. Since
c
is compact and positively invariant,
T
+
x
0
= R
+
. Moreover, there exists d 0 such that lim
t
V((t, x
0
)) = d. If d > 0, then we can nd
> 0 such that B
n
(0)
d
. Since the set x
d
: |x| is compact, and since on it L
f
V < 0,
there exists > 0 such that = max
x
d
:|x|
L
f
V. For all t 0 such that (t, x
0
) , B
n
(0), we have
V((t, x
0
)) V(x
0
) t, contradicting the fact that V((t, x
0
)) d > 0. Therefore, V((t, x
0
)) 0.
For all b > 0, let a (0, c) be such that a < min
x
c
:|x|b
V(x) (the minimum exists and is positive
because V is continuous and p.d.). Then, we claim that
a
B
n
b
(0). If not, then there exists p
a
with p , B
n
b
(0), i.e., V(p) a < c and |p| > b. However, V(p) < c and |p| b implies V(x) > a,
a contradiction. Next, since V((t, x
0
)) 0 there exists T > 0 such that for all t T, V((t, x
0
)) a,
implying (t, x
0
)
a
, implying (t, x
0
) B
n
b
(0). We have shown that
(b > 0)(T > 0)(t T) (t, x
0
) B
n
b
(0),
proving that for all x
0

c
, (t, x
0
) 0.
Remark 4.13. Finding a Lyapunov function is not an easy task, but we will see in Section 4.7 that if the
linearization of the vector eld f at x
has all eigenvalues with negative real part, then one can generate
a quadratic Lyapunov function for the nonlinear system, and use it to estimate T
x
. If the level sets of
V are not all compact, one can show that there exists c (0, c
) such that for all c (0, c) the level set

x A : V(x) = c is compact.
Remark 4.14. Geometrically, the set
:= x : V(x) < c
is the largest sublevel set of V contained in D.

To illustrate, let
0 < c
1
< c
< c
2
. We claim that the set
c
1
:= x

D :
V(x) c
1
is contained in the interior of D, while the set
c
2
= x

D : V(x) c
2
is not. The situation is illustrated
on the right-hand side. To prove the claim above, suppose
that there exists x
c
1
such that x D. Then, V(x)
c
1
< c
= inf
xD
V(x), which gives a contradiction. Next,
since inf
xD
V(x) = c
, by Denition 1.30 there exists x

D such that c
< V(x) < c

2
. Thus, x
c
2
D. This
shows that
c
2
is not entirely contained in the interior of D.
x
c
1
c
2
c
1
< c
< c
2
Example 4.15. Consider the planar system
x
1
= x
1
(1 [x
1
[ [x
2
[) x
2
x
2
= x
2
(1 [x
1
[ [x
2
[) + x
1
.
The origin is asymptotically stable. For, using V = 1/2(x
2
1
+ x
2
2
), we have
L
f
V = (x
2
1
+ x
2
2
)(1 [x
1
[ [x
2
[).
L
f
V is n.d. at 0 as long as x belongs to the domain D = x : [x
1
[ +[x
2
[ < 1. We have
c
= inf [x
1
[ +[x
2
[ = 1
_
1
2
(x
2
1
+ x
2
2
)
_
=
1
4
.
The estimate of T
x
is
= x : V(x) < 1/4, the open disk of radius

2/2.
T
x
x
1
x
2
D
1
The gure illustrates the actual T
x
(the region enclosed by the thick phase curve) and its estimate
.
Note that D is not entirely contained in T
x
, so it would be wrong to say that the region D where L
f
V
is n.d. is an estimate of the domain of attraction.
4.4 Global asymptotic stability
In this section we present conditions for global asymptotic stability, assuming throughout that A =
R
n
. In Section 4.3 we presented a procedure to estimate the domain of attraction of an asymptotically
stable equilibrium by means of compact sublevel sets of a Lyapunov function. Without compactness,
the estimate may be erroneous because solutions may grow unbounded while crossing smaller and
smaller level sets of the Lyapunov function. This observation suggests that even if the assumptions
of Theorem 4.6 hold on all of the state space (i.e., D = R
n
), the equilibrium may not be globally
asymptotically stable, as shown in the next example.
x
1
=
6x
1
(1 + x
2
1
)
2
+ 2x
2
x
2
=
2(x
1
+ x
2
)
(1 + x
2
1
)
2
4.4 GLOBAL ASYMPTOTIC STABILITY 87
and the function V = x
2
1
/(1 + x
2
1
) + x
2
2
, which is p.d. at the origin. We have
L
f
V = 4
3 x
1
2
+ x
2
2
+ 2 x
2
2
x
1
2
+ x
2
2
x
1
4
(1 + x
1
2
)
4
,
and so L
f
V is n.d. at the origin. Despite the fact that here D = R
2
, the origin is not globally asymptoti-
cally stable. The problem is illustrated below.
8 6 4 2 2 4 6 8
4
3
2
1
0
0
1
2
3
4
H
The level sets of V are not all compact. In fact, it is not difcult to see that for all c 1,
c
is unbounded.
For initial conditions not close to the origin, the solution escapes to innity while at the same time
entering the level sets of V. This fact can be proved rigorously using the set
H = (x
1
, x
2
) : (x
1
2)x
2
> 2, x
1
>
2,
which is shaded in the gure above.
Exercise 4.6. Letting h(x
1
, x
2
) = (x
1
_
(2))x
2
2, verify that
(h(x) = 0, x
1
>
2) = L
f
h(x) > 0.
By Nagumos theorem (see Theorem 3.17), the exercise above implies that H is positively invariant. Since
H does not contain the origin, the origin cannot be globally asymptotically stable.
The example above demonstrates that in order to assert global asymptotic stability, the assumptions of
Theorem 4.6 need to be strenghtened by assuming not only that D = R
n
, but also that all level sets of V
are compact. This condition translates into the notion of radial unboundedness.
Denition 4.17. A function V : R
n
R is radially unbounded if lim
|x|
V(x) = .
If V is p.d. and radially unbounded, then all its level sets are compact. In order to see this fact, use the
denition of limit to state that lim
|x|
V(x) = ,
(c > 0)(K > 0) |x| > K = V(x) > c,
or, equivalently,
(c > 0)(K > 0) V(x) c = |x| K.
Hence, we have established that for all c > 0 there exists K > 0 such that the sublevel set V(x) c is
contained in the ball B
n
K
(0), proving that the sublevel set in question is bounded, and therefore compact.
Theorem 4.18 (Barbashin-Krasovskii). Let x = 0 be an equilibrium of (4.1) with A = R
n
, and suppose there
exists a C
1
function V : R
n
R function s.t. V is p.d. at 0, radially unbounded, and L
f
V is n.d. at 0. Then,
x = 0 is globally asymptotically stable.
Proof. By Theorem 4.6, x = 0 is asymptotically stable. By radial unboundedness, for any c > 0,
c
=
x R
n
: V(x) c is compact, and therefore, by Proposition 4.12,
c
T
0
. Since c > 0 is arbitrary,
T
0
= R
n
.
4.5 LaSalles Invariance Principle
The LaSalle invariance principle is an improvement of Lyapunovs direct method. It deals with the case
when the derivative of the Lyapunov function along the system vector eld is only nonpositive, and
allows one to conclude more than just stability of the equilibrium. The principle follows directly from
Birkhoffs theorem on limit sets presented in Section 3.1.
Theorem 4.19 (LaSalles Invariance Principle). Let D A be a domain and D be a compact positively
invariant set. Let V : D R be a C
1
function such that for all x , L
f
V 0. Let E = x : L
f
V = 0
and let M be the largest
2
invariant subset of E. Then, for all x
0
, (t, x
0
) M as t .
This theorem was published by LaSalle in the IRE Transactions on Circuit Theory in 1960. A similar
result, stated in a rather different way, was published by Nikolai Nikolaevich Krasovskii in 1959 in
a Soviet journal, and for this reason the theorem is sometimes referred to as the Krasovskii-LaSalle
invariance principle.
Note that the theorem does not require V to be p.d., but it does assume the existence of a compact
positively invariant set . Often, is generated as the sublevel set of a p.d. Lyapunov function.
Proof. This proof has four parts.
(Part 1) Let x
0
be arbitrary. We have T
+
x
0
= R
+
because is a compact and positively invariant.
We claim that there exists a R such that lim
t
V((t, x
0
)) = a. (1) Since L
f
V 0 on , the function
t V((t, x
0
)) is nonincreasing. (2) Since V is continuous and is compact, by Theorem 1.24 V has a
minimum in . By Theorem 1.29, facts (1) and (2) prove the claim.
(Part 2) We claim that L
+
(x
0
) ,= and (x L
+
(x
0
)) V(x) = a. Since the positive semiorbit through
x
0
is a bounded set (because it is contained in the compact set ), by Theorem 3.8 L
+
(x
0
) is nonempty,
compact, and invariant. By denition, for all x L
+
(x
0
) there exists a sequence t
i
, with t
i
, such
that (t
i
, x
0
) x. By continuity of V, V((t
i
, x
0
)) V(x). At the same time, V((t
i
, x
0
)) a. Since
the limit is unique (Theorem 1.28), we have V(x) = a, proving the claim.
2
By largest it is meant that if N is any invariant subset of E, then N M.
4.5 LASALLES INVARIANCE PRINCIPLE 89
(Part 3) Now we show that L
+
(x
0
) E = x : L
f
V(x) = 0. Since L
+
(x
0
) is invariant, for all
p L
+
(x
0
) and all t R
+
, (t, p) L
+
(x
0
). Using the result in part 2, we have (t 0) V((t, p)) = a,
which implies
dV((t, p))
dt
0 L
f
V((t, p)) 0 = L
f
V(p) = 0.
Moreover,
x
0
=
()
L
+
(x
0
) = p .
Therefore, for all p L
+
(x
0
) we have p and L
f
V(p) = 0, as required.
(Part 4) Since L
+
(x
0
) is invariant and contained in E, necessarily L
+
(x
0
) M. By Theorem 3.8, (t, x
0
)
L
+
(x
0
) as t , and therefore (t, x
0
) M.
Exercise 4.7. Prove implication () in the proof above.
Example 4.20. We return to the pendulum affected by viscous friction of Example 4.7 with the objec-
tive of proving asymptotic stability of the equilibrium (0, 0), corresponding to the pendulum pointing
downward, using the energy. Recall that the total energy of the pendulum is
V(x
1
, x
2
) = Mgl(1 cos x
1
) +
1
2
Ix
2
2
,
and

V = Kx
2
2
. As in Example 4.7, let D = (x
1
, x
2
) R
2
: [x
1
[ < . We have
inf
xD
V(x) = 2 Mgl =: c
.
Exercise 4.8. Show that for all c (0, c
), the sublevel set

c
= x D : V(x) c is compact.
For an arbitrary c (0, c
),
c
is compact, positively invariant, and contained in D. Following Theo-
rem 4.19, we have E = x
c
: x
2
= 0 and we need to nd the largest invariant subset of E. If x
0
E,
we look for conditions under which (t, x
0
) remains in E at all time. Denoting (x
1
(t), x
2
(t)) = (t, x
0
),
we need x
2
(t) 0, and therefore also x
2
(t) 0, or
Mgl
I
sin x
1
(t)
K
I
x
2
(t) =
Mgl
I
sin x
1
(t) 0.
Therefore, we need sin x
1
(t) 0. Since
c
D, and since D does not include the lines x
1
= , we
conclude that the only way that (t, x
0
) can remain in E is that x
1
(t) = x
2
(t) = 0 for all t R, which
can only happen if x
0
= (0, 0). This reasoning shows that M = (0, 0). By the LaSalle invariance
principle the positive semiorbit through any point in
c
approaches the origin, proving that the origin
is attractive (and hence asymptotically stable), with
c
contained in its domain of attraction. Since
c (0, c
) is arbitrary, the set
= x D : V(x) < 2Mgl is an estimate of T

(0,0)
. The gure below
was generated with all pendulum parameters set at 1.
6 4 2 0 2 4 6
4
3
2
1
0
1
2
3
4
x
1
x
2
D
The example above illustrates a typical application of the LaSalle invariance principle, whereby V is a
Lyapunov function which is used to produce the compact positively invariant set . This special case,
where the objective is to prove asymptotic stability of an equilibrium using a Lyapunov function whose
Lie derivative is only n.s.d., is the subject of the next Corollary.
Corollary 4.21. Let x = 0 be an equilibrium of (4.1), and suppose there exists a C
1
function V : D R, with
D A a domain containing 0, such that V is p.d. at 0, L
f
V is n.s.d. at 0, and the largest invariant subset of
x D : L
f
V = 0 is x = 0. Then, x = 0 is asymptotically stable. Moreover, if A = R
n
, D = R
n
, and V is
radially unbounded, then x = 0 is globally asymptotically stable.
Example 4.22. Consider the scalar control system
x = ax + u,
where a is an unknown real number. We want to design a feedback controller that stabilizes x = 0
without knowing a. If a were known, a stabilizing feedback would be given by
u(x
1
, x
2
) = ax kx, k > 0.
In this case, V = 1/2x
2
would be a Lyapunov function for the closed-loop system. Since a is not known,
we set
u(x
1
, x
2
) = ax kx, k > 0,
where
a = (x)
and (x) is to be determined in such a way that convergence of x to 0 is guaranteed. The vector eld of
the closed-loop system is
f (x, a) =
_
(a a)x kx
(x)
_
.
4.5 LASALLES INVARIANCE PRINCIPLE 91
We think of the a-subsystem as an estimator that utilizes x to estimate a. The resulting controller, which
we call adaptive, will be dynamic because it has a state.
Consider the Lyapunov function candidate
W(x, a) =
1
2
x
2
+
1
2
( a a)
2
, > 0,
which is p.d. at (0, a) and radially unbounded (if x
2
+ a
2
, then W(x, a) ). It is the sum of
the Lyapunov function of the ideal closed-loop system (i.e., the closed-loop system we would have if
we knew a) with a term that weighs the difference between the actual value of a and its estimate. The
constant is a design parameter. We have
L

f
W = (a a)x
2
kx
2
+
1
( a a)(x).
At this point, notice that by setting (x) = x
2
we cancel the term (a a)x
2
and obtain L

f
W = kx
2
.
It is important that we are able to do this with a function that does not depend on the unknown
parameter a. The Lie derivative L

f
W is only n.s.d. because it does not depend on the state a. Since
W is radially unbounded and L

f
W is n.s.d., all level sublevel sets of W are compact and positively
invariant. Let c > 0 be arbitrary and let
c
= (x, a) : W(x, a) c. Following Theorem 4.19, we have
E = (x, a)
c
: x = 0. In order to nd the largest invariant subset M of E, note that every point of
E is an equilibrium of the closed-loop system. Therefore, the entire set E is invariant for the closed-loop
system, and M = E. We have obtained that for any initial condition (x
0
, a
0
)
c
the corresponding
solution (x(t), a(t)) is such that x(t) 0 as t . Since c is arbitrary, we conclude that the convergence
property holds for all initial conditions in R
2
. While the equilibrium (x, a) = (0, a) is stable, this analysis
does not conclude that it is attractive, because it only guarantees that x(t) 0. As a matter of fact, the
closed-loop system
x = (a a)x kx
a = x
2
has a continuum of equilibria on the entire a axis and it is possible to show that any equilibrium ( a, 0)
with a a k is stable (in particular, this holds true for a = a, as we have already seen), while all other
equilibria are unstable. The special case when a = k = = 1 is illustrated below.
2 1.5 1 0.5 0 0.5 1 1.5 2
2
1.5
1
0.5
0
0.5
1
1.5
2
a = a
a
x
M
Exercise 4.9. Consider the system
x
1
= x
2
+ x
1
(1 x
2
1
x
2
2
)
x
2
= x
1
+ x
2
(1 x
2
1
x
2
2
).
Using LaSalles invariance principle, prove that the unit circle is a stable limit cycle of the system. Additionally, prove that the
positive semiorbit through any nonzero initial condition asymptotically approaches the unit circle.
4.6 Stability of LTI systems
In this section we turn our attention to the special case of linear time-invariant (LTI) systems
x = Ax (4.4)
with state space A = R
n
. If det A ,= 0, (4.4) has a unique equilibrium at x = 0. We recall the following
stability result.
Theorem 4.23. The equilibrium x = 0 is stable for (4.4) if and only if
(i) all eigenvalues of A have real part 0, and
(ii) every eigenvalue
i
of A with zero real part has m
i
associated linearly independent eigenvectors, where m
i
is
the algebraic multiplicity
3
of
i
.
The equilibrium is asymptotically stable if and only A is Hurwitz, i.e., if all eigenvalues of A have real part < 0.
The question we investigate in this section is whether asymptotic stability of an LTI system can be tied to
the existence of a special Lyapunov function. Let us consider a quadratic Lyapunov function candidate,
V = x
Px,
where P is a symmetric p.d. matrix. Then,
L
Ax
V(x) =
Exercise 1.25
2x
PAx = x
PAx + x
Px = x
(PA + A
P)
. .
Q
x.
We have thus obtained that the Lie derivative of a quadratic function along an LTI vector eld is itself a
quadratic function. If, given a symmetric p.d. matrix P, the symmetric matrix Q = (PA + A
P) turns
out to be positive denite, then L
Ax
V is n.d. and V is a Lyapunov function for (4.4). This observation is
not that useful though, because it isnt clear how to choose P in order to get a p.d. Q. Turn the problem
around and ask whether, given a symmetric p.d. matrix Q, does there exist a symmetric p.d. matrix P
solving the Lyapunov equation
PA + A
P = Q. (4.5)
The answer is yes for any Q, provided that A is Hurwitz.
Theorem 4.24. The matrix A is Hurwitz if and only if for any symmetric p.d. matrix Q there exists a symmetric
p.d. matrix P satisfying Lyapunovs equation (4.5). Furthermore, if A is Hurwitz then P is the unique solution
of (4.5).
3
The algebraic multiplicity of an eigenvalue
i
is the number of roots of the polynomial det(I A) = 0 at =
i
.
4.6 STABILITY OF LTI SYSTEMS 93
Proof. Sufciency follows directly from Lyapunovs Theorem 4.6 and Theorem 4.23. Concerning the
necessity, assuming that A is Hurwitz and pick a positive denite Q. We need to prove the existence of
a symmetric p.d. P solving (4.5). Let x
0
R
n
be arbitrary and denote x(t) = e
At
x
0
. We need to nd a
matrix P such that d/dt[x(t)
Px(t)] = x(t)
Qx(t), or
d
dt
_
x
0
e
A
t
Pe
At
x
0
_
= x
0
e
A
t
Qe
At
x
0
. (4.6)
Integrate both sides from 0 to +
x
0
e
A
t
Pe
At
x
0
0
=
_

0
x
0
e
A
Qe
A
x
0
d = x
0
_
_

0
e
A
Qe
A
d
_
x
0
.
Since A is Hurwitz, e
At
x
0
0 exponentially as t , and so we obtain
x
0
Px
0
= x
0
_
_

0
e
A
Qe
A
d
_
x
0
.
Since we want this identity to be satised for arbitrary initial conditions x
0
, it must be that (see Exer-
cise 1.12)
P =
_

0
e
A
Qe
A
d.
The matrix P above is well-dened because the integrand can be written as the sum of terms of the form
t
k
e
i
t
e
j
t
which tend to zero exponentially. By construction, P solves Lyapunovs equation and it is its
unique solution because any other solution must satisfy (4.6) for all x
0
R
n
, and therefore it must also
satisfy all the implications that led us to the denition of P. Clearly P
= P, so we are left to show that

P is positive denite. For any x R
n
, x
Px =
_
0
x
e
A
Qe
A
xd. The integrand is nonnegative for
all 0 because Q is p.d., and so the integral is nonnegative as well. Suppose that there exists x R
n
such that x
Px = 0, then
_
0
x
e
A
Qe
A
xd = 0. The integral of a nonnegative function can be zero
only if the function is zero, so we have
x
e
A
Qe
A
x 0 = e
A
x 0 = x = 0,
proving that P is positive denite.
Lyapunovs equation can always be transformed into a linear system of equations.
Example 4.25. In Example 4.4 we gave without justication a Lyapunov function for the mass-spring
system with friction
x
1
= x
2
x
2
= x
1
2x
2
,
which is an asymptotically stable LTI system. Here, we derive V. Letting Q = I
2
, we need to solve the
equation
_
p
1
p
2
p
2
p
3
_ _
0 1
1 2
_
+
_
0 1
1 2
_ _
p
1
p
2
p
2
p
3
_
=
_
1 0
0 1
_
,
or, componentwise,
p
2
p
2
= 1
p
1
2p
2
p
3
= 0
p
2
2p
3
+ p
2
2p
3
= 1.
The system of equations above can be rewritten as
_
_
0 2 0
1 2 1
0 2 4
_
_
_
_
p
1
p
2
p
3
_
_ =
_
_
1
0
1
_
_.
The matrix above is nonsingular (it must be, by Theorem 4.24), and the unique solution of the linear
system is
_
_
p
1
p
2
p
3
_
_ =
_
_
3/2
1/2
1/2
_
_,
which gives the same Lyapunov function as in Example 4.4
V(x
1
, x
2
) =
1
2
_
3x
2
1
+ 2x
1
x
2
+ x
2
2
_
.
4.7 Linearization and exponential stability
In the previous section we have seen that there is a systematic way to generate Lyapunov functions
for asymptotically stable linear systems. In this section, we establish a relationship between the stability
properties of the linearization and those of the nonlinear system. Throughout the section we assume that
the vector eld f is C
1
. We begin by characterizing the property of exponential stability via Lyapunov
functions.
Theorem 4.26. Let x = 0 be an equilibrium of (4.1), and suppose there exists a C
1
function V : D R, with
D A a domain containing 0, such that, for some positive real numbers c
1
, c
2
, c
3
, and some k N,
(i) c
1
|x|
k
V(x) c
2
|x|
k
for all x D
(ii) L
f
V(x) c
3
|x|
k
for all x D.
Then, x = 0 is exponentially stable. If D = A = R
n
, then x = 0 is globally exponentially stable.
Proof. Let r > 0 be such that B
n
r
(0) D, c > 0 be such that
c
= x

B
n
r
(0) : V(x) c B
n
r
(0),
and > 0 be such that B
n
(0)
c
. The existence of c and was proved in Theorem 4.6. Moreover,
in there we also proved that
c
is positively invariant. By assumptions (i) and (ii), on D we have
V(x) c
2
|x|
k
and L
f
V(x) c
3
/c
2
V(x). By the positive invariance of
c
and the fact that
c
D we
have
(x
0

c
)(t 0)
dV((t, x
0
))
dt

c
3
c
2
V((t, x
0
)),
integrating both sides from 0 to t we have
(x
0

c
)(t 0) V((t, x
0
)) V(x
0
)
c
3
c
2
_
t
0
V((, x
0
))d.
4.7 LINEARIZATION AND EXPONENTIAL STABILITY 95
By Gronwalls inequality in Lemma 2.30 we have
(x
0

c
)(t 0) V((t, x
0
)) V(x
0
)e
(c
3
/c
2
)t
.
By assumption (i), the above inequality implies
(x
0

c
)(t 0) c
1
|(t, x
0
)|[
k
c
2
|x
0
|
k
e
(c
3
/c
2
)t
,
so that |(t, x
0
)|
_
c
2
c
1
_
1/k
|x
0
|e
c
3
c
2
k
t
. In conclusion,
x
0
B
n
(0) = x
0

c
D = (t 0)|(t, x
0
)|
_
c
2
c
1
_
1/k
|x
0
|e
c
3
c
2
k
t
,
proving that x
0
is exponentially stable. If D = A = R
n
, then c, and hence , can be chosen arbitrarily
large and therefore the result becomes global.
In Theorem 4.24 we have seen that any asymptotically stable linear system admits a quadratic p.d.
Lyapunov function V(x) = x
Px with quadratic n.d. Lie derivative L

f
V = x
Qx. Recalling the

inequalities in (1.3), we have
min
(P)|x|
2
2
V(x)
max
(P)|x|
2
2
and
L
f
V(x)
min
(Q)|x|
2
2
,
so by Theorem 4.26 an asymptotically stable LTI system is also globally exponentially stable. While
this conclusion is not surprising, it sets the stage for the next result linking exponential stability of an
equilibrium for a nonlinear system with that of its linearization.
Theorem 4.27 (Lyapunovs indirect method). Let the vector eld f in (4.1) be C
1
and such that, for some
r > 0, the function x d f
x
is Lipschitz on B
n
r
(0). Then, x = 0 is an exponentially stable equilibrium of the
nonlinear system (4.1) if and only if it exponentially stable for the linearization z = d f
0
z.
We will only prove the if part of the statement, which is due to Lyapunov. The only if part requires
a converse Lyapunov theorem which will be presented in the next section.
Proof of sufciency. Suppose that the linearization z = d f
0
z is exponentially stable. Then, by Theo-
rem 4.24, for any symmetric p.d. Q there exists a symmetric p.d. matrix P such that V(z) = z
Pz is
a Lyapunov function for the linearization, with Lie derivative L
d f
0
z
V(z) = z
Qz. We use the same

function as a Lyapunov function candidate for the nonlinear system, V(x) = x
Px. The Lie derivative

along the nonlinear vector eld reads as
L
f
V(x) = 2x
Pf (x) = 2x
P[d f
0
x + ( f (x) d f
0
x)]
= x
(Pd f
0
+ d f
0
P)x + 2x
P( f (x) d f
0
x)
= x
Qx + 2x
P( f (x) d f
0
x). (4.7)
Now we will bound the second term of the sum. First, using the Cauchy-Schwarz inequality, we get
[2x
P( f (x) d f
0
x)[ 2|x|
2
|P( f (x) d f
0
x)|
2
.
Using the property of induced norms in Exercise 1.7,
[2x
P( f (x) d f
0
x)[ 2|x|
2
|P|
2
|f (x) d f
0
x|
2
.
Now we apply the Mean Value Theorem 1.45 to each component f
i
of f . For all x B
n
r
(0) and all
i 1, . . . , n, there exists z
i
in the segment connecting x to 0 (which is contained in B
n
r
(0)) such that
f
i
(x) f
i
(0) = (d f
i
)
z
i
x. Since 0 is an equilibrium, we have f
i
(x) = (d f
i
)
z
i
x and so
[ f
i
(x) (d f
i
)
0
x[ =
[(d f
i
)
z
i
(d f
i
)
0
]x
|(d f
i
)
z
i
(d f
i
)
0
|
2
|x|
2
.
Since x d f
x
is Lipschitz on B
n
r
(0), so are the maps x (d f
i
)
x
, and there exists L > 0 such that for all
i n,
|(d f
i
)
z
i
(d f
i
)
0
|
2
L|z
i
|
2
L|x|
2
.
The second inequality follows from the fact that z
i
lies on the segment connecting 0 to x, and thus its
distance to the origin is less than or equal to that of x. We have
|f (x) d f
0
x|
2
=
_
i
[ f
i
(x) (d f
i
)
0
x[
2
L
n|x|
2
2
,
and so
[2x
P( f (x) d f
0
x)[ 2L
n|P|
2
|x|
3
2
,
which we use in (4.7) to get
L
f
V(x)
min
(Q)|x|
2
2
+ 2L
n|P|
2
|x|
3
2
|x|
2
2
_
min
(Q) 2L
n|P|
2
|x|
2
_
.
This inequality holds for all x B
n
r
(0). If we let = min
_
r,

min
(Q)
4L
n|P|
2
_
, then for all x B
n
(0) we have
L
f
V(x)
1
2
min
(Q)|x|
2
2
,
which, together with the fact that
min
(P)|x|
2
2
V(x)
max
(P)|x|
2
2
, proves the exponential stability
of x = 0.
Theorem 4.27 is useful because it states that checking exponential stability of an equilibrium of a non-
linear system amounts to check stability of the systems linearization at the equilibrium, which is a
straightforward matter.
Example 4.28. The origin of the scalar differential equation
x = x
3
is globally asymptotically stable (use, for instance, V(x) = x
2
/2), but not exponentially stable because
the linearization at the origin is z = 0, which is only stable, not asymptotically stable.
4.7 LINEARIZATION AND EXPONENTIAL STABILITY 97
A byproduct of the proof of Theorem 4.27 is a methodology to nd Lyapunov functions and estimate the
domain of attraction of exponentially stable equilibria. The following procedure is an adaptation of the
method presented in Section 4.3. It is important to bear in mind that this procedure generally produces
rather conservative estimates of the domain of attraction, due to the approximation in the linearization.
Procedure to nd an estimate of T
x
using the linearization
1. Suppose the Jacobian d f
x
is Hurwitz. Pick a symmetric p.d. matrix Q (for instance, Q = I
2
) and
solve Lyapunovs equation to get a symmetric p.d. P.
2. Let D be the connected component of the set x A : (x x
Q(x x
) +2(x x
P[ f (x)
d f
x
(x x
)] 0 containing x
. If D = R
n
, then set c
= . Otherwise, let c
= inf
xD
(x
x
P(x x
).
3. An estimate of T
x
is the set
= x A : (x x
P(x x
) < c
.
The procedure above is rather practical. It involves solving, in step 2, a constrained minimization prob-
lem, which can be done numerically.
Example 4.29. We return to the pendulum with friction of Example 4.20, but this time we want to
estimate the domain of attraction of the equilibrium (0, 0) without using the energy. For convenience,
set all pendulum parameters to 1. The linearization at x
= (0, 0) has system matrix

d f
(0,0)
=
_
0 1
1 1
_
.
With Q = I
2
, the solution to Lyapunovs equation is
P =
_
3/2 1/2
1/2 1
_
.
The set D is given by
D =
_
x R
2
: (x
2
1
+ x
2
2
) + 2
_
x
1
x
2
_
P
_
0
sin x
1
+ x
1
_
0
_
= x R
2
: (x
2
1
+ x
2
2
) + (x
1
+ 2x
2
)(sin x
1
+ x
1
) 0.
To nd c
, we need to solve the constrained minimization problem

minimize
3
2
x
2
1
+ x
1
x
2
+ x
2
2
subject to (x
2
1
+ x
2
2
) + (x
1
+ 2x
2
)(sin x
1
+ x
1
) = 0, x ,= 0.
One could use the Lagrange multiplier method to calculate the minimum analytically. Instead, here we
use Matlabs fmincon function to get c
= 10.0957. The results are displayed below.

6 4 2 0 2 4 6
4
3
2
1
0
1
2
3
4
x
1
x
2
D
It is interesting to compare
to the estimate displayed in Example 4.20. The union of the two sets
found in the two examples is a better estimate of the actual domain of attraction. This example illustrates
that by estimating T
x
in different ways, one can incrementally improve the approximation of T
x
. Here,
one could further improve the estimate by choosing different matrices Q and repeating the procedure.
Exercise 4.10. Consider the following class of nonlinear systems
x = Ax + f (x)
y = Cx
(4.8)
where A R
nn
, f : R
n
R
n
is globally Lipschitz, and C R
1n
. Assume that the state of the system is unknown,
while y(t) can be measured. An observer for (4.8) is a dynamical system with state x R
n
and input y such that, (x
0
, x
0
)
R
n
R
n
, lim
t
((t, x
0
)

(t, x
0
)) = 0. Hence, an observer is a device which estimates the state of system (4.8) by only using
the information given by y.
Assume that there exists L R
n1
such that A LC is Hurwitz. Find conditions on the Lipschitz constant of f such that
x = A x + f ( x) + L(y C x) is an observer for system (4.8).

[Hint: consider the estimation error x = x x and write the error dynamics. Find a quadratic Lyapunov function for the linear
part and use it for the nonlinear error dynamics. Then use the fact that f is globally Lipschitz to nd an upper bound for the
Lie derivative of V. ]
Exercise 4.11. Consider the dynamical system x = Ax + g(x), where A R
nn
is a Hurwitz matrix and g : R
n
R
n
is locally
Lipschitz on R
n
and such that ( > 0), (x R
n
) |g(x)|
2
|x|
2
2
.
1. Prove that the origin is an exponentially stable equilibrium;
2. Find an estimate of its domain of attraction;
3. Let now n = 2 and
A =
_
0 1
1 2
_
, g(x) =
_
x
2
1
+ x
2
2
0
_
.
Find the value of for this example and, using the Matlab command streamslice, generate a phase portrait, high-
lighting the domain of attraction. Compare your estimate of the domain of attraction to the simulation results: how
conservative is your estimate?
Exercise 4.12. Consider the model of a controlled pendulum with unknown mass
x
1
= x
2
x
2
= sin x
1
+
1
I
u.
4.8 CONVERSE STABILITY THEOREMS 99
The parameter is unknown. We want to design an adaptive controller that stabilizes the inverted conguration x = (, 0).
Note that, if were known, the control law u = I (c
1
(x
1
) c
2
x
2
+ sin x
1
), c
1
, c
2
> 0, would globally exponentially sta-
bilize the equilibrium (, 0). Consider the adaptive control law u(x
1
, x
2
,

) = I
_
c
1
(x
1
) c
2
x
2
+

sin x
1
_
,

= (x
1
, x
2
),
where (x
1
, x
2
) is to be designed.
1. Using a function of the form V = x
P x +
1
2
(
)
2
, where x = ( x
1
, x
2
) = (x
1
, x
2
) and where P is a matrix you
need to choose wisely, nd (x
1
, x
2
) such that
(a) All trajectories (x
1
(t), x
2
(t), (t)) of the closed-loop system are bounded.
(b) The straight line L = (x
1
, x
2
,

) R
3
: x
1
= , x
2
= 0 is a set of equilibria for the closed-loop system.
(c) All trajectories of the closed-loop system approach L as t .
2. Choose a numerical value for , and simulate the closed-loop system with your adaptive controller. Plot (x
1
(t), x
2
(t))
on the x
1
-x
2
plane for several initial conditions to verify that (x
1
(t), x
2
(t)) (, 0). Does

(t) converge to ?
Exercise 4.13. Consider again the controlled pendulum model of the previous exercise, but now suppose that both its mass
and its inertia are unknown. We write the model as follows
x
1
= x
2
x
2
=
1
sin x
1
+
2
u,
where
1
and
2
are unknown. Although
2
is unknown, we know that its sign is positive. If
1
and
2
were known, the con-
troller u(x
1
, x
2
) =

1
2
sin x
1
2
(c
1
(x
1
) +c
2
x
2
), c
1
, c
2
> 0, would make the equilibrium (, 0) globally exponentially stable.
Let a
1
=

1
2
and a
2
=
1
2
and consider the adaptive control law u(x
1
, x
2
, a
1
, a
2
) = a
1
sin x
1
a
2
(c
1
x
1
+c
2
x
2
),

a
1
=
1
(x
1
, x
2
),

a
2
=
2
(x
1
, x
2
), where x is dened in the previous exercise. Using a function of the form V = x
P x +

2
2
( a
1
a
1
)
2
+

2
2
( a
2
a
2
)
2
,
nd
1
(x
1
, x
2
) and
2
(x
1
, x
2
) yielding properties analogous to (a)-(c) in the previous exercise. Produce simulations analogous
to those in the previous exercise that conrm your theoretical results.
4.8 Converse stability theorems
We have seen that the existence of a Lyapunov function implies asymptotic (or exponential) stability. Is
the converse true? The answer is yes, and it is the subject of this section.
The next landmark theorem was published by Jos Luis Massera in 1956 [Mas56].
Theorem 4.30 (Massera 1956). (a) Let x = 0 be an asymptotically stable equilibrium of (4.1), and suppose that
the vector eld f is locally Lipschitz on A. Then, there exists a ball B
n
r
(0) and a C
1
function V : B
n
r
(0) R
which is p.d. at 0 and such that L
f
V is n.d. at 0.
(b) If x = 0 is globally asymptotically stable and A = R
n
, then there exists a radially unbounded C
1
function
V : R
n
R which is p.d. at 0 and such that L
f
V is n.d. at 0.
At about the same time, J. Kurzweil published a converse theorem which takes into account the domain
of attraction of the equilibrium, and is therefore more general than Masseras. His paper [Kur63], written
in Russian, appeared in 1956 in the Czechoslovak Mathematical Journal, and was translated into English
in 1963.
Theorem 4.31 (Kurzweil). Let x = 0 be an asymptotically stable equilibrium of (4.1) with domain of attraction
T
0
, and suppose that f is locally Lipschitz on A. Then, there exists a C
1
function V : T
0
R such that
(a) V(x) is p.d. at 0 and V(x) as x T
0
.
(b) For all c > 0, the sublevel set x : V(x) c is a compact subset of T
0
.
(c) L
f
V(x) is n.d. at 0.
The proofs of the two theorems above are rather involved and are omitted. We conclude with a converse
theorem for exponential stability.
Theorem 4.32. Let x = 0 be an exponentially stable equilibrium of (4.1), and suppose that the vector eld f is
locally Lipschitz on A. Then, there exists a ball B
n
r
(0) and a C
1
function V : B
n
r
(0) R such that, for some
c
1
, c
2
, c
3
, c
4
> 0, and for all x B
n
r
(0),
(a) c
1
|x|
2
V(x) c
2
|x|
2
(b) L
f
V c
3
|x|
2
(c) |dV
x
| c
4
|x|.
Moreover, if A = R
n
and x = 0 is globally exponentially stable, then V and the properties above are dened on
the whole R
n
.
Interestingly, there isnt a converse Lyapunov theorem for the property of stability, and in fact the stabil-
ity of an equilibrium of an autonomous dynamical system does not imply the existence of a Lyapunov
function. The most one can say is that there exists a time-varying function V(t, x) which satises prop-
erties analogous to V in Theorem 4.6.
101
Chapter 5
Introduction to Nonlinear Stabilization
This chapter presents some basic results in the theory of nonlinear stabilization. The rst part presents
the notion of control-Lyapunov function, the Artstein-Sontag theorem on the existence of C
1
stabilizing
feedbacks, and Brocketts necessary conditions for stabilization. In the second part of the chapter, we
focus on a specic class of nonlinear control systems, the so-called passive systems, which model a large
class of electro-mechanical systems, and for which the stabilization problem is well-understood.
5.1 Regular Feedbacks
In this chapter we consider nonlinear control systems modeled as
x = f (x) +
m
i=1
g
i
(x)u
i
= f (x) + g(x)u (5.1)
where f and g
i
, i m, are locally Lipschitz vector elds on R
n
and u = [u
1
. . . u
m
]
is the vector
of control inputs. The above model is called control-afne because the control input u appears in the
equation in a linear afne manner. A more general model would be given by the non-afne equation x =
f (x, u). Before talking about stabilization we need to dene the class of controllers under consideration.
Denition 5.1. A function u which is locally Lipschitz on R
n
0 and continuous at x = 0 is called a
regular feedback.
The term feedback is used to highlight the dependence of the control function u(x) on the system
state: the information about the current state of the system is fed back by the controller to the system.
In contrast to this, an open-loop controller is a function of time, u(t), which has been computed in
advance. Sometimes, the controller is itself a dynamical system with a state, as in the adaptive control
example 4.22, in which case one speaks of dynamic feedback. The controller could also depend on both
x and t, in which case it is called a time-varying feedback.
102 CHAPTER 5. INTRODUCTION TO NONLINEAR STABILIZATION
The reason why, in Denition 5.1, we consider feedbacks that are only continuous at the origin is that
this is precisely the type of regularity needed in the Artstein-Sontag theorem, presented in what fol-
lows. Continuity at the origin, however, raises the question of uniqueness of solutions. For instance, in
Example 2.18 we have seen that the vector eld x = x
1/3
, which is continuous at 0, but not Lipschitz
continuous at 0, has two solutions satisfying x(0) = 0, namely the null solution and x(t) = (2/3t)
3/2
.
This latter solution leaves the origin, so x = 0 is not an equilibrium! Consider, however, the equation
x = x
1/3
. In this case, all solutions are unique in forward time
1
and the origin is globally asymptoti-
cally stable. This fact is a consequence of the next theorem.
Theorem 5.2. Let f : R
n
R
n
be locally Lipschitz on R
n
0 and continuous at x = 0, and let f (0) = 0.
Assume that there exists a C
1
Lyapunov function V : D R, where D R
n
is a domain containing 0, which is
p.d. at 0, and such that L
f
V is n.d. at 0. Then, all solutions are unique in forward time and x = 0 is asymptotically
stable.
Exercise 5.1. Prove the theorem above.
[Hint: For all x
0
,= 0, solutions are dened and unique for some time, because f is locally Lipschitz away from x = 0. Using
V, show that all solutions originating in a neighborhood of the origin are dened for all t 0. Using the properties of V,
show that there is a unique solution through the origin. Use this fact to deduce that for all x
0
R
n
there is a unique solution
through x
0
in forward time. Asymptotic stability is then obvious.
Exercise 5.2. In the context of Theorem 5.2, are solutions guaranteed to be unique for t 0? If not, which initial conditions
can give rise to non-uniqueness in backward time?
5.2 Control-Lyapunov functions
Problem5.3 (Equilibrium Stabilization Problem). Suppose that x = 0 is a controlled equilibriumof (5.1),
i.e., that there exists u
R
m
such that f (0) + g(0)u
= 0. Find, if possible, a regular feedback u : R

n
R
m
such that x = 0 is a [globally] asymptotically stable equilibrium of the closed-loop system
x = f (x) + g(x) u(x).
We begin our discussion by nding necessary conditions for the existence of a global solution to Prob-
lem 5.3 in terms of Lyapunov functions. Suppose that a locally Lipschitz feedback u(x) has been found
that globally asymptotically stabilizes x = 0. The closed-loop system is given by
x =

f (x) := f (x) +
m
i=1
g
i
(x) u
i
(x).
By Masseras theorem, Theorem 4.30, there exists a radially unbounded C
1
function V : R
n
R which
is p.d. at 0 and such that
L

f
V(x) = L
f
V(x) +
m
i=1
L
g
i
V(x) u
i
(x)
1
The property of uniqueness of solutions in forward time can be stated as follows. For all x
0
R
n
, if x(t) : T
x
0
R
n
and
x(t) :

T
x
0
R
n
are two solutions of x = f (x) with initial condition x
0
, then T
+
x
0
=

T
+
x
0
and x(t) = x(t) for all t T
+
x
0
.
5.3 ARTSTEIN-SONTAG THEOREM 103
is n.d. at 0. Obviously, the function V has the property that
(x R
n
, x ,= 0)(( u
1
, . . . , u
m
) R
m
) L
f
V(x) +
m
i=1
L
g
i
V(x) u
i
< 0
(simply let u
i
= u
i
(x)). In other words, at each x one can nd a control vector u that makes f (x) + g(x) u
point to the interior of the sublevel set of V through x. The condition above is equivalent to
(x R
n
)(x ,= 0 and L
g
1
V(x) = = L
g
m
V(x) = 0) = L
f
V(x) < 0, (5.2)
and can also be written as
(x R
n
, x ,= 0) inf
uR
m
_
L
f
V +
m
i=1
L
g
i
V(x)u
i
_
< 0 (5.3)
Denition 5.4. A C
1
control-Lyapunov function (clf) is a C
1
function V : R
n
R which is p.d., radially
unbounded, and which satises condition (5.2) (or, equivalently, condition (5.3)).
Our discussion illustrates that if a locally Lipschitz feedback u(x) is to exist that globally stabilizes
x = 0, then there must exist a C
1
clf. Moreover, by the continuity of u(x) at x = 0, we have that for
all > 0 there is a > 0 such that, if x ,= 0 is such that |x| < , then there is u R
m
satisfying
L
f
V(x) +
m
i=1
L
g
i
V(x) u
i
< 0 and such that | u| < . Once again, this statement follows by setting
u = u(x) and using the continuity of u(x). This latter observation motivates the next denition.
Denition 5.5. A clf V : R
n
R satises the small control property if
( > 0)( > 0)(x ,= 0) |x| < = ( u R
m
, | u| < ) L
f
V(x) +
m
i=1
L
g
i
V(x) u
i
< 0.
5.3 Artstein-Sontag Theorem
So far we have established that a necessary condition for the existence of a locally Lipschitz stabilizing
feedback is that there exists a C
1
clf V satisfying the small control property. Now suppose that we are
given such a clf. Can we use it to determine a regular stabilizing feedback? One idea would be to
dene, for each x ,= 0, u(x) to be the minimizer of the function L
f
V(x) +
m
i=1
L
g
i
V(x)u
i
. We know
by (5.3) that the inmum of this function must be < 0, and so this denition would give a feedback
that makes L

f
V n.d.. This feedback, however, has some problems. First, for a given x there may be more
than one minimum, and the minimization process may result in a function u(x) which is everywhere
discontinuous, leading to a problem of existence of solutions of the closed-loop system. Second, it is
rarely the case than one can analytically determine the minimizer of a nonlinear function, and it would
be impractical to have to compute this minimizer numerically for every x. Is it possible, at least in
principle, to nd a continuous stabilizer? The afrmative answer to this question was provided in the
early eighties by Zvi Artstein [Art83], who proved that the existence of a C
1
clf implies the existence
of a global stabilizer which is locally Lipschitz on R
n
0 and, if the small control property holds,
also continuous at x = 0. Artsteins proof did not provide an explicit feedback solving the stabilization
problem. A few years later, Eduardo Sontag [Son89] gave an explicit construction of a global regular
stabilizer. This fundamental result is today known as Artstein-Sontags theorem.
Theorem 5.6 (Artstein-Sontag). Consider the control system (5.1) and suppose that f (0) = 0. Then, there exists
a global regular stabilizer if and only if there exists a C
1
clf V satisfying the small control property. Moreover one
such stabilizer is given by
u
i
(x) =
_
_
L
g
i
V(x)
L
f
V(x) +
_
[L
f
V(x)]
2
+
_
m
j=1
[L
g
i
V(x)]
2
_
2
m
j=1
[L
g
j
V(x)]
2
,
m
j=1
[L
g
j
V(x)]
2
,= 0
0,
m
j=1
[L
g
j
V(x)]
2
= 0,
(5.4)
i = 1, . . . , m.
Proof Sketch. The necessity of the statement has already been proved. For the sufciency, we will only
prove that L

f
V is n.d., and will omit the more involved proof that u in (5.4) is locally Lipschitz on
R
n
0 and continuous at 0.
Use V as a Lyapunov function candidate. If x ,= 0 is such that
m
j=1
[L
g
j
V(x)]
2
= 0, the Lie derivative of
V along the vector eld of the closed-loop system is L

f
V = L
f
V which, by property (5.2) of clfs, is < 0.
Otherwise, if
m
j=1
[L
g
j
V(x)]
2
,= 0,
L

f
V = L
f
V +
m
i=1
L
g
i
V(x) u
i
=
_
[L
f
V]
2
+
_
[L
g
j
V]
2
_
< 0.
Therefore, L

f
V is n.d.. This fact and Theorem 5.2 yield the required result.
Example 5.7. Consider the equation of a pendulum without friction, whose pivot point is subject to a
control torque u,
x
1
= x
2
x
2
= sin x
1
+ u.
For convenience, all coefcients in the model have been set to 1. Consider the function V(x
1
, x
2
) =
2x
2
1
+ x
2
2
+2x
1
x
2
, which being a p.d. quadratic form is automatically radially unbounded. We now show
that V is a clf. We have
L
f
V(x
1
, x
2
) = 4x
1
x
2
+ 2x
2
2
2x
2
sin x
1
2x
1
sin x
1
L
g
V(x
1
, x
2
) = 2x
2
+ 2x
1
.
If L
g
V(x
1
, x
2
) = 0, i.e., if x
2
= x
1
, then L
f
V(x
1
, x
2
) = 2x
2
2
which is equal to zero only when x
1
=
x
2
= 0, proving that V is a clf. A global stabilizer is
u(x
1
, x
2
) =
_
_
L
g
V
L
f
V +
_
(L
f
V)
2
+ (L
g
V)
4
(L
g
V)
2
L
g
V ,= 0
0 L
g
V = 0.
Note that we have treated x
1
as a real variable and in so doing we considered two angles x
1
and x
1
+2
to be distinct points of the real line. Because of this fact, the point x = [2 0]
is not an equilibrium of the

closed-loop system and if the system were initialized there, the controller would make the pendulum
perform one complete revolution in order to unwind the angle x
1
and make it converge to 0. This
behavior is undesirable in practice.
Note also that the controller coming out of the clf-based design is unreasonably complicated! A much
simpler feedback solving the same problem is given by u(x
1
, x
2
) = sin x
1
K
1
x
1
K
2
x
2
, where K
1
, K
2
> 0
are two design parameters. With this feedback, the closed-loop system is an asymptotically stable LTI
system.
Exercise 5.3. Consider again the controlled pendulum in the example above. Show that the energy of the pendulum is not a
clf.
The example and exercise above illustrate some of the drawbacks of clfs. Namely, it may be difcult
to nd a clf without resorting to trial and error and intuition. This is true even for electro-mechanical
systems, where the energy of the system is generally not a clf. Further, even if a clf were available, the
feedback resulting from the Artstein-Sontag theorem may be unnecessarily complicated.
Having established that the existence of a C
1
clf is both necessary and sufcient for stabilizability by
means of regular feedback, we conclude this section with two examples illustrating some of the chal-
lenges in nonlinear stabilization. The rst example, due to E. Sontag, illustrates that some systems do
not admit continuous stabilizers, and yet they can be stabilized by discontinuous feedback.
Example 5.8. The phase curves of the dynamical system
x
1
= x
2
1
x
2
2
x
2
= 2x
1
x
2
are circles with centre on the x
2
axis, except for the x
1
axis, which is itself a phase curve.
2 1.5 1 0.5 0 0.5 1 1.5 2
2
1.5
1
0.5
0
0.5
1
1.5
2
x
1
x
2
Now consider the control system
x
1
= (x
2
1
x
2
2
)u
x
2
= (2x
1
x
2
)u.
Because u multiplies the entire vector eld, its effect is solely to change the speed of a point traveling on
the circles. The result is that, no matter what feedback u one chooses, the phase portrait of the closed-
loop system looks the same as that of the open-loop system, but the orientation of the phase curves
depends on the choice of u, and some points on the plane may be equilibria. If, for instance, u < 0, then
the orientation of the phase curves is reversed. We now show that no continuous feedback u(x
1
, x
2
) can
exist that globally asymptotically stabilizes the origin. For, suppose such a feedback exists, and consider
the semicircle S connecting the point [1 0]
to [1 0]
, as in the gure below.

2 1.5 1 0.5 0 0.5 1 1.5 2
2
1.5
1
0.5
0
0.5
1
1.5
2
x
1
x
2
S
A parametrization of S is
(x
1
(), x
2
()) = (cos(), sin()),
with [0, 1]. Since the x
1
axis is invariant no matter what feedback one chooses, for the origin to be
globally asymptotically stable it is necessary that x
1
< 0 when = 0 and x
1
> 0 when = 1. On S, we
have
x
1
= (cos
2
() sin
2
()) u(cos(), sin()).
In order for x
1
[
=0
< 0, it must be that u(cos(), sin())[
=0
< 0. Similarly, the condition x
1
[
=1
> 0
gives u(cos(), sin())[
=1
> 0. If u is continuous, then so is the function
g() = u(cos(), sin()).
Since g is continuous, g(0) < 0, and g(1) > 0, there exists

(0, 1) such that
g(
) = u(cos(
), sin(
)) = 0,
and so the closed-loop system has an equilibrium point on the semicircle S. Hence, the closed-loop
system has at least two distinct equilibria, and so the origin cannot be globally asymptotically stable.
Having shown that a regular stabilizer does not exist (and therefore there cannot exist a C
1
clf), consider
the discontinuous controller
u(x
1
, x
2
) =
_
_
_
1 x
1
0
1 x
2
< 0.
The controller is discontinuous on the x
2
axis and, leaving aside the issue of existence of solutions
through the x
2
axis, it should be intuitively clear that the origin of the closed-loop system is globally
asymptotically stable.
The example below, due to M. Kawski, illustrates a different complication, namely the fact that some
nonlinear control systems cannot be stabilized by C
1
feedback, but they admit a regular stabilizer.
Example 5.9. Consider
x
1
= u
x
2
= x
2
x
3
1
If there existed a C
1
stabilizer u(x
1
, x
2
), then in order for (x
1
, x
2
) = (0, 0) to be an equilibrium of the
closed-loop system, we should have u(0, 0) = 0. Moreover, the linearization at (0, 0) of the closed-loop
system should not have eigenvalues with real part > 0. However, the linearization is
_
u
x
1
(0)
u
x
2
(0)
0 1
_
which always has an eigenvalue at 1! Therefore, there cannot exist a C
1
stabilizer. Kawski, however,
showed that this system admits the regular stabilizer
u(x
1
, x
2
) = x
1
+ x
2
+
4
3
x
1/3
2
x
3
1
.
The phase portrait of the closed-loop system is below.
1 0.5 0 0.5 1
1
0.5
0
0.5
1
x
1
x
2
5.4 Brocketts necessary conditions for continuous stabilizability
We have seen that there are systems which are not stabilizable by regular feedback, but admit discon-
tinuous stabilizers. In this section, we present necessary conditions for a regular stabilizer to exist,
developed by Roger Brockett [Bro83]. These conditions are useful because they will allow us to rule out
the existence of continuous stabilizers for certain classes of control systems. Brocketts theorem applies
not just to the class of control-afne systems, but to the more general class
x = f (x, u), (x, u) R
n
R
m
. (5.5)
Before presenting the theorem, we develop some intuition in the special case when n = m = 1. In this
case, if there exists a continuous feedback u(x) that stabilizes the origin, then necessarily
f (x, u(x)) > 0 if x < 0,
f (x, u(x)) = 0 if x = 0,
f (x, u(x)) < 0 if x > 0.
x
y = f (x, u(x))
We see that, for any > 0, the image of the interval (, ) under the map x f (x, u(x)) must contain
a neighborhood of y = 0.
5.4 BROCKETTS NECESSARY CONDITIONS FOR CONTINUOUS STABILIZABILITY 109
Exercise 5.4. Using the fact that the image of any neighborhood (, ) under the map x f (x, u(x)) contains a neighborhood
of y = 0, show that the image of any neighborhood (, ) (, ) under the map (x, u) f (x, u) contains a neighborhood
of y = 0.
Brocketts theorem generalizes the necessary condition established in the above exercise to the case when
the dimensions n and m are arbitrary.
Theorem 5.10 (Brockett). Consider system (5.5), with f locally Lipschitz on R
n
R
m
, and suppose that
f (0, 0) = 0. If (5.5) admits a continuous stabilizer u(x) with u(0) = 0 then, for all > 0 there exists > 0 such
that B
n
(0) f (B
n
(0) B
m
(0)).
The assumption of the theorem says that given any > 0, there exists > 0 such that for any y R
n
satisfying |y| < , the equation
y = f (x, u)
has a solution (x, u) such that |x| < and |u| < . A map f satisfying this property is said to be open
at (x, u) = (0, 0).
Remark 5.11. By taking the contrapositive version of the theorem statement, we get a sufcient condition
for the non-existence of continuous (and hence regular) stabilizing feedbacks: if there exists > 0 such
that, for all > 0, the set f (B
n
(0) B
m
(0)) does not entirely contain the ball B

n
(0), then there does not

exist a continuous stabilizer of x = 0. Note that the theorem only assumes continuity of the feedback,
so there may be more than one solution of the differential equation. Our proof, however, will assume
that solutions are unique. Additionally, for the sake of simplicity, in our proof we will assume that the
stabilizer is global.
The proof of Theorem 5.10 rests upon the following xed point theorem.
Theorem 5.12. Let be a compact subset of R
n
and f : a continuous map. Suppose that there exists a
continuous function r : R
n
which is surjective and such that, for all x , r(x) = x. Then, there exists a
point x such that f ( x) = x.
The function r in the theorem above is called a retraction, and is called a retract of R
n
. Intuitively,
think of a retraction R
n
as a way to continuously deformR
n
into in such a way that points of R
n
that are in are left unchanged by the deformation. For instance, a closed disk in the plane is a retract
of the plane, but a circle is not a retract of R
2
.
Proof of Theorem 5.10. Suppose that a global continuous stabilizer u(x) exists. We will assume that
solutions of the closed-loop system are unique in forward time. By Masseras theorem, Theorem 4.30,
there exists a C
1
p.d. and radially unbounded Lyapunov function V : R
n
R for the closed-loop system
x = f (x, u(x)). If we let

f (x) := f (x, u(x)), then L

f
V < 0 for all x ,= 0. Let > 0 be arbitrary and pick
c > 0 such that
c < min
|x|=
V(x).
Then, the sublevel set
c
= x

B
n
(0) : V(x) c is contained in B

n
(0) and is compact. Its boundary
c
= x B
n
(0) : V(x) = c is also compact and does not contain the origin, so L
f
V has a maximum
M
1
< 0 on
c
. Moreover, since x |dV
x
| (where |dV
x
| is the induced matrix norm of dV
x
) is a
continuous function, it has a maximum M
2
> 0 on
c
. Consider the perturbed vector eld

f (x) y,
where y R
n
is a constant vector. For all x
c
, we have dV
x
(

f (x) + y) = L

f
V(x) + dV
x
y
M
1
+ M
2
|y|. Letting = M
1
/M
2
, we have that if |y| < , L

f y
V < 0 on
c
, and so
c
is positively
invariant for x =

f (x) y. Denote by (t, x
0
) the ow of

f (x) and by

(t, x
0
) the ow of

f (x) y. Let
t
(x
0
) =

(t, x
0
) be the t-advance mapping of

f (x) y. The positive invariance of
c
under the vector
eld

f (x) y implies that, for all h > 0,

h
(
c
)
c
and so

h
[
c
:
c

c
. We claim that
c
is a
retract of R
n
. Since x = 0 is globally asymptotically stable for x =

f (x), for each x
0
R
n
there exists
t 0 such that (t, x
0
)
c
. Let T(x
0
) be the least such t (if x
0

c
, then obviously T(x
0
) = 0). It can
be shown that the function T(x
0
) just dened is continuous. The continuous map r(x
0
) = (T(x
0
), x
0
)
is the desired retraction of R
n
onto
c
. For, if x
0

c
, then T(x
0
) = 0 and r(x
0
) = x
0
. Moreover, by
denition of T(x
0
), for any x
0
R
n
we have r(x
0
)
c
. So far we have established that
c
is a retract
of R
n
and that, for each h > 0,

h
[
c
:
c

c
. Since, for each h > 0, the map x

h
(x) is continuous,
by Theorem 5.12 there exists x
h

c
such that

h
(x
h
) = x
h
. Take a sequence h
i
converging to zero,
which gives a sequence x
h
i

c
such that

h
i
(x
h
i
) = x
h
i
. Since
c
is compact, x
h
i
is bounded, and
therefore it has a convergent subsequence with limit x
c
. Replace h
i
and x
h
i
by the respective
convergent subsequences. We have
lim
h
i
0
(h
i
, x
h
i
) x
h
i
h
i
= lim
h
i
0
(h
i
, x) x
h
i
=

f ( x) y.
At the same time,

(h
i
, x
h
i
) = x
h
i
, so the limit above must be zero, and

f ( x) = y. We have thus found
that for all > 0, there exists > 0 such that for all y B
n
(0), the equation f (x, u(x)) = y has a solution

x B
n
(0). This fact implies the statement of the theorem (see Exercise 5.4).
Exercise 5.5. Consider the LTI control system x = Ax + Bu. Show that Brocketts necessary condition for this system amounts
to requiring that rank[A, B] = n. In other words, 0 must be a controllable eigenvalue.
Corollary 5.13. Consider a driftless control system on R
n
x =
m
i=1
g
i
(x)u
i
, m < n.
If rank [g
1
(0) g
m
(0)] = m, then there is no regular stabilizer x = 0.
Proof. The matrix G(x) = [g
1
(x) g
m
(x)] has m linearly independent rows when x = 0. Without
loss of generality, we can assume that the rst m rows are linearly independent when x = 0, so that
partitioning G as
G(x) =
_
G
1
(x)
G
2
(x)
_
m
n m
G
1
(0) is a nonsingular m m matrix. By continuity of the vector elds g
i
(x), the rst m rows of G
remain linearly independent in a neighborhood B
n
(0) of the origin, and so G

1
(x) is nonsingular for all
x B
n
(0). From this we deduce that any point of the form [0

m
a
, where 0
m
is the vector with m zeros
and a ,= 0 is a vector in R
nm
, cannot be in the image of the map
f : B
n
(0) B
n
(0) R
n
, f (x, u) = G(x)u.
5.4 BROCKETTS NECESSARY CONDITIONS FOR CONTINUOUS STABILIZABILITY 111
To see that, write
_
G
1
(x)
G
2
(x)
_
u =
_
0
a
_
.
Since G
1
(x) is nonsingular for all x B
n
(0), the equation G

1
(x)u = 0 gives u = 0. Since u = 0,
G
2
(x)u = 0 ,= a. In conclusion, we have shown that there exists > 0 such that, for all > 0, the set
f (B
n
(0), B
n
(0)) does not entirely contain the ball B

n
(0), proving that the map f is not open at 0.

Exercise 5.6. Consider the front-wheel drive automobile illustrated below.
(x
1
, x
2
)
x
3
x
4
d
Denote by (x
1
, x
2
) the coordinates of the centre of the back-wheel axle, by x
3
the heading angle of the car, and by x
4
the angle
of the front wheels relative to the heading of the car (the steering angle). The control inputs are the linear velocity u
1
of the
middle point of the front axle and the angular velocity u
2
with which the front wheels are turned. The kinematic model of the
car is
x
1
= u
1
cos x
4
cos x
3
x
2
= u
1
cos x
4
sin x
3
x
3
=
u
1
d
sin x
4
x
4
= u
2
,
where d is the distance between the front and rear axles. This system is driftless. Show that given any x
R
4
there is no
regular feedback stabilizing x = x
. Thus, the only way to stabilize x
is to use either discontinuous control, time-varying

control, or dynamic feedback.
Exercise 5.7. Show, by means of a counterexample, that if rank[g
1
(0) g
m
(0)] < m then the statement of Corollary 5.13 is
no longer true. In other words, give an example of a driftless control system with rank g(0) < m which can be stabilized to
the origin by means of a regular stabilizer.
Exercise 5.8. Consider the non-afne control system (5.5), assume that f is C
1
and suppose that f (x
, u
) = 0 for some
(x
, u
) R
n
R
m
. Dene
A =
f
x
(x
, u
), B =
f
u
(x
, u
),
where
f
x
and
f
u
are the Jacobian matrices of the maps x f (x, u) and u f (x, u), respectively. Show that if (A, B) is
stabilizable, then there exists a feedback of the form u = u
+ K(x x
) which stabilizes the equilibrium x
of the nonlinear
system. Thus, if the linearization is stabilizable, the nonlinear control system can be stabilized by linear feedback.
Exercise 5.9. Brocketts theorem provides a necessary condition for stabilizability by regular feedback. In this exercise youll
show that Brocketts condition is not sufcient. Consider the control system in Example 5.8, with
f (x, u) =
_
x
2
1
x
2
2
2x
1
x
2
_
u.
Show that f is open at (0, 0) and thus the condition of Brocketts theorem is satised despite the fact, showed in Example 5.8,
that the origin is not stabilizable by any continuous feedback.
5.5 Passive Systems
A passive system is one that stores energy, may have dissipative components, and exchanges its energy
with the outside world by transferring power through input-output ports. The origin of the notion of
passivity can be traced back to the second half of the 1950s, with work on linear passive network theory,
but the rst general characterization of this property came with the pioneering work of Jan Willems in
the early 1970s, [Wil72a, Wil72b]. Examples of passive systems include passive electrical networks and
large classes of electromechanical systems.
In the rest of the chapter we consider control-afne systems with output,
x = f (x) +
m
i=1
g
i
(x)u
i
= f (x) + g(x)u
y = [h
1
(x) h
m
(x)]
= h(x).
(5.6)
The model above has as many inputs as outputs. The state space is an open set A R
n
. We assume
that f , g
i
, h
i
, i m, are locally Lipschitz on A, and that x = 0 is an equilibrium of the open-loop system,
i.e., f (0) = 0. We let | be the class of piecewise continuous input signals u(t) : R R
m
.
Denition 5.14. System (5.6) is passive if there exists a continuous nonnegative function V : A R,
called the storage function, such that
(x
0
A)(u(t) |)(t T
x
0
) V((t, x
0
, u)) V(x
0
)
_
t
0
h((, x
0
, u))
u()d, (5.7)
where (t, x
0
, u) denotes the solution of (5.6) at time t with initial condition x
0
and input u(t), and T
x
0
is
its maximal interval of existence. If condition (5.7) holds with equality, rather than inequality, then the
system is said to be lossless.
Remark 5.15. It can be shown that the piecewise continuity of u(t) guarantees that the solution (t, x
0
, u)
does in fact exist and is unique. Condition (5.7) is called the dissipation inequality. If one thinks of
each vector eld g
i
as a port in an electrical network, u
i
as the voltage supplied to this port, and y
i
as the
current owing into the network through this port, then the product y
i
u
i
is the power supplied to the
system through the port, and the inner product y
u is the total power supplied to the system through

all ports. With this analogy in mind, the integral in (5.7) is the energy supplied to the system through its
ports, and the dissipation inequality states that the variation in the stored energy V is less than or equal
to the supplied energy. The inequality signies the fact that the system may dissipate some of its stored
energy through resistive elements.
Remark 5.16. One can also give a mechanical interpretation of the dissipation inequality. Think of V(x)
as the total energy of a mechanical system; think of y as a generalized velocity vector, and of u as
a generalized external force. Then,
_
t
0
y()
u()d is the work made by the external force along a

trajectory. In this context, inequality (5.7) states that the variation in total energy of the system is less or
equal than the work made by the external force. The difference between the energy variation and the
work produced by the external force is due to dissipation effects such as friction.
5.5 PASSIVE SYSTEMS 113
Often, the storage function V is not just continuous, but C
1
, in which case by dividing both sides of the
inequality (5.7) by t and taking the limit as t tends to 0 from the right we obtain
(x
0
A)(u(t) |)
d
dt
t=0
V((t, x
0
, u)) h(x
0
)
u(0),
or, what is the same,
(x A)(u R
m
)

V = L
f
V(x) + L
g
V(x)u h(x)
u, (5.8)
where L
g
V(x) := [L
g
1
V L
g
m
V]. The inequality above and (5.7) are equivalent because the latter can
be obtained from the former through integration with respect to time. The lemma below gives a useful
characterization of passivity when the storage function is C
1
.
Lemma 5.17. System (5.6) is passive with a C
1
storage function V if and only if
(x A) L
f
V(x) 0, L
g
V(x) = h(x)
. (5.9)
The specialization of this result to LTI systems is referred to as the Kalman-Yakubovich-Popov Lemma.
Note that if the storage function V is p.d. at 0, then by (5.9) we deduce that x = 0 is stable for the
open-loop system x = f (x).
Proof. It is obvious that condition (5.9) implies (5.8), so we only need to show that the converse is true
as well. To this end, suppose that
(x A)(u R
m
) L
f
V(x) + L
g
V(x)u h(x)
u.
Setting u = 0, we obtain L
f
V(x) 0 for all x A. Suppose, by way of contradiction, that there exists
x A such that L
g
V( x) ,= h( x)
. Let
u =
h( x) L
g
V( x)
|h( x) L
g
V( x)
|
2
2
(L
f
V( x) 1).
Substituting this choice of u in the inequality L
f
V( x) [h( x) L
g
V( x)
u, we obtain
L
f
V( x)
[h( x) L
g
V( x)
[h( x) L
g
V( x)
]
|h( x) L
g
V( x)
|
2
2
(L
f
V( x) 1),
and so L
f
V( x) L
f
V( x) 1, a contradiction.
The next two examples illustrate the importance of passive systems.
Example 5.18. Consider the nonlinear RLC circuit displayed below.
+
_
u
y
R
H
1
(Q)
H
2
()
In the above, H
1
(Q) is the electric energy of the capacitor, Q is the capacitor charge, H
2
() is the
magnetic energy of the inductor, and is the ux linkage (if the elements were linear, we would have
H
2
() =
2
/(2L) and H
1
(Q) = Q
2
/(2C)). We have the following relations for voltages and currents
v
capacitor
=
dH
1
dQ
, i
capacitor
=

Q
v
inductor
=

, i
inductor
=
dH
2
d
.
The total energy of the system is V(Q, ) = H
1
(Q) + H
2
(). Letting x = [Q ]
and y = i
inductor
, we
have
x =
_
0 1
1 R
_ _
V
x
1
V
x
2
_
+
_
0
1
_
u
y = h(x
1
, x
2
) =
V
x
2
.
For this system we have
f (x
1
, x
2
) =
_
0 1
1 R
_ _
V
x
1
V
x
2
_
, g(x
1
, x
2
) =
_
0
1
_
,
and
L
f
V(x) = dV
x
_
0 1
1 R
_ _
V
x
1
V
x
2
_
= R
_
V
x
2
_
2
0
L
g
V(x) = dV
x
_
0
1
_
=
V
x
2
= h(x
1
, x
2
),
proving that the circuit is passive with storage V.
A class of mechanical systems is passive, as the next example illustrates.
Example 5.19. Fully actuated mechanical systems with n degrees-of-freedom are modeled by the Euler-
Lagrange equations
d
dt
_
L
q
(q, q)
_
L
q
(q, q) = ,
where q R
n
is the vector of generalized conguration coordinates, is the vector of generalized forces,
and L(q, q) is the Lagrangian function. The expression
L
q
denotes a column vector whose i-th entry is
L
q
i
. The column vector
L
q
is dened analogously. Typically, L(q, q) has the form
L(q, q) =
1
2
q
M(q) q P(q),
where M(q) is the positive denite inertia matrix,
1
2
q
M(q) q is the kinetic energy of the system, and

P(q) is its potential energy. The total energy of the system is
E(q, q) =
1
2
q
M(q) q + P(q).
5.5 PASSIVE SYSTEMS 115
The Euler-Lagrange equations can be transformed into Hamiltonian form, which is particularly conve-
nient to expose the passivity property of the system. Let
p = M(q) q =
L
q
be the vector of generalized momenta and let H(q, p) be the total energy in (q, p) coordinates, namely
H(q, p) =
1
2
p
M
1
(q)p + P(q).
We have
H
p
= M
1
(q)p = q. Since p =
L
q
, we have
p =
d
dt
_
L
q
_
=
L
q
+ =

q
_
1
2
q
M(q) q
_
P
q
+ .
It can be shown that
q
_
1
2
q
M(q) q
_
=

q
_
1
2
p
M
1
(q)p
_
p=M(q) q
,
and since
q
_
1
2
p
M
1
(q)p
_
=
H
q

P
q
,
we conclude that
p =
H
q
+ .
In summary, the dynamics in (q, p) coordinates read as
q =
H
p
p =
H
q
+ .
The above is called a fully actuated n degrees-of-freedom Hamiltonian control system, and H(q, p) is
called the Hamiltonian. We choose q to be the output of the system,
y =
H
p
,
and claim that the system with input and output y is passive, with storage function given by the
Hamiltonian. Lets verify:
H =
_
H
q
_
q +
_
H
p
_
p =
_
H
p
_
u.
Therefore, condition (5.8) is satised and the system is passive (actually, lossless).
The Hamiltonian control system above is a special case of the model
x = [J(x) R(x)]
H
x
+ g(x)u
y = g(x)
H
x
,
(5.10)
This is the model of a so-called port-Hamiltonian system where, for each x, J(x) is a skew-symmetric
matrix, i.e., J(x)
= J(x), and R(x) is a symmetric p.s.d matrix.

Exercise 5.10. Show that, if H(x) is nonnegative, the port-Hamiltonian system (5.10) is passive and H is a storage function.
Show that the Hamiltonian control system with state (q, p) is a special case of (5.10).
5.6 Passivity-Based Stabilization
In this section we explore the stabilization of the origin of passive systems by means of passivity-based
feedback, dened next.
Denition 5.20. A passivity-based feedback (PBF) for the passive system (5.6) is a function u =
(h(x)) such that : R
m
R
m
is locally Lipschitz on R
m
and
(a) h(x)
(h(x)) 0 for all x A

(b) h(x)
(h(x)) = 0 if and only if x = 0.

The simplest example of a PBF is u = h(x). A PBF dissipates the system energy, since
d
dt
V((t, x
0
)) = L
f
V((t, x
0
)) + L
g
V((t, x
0
)) [(h((t, x
0
)))]
by (5.9)
h((t, x
0
))
(h((t, x
0
))) < 0 if h((t, x
0
)) ,= 0.
Thus, the energy decreases as long as the output is not zero. Suppose, for the sake of discussion, that
there exists a compact set containing x = 0 which is positively invariant for the closed-loop system
x =

f (x) := f (x) g(x)(h(x)).
Then, the LaSalle invariance principle implies that all solutions originating in asymptotically approach
the largest invariant subset of
E = x : L

f
V(x) = 0 = x : L
f
V(x) h(x)
(h(x)) = 0
= x : L
f
V(x) = 0, h(x) = 0.
In particular, since E x : h(x) = 0, for all x
0
, h((t, x
0
)) 0 as t . Summarizing, in
order to stabilize the origin using a PBF, we need to address two problems:
Find a compact set which is positively invariant for the closed-loop system.
Determine what conditions are needed to guarantee that the property (x
0
) h((t, x
0
)) 0
implies that (x
0
) (t, x
0
) 0.
Concerning the rst problem, if the storage function V is p.d. at 0, then it is a Lyapunov function for
the closed-loop system, and we can take to be any compact sublevel set of V. In order to address the
second problem, we need the notion of zero-state detectability.
5.6 PASSIVITY-BASED STABILIZATION 117
Denition 5.21. System (5.6) is locally zero-state detectable if there exists > 0 such that for all x
0

B
n
(0),
_
h((t, x
0
)) = 0 and u(t) = 0 for all t 0
_
= (t, x
0
) 0 as t . (5.11)
The system is zero state detectable if the condition above holds for all x
0
A.
Exercise 5.11. Show that the LTI system
x = Ax + Bu
y = Cx
is zero-state detectable if and only if the pair (C, A) is detectable.
Remark 5.22. When V is p.d. at 0, the property of zero-state detectability can equivalently be restated
as follows. Let O be the largest subset of x A : h(x) = 0 with the property of being positively
invariant for the open-loop system x = f (x). Then, the system is locally zero-state detectable if and only
if x = 0 is asymptotically stable for the open-loop system relative to O, i.e., it is asymptotically stable
when initial conditions are restricted to be in O. The system is zero-state detectable if and only if x = 0
is globally asymptotically stable relative to O. The intuition behind this remark is this: the set O is the
collection of all phase curves of the open-loop system associated to solutions x(t) such that h(x(t)) is
identically zero for all t 0. Thus, the attractivity of x = 0 relative to O is equivalent to condition (5.11).
The stability of x = 0 relative to O is automatically satised since, by (5.9), x = 0 is a stable equilibrium
of the open-loop system.
Exercise 5.12. Consider again the LTI system of Exercise 5.11. Show that O is the unobservable subspace of the system (the
null space of the observability matrix).
Example 5.23. The LTI system
x
1
= x
1
+ x
2
x
2
= x
2
+ u
y = x
2
is passive with storage function V(x
1
, x
2
) =
1
2
x
2
2
. Indeed,

V = x
2
2
+ x
2
u yu. The system is zero-state
detectable since, if y(t) 0 and u(t) 0 we have
x
1
= x
1
,
and hence (x
1
(t), x
2
(t)) = (x
1
(t), 0) 0 at t . An equivalent way of checking zero-state detectabil-
ity is to nd the set O, the largest subset of (x
1
, x
2
) : x
2
= 0 which is positively invariant for the
open-loop system. In this case, x
2
= 0 is invariant for the open-loop system, and so O = x
2
= 0.
The dynamics on O are given by x
1
= x
1
, and hence the origin is globally asymptotically stable relative
to O. As expected, O is the unobservable subspace of the LTI system.
Later, in Theorem 5.30, we will present sufcient conditions for zero-state detectability. The next fun-
damental result, due to Chris Byrnes, Alberto Isidori, and Jan Willems [BIW91], states that zero-state
detectability is both necessary and sufcient for x = 0 to be stabilizable by a PBF in the case when the
storage function is p.d. at 0.
Theorem 5.24 (Byrnes-Isidori-Willems). Suppose that system (5.6) is passive with a C
1
storage function with
the property that there is a domain D A containing 0 such that V : D R is p.d. at 0. Then, a PBF
asymptotically stabilizes x = 0 if and only if (5.6) is locally zero-state detectable. Moreover, if V is proper (i.e., all
level sets of V are compact) and D = A, then a PBF globally asymptotically stabilizes x = 0 if and only if (5.6)
is zero-state detectable.
Proof. ( =) Suppose that (5.6) is locally zero-state detectable and let

f (x) = f (x) g(x)(h(x)) be the
vector eld of the closed-loop system with a PBF u = (h(x)). Let (t, x
0
) be the phase ow of

f (x).
We have shown earlier that L

f
V 0, so V : D R is a Lyapunov function and 0 is a stable equilibrium
of the closed-loop system. Let > 0 be as in the denition of local zero-state detectability. If needed,
make smaller so that B
n
(0) D. Let c > 0 be small enough that

c
= x

B
n
(0) : V(x) c B
n
(0)
(the existence of c was demonstrated in the proof of Theorem 4.6). Let x
0
be an arbitrary initial condition
in
c
. Since
c
is compact and positively invariant for the closed-loop system, the positive limit set
L
+
(x
0
) is nonempty, compact, invariant, and (t, x
0
) L
+
(x
0
) as t . We will show that L
+
(x
0
) = 0.
The function t V((t, x
0
)) is continuous, nonincreasing (because L

f
V(x) 0), and bounded from
below by 0 (because V is p.d.). Therefore, lim
t
V((t, x
0
)) = a, for some a 0. By the arguments
presented in parts 2 and 3 of the proof of Theorem 4.19, we have
(i) L
+
(x
0
) x
c
: V(x) = a,
(ii) L
+
(x
0
) E = x
c
: L
f
V(x) = 0, h(x) = 0 x
c
: h(x) = 0.
Since L
+
(x
0
) is an invariant set for the closed-loop system, for any y
0
L
+
(x
0
), (t, y
0
) L
+
(x
0
) for
all t R. By property (ii), h((t, y
0
)) 0, and thus u(t) = (h((t, y
0
))) = 0 for all t R. We have
y
0
L
+
(x
0
)
c
B
n
(0) and thus, by local zero-state detectability, (t, y

0
) 0. By continuity of V,
V((t, y
0
)) 0 as well. At the same time, by property (i), V((t, y
0
)) a, and so a = 0, proving that
y
0
x
c
: V(x) = 0 = 0. Since y
0
is an arbitrary point in L
+
(x
0
), we have that L
+
(x
0
) = 0
and so x = 0 is attractive for the closed-loop system. If V is proper, D = A, and (5.6) is zero-state
detectable, then and the constant c may be chosen arbitrarily large, and so x = 0 is globally attractive
for the closed-loop system.
( =) Suppose that a PBF asymptotically stabilizes x = 0. Pick > 0 small enough that B
n
(0) is
contained in the domain of attraction of 0. For any x
0
B
n
(0) such that h((t, x

0
)) = 0 for all t 0, we
have u(t) = (h((t, x
0
))) 0, and, obviously, (t, x
0
) 0, proving local zero-state detectability. The
proof that global asymptotic stabilizability implies zero-state detectability is analogous.
5.7 APPLICATION TO HAMILTONIAN CONTROL SYSTEMS 119
5.7 Application to Hamiltonian control systems
We return to the fully-actuated n degrees-of-freedom Hamiltonian control system of Example 5.19
q =
H
p
(q, p)
p =
H
q
(q, p) +
y =
H
p
,
(5.12)
with Hamiltonian
H(q, p) =
1
2
p
M
1
(q)p + P(q).
The inertia matrix M(q) is assumed to be p.d. for each q. Our objective in this section is to relate the
property of zero-state detectability to properties of the potential function P(q), and to use Theorem 5.24
to design controllers for this class of systems.
The equilibria of the open loop system are points (q, p) such that
H
p
= 0 and
H
q
= 0. We have
H
p
= 0
if and only if p = 0, while
H
q
= 0

q
_
1
2
p
M
1
(q)p
_
+
P
q
(q) = 0.
We have
q
i
_
1
2
p
M
1
(q)p
_
,
and every term of the sum is zero when p = 0. In conclusion, the equilibria of the open loop system are
points (q
, 0) such that q
is a stationary point of P, i.e., dP

q
= 0. We claim that if P(q) has an isolated
stationary point at q = 0 (i.e., there is > 0 such that q = 0 is the only stationary point of P in B
n
(0)),
then the system is locally zero-state detectable; and if q = 0 is the only stationary point of P(q), then the
system is zero-state detectable. To prove the claim, pick > 0 so that q = 0 is the only stationary point of
P in B
n
(0). Following Denition 5.21, pick (t) 0 and consider initial conditions (q
0
, p
0
) B
n
(0) R
n
such that y(t) =
H
p
(q(t), p(t)) 0 or, what is the same, M
1
(q(t))p(t) 0. Since M
1
is p.d., we have
p(t) 0. In turn,
p(t) 0 = p(t) 0 =
H
q
(q(t), p(t)) 0 =
P
q
(q(t)) 0.
In particular,
P
q
(q
0
) = 0, and thus q
0
= 0 because 0 is the only stationary point of P in B
n
(0). We have
established that (q
0
, p
0
) = (0, 0), and thus (q(t), p(t)) 0, proving local zero-state detectability. If q = 0
is the only stationary point of P, then we can let be arbitrarily large and we get zero-state detectability.
Our discussion is summarized in the next result.
Lemma 5.25. If q = 0 is an isolated stationary point of P(q), then the Hamiltonian control system (5.12) is
locally zero-state detectable. If q = 0 is the only stationary point of P(q), then (5.12) is zero-state detectable.
We are now ready to apply Theorem 5.24 to the Hamiltonian system (5.12).
Theorem 5.26. Consider system (5.12) and suppose that P(q) has an isolated stationary point at q = 0 which is
a local minimum. Then, any PBF feedback of the form u = ( q) asymptotically stabilizes the origin (q, p) =
(0, 0). If P is proper and if q = 0 is the only stationary point of P(q), and a global minimum, then any PBF
globally asymptotically stabilizes (q, p) = (0, 0).
Proof. Let > 0 be small enough that q = 0 is the only stationary point of P in B
n
(0), and that for all

q ,= 0 in B
n
(0), P(q) > P(0). Dene D = B

n
(0) R
n
. Since q = 0 is a local minimum of P, the function
V : D R,
V(q, p) = H(q, p) P(0) =
1
2
p
M
1
(q)p + P(q) P(0),
is p.d. at (0, 0) and therefore it is a storage function. By Lemma 5.25, system (5.12) is zero-state de-
tectable. By Theorem 5.24, any PBF asymptotically stabilizes the origin. If q = 0 is the only stationary
point of P, then we can take D = A R
n
. If P is proper, then so is V(q, p) and so by Theorem 5.24 any
PBF globally asymptotically stabilizes the origin.
Theorem 5.26 has the following physical interpretation. A PBF of the form u = ( q) introduces viscous
friction in the system. If the potential energy has a minimum at q = 0, this energy dissipation drives
solutions of the system originating near the equilibrium to a minimum of the potential function. If the
minimum is unique and there is no other stationary point, then the result is global provided that P is
proper.
Example 5.27. Consider the controlled pendulum
x
1
= x
2
x
2
=
Mgl
I
sin x
1
+
u
I
,
with output y = x
2
. Let q = x
1
, p = Ix
2
, and
H(q, p) =
1
2I
p
2
+ Mgl(1 cos q).
In (q, p) coordinates, the pendulum model takes on the Hamiltonian form (5.12), namely
q =
p
I
=
H
p
p = Mgl sin q + u =
H
q
+ u
y = q =
H
p
.
The stationary points of the potential function P(q) = Mgl(1 cos q) are q = 2k (local minima) and
q = (2k + 1) (local maxima). By Theorem 5.26, any feedback of the form u = ( q) asymptotically
stabilizes all points (q, p) = (2k, 0). This result is obvious if we think of the PBF as a viscous friction
term in the torque.
5.7 APPLICATION TO HAMILTONIAN CONTROL SYSTEMS 121
For fully actuated Hamiltonian control systems, the control strategy in Theorem 5.26 allows one to only
stabilize equilibria (q
, 0) where q
is an isolated local minimum of the potential. If the equilibrium one

wants to stabilize is not a minimum of the potential, one can shape the potential by means of a suitable
feedback in such a way that the minimum of the new potential is at the desired conguration q
.
Theorem 5.28 (Potential energy shaping). Consider the Hamiltonian control system (5.12) and let P
d
(q) be a
desired potential function with an isolated stationary point at q
which is a local minimum. Then, the feedback

u =
P
q

P
d
q
( q),
where the term ( q) is a PBF, asymptotically stabilizes the equilibrium (q
, 0). If P is proper, and if q
is the
only stationary point of P, and a global minimum, then the stabilization is global.
Proof. Consider the feedback transformation
u =
P
q

P
d
q
+ v,
where v R
m
is the new control input. The closed-loop system reads as
q =
H
p
p =
H
q
+
P
q

P
d
q
+ v.
Letting H
d
(q, p) =
1
2
p
M
1
(q)p + P
d
(q), and using the identities
H
d
p
=
H
p
,
H
d
q
=
H
q

P
q
+
P
d
q
,
we can rewrite the closed-loop system as
q =
H
d
p
p =
H
d
q
+ v.
We have thus obtained a new Hamiltonian control system with shaped potential P
d
(q). By Theorem 5.26,
a PBF v = ( q) yields the required result.
Example 5.29. We return to the controlled pendulum example, but this time we seek to stabilize an
unstable equilibrium (k, 0). Consider the desired potential energy
P
d
(q) = Mgl(1 + cos q)
which has local minima at k. For any K > 0, the feedback
u =
P
q

P
d
q
K q = 2Mgl sin q K q
asymptotically stabilizes all equilibria (k, 0). This feedback has a very natural interpretation. If we
substitute it into the original pendulum model in (x
1
, x
2
) coordinates, we get the closed-loop system
x
1
= x
2
x
2
=
Mgl
I
sin x
1
Kx
2
I
.
The above is the equation of a pendulum with viscous friction subject to a gravity force pointing up-
wards! Thus, the potential energy compensation term articially changes the gravity force in a benecial
way.
To obtain global asymptotic stabilization of, say, (q, p) = (, 0), we could have chosen the proper func-
tion
P
d
(q) =
1
2
(q )
2
,
which has a global minimum at . This choice leads to the feedback
u = Mgl sin q (q ) K q,
which achieves global asymptotic stabilization of (, 0). Such a feedback, however, is undesirable in
practice because any point (k, 0) with k ,= 1 is not an equilibrium of the closed-loop system, even
though all congurations k are physically equivalent.
Exercise 5.13. Let be a desired angle. Using potential energy shaping, design a feedback that stabilizes all poins (q, p) =
( + 2k, 0), k Z.
5.8 Damping Control
In this section we illustrate another application of the passivity-based stabilization theorem of Section 5.6
to control systems that are open-loop stable. We return to the control afne model
x = f (x) +
m
i=1
g
i
(x)u
i
= f (x) + g(x)u
and assume that a Lyapunov function V : A R is available for the open-loop system, i.e., V is p.d. at
0 and L
f
V 0. If we dene the dummy output
y = L
g
V(x)
,
then we have L
f
V 0 and L
g
V = h
, and thus by Lemma 5.17 the system with input u and output y is
passive with storage V. If zero-state detectability holds, then a feedback of the form
u = KL
g
V(x), K > 0,
asymptotically stabilizes the origin. The feedback above is very popular in the control literature and has
been discovered many times. Some researchers call it damping controller, researchers in the Russian
literature call it L
g
V-controller. Others call it a Jurdjevic-Quinn controller. We conclude this chapter,
5.8 DAMPING CONTROL 123
and these notes, with sufcient conditions for zero-state detectability. We need the following notation
for iterated Lie derivatives
L
0
f
L
g
i
V := L
g
i
V, i = 1, . . . , m
L
k
f
L
g
i
V := L
f
(L
k1
f
L
g
i
V), k = 0, 1, . . . , i = 1, . . . , m.
Theorem 5.30. Suppose that, for some k
1, the iterated Lie derivatives L

k
f
V, L
g
L
k
f
V, k = 0, . . . , k
exist and
are continuous. If
S = x A : L
f
V(x) = 0, L
k
f
L
g
i
V(x) = 0, i = 1, . . . , m, k = 0, . . . , k
= 0,
then the system with input u and output y = L
g
V(x)
is locally zero-state detectable. Moreover, if V is proper

then the system is zero-state detectable.
Proof. Since V is p.d. at 0, x = 0 is a stable equilibrium of the open-loop system, so there exists > 0 such
that, for all x
0
B
n
(0), the solution x(t) is bounded and L

+
(x
0
) is nonempty. Following Denition 5.21,
set u(t) 0 and let x
0
B
n
(0) be such that y(t) = L

g
V(x(t))
0 (where x(t) = (t, x

0
)). We must
have
0
d
dt
L
g
V(x(t)) = d(L
g
V)
x(t)
x(t) = d(L
g
V)
x(t)
f (x(t)) = L
f
L
g
V(x(t)),
and, in particular, L
f
L
g
V(x
0
) = 0. Taking further time derivatives, we reach the conclusion that
(t 0)(t, x
0
)

S = x A : L
k
f
L
g
i
V(x) = 0, i = 1, . . . , m, k = 0, . . . , k
.
In particular, x
0

S. Since all points in L
+
(x
0
) are limits of sequences (t
i
, x
0
) contained in

S, and
since

S is a closed set, we have L
+
(x
0
)

S. Moreover, by the arguments presented in part (3) of the
proof of Theorem 4.19, L
+
(x
0
) x A : L
f
V(x) = 0. We have thus obtained that L
+
(x
0
) S = 0,
and so (t, x
0
) 0, proving local zero-state detectability. If V is proper, then all solutions of the
open-loop system are bounded, and can be chosen arbitrarily large, so in this case we have zero-state
detectability.
BIBLIOGRAPHY 125
Bibliography
[Arn73] V. I. Arnold. Ordinary differential equations. MIT Press, 1973. Translated from the Russian by
R. A. Silverman.
[Art83] Zvi Artstein. Stabilization with relaxed controls. Nonlinear Analysis: Theory, Methods & Appli-
cations, 7(11):11631173, 1983.
[Bir27] G. D. Birkhoff. Dynamical Systems. American Mathematical Society Colloquium Publications,
1927.
[BIW91] C. Byrnes, A. Isidori, and J. C. Willems. Passivity, feedback equivalence, and the global
stabilization of nonlinear systems. IEEE Transactions on Automatic Control, 36:12281240, 1991.
[Bre70] H. Brezis. On a characterization of ow invariant sets. Communications On Pure & Applied
Mathematics, 23:261263, 1970.
[Bro83] R. W. Brockett. Asymptotic stability and feedback stabilization. In R. S. Millman R. W. Brockett
and H. Sussmann, editors, Differential Geometric Control Theory, pages 181191. Birkhauser,
1983.
[Bro06] M. E. Broucke. Linear differential equations, course notes for ECE557, 2006. Available in the
handouts section on the course website.
[DVM02] I.C. Dolcetta, S.F. Vita, and R. March. Area preserving curve shortening ows: from phase
transitions to image processing. Interfaces Free Boundaries, 4(4):325343, 2002.
[Fra03] B. A. Francis. Linear algebra, course notes for ECE1619, 2003. Available in the handouts
section on the course website.
[GH83] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of
Vector Fields. Springer-Verlag, 1983.
[Gra87] M. Grayson. The heat equation shrinks embedded plane curves to round points. Journal of
Differential Geometry, 26(2):285314, 1987.
[Hal80] J. K. Hale. Ordinary differential equations. Robert E. Krieger Publishing Company, second
edition, 1980.
126 BIBLIOGRAPHY
[Har02] P. Hartman. Ordinary differential equations. SIAM Classics in Applied Mathematics, second
edition, 2002.
[HS74] M. W. Hirsch and S. Smale. Differential equations, Dynamical Systems, and Linear Algebra. Aca-
demic Press, 1974.
[KF70] A. N. Kolmogorov and S. V. Fomin. Introductory Real Analysis. Dover Publications, 1970.
Translated from the Russian by R. A. Silverman.
[Kha02] H. K. Khalil. Nonlinear Systems. Prentice Hall, third edition, 2002.
[Kur63] J. Kurzweil. On the inversion of Lyapunovs second theorem on stability of motion. American
Mathematical Society Translations, Series II, 24:1977, 1963. Originally published in Czechoslovak
Math. J. in 1956.
[Mas56] J.L. Massera. Contributions to stability theory. Annals of Mathematics, 64:182206, 1956.
[Nag42] M. Nagumo. ber die Lage der Integralkurven gewhnlicher Differentialgleichungen. In
Proceedings of the Physico-Mathematical Society of Japan, volume 24, pages 551559, 1942.
[Pet66] I. G. Petrovski. Ordinary Differential Equations. Prentice Hall, 1966. Translated from the Russian
by R. A. Silverman.
[Roy88] H. L. Royden. Real Analysis. Prentice Hall, third edition, 1988.
[Rud76] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, third edition, 1976.
[SBF07] S.L. Smith, M.E. Broucke, and B.A. Francis. Curve shortening and the rendezvous problem
for mobile autonomous robots. IEEE Transactions on Automatic Control, 52(6):11541159, 2007.
[Son89] Eduardo D. Sontag. A "universal" construction of Artsteins theorem on nonlinear stabiliza-
tion. Systems & Control Letters, 13(2):117123, 1989.
[Wil72a] J. C. Willems. Dissipative dynamical systems - Part I: General theory. Arch. of Rational Mechan-
ics and Analysis, 45:321351, 1972.
[Wil72b] J. C. Willems. Dissipative dynamical systems - Part II: Linear systems with quadratic supply
rates. Arch. of Rational Mechanics and Analysis, 45:352393, 1972.

ECE1647 NonlinearControl

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ECE1647 NonlinearControl

Uploaded by

Copyright:

Available Formats

Introduction to Nonlinear Control Systems

(p) Open ball of radius centred at p R

(p) Closed ball of radius centred at p R

S The closure of the set S 8

(x) The Bouligand tangent cone to at x 50

The restriction of the function f to a set 2

denotes the restriction of f to , so we have

y. The vector space R

(x y) 0. Then, use the fact that x

Qx or the associated symmetric matrix Q are said to be

(p). Remarkably, this

(p), there exist two neighborhoods dened using | |,

(s) is tangent to the curve at x(s), N(s) is the

(s)| by counter-clockwise rotation by /2.

(s) ,= 0. The evolution of the three processes is deterministic in that

denote the largest

= I. If not, then there exists t

) are in fact the same.

). By existence and uniqueness (Theorem 2.20), (t, x

> b. Consider the case

> b (the other case is similar). Take

). Since the interval [0,

A is an equilibrium point of (3.1) (or an equilibrium) if x(0) = x

) = 0. Equilibria are classied, according to their stability properties, as stable,

be diverging time sequences such that t

does not belong to either A or B, a contradiction.

0 for all small t. If x , , then

(x) = because |x + tv|

> > 0 for all small t. So T

(x) is nontrivial only when x . In this

= 0, and the quantity liminf

/t is the lower Dini derivative

at t = 0. This derivative generalizes the directional derivative of Denition 1.33

to non-differentiable functions, and it is always well-dened. Roughly

illustrated on the right-hand side,

/t = 0. To apply the denition of T

(x) for all x , if and

be a unit vector orthogonal to f (p). For some > 0, let S = x R

(p), implying that f is transversal to v on B

(p). Picking (0, ) we

(p), implying that S is a local section of f at p.

(p) are local sections of f at p. Dene the closed set V B

(p) whose boundary is

(p). By construction, V is a owbox at p.

> 0 such that, for all c (0, E

), is closed (and hence, by part 2, compact) and invariant.

> 0 be such that

around is small enough that its intersection with S

for all t 0. Pick > 0 small enough that

, which gives |(t, x

for all t 0. Hence, in particular,

< for all t [0, 2T)

can be made smaller in order to guarantee

(p) W. Since p is asymptotically stable, there exists

> 0 such that if x

0 such that (t, x

[0, 2T). By the way W was dened, the

< . Now let > 0 be small enough that x : |x|

< for all t 0.

0 as t +. This result follows from the continuity of , the

), we have that y = 0 is an equilibrium of y =

_. The eigenvalues are

,= 0, then the origin of the subspaces E