Lyapunov Operator

Let A F
nn
be given, and dene a linear operator L
A
: C
nn
C
nn
as
L
A
(X) := A

X + XA
Suppose A is diagonalizable (what follows can be generalized even if this is
not possible - the qualitative results still hold). Let
1
,
2
, . . . ,
n
be the
eigenvalues of A. In general, these are complex numbers. Let V C
nn
be
the matrix of eigenvectors for A

, so
V = [v
1
v
2
v
n
] C
nn
is invertible, and for each i
A

v
i
=

i
v
i
Let w
i
C
n
be dened so that
V
1
=
_

_
w
T
1
w
T
2
.
.
.
w
T
n
_

_
Since V
1
V = I
n
, we have that
w
T
i
v
j
=
ij
:=
_

_
0 if i ,= j
1 if i = j
For each 1 i, j n dene
X
ij
:= v
i
v

j
C
nn
130
Eigenvalues Lyapunov Operator
Lemma 63 The set X
ij

i=1,.,n;j=1,.,n
is a linearly independent set, and this
set is a full set of eigenvectors for the linear operator L
A
. Moreover, the
eigenvalues of L
A
is the set of complex numbers
_

i
+
j
_
i=1,.,n;j=1,.,n
Proof: Suppose
ij

i=1,.,n;j=1,.,n
are scalars and
n

j=1
n

i=1

ij
X
ij
= 0
nn
(7.19)
Premultiply this by w
T
l
and postmultiply by w
k
0 = w
T
l
_
_
n

j=1
n

i=1

ij
X
ij
_
_
w
k
=
n

j=1
n

i=1

ij
w
T
l
v
i
v

j
w
k
=
n

j=1
n

i=1

ij

li

kj
=
lk
which holds for any 1 k, l n. Hence, all of the s are 0, and the set
X
ij

i=1,.,n;j=1,.,n
is indeed a linearly independent set. Also, by denition
of L
A
, we have
L
A
(X
ij
) = A

X
ij
+X
ij
A
= A

v
i
v

j
+v
i
v

j
A
=

i
v
i
v

j
+ v
i
_

j
v

j
_
=
_

i
+
j
_
v
i
v

j
=
_

i
+
j
_
X
ij
So, X
ij
is indeed an eigenvector of L
A
with eigenvalue
_

i
+
j
_
.
131
Lyapunov Operator Alternate Invertibility Proof
Do a Schur decomposition of A, namely A = QQ

, with Q unitary and

upper triangular. Then the equation A

X +XA = R can be written as

A

X +XA = R Q

X +XQQ

= R

XQ +Q

XQ = Q

RQ

X +

X =

R
Since Q is invertible, the redenition of variables is an invertible transfor-
mation.
Writing out all of the equations (we are trying to get that for every

R, there
is a unique

X satisfying the equation) yields a upper triangular system with
n
2
unknowns, and the values
i
+

j
on the diagonal. Do it.
This is an easy (and computationally motivated) proof that the Lyapunov
operator L
A
is an invertible operator if and only if
i
+

j
,= 0 for all eigen-
values of A.
132
Lyapunov Operator Integral Formula
If A is stable (all eigenvalues have negative real parts) then
_

i
+
j
_
,= 0
for all combinations. Hence L
A
is an invertible linear operator. We can
explicitly solve the equation
L
A
(X) = Q
for X using an integral formula.
Theorem 64 Let A F
nn
be given, and suppose that A is stable. The L
A
is invertible, and for any Q F
nn
, the unique X F
nn
solving L
A
(X) =
Q is given by
X =
_

0
e
A

Qe
A
d
Proof: For each t 0, dene
S(t) :=
_
t
0
e
A

Qe
A
d
Since A is stable, the integrand is made up of decaying exponentials, and
S(t) has a well-dened limit as t , which we have already denoted
X := lim
t
S(t). For any t 0,
A

S(t) +S(t)A =
_
t
0
_
A

e
A

Qe
A
+ e
A

Qe
A
A
_
d
=
_
t
0
d
d
_
e
A

Qe
A
_
d
= e
A

t
Qe
At
Q
Taking limits on both sides gives
A

X + XA = Q
as desired.
133
Lyapunov Operator More
A few other useful facts
Theorem 65 Suppose A, Q F
nn
are given. Assume that A is stable and
Q = Q

0. Then the pair (A, Q) is observable if and only if

X :=
_

0
e
A

Qe
A
d > 0
Proof: Suppose not. Then there is an x
o
F
n
, x
o
,= 0 such that
Qe
At
x
o
= 0
for all t 0. Consequently x

o
e
A

t
Qe
At
x
o
= 0 for all t 0. Integrat-
ing gives
0 =
_

0
_
x

o
e
A

t
Qe
At
x
o
_
dt
= x

o
_
_

0
e
A

t
Qe
At
dt
_
x
o
= x

o
Xx
o
Since x
o
,= 0, X is not positive denite.
Suppose that X is not positive denite (note by virtue of the deni-
tion X is at least positive semidenite). Then there exists x
o
F
n
,
x
o
,= 0 such that x

o
Xx
o
= 0. Using the integral form for X (since
A is stable by assumption) and the fact tha Q 0 gives
_

0
_
_
_
_
Q
1
2
e
A
x
o
_
_
_
_
d = 0
The integrand is nonnegative and continuous, hence it must be 0
for all 0. Hence Qe
A
x
o
= 0 for all 0. Since x
o
,= 0, this
implies that (A, Q) is not observable.
134
Lyapunov Operator More
Theorem 66 Let (A, C) be detectable and suppose X is any solution to
A

X +XA = C

C (there is no apriori assumption that L

A
is an invertible
operator, hence there could be multiple solutions). Then X 0 if and only
if A is stable.
Proof Suppose not, then there is a v C
n
, v ,= 0, C, Re () 0
with Av = v (and v

=

v

). Since (A, C) is detectable and the

eigenvalue in the closed right-half plane Cv ,= 0. Note that
|Cv|
2
= v

Cv
= v

(A

X + XA) v
=
_

+
_
v

Xv
= 2Re () v

Xv
Since |Cv| > 0 and Re () 0, we must actually have Re () > 0
and v

Xv < 0. Hence X is not positive semidenite.

Since A is stable, there is only one solution to the equation A

X +
XA = C

C, moreover it is given by the integral formula,

X =
_

0
e
A

Ce
A
d
which is clearly positive semidenite.
135
Lyap Ordering
Finally, some simple results about the ordering of Lyapunov solutions.
Let A be stable, so L
A
is guaranteed to be invertible. Use L
1
A
(Q) to
denote the unique solution X to the equation A

X +XA = Q.
Lemma 67 If Q

1
= Q
1
Q
2
= Q

2
, then
L
1
A
(Q
1
) L
1
A
(Q
2
)
Lemma 68 If Q

1
= Q
1
> Q
2
= Q

2
, then
L
1
A
(Q
1
) > L
1
A
(Q
2
)
Proofs Let X
1
and X
2
be the two solutions, obviously
A

X
1
+X
1
A = Q
1
A

X
2
+X
2
A = Q
2
A

(X
1
X
2
) + (X
1
X
2
) A = (Q
1
Q
2
)
Since A is stable, the integral formula applies, and
X
1
X
2
=
_

0
e
A

(Q
1
Q
2
) e
A
d
Obviously, if Q
1
Q
2
0, then X
1
X
2
0, which proves Lemma 5. If
Q
1
Q
2
> 0, then the pair (A, Q
1
Q
2
) is observable, and by Theorem
3, X
1
X
2
> 0, proving Lemma 6.
Warning: The converses of Lemma 5 and Lemma 6 are NOT true.
136
Algebraic Riccati Equation
The algebraic Riccati equation (A.R.E.) is
A
T
X + XA + XRX Q = 0 (7.20)
where A, R, Q R
nn
are given matrices, with R = R
T
, and Q = Q
T
, and
solutions X C
nn
are sought. We will primarily be interested in solutions
X of (7.20), which satisfy
A+ RX is Hurwitz
and are called stabilizing solutions of the Riccati equation.
Associated with the data A, R, and Q, we dene the Hamiltonian matrix
H R
2n2n
by
H :=
_

_
A R
Q A
T
_

_ (7.21)
137
Invariant Subspace Denition
Suppose A C
nn
. Then (via matrix-vector multiplication), we view A as
a linear map from C
n
C
n
.
A subspace 1 C
n
is said to be A-invariant if for every vector v 1, the
vector Av 1 too.
In terms of matrices, suppose that the columns of W C
nm
are a basis for
1. In other words, they are linearly independent, and span the space 1, so
1 = W : C
m

Then 1 being A-invariant is equivalent to the existence of a matrix S C

mm
such that
AW = WS
The matrix S is the matrix representation of A restricted to 1, referred to
the basis W. Clearly, since W has linearly independent columns, the matrix
S is unique.
If (Q, ) are the Schur decomposition of A, then for any k with 1 k n,
the rst k columns of Q span a k-dimensional invariant subspace of A, as
seen below
A[Q
1
Q
2
] = [Q
1
Q
2
]
_

11

12
0
22
_

_
Clearly, AQ
1
= Q
1

11
.
138
Eigenvectors in Invariant subspaces
Note that since S C
mm
has at least one eigenvector, say ,= 0
m
, with
associated eigenvalue , it follows that W 1 is an eigenvector of A,
A(W) = AW = WS = W = (W)
Hence, every nontrivial invariant subspace of A contains an eigen-
vector of A.
139
Stable Invariant Subspace Denition
If S is Hurwitz, then 1 is a stable, A-invariant subspace.
Note that while the matrix S depends on the particular choice of the basis for
1 (ie., the specic columns of W), the eigenvalues of S are only dependent
on the subspace 1, not the basis choice.
To see this, suppose that the columns of

W C
nm
span the same space as
the columns of W.
Obviously, there is a matrix T C
mm
such that W =

WT. The linear
independence of the columns of W (and

W) imply that T is invertible. Also,
A

W = AWT
1
= WST
1
=

W TST
1
. .

S
Clearly, the eigenvalues of S and

S are the same.
The linear operator A[
1
is the transformation from 1 1 dened as A,
simply restricted to 1.
Since 1 is A-invariant, this is well-dened. Clearly, S is the matrix repre-
sentation of this operator, using the columns of W as the basis choice.
140
A
T
X + XA +XRX Q = 0 Invariant Subspace
The rst theorem gives a way of constructing solutions to (7.20) in terms of
invariant subspaces of H.
Theorem 69 Let 1 C
2n
be a n-dimensional invariant subspace of H (H
is a linear map from C
2n
to C
2n
), and let X
1
C
nn
, X
2
C
nn
be two
complex matrices such that
1 = span
_

_
X
1
X
2
_

_
If X
1
is invertible, then X := X
2
X
1
1
is a solution to the algebraic Riccati
equation, and spec (A+ RX) = spec (H[
1
).
Proof: Since 1 is H invariant, there is a matrix C
nn
with
_

_
A R
Q A
T
_

_
_

_
X
1
X
2
_

_ =
_

_
X
1
X
2
_

_ (7.22)
which gives
AX
1
+ RX
2
= X
1
(7.23)
and
QX
1
A
T
X
2
= X
2
(7.24)
Pre and post multiplying equation (7.23) by X
2
X
1
1
and X
1
1
respec-
tively gives
_
X
2
X
1
1
_
A+
_
X
2
X
1
1
_
R
_
X
2
X
1
1
_
= X
2
X
1
1
(7.25)
and postmultiplying 7.24 by X
1
1
gives
QA
T
_
X
2
X
1
1
_
= X
2
X
1
1
(7.26)
Combining (7.25) and (7.26) yields
A
T
(X
2
X
1
1
) + (X
2
X
1
1
)A+ (X
2
X
1
1
)R(X
2
X
1
1
) Q = 0
so X
2
X
1
1
is indeed a solution. Equation (7.23) implies that A+R
_
X
2
X
1
1
_
=
X
1
X
1
1
, and hence they have the same eigenvalues. But, by denition,
is a matrix representation of the map H[
1
, so the theorem is proved.

141
A
T
X + XA +XRX Q = 0 Invariant Subspace
The next theorem shows that the specic choice of matrices X
1
and X
2
which
span 1 is not important.
Theorem 70 Let 1, X
1
, and X
2
be as above, and suppose there are two
other matrices

X
1
,

X
2
C
nn
such that the columns of
_

X
1

X
2
_

_ also span
1. Then X
1
is invertible if and only if

X
1
is invertible. Consequently, both
X
2
X
1
1
and

X
2

X
1
1
are solutions to (7.20), and in fact they are equal.
Remark: Hence, the question of whether or not X
1
is invertible is a property
of the subspace, and not of the particular basis choice for the subspace.
Therefore, we say that an n-dimensional subspace 1 has the invertibility
property if in any basis, the top portion is invertible. In the literature,
this is often called the complimentary property.
Proof: Since both sets of columns span the same n-dimensional subspace,
there is an invertible K C
nn
such that
_

_
X
1
X
2
_

_ =
_

X
1

X
2
_

_ K
Obviously, X
1
is invertible if and only if

X
1
is invertible, and X
2
X
1
1
=
_

X
2
K
_ _

X
1
K
_
1
=

X
2

X
1
1
. .
142
A
T
X + XA +XRX Q = 0 Invariant Subspace
As we would hope, the converse of theorem 69 is true.
Theorem 71 If X C
nn
is a solution to the A.R.E., then there exist
matrices X
1
, X
2
C
nn
, with X
1
invertible, such that X = X
2
X
1
1
and the
columns of
_

_
X
1
X
2
_

_ span an n-dimensional invariant subspace of H.

Proof: Dene := A + RX C
nn
. Multiplying this by X gives X =
XA+XRX = QA
T
X, the second equality coming from the fact that
X is a solution to the Riccati equation. We write these two relations as
_

_
A R
Q A
T
_

_
_

_
I
n
X
_

_ =
_

_
I
n
X
_

_
Hence, the columns of
_

_
I
n
X
_

_ span an n-dimensional invariant subspace

of H, and dening X
1
:= I
n
, and X
2
:= X completes the proof. .
Hence, there is a one-to-one correspondence between n-dimensional invari-
ants subspaces of H with the invertibility property, and solutions of the
Riccati equation.
143
A
T
X + XA +XRX Q = 0 Eigenvalue distribution of H
In the previous section, we showed that any solution X of the Riccati equa-
tion is intimately related to an invariant subspace, 1, of H, and the matrix
A + RX is similiar (ie. related by a similarity transformation) to the re-
striction of H on 1, H[
1
. Hence, the eigenvalues of A + RX will always
be a subset of the eigenvalues of H. Recall the particular structure of the
Hamiltonian matrix H,
H :=
_

_
A R
Q A
T
_

_
Since H is real, we know that the eigenvalues of H are symmetric about the
real axis, including multiplicities. The added structure of H gives additional
symmetry in the eigenvalues as follows.
Lemma 72 The eigenvalues of H are symmetric about the origin (and hence
also about the imaginary axis).
Proof: Dene J R
2n2n
by
J =
_

_
0 I
I 0
_

_
Note that JHJ
1
= H
T
. Also, JH = H
T
J = (JH)
T
. Let p () be
the characteristic polynomial of H. Then
p() := det (I H)
= det
_
I JHJ
1
_
= det
_
I + H
T
_
= det (I + H)
= (1)
2n
det (I H)
= p()
(7.27)
Hence the roots of p are symmetric about the origin, and the symmetry
about the imaginary axis follows.
144
A
T
X + XA +XRX Q = 0 Hermitian Solutions
Lemma 73 If H has no eigenvalues on the imaginary axis, then H has n
eigenvalues in the open-right-half plane, and n eigenvalues in the open-left-
half plane.
Theorem 74 Suppose 1 is a n-dimensional invariant subspace of H, and
X
1
, X
2
C
nn
form a matrix whose colums span 1. Let C
nn
be a
matrix representation of H[
1
. If
i
+
j
,= 0 for all eigenvalues
i
,
j

spec(), then X

1
X
2
= X

2
X
1
.
Proof: Using the matrix J from the proof of Lemma 72 we have that
[X

1
X

2
] JH
_

_
X
1
X
2
_

_
is Hermitian, and equals (X

1
X
2
+X

2
X
1
) . Dene W := X

1
X
2
+
X

2
X
1
. Then W = W

, and W +

W = 0. By the eigenvalue
assumption on , the linear map L

: C
nn
C
nn
dened by
L

(X) = X +

X
is invertible, so W = 0.
Corollary 75 Under the same assumptions as in theorem 74, if X
1
is in-
vertible, then X
2
X
1
1
is Hermitian.
145
A
T
X + XA +XRX Q = 0 Real Solutions
Real (rather than complex) solutions arise when a symmetry condition is
imposed on the invariant subspace.
Theorem 76 Let 1 be a n-dimensional invariant subspace of H with the
invertibility property, and let X
1
, X
2
C
nn
form a 2n n matrix whose
columns span 1. Then 1 is conjugate symmetric ( v : v 1 = 1) if and
only if X
2
X
1
1
R
nn
.
Proof: Dene X := X
2
X
1
1
. By assumption, X R
nn
, and
span
_

_
I
n
X
_

_ = 1
so 1 is conjugate symmetric.
Since 1 is conjugate symmetric, there is an invertible matrix K
C
nn
such that
_

_
X
1
X
2
_

_ =
_

_
X
1
X
2
_

_ K
Therefore, (X
2
X
1
1
) = X
2
X
1
1
= X
2
K(X
1
K)
1
= X
2
X
1
1
as de-
sired.
146
A
T
X + XA +XRX Q = 0 Stabilizing Solutions
Recall that for any square matrix M, there is a largest stable invariant
subspace. That is, there is a stable invariant subspace 1

S
that contains every
other stable invariant subspace of M. This largest stable invariant subspace
is simply the span of all of the eigenvectors and generalized eigenvectors
associated with the open-left-half plane eigenvalues. If the Schur form has
the eigenvalues (on diagonal of ) ordered, then it is spanned by the rst
set of columns of Q
Theorem 77 There is at most one solution X to the Riccati equation such
that A+RX is Hurwitz, and if it does exist, then X is real, and symmetric.
Proof: If X is such a stabilizing solution, then by theorem 71 it can be
constructed from some n-dimensional invariant subspace of H, 1. If
A + RX is stable, then we must also have spec (H[
1
). Recall that the
spectrum of H is symmetric about the imaginary axis, so H has at
most n eigenvalues in

C
, and the only possible choice for 1 is the
stable eigenspace of H. By theorem 70, every solution (if there are any)
constructed from this subspace will be the same, hence there is at most
1 stabilizing solution.
Associated with the stable eigenspace, any as in equation (7.22) will
be stable. Hence, if the stable eigenspace has the invertibility property,
from corollary 75 we have X := X
2
X
1
1
is Hermitian. Finally, since H
is real, the stable eigenspace of H is indeed conjugate symmetric, so the
associated solution X is in fact real, and symmetric.
147
A
T
X + XA +XRX Q = 0 dom
S
Ric
If H is (7.21) has no imaginary axis eigenvalues, then H has a unique n-
dimensional stable invariant subspace 1
S
C
2n
, and since H is real, there
exist matrices X
1
, X
2
R
nn
such that
span
_

_
X
1
X
2
_

_ = 1
S
.
More concretely, there exists a matrix R
nn
such that
H
_

_
X
1
X
2
_

_ =
_

_
X
1
X
2
_

_ , spec ()

C
This leads to the following denition
Denition 78 (Denition of dom
S
Ric) Consider H R
2n2n
as dened
in (7.21). If H has no imaginary axis eigenvalues, and the matrices X
1
, X
2

R
nn
for which
span
_

_
X
1
X
2
_

_
is the stable, n-dimensional invariant subspace of H satisfy det (X
1
) ,= 0,
then H dom
S
Ric. For H dom
S
Ric, dene Ric
S
(H) := X
2
X
1
1
, which is
the unique matrix X R
nn
such that A+RX is stable, and
A
T
X + XA +XRX Q = 0.
Moreover, X = X
T
.
148
A
T
X + XA +XRX Q = 0 Useful Lemma
Theorem 79 Suppose H (as in (7.21)) does not have any imaginary axis
eigenvalues, R is either positive semidenite, or negative semidenite, and
(A, R) is a stabilizable pair. Then H dom
S
Ric.
Proof: Let
_

_
X
1
X
2
_

_ form a basis for the stable, n-dimensional invariant sub-

space of H (this exists, by virtue of the eigenvalue assumption about
H). Then
_

_
A R
Q A
T
_

_
_

_
X
1
X
2
_

_ =
_

_
X
1
X
2
_

(7.28)
where C
nn
is a stable matrix. Since is stable, it must be that
X

1
X
2
= X

2
X
1
. We simply need to show X
1
is invertible. First we prove
that KerX
1
is -invariant, that is, the implication
x KerX
1
x KerX
1
Let x KerX
1
. The top row of (7.28) gives RX
2
= X
1
AX
1
.
Premultiply this by x

2
to get
x

2
RX
2
x = x

2
(X
1
AX
1
) x
= x

2
X
1
x
= x

1
X
2
x since X

2
X
1
= X

1
X
2
= 0
.
Since R 0 or R 0, it follows that RX
2
x = 0. Now, using (7.28)
again gives X
1
x = 0, as desired. Hence KerX
1
is a subspace that is
-invariant. Now, if KerX
1
,= 0, then there is a vector v ,= 0 and
C, Re () < 0 such that
X
1
v = 0
v = v
RX
2
v = 0
149
We also have QX
1
A
T
X
2
= X
2
, from the second row of (7.28).
Mutiplying by v

gives
v

2
_
A
_

I
_
R
_
= 0
Since Re
_

_
< 0, and (A, R) is assumed stabilizable, it must be that
X
2
v = 0. But X
1
v = 0, and
_

_
X
1
X
2
_

_ is full-column rank. Hence, v =

0, and KerX
1
is indeed the trivial set 0. This implies that X
1
is
invertible, and hence H dom
S
Ric. .
150
A
T
X + XA +XRX Q = 0 Quadratic Regulator
The next theorem is a main result of LQR theory.
Theorem 80 Let A, B, C be given, with (A, B) stabilizable, and (A, C)
detectable. Dene
H :=
_

_
A BB
T
C
T
C A
T
_

_
Then H dom
S
Ric. Moreover, 0 X := Ric
S
(H), and
KerX
_

_
C
CA
.
.
.
CA
n1
_

_
Proof: Use stabilizablity of (A, B) and detectability of (A, C) to show that
H has no imaginary axis eigenvalues. Specically, suppose v
1
, v
2
C
n
and R such that
_

_
A BB
T
C
T
C A
T
_

_
_

_
v
1
v
2
_

_ = j
_

_
v
1
v
2
_

_
Premultiply top and bottom by v

2
and v

1
, combine, and conclude v
1
=
v
2
= 0. Show that if (A, B) is stabilizable, then
_
A, BB
T
_
is also sta-
bilizable. At this point , apply Theorem 79 to conclude H dom
S
Ric.
Then, rearrange the Riccati equation into a Lyapunov equation and
use Theorem 64 to show that X 0. Finally, show that KerX KerC,
and that KerX is A-invariant. That shows that KerX KerCA
j
for
any j 0. .
151
Derivatives Linear Maps
Recall that the derivative of f at x is dened to be the number l
x
such that
lim
0
f(x +) f(x) l
x

= 0
Suppose f : R
n
R
m
is dierentiable. Then f

(x) is the unique matrix

L
x
R
mn
which satises
lim
0
n
1
||
(f(x +) f(x) L
x
) = 0
Hence, while f : R
n
R
m
, the value of f

at a specic location x, f

(x),
is a linear operator from R
n
R
m
. Hence the function f

is a map from
R
n
L(R
n
, R
m
).
Similarly, if f : H
nn
H
nn
is dierentiable, then the derivative of f at x
is the unique linear operator L
X
: H
nn
H
nn
such that satisfying
lim
0
nn
1
||
(f(X + ) f(X) L
X
) = 0
152
Derivative of Riccati Lyapunov
f(X) := XDX A

X XAC
Note,
f(X + ) = (X + ) D(X + ) A

(X + ) (X + ) A C
= XDX A

X XA C + DX + XDA

A+ D
= f(X) (ADX) (ADX)

+ D
For any W C
nn
, let L
W
: H
nn
H
nn
be the Lyapunov operator
L
W
(X) := W

X +XW. Using this notation,

f(X + ) f(X) L
A+DX
() = D
Therefore
lim
0
nn
1
||
[f(X + ) f(X) L
A+DX
()] = lim
0
nn
1
||
[D] = 0
This means that the Lyaponov operator L
A+DX
is the derivative of the
matrix-valued function
XDX A

X XA C
at X.
153
Iterative Algorithms Roots
Suppose f : R R is dierentiable. An iterative algorithm to nd a root
of f(x) = 0 is
x
k+1
= x
k

f(x
k
)
f

(x
k
)
Suppose f : R
n
R
n
is dierentiable. Then to rst order
f(x + ) = f(x) +L
x

An iterative algorithm to nd a root of f(x) = 0

n
is obtained by setting
the rst-order approximation expression to zero, and solving for , yielding
the update law (with x
k+1
= x
k
+)
x
k+1
= x
k
L
1
x
k
[f(x
k
)]
Implicitly, it is written as
L
x
k
[x
k+1
x
k
] = f(x
k
) (7.29)
For the Riccati equation
f(X) := XDX A

X XAC
we have derived that the derivative of f at X involves the Lyapunov operator.
The iteration in (7.29) appears as
L
A+DX
k
(X
k+1
X
k
) = X
k
DX
k
+ A

X
k
+X
k
A+C
In detail, this is
(A DX
k
)

(X
k+1
X
k
)(X
k+1
X
k
) (A DX
k
) = X
k
DX
k
+A

X
k
+X
k
A+C
which simplies (check) to
(ADX
k
)

X
k+1
+ X
k+1
(ADX
k
) = X
k
DX
k
C
Of course, questions about the well-posedness and convergence of the itera-
tion must be addressed in detail.
154
Comparisons Gohberg, Lancaster and Rodman
Theorems 2.1-2.3 in On Hermitian Solutions of the Symmetric Algebraic
Riccati Equation, by Gohberg, Lancaster and Rodman, SIAM Journal of
Control and Optimization, vol. 24, no. 6, Nov. 1986, are the basis for compar-
ing equality and inequality solutions of Riccati equations. The denitions,
theorems and main ideas are below.
Suppose that A R
nn
, C R
nn
, D R
nn
, with D = D
T
_ 0, C = C
T
and (A, D) stabilizable. Consider the matrix equation
XDX A
T
X XA C = 0 (7.30)
Denition 81 A symmetric solution X
+
= X
T
+
of (9.39) is maximal if
X
+
_ X for any other symmetric solution X of (9.39). Check: Maximal
solutions (if they exist) are unique.
Theorem 82 If there is a symmetric solution to (9.39), then there is a
maximal solution X
+
. The maximal solution satises
max
i
Re
i
(ADX
+
) 0
Theorem 83 If there is a symmetric solution to (9.39), then there is a
sequence X
j

j=1
such that for each j, X
j
= X
T
j
and
0 _ X
j
DX
j
A
T
X
j
X
j
AC
X
j
_ X for any X = X
T
solving 9.39
A DX
j
is Hurwitz
lim
j
X
j
= X
+
Theorem 84 If there are symmetric solutions to (9.39), then for every sym-
metric

C _ C, there exists symmetric solutions (and hence a maximal solu-
tion

X
+
) to
XDX A
T
X XA

C = 0
Moreover, the maximal solutions are related by

X
+
_ X
+
.
155
Identities
For any Hermitian matrices Y and

Y , it is trivial to verify
Y (ADY ) + (ADY )

Y + Y DY
= Y (AD

Y ) + (AD

Y )

Y +

Y D

Y
_
Y

Y
_
D
_
Y

Y
_
(7.31)
Let X denote any Hermitian solution (if one exists) to the ARE,
XDX A

X XA C = 0 (7.32)
This (the existence of a hermitian solution) is the departure point for all
that will eventually follow.
For any hermitian matrix Z
C = XDX + A

X + XA
= X (ADX) + (ADX)

X + XDX
= X (ADZ) + (A DZ)

X +ZDZ (X Z) D(X Z)
(7.33)
This exploits that X solves the original Riccati equation, and uses the iden-
tity (7.31), with Y = X,

Y = Z.
156
Iteration
Take any

C _ C (which could be C, for instance). Also, let X
0
= X

0
_ 0
be chosen so that A DX
0
is Hurwitz. Check: Why is this possible?
Iterate (if possible, which we will show is...) as
X
+1
(ADX

) + (ADX

X
+1
= X

DX

C (7.34)
Note that by the Hurwitz assumption on ADX
0
, at least the rst iteration
is possible ( = 0). Also, recall that the iteration is analogous to a rst-order
algorithm for root nding, x
n+1
:= x
n

f(x
n
)
f

(x
n
)
.
Use Z := X

in equation (7.33), and subtract from the iteration equation

(7.34), leaving
(X
+1
X) (ADX

) + (ADX

(X
+1
X)
= (X X

) D(X X

)
_

C C
_ (7.35)
which is valid whenever the iteration equation is valid.
157
Main Induction
Now for the induction: assume X

= X

and A DX

is Hurwitz. Note
that this is true for = 0.
Since ADX

is Hurwitz, there is a unique solution X

+1
to equation (7.34).
Moreover, since ADX

is Hurwitz, and the right-hand-side of (7.35) is _ 0,

it follows that X
+1
X _ 0 (note that X
0
X is not necessarily positive
semidenite).
Rewrite (7.31) with Y = X
+1
and

Y = X

.
X
+1
(ADX
+1
) + (ADX
+1
)

X
+1
+ X
+1
DX
+1
= X
+1
(ADX

) + (ADX

X
+1
+ X

DX

. .

C
(X
+1
X

) D(X
+1
X

)
Rewriting,

C = X
+1
(A DX
+1
) + (ADX
+1
)

X
+1
+ X
+1
DX
+1
+ (X
+1
X

) D(X
+1
X

)
Writing (7.33) with Z = X
+1
gives
C = X (A DX
+1
) + (A DX
+1
)

X
+ X
+1
DX
+1
(X X
+1
) D(X X
+1
)
Subtract these, yielding

_

C C
_
= (X
+1
X) (ADX
+1
) + (A DX
+1
)

(X
+1
X)
+ (X
+1
X

) D(X
+1
X

) + (X X
+1
) D(X X
+1
)
(7.36)
Now suppose that Re () 0, and v C
n
such that (ADX
+1
) v = v.
Pre and post-multiply (7.36) by v

and v respectively. This gives

v

_

C C
_
v
. .
0
= 2Re()
. .
0
v

(X
+1
X) v
. .
0
+
_
_
_
_
_
_
_
_

_
D
1/2
(X
+1
X

)
D
1/2
(X X
+1
)
_

_ v
_
_
_
_
_
_
_
2
. .
0
Hence, both sides are in fact equal to zero, and therefore
v

(X
+1
X

) D(X
+1
X

) = 0.
158
Since D _ 0, it follows that D(X
+1
X

) v = 0, which means DX
+1
v =
DX

v. Hence
v = (A DX
+1
) v = (ADX

) v
But A DX

is Hurwitz, and Re() 0, so v = 0

n
as desired. This means
ADX
+1
is Hurwitz.
Summarizing:
Suppose there exists a hermitian X = X

solving
XDX A

X XA C = 0
Let X
0
= X

0
_ 0 be chosen so ADX
0
is Hurwitz
Let

C be a Hermitian matrix with

C _ C.
Iteratively dene (if possible) a sequence X

=1
via
X
+1
(ADX

) + (ADX

X
+1
= X

DX

C
The following is true:
Given the choice of X
0
and

C, the sequence X

=1
is well-dened,
unique, and has X

= X

.
For each 0, A DX

is Hurwitz
For each 1, X

_ X.
It remains to show that for 1, X
+1
_ X

(although it is not necessarily

true that X
1
_ X
0
).
159
Decreasing Proof
For 1, apply (7.31) with Y = X

and

Y = X
1
, and substitute the
iteration (shifted back one step) equation (7.34) giving
X

(ADX

) + (ADX

+ X

DX

C (X

X
1
) D(X

X
1
)
The iteration equation is
X
+1
(A DX

) + (ADX

X
+1
= X

DX

C
Subtracting leaves
(X

X
+1
) (ADX

) + (ADX

(X

X
+1
)
= (X

X
1
) D(X

X
1
)
The fact that ADX

is Hurwitz and D _ 0 implies that X

X
+1
_ 0.
Hence, we have shown that for all 1,
X _ X
+1
_ X

160
Monotonic Matrix Sequences Convergence
Every nonincreasing sequence of real numbers which is bounded below has
a limit. Specically, if x
k

k=0
is a sequence of real numbers, and there is a
real number such that x
k
and x
k+1
x
k
for all k, then there is a real
number x such that
lim
k
x
k
= x
A similar result holds for symmetric matrices: Specically, if X
k

k=0
is a
sequence of real, symmetric matrices, and there is a real symmetric matrix
such that X
k
_ and X
k+1
_ X
k
for all k, then there is a real symmetric
matrix X such that
lim
k
X
k
= X
This is proven by rst considering the sequence
_
e
T
i
X
k
e
i
_

k=0
, which shows
that all of the diagonal entries of the sequence have limits. Then look at
_
(e
i
+ e
j
)
T
X
k
(e
i
+ e
j
)
_

k=0
to conclude convergence of o-diagonal terms.
161
Limit of Sequence Maximal Solution
The sequence X

=0
, generated by (7.34), has a limit
lim

=:

X
+
Remember that

C was any hermitian matrix _ C.
Facts:
Since each A DX

is Hurwitz, in passing to the limit we may get

eigenvalues with zero real-parts, hence
max
i
Re
i
(ADX
+
) 0
Since each X

= X

, it follows that

X

+
=

X
+
.
Since each X

_ X (where X is any hermitian solution to the equality

XDX A

X XA C = 0, equation (7.32)), it follows by passing to

the limit that

X
+
_ X.
The iteration equation is
X
+1
(ADX

) + (ADX

X
+1
= X

DX

C
Taking limits yields

X
+
_
A D

X
+
_
+
_
AD

X
+
_

X
+
=

X
+
D

X
+

C
which has a few cancellations, leaving

X
+
A+A

X
+

X
+
D

X
+
+

C = 0
Finally, if X
+
denotes the above limit when

C = C, it follows that the
above ideas hold, so X
+
_ X, for all solutions X.
And really nally, since X
+
is a hermitian solution to (7.32) with C, the
properties of

X
+
imply that

X
+
_ X
+
.
162
Analogies with Scalar Case dx
2
2ax c = 0
Matrix Statement Scalar Specialization
D = D

_ 0 d R, with d 0
(A, D) stabilizable a < 0 or d > 0
existence of Hermitian solution a
2
+ cd 0
Maximal solution X
+
x
+
=
a+

a
2
+cd
d
for d > 0
Maximal solution X
+
x
+
=
c
2a
for d = 0
ADX
+

a
2
+cd for d > 0
ADX
+
a for d = 0
ADX
0
Hurwitz x
0
>
a
d
for d > 0
ADX
0
Hurwitz a < 0 for d = 0
ADX Hurwitz x >
a
d
for d > 0
ADX Hurwitz a < 0 for d = 0
If d = 0, then the graph of 2ax c (with a < 0) is a straight line with a
positive slope. There is one (and only one) real solution at x =
c
2a
.
If d > 0, then the graph of dx
2
2ax c is a upward directed parabola.
The minimum occurs at
a
d
. There are either 0, 1, or 2 real solutions to
dx
2
2ax c = 0.
If a
2
+ cd < 0, then there are no solutions (the minimum is positive)
If a
2
+ cd = 0, then there is one solution, at x =
a
d
, which is where the
minimum occurs.
If a
2
+ cd > 0, then there are two solutions, at x

=
a
d

a
2
+cd
d
and
x
+
=
a
d
+

a
2
+cd
d
which are equidistant from the minimum
a
d
.
163