Optimal Control (Unit 3/CY/E0)

Lecturer: Dr. V.M. Becerra, Cybernetics 178.

E-mail: v.m.becerra@reading.ac.uk
Oce hours: Thursdays and Fridays 2-5pm.
Contents

Introduction to static optimisation. Dynamic optimisation. Discrete time systems. Continuous time systems. Open nal time.
Pontryagin's minimum principle. Tracking optimal control. LQG
optimal control. Kalman ltering. Predictive control. Numerical
methods.

Lecture notes, etc

http://www.rdg.ac.uk/~shs99vmb/notes

Recommended books

Lewis F.L. and Syrmos V.L. (1995) Optimal Control. Second Edition. Wiley.

Linear Optimal Control. Addison-Wesley.

Bryson A.E. (2000). Dynamic Optimization. Addison-Wesley.

Burl J.B. (2000).

Assessment

Exam (100%)

Introduction

What is optimal control?

Optimal control is the process of nding control and state histories for a dynamic system
over a period of time to minimise a performance index.
Optimal control is used in many elds, for example:

To determine ecient maneuvers of aircraft, spacecraft and robots.

To design feedback controllers that are optimal in some sense.

2

Introduction to static optimization

In order to learn the principles of optimal control, it is important to understand the theory
of static optimisation.
Unconstrained optimization

A simple type of static optimization problem is

to nd the values of n variables x1; x2;


; xn
to minimise a function of these variables:
L(x1; x2;


; xn)

(1)

For convenience, vector notation is used, so

let:
2
6
6
6
x = 66
6
4

x1
x2
x3

..

xn

3
7
7
7
7
7
7
5

(2)

then the optimisation problem is written as follows:

min
L(x)
x

(3)

Assuming that function L has rst and second

partial derivatives, then using Taylor series the
function can be approximated about x:
1
L(x)  L(
x)+Lx(x x)+ (x x)Lxx(x x) (4)
2
where the partial derivatives Lx = @L=@x (row
vector) and Lxx = @ 2L=@x2 (a n  n matrix
called the Hessian) are computed at x.
The values at which Lx = 0 are called stationary points.

From the approximation, we can infer that the

necessary conditions for a minimum are:
Lx = 0
Lxx  0

(5)

whereas sucient conditions for a minimum

are:
Lx = 0
(6)
Lxx > 0
When a matrix satises the relation > 0 ( 0)
we say that the matrix is positive denite (positive semi{denite). A positive denite matrix
has positive eigenvalues.
6

Example 1.1

"

1
11 Q12
L(x) = [x1 x2] Q
Q21 Q22
2

#"

"

x1 +[b b ] x1
1 2
x2
x2

(7)

Or, in matrix-vector notation:

1 T
L(x) = x Qx + bT x
2

(8)

A stationary point is given by:

Lx = Qx + b = 0 ) x = Q 1b

(9)
7

The type of stationary point depends on the

Hessian Lxx = Q

If Lxx > 0 we have a minimum.

If Lxx < 0 we have a maximum.

If Lxx  or Lxx  0 we have a singular

point. We cannot say if the stationary
point is a minimum or a maximum.

If Lxx has both positive and negative eigenvalues, then we have a saddle point.

Optimisation with equality constraints

A more general class of static optimisation problem involves a set of p constraint relations:
f (i)(x) = 0; i = 1;


; p

(10)

where p < n.
Using vector notation, let
2

f (1)

f = 64 .. 75
f (p)

(11)

Then the static optimisation problem can be

written as follows:
min
L(x)
x

(12)

f (x) = 0

(13)

subject to

10

Suppose we have a feasible x, such that f (x) =

0. We wish to analyse the conditions for which
x provides a minimum for L(x).
We look for an innitesimal change in x that
will decrease L while keeping f = 0 to rst
order in dx:
dL = Lx dx < 0
df = fx dx = 0

(14)

where fx = @f=@x is a p  n matrix known as

the Jacobian. The rows of fx are the gradients
of f (i) with respect to x, i = 1;


; n. These
gradients are perpendicular to the curve f = 0.
At a minimum, Lx must have no component
parallel to the curve f = 0; otherwise a dx
could be found to make dL < 0 while keeping
df = 0. Then Lx must be perpendicular to
f = 0 at a minimum.
11

Then, Lx must be a linear combination of the

n constraint gradients fx(i):

Lx =

n
X
i=1

ifx(i)

(15)

where i is a set of constants known as the

Lagrange multipliers. In vector notation:

where

Lx = T fx

(16)

T = [1;


; n]

(17)

12

Then, the necessary conditions for a stationary

point are:
f (x) = 0

(18)

Lx + T fx = 0

(19)

which are n + p equations for the n + p unknowns x and .

Dene a function H as follows:

H = L + T f

(20)

Then, optimality condition (19) can be written

as follows:
Hx = 0

(21)
13

Example 1.2
2
2
min
L
=
x
+
x
1
2
x

subject to:

2x1 + x2 + 4 = 0
Solution

H = x21 + x22 + (2x1 + x2 + 4)

Optimality conditions:
f (x) = 0
Hx = 0

) (2x1 + x2 + 4 = 0
2 = 0
) 22xx12 +
+ =0

Solving for the unknowns results in x1 = 1:6,

x2 = 0:8, and  = 1:6
14

8
7
6
5
4
3
L(x) = x21 + x22

2
1
0
2

2x1+x2+4=0

2
1

2
0
4

1
6

Example 1.2 illustrated

15