Optimal Control of a Hybrid Dynamical

System: Two Coupled Tanks

T. Hirmajer and M. Fikar
Department of Information Engineering and Process Control,
FCFT STU, Radlinskeho 9, 812 37 Bratislava, Slovakia,
fax: +421 2 52 49 64 69
e-mail: {tomas.hirmajer, miroslav.fikar}@stuba.sk
Technical Report TH0001
http://www.kirp.chtf.stuba.sk/hirmajer/

27.12.2005

Version 1.11

Abstract
In this report we present dynamic optimization of a hybrid system - coupled tanks exhibiting hybrid dynamics. Dynamic optimization of this hybrid system is based on a control
vector parameterization (CVP) approach which allows the computation of the optimal
operating policies. The CVP approach is a direct method which transforms the original
problem into a non-linear programming (NLP) problem, which must be solved by a suitable
(efficient and robust) solver.
The main aim of the report is to show a detailed derivation of the hybrid optimal
control problem. Based on this result, CVP approach can be used and gradients of the
NLP problem can be derived.
Several simulation studies including minimal time problems and quadratic performance
criteria are presented.
Keywords:
Hybrid systems, dynamic optimization, control vector parameterization.

Contents
1 Introduction

2 Problem Formulation

3 Derivation of Gradients

4 Procedure
4.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Implementation of Algorithm . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Integration of Adjoint Equations . . . . . . . . . . . . . . . . . . . . . . . .

11
11
12
13

5 Illustrative Examples
5.1 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Minimum Time Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 LQ Cost Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

14
14
15
17

6 Results and Discussion

6.1 Minimum Time Problems . . . . . . . . . .
6.1.1 Multivariable Time Optimal Control
6.1.2 Singlevariable Time Optimal Control
6.2 LQ Cost Problem . . . . . . . . . . . . . . .
6.3 Graphic Output . . . . . . . . . . . . . . . .

18
18
18
18
18
19

7 Conclusions

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

22

List of Figures
1
2
3
4
5
6
7
8
9

Algorithm for CVP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Coupled tanks system . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simulation of the hybrid system with constant control values . . . . . . .
Optimal state trajectories with time optimal control . . . . . . . . . . . .
Optimal control policy for the Example 6.1.1 . . . . . . . . . . . . . . . .
Optimal state trajectories with singlevariable time optimal control . . . .
Optimal control policy for the Example 6.1.2 . . . . . . . . . . . . . . . .
Level of the liquid in the second tank for various choices of the coefficient
Optimal control policy for Example 6.2 . . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
r
.

11
14
15
19
19
20
20
21
21

1 Introduction

Introduction

Many processes in chemical and biotechnological industries are described by multiple sets
of differential and algebraic equations. As such they are difficult to control and optimize
in transient regimes if switching between the sets is to be taken into account. The switches
can involve different regimes of operation (occurrence of flooding in distillation columns,
explosive areas in mixtures of gasses, etc) or external actions (addition of second reactor
when production increases, etc).
There are several approaches to solution of such an dynamic optimization problem.
If the process to be optimized can be described accurately enough by piece-wise linear
and logic formulation, powerful algorithms in the area of explicit model predictive control
exists Bemporad and Morari (1999).
If fully nonlinear processes are concerned, original dynamic optimization problem has
to be approximated by some simplified formulation. The usual approaches are complete
discretisation of state and control variables orthogonal collocation. Such formulation
can be found in Avraam et al. (1998). Other possibility is to leaves the states intact and
approximate only the control variables as piece-wise constant, or with some higher order
approximations. This approach is known as control vector parameterization. Here different formulations can be found, depending on how gradients of the resulting nonlinear
programming problem (NLP) are calculated. In Vassiliadis et al. (1994), system of sensitivity equations is formed and the gradients are calculated from its solution. The advantage
of this method is easy formulation of the problem and forward integration of both states
and sensitivity equations. The drawback of this method lies in a large system of differential
equations as each optimized parameter generates a set of differential equations with the
same dimension as the number of states of the optimized process.
Another possibility, that is pursued in this work, is to calculate the gradients of NLP
via optimal control theory using the so-called co-state, or adjoint equations (Ruban 1997).
The advantage is that the number of differential equations is not proportional to number of
optimized parameters, but to number of constraints. On the other side, adjoint equations
have to be solved in opposite direction of time which makes the implementation more
difficult. When dealing with processes comprised of a large number of state equations and
only a small number of state-dependent constraints, this approach has favorable properties
compared to calculation of sensitivities.
The main aim of this work is to show a detailed derivation of the dynamic optimization
procedure for hybrid systems based on the optimal control theory. The results obtained
will then be presented on a simple example of two tank level control that exhibits hybrid
dynamics.

http://www.kirp.chtf.stuba.sk/ hirmajer

2 Problem Formulation

Problem Formulation

We consider a process to be optimized described by sets of differential equations

x = f i (x(t), u, t), ti1 t ti , i = 1, n,
x = f n+1 (x(t), u, t), tn t tP

(1)
(2)

with initial and terminal conditions (t0 = t0 (u), x+ (t0 ) = x0 (t0 , u), and tP = tP (u)). We
consider here the z dimensional process state vector x(t) and a constant m dimensional
vector of optimized parameters u. It is assumed that system states are governed by different
state equations f i () that are continuously differentiable.
The switching instant ti , at which one set of equations is replaced by another is determined by the switching conditions
g i (x
i , ti , u) = 0,

i = 1, n

(3)

We assume that the functions g i () are continuously differentiable with respect to all
variables. At switching instants, the vector x(t) can have breaks defined by the equations

x+
i = xi + i (xi , ti , u),

i = 1, n

(4)

where i () are also continuously differentiable vector functions and x

i = x(ti ), xi =
x(t+
i ) are the values of the vector x(t) before and after the switching instant, respectively.
In (4) we consider additive jumps that are superimposed on the continuous trajectory at
points ti (e.g. if i = 0, then x(t) is continuous at the point ti ).
Next, we assume model of the measurement unit, which is given in the form

(t) = (x(t), u, t),

t [t0 , tP ]

(5)

where () is also a continuous, continuously differentiable, bounded (together with the

first-order derivatives) vector function. However, the dimensions of vectors x(t) and (t)
may be different.
In the next step we define the cost J to be optimized (or constraint to be satisfied) in
a general Bolza form (Bryson and Ho 1975).
J(u) = G((tP ), u, tP ) +

ZtP

F ((t), u, t)dt

(6)

t0

where J is the scalar performance index to be minimized (functional of the quality of the
dynamic system), G defines final time conditions and F requirements along the time axis.

http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

Derivation of Gradients

For the purpose of NLP, we need to derive gradients of the cost function with respect to
optimized parameters u. These will be obtained by the variational method.
We supplement the functional (6) by constraints on system dynamics (1), (2), and by
initial value constraint, thus
J = J(u) +

ZtP

t0




T (t) [f (t) x(t)]

dt + T (t0 ) x0 (t0 , u0 ) x+ (t0 )

(7)

where we have introduced the following simplifications:

f (x(t), u, t) = f (t),
(x(tP ), u, tP ) = (tP ),
G((tP ), u, tP ) = G()

F ((t), u, t) = F (t),
(x(t), u, t) = (t),

(8)
(9)
(10)

When taking time of switch into account, the integral function can be separated into
following parts

ZtP

T (t)x(t)dt
=

t0

n Zti
X
i=1 t

T (t)x(t)dt

Ztp

T (t)x(t)dt

(11)

tn

i1

After integration by parts this gives

ZtP

(t)x(t)dt
=

ZtP

T
(t)x(t)dt

t0

t0

T
+

+ T (t+
0 )x(t0 ) (tP )x(tP ) +

n
X

 T +
T

(ti )x(t+
i ) (ti )x(ti )

(12)

i=1

The cost function (7) can now be written as

J =G() + T (t0 )x0 (t0 , u0 ) T (tP )x(tP )
ZtP
n
X


 T +

T

+
F (t) + T (t)f (t) + xT (t)(t)
dt +
(ti )x(t+
i ) (ti )x(ti )
t0

(13)

i=1

Taking into account the change of the right side of object equations (1)(2), we split

http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

the integral in (13) in the time intervals

J =G() + T (t0 )x0 (t0 , u0 ) T (tP )x(tP )
ZtP h
i

+
F (t) + T (t)f n+1 (t) + xT (t)(t)
dt
tn

n Zti h
i
X
T
+
F (t) + T (t)f i (t) + (t)x(t) dt
i=1 t

i1

n
X
i=1

 T +

T

(ti )x(t+
i ) (ti )x(ti )

(14)

Now, we can find the first variation of J taking into account correlations (5) between
(t) and x(t) and dependencies of t0 and tP upon u






G()
G()
G()
(tP ) +
u +
tP
J =

u
tP
+T (t0 )x0 (t0 , u0 ) + T (t0 )x0 (t0 , u0 )
T (tP )xP (tP , uP ) T (tP )xP (tP , uP )
(ZtP
h
i

+
F (t) + T (t)f n+1 (t) + xT (t)(t)
dt
tn

)
n Zti h
i
X
T
+
F (t) + T (t)f i (t) + (t)x(t) dt
i=1 t

i1

)
( n
X


T
+
+
T (t+
i )x(ti ) (ti )x(ti )

(15)

i=1

Calculating the first variation of the last formula in the equation (15), we find the
variation of the additional component with respect to x(t) and ti , i = 1, n having regard
to the break of the point which is represented as time of switch
)
( n
X

T

T (t+
i )x(ti ) (ti )x(ti )
i=1
n 
X






 
n 
X
i ()
i ()
T
T +

I+
(ti )
=

(ti ) x(ti ) +
ti
ti
x
i
i=1
i=1


n 
n h
i
X
X
i ()
T +
T
T +
+

(ti )x(ti ) (ti )x(ti ) ti

(16)
+
(ti )
u +
u
i=1
i=1
T

(t+
i )

http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

where I is the identity matrix. Relation between x(t+

i ) and x(ti ) with regard to time
of switch which was used in (16) is given from equation (4)






i ()
i ()
i ()
+

x(ti ) = x(ti ) +
x(ti ) +
ti +
u
(17)
ti
u
x
i

The variation of integrals in (15) with respect to time of switch ti , i = 1, n gives the
following result:

ZtP

n Zti
X

{}dt +

T
F (t
i ) + (ti )f i (ti ) + (ti )x(ti )

i=1 t

tn

i1

{}dt

n h
X
i=1

i
T +
+
T (t+ )x(t+ ) ti
F (t+
)

(t
)f
(t
)

i+1 i
i
i
i
i

(18)

Finally, we can say that the discontinuities of state variables are satisfied with following
additional summands (caused by time of switch)




n 
X
i ()
T
T +
(ti ) x(t
(ti ) I +
i )

xi
i=1


n 
X
i ()
T
T +
T +

+
+
+
F (ti ) + (ti )f i (ti ) F (ti ) (ti )f i+1 (ti ) + (ti )
ti
t
i
i=1


n 
X
i ()
T +
+
(ti )
u
(19)
u
i=1
Now, we can find the dependence of the variation of the break instant ti upon the
variation of phase coordinates x(t
i ) and upon the variation of parameters u by using
switching condition (3)






g i ()
g i ()
g i ()

g i () =
xi (ti ) +
ti +
u,
(20)
ti
u
xi (t
i )


1 

1 
g i ()
g i ()
g i ()
g i () 

ti =
x
(t
u,
(21)
i i
ti
ti
u
xi (t
i )
ti = ai xi (t
(22)
i ) + bi u,

1 


g i (xi , ti , u)
g i (xi , ti , u)
ai =
,
(23)
ti
xi (t
i )

1 

g i (x
g i (x
i , ti , u)
i , ti , u)
bi =
(24)
ti
u
http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

is not equal to zero, because equation (3)

In equations (23)(24) the derivative gti ()
i
must be resolved with respect to the time of switch ti . Combining results, which we
acquired above, we obtain


G() (tP )
T
J =
(tP ) x(tP )
(tP ) x(tP )

ZtP 
f (t)
F (t) (t)
T
T

+ (t)
+ (t) x(t)dt
+
(t) x(t)
x(t)
t0
(




n
X
i ()
T +
T
(ti ) I +
+
(ti )
x
i
i=1

T
T +

+
+ F (t
i ) + (ti )f i (ti ) (ti )f i+1 (ti )
 )


()
i
T +
ai x(t
F (t+
i )
i ) + (ti )
ti
(


dt0
x0 (t0 , u)
T
f (t0 )
+
F (t0 ) + (t0 )
t0
du
+

ZtP 

F (t) (t) F (t)

+
+ T (t)
(t) u(t) u(t)

t0

f (t)
u(t)



dt


G() (tP )
G()
dtP
T
+
+ (tP )f (tP ) +
+ F (tP )
(tP ) tP
tP
du
"


n
X
x
(t
,
u
)
i ()
G()
(t
)
G()
0 0
0
P
T
T +
+ (t0 )
+
+
+
(ti )
u
(tP ) u(tP )
u
u
i=1


 #)

()
i
T +
T +
+
+

+ T (t
bi u
i )f i (ti ) + F (ti ) F (ti ) (ti )f i+1 (ti ) + (ti )
ti
(25)
Now, we can set coefficients x(t) in equation (25) to zero, we obtain adjoint equations
G() (tP )
,
(tP ) x(tP )
f (t) F (t) (t)
T
(t) = T (t)

,
x(t) (t) x(t)

T (tP ) =

(26)
t0 t < tP

(27)

with the boundary value at the point tP and the switching condition of the Lagrange

http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

multipliers at switching instants

T (t
)=
( i



 
i
i
+
I+
+
f i+1 (ti ) ai

ti
x
i
)

1
+
+(F (t
I f i (t
i ) F (ti ))ai
i )ai
T

(t+
i )

(28)

where i = 1, n and the instants depend explicitly on the parameters. If ai = 0 the switching
conditions are simplified considerably (if the value of the jump of state variables of the
object i does not depend on x
i , then the multipliers are continuous):



i
T +
T
(ti ) = (ti ) I +
(29)
, i = 1, n
x
i
Now, we can see the coefficients of the variations of parameters u in the functional
(25) from whose gradients of the cost with respect to optimized variables can be derived
(
G() (tP ) G()
dJ
=
+
du
(tP ) u(tP )
u



x0 (t0 , u)
dt0
T
+ F (t0 ) + (t0 )
f (t0 )
t0
du


ZtP 
f (t)
F (t) (t) F (t)
T
+
+ (t)
dt
+
(t) u(t) u(t)
u(t)
t0


G() (tP )
G()
dtP
T
+
+ (tP )f (tP ) +
+ F (tP )
(tP ) tP
tP
du
"



n
x0 (t0 , u0 ) X
i ()

+ T (t
+ T (t0 )
+
T (t+
i )f i (ti )
i )
u
u
i=1

  #)

()
i
T
T
+
+
+
+
+ F (t
bi
(30)
i ) F (ti ) (ti )f i+1 (ti ) + (ti )
ti
Additional summands contain information about switch
(


n
X
i ()
T +
(ti )
u
i=1


 )

()
i
T +
T +
+
+

+ T (t
bi
(31)
i )f i (ti ) + F (ti ) F (ti ) (ti )f i+1 (ti ) + (ti )
ti
http://www.kirp.chtf.stuba.sk/ hirmajer

3 Derivation of Gradients

10

If state variables of the process are continuous at switching points (i = 0), then
the integrand function F of the quality index is continuous, but the Lagrange multipliers
contain discontinuity at the moment of switch



1
T +
+

T (t
(32)
, i = 1, n
i ) = (ti ) I f i+1 (ti )ai x I f i (ti )ai

and additional summands (31) of the calculation formula for necessary coefficients (30) are
simplified to form
n
X

i=1


T +

+
T (t
i )f i (ti ) (ti )f i+1 (ti ) bi

If the time of switch depend explicitly on parameters (ti = ti (u), ai = 0, and bi =

then the Lagrange multipliers are continuous.

http://www.kirp.chtf.stuba.sk/ hirmajer

(33)
dti
),
du

4 Procedure

11

Procedure

Figure 1: Algorithm for CVP

4.1

Algorithm

We assume that the continuous control trajectory is considered to be piece-wise constant

over P intervals. This makes it possible to convert the original problem of dynamic optimization into nonlinear programming problem (Fikar and Latifi 2001). In this algorithm
we assume that we have the functional J0 and k constraints Jj , where j = 1, k. For simplicity assume that the initial time t0 and state x(t0 ) are given and constant. Then we can
write the following algorithm:
1. Integrate the system (1)(2) and integral terms Fj together from t = t0 to t =
tP . Restart integration with switching conditions (3), states can be discontinuous
following the equation (4).
2. For j = 0, k repeat
(a) Initialize adjoint variables Tj (tP ), according to equations (26) as
j T (tP ) =

Gj () (tP )
(tP ) x(tP )

(b) Initialize intermediate variables J D,j as zero.

http://www.kirp.chtf.stuba.sk/ hirmajer

(34)

4.2

Implementation of Algorithm

12

(c) Integrate backwards from t = tP to t = t0 the adjoint system (27) and intermediate variables. Allow for discontinuities of the adjoint equations as given in (29),
restart integration at these points and at the points of changes of dynamics
Hj
f () Fj (t) (t)
T
j = j T (t)

=
,
x(t)
(t) x(t)
x
f () Fj (t) (t) Fj (t)
Hj
JTD,j =j T (t)
+
+
=
u(t)
(t) u(t)
u(t)
u

(35)
(36)

Equations (35), (36) are computed from (25) and JD,j represent integral parts
of the gradients.
(d) Calculate the gradients of Jj with respect to times ti and control u, with help
of (25).
Jj
Gj ()
= Hj (t
,
(37)
P) +
tP
tP



Jj
i ()
T +

+
(38)
= Hj (ti ) Hj (ti ) + j (ti )
bi , i = 1, N,
ti
ti


i ()
Gj ()
Jj
T +
= J D,j (ti1 ) J D,j (ti ) +
+ (ti )
, i = 1, P
ui
ui
u
(39)
In this manner, the values of Jj are obtained in the step 1 and the values of gradients in
the step 2d. This is all what is needed in NLP algorithm. The scheme of the forward and
backward integrations is shown in Fig. 1.
For numerical reasons, time increments ti will be optimized, rather than absolute
time values ti . Therefore, the gradients have to be modified correspondingly. The relations
between times and their increments are given as
tP =

N
X

ti

(40)

i=1

Therefore, the following holds for the derivatives

N

X Jj tr
Jj
=
ti
tr ti
r=1

4.2

(41)

Implementation of Algorithm

The algorithm was implemented FORTRAN 77. The first part of this program contains
a module for forward and backward integrations LSODAR (Petzold and Hindmarsh 1997)
that is able to handle state events. The second module of the program computes gradients
and calls the NLP solver NLPQL (Schittkowski 1981).
The user specifies initial conditions:
http://www.kirp.chtf.stuba.sk/ hirmajer

4.3

Integration of Adjoint Equations

13

u0 , x0 , lower and upper control bounds, number of time intervals,

cost function, constraints,
differential equations of the process.

4.3

Integration of Adjoint Equations

When the adjoint equations are integrated backwards in time, the knowledge of states
x(t) is needed. In our case the program stores at first in the forward pass the states at a
predefined grid points and interpolates them when adjoint equations are solved.
Two interpolations were implemented into the program: linear, and the approximations
having continuous states and continuous first order derivatives across boundaries. All
examples were simulated with the second one.

http://www.kirp.chtf.stuba.sk/ hirmajer

5 Illustrative Examples

14

Illustrative Examples

Figure 2: Coupled tanks system

5.1

Process

We assume a non-linear system of two tanks shown in Fig. 2 that is described by two sets
of differential equations
!

 dx11 
k11 x1
u1

f 1 = dxdt12 = u2 Fk111 x1 F1 k22 x2 ,

(42)
+ F2 F2
dt
F2

 dx21 
k11 x1 (x2 h)
u1

F1
F1

f 2 = dxdt22 =
(43)

k
x1 (x2 h)
11
k22 x2
u
2
dt
+

F2
F2
F2
where F1 , F2 [m2 ] are the cross-sectional areas of the tanks; k11 , k22 [l2.5 s1 ] constants;
x1 , x2 state values levels of liquid in the first and the second tank; u1 , u2 [l s1 ] control
values flow rates; h [m] vertical distance between tanks.
The first dynamics with no interactions between tanks takes place if the height h2 in the
second tank is smaller than h. In the opposite case, the tanks interact and are described
by the dynamics f 2 .
Thus, the height h specifies the switching condition between dynamics f 1 and f 2 as
g1 = h x2
http://www.kirp.chtf.stuba.sk/ hirmajer

(44)

5.2

Minimum Time Problem

15

x(1)
x(2)

0.6

h [m]

0.5
0.4
0.3
0.2
0.1
0

10
time [s]

15

20

Figure 3: Simulation of the hybrid system with constant control values

The Jacobians of the differential equations with respect to states and controls are of
the form


1
1
0
0
f 2
f 1
F1
F1
,
,
(45)
=
=
0 F12
0 F12
u
u
!

2Fk111
0
f 1
x1
=
,
(46)
k11

2Fk222
x
2F2 x1
x2

k11
k11

f 2 2F1 x1 (x2 h)
2F1 x1 (x2 h)

=
(47)
k11
k11

2Fk222
x
x2
2F2

x1 (x2 h)

2F2

x1 (x2 h)

The concrete values of parameters are defined as: resistances k11 = 1.75 l2.5 s1 , k22 =
1.50 l2.5 s1 , surfaces F1 = 2.00 m2 , F2 = 4.00 m2 , vertical distance h = 0.40 m, initial states
x1 (0) = x2 (0) = 0.10 m, desired state xw
2 = 1.00 m.
The initial values of optimized parameters are ui = 1 l s1 , ti = 1 s with bounds
defined as ui [0, 3] and ti [0.01, 10.00].
Fig. 3 shows the response of the plant to these initial optimized parameters with final
time of simulation set to tP = 20.00 s. We can observe the change of dynamic at the switch
time t = 2.19 s.

5.2

Minimum Time Problem

The aim of the optimal operation is in this case defined as to reach in a minimum time
a new steady-state defined by a desired height in the second tank that is located in the
opposite region. Thus, the cost functional is defined as
min J0 = tP
tp ,u

http://www.kirp.chtf.stuba.sk/ hirmajer

(48)

5.2

Minimum Time Problem

16

subject to the constraints:

J1 = x2 (tP ) xw
2 = 0,
dx1
J2 =
= 0,
dt
dx2
=0
J3 =
dt

(49)
(50)
(51)

where xw
2 is the required level of the liquid in the second tank. Constraints J2 , J3 represent
the requirement of the new steady-state in the final time.
We consider that the control vector u contains incoming flowrates u1 and u2 and that
these are piece-wise constants. Then, we optimize the duration and value of piece-wise
controls u over specified number of time intervals.
As J0 does not depend on state variables, it is easy to obtain the required gradients as
 
J0
J0
0
=
= 1,
(52)
0
tP
u
The situation is more complicated with the constraints J1 , J2 , and J3 . We calculate
final time conditions for the Langrange multipliers and set subintegral variables J D,1 , J D,2 ,
J D,3 as zero
!  
(x2 xw
x1
2)
0
x1 (tP )
1
1 (tP ) = (xx
=
(53)
w
2 x2 )
x2
1
x2

x2 (tP )

Then, we integrate backwards from t = tP to t = t0 with help of following equations

f (t)
T
1 (t) = T1 (t)
,
x(t)
f (t)
T
J D,1 = T1
u(t)

(54)
(55)

Solutions of 1 (t) and J D,1 are obtained by backward integration of these differential
equations. The desired gradients are then obtained as
J1
G1

= H1 (t
= T1 (t
P) +
P )f 2 (tP ),
tP
tP
G1
J1
+
T +
+

= H1 (t
= T1 (t
i ) H1 (ti ) +
i )f (ti ) 1 (ti )f (ti ),
ti
ti
J1
= J D,1 (ti1 ) J D,1 (ti )
ui

(56)
(57)
(58)

The same situation is with the remaining constraints. The only problem lies in the fact
that initialization of adjoint variables at the terminal state depends on the location of the
state h2 (tP ). Also, as the constraints depend directly on the control vector in the final
.
time, equation (58) contains additional term G
u
http://www.kirp.chtf.stuba.sk/ hirmajer

5.3

LQ Cost Problem

5.3

17

LQ Cost Problem

Consider now a situation with fixed terminal time tP and the objective to optimize a LQ
cost functional
Z tP


(x2 xs2 )2 + r (u1 us1 )2 dt
(59)
min J0 =
u

t0

where xs2 , us1 are steady states of the values and r is a positive weight coefficient.
Algorithm and computation of the gradients considering the constrains are same that
the previous case and are given in (53)(58).
However, we need to calculate value of the cost J0 in the each forward integration
step. At the end of each integration we have to obtain value of the cost. Next we need to
calculate gradients of J0 with respect to time intervals and control values.
These gradients we obtain with help of backward integration. To integrate backwards
from t = tP to t = t0 we need to set initial conditions of 0 (tP ) and J D,0 (tP ). The
initial conditions are the same as were presented in Example 5.2. Then we can continue
to integrate backward with help of following equations
f (t) F0 (t) (t)
T
0 (t) = T0 (t)

,
x(t)
(t) x(t)
f (t) F0 (t)
T
+
J D,0 = T0
u(t)
u(t)

(60)
(61)

Finally, we can use Algorithm 4.1 to computate the desired gradients

J0

s 2
s 2
+ T0 (t
+ r u1 (t
= x2 (t
P )f 2 (tP ),
P ) u1
P ) x2
tP





J0

T +
+
+
s 2

s 2
+ T0 (t
= r u1 (tP ) u1 u1 (tP ) u1
i )f (ti ) 0 (ti )f (ti ),
ti
J0
= J D,0 (ti1 ) J D,0 (ti )
ui

http://www.kirp.chtf.stuba.sk/ hirmajer

(62)
(63)
(64)

6 Results and Discussion

6
6.1
6.1.1

18

Results and Discussion

Minimum Time Problems
Multivariable Time Optimal Control

In order to drive the process to the desired new steady-state as fast as possible, we consider
both inflows to be optimized and set the number of optimized time intervals to two.
The optimization and integration tolerances were set as 106 and 1010 , respectively.
The minimum final time was obtained as tP = 1.47 s after 8 NLP iterations. The optimal
state variables can be found in Fig. 4 and the corresponding controls in Fig. 5. The control
variables show typical bang-bang behavior that follows from the desired objective.
6.1.2

Singlevariable Time Optimal Control

A more realistic example considers only one control variable the inlet flowrate to the first
tank. The number of time intervals was set to 8. Optimization and integration tolerances
were set as 104 and 1012 , respectively.
The optimal minimum time in this case was higher and was found as tP = 8.623 s.
The optimal state variables can be found in Fig. 6 and the corresponding controls in
Fig. 7. Again, the control variables show typical bang-bang behavior. Although 8 optimized
intervals were chosen, only three would be enough two for optimal control of the second
order dynamics and the last one to achieve the desired steady-state.

6.2

LQ Cost Problem

In this case, we fix the number of time intervals to 15 and their duration to 1 s and optimize
only the first control variable with the optimization and integration tolerances, which were
set as 105 and 1012 , respectively. Steady-state analysis gives the values of the new
steady-state as xs2 = 1.00 m, us1 = 2.50 l s1 .
Various values of the penalization factor r have been considered, the optimal simulations
in Fig. 8 (states) and Fig. 9 show three of them.
Decreasing value of r leads to behavior similar to that of time optimal control. On the
other hand, its increase smoothes control variable considerably with only a small loss of
performance.
The optimal value of the cost function was 2.7517 with 7 NLP iterations if the weight
coefficient r = 1 was considered.

http://www.kirp.chtf.stuba.sk/ hirmajer

6.3

6.3

Graphic Output

19

Graphic Output

x(1)
x(2)

1.2
1.00m

h [m]

1
0.8
0.6
0.4
0.2
0

0.5

1.5
time [s]

2.5

Figure 4: Optimal state trajectories with time optimal control

3.5

3.5
u(2)

2.5

2.5

u [l/s]

u [l/s]

u(1)

1.5

1.5

0.5

0.5

time [s]

2
time [s]

Figure 5: Optimal control policy for the Example 6.1.1

http://www.kirp.chtf.stuba.sk/ hirmajer

6.3

Graphic Output

20

x(1)
x(2)

3
2.5

h [m]

2
1.5
1.00m

1
0.5
0

10

15

time [s]

Figure 6: Optimal state trajectories with singlevariable time optimal control

3.5
u(1)
3

u [l/s]

2.5
2
1.5
1
0.5
0

10

15

time [s]

Figure 7: Optimal control policy for the Example 6.1.2

http://www.kirp.chtf.stuba.sk/ hirmajer

6.3

Graphic Output

21

1.00m

h [m]

0.8

0.6

0.4
r = 0.10
r = 1.00
r = 10.00

0.2

10

15

time [s]

Figure 8: Level of the liquid in the second tank for various choices of the coefficient r

3
2.8
2.6
u [l/s]

2.4
2.2
2
1.8
r = 0.10
r = 1.00
r = 10.00

1.6
1.4
0

10
time [s]

Figure 9: Optimal control policy for Example 6.2

http://www.kirp.chtf.stuba.sk/ hirmajer

15

7 Conclusions

22

Conclusions

In this report we have investigated the problem of dynamic optimization of systems described by multiple sets of differential equations. Control vector parameterization has been
used and gradients for the nonlinear programming have been calculated based on the optimal control theory. This is in contrast to usual approaches where sensitivity equations
are preferred due to simplicity of the implementation.
On the other hand, the adjoint variable approach has its advantage in systems described
by a larger set differential equations and it can reduce the computational time considerably.
Simulations with a simple chemical engineering example confirmed attractivity of the
proposed approach.

Acknowledgments
The authors are pleased to acknowledge the financial support of the Scientific Grant Agency
of the Slovak Republic under grants No. 1/0135/03, 1/1046/04.

References
M. P. Avraam, N. Shah, and C. C. Pantelides. Modelling and optimisation of general
hybrid systems in the continuous time domain. Computers chem. Engng., 22(Suppl.):
S221S228, 1998. 3
A. Bemporad and M. Morari. Control of systems integrating logic, dynamics, and constraints. Automatica, 35(3):407 427, 1999. 3
A. E. Bryson and Yu-Chi Ho. Applied Optimal Control - Optimization, Estimation and
Control. Hemisphere publishing corporation, Taylor & Francis, Inc., 1975. 4
M. Fikar and M. A. Latifi. Users guide for FORTRAN dynamic optimisation code DYNO.
Technical Report mf0201, LSGC CNRS, Nancy, France; STU Bratislava, Slovak Republic, 2001. 11
L.R. Petzold and A.C. Hindmarsh. LSODAR. Technical report, Computing and mathematics research division, lawrence livermore national laboratory livermore, ca 94550,
1997. 12
A. I. Ruban. Sensitivity coefficients for discontinuous dynamic systems. Computer and
Systems Sciences International, 36:536542, 1997. 3
K. Schittkowski. NLPQL: A new fortran implementation of a sequential quadratic programming algorithm for parallel computing. Technical report, Department of Mathematics,
University of Bayreuth, D - 95440 Bayreuth, 1981. 12

REFERENCES

23

V. S. Vassiliadis, R. W. H. Sargent, and C. C. Pantelides. Solution of a class of multistage

dynamic optimization problems. 1. Problems without path constraints, 2. Problems with
path constraints. Ind. Eng. Chem. Res., 33:21112122, 21232133, 1994. 3