Project-2

December 24, 2017

1 Problem 5.1
Define the horizontal distance from point 1 to point 2 as XH and the vertical distance as XV . The problem has final and
initial conditions specified as the following:

x1 (tf ) = XH , x2 (tf ) = −XV , x1 (t0 ) = 0, x2 (t0 ) = 0

Final time is not specified, thus the problem is classified as free-final time problem.
Since this is an optimal time problem, and there is no other constraints on the states nor the input, the functional to be
minimized (the performance index) is simply:
Z tf Z tf
J = L dt = 1 dt
to to

T
We can then introduce the co-states as the vector p(t) = [p1 (t), p2 (t)] , and use that definition in writing out the hamiltonian
of the system as follows:
H = pt ẋ − L
Next we need to find the equations of motion governing the dynamics of the system, which is done by resolving the velocity
vector and expressing it in the generalized coordinates x = [x1 , x2 ]T , the control input β and the current distribution δ(x1 )

x˙1 = v cos(β)
x˙2 = −v sin(β) + δ(x1 )

Substituting from the dynamics into the expression of the hamiltonian and substituting for the lagrangian with unity, the
expression of the hamiltonian becomes:

H = p1 v cos(β) + p2 (−v sin(β) + δ(x1 )) − 1

The augmented functional for the system then becomes:

Z tf Z tf
J = H dt = [ p1 v cos(β) + p2 (−v sin(β) + δ(x1 )) − 1 ] dt
to to

Thus applying the general necesary conditions for a problem of this type gives the following set of equations:
∂H T
ẋ = = [v cos(β) , −v sin(β) + δ(x1 )]
∂p
 T
∂H ∂δ(x1 )
ṗ = − = − ,0
∂x x1
∂H
= −p1 v sin(β) − p2 v cos(β) = 0
∂β

and
pT δxf + H|tf δtf = 0
Then applying the boundary conditions for this type of problem, namely we substitute with δxf = 0 and let δtf be arbitrary
we get the condition that:
H|tf = 0

1
 
2 
We substitute with the expression of the current as δ(x1 ) = U 1 − X42 x1 − X2H Expanding the equations for the
H
augmented system of states and co-states we get the following system of differential equations:

x˙1 = v cos(β)
 2 !
4 XH
x˙2 = −v sin(β) + U 1 − 2 x1 −
XH 2
 
8U XH
p˙1 = 2 x1 −
XH 2
p˙2 = 0
−p2
β = tan−1 ( )
p1
Subject to the four boundary conditions:

x1 (tf ) = XH , x2 (tf ) = −XV , x1 (t0 ) = 0, x2 (t0 ) = 0

and the constraint on tf such that:

p1 |tf v cos(β|tf ) + p2 |tf −v sin(β|tf ) − 1 = 0

The problem as stated above is a complicated boundary value problem. The complications arise from the fact that
the final time at which the boundary conditions are stated is the solution to the constraint specified above which requires
knowledge of the values of the co-states at the final time (which are unknown!!). A simple way to solve this problem is to
convert it into a least-squares optimization problem and use Levenberg-Marquardt algorithm to solve it. This is done by first
constructing the residual vector which is, in our case, the following:
 
x1 |tf − XH
R= x2 |tf − XV


p1 |tf v cos(β|tf ) + p2 |tf −v sin(β|tf ) − 1

Then we use the LMA to optimize for the list of parameters [p1 |t0 , p2 |t0 , tf ] that minimize this residual vector. We used
MATLAB function lsqnonlin to solve this optimization problem and the resulting normalized curves are shown below:

2
Figure 1: History of p1 with normalized time τ

Figure 2: History of p2 with normalized time τ

3
 Figure 3: Normalized path of the boat

Figure 4: History of β with normalized time τ

4
 As expected, the control variable β tries to counter the flow that works against the motion of the body towards north.
Thus, the maximum of the control variable would naturally occur at the maximum of the counter current (i.e at x1 = X2H ).
We also expect it to decrease again to maximize horizontal velocity. Since we know that the shortest path between two
points is a straight line, the absence of any external forces or fields, we expect that there is a transformation of coordinates
which transforms the path above into a straight line in the new space of transformed coordinates. We expect the coordinates
transformation to involve only x1 since the only external force is dependent on it.

2 Problem 5.6
The problem requires minimizing the functional given by:
Z T
1.5x2 + 0.5u2 dt
 
J =
0

So we can deduce that the lagrangian of the system is given by:

L = 1.5x2 + 0.5u2

We can then introduce the co-state p and write the hamiltonian of the system as:

H = pẋ − L = p [x + u] − 1.5x2 − 0.5u2

and then the new functional to be minimized is:

Z T
p [x + u] − 1.5x2 − 0.5u2 ] dt
 
Ja =
0

subject to the boundary conditions that δxf is free and δtf = 0, in other words:

p|tf δxf + H|tf δtf = 0

after substitution becomes:

p|tf = 0
Applying pontryagin’s principle in the unbounded control input form, we have:
∂H
=p−u=0
∂u
Applying the necessary conditions on H for optimality, we can find the governing equations for evolution of states and
co-states:
∂H
ẋ = =x+u
∂p
∂H
ṗ = − = −p + 3x
∂x

subject to the conditions that:

x|t0 = x0 p|tf = 0
The problem now is a boundary value problem with specified integration time. To solve this, we first substitute from
Pontryagin’s principle for u, thus making the set of equations become:

ẋ = x + p
ṗ = −p + 3x

Which can be written in matrix form as:     

ẋ 1 1 x
=
ṗ 3 −1 p
Which has the form: ẏ = [A] y and the general solution form of

y = eAt y|t0

5
 This solution can be expanded in terms of x and p as:

x = Φ11 x|t0 + Φ12 p|t0

p = Φ21 x|t0 + Φ22 p|t0

Where:  
Φ11 Φ12
Φ = eAt =
Φ21 Φ22
By inspecting the matrix A we find that it has two eigen valuesλ1 = 2 , λ2 = −2, in other words, it is a non-defective matrix,
and can be diagonalized. Thus, we can write the exponential of this matrix in terms of the exponential of its diagonalized
form as follows:
eAt = V eΛt V −1
Where the matrix Λ is the diagonal matrix with the eigen values of matrix A on the diagonals, and the matrix V is the
matrix of egien vectors ordered in the same order as the eigen values in matrix Λ are ordered. We use MATLAB to find the
eigenvalues and eigen vectors of the matrix A. The result is:
   
2 0 0.7071 −0.3162
Λ= , V =
0 −2 0.7071 0.9487

We can now write: −1

e2t
  
At 0.7071 −0.3162 0 0.7071 −0.3162
e =
0.7071 0.9487 0 e−2t 0.7071 0.9487
Carrying out the multiplication and simplifying (using MATLAB) we find the exponential of the matrix A becomes:
" 2t −2t 2t −2t
#
3e +e e −e
eAt = 4
3e2t −3e−2t
4
e2t +3e−2t
4 4

And thus, we have:

3e2t + e−2t
Φ11 =
4
e2t − e−2t
Φ12 =
4
3e2t − 3e−2t
Φ21 =
4
e2t + 3e−2t
Φ22 =
4

Substituting with the conditions we have:

x|t0 = Φ11 |t0 x|t0 + Φ12 |t0 p|t0 ⇒ x0 + 0 ∗ p|t0 = x0 ⇒ x0 = x0

Which is a trivial result. Next we use the second boundary condition:

3e2T − 3e−2T e2T + 3e−2T 3e−2T − 3e2T

p|tf = Φ21 |tf x|t0 + Φ22 |tf p|t0 ⇒ x0 + p|t0 = 0 ⇒ p|t0 = 2T x0
4 4 e + 3e−2T
Thus the solution becomes:
3e2t + e−2t e2t − e−2t 3e−2T − 3e2T
 
x= + · 2T x0
4 4 e + 3e−2T
3e2t − 3e−2t e2t + 3e−2t 3e−2T − 3e 2T
 
p= + · 2T x0
4 4 e + 3e−2T
3e2t − 3e−2t e2t + 3e−2t 3e−2T − 3e2T
 
u= + · 2T x0
4 4 e + 3e−2T
We can put the control law in closed form by substituting from the first equation for x0 into the third as follows:
x
x0 =  
3e2t +e−2t e2t −e−2t 3e−2T −3e2T
4 + 4 · e2T +3e−2T

6
  
3e2t −3e−2t e2t +3e−2t 3e−2T −3e2T
4 + 4 · e2T +3e−2T
u=   x
3e2t +e−2t e2t −e−2t 3e−2T −3e2T
4 + 4 · e2T +3e−2T

As integration limit goes to infinity T → ∞, we find that the above expression simplifies to:
 2t −2t 
3e −3e e2t +3e−2t
4 + 4 · −3
u =  2t −2t  x = −3x
3e +e e2t −e−2t
4 + 4 · −3

Thus, we find that the closed-loop control law is u = −3x in the case of an infinite horizon problem.

3 Problem 5.35
We have the initial states specified and only half of the final states specified,
R t additonally, we have free final time. We start
out by writing the lagrangian of the system: L = 1, and accordingly J = 0 f L dt, subject to the constraint that ẋ = f (x, u).
Thus we construct the augmented functional
Z tf
Ja = [p| f (x, u) − L] dt
0

The above optimization problem is solved by applying the necessary conditions for optimality (Hamilton’s Equations).

∂H
ẋ =
∂p
∂H
ṗ = −
∂x
We have the hamiltonian, expressed in components of x and p as:
 q   q 
2 2 2 2
H = p2 u1 − αx2 x2 + x4 + p4 u2 − αx4 x2 + x4 + p1 x2 + p3 x2 − 1

Which yields the following expressions for the co-state equations:

 
  0 
p˙1 
 −p1 + p2 + √ αp4 x2 x4 √ αx22
p
2 2

 p˙2   − p2 − α x2 + x4 
x22 +x24 x22 +x24 
 = 
 p˙3   0 
   
p˙4  αp2 x2 x4
√ √ αx 2 p
2 2

2 2
− p4 2
4
2
− α x2 + x4
x2 +x4 x2 +x4

At tf , the boundary conditions are specified by substituting into the boundary term in the first variation of the augmented
function:
pT δxf + H|tf δtf = 0
with the conditions that δx1 |tf = 0, δx3 |tf = 0 and δx|0 = 0, which gives us the following expression for δx|tf
 
0
 δx2 |tf 
δx|tf =  
 0 
δx4 |tf

Substituting in the expression for the boundary term, we get:

(p2 δx2 ) |tf + (p4 δx4 ) |tf + H|tt δtf = 0

Which necessitates that:

p2 |tf = 0
p4 |tf = 0


H|tf = p1 |tf + p3 |rf x2 |tf − 1 = 0

7
The above equations give two more boundary conditions, thus the full set of boundary conditions for this problem become:
x1 |0 = 0 , x2 |0 = 0 , x3 |0 = 0 , x4 |0 = 0 , x1 |tf = e1 , x3 |tf = e3 , p2 |tf = 0 , p4 |tf = 0
With the additional equation to solve for the final time tf :


p1 |tf + p3 |rf x2 |tf − 1 = 0
Next, we need to obtain the control law. To do that, we will make use of Pontryagin’s principle which states that the optimal
control law maximizes the hamiltonian, which means:
  q   q  
∗ 2 2 2 2
u = argmax H = argmax p2 u1 − αx2 x2 + x4 + p4 u2 − αx4 x2 + x4 + p1 x2 + p3 x2 − 1
u∈U u∈U
  q   q  
0| 2 2 2 2
= argmax p u − p2 αx2 x2 + x4 − p4 −αx4 x2 + x4 + p1 x2 + p3 x2 − 1
u∈U

|
Where U = {u | u u ≤ 1}. Since we know that p˙1 = 0 and p˙3 = 0, we can conclude that the vector p is constrained to lie
in a plane in 4-D. This plane can effectively be mapped to a 2D plane making all possible solutions for the 4-D vector p
isomorphic to vectors in 2D. Furthermore, since the type of constraint is linear (p1 = C1 , p3 = C3 ) we can conclude that
the transformation is linear, and accordingly an extremum in the 2-D space on which the hamiltonian is defined on u andp0
|
corresponds to an extremum in the 4D space where u0 and p are defined. In the above argument, p = p1 p2 p3 p4 ,
| | |
u0 = 0 u1 0 u2 , p0 = p2 p4 , u = u1 u2 .
     
 
According to the definition of the hamiltonian, we can see that the term ( p2 p4 u) is the only term involving the control
effort u. Thus maximizing this term is the only way to maximize H with respect to u. This term is basically the dot product
between the two vectors u and p0 . Thus the best way to maximize this expression is to make the control vector always in the
same direction as the projection of the co-state vector p0 on the 2D plane in the 4D space where u lies with the maximum
possible value for the magnitude of u, i.e:
p
u21 + u22 p2
   
∗ 1 p2
u = =
p22 + p24 p4 p22 + p24 p4
p p

An alternative approach is to use Lagrange Multipliers to perform the constrained optimization, first we construct the
augmented function by introducing the lagrange multiplier λ:
Ha = H + λ (1 − u| u)
 q   q 
= p2 u1 − αx2 x2 + x4 + p4 u2 − αx4 x2 + x4 + p1 x2 + p3 x2 − 1 + λ 1 − u21 + u22
2 2 2 2

 |
Then we obtain the gradient of the augmented function Ha with respect to the vector u1 u2 λ and equate it with
the zero vector to find the maximizing argument:
   
p2 + 2λu1 0
∇Ha =  p4 + 2λu2  =  0 
u21 + u22 − 1 0
Solving the above equations for u1 , u2 and λ yields, in the case of maximization, the expressions:
p2
u1 = p 2
p2 + p24
p4
u2 = p 2
p2 + p24
" #
1 p22 p24
λ=− p +p 2
2 p22 + p24 p2 + p24

In the second case, the constraint is | u1 | + | u2 |≤ 1. We expect the second case to produce smaller value for the Hamiltonian
because it limits the admissible magnitude of the control variable allowed for each possible value for p except at the conditions
where p2 = 0 or p4 = 0 . This is apparent from the next page. We can conclude that a similar expressioin will result for the
control input but with replacing the magnitude with the expression: 1 − 2 | u1 u2 |, thus making the equation expressing the
control:    
u1 1 − 2 | u1 u2 | p2
= p 2
u2 p2 + p24 p4
The above expression needs to be further simplified considering each possible case for the product | u1 u2 | which yields a
piece-wise defined control law.

8
 u2

u21 + u22 ≤ 1

−2 −1 1
u1

−1

−2

Figure 5: A Visualization of control input u constraint