Section 20.1 from Fundamental methods of Mathematical Economics, McGraw Hill 2005, 4th Edition. By A. C.

Chiang & Kevin Wainwright is covered. The nature of optimal control Read page 631-632 from book. The summary is as follows: The optimal control problem is Maximize Subject to ( ( ( ) ) ) ( ) and u(t) is unrestricted.

t is time, y(t) is known as state variable, u(t) as control variable. The objective function is an integral ( ) whose integrand ( ) stipulates how

the choice of the control variable u at time t, along with the resulting y at time t, determines our object of maximization at t.

The equation ( ) is equation of motion for state variable. This equation provides the mechanism whereby our choice of control variable u can be translated into a specific pattern of movement of the state variable y.

( ) In equation ( ) , we indicate that initial state, the value of y at t=0, is a constant A, but the terminal state ( ) is left unrestricted. u(t) is unrestricted its possible for u to be limited to a control region. Pontryagins maximum principal Read page 633-634 from book. The summary is as follows: The Hamiltonian The Hamiltonian is defined as ( ) ( ) ( ) (

Where ( ) is known as the costate variable, H denotes the Hamiltonian and is a function of four variables: t, y, u and Like the Lagrange multiplier, the costate variable ( ) measures the shadow price of the state variable.

The maximum principal The various components of maximum principal are as follows: (i) (ii) (iii) (iv) ( ) and (state equation) (costate equation) (transversality condition)

Condition (i) states that at every time t the value of u(t), the optimal control, must be chosen so as to maximize the value of Hamiltonian over all admissible values of u(t).

Conditions (ii) and (iii) of maximum principal,

and

give us two equations of

motion, referred to as the Hamiltonian system for the given problem.

Condition (iv), is the transversality condition appropriate for the free-terminal-state problem only. Example 1: Find the optimal control path that will Maximize ( )

Subject to ( ) And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ()

(ii)

The state equation is

and for this problem

( ) (iii) (iv) The costate equation is The transversality condition is and for this problem ( ) ( ) ( ) Equations (ii) and (iii) constitute differential equation system for this problem. Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the transversality condition (iv) and initial condition ( ) . We can first solve for by straight integration of (iii) to get ( ) ( ) in (vi), we have

( ) Using the transversality condition ( )

Thus the optimal costate path is ( ) ( ) Using (t) from (vi) in (i), the corresponding optimal control path is ( ) Using (vii) in (ii), we have ( ) ( )

and by integration, we have ( ) Setting t=0 in the preceding equation, we get variable is ( ) The optimal control path is ( ) ( ) ( ) with costate variable ( ) ( ) ( ) . . Thus the optimal path for state .

Example 2: Find the optimal control path that will Maximize Subject to
3

( ) And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is (

( ) Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i)
( )

() ( )

(ii)

The state equation is

and for this problem ( )

(iii) (iv)

The costate equation is

and for this problem ( )

The transversality condition is ( ) ( ) Equations (ii) and (iii) constitute differential equation system for this problem. Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the transversality condition (iv) and initial condition ( ) . We can first solve for by straight integration of (iii) to get ) ( ) in (vi), we have

( ) ( Using the transversality condition ( )

Thus the optimal costate path is ( ) ( ) Using (t) from (vi) in (i), the corresponding optimal control path is ( ) ( ) Using (vii) in (ii), we have and by integration, we have ( ) ( ) ( ) .

Setting t=0 in the preceding equation, we get variable is ( ) The optimal control path is ( ) costate variable ( )

. Thus the optimal path for state . ( ) with

Example 3: Find the optimal control path that will Maximize ( )

Subject to ( ) And ( ) Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ()

(ii)

The state equation is ( )

and for this problem

(iii)

The costate equation is

and for this problem

( ) (iv) The transversality condition is ( ) ( ) Equations (ii) and (iii) constitute differential equation system for this problem. Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the transversality condition (iv) and initial condition ( ) . We can first solve for by straight integration of (iii) to get ( )
5

) ( )

Using the transversality condition ( )

in (vi), we have

Thus the optimal costate path is ( ) ( ) Using (t) from (vi) in (i), the corresponding optimal control path is ( ) Using (vii) in (ii), we have ( ) ( )

and by integration, we have ( ) Setting t=0 in the preceding equation, we get variable is ( ) The optimal control path is ( ) ( ) ( ) with costate variable ( ) Example 4: Find the optimal control path that will Maximize ( ) ( ) ( ) . . Thus the optimal path for state .

Subject to ( ) And ( ) Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is ( )

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ( ) ()

(ii)

The state equation is ( )

and for this problem

(iii)

The costate equation is

and for this problem

( ) (iv) The transversality condition is ( ) ( ) Equations (ii) and (iii) constitute differential equation system for this problem. Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the transversality condition (iv) and initial condition ( ) . We can write equation (iii) as ( ) This linear first order ordinary differential equation for

and its integrating factor is

Multiplying (v) by this yields

, we have

[ Integrating both sides, we have [ ]

( ) Using the transversality condition ( )

( ) ( ) in (vi), we have

Thus the optimal costate path is ( ) ( ) ( ) Using (t) from (vii) in (i), the corresponding optimal control path is ( ) Using (viii) in (ii), we have ( Solving (ix) by integrating factor method, we have ( )
7

) (

) ( )

( ) Setting t=0 in the preceding equation, we get state variable is ( ) The optimal control path is ( ) ( ) ( ( ) ) with costate variable ( ) ( ) ( ) -

. Thus the optimal path for

Example 5: Find the optimal control path that will Maximize ( )

Subject to ( ) And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is ( )

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ( ) ()

(ii)

The state equation is

and for this problem ( )

(iii) (iv)

The costate equation is

and for this problem ( )

The transversality condition is ( ) ( ) Equations (ii) and (iii) constitute differential equation system for this problem.
8

Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the transversality condition (iv) and initial condition ( ) . We can write equation (iii) as ( ) This linear first order ordinary differential equation for

and its integrating factor is

Multiplying (v) by this yields

, we have

[ Integrating both sides, we have

( ) ( ) ( ) Using the transversality condition ( ) in (vi), we have Thus the optimal costate path is ( ) ( ) Using (t) from (vi) in (i), the corresponding optimal control path is ( ) Using (vii) in (ii), we have ( )

( The integrating factor for (viii) is

Multiplying (viii) by

, we have

this yields [ Integrating both sides, we have [ ] ( )

9

( ) ( )

( ) Setting t=0 in the preceding equation, we get state variable is ( ) The optimal control path is ( ) costate variable ( ) Example 6: If ( ) ( ) .

. Thus the optimal path for

( )

with

Subject to ( ) And ( ) (a) Write Hamiltonian and conditions of maximum principal. (b) Express conditions of maximum principal as first order differential system for . Solution: (a) The Hamiltonian for this problem is (b)

The various components of maximum principal are as follows: (v) ( ) ( ) ()

(vi)

The state equation is

and for this problem ( )

(vii)

The costate equation is

and for this problem ( )

(viii) The transversality condition is ( ) (c) Using (i) in (ii) and (iii), we have (
10

( )

( ) And ( Or ( ) Equation (v) and (vi) form first order differential system for We can solve it for subject to transversality condition ( ) and initial condition ( ) )

Section 20.2 from Fundamental methods of Mathematical Economics, McGraw Hill 2005, 4th Edition. By A. C. Chiang & Kevin Wainwright is covered. Fixed terminal point The optimal control problem is Maximize Subject to ( ( ( ) Both ) ) ( ) and u(t) is unrestricted.

are given. u(t) is unrestricted its possible for u to be limited to a control region.

The Hamiltonian is defined as ( ) ( ) ( ) ( ) The various components of maximum principal are as follows: (v) (vi) (vii) and (state equation) (costate equation)

In this case, no transversality condition is needed as the terminal condition itself should provide information to definitize one constant. Example 1: Find the optimal control path that will

11

Maximize

Subject to ( ) And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (v) ( ) ()

(vi)

The state equation is

and for this problem ( )

(vii)

The costate equation is

and for this problem

( ) Equations (ii) and (iii) constitute differential equation system for this problem. Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the initial condition ( ) and terminal condition ( ) Combining (i) and (ii), we have ( ) We can first solve for by straight integration of (iii) to get ( ) Using (v) in (iv), we have ( and by integration, we have ) ( ) ( )

12

( ) and terminal condition ( ) Using ( ) in (vi), we have Using ( ) in (vi), we have

)( ) ( ) and

( Or Thus the optimal path for state variable is ( ) ( ) (

The optimal path for costate variable and control variable is ( ) And ( ) Example 2: Find the optimal control path that will Maximize Subject to And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is . ( ( ) ) ( ( )) ( ( ))

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ()

13

(ii)

The state equation is

and for this problem ( )

(iii)

The costate equation is

and for this problem ( )

Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the initial condition ( ) and terminal condition ( ) Combining (i) and (ii), We have ( ) We can first solve for by straight integration of (iii) to get ( ) from (v) in (iv), we have ( ( ) And by straight integration, we have ( ) ) ( )

Substituting

( ) Using ( ) Using ( )

( ) in (vii), we have ( in (vii), we have ( ) and , we have )

Solving (viii) and (ix) for

Thus the optimal path for state variable is ( ) The optimal path for costate variable and control variable is ( ) and ( )
14

( (

( )) ( ))

Example 3: Find the optimal control path that will Maximize Subject to And ( ) Solution: Step I: Write the Hamiltonian and calculate its partial derivatives with respect to The Hamiltonian for this problem is ( ) . ( )

Step II: Write the various components of maximum principal. The various components of maximum principal are as follows: (i) ( ) ()

(ii)

The state equation is

and for this problem ( )

(iii)

The costate equation is

and for this problem ( )

Step III: We want to find ( ), ( ) and ( ) from equations (i) to (iii) satisfying the initial condition ( ) and terminal condition ( ) Combining (i) and (ii), We have

( ) We can first solve (iii) for

15

( ) This linear first order ordinary differential equation for

and its integrating factor is

Multiplying (v) by this yields

, we have

[ Integrating both sides, we have ( ) Substituting (

) ( )

from (vi) in (iv), we have ( )

The integrating factor for (vii) is

Multiplying (vii) by

, we have

this yields [ Integrating both sides, we have [ ] ]

( )

) (

( ) Using ( ) Using ( )

( ) in (viii), we have ( ) in (viii), we have ( )

Solving (ix) and (x) for

and

, we have

16

Thus the optimal path for state variable is ( ) [ ]

The optimal path for costate variable and control variable is ( ) and ( ) ( ( )) ( ( ))

Section 20.3 Fundamental methods of Mathematical Economics, McGraw Hill 2005, 4th Edition. By A. C. Chiang & Kevin Wainwright is covered.

17

The optimal control problem is Maximize Subject to and conditions ( ( ) )

18

t is time, y(t) is known as state variable, u(t) as control variable. Hamiltonian function and Pontryagins maximum principal The Hamiltonian is defined as ( ) ( ) ( ) ( )( ) Where ( ) is known as the costate variable, H denotes the Hamiltonian and is a function of four variables: t, y, u and Like the Lagrange multiplier, the costate variable ( ) measures the shadow price of the state variable. The various components of maximum principal are as follows: (viii) (ix) (x) and (state equation) (costate equation)

Current value Hamiltonian and maximum principal The current value Hamiltonian is defined as ( ) ( ) ( ) (

Where ( ) is known as the costate variable, H denotes the Hamiltonian and is a function of four variables: t, y, u and Like the Lagrange multiplier, the costate variable ( ) measures the shadow price of the state variable. The various components of maximum principal are as follows: (i) (ii) (iii) and (state equation) (costate equation)

1. Find the optimal control path that will Maximize ( ) Subject to (d) Write present value Hamiltonian and conditions of maximum principal. (e) Write current value Hamiltonian and conditions of maximum principal. 2. Find the optimal control path that will Maximize Subject to
19

(a) Write current value Hamiltonian and conditions of maximum principal (b) Express conditions of maximum principal as first order differential system for . (c) Solve the differential system obtained in part (b) for ( ) ( ). Also find the control variable ( ). 3. Find the optimal control path that will Maximize ( ) Subject to (a) Write current value Hamiltonian and conditions of maximum principal. (b) Express conditions of maximum principal as first order differential system for . (f) Solve the differential system obtained in part (b) for ( ) ( ). Also find the control variable ( ). 4. Find the optimal control path that will Maximize ( ) Subject to (a) Write current value Hamiltonian and conditions of maximum principal (b) Express conditions of maximum principal as first order differential system for . (c) Solve the differential system obtained in part (b) for ( ) ( ). Also find the control variable ( ).

20