Automatica 41 (2005) 915 920 www.elsevier.

com/locate/automatica

Technical Communique

A closed-form optimal control for linear systems with equal state and input delays
Michael Basin , Jesus Rodriguez-Gonzalez
Department of Physical and Mathematical Sciences, Autonomous University of Nuevo Leon, Apdo postal 144-F, C.P. 66450, San Nicolas de los Garza, Nuevo Leon, Mexico Received 11 October 2003; received in revised form 19 November 2004; accepted 8 December 2004

Abstract This paper presents the optimal regulator for a linear system with equal delays in state and input and a quadratic criterion. The optimal regulator equations are obtained using the maximum principle. Performance of the obtained optimal regulator is veried in the illustrative example against the best linear regulators available for the linear system without delays and for a rational approximation of the original time delay system. Simulation graphs demonstrating better performance of the obtained optimal regulator are included. 2005 Elsevier Ltd. All rights reserved.
Keywords: Optimal control; Time-delay system; Linear system

1. Introduction Although the optimal control (regulator) problem for linear system states was solved, as well as the ltering one, in the 1960s (Kwakernaak & Sivan, 1972; Fleming & Rishel, 1975), the optimal control problem for linear systems with delays is still open, depending on the delay type, specic system equations, criterion, etc. Such complete reference books in the area as (Kolmanovskii & Shaikhet, 1996; Kolmanovskii & Myshkis, 1999; Malek-Zavarei & Jashmidi, 1987; Dion, Dugard, & Fliess, 1999; Boukas & Liu, 2002) note, discussing the maximum principle (Kharatashvili, 1967) or the dynamic programming method (Oguztoreli, 1966) for systems with delays, that nding a particular explicit form of the optimal control function might still remain difcult. A specic form of the criterion must also be taken into account: the studies mostly focused on the time-optimal
This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor R. Middleton under the direction of Editor P. Van den Hof. Corresponding author. E-mail addresses: mbasin@fcfm.uanl.mx (M. Basin), jgrg17@yahoo.com.mx (J. Rodriguez-Gonzalez). 0005-1098/$ - see front matter 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.automatica.2004.12.004

criterion (see the paper (Oguztoreli, 1963) for linear systems) or the quadratic one (Eller, Aggarwal, & Banks, 1969; Alekal, Brunovsky, Chyung, & Lee, 1971; Delfour, 1986; Uchida, Shimemura, Kubo, & Abe, 1988). There also exists a considerable bibliography related to the robust control problem for time-delay systems (such as (Dugard & Verriest, 1998; Mahmoud, 2000)). The current state-of-the-art in the time-delay system theory has recently been reviewed in two survey papers (Gu & Niculescu, 2003; Richard, 2003). This paper concentrates on the solution of the optimal control problem for a linear system with equal delays in state and input and a quadratic criterion. Using the maximum principle (Pontryagin, Boltyanskii, Gamkrelidze, & Mishchenko, 1962; Kharatashvili, 1967), the solution to the stated optimal control problem is obtained in a closed form, i.e., it is represented as a linear feedback control law, whose gain matrix satises an ordinary differential (quasi-Riccati) equation, which does not contain time-advanced or delayed arguments and does not depend on the state variables. The obtained optimal regulator makes an advance with respect to general optimality results for time-delay systems (such as given in Alekal et al., 1971; Malek-Zavarei & Jashmidi, 1987; Kolmanovskii &

916

M. Basin, J. Rodriguez-Gonzalez / Automatica 41 (2005) 915 920

Shaikhet, 1996; Kolmanovskii & Myshkis, 1999), since it is realizable using only two equations: a delay-differential equation for the state and an ordinary differential one for the gain matrix. Taking into account that the state space of a delayed system is innite-dimensional (Malek-Zavarei & Jashmidi, 1987), this seems to be a signicant advantage. Finally, performance of the obtained optimal control for a linear system with equal delays in state and input and a quadratic criterion is veried in the illustrative example against the best linear regulator available for the linear system without delays. The simulation results show a signicant (about twenty times) difference in the values of the cost function in favor of the obtained optimal regulator. Moreover, the obtained optimal regulator is compared to the best linear regulator based on a rational approximation of the original time-delay system. In this case, the simulations show that the approximation-based regulator generates an increase in the problem dimension, producing additional computational difculties, and yields still unsatisfactory values of the cost function in comparison to the optimal regulator. The paper is organized as follows. Section 2 states the optimal control problem for a linear system with equal delays in state and input. The solution to the optimal control problem is given in Section 3. The proof of the obtained results, based on the maximum principle, is given in appendix. Section 4 presents an example illustrating the efciency of control provided by the obtained optimal regulator for linear systems with equal delays in state and input against the best linear regulators available for the linear system without delays and for a rational approximation of the original time-delay system. Simulation graphs demonstrating better performance of the obtained optimal regulator are included. 2. Optimal control problem for linear system with equal state and input delays Consider a linear system with time delay in the state x(t) = a(t)x(t h) + B(t)u(t h), (1)

where R(s) and L(s) are positive and nonnegative-denite symmetric continuous matrix functions, is a nonnegativedenite symmetric matrix, and T1 > t0 + h is the terminal time (control horizon). The optimal control problem is to nd the control u (t), t [t0 , T1 ], that minimizes the criterion J along with the trajectory x (t), t [t0 , T1 ], generated upon substituting u (t) into the state equation (1). The solution to the stated optimal control problem is given in the next section and then proved using the maximum principle (Pontryagin et al., 1962; Kharatashvili, 1967) in appendix. 3. Optimal control problem solution The solution to the optimal control problem for the linear system with state delay (1) and the quadratic criterion (2) is given as follows. The optimal control law is given by u (t) = (R(t))1 B T (t)Q(t)x(t),
T Q(t) = L(t) Q(t)M1 (t)a(t) a T (t)M1 (t)Q(t) Q(t)B(t)R 1 (t)B T (t)Q(t),

(3)

where the matrix function Q(t) satises the matrix equation (4)

with the terminal condition Q(T1 ) = . The auxiliary matrix M1 (t) is dened as M1 (t) = (jx(t h)/jx(t)), whose value is equal to zero, M1 (t) = 0, if t [t0 , t0 + h), and is equal to I, M1 (t) = I , where I is the identity matrix of dimension n, if t t0 + h. Upon substituting the optimal control (3) into the state equation (1), the optimally controlled state equation is obtained x(t) = a(t)x(t h) + B(t)R 1 (t h)B T (t h) Q(t h)x(t h), (5)

with the initial condition x(s) =

(s), s [t0 h, t0 ], where x() C([t0 h, t0 ]; R n ) is the system state, u(t) R m is the control input variable, coefcients a(t) and B(t) are continuous functions of time, and
(s) is a continuous function given in the interval [t0 h, t0 ]. Note that the state of a delayed system (1) is innite-dimensional (Malek-Zavarei & Jashmidi, 1987). Under the made assumptions, existence of the unique solution of the unforced Eq. (1) with zero input is assured by the results given in (Malek-Zavarei & Jashmidi, 1987) (see Theorem 3.2.1 on p. 62 therein and the preceding comments). The quadratic cost function to be minimized is dened as follows: 1 T1 T 1 u (s)R(s)u(s) ds J = [x(T1 )]T [x(T1 )] + 2 2 t0 1 T1 T x (s)L(s)x(s) ds, (2) + 2 t0

with the initial condition x(s) =

(s), s [t0 h, t0 ]. Note that system (1) is not controlled in the interval [t0 , t0 + h). It should also be noted that the obtained optimal regulator makes an advance with respect to general optimality results for time delay systems (such as given in Alekal et al., 1971; Malek-Zavarei & Jashmidi, 1987; Kolmanovskii & Shaikhet, 1996; Kolmanovskii & Myshkis, 1999), since (a) the optimal control law is given explicitly and not as a solution of a system of integro-differential or PDE equations, and (b) the quasi-Riccati equation for the gain matrix does not contain any time-advanced arguments and does not depend on the state variables and, therefore, leads to a conventional two points boundary-valued problem generated in the optimal control problems with quadratic criterion and nite horizon (see, for example, Kwakernaak & Sivan, 1972). Thus, the obtained optimal regulator is realizable using a delay-differential equation for the state and an ordinary differential one for the gain matrix. Remark. Existence of the unique solution to the forwardtime Cauchy problem for the linear state delay equation

M. Basin, J. Rodriguez-Gonzalez / Automatica 41 (2005) 915 920

917

(5) readily follows from the results given in Malek-Zavarei and Jashmidi (1987) (see Theorem 3.2.1 on p. 62 therein and the preceding comments), in view of continuity of the initial condition and coefcients of Eq. (1) and continuity of the weight matrix functions in the criterion (2). Thus, the obtained optimal regulator (3)(5) can be equally applied to unstable as well as stable plants (1), since the forward-time Cauchy problem for linear state delay systems (5) always has a unique solution in the whole interval [t0 h, ), which does not diverge to innity for a nite time. 4. Example This section presents an example of designing the optimal regulator for a system (1) with a criterion (2), using the scheme (3)(5), and comparing it to the regulator where the matrix Q is selected as in the optimal linear regulator for a system without delays. Let us start with a scalar linear system x(t) = 10x(t 0.25) + u(t 0.25), (6)

150

criterion

100

50

0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

time
10 0

control

10 20 30 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

time
Fig. 1. Best linear regulator available for linear systems without delays. Graphs of criterion (7) J (t) and control (8) u(t) in the interval [0, 0.5].

with the initial conditions x(s) = 1 for s [0.25, 0]. The control problem is to nd the control u(t), t [0, T1 ], T1 = 0.5, that minimizes the criterion J= 1 2
T1 0

u2 (t) dt +
0

T1

x 2 (t) dt .

(7)

The results of applying the regulator (8)(9) to the system (6) are shown in Fig. 1, which presents the graphs of the criterion (7) J (t) and the control (8) u(t) in the interval [0, T1 ]. The value of criterion (7) at the nal moment T1 =0.5 is J (0.5) = 140.32. Let us now apply the optimal regulator (3)(5) for linear states with time delay to the system (6). Control law (3) takes the same form as (8) u (t) = Q (t)x(t), where Q (t) satises the equation Q (t) = 1 20Q (t)M1 (t) Q 2 (t), Q (0.5) = 0, (12) (11)

In other words, the control problem is to minimize the overall energy of the state x using the minimal overall energy of control u. Let us rst construct the regulator where the control law and the matrix Q(t) are calculated in the same manner as for the optimal linear regulator for a linear system without delays, that is u(t) = R 1 (t)B T (t)Q(t)x(t) (see Kwakernaak & Sivan, 1972 for reference). Since B(t) = 1 in (6) and R(t) = 1 in (7), the optimal control is actually equal to u(t) = Q(t)x(t), where Q(t) satises the Riccati equation Q(t) = a T (t)Q(t) Q(t)a(t) + L(t) Q(t)B(t)R 1 (t)B T (t)Q(t), with the terminal condition Q(T1 ) = . Since a(t) = 10 in (6) and L(t) = 1 and = 0 in (7), the last equation turns to Q(t) = 1 20Q(t) Q2 (t), Q(0.5) = 0, (9) (8)

where M1 (t) = 0 for t [0, 0.25) and M1 (t) = 1 for t [0.25, 0.5]. Since the solution Q (t) of Eq. (12) is not smooth, it has been numerically solved with the approximating terminal condition Q (0.5) = 0.04, in order to avoid chattering. Upon substituting control (11) into (6), the optimally controlled system takes the same form as (10) x(t) = 10x(t 0.25) + Q (t 0.25)x(t 0.25). (13)

whose solution can be found analytically and is equal to Q(t) = [1 exp( 404(0.5 t))][10 + 101 exp( 404(0.5 t))(10 101)]1 . Upon substituting the control (8) into (6), the controlled system takes the form x(t) = 10x(t 0.25) + Q(t 0.25)x(t 0.25). (10)

The results of applying the regulator (11)(13) to the system (6) are shown in Fig. 2, which presents the graphs of criterion (7) J (t) and the control (11) u (t) in the interval [0, T1 ]. The value of criterion (7) at the nal moment T1 = 0.5 is J (0.5) = 6.218. There is a signicant (about twenty times) improvement in the values of the cost function in comparison to the preceding case, due to the optimality of the regulator (3)(5) for a linear system with equal delays in state and input. Finally, let us compare the optimal regulator (3)(5) to the best linear regulator based on a rational approximation of the time-delay system (6). The input-state transfer function of (6), G(s)=(s exp(sh)10)1 , h=0.25, is approximated

918
7 6 5 4 3 2 1 0 0 1 0 0.05 0.1 0.15 0.2 0.25

M. Basin, J. Rodriguez-Gonzalez / Automatica 41 (2005) 915 920

12 10

criterion

criterion

8 6 4 2 0

0.3

0.35

0.4

0.45

0.5 0.5 0 0.5 1 1.5 2 2.5

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

time

time

control

1 2 3 4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

control

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

time
Fig. 2. Optimal regulator obtained for linear systems with equal delays in state and input. Graphs of criterion (7) J (t) and optimal control (11) u (t) in the interval [0, 0.5].

time
Fig. 3. Regulator based on a rational approximation of the original time-delay system. Graphs of criterion (7) J (t) and control (15) u1 (t) in the interval [0, 0.5].

by a rational function up to the rst order of h: G1 (s) = s + s 2 h 10 + O(h2 ). Note that since the third derivative of the solution of Eq. (13) is virtually a constant equal to zero, the omission of more than quadratic powers of s should not introduce a considerable error into the dynamics. This approximation results in the time-domain realization (h = 0.25) 0.25(d2 (x(t))/dt 2 ) + d(x(t))/dt = 10x(t) + u(t), which is represented in the standard form of two rst-order differential equations x(t) = z(t), z(t) = 40x(t) 4z(t) + 4u(t), (14)

producing additional computational difculties, and yields still unsatisfactory values of the cost function in comparison to the optimal regulator (3)(5).

Appendix Proof of the optimal control problem solution. Dene the Hamiltonian function (Pontryagin et al., 1962; Kharatashvili, 1967) for the optimal control problem (1), (2) as
1 H (x, u, q, t) = 2 (uT R(t)u + x T L(t)x)

+ q T [a(t)x1 + B(t)u1 ],

(17)

with the initial conditions x(0) = 1, z(0) = 0.25. The control law is calculated as the optimal control for the linear system without delays (14):
T u1 (t) = B1 Q1 (t)[x(t) z(t)]T ,

where x1 (t) = x(t h) and u1 (t) = u(t h). Applying the maximum principle condition jH /ju = 0 to this specic Hamiltonian function (17) yields

(15)

jH /ju = 0 R(t)u(t) + (ju1 (t)/ju)T B T (t)q(t) = 0.

Upon denoting (ju1 (t)/ju) = M(t), the optimal control law is obtained as u (t) = R 1 (t)M T (t)B T (t)q(t). Taking linearity and causality of the problem into account, the co-state q(t) is sought as a linear function in x(t) to readily satisfy the transversality condition (Pontryagin et al., 1962; Kharatashvili, 1967) induced by the cost function (2): q(t) = Q(t)x(t), (18)

and the two-dimensional matrix Q1 (t) satises the Riccati equation

T Q1 (t) = a1 Q1 (t) Q1 (t)a1 + L1 T Q1 (t)B1 B1 Q1 (t),

(16)

with the terminal condition Q1 (T1 ) = 0, where B1 = [0 4]T , a1 = [0 1|40 4] and L1 = diag[1 0]. Control (14) is then substituted into the original time-delay system (6). The results of applying the regulator (14)(16) to system (6) are shown in Fig. 3, which presents the graphs of criterion (7) J (t) and control (15) u1 (t) in the interval [0, T1 ]. The value of criterion (7) at the nal moment T1 = 0.5 is J (0.5) = 10.296. Thus, the simulation results show that application of the regulator (14)(16), based on a rational approximation, leads to an increase in the problem dimension,

where Q(t) is a square symmetric matrix of dimension n. This yields the complete form of the optimal control u (t) = R 1 (t)M T (t)B T (t)Q(t)x(t), (19)

M. Basin, J. Rodriguez-Gonzalez / Automatica 41 (2005) 915 920

919

and the transversality condition for q(T1 ) implies that q(T1 ) = jJ /jx(T1 ) = x(T1 ), therefore, Q(T1 ) = . Using the co-state equation dq(t)/dt = jH /jx and denoting (jx1 (t)/jx) = M1 (t) yields
T dq(t)/dt = L(t)x(t) + a T (t)M1 (t)q(t),

(jx(t + h)/jx(t)) = I , if t t0 , and, consequently, M1 (t) = (jx(t h)/jx(t)) =I , where I is the identity matrix of dimension n, if t t0 + h. Let us now nd the value of matrix M(t) for this problem. Note that integrating (24) also yields the formula x(t + h) = x( ) +
t+h t+h

(20)

and substituting (18) into (20), we obtain Q(t)x(t) + Q(t) d(x(t))/dt T = L(t)x(t) a T (t)M1 (t)Q(t)x(t).

a(s)x(s h) ds +

B(s) (27)

R 1 (s h)B T (s h)Q(s h)x(s h) ds, (21)

Substituting the expression for x(t) from the state equation (1) into (21) yields Q(t)x(t) + Q(t)a(t)x(t h) + Q(t)B(t)u(t h) T = L(t)x(t) a T (t)M1 (t)Q(t)x(t).

(22)

In view of linearity of the problem, differentiating the last expression in x does not imply loss of generality. Upon substituting the optimal control law (19) into (22), taking into account that (jx(t h)/jx(t))=M1 (t) and (ju(t h)/ju)= M(t), and differentiating the equation (22) in x, it is transformed into the quasi-Riccati equation
T Q(t) = L(t) Q(t)M1 (t)a(t) a T (t)M1 (t)Q(t) Q(t)B(t)M(t)R 1 (t)M T (t)B T (t)Q(t),

(23)

where t t0 and is taken in the interval [t, t + h]. Since the integral terms do not depend on u(t), jx(t + h)/ju(t) = jx( )/ju(t). Inverting the last equality implies that ju(t)/jx(t + h) = ju(t)/jx( ) for any [t, t + h]. Hence, u(t) = K1 (t)x( ), where K1 (t) is the same for any [t, t + h], therefore, K1 (t) = K1 = const in the interval [t, t + h]. Analogously, since the integral terms in (27) do not depend on u(t + h) as well, u(t + h) = K2 x( ), where K2 (t) = K2 = const. Thus, ju(t)/ju(t +h)=K1 (K2 )1 =K =const for any t t0 . Since ju(t)/ju(t) = I , K = I and M(t) = (ju(t h)/ju(t)) = I for t t0 + h. In addition, analysis of formula (25) implies that jx(t0 + r)/ju(t) = jx(t0 + r)/ju(t + h) for any t [t0 h, t0 ] and any r [0, h], yielding again M(t) = I for t [t0 , t0 + h]. Thus, Eq.(23) for Q(t) nally takes the form (4). The optimal control problem solution is proved.

with the terminal condition Q(T1 ) = . Let us obtain the value of M1 (t). By denition, M1 (t) = (jx(t h)/jx(t)). Substituting the optimal control law (19) into Eq. (1) gives x(t) = a(t)x(t h) + B(t)R 1 (t h)B T (t h) Q(t h)x(t h),

Acknowledgements The authors thank the Mexican National Science and Technology Council (CONACyT) for nancial support in the framework of Grant 39388-A. The authors are grateful to the Associate Editor and reviewers for their useful comments, which helped to improve this paper. References
Alekal, Y., Brunovsky, P., Chyung, D. H., & Lee, E. B. (1971). The quadratic problem for systems with time delays. IEEE Transactions on Automatic Control, 16, 673687. Boukas, E.-K., & Liu, Z.-K. (2002). Deterministic and stochastic timedelay systems. Babel: Birkhauser. Delfour, M. C. (1986). The linear quadratic control problem with delays in state and control variables: A state space approach. SIAM Journal of Control and Optimization, 24, 835883. Dion, J. M., Dugard, J. L., & Fliess, M. (1999). Linear time-delay systems. London: Pergamon. Dugard, J. L., & Verriest, E. I. (1998). Stability and control of time-delay systems. Berlin: Springer. Eller, D. H., Aggarwal, J. K., & Banks, H. T. (1969). Optimal control of linear time-delay systems. IEEE Transactions on Automatic Control, 14, 678687. Fleming, W. H., & Rishel, R. W. (1975). Deterministic and stochastic optimal control. Berlin: Springer. Gu, K., & Niculescu, S.-I. (2003). Survey on recent results in the stability and control of time-delay systems. ASME Transactions, Journal of Dynamic Systems, Measurement, and Control, 125, 158165.

(24)

with the initial condition x(s) =

(s), s [t0 h, t0 ]. Integrating (24) yields x(t0 + h) = x(t0 ) +
t0 +h t0 t0 +h t0

a(s)x(s h) ds +

B(s) (25)

R 1 (s h)B T (s h)Q(s h)x(s h) ds.

Analysis of formula (25) shows that x(t) does not depend on x(t h), if t [t0 , t0 + h). Therefore, M1 (t) = 0 for t [t0 , t0 + h). On the other hand, if t t0 , the following formula is valid for the solution x(t) of Eq. (24) x(t + h) = x(t) +
t t+h t+h t

a(s)x(s h) ds +

B(s) (26)

R 1 (s h)B T (s h)Q(s h)x(s h) ds.

Analysis of formula (26) shows that x(t + h) depends on x(t) only via the non-integral term. Therefore,

920

M. Basin, J. Rodriguez-Gonzalez / Automatica 41 (2005) 915 920 Oguztoreli, M. N. (1963). A time optimal control problem for systems described by differential difference equations. SIAM Journal of Control, 1, 290310. Oguztoreli, M. N. (1966). Time-lag control systems. New York: Academic Press. Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., & Mishchenko, E. F. (1962). The mathematical theory of optimal processes. New York: Wiley-Interscience. Richard, J.-P. (2003). Time-delay systems: An overview of some recent advances and open problems. Automatica, 39, 16671694. Uchida, K., Shimemura, E., Kubo, T., & Abe, N. (1988). The linearquadratic optimal control approach to feedback control design for systems with delay. Automatica, 24, 773780.

Kharatashvili, G. L. (1967). A maximum principle in external problems with delays. Mathematical theory of control. New York: Academic Press. Kolmanovskii, V. B., & Myshkis, A. D. (1999). Introduction to the theory and applications of functional differential equations. New York: Kluwer. Kolmanovskii, V. B., & Shaikhet, L. E. (1996). Control of systems with aftereffect. Providence: American Mathematical Society. Kwakernaak, H., & Sivan, R. (1972). Linear optimal control systems. New York: Wiley-Interscience. Mahmoud, M. S. (2000). Robust control and ltering for time-delay systems. New York: Marcel Dekker. Malek-Zavarei, M., & Jashmidi, M. (1987). Time-delay systems: Analysis, optimization and applications. Amsterdam: North-Holland.