Recursively Updated Least Squares Based Modification Term for

Adaptive Control
Girish Chowdhary and Eric Johnson
Abstract We present an approach for combining standard
recursive least squares based regression with proven direct
model reference adaptive control using a recursively updated
modification term. This approach is applicable to adaptive
control problems where the uncertainty can be linearly parameterized. The combined training law drives the adaptive weights
smoothly to a recursively updated least squares estimate of the
ideal weights and is shown to have a stability proof. Expected
improvement in performance of the adaptive law is validated
through simulation.

I. INTRODUCTION
This paper presents a combined direct and indirect approach to adaptive control for a class of adaptive control
problems where the uncertainty can be linearly parameterized using a weighted combination of known nonlinear
functions. The presented approach augments any baseline
direct adaptive law by smoothly incorporating a recursively
updated least squares estimate of the ideal weights through
a modification term. The inclusion of the least squares based
estimate provides additional information to the adaptive law
and aids in convergence of the weights to optimal estimates
of the ideal weights. The unique idea behind this approach
is that it combines proven recursive least squares based
methods such as Kalman filter based least squares regression
[5] with standard gradient based adaptive laws while ensuring
that the system states remain uniformly ultimately bounded.
In plants where the plant uncertainty can be linearly
parameterized using a weighted combination of known nonlinear functions, the standard approach to the design of
adaptive controllers is to use a weight training law that
attempts to estimate the ideal weights in order to cancel
the uncertainty. In the direct adaptive control methodology,
adaptive laws are designed using a Lyapunov based approach
and can be thought of as attempting to minimize a quadratic
cost function of the instantaneous tracking error by using
a gradient descent type method [9]. These adaptive laws
however, are susceptible to local adaptation and weight drift
[11],[4],[15]. Furthermore, it can be shown that adaptive
laws formulated using the gradient descent methodology are
always at most rank 1 [2]. In order to ensure that adaptive
weights remain bounded, Ioannou et al. [4] suggested the use
of a modification term which adds damping to the adaptive
law. Narendra et al. [11] suggested the e modification term
which uses the norm of the tracking error to scale the added
damping. The methods are primarily designed to ensure
G. Chowdhary and Professor E. N. Johnson are with the Daniel
Guggenheim school of aerospace engineering at the Georgia Institute
of Technology, Atlanta GA, Girish.Chowdhary@gatech.edu,

Eric.Johnson@ae.gatech.edu

boundedness of adaptive weights and they do little to ensure

the convergence of the weights to ideal values. In order to
improve weight convergence, Volyanskyy et al. suggested
a novel Q modification [16] term which uses an integral
over a moving window of the system uncertainty to improve
weight convergence. Chowdhary and Johnson suggested the
Concurrent Learning [2] method for incorporating long term
learning in adaptive laws by using past and current data
concurrently for adaptation. Concurrent learning constrains
the weight update based on simultaneous gradient descent
over a set of stored data point to the null space of the weight
update based on current data. This augments the rank of
the adaptive law while ensuring that it stays responsive to
dynamic changes in the environment.
In this paper we maintain the idea of using past and current
data concurrently for learning, however, we use optimal
recursive least squares for adaptation based on past data
combined with gradient descent for adaptation based on
current data. This choice is motivated from the well known
fact in linear algebra that the best linear fit for a given set of
data can be obtained by solving the least squares problem [1].
The linear least squares regression problem can be viewed as
a special case of the discrete Kalman filter and can be solved
in a recursive way such that the update of the estimates
become linear function of the error in the output [5]. Nguyen
has shown improved tracking performance in direct adaptive
controllers by using recursive least squares directly for
weight adaptation [12]. We differ from Nguyens approach
by using recursive least squares to update a modification
term rather than using it directly as the adaptive law. This
allows us to augment any baseline law without affecting
its responsiveness and other desirable properties. We show
that inclusion of the modification term does not affect the
uniform ultimate boundedness properties of the adaptive law.
Furthermore, we show that the convergence of the weights
to their ideal values is guaranteed if the system states are
uniformly completely observable. The presented approach is
recursive and hence conducive for online implementation.
To that effect, we show the feasibility of using this approach
on a wing rock dynamics model of a fixed wing aircraft.
The simulation results confirm expected improvement in
performance.
II. M ODEL R EFERENCE A DAPTIVE C ONTROL
This section discusses the formulation of Model Reference
Adaptive Control, the reader is referred to [9],[6],[8],[7] for
further details. Let x(t) <n be the known state vector, let

u < denote the control input, and consider the following

system where the uncertainty can be linearly parameterized:

For this case it is well known that the following adaptive

law
= (x)eT P BW
W
(8)

x = Ax(t) + B(u(t) + (x(t))),

where W is a positive definite matrix of appropriate dimensions results in ultimate boundedness of the weights and
e(t) 0 if the reference input r(t) is persistently exciting
[11],[4],[15]. Equation 8 will be referred to as the baseline
adaptive law.
Case 2: Unstructured Uncertainty: In this more general
case it is only known that the uncertainty (x) can be
linearly parameterized; hence the adaptive part of the control
law is represented using a Radial Basis Function (RBF)
Neural Network (NN):

(1)

where A <nn , B <n , and (x) is a continuously

differentiable function representing the scalar uncertainty. We
assume that the pair A, B is completely controllable.
A reference model can be designed that characterizes the
desired response of the system:
x rm = Arm xrm (t) + Brm r(t),

(2)

where Arm <nn is a Hurwitz matrix and r(t) denotes

a bounded reference signal. A tracking control law consisting
of a linear feedback part upd = Kx, a linear feedforward
part ucrm = Kr [xTrm , r(t)]T , and an adaptive part uad (x) is
proposed to have the following form:
u = ucrm + upd uad .

(3)

Define the tracking error e as e(t) = xrm (t) x(t), with

an appropriate choice of ucrm such that Bucrm = (Arm
A)xrm + Brm r(t), the tracking error dynamics are found to
have the form:
e = Am e + B(uad (x) (x)),

(4)

where the baseline full state feedback controller upd =

Kx is assumed to be designed such that Am = A BK
is a Hurwitz matrix. Hence for any positive definite matrix
Q <nn , a positive definite solution P <nn exists to
the Lyapunov equation:
ATm P + P Am + Q = 0.

(5)

We now state the following assumptions:

Assumption 1: For simplicity, assume that B =
[0, 0, ..., 1]T and (x) <.
Assumption 2: The uncertainty (x) can be linearly
parameterized, that is, there exist a vector of constants
W = [W1 , W2 , ...., Wm ]T and a vector of continuously
differentiable functions (x) = [1 (x), 2 (x), ...., m (x)]T
such that
(x) = W T (x).

(6)

This results in the following two cases:

Case 1: Structured Uncertainty: In this case the form of
the linearly parameterized uncertainty is known, that is the
denote the
mapping (x) is known. In this case letting W
estimate W the adaptive law can be written as
T (x).
uad (x) = W

(7)

= W
On examining equation 4 we see that if W
then the tracking error dynamics are globally exponentially
stable. Hence, we desire to design an adaptive law such that
(t) W asymptotically.
W

T (x).
uad (x) = W

(9)

< and = [1, 2 (x), 3 (x), ....., l (x)] is

where W
a vector of known radial basis functions. For i = 2, 3..., l let
ci denote the RBF centroid and i denote the RBF width
then for each i The radial basis functions are given as:
l

i (x) = ekxci k

/i

(10)

Appealing to the universal approximation property of RBF

NN we have that given a fixed number of radial basis
functions l there exists a vector of ideal weights W <l
and a positive real number  such that
(x) = W T (x) + ,

(11)

where  can be made arbitrarily small given sufficient number

of radial basis functions [13]. For a fixed number of radial
basis functions, the control design task is to design an
(t) W asymptotically. This
adaptive law such that W
will result in a uniform ultimate bound on the tracking error
of equation 4 which is a function of .
III. R ECURSIVE L EAST S QUARES BASED C OMBINED
D IRECT I NDIRECT A DAPTIVE C ONTROL
In this section we propose a combined direct and indirect
adaptive law that uses a recursive optimal least squares
estimate of the ideal weights for the linearly parameterized
uncertainty. In the following, we will consider the case where
the structure of the linearly parameterized uncertainty is
exactly known (Case 1 in section II), the results will then
be generalized to the case where it is only known that the
uncertainty can be linearly parameterized (Case 2 in section
II).
A. Least Squares Regression
We begin by describing a method by which least squares
Regression can be performed online. Let N denote the
number of available state measurements at time t, and
denote an estimate of the ideal weighs W . For a given set
of k samples, the model error (k) can be observed by
appealing to assumption 1 and noting that:
(k) = B T [x(k)

Ax(k) Bu(k)].

(12)

Since A, B, x(k), u(k) are known, the problem of estimating system uncertainty can be reduced to that of estimation

of x by using 12. In cases where an explicit measurement for

x is not available, for a given sample k, x can be estimated
using an implementation of a fixed point smoother [3], the
details of which are given in Appendix A.
Define the error (k) = (x(k)) (x(k))T , then the
error for N discrete samples can be written in vector form
as  = [(1), (2), ..., (N )]T . In order to arrive at the ideal
estimate of the true weights W we must solve the following
least squares problem:

the ideal weights smoothly into the baseline direct adaptive

training law of equation 8. This term acts as a modification
term to the baseline adaptive law.
In the following, we present Lyapunov based stability
analysis for the chosen adaptive law.

min T .

=W
W , tr denote the trace operator, and
Proof: Let W
consider the following Lyapunov candidate function:

(13)

Let Y (k) = [(1), (2), ..., (k)]T

lowing matrix:

1 (x(1)) 2 (x(1)) ...

1 (x(2)) 2 (x(2)) ...

X(k) =
1 (x(k)) 2 (x(k)) ...

and define the fol

m (x(1))
m (x(2))
. (14)

m (x(k))

The solution to the above least squares problem can be

obtained in a recursive form [5]. Alternatively, the solution
to the least squares problem can be found through Kalman
filtering theory by casting the least squares problem as
parameter estimation problem. Appealing to assumption 2,
we have the following model for the ideal weights ,
(k)

= (k 1),

(15)

(k)

(16)

(x(k))(k).

Let S(k) denote the Kalman filter error covariance matrix,

denote the estimate of the ideal weights , then setting
the Kalman filter process noise covariance matrix Q(k) = 0,
and the measurement covariance R > 0, the Kalman filter
based least squares estimate can be updated in the following
manner:
T

(k+1)
= (k)+K(k+1)[(k+1)
(k+1)(k)],
(17)

K(k + 1) = S(k)T (k + 1)[R + T (k + 1)S(k)(k + 1)]1 ,

(18)
S(k + 1) = [I K(k + 1)(k + 1)]S(k).
(19)
B. Recursive Least Squares Based Modification

T W 1 W
).
) = 1 eT P e + 1 tr(W
V (e, W
2
2

(20)

(t) (t))) serves

In the above equation, the term (W
to combine the indirect recursive least based estimate of

(21)

The time derivative of the Lyapunov candidate along the

system trajectory 4 is:
T (x) W T (x))
) = 1 eT Qe + rT (W
V (e, W
2
W 1 W
T ).
+ tr(W

(22)

Let  be such that W = + , adding and subtracting

T )T (W
T ) to equation 22 and using the
(W
definition of  yields,
T (x)) + tr(W
W 1 W
T)
) = 1 eT Qe + rT (W
V (e, W
2
T (W
) W
T ( W
).
+W
(23)
Rearranging yields:
) = 1 eT Qe
V (e, W
2
W 1 + (x)rT + (W
))W
T)
+ tr((W
T (W
).
W
(24)
W 1 + (x)rT + (W
))W
T) = 0
Setting tr((W
yields the adaptive law. Consider the last term in 24,
T (W
) =
W
=

We now describe a method by which the least squares

estimate of the ideal weights can be incorporated in the
adaptive control law. Let rT = eT P B where e, P, B are as
in section II, W , are positive definite matrices denoting
the learning rate. Let (t) denote the interval between two
successive samples k and k + 1, let T denote the time when
sample k was obtained, for the current instant in time t,

define the piece wise continuous sequence (t) = (k)

for

T t < T + (t), where (k) is as in 17.

is chosen as:
The adaptive law for weight estimates W
(t) = ((x(t))rT (t) (W
(t) (t)))W .
W

Theorem 1 Consider the control law of equation 3, and

the weight update law of 20, then all signals in the system
1 are bounded.

W )T (W
)
(W
T
W ) ( W
W)
(W
W )T .
+(W

(25)

Letting min (Q) and min ( ) denote the minimum

eigenvalues of Q and we have that equation 24 becomes:
k2 min ( ) W
T .
) = 1 kek2 min (Q) kW
V (e, W
2
(26)
With appropriate choice of S(0) and R, the Kalman filter

estimation error (k) (k)

and S(k) of equation 17, 19
remain bounded, hence  remains bounded. Therefore, for a
) < 0 outside of a compact
given choice of Q and , V (e, W
remain uniformly ultimately
set, which shows that e and W
bounded, since Arm is Hurwitz, xrm is bounded, therefore
it follows that x is bounded. This completes the proof.

Remark I : The above proof shows uniform ultimate boundedness of the tracking error and adaptive weights. It can be clearly seen that if  0
then tracking error e 0. This condition will be
achieved when W , that is when the Kalman
filter estimate of the ideal weights in 17 converges.
The convergence of the Kalman filter estimate is
related to choice of S(0), R and uniform complete
observability of the process model [3]. The uniform
complete observability for the process model in 15
can
if for some T > 0 and > 0
R t+Tbe guaranteed
T
(x(
))
(x(
))d I
t
Remark II : The above proof can be easily extended to the case where the structure of the
uncertainty is unknown (case 2 from section II)
by using Radial Basis Function Neural Networks
for approximating the uncertainty. The following
adaptive law will result in uniform ultimate boundedness of all states:
= ((x)rT (W
))W .
W
(27)
Furthermore, referring to equation 11 and noting
that in this case  = , it can be shown that if
the Kalman filter estimates of the ideal weights
converge, then the weights will approach a neighborhood of the best linear approximation of the
uncertainty.
Remark III : Although in this paper we restricted
our attention to nonlinear systems which have
a linear part and a scalar linearly parameterized
uncertainty, it is straight forward to extend the
analysis to more general nonlinear systems with the
uncertainty in vector form as long as it is linearly
parameterized.
Remark IV : The increased computational burden
when using the adaptive law of equation 20 consists mainly of evaluating equations 17,18, and 19.
It should be noted that since (x) <m , the
inversion in equation 18 is reduced to a division
by a scalar.


Ref
Model



u
+

Plant

 


Estimation of

Adaptive Law



 



Least Squares
estimation

  



Fig. 1.
Schematics of adaptive controller with recursive least squares
Modification

Figure 1 shows the schematic of the presented adaptive

control method with recursive least squares modification.

IV. S IMULATION RESULTS

In this section we use the method of Theorem 1 for the
control a wing rock dynamics model.
Let denote the roll angle of an aircraft, p denote the roll
rate, a denote the aileron control input, then a model for
wing rock dynamics is [10]:

= p

(28)

= a + (x).

(29)

Where (x) = W0 + W1 + W2 p + W3 ||p + W4 |p|p +

W5 3 . The parameters for wing rock motion are adopted
from [14] and [16], they are W0 = 0.0, W1 = 0.2314, W2 =
0.6918, W3 = 0.6245, W4 = 0.0095, W5 = 0.0214. Initial
conditions for the simulation are arbitrarily chosen to be
= 1deg, p = 1deg/s. The task of the controller is to
drive the state to the origin. To that effect, a stable second
order reference model is used with a natural frequency and
a damping ratio of 1. The proportional gain Kx and the
feedforward gain Kr in equation 3 are held constant for all
of the presented simulation results.
The structure of the uncertainty and the ideal weights W
are known for the wing rock dynamics model, hence the
performance of the adaptive law can be accurately evaluated
to the ideal
in terms of convergence of adaptive weights W
weights. The least squares problem is solved recursively
using equations 17, 18, and 19. It is assumed that no a priori
information is available about the ideal weights,hence we
= 0, consequently, the initial Kalman filter error
choose (0)
covariance matrix S(0) is chosen to have diagonal elements
with large positive values. Figure 2 shows the performance
of the baseline adaptive control law of equation 8 without the
recursive least squares modification. The learning rate used
was W = 3 for the low gain case, and W = 10 for the high
gain case. It is seen that the performance of the controller
in both cases is unsatisfactory. Figure 3 shows the phase
portrait of the states when the combined direct and indirect
adaptive law of Theorem 1 is used. It is seen that in both the
low gain and the high gain case the system follows a smooth
trajectory to the origin. Figure 4 shows the evolution of the
adaptive control weights when only the baseline adaptive
law of equation 8 is used. It is seen that the weights do not
converge to the ideal values (W ) and evolve in an oscillatory
manner. In contrast, figure 5 shows the convergence of the
weights when the combined direct and indirect adaptive law
of Theorem 1 used. Figure 6 compares the reference model
states with the plant states for the baseline adaptive law,
while 7 compares the reference model and state output when
the combined direct and indirect adaptive law is used. It can
be seen that the performance of the adaptive law of Theorem
1 is superior to that of the baseline adaptive law. Although
not shown in this paper, the adaptive law of Theorem 1 also
showed improvement in control effort required by reducing
the oscillation in the control input, and only a modest
increase in computational efforts. Furthermore, we note that

parameter convergence was observed despite using a nonpersistently exciting reference input (r(t) = 0t), this is
because (x(t)) was uniformly completely observable.

1.2
adaptive weights
true weights

1
0.8
0.6

0.4
0.5

0.2

p deg/sec

0
0.2

0.4
0.6

0.5

0.8
baseline low gain
baseline high gain

0.2

0.4

0.6
degrees

0.8

1.2

10

15

time (sec)

Fig. 4.
1.5
0.2

Evolution of adaptive weights with only baseline adaptive control

1.4
0.8

Fig. 2. Phase portrait of system states with only baseline adaptive control

0.6
adaptive weights
true weights

0.4
0.2
W

0.5

0
0.2

p deg/sec

0.4
0
0.6
0.8

0.5

10

15

time (sec)
LS mod low gain
LS mod high gain

1.5
0.2

Fig. 5. Evolution of adaptive weights with combined direct and indirect

adaptive controller
0

0.2

0.4

0.6
degrees

0.8

1.2

1.4

Fig. 3. Phase portrait of system states with combined direct and indirect
adaptive controller

V. CONCLUSION
In this work we presented an approach to combined
direct and indirect adaptive control for plants with a linearly
parameterized uncertainty by augmenting a baseline direct
adaptive law with recursive least squares. The presented
method ensures that the adaptive weights arrive smoothly
at the estimates of the ideal weights updated recursively
using a Kalman filter based parameter estimator. We showed
that the proposed combined adaptive law does not affect
the uniform ultimate boundedness properties of the model
reference adaptive controller. Furthermore, we indicated that
if the Kalman filter estimates of the ideal weights converge,
asymptotic stability can be guaranteed. We showed that the

problem of estimating the system uncertainty required by

the Kalman filter for estimating the ideal weights can be
effectively reduced to that of estimating the state derivative
x and suggested a time domain method for estimating the
state derivative using optimal smoothing. Online optimal
smoothing uses past as well as current data for generating
a delayed estimate of x.
However, this delay does not
affect the stability of the tracking error in the presented
framework, since the indirect part of the adaptive law acts
as a modification to a stable baseline direct adaptive law. An
adaptive controller with the presented combined direct and
indirect adaptive control modification was found to perform
significantly better than the baseline adaptive controller for
the exemplary problem of wing rock dynamics.
VI. ACKNOWLEDGMENTS
This work was supported in part by NSF ECS-0238993
and NASA Cooperative Agreement NNX08AD06A.

roll angle

[10] Morgan M. Monahemi and Miroslav Krstic. Control of wingrock

motion using adaptive feedback linearization. Journal of Guidance
Control and Dynamics, 19(4):905912, August 1996.
[11] K. Narendra and A. Annaswamy. A new adaptive law for robust adaptation without persistent excitation. IEEE Transactions on Automatic
Control, 32(2):134145, February 1987.
[12] Nhan Nguyen, Kalamanje Krishnakumar, John Kaneshige, and Pascal
Nespeca. Dynamics and adaptive control for stability recovery of
damaged asymmetric aircraft. In AIAA Guidance Navigation and
Control Conference, Keystone, CO, 2006.
[13] J. Park and I.W. Sandberg. Universal approximation using radial basis
function networks. Neural Computations, 3:246257, 1991.
[14] S. N. Singh, W. Yim, and W. R. Wells. Direct adaptive control of wing
rock motion of slender delta wings. Journal of Guidance Control and
Dynamics, 18(1):2530, Feb. 1995.
[15] Gang Tao. Adaptive Control Design and Analysis. John Wiley and
Sons, 2003.
[16] Konstantine Volyanskyy and Anthony Calise. An error minimization
method in adaptive control. In Proceedings of AIAA GNC, 2006.

1.5
actual
ref model

pirad

1
0.5
0
0.5

10

15

10

15

time (sec)
roll rate

xDot (pirad/s)

1
actual
ref model

0.5

0.5

5
time (sec)

VII. A PPENDIX A: O PTIMAL F IXED P OINT S MOOTHING

Fig. 6. Performance of adaptive controller with only baseline adaptive law

roll angle
1.4
1.2

actual
ref model

pirad

1
0.8
0.6
0.4
0.2
0

10

15

time (sec)
roll rate

xDot (pirad/s)

1
actual
ref model
0.5

0.5

10

15

time (sec)

Fig. 7.

Performance of combined direct and indirect adaptive controller

Numerical differentiation for estimation of state derivatives suffers from high sensitivity to noise. An alternate
method is to use a Kalman filter based approach. Let x,
be the state of the system and x be its first derivative, and
consider the following system:

 


x
0 1
x
=
(30)
x

0 0
x
Suppose x is available as sensor measurement, then optimal fixed point smoothing can be used for estimating x
from available noisy measurements using the above system
[3]. Optimal Fixed Point Smoothing is a non real time
method for arriving at a state estimate at some time t, where
0 t T , by using all available data up to time T . For
ease of implementation on modern avionics, we present the
relevant equations in the discrete form. Let x
(k|N ) denote
the estimate of the state x = [ x x ]T , let Zk denote the
measurements, () denote predicted values, and (+) denote
corrected values, dt denote the discrete time step, Q and
R denote the process and measurement noise covariance
matrices respectively, while P denotes the error covariance
matrix. Then the forward Kalman filter equations can be
given as follow:

R EFERENCES
[1] Otto Bretscher. Linear Algebra with Applications. Prentice Hall, 2001.
[2] Girish V. Chowdhary and Eric N. Johnson. Theory and flight
test validation of long term learning adaptive flight controller. In
Proceedings of the AIAA GNC Conference, held at Honolulu, HI, 2008.
[3] Arthur Gelb. Applied Optimal Estimation. MIT Press, 1974.
[4] Petros A. Ioannou and Jung Sun. Robust Adaptive Control. Prentice
hall, out of print, 1996.
[5] Ravindra V. Jategaonkar. Flight Vehicle System Identification A
Time Domain Approach. American Institute of Aeronautics and
Astronautics, 2006.
[6] E. N. Johnson. Limited Authority Adaptive Flight Control. PhD thesis,
Georgia Institute of Technology, Atlanta Ga, 2000.
[7] Suresh K. Kannan. Adaptive Control of Systems in Cascade with
Saturation. PhD thesis, Georgia Institute of Technology, Atlanta Ga,
2005.
[8] Nakawan Kim. Improved Methods in Neural Network Based Adaptive
Output Feedback Control, with Applications to Flight Control. PhD
thesis, Georgia Institute of Technology, Atlanta Ga, 2003.
[9] Lewis F. L. Nonlinear network structures for feedback control.
Asian Journal of Control, 1:205228, 1999. Special Issue on Neural
Networks for Feedback Control.

0
0

1
0

dt


Zk

[ 1

x
k ()

k x
k1 ,

Pk ()
Kk

0 ]

,

x
,
x

(31)
(32)
(33)

k Pk1 k + Qk ,
T

(34)
T

= Pk ()Hk [Hk Pk ()Hk + Rk ]

x
k (+)

= x
k () + Kk [Zk Hk x
k ()],

Pk (+)

[I Kk Hk ]Pk ().

, (35)
(36)
(37)

Letting x
k|k = x
k the smoothed state estimate can be given
as:
x
k|N = x
k|N 1 + BN [
xN (+) x
N ()].
(38)