Copyright C IFAC ModeliDa and Comrol in Biomedical
Syaans. Galveam. Teua. USA. 1994
Regularization Networks for System Identification and Prediction
of Glucose Kinetics
Z. Trajanoskitl, W. Regittnig§, P.
*Institute of Biomedical Engineering, Graz University of Technology. Inffeldgasse 18. 8010 Graz
§Deparunent of Internal Medicine. Krankenhaus der Barmherzigen Brtider. Marschallgasse 12. 8020 Graz. Austria
Abstract - A neural network for non-linear system
identification and prediction is presented. A
regularization network which is equivalent to radial
basis function neural network was used for
identification of a Non-linear AutoRegressive Moving
Avarage model with/without eXogenous inputs
(NARMAXINARMA). NARMAX system identification
and validation was performed from simulated data from
a publisbed model for the dynamics of blood glucose in a
diabetic state. A NARMA model was used for short-
term prediction of subcutaneous tissue glucose time-
series from healthy SUbjects.
I. IN1RODUCTlON
In recent years. both theory and praxis of artificial neural
networks and non-linear systems have advanced
considerably. Feed forward multi-layered neural networks
have been increasingly employed for dynamic system
modelling and identification of systems which exhibit
complex non-linear characteristics.
In the present study we have applied regularization
networks for identification of the glucose metabolism in a
diabetic state and for modeling subcutaneous tissue glucose
time-series from healthy subjects.
11. MErnODS
A. System Representation: the NARMAX -Model
The Non-linear AutoRegressive Moving Avarage model
with eXogenous inputs (NARMAX) is a concise
description of a system in terms of a non-linear functional
expansion of past inputs. outputs. and prediction errors [2]:
y(t) = f(y{t -I) .... y{t - nJ,u{t -I) ... .u(t - nJ,
(1)
E(t -1) .... E(t - nE) + E(t)
where y(t).U(t).E(t) are the system output (glucose), input
(insulin), and prediction error. or residual respectively. and
f( x) is some non-linear function. For nu = 0 the
relationship (I) represents a non-linear time-series or a
tEmail : uajanoS@ibmLw-graz.ac.at
303
Ko~o§, F. Skrabal§, and P. Wach l
NARMA model. The exact form of the function f
describing a real-world system is usually unknown. Hence.
the uknown function f had to be recovered from data set
obtained by sampling.
B. Regularization Network
The inverse problem of recovering the function f . or an
estimate of it from a sparse data set is clearly an ill-posed
one. Hence, in order to reconstruct the function f one
needs some a priori knowledge about it. In many cases the
a priori assumption about the function f is that it is
smooth. Thus, it can be shown either in the context of
functional analysis [10] or in a probabilistic framework [7]
that the solution of the ill-posed problem can be obtained
by minimizing the functional:
N
H[J] = L (J(xJ- y} +Njl[J] (2)
;=1
where cp[J] is a smoothness functional or stabilizer and A.
is the regularization parameter which controls the
compromise between the degree of smoothness of the
solution and its closeness to the data. Depending on the
choice of the stabilizer (which corresponds to different
assumptions of smoothness) different solutions can be
obtained [9]. For the class of radial stabilizers
cp[J(x)] = cp[f(Rx)] for any rotation matrix R the well
known radial basis function neural network [3, 8] is
recovered. It can be shown, that a radial stabilizer of the
form :
(3)
-~
where ](s) is the Fourier transform of f{x) , G(s) = e ~
and ~ is a fixed parameter, leads to the approximation:
n
f(x}= Lc;G(lIx-t;ID (4)
;=1
where n < N , t; are some appropriately chosen centers.
G(r) =e-r' , and c; are constant coefficients.
 In many practical applications, the components of the
input vector are of different type and the norm Ix - I
tj must
be considered as a weighted norm:
(5)
since the scale of the components would be otherwise
arbitrary. In this study, a diagonal matrix W was used, in
which case the elements of the matrix are equivalent to the
inverse of the variance ~ of each component of the
multidimensional Gaussian in equation (4).
C.ldentification and Validation
Learning of the network (4), i.e. identification of the
NARMAX-Model is a non-linear optimization problem:
given a set of data, fmd the coefficients Cj' the coordinates
of the centers tj and the elements of the matrix W. The
functional H[t] is in general non-convex and a stochastic
gradient descent can be applied [9]. However, for a given
matrix W and for known , centers tj the network (4) is
linear in the parameters and the coefficients Cj can be
determined from the equation [9]:
c=(GTG+).gtGTy (6)
with the n x n matrix gall = G( ta; t~). and the N x n matrix
G,n = G(x,;t a ). Here we have used the following approach:
1) use a priori knowledge for determining the matrix W,
2) select the significant centers, and 3) calculate the
coefficients Cj from equation (6). For the elements of the
scaling matrix W the modified indices of insulin
sensitivity and glucose effectiveness [1] were used. The
centers can be selected from the available data points by the
lC-means clustering technique (for on-line identification)
[8] or the orthogonal least squares method (for off-line
identification) [3]. The regularization parameter A was
determined by the generalized cross-validation method [5].
After identification, the identified model was supplied to
validation tests for non-linear systems [6].
III. REsULTS
A published model [4] , adjusted to a diabetic state was
used to generate the data for identification and validation.
The simulated data were corrupted by additive white noise
of zero mean and 2 % coefficient of variation and were
then submitted to the network for identification. 300 data
points from a simulated intravenous glucose tolerance test
and 200 data points from an oral glucose tolerance test
were used for identification and validation, respectively.
Additionally, subcutaneously measured glucose time-series
from healthy subjects were used for NARMA identification
and one step a-head prediction as previously shown [11].
304
The identified networks have succesfully passed 11
validation tests for non-linear systems [6]. However, iJ
contrast to out previuos worK [11], with the presentcG
regularization networK it was not necessary to use movin!
average terms in the equation (1).
IV. DISCUSSION
In the case of system identification with neural networks,
it is obvious that different prior assumptions about Ihe
smoothness of the function which should be reconstructcG
lead to different network architectures. The presented stOOy
. demonstrates that a regUlarization network can be obtained
for NARMAX identification of glucose kinetics. The resuJlS
of this worK encourage further studies of netwoR
application for an adaptive closed loop control of glucose (I'
hypoglycemia prediction in patients with diabetes mellitus.
ACKNOWLEDGEMENT
This worK was supported by the Austrian Science
Foundation, Grant No. S4906 and P09142-Med.
REfERENCES
[l] BERGMAN R.N., Y2. iDER, c.R. BOWDEN, AND C. COBEut
"Quultitative estimation of insulin sensitivity". Am J Physioi, 236:
E667-E677, 1979
[2] CHEN S. AND S.A. Bn..LINGs, "Representation of non-linear systems: die
NARMAX model",lnll Control, 49 : 1013-1042, 1989.
[3] CHEN. S .• C.F.N. COWAN, AND P.M. GRANT, "Orthogonalleast squares
learning algorithm for radial basis function networks" , IEEE TraIlS
Neural Networks. 2: 302-309,1991.
[4] COB ELL! c., G. FEDERSPIL. G. PACINI, A. SALVAN, AND C.
SCANDELLARl; "An integrated mathematical model of the dynamics cl
blood glucose and its hormonal control". Math Biosci, 58:27-60. 1982
(5) CRAVEN P. ANDG. WAHBA: "Smoothing noisy data with splinefunctioru
- estimating the correct degree of smoothing by the method of generalized
cross-validation ", Numer Math, 31 : 377-403, 1979
(6) LEONTARITlS l.J. AND SA. Bn..LINGS. "Model selection and validatioo
methods for non-linear systems".lnt J Control. 45: 311-341, 1987.
[7] MARROQulN J.L., S. MrrrER. AND T. POGGIO: "Probabilistic solution cl
ill-posed problems in computational vision. J Amer Stat Assoc, 82:76-
89, 1987
(8) MOODY J. AND C. DARKEN, "Fast learning in networks of locaIly·tuned
processing writs", Neural Computation. I: 281-294,1989.
[9] POGGIO T., AND F. GJRQ5I; "A theory of networks for approximation and
learning", A.1. Memo 1140, Artificial Intellegence Laboralory,
Massachusetts Institute of Technology, 1990
(10)TIKHONOV A.N. AND V.Y. ARsENIN: Solutions of ill.posed problenu,
New York. Wiley. 1977
[1 I]TRAJANOSKI Z.• P. WArn. R. GROBLER. H. KONTSCHIEDER, P.
KOTANKO. AND F. SKRABAL: "NARMA modeling of subcutaneous
tissue glucose time-series using neural networks", Proc 15th Ann 1nl
ConflEEE Eng Med Bioi Soc, San Diego, USA. 538-539,1993