MATHEMATICS FOR

AUTOMATIC CONTROL
AND SIGNAL PROCESSING
Professor Jean Claude CARMONA

Some references
Analisis de seales, IRARRAZAVAL M. PABLO, Mac Graw Hill, 1999. Class.
TK 5102.9 .I73 1999
Digital Signal Processsing: principles, algorithms and applications,
PROAKIS J.G., and al., PrinticeHall 1996, TK 5102.9 .P757 1996
Introduction to random signals and applied Kalman filtering, GROVER
BROWN R. and , PATRICK Y.C. HWANG P., Class. TK 5102.5 .B768I
Mathematical methods in Signal Processing and Digital Image,
DAHLHAUS, R. ED, Springer 2008, Class. TK 5102.9 M3837 2008
Digital Signal Processsing laboratory: LabView based FPGA
implementation, KEHTARNAVAZ N. and MAHOTRA S.,Bown Walker Press,
TK 5102.9 .K443 2010
Instrumentacion virtual: adquisicion, procesado y analisis de senales,
ANTONI M., BIEL D., 2002, Class: TK 5102.9 .M36 2002

SUMMARY

Fourier transforms
Classical filtering
Adaptive filtering
Estimation: some basical concepts
Linear optimal festimation: Kallman filter
Optimisation: classical algorithms:
- application to system identification
- application to optimal control laws

Evaluation
Written evaluations: theoretical evaluations
mid term and final evaluations
Mini project:
work on experimental data coming from a real
process:
- data acquisition
- treatment: filtering, estimation, .
- application to control ? .

Chapter 1: Fourier analysis of an analogical signal

I.1. Fourier series of a periodical signal
I.1.1. Definition

For s(t) with period T :

+2

(1)

, and conversely :

example

1
=

()

(2)

ip(t) : periodic comb (T) , duration t and magnitude a

therefore :

1
=

+/2

/2

( )
=

= ( )

then :

(3)

+2

( )

(4)

I.1.2. Main properties

a. Power conservation ( Parceval theorem)
+

1
=

b. Magnitude spectrum features

()2

(5)

i.e. function

Periodic signal (T) discrete spectrum (k/T)

ray spectrum : the harmonics

I.2. Fourier transform of a function

I.2.1. Definition

For any summable signal s(t) :

() +2

(6)

with :

() 2

(7)

S(f) is called the Fourier transform (FT) of hte signal s(t)

(6) is called the reverse Fourier transform formula.
example

i(t) : single pulse signal , duration t and magnitude a

Let us compute its Fourier transform I(f) :

+

+/2

() 2

=
/2

()
= ()

Its cancellation points are : = 0

(8)

, , and: IMax =I(0) at

Infinite and multi lobed magitude spectrum (lobe width 1/t)

I.2.2. Main properties
a. Power conservation ( Parceval theorem)
+

()

()2

(9)

b. Differential theorem

2 = 2()

(10)

c. Convolution theorem
+

= =

()

() 2 .

() 2

= .

(11)

I.2.3. Case of the Dirac distribution

+

a. Definitions

= 1

(12)

b. Main property
+

, ,

= ()

(13)

Remarks :
1.
2.

= ,

(14)

I.2.4. Fourier transform of a distibution

From definition of d(t) :

() 1

(15)

d(t) is the unit (simplest one) in the "fequency" domain (TF)

Important consequence :
u(t) : the time unity comb (distribution)
period : T
level : 1
+

= lim =
0

( )
=

u(t) periodic Fourier series development:

+

its Fourier transform:

(16)

=
=

( )

(17)

mind the difference (1/T) between FS and FT !

U(f) is also a unit comb but in the frequency doamin (period 1/T) !!

I.3. Application to continuous time signal sampling

1.3.1. modeling sampled continuous time signal

a. additive model :

(18)

( )
=

b. multiplicative model :
(19)

= .

remark : u(t) is already a sampled signal , since:

= lim

( )
=

Dont forget that u(t) and U(f) are very particulars and play particular roles:
u(t) is the unique signal that have teh same shape that its TF U(f)

1.3.2. Spectrum of a sampled continuous time signal

Idea: use the multiplicative model and the convolution theorem
+

= . = =

replace with :

and use :

( ) =

( )
=
+

= ()

But, in the frequency domain !

Finally, it holds:
+

( )

(20)

Important features:
The spectrum X*(f) is peridodic (Fe)
We dont have loss of information, on the contrary !

1.3.3. Sampling fundamental theorem, Shannon theorem

Principle: signal " well sampled" : we can retrieve it (the information it contains)

in fact, the problem is:

Let us draw the shape of the magnitude spectrum

2 cases :
1. Fe sufficiently large
It is easy to retrieve

a low pass filter at Fe /2

2. Fe too small
Spectrum aliasing problem !
(distorsion )
good sampling iff Fe > 2 fMax (the spectrum width of the signal) Shannon
9

I.4. Training exercices

1.4.1. FT of a rectangular impulse

(see fig. 1.2.)

Let the signal s(t) be a rectangular impulse signal with length a and height
b.
Show that its TF is given as follows:
= ()
Draw the shape of the magnitude spectrum of s(t).
1.4.2. FT of a exponential impulse
Let the signal s(t) be an exponential impulse signal given as follows:
= 2 /
Show that its TF is given as follows:
() =

1 + ()2

Draw the shape of the magnitude spectrum of s(t).

Compare this result to the case of a rectangular impulse (the shape changes)

10

1.4.3. FT of a Lorentz impulse

Let the signal s(t) be a Lorentz impulse signal given as follows:
Show that its TF is given as follows:
1
=
1 + (/)2
Draw the shape of the magnitude spectrum of s(t).
hint : use the reciprocity theorem and the scale change theorem

= ()
= ()

= ()
1
() =
( )

1.4.4. FT of a periodic signal

Let the signal s(t) be a positive square periodic signal, with period T,
Height 1 a and duty cycle 0,5.
Start with computing its Fourier series (FS).
Using:
( 0 ) = 20
and the reciprocity theorem (see above), compute its Fourier transform S(f).
11

I.5. Practical application:

Let us consider here the problem of (numerical) data acquisition on a process.
We calssically use a so called "acquistion chain" represented by the following scheme:

process

sensor

conditionning

IN
Intranet/Internet

anti-aliasing filter
Coding/decoding

x(t)
Sample and hold

m-

processor

x*(t)

ADC

xn
coding/dcodage

coding

bus

Data acquisition chain

12

Problem :
We are interested in the signals:
x(t) : the measure
/ analogic signal
i.e. exploitable signal given by the sensor + conditionning (pre treatment)
x*(t) : sampled (Te) / analogic signal
xn : converted (Te) / numerical signal ( i.e. the data )
and the way to obtain them in good conditions in order to conserve the information!

1. How sample x(t) without loosing information (choice of Te ? )

2. How convertind (ADC) without loosing information ?
Idea: The frequency approach, i.e. using Fourier transforms
and analysing the magnitude spectrum of x(t) , x*(t) and xn .
The first question here is :
How choosing the good Fe for a given signal, in particular in the general case ?
Methodology:
1. compute the TF X(f) of the signal x(t)
2. draw its magnitude spectrum , i.e. X(f)
3. determine the upper bound of its spectrum, i.e. fMax
4. apply Shannon theorem , i.e. Fe > 2 fMax

13

Difficulties :
1. compute the TF X(f) of the signal x(t)
easy for simple signals but for a general case (real) signal !!
Idea : the more a signal is rapid, the more it is difficult to sample it: fMax
Analysis the typical cases of:
1. a very rapide variation in the signal (i.e. a step dicontinuity type)

2. a progressive variation in the signal (i.e. a linear signal, triangular)

practically, we will analyse:
1. the case of a rectangular impulse signal
2. the case of a triangular signal
and determine the frequency cost , i.e. fMax , to well sample them.
For a general signal, it will depend on the fact it contains these cases , for
choosing the convenient estimation of fMax , and then deduce Fe.
14

1. Rectangular impulse signal

duration t , height a
a. Show that the frequency harmonics decrease in 1/f .
b. The spectrum is theoritically infinite, so we must decide to
make an approximation in order to fix fMax.
Classically, we decide of a quality criterion, i.e. fMax for less
than 1% of magnitude.
Compute the cooresponding fMax , then propose Fe.
c. Numerical applications: t =100ms , t = 1 ms and t = 100 ms;
Conclude on the sampling cost of these signals.

2. Triangular signal
To facilitate the calculus, choose a traingle with a for the base
and the height, namely t.
The same questions and conclusions.
To numerically compare the two cases, take the same values of t,
as in 1.
15

Chapter 2: Fourier analysis of an numerical signal

Interest:
- manipulate data records relevant information
ex: RADAR signal extraction position/speed of an air plane
-

modifications: add "good properties

ex: noise reduction/caneelation
sentences synthesis for telephone recording
information transmisson on a specific canal,

Interest of numerical signals

II.1. Time representation of a numerical signal

II.1.1. Definition
A numerical signal xn or x(n) can be modelised by a function from the Z set to the real set R.

II.1.2. Basical numerical signals

1. unit impulse defined as follows:
=

1
= 0
0

2. unit step defined as follows:

1 0
=
0
3.

Sign function defined as follows:

Signn = 2 Un 1

4.

Gate function defined as follows:

RectN n = Un+N Un-N

5. Sine function defined sas follows: (frequency f0)

xn= = X0 Sin (2pf0n + j)
6. Complex exponential function :

= 0 20

II.2. Discrete time Fourier transform: dtFT

II.2.1. Definition
+

(21)

where: g = f / Fe is the reduced (normalized) frequency, related to the discret time : n = t / Te

2 = 2 = = = = ( )

(22)

(23)

II.2.2. Main properties

X(g) is continue in g , and periodic in g, with period 1

(24)

vocabular: the basic period [0; 1[ or [-1/2 ; +1/2 [ is called the fundamental interval
+1/2

the inverse dtFT :

() +2

=
1/2

linearity :

axn + byn aX() + bY()

time translation :

0 20

(25)

frequency translation :

20 0

time inversion:

x-n

X(- )

convolution product:

(modulation)

(26 )

=
+

Parseval formula :

+1/2

II.2.3. Main example

() 2

(27 )

1/2

the dtFT of the unit gate signal (duration t 2N+1 samples)

2 = +2 (1 + 2 . + 2(2+1) )

=
=

( 2+1 )
()

the magnitude spectrum

(28)

is not evident to draw in the fundametnal interval !

from Matlab:

application: multilobated spectrum but with a very fast decreasing

it is shown that almost 90% of the power spectrum
is located in the first lobe !
We shall consider only this first (main) lobe. We measured appromitaly:

()

D 0,15 fMax 0,03 Fe : upper bound of the numerical signal.

Let us resume the analogical signal with width t =1 ms , sampled at Fe 2 . 33/t , 100kHz for example.
For the numerical signal obtain after numerization, we have: fMax 0,03 .105 = 3 kHz
conclusion:

the numerical signal "occupies" 3 kHZ since the original continuous time signal
"occupies" 33 kHz . NO COMMENT !!

II.3. Discrete Fourier transform: DFT

Motivation :

Computing dtTF is too bulky :

infinite sum in data (xn) + infinite number of frequencies (g)

Approximation :

the DTF is an approxilation of the dtTF defined on:

a large but finite number of data (xn) + contribution of a finite number of frequencies (g)
1

, where: k=1,2 ..N , g = k/N and: W e-i2p/N

=0

In fact, we compute the DFT X(k) for N frequencies , on the base of N data xn of the signal.
W is the Nth root of the unity, in the complex set C, i.e. the solution of zN = 1 in C
the DFT inverse is:

=0 ()

In practical application, we want to optimize (reduce) the computaional time of DFT.

The FFT (Fast Fourier Transform)
remark : it is not an approximation of the DFT. Simply, it is a better way to compute it !

(29 )

II.4. the Fast Fourier Transform (FFT ):

It is an optimization algorithm due to Cooley Tuckey
1.

first idea: N = 2p . We say : a FFT on 2P points (4, 8, 16, 64, 128, 256, points)

2.

second idea: Rewrite cleverly X(k) , that is:

for example: N = 8 points : data x0 , x1 , x2, x3 , x4 , x5 , x6 and x7
frequency contributions: X(1) , X(2) , X(3) , X(4) , X(5) , x(6) and X(7)

= 0 0 + 1 1 + 2 2 + 3 3 + 4 4 + 5 5 + 6 6 + 7 7

= 0 0 + 2 2 + 4 4 + 6 6 + ( 1 1 + 3 3 + 5 5 + 7 7 )
Saying that: W0k = 1

= 0 + 2 2 + 4 4 + 6 6 + 1 (1 + 2 3 + 4 5 + 6 7 )
Moreover:

= [(0 + 2 2 )+ 4 (4 + 2 6 )]+ 1 [(1 + 2 3 )+ 4 (5 + 2 7 )]

We notice that we can operate in the same time, x0 and x2, and , x4 and x6, and, x1 and x3, and , x5 and x7,
with the same operator:
1 time the first number + W2k the second number. Oreover
In fact, we can compute in parallel the 4 parenthesis ( xi + W2k xj) ,
i.e. the same operation with different entries time improvement (optimization)

 The following (second) step: compute the brackets [ () + W4k () ]

once more , using tehe same operator (a second one) with two couple of different entries , i.e. parallel computing.
Final step (third step); compute X(k) = [ ] + W8k []
Remark: the 3 calculus corresponding to the three steps , i.e. compute the 4 parenthesis (), than the 2 brackets [],
and finally X(k) itself, are of the same type:
a and b a + Wr.b with a certain power of W, Wr : W2k , then W4k and finally W8k
Let us introduce the following symbolic operator:

In fact, using the properties of the 8th roots of unity, we have some
useful symetric propereties of Wr, namely:
W4k = - W0k = -1 , W5k = -Wk , W6k = - W2k and W7k = - W3k

we can use the second (bottom) output of this operator !

Finally, sumarizing the three computing steps and using this operator, we have:

Chapter 3: Linear Invariant Discrete Systems (LIDS)

Idea: deal about nuemrical filters almost, linear invariant

Linearity : x1n y1n and x2n y2n ax1n + bx2n ay1n + by2n
xn yn xn-1 yn-1

Time invariance :
Stability (1st form) :
Causality:

If xn is bounded, i.e. > 0 , , < > 0 , , <B

is independent of

, >

III.1. Recurent equations:

example:

yn = acxn-1 + bcxn-2 + cyn-1 (recursivity order 1 since yn-1 )

model closest to the implementation algorithm !

III.2. Impulse response:

Definition : input xn dn yn hn output

Fundamental property : (basis of linear dynamic systems) : convolution theorem

, ,

(30)

< +

Stability (2d form) :

(31)

Causality:

< 0 ,

= 0

(32)
+

Discrete convolution:

= =

=
=

example:

hn = an Un

and :

xn = bn Un

yn = ?

(33)

example :

Classically, we introduce the elementary following operators

Addition operator:

Multiplication operator:

Delay operator

1. give the recursive equation of this "filter

2. deduce its impulse response hn
3. draw its shape

remark :

examples of nonlinear and/or time varying disrete systems

Some basical relations

= ,
,
+

=
=

III.3. Frequency response:

Principle : Let us calculate the output of an DLIS with sine input with frequency f :

= =

+ = 1 + 2
2

Let us take the DFT:

= = .

= 1,2

where H(jf) is the DFT of the impulse response hn.

Then , the output signal yn is a sine signal with frequency f , where:

= cos + , : = () , = ()

(34)

Some examples:

Give the recusrisve equations, the impulse responses hn and the frequency transfer H(if).

III.4. The z Transform of a numerical signal: (zT)

II.4.1. Definition
Let us remember the Laplace transfom (LT) of a continuous time signal x(t) :
+

()

(35)

For a numerical signal xn , for example the one given by the discretization of x(t) :
+

() =

(36)

remark: we can say its a development in integer series of z .

examples :

= = ?
= = ?
gate signal: = 1 < , =
exponential signal:

=?

does X(z) ever exists ? pb of the integer series convergence !!

=?

II.4.2. Convergence of the zT

Some results:

1. xn left bounded, i.e. 0 < 0 , = 0 X(z) converges for all z outside a given circle C
2. xn right bounded, i.e. 0 > 0 , = 0 X(z) converges for all z inside a given circle C
3. xn left-right bounded, i.e. of finite length ( n1 n n2 ) X(z) converges for all z except , maybe, for 0 and .
Practical consequence: We almost always use finite record "of signal" case 3 always convergence !
Examples: let us study the convergence of the folowing time series
1. xn = Un and yn = - Un
2. xn = an Cos (n).Un , with: a = 0.9 and = p/4
3. xn = an Sin (n).Un
Inverse transform:
from the residual theorem:

1
2

() 1

(37)

, where C is a closed curve in the complex plane containing all the poles of X(z) and the origin O.
practically unusefull and unused !!
Therefore, how can we do to retrieve xn from X(z) ?

1st method: the elementary decomposition of X(z)

Let us consider X(z) as he following rational fraction:

()
0 + 1 1 + +
=
=
()
1 + 1 1 + +
Idea: let us decompose X(z) in simple elements in order to use their well-known ZT; i.e.

1
=
1 1

1 1

xn = ..

2d method: the direct division of N(z) by D(z)

example:

1+2 1 + 2 + 3
1 1 8 2 +12 3

(36)

0 = 1,

= 1 + 3 1 + 12 2 + 25 3 +

1 = 3 , 2 = 12,

3 = 25,

II.4.3. Usual Ztransforms table

II.4.4. Main properties of the zT

1. linearity:

ax1n + bx2n aX1(z) + bX2(z)

2. time delay:

xn-i z-iX(z)

3. product with an exponential time series:

4. final and initial theorems:
0 = lim

= lim 1 ()

6. frequantial convolution:

7. Parseval theorem

x1n * x2n X1(z).X2(z)

5. time convolution:

1 . 2

anxn X(z/a)

1
2

(practically unexploitable !!)

1 ()2 ( )1

1
=
2

( 1 ) 1

8. relation zT/dtFT :
+

with the formal link :

2 =

+
2
=

= 2

II.4.5. Discrete transfer function H(z)

a. definition :

(38)

main interests:
1. provide the frequantial behaviour of the system:

()

(39)

2. avoid the convolution difficulty:

=

= . ()

(40)

H(z) computation
1. From hn , compute H(z) with (38) : for simple cases only !
2. Applied the delay theorem xn-i z-iX(z) to the recusive equation
example : give H(z) of the following system

II.4.6. Poles/Zeros diagram (description)

In a general way:

=0

(41)

=1

, the recursive equation of a LIDS and therefore its distcrete tranfer function H(z) :

0 + 1 1 + +
=
= 0
1 + 1 1 + +

=1( )

=1( )

(42)

 The knowledge of the nA poles pj and the nB zeros zi + one constant defines the system !

Main properties
1. Algebraic stability criterion
a LIDS is stable ifff all its poles pi are with modulus less than 1, i.e.

, < 1

(43)

2. Minimum phase system (stable inverse system)

a LIDS is a minimum phase system, or is inverse stable, ifff all its zeros zji are "stable",
i.e.with modulus less than 1.

, < 1

(44)

3. Causality or feasability of a LIDS

In practical applications, systems are, of course, real and feasable, so nB nA
Important remark
There are strong lengths between the place of the poles and teh zeros of a model ,
and tits dynamical behaviour. We have seen the satbility (inside the unit circle), but also
The rapidity, the damping, as it can be easily shown on a 1st and 2d order models.

1st order behaviour:

2d order behaviour:

III.4. The q backshift operator: application to SIDL modelling

II.4.1. Definition :
Let us introduce the time operator corresponding to a delay of one sampling period Te :

n, un-1 = q-1 un

, for all numerical signal (time series) {un}n

(45)

remark : taking zT of (45) on using the delay theorem of zT, we have :

zT{un} = z-1 U(z)
obviously the frequancy delay operator z-1 correspond to the time delay operator q-1
interest: use the practical notation q-1 while staying in time domain !
II.4.2. Important consequences :
a. Recursive equation (algorithm):

= b.
=0

=1

see (45)

=
=0

=1

(47)

 We can then derive a discrete time transfer function:

0 + 1 1 + +
=
1 + 1 1 + +

(48)

Interest: we can know simply write in time domain:

= ( 1 )

(49)

b. Regression form:
We are using parameter models: recursive equation , transfer fucntions
Let us consider (47) introducing
notations: 1. let us define a parameter vector:
= 1 2 0 1
we have nA + nB +1 model parameters , i.e. dim q .

(50)

2. let us introduce the observation vector

= 1 2 11

(51)

 We can use the regressive form of the model, rewriting equ (47)
(very helpfull for the Estimation Theory (and model Identification !)
=

(48)

remarks: It is an almost general framawork since:

case 1 . in the basic case , j do not depend on the parameter q.
(48) is a pure linear regression.
Example in System Identification: we often employ the concept of true
system (see Ljung 99) to define the model of the system we are looking for:
* the true model with its true parameter vector q0 :
= 0

(49)

we use a family of parameter model within which we are looking for the best
estimate , with output signal the estimation of our systme output, i.e.
=
where the vector parameter is extimated on the base of N data.

(50)

See estimation theory and/or system identification theory,

we solve the following optimization problem :
= min ( )

(51)

Hoping
1.

that problem has solution ( a set of solutions)

2.

q0 belongs to its set (in general, we suppose it !)

3.

lim = 0

case 2 . , j depends on the parameter q , i.e. j j (q)

(48) is a pseudolinear regression
solving (51) is more complicated !
See further fondamental regressives models in control theory

III.5. Black-Box models used in Automatic Control

Based on a parametric model structure as follows:
= 1 , + 1 ,
, =
1

, (, ) = 1 +

(52)

(53)

where:

un , yn are the input/output signals of the system

en the noise signal (centered white sequence with finite variance)
with pdf fe(q)
q is the parameter vector of a given model of the structure
d the interger part of the time delay of the system
G the transfer function of the deterministic part of the model
H the transfer of the disturbance model (stochastic part)
{gk}k the impulse response of the process model
{hk}k the impulse response of the noise model
remark:

noise output disturbance 1 ,

remark: classically we define the one-step-ahead predictor related to, as:

() = 1 , , + [1 1 (, )]

A better approach, i.e. a polynomial approach facilitates estiamtion of Balck-Box mdels.

Teh general structure used is :

1
( 1 )
( 1 )
=
+

( 1 ) ( 1 ) ( 1 )

(53)

II.5.1. AR model :
= 1 1 2 2 . +

(54)

Let us introduce the predicted output (estimation)

=
where:

linear regression

= 1 2 .
= 1 2 .

II.5.2. ARX model :

= 1 1 2 2 . + 1 1 . + +

(55)

Let us introduce the predicted output (estimation)

=

linear regression

where:
= 1 . 1
II.5.3. ARMAX model :

= 1 2 . 1

= 1 1 . + 1 1 . + + + 1 1 +

(56)

Let us introduce the predicted output (estimation)

= ()

pseudo linear regression

where:
= 1 2 . 1 1

and
where

= 1 . 1 1 ()
= ()

is the prediction error at time n-i

II.5.4. Output Error model (OE):

= 1 1 . + 1 1 . + + + 1 1 +

We have the pseudo linear regression

= ()
with

= 1 . 1
II.5.5. Exercices:

= 1 1

1. Demonstrate the preceeding expressions for ARMAX and OE models

2. Establish these expressions for the general case
of the following BoxJ enkins model

(57)

III.6. Discretization of a continuous time system

Pb: What is exactly the system seen by the computer we are looking for its G(z-1) ?

Let us define the different functions to model:

- Sampling procedure

- "Hold" treatment

- Conversion A D

- Continuous time process G(s)

- Conversion D A

Conclusion

(under ideal asumptions)

"transparent "

HBOZ(s)
"transparent "

G(s)
"transparent "

1 = () = 1 1

()

(58)

examples
1st order:

=
1 +

2d order (with an integer):

=
(1 + )

1 ( + 1 )
=
(1 1 )(1 1 )

with:
= (1 / )
= (1 / ) / )
= /