Professional Documents
Culture Documents
'
2. Unit step
1 0
[ ]
0 0
n
u n
n
'
<
3. Real exponential:
2-3
[ ]
n
x n A
where both and A are real. If
0 A >
and 0 1 < < , then
[ ] x n
decreases as
n
increases; see figure below.
4. Sinusoidal
( )
[ ] cos
o
x n A n +
where A is the amplitude,
o
Example: Express
1 2,3,...,10
[ ]
0 otherwise
n
x n
'
+
where
*
1 2
2
j
Ae
A A
*
1 2
o
j
e
It is also possible to express the signal as
( )
{ } 1 1
[ ] 2Re
n
x n A
Example: Express an arbitrary DT signal in terms of the unit impulse
Answer
[ ]
[ ] [ ]
k
x n x k n k
2-6
2.2 Discrete Time Systems
Let
[ ] T
represents the transformation a discrete time system performed
on its input
[ ] x n
. The corresponding output signal of the system is
[ ]
[ ] [ ] y n T x n
.
The system is linear if
[ ]
1 2 1 2
[ ] [ ] [ ] [ ] T ax n bx n ay n by n + +
where
1 2
[ ] and [ ] y n y n
are the responses of the system to inputs of
1 2
[ ] and [ ] x n x n
respectively.
The above equation illustrates the principle of superposition.
Assume the system is linear and let
[ ]
h
h n
be the output of the system when
the input is
[ ] [ ]
k
p n n k
(i.e. a unit-impulse at time n k ). Then according to the linearity property,
[ ] y n [ ]
T [ ] x n
2-7
when the input is
[ ]
[ ]
[ ] [ ]
[ ]
k
k
k
x n x k n k
x k p n
,
the output will be
[ ]
[ ] [ ]
k
k
y n x k h n
The system is time-invariant if
[ ] [ ]
k
h n h n k
,
i.e. the output is delayed if the input is delayed. In this case
[ ] [ ] [ ]
[ ] [ ]
k
y n x k h n k
x n h n
The signal
[ ] h n
is called the impulse response of the time-invariant
system.
While the focus of this course is on linear, time-invariant (LTI) system,
there are many real-life applications where the system is non-linear and
time-variant. A good example is a digital FM demodulator operating in the
mobile radio environment.
2-8
Example: Provide a physical interpretation of a LTI system whose impulse
response is
[ ] [ ] h n u n
Answer: The output of the system is
[ ]
[ ]
1
[ ] [ ]
[ ]
[ ]
[ ] [ ]
[ 1] [ ]
k
k
n
k
n
k
y n x k h n k
x k u n k
x k
x k x n
y n x k
_
+
,
+
,
this means the impulse response
[ ] h n
is zero when
0 n <
.
2-9
Example: A moving averager computes the signal
2
1
1 2
1
[ ] [ ]
1
M
k M
y n x n k
M M
+ +
from its input [ ] x n . Here
1
M and
2
M
are positive integers. What is the
impulse response of the system? Is the system casual?
Answer:
The output can be rewritten as
1
2
1 2
1 2
1
1 2
1
[ ] [ ]
1
1
[ ] [ ]
1
n M
m n M
n M n M
m m
y n x m
M M
x m x m
M M
+
+
+ +
_
+ +
,
Compared to the output of the integrator, we can deduce that the impulse
response of the system is
[ ] [ ] ( )
1 2
1 2
1
[ ] 1
1
h n u n M u n M
M M
+
+ +
Since the impulse response is non-zero when
0 n <
, so the system is not
casual.
A real physical system can not be non-casual, i.e. it can not generate an
output before there is an input. So in practice what a non-casual system
2-10
means is that there is a processing delay. For example, you can view the
moving averager as a device that computes the local mean of the signal
[ ] x n
at time
n
after it observes the sample
1
[ ] x n M +
. So
1
M
is the delay.
A system is stable if a bounded input results in a bounded output. The
requirement for having a stable system can be derived from the
input/output relationship of a LTI system, which states that
[ ]
[ ] [ ]
k
y n x k h n k
This means
[ ]
[ ]
[ ] [ ]
max max
[ ] [ ]
[ ]
k
k
k m
y n x k h n k
x k h n k
x h n k x h m
where is the absolute value operator and
max
x
is the largest magnitude of
the input signal.
So if the impulse response of the system is absolute-summable, i.e. when
[ ]
k
S h k
<
'
Solution:
0 0
[ ]
k
k
k k k
h k a a
2-12
This is summable if
1 a <
. In this case,
1
[ ]
1
k
S h k
a
The ZIR is
1
1
[ ] [ ]
n
h n a bu n
+
+
+ +
,
which is different from
3 1 1 2 2
[ ] [ ] [ ] y n w y n w y n +
,
where
1
1 1
0
1
2 2
0
[ ] [ ],
[ ] [ ]
n
n n k
k
n
n n k
k
y n a b a x k
y n a b a x k
+
+
+
2-16
2.3 Fourier Transform of Discrete Time Signals
Consider the sinusoidal signal
( )
[ ] cos x n A n +
It can be written in terms of two complex exponential functions as
( ) ( )
1 2
[ ]
2
j n j n
j n j n
e e
x n A Ae A e
+ +
+
+
where
*
1 2
2
j
A
A e A
The complex signal
j n
e
is an important signal in discrete time signal
processing it is an eigenfunction of a linear system and it leads us to the
concept of Fourier Transform of a discrete-time signal.
Again let us use
[ ] T
to represent the operation a discrete time system
performs on its input. A signal
[ ] f n
is an eigenfunction of the system if
[ ]
[ ] [ ] T f n a f n
,
where the constant
a
is called an eigenvalue. This definition is consistent
with that in matrix theory where the eigenvector
v
and the eigenvalue
b
of
a matrix A is defined as
b Av v
.
2-17
Here the matrix A is analogous to our linear system.
As shown in Section 2.2, the transformation performed by a LTI on its
input
[ ] x n
is described by the convolution formula:
[ ] [ ] [ ]
k
y n h k x n k
,
where
[ ] h n
is the impulse response of the system and
[ ] y n
is the
transformed signal or output of the system. If
[ ]
j n
x n e
,
then the output signal becomes
( )
( )
[ ] [ ]
[ ]
[ ]
j n k
k
j n j k
k
j n j k
k
j n j
y n h k e
h k e e
e h k e
e H e
,
where
( )
[ ]
j j k
k
H e h k e
.
2-18
It is clear from the above analysis that
j n
e
is indeed an eigenfunction of a
discrete-time LTI system with ( )
j
H e
being the corresponding eigenvalue.
In the linear system literature, ( )
j
H e
is called the frequency response of a
discrete-time LTI system.
In general, the expression
( )
[ ]
j j k
k
X e x k e
is called the Fourier Transform of the discrete-time signal
[ ] x n
.
One important property of the Fourier Transform of a discrete time signal
is that it is periodic in
with a period of
2
. This is quite different from
the Fourier Transform of a continuous time signal, which in general is not
periodic.
Example: Express the output of a LTI system in terms of its frequency
response when the input is the sinusoid
( )
[ ] cos x n A n +
. Assume the
impulse response of the sytem is a real signal.
Solution:
- The sinusoidal input can be written as a weighted sum of two complex
exponential functions as
1 2
[ ]
j n j n
x n Ae A e
+
2-19
where
*
1 2
/ 2
j
A Ae A
are the weighting coefficients.
- Since the system is linear, the sinusoidal response is
1 1 2 2
[ ] [ ] [ ] y n A y n A y n +
where
( )
1
[ ]
j j
y n H e e
and
( )
2
[ ]
j j
y n H e e
,
are the outputs of the system when the inputs are
1
[ ]
j n
x n e
and
2
[ ]
j n
x n e
respectively.
- Since the impulse response is real,
( ) ( )
*
*
[ ] [ ]
j j k j k j
k k
H e h k e h k e H e
_
,
.
This means we can write the output of the system as
( ) ( )
( ) ( )
( ) { }
( ) ( )
1 2
1 2
[ ] ]
* *
1 1
1
[ ]
2Re
cos ( )
j j n j j n
y n y n
j j n j j n
j j n
j
y n A H e e A H e e
A H e e A H e e
A H e e
A H e n
+
+
+ +
14243 1442443
2-20
Note that
( )
j
H e
and
( )
are respectively the magnitude and phase of
the frequency response, i.e.
( ) ( )
( ) j j j
H e H e e
This means
( )
1
j
H e
Since
( )
( )
( ) ( )
( ) ( ) ( ) { }
( )
2
2 2
2 2
2 2 2 2
2
2 2
2
1 1 cos( ) sin( )
1 cos( ) sin( )
1 cos( ) sin ( )
1 cos( ) sin ( ) 1 cos( ) sin ( )
1 cos( ) sin ( ) cos ( ) sin ( )
1 cos( )
j
ae a ja
a a
a a j
a a a a
a a j
a a
+
_
+ +
+ +
,
+ +
+ ( )
2 2
sin ( )exp ( )
j
where
sin( )
( )
1 cos( )
a
a
,
this means the magnitude of the Fourier transform is
( )
( )
2
2 2
1
1 cos( ) sin ( )
j
X e
a a
+
and the phase is simply
( ) ( )
.
2-22
Existence of the Fourier Transform:
- If we set the parameter
a
in the above example to unity, then the signal
becomes a unit-step. The Fourier Transform in this case, however, does
not exist in the finite magnitude sense.
- A sufficient condition for the existence of the Fourier transform (in the
finite-magnitude sense) is that the signal is absolute-summable, i.e.
[ ]
k
S x k
<
The proof is the same as that we used to proof the stability of a LTI
system.
- We can deduce from the above that the Fourier Transform always exists
for signals with finite duration.
Example: Determine the Fourier Transform of the signal
1/( 1) 0
[ ]
0 otherwise
M n M
x n
+
'
Solution
( )
( )
( ) ( ) ( )
{ }
{ }
( ) ( )
( )
1
0
1 / 2 1 / 2 1 / 2
/ 2 / 2 / 2
1
2 / 2
2
1 1 1
[ ]
1 1 1
1
1
sin
1
1 sin
j M
M
j j n j n
j
n n
j M j M j M
j j j
M
j M
e
X e x n e e
M M e
e e e
M e e e
e
M
+ + +
+
+ +
+
2-23
The magnitude of the transform is
( )
( ) ( )
( )
1
2
2
sin
1
1 sin
M
j
X e
M
+
At a first glance, the phase of the Fourier Transform is
( ) / 2 M
.
However, the
sin( ) / sin( ) g
function can take on either + or ve value.
When there is a sign change in this function, that corresponds to an
additional 180 degree phase shift.
Plots of the magnitude and phase of the transform for the case of 4 M
are shown below.
2-24
The Inverse Fourier Transform is defined mathematically as
( )
1
[ ]
2
j j n
x n X e e d
Proof:
( )
( )
( )
1 1
[ ]
2 2
1
[ ]
2
1
[ ]
2
j j n j k j n
k
j n k
k
j n k
k
X e e d x k e e d
x k e d
x k e d
Since
( )
( )
( )
( ) ( )
( ) ( )
1 1
2 2
1
2
1
2
sin sin
sinc
c c
c c
c
c
c c
j m j m
j m
j m j m
c c
c
c
c c
e d e d j m
j m
e
j m
e e
m j
m m
m m
m
,
this means
2-25
( )
( )
( )
1 1
[ ]
2 2
[ ] sinc
[ ] [ ]
[ ]
j j n j n k
k
k
k
X e e d x k e d
x k n k
x k n k
x n
Physically, the inverse Fourier transform states that the time domain signal
is the sum of infinitesimally small complex sinuoids of the form
( )
j j n
X e e d
where ( )
j
X e
denotes the relative amount of each complex sinusoidal
component. Consequently, it is a synthesizing formula.
Example: Determine the impulse response of a LTI system whose
frequency response is given by
( )
4
1
2
1
1
j
j
H e
e
Solution:
We first rewrite the frequency response as
2-26
( )
( )
( )
4
1
2
4
1
2
0
4
1
2
0
1
1
j
j
k
j
k
k
j k
k
H e
e
e
e
'
'
1
( )
j
lp
H e
2-27
Solution:
( )
( )
1
[ ]
2
1
2
sin
c
c
j j n
lp lp
j n
c
h n X e e d
e d
n
n
<
is
( )
k
k h k
<
With the introduction of this function, we can now have proper definitions
for the Fourier transforms of signals such as a DC signal, a complex
sinusoidal, and the unit-step signal. The fact that these signals exist at
discrete frequencies is consistent with the above properties of the impulse
function.
2-29
The Fourier Transform of the DC signal
[ ] 1 x n
is
( )
( )
[ ]
2 2
j j n
n
j n
n
n
X e x n e
e
n
Proof:
The function
( )
j
X e
can be treated as an analog signal in
. Since this
analog signal has a period of 2 P , it can be represented by the
complex Fourier series
( )
2
exp
j
k
k
X e X j k
P
,
(1)
where
( )
/ 2
/ 2
1 2
exp
P
j
k
P
X X e j k d
P P
_
,
(2)
is the k-th complex Fourier coefficient. Substituting
( ) ( )
2 2
j
n
X e n
2-30
into (2) yields
( )
( )
/ 2
/ 2
1 2
exp
1 2
2 2 exp
2 2
1
P
j
k
P
n
X X e j k d
P P
n j k d
_
,
_ _
+
, ,
Substituting
1
k
X
into (1) yields
( )
( )
2
exp
2
exp
2
exp
j
k
k
k
k
X e X j k
P
j k
jk
,
_
is
( )
( )
0
o
j n j j n
n
j n
n
X e e e
e
2-31
This is simply the Fourier Transform of the DC signal shifted to the
frequency
o
. Consequently,
( ) ( )
2 2
j
o
n
X e n
+ +
2.4 Properties of Fourier Transforms
Linearity:
If
( )
( )
1 1
2 2
[ ] ,
[ ]
j
j
x n X e
x n X e
then
( ) ( )
1 2 1 2
[ ] [ ]
j j
ax n bx n aX e bX e
+ +
2-32
Time Shifting and Frequency Shifting
[ ] [ ]
[ ]
( )
[ ]
( )
j n
n
j n d j d
n
j m j d
m
j d j
x n d x n d e
x n d e e
x m e e
e X e
[ ] [ ]
[ ]
( )
( )
( )
0
0
o o
j n j n j n
n
j n
n
j
e x n x n e e
x n e
X e
Time reversal
[ ] [ ]
[ ]
[ ]
( )
( )( )
( )
j n
n
j n
n
j m
m
j
x n x n e
x n e
x m e
X e
2-33
We showed earlier that if
[ ] x n
is real, then
( ) ( )
* j j
X e X e
.
So if
[ ] x n
is real, then
[ ] ( )
* j
x n X e
Differentiation in Frequency
( ) [ ]
[ ]
[ ] ( )
[ ]
j j k
k
j k
k
j k
k
j k
k
d d
X e x k e
d d
d
x k e
d
x k jk e
j kx k e
Convolution: If
3 1 2
[ ] [ ] [ ] x n x n x n
, then
( ) ( ) ( )
3 1 2
j j j
X e X e X e
2-34
Proof: Since
3 1 2
1 2
[ ] [ ] [ ]
[ ] [ ]
k
x n x n x n
x k x n k
,
then
( )
( )
( )
3 3
1 2
1 2
1 2
1 2
[ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
j j n
n
j n
n k
j n k
j k
n k
j n k j k
k n
j m
X e x n e
x k x n k e
x k x n k e e
x k x n k e e
x k x m e
_
,
( )
( )
( ) ( )
1 2
2 1
1 2
[ ]
[ ]
j k
k m
j j k
k
j j k
k
j j
e
x k X e e
X e x k e
X e X e
_
,
The result is known as the Parsevals Theorem.
Proof:
Let
[ ]
*
[ ] h n x n
This means
( ) ( )
* j j
H e X e
.
If
*
[ ] [ ] [ ]
[ ] [ ]
[ ] [ ]
k
k
y n x n h n
x k h n k
x k x k n
,
then
( ) ( ) ( ) ( )
2
j j j j
Y e X e H e X e
Taking the inverse Fourier Transform of ( )
j
Y e
at
0 n
yields
2-36
( )
( )
2
2
1
[0] [ ]
2
1
2
j
k
j
y x k Y e d
X e d
_ _
, ,
_ _
, ,
( )
( )
( )
( )
( )
( )
1 1
2 2
1
2
j jn
j j jn
j jn
e e d d
X e W e d e d
Y e e d
where
( ) ( )
( )
( )
1
2
j j j
Y e X e W e d
2-37
It is important to realize that the above is a circular convolution in the
frequency domain.
Example: Use the Windowing Theorem to determine the Fourier
Transform of the signal
( )
sin
[ ]
0 otherwise
c
n
n
M n M
y n
'
Solution
This signal can be considered as the product of the ideal low pass signal
( )
sin
[ ]
c
lp
n
h n
n
'
As shown earlier,
( )
1
0 otherwise
j c
lp
H e
'
.
It can also be shown that (please verify)
( )
( 1) ( 1)
1 1
1
1 1
j M j M
j
j j
e e
W e
e e
+ + +
+
+
2-38
Results for different values of M are shown below
Note that
( )
j
M
H e
is this figure is equivalent to
( )
j
Y e
Observations:
- the oscillation (also known as the Gibbs phenomenon) is more rapid for
larger M,
- the amount of ripples does not decrease though
Exercise: Plot ( )
j
W e
and find out what causes the ripples in these
diagrams.
2-39
2.5 Sampling of an Analog Signal
Reference: Sections 4.0-4.3, 4.6 of Text
Now that we know what the Fourier Transform of a discrete time signal
[ ] x n
is, we want to relate it to the Fourier transform of its continuous-time
counterpart
( )
c
x t
, under the condition that
[ ] ( )
c
x n x nT
,
where T is the sampling period and
1
s
f
T
.
Since
[ ] ( )
c
x n x nT
, this means
2-40
( )
( )
( )
( )
( ) ( )
( )
( )
( ) ( )
2 1
2
2 1
2
2
/
2 2
/
/
2
/
1
[ ] ( )
2
1
2
1
2
1
2
T
T
T
j nT
c c
k
j nT
c
k
k
T
j k nT
c T T
k
T
T
j nT
c T
T
x n x nT X j e d
X j e d
X j k e d k
X j k e d
+ +
+
( ) ( )
( ) ( )
( ) ( )
/
2
/
/
2
/
2
1
2
1
2
1 1
2
k
T
j nT
c T
k
T
T
j nT
c T
k
T
j n
c T T
k
X j k e d
X j k e d
X j k e d
T
+
' )
+
' )
+
' )
with a
period of
2
.
( ) ( ) ( )
2
1
j
c T T
k
X e X j k
T
2-41
To construct ( )
j
X e
from
( )
c
X j
, we can adopt the following procedure
1. Divide
( )
c
X j
into intervals of width
2 / T
according to the formula
k-th interval:
2 2
k k
T T T T
< +
2. Define the spectral segment in the k-th interval mathematically as
( )
( ) ( ) ( )
1 1
2 2
2 / 2 /
0 otherwise
c
k
X j k T k T
X j
< +
'
3. Shift the different spectral segments to the center of the spectrum and
add them together according to
( ) ( ) ( )
2
k T
k
Y j X j k
_
<
,
Although mathematically this equation is only confined to the interval
< , we can obtain the value of ( )
j
X e
outside this interval by
making use of the fact that ( )
j
X e
is periodic in
with a period of
2
.
2-42
Example: Sampling of a triangular analog spectrum with a one-sided
bandwidth of 480 W rad/s. The sampling frequency, in rad/s, is
2
640
s
T
Fig: (a)
( )
c
X j
, (b)
1
( ) X j
, (c)
0
( ) X j
, and (d)
1
( ) X j
2-43
Fig: (a) the various shifted spectra
( )
k
X j
, (b) the summed spectrum ( ) Y j ,
and (c) the Fourier transform
( )
j
X e
. Note that the x-variable in diagram (c)
is the digital frequency divided by .
It is observed that the summed spectrum ( ) Y j is no longer triangular.
Consequently the Fourier transform
( )
j
X e
in the interval
[ , ]
is also not
triangular. This is known as the aliasing effect and is caused by using too
small a sampling frequency.
The aliasing effect can be eliminated if the sampling frequency satisfies the
Nyquist criterion :
2-44
2
2
s
W
T
where W is the (one-sided) bandwidth of the analog signal
( )
c
x t
in rad/s.
Proof:
Consider the term ( )
1
2
2 / k T
for any
0 k >
. If the Nyquist sampling
criterion is satisfied, this means
1 2 1 1
2
2 2 2
s
k k k W W
T
_ _ _
, , ,
This means the spectral segments
( )
k
X j
, k=1,2,, are all zero.
Similarly, it can be shown that all the spectral segments with negative k are
zero. Consequently,
( )
1
j
c
X e X j
T T
,
,
i.e. the shape of
( )
c
X j
is preserved after sampling. In this case, the
Fourier transform of the sampled signal is simply a (frequency and
amplitude) scaled version of
( )
c
X j
.
For the previous example involving the triangular
( )
c
X j
, a sampling
frequency of 640 rad/s was used. This is less than the Nyquist frequency of
2 960 W
rad/s. Consequently there is aliasing in the sampled signal.
When the sampling frequency is at or above the Nyquist frequency, say
960, 2 960, 3 960
s
rad/s, then the aliasing disappears and ( )
j
X e
becomes
2-45
Fig: Effect of varying the sampling frequency on
( )
j
X e
: (a) the sampling frequency
is at the Nyquist rate of 960 rad/s, (b) the sampling frequency is twice the
Nyquist rate or 1920 rad/s, and (c) the sampling frequency is three times the
Nyquist rate or 2880 rad/s.
It is evident from the diagram that the spectral width of ( )
j
X e
in the
interval [ , ] is proportional to the bandwidth to sampling-rate ratio
2
s
W
The larger
s
+
+
+ +
+ +
' )
2-48
( )
( )
( )
1
2 2 1
0
1
[ 2 ]/
0
1 1
1
M
i
c M T T
i k
M
j i M
i
X j k
M T
X e
M
+
+ 1
' )
]
.
2-49
If this is the case, then the effective sampling frequency of
[ ]
d
x n
is
( )
' 2
2
s
s
W
which satisfies the Nyquist criterion for zero-aliasing. Alternatively you
can prove this using the diagram below.
Fig: (a) ( )
j
X e
when the sampling frequency is twice the Nyquist frequency, (b)
( )
/ 2
1
2
j
X e
, (c) ( )
( 2 ) / 2
1
2
j
X e
, and (d) ( )
j
d
X e
.
In general, the sampling frequency of
[ ] x n
must be at least M times the
Nyquist rate in order to avoid aliasing in
[ ]
d
x n
.
2-50
2.5.2 Up Sampling by an I nteger Factor
When there is no aliasing in the sampled signal, then theoretically it is
possible to compute
( )
c
x t
exactly at any value of
t
from
[ ] x n
. This
process is known as interpolation.
Argument:
( ) ( ) [ ] ( )
j
c c
x n X e X j x t
_ _
, ,
[ ]sinc
n
n
t
x n n
T
_
,
_
,
2-51
In practice, it is not possible to calculate
( )
c
x t
exactly from its samples
because the sinc function is, straightly speaking, infinitely long.
Interpolators that are commonly used in practice are:
- Lagrange,
- Cubic spline
Note that
sinc( / ) t T
is the inverse Fourier transform of the following ideal
low pass spectrum:
Thus ideal interpolation is equivalent to feeding the discrete time signal
[ ] x n
as a sequence of (continuous-time) impulses to an ideal low pass
filter, i.e.
( ) [ ] ( ) sinc
c
n
t
x t x n t nT
T
_
_
, ,
Again, since the ideal low pass filter can never be implemented exactly
because of the abrupt transitions, we can never interpolate the analog signal
exactly from its samples.
/ T / T
T
( )
lp
H j
2-52
Let
[ ] ( ')
i c
x n x nT
be the DT signal obtained by sampling
( )
c
x t
at a rate of
( )
'
1 1
Hz
' /
s
L
f
T T L T
Since this signal can be obtained from the signal
[ ] ( )
c
x n x nT
according
'
[ ] [ ]sinc
[ ]sinc
[ ]sinc ,
i
k
k
k
nT
x n x k k
T
nT
x k k
LT
n
x k k
L
_
,
_
,
_
,
,
- Since
1/
s
f T
satisfies the Nyquist criterion, the new sampling rate
' /
s
f L T
used in generating
[ ]
i
x n
will also satisfy the Nyquist
criterion. This means
( )
( )
1
' '
/
0 /
j
i c c
j L
L L
X e X j X j
T T T T
LX e L
L
_ _
, ,
'
<
In otherword, ( )
j
i
X e
is simply a compressed version of ( )
j
X e
.
The existence of the two distinct frequency bands in ( )
j
i
X e
suggests
(again) a low pass filtering effect in interpolation. This can be traced back
to the sinc function
sinc
n
k
L
_
,
in
[ ]
i
x n
.
2-54
2-55
Exercise: What is the Fourier Transform of the signal
[ ] [ ]sinc
i
k
n
x n x k k
L
_
,
'
,
where
[ ]
x
m E x
and
( )
2
2
x x
E m
1
]
x
are respectively the mean and variance of x , with [ ]
E
being
the average operator.
Fig: pdf of Gaussian random variables.
2-58
Exercise: Determine the mean and variance of a uniform random variable.
Note that
[ ]
Pr ( )
b
x
a
a b p v dv
x
with
( ) 1
x
p v dv
.
A discrete time signal
[ ] x n
is a random process when every
[ ] x n
is a
random variable.
Notations:
1.
[ ]
n
x n x
2. The pdf of
n
x
is
( )
n
x
p v
3. The mean of
n
x
is
n
x
m
4. The variance of
n
x
is
2
n
x
]
,
depends only on the time difference
n m
, i.e.
[ ]
[ ]
*
, [ ] [ ]
xx
xx
n m E x n x m
n m
1
]
Basically what we are saying is that the first and second other statistics of a
(wide-sense) stationary random process do not depend on absolute time. In
this course, we will focus only on these random processes.
Exercise: Show that
[ ] [ ]
*
xx xx
n n
Subsequently show that for a real random process
( ) ( )
j j
e e
The autocorrelation function at a time difference of
0 n
is
[ ]
2
*
0 [ ] [ ] [ ]
xx
E x n x n E x n
1
1
]
]
and is called the average power of the random process
[ ] x n
. If the random
process has zero mean, i.e.
0
n
x
m
, then
2-60
[ ]
2
2
2
0 [ ] [ ]
,
n
xx x
x
E x n E x n m
1
1
1 ]
]
where
2
x
is the variance of
[ ] x n
.
We will only focus on zero-mean processes in this course.
The autocorrelation function provides information as to how fast a random
process varies with time. A fast process will have a relatively narrow
autocorrelation function. The exact frequency contents of a random process
can be obtained from its power spectral density (psd) function, defined as
( )
[ ]
j j n
xx xx
n
e n e
Since this equation is simply the Fourier transform of
[ ]
xx
n
, consequently
1
[ ] ( )
2
j j n
xx xx
n e e d
Note that
1
[0] ( )
2
area under psd function
= average power
j
xx xx
e d
2-61
White noise:
A random process is white if there is no correlation between the values of
the process at different time instants. Mathematically this means
2
[ ] [ ]
xx x
n n
and
( )
2
2
[ ] [ ]
j j n j n
xx xx x
n n
x
e n e n e
Exercise: Convince yourself that a white noise must have zero mean.
A commonly encountered random process is the white Gaussian noise.
Remember, white refers to the spectral shape, and Gaussian refers to
the pdf.
Let the (stationary) random process
[ ] x n
be the input to a LTI system with
an impulse response
[ ] h n
. Then the output of the system is
[ ] [ ] [ ]
k
y n x k h n k
.
We want to examine the statistical properties of the output process.
The mean value of
[ ] y n
is
2-62
[ ]
[ ]
( )
0
[ ] [ ] [ ]
[ ] [ ]
[ ]
[ ]
n
y
k
k
x
k
x
k
j
x
m E y n E x k h n k
E x k h n k
m h n k
m h n k
m H e
1
1
]
,
y
m
where
( )
[ ]
j j m
m
H e h m e
is the frequency response of the system.
Basically, what the result is saying is that if the mean of the input is
stationary, then the mean of the output is also stationary.
How about the autocorrelation function?
By definition
[ ]
*
*
, [ ] [ ]
[ ] [ ] [ ] [ ]
yy
k r
n m E y n y m
E h k x n k h r x m r
]
1
_ _
1
, ,
1
]
2-63
* *
* *
* *
*
[ ] [ ] [ ] [ ]
[ ] [ ] [ ] [ ]
[ ] [ ] [ ] [ ]
[ ] [
k r
k r
k r
E h k x n k h r x m r
E h k h r x n k x m r
h k h r E x n k x m r
h k h
1
_ _
1
, ,
]
1
_
1
,
]
1
]
[ ]
]
[ ]
xx
k r
yy
r n m k r
n m
+
Thus the autocorrelation function of the output process is also independent
of absolute time.
Since both the first and second order statistics of the output process are
independent of absolute time, the process is (wide-sense) stationary.
Let
d n m
, then the autocorrelation function of the output process can
be written as
[ ] [ ]
*
[ ] [ ]
yy xx
k r
d h k h r d k r
+
Taking Fourier transform of both sides yields
2-64
( ) [ ]
[ ]
[ ]
[ ]
*
* ( )
*
[ ] [ ]
[ ] [ ]
[ ] [ ]
j j d
yy yy
d
j d
xx
d k r
j d k r j r j k
xx
k r d
j m
xx
m
e d e
h k h r d k r e
h k h r d k r e e e
h k h r m e
+
_
+
' )
,
( )
( ) ( )
( ) ( ) ( )
( ) ( )
*
*
*
2
[ ] [ ]
[ ]
j r j k
k r
j j r j k
xx
k r
j j j k
xx
d
j j j
xx
j j
xx
e e
h k e h r e e
e H e h k e
e H e H e
e H e
_
' )
,
' )
1
]
2-65
If
( )
j
H e
is a very narrow band filter centered at
c
t
, then
( ) ( ) ( ) { }
2
2
c c c
j j j
y xx xx
H e e e
+
,
where 0 is the bandwidth of the filter.
For simiplicity assume a real
[ ] x n
1
. Then
[ ] [ ]
xx xx
n n
(see an earlier
exercise) and ( ) ( )
j j
e e
. Consequently,
( ) ( )
2
2
2 0
c c
j j
y xx
H e e
or simply
( )
0
c
j
xx
e
.
This property of ( )
c
j
xx
e
1 1 +
] ]
1 + +
]
1 1 + +
] ]
+ +
[ ] [ ]
2 2 2
] [ ]
y x
E x n
a +
or
2
2
2 2
1
1 1
x
y
a a
- The pdf of
[ ] y n
is simply
( )
2
2
2
1
exp
2
y
y
y
v
p v
_
,
- The output
[ ] y n
can be expressed recursively as
( )
[ ] [ 1] [ ]
[ 2] [ 1] [ ]
y n ay n x n
a ay n x n x n
+
+ +
2-67
2
3 2
1
0
[ 2] [ 1] [ ]
[ 3] [ 2] [ 1] [ ]
[ ] [ ]
m
m k
k
a y n ax n x n
a y n a x n ax n x n
a y n m a x n k
+ +
+ + +
+
1 _
+
1
, ]
1 _
+
1
, ]
1 +
]
2
[ ]
m
y
yy
a
m
2-68
( ) ( ) ( )( )
( )
0
2 2 2
0
2 2 2
0 0
2 2
2
2 2 2
1 1
1 1 1 1
1 1
m m
j m j m
y y y
m m
n j n m j m
y y y
n m
y y
y
j j
j j j j
y y y
j
a e a e
a e a e
ae ae
ae ae ae ae
ae a
+
+
+
+
( )
( )
( )
( )
2 2
2
2
1
1 2 cos
1
1 2 cos
j
y
e
a
a a
a a
+
Solution 2
- The LTI has an impulse response of
[ ] [ ]
n
h n a u n
where
[ ] u n
is the unit-step function.
- The corresponding frequency response is
( )
0
1
[ ]
1
j j n n j n
j
n n
H e h n e a e
ae
This means
2-69
( )
( ) ( )
2
2
2 2
2
1
1
1
1 cos( ) sin( )
1
1 2 cos( )
j
j
H e
ae
a a
a a
+
- Since
[ ] x n
is white and has a variance of unity, its psd is simply
( )
2
1
j
xx x
e
- The psd of
[ ] y n
is
( ) ( ) ( )
2
2
1
1 2 cos( )
j j j
yy xx
e e H e
a a
+
2.7 Linear Predictiive Coding and the Autocorrelation Function
Consider the last example in Section 2.6 where we have a random process
[ ] y n
governed by the equation
[ ] [ 1] [ ] y n ay n x n +
2-70
Here
a
is a positive constant less than 1 and
[ ] x n
is a white process with
zero mean and unit variance. As shown in Section 2.6, the variance of
[ ] y n
is related to the variance of
[ ] x n
according to the equation
2
2
2 2
1
1 1
x
y
a a
Suppose
[ ] y n
is a sampled-speech signal and we want to send this speech
signal digitally through a communication channel. A simple way is to
quantize each sample into
y
B
bits and feed the resultant bit stream to a
digital modulator to generate the transmitted signal
This encoding method, which is referred to as Pulse Code Modulation
(PCM) in the literature, is not very efficient because
[ ] y n
has a relatively
large dynamic range, at least compared to
[ ] x n
.
Suppose the parameter parameter
a
(which will vary from speaker to
speaker) is known to both the encoder and the decoder. Then a more
efficient encoding scheme can be obtained by first subtracting
[ ] [ 1]
p
y n ay n
from
[ ] y n
to obtain
[ ] y n
bit
Quantizer
y
B
Digital
Modulator
2-71
[ ] [ ] [ ] [ ] [ 1]
p
x n y n y n y n ay n
and then quantize
[ ] x n
using a
x
B
-bit quantizer. The resultant bits,
together with the parameter
a
(also in quantized form), are then sent to the
receiver. Upon receiving these information, the decoder can approximate
[ ] y n
according to
[ ] [ 1] [ ] y n ay n x n + % %
,
where
[ ] x n %
and
a%
are respectively the quantized versions of
[ ] x n
and
a
.
This encoding scheme, known as linear predictive coding (LPC), is more
efficient than PCM because
[ ] x n
has a smaller dynamic range than
[ ] y n
.
In the speech coding literature,
[ ] [ 1]
p
y n ay n
is referred to as the
predicted value of
[ ] y n
and
[ ] [ ] [ ]
p
x n y n y n
is called the residual or
excitation.
[ 1] ay n
+
[ ] x n
[ ] x n
( )
Speech Model
1
1
j
j
H e
ae
( )
Prediction Filter
j j
F e ae
[ ] y n
Eqv. response = 1
j
ae
2-72
As shown earlier, the autocorrelation function of
[ ] y n
is
2
[ ]
m
yy x
m a
.
This means
[1] [0]
yy yy
a
or
[1]
=
[0]
yy
yy
a
So as long as
[0] and [1]
yy yy
are known, then the parameter
a
will be
known to both the encoder and decoder. In practice,
[0] and [1]
yy yy
can be
estimated according to
1
1
[ ] [ ] [ ].
N d
yy
k
d y n y n d
N
This suggests that in order to implement the LPC encoder, the signal [ ] y n
must first be analyzed to obtain its autocorrelation function. Once the
autocorrelation function is estimated, then the information will be used to
obtain the prediction filter at the encoder and the synthesizing filter at the
decoder.
[ 1] ay n
[ ] y n
[ ] x n %
[ ] x n
[ ]
Q
Quantizer
1
az
Synthesizing filter
2-73
As the parameter
1 a
, the variance of [ ] y n is going to much greater than
that of [ ] x n . Consequently LPC will be much more efficient than PCM.
As
0 a
, the variance of [ ] y n approaches that of [ ] x n and LPC provides
little improvement to the encoding efficiency.
Since
0 a
implies little correlation between samples in [ ] y n while
1 a
corresponds to high correlation, we can conclude that LPC achieves
a higher encoding efficiency because it removes the redundancy in [ ] y n
through linear prediction.
The expression
[ ] [ 1] [ ] y n ay n x n +
represents a first-order model. In
general, speech signals can be modeled accurately using a N-th order model
(typical value of N is 10):
1
[ ] [ ] [ ]
N
k
k
y n a y n k x n
where
[ ] x n
is a white noise. The model also represents a system described
by a Linear Constant Coefficient Difference Equation with
[ ] x n
being the
excitation and [ ] y n being the output.
For the N-th order model, the prediction filter in the encoder computes
1
[ ] [ ]
N
p k
k
y n a y n k
1
1
1
]
A
M
are then quantized and transmitted to the receiver. Upon receiving these
information, the decoder synthesizes the original speech signal according to
1
[ ] [ ] [ ]
N
k
k
y n a y n k x n
% %
,
where
[ ] x n %
and
k
a%
are respectively the quantized versions of
[ ] x n
and
k
a
.
How to determine linear predictor A? If we multiply both sides of
[ ] y n
by
[ 1] y n
and taking average, we obtain
[ ]
[ ] [ ]
1
1
1
[ ] [ 1] [ ] [ 1] [ ] [ 1]
[ ] [ 1] [ ] [ 1]
[ 1]
N
k
k
N
k
k
N
k yy
k
E y n y n E a y n k y n x n y n
a E y n k y n E x n y n
a k
1
+
1
]
+
2-75
or simply
1
[1] [ 1]
N
yy k yy
k
a k
1
+
1
]
+
or
1
[2] [ 2]
N
yy k yy
k
a k
1
]
2-76
These equations can be written in matrix form as
2
1
2
3
1
[0] [1] [2] [ 2] [ 1] [1]
[1] [0] [1] [ 3] [ 2] [2]
[2] [1] [0] [ 4] [ 3] [3
[ 2] [ 3] [ 4] [0] [1]
[ 1] [ 2] [ 3] [1] [0]
N
N
a N N
a N N
a N N
a N N N
a N N N
1 1
1 1
1 1
1 1
1 1
1 1
1 1
1 1
1 1
] ]
L
L
L
M M M M M M M
L
L
]
[ 1]
[ ]
N
N
1
1
1
1
1
1
1
1
1
]
M
or
N N N
U A V
,
where
[0] [1] [2] [ 2] [ 1]
[1] [0] [1] [ 3] [ 2]
[2] [1] [0] [ 4] [ 3]
[ 2] [ 3] [ 4] [0] [1]
[ 1] [ 2] [ 3] [1] [0]
N
N N
N N
N N
N N N
N N N
1
1
1
1
1
1
1
1
1
]
U
L
L
L
M M M M M M
L
L
,1 1
,2 2
,3 3
, 1 1
,
N
N
N
N
N N N
N N N
a a
a a
a a
a a
a a
1 1
1 1
1 1
1 1
1 1
1 1
1 1
1 1
1 1
] ]
A
M M
and
[1]
[2]
[3]
[ 1]
[ ]
N
N
N
1
1
1
1
1
1
1
1
1
]
V
M
2
For convenience, we drop the subscript yy in [ ]
yy
m .
2-77
So the coefficients for the prediction filter can be obtained by solving
1
N N N
A U V
,
provided that the autocorrelation function
[ ] m
is known.
A brute-force computation of
N
A
results in a complexity in the order of
( )
3
O N . However, since the matrix is Toeplitz, i.e. all elements along any
diagonal are identical, a more efficient algorithm called the Levison and
Durbin (LD) algorithm can be used.
The LD algorithm only has a complexity of only
( )
2
O N .
The LD algorithm is a order-recursive algorithm, i.e. the m-th order
predictor
m
A
can be obtained from the
( 1) m
-th predictor
1 m
A
.
Specifically, we rewrite
m
A
as
,1
,2 1 1
,
0
m
m m m
m
m
m m
a
a
k
a
1
1
1 1
1
+
1 1
1
] ]
1
1
]
d A
A
M
(1)
where the vector
1 m
d
and the scaler
m
k
are quantities to be determined.
The covariance matrix
m
U
itself can be written in terms of
1 m
U
and
1 m
V
as:
2-78
( )
1 1
1
[0]
r
m m
t
m
r
m
1
1
1
]
U V
U
V
(2)
where
1
[ 1]
[ 2]
[ 3]
[2]
[1]
r
m
m
m
m
1
1
1
1
1
1
1
1
1
]
V
M
(3)
is the correlation vector
1 m
V
arranged in reverse order, and [ ]
t
stands for
the transpose of a matrix.
Substituting (1)-(3) into the equation
m m m
U A V
implies
( )
1 1
1 1 1
1
0 [ ]
[0]
r
m m
m m m
t
r
m
m
k m
1
_
1 1 _
1
+
1 1
1
] , ] ,
]
U V
d A V
V
(4)
One of the equation we can obtain from (4) is
1 1 1 1 1 1
1 1 1 1
r
m m m m m m m
r
m m m m m
k
k
+ +
+ +
V U A U d V
V U d V
or
2-79
1
1 1 1
r
m m m m
k
d U V
(5)
Since
1 1 1 m m m
V U A
,
1 1
1 1
1 1
1 1
r
m m
m m
r
m m
r
m m
V PV
PU A
PU PA
W A
, (6)
where
0 0 1
0 1 0
1 0 0
1
1
1
1
1
]
P
L
L
M M M M
L
is the permutation matrix representing vector reversal, and
1 1 m m
W PU P
Because of the property of
1 m
U
,
1 1 m m
W U
(7)
Combining (5)-(7) implies
1
1 1 1
1
1 1 1
1
1 1 1
1
,
r
m m m m
r
m m m m
r
m m m m
r
m m
k
k
k
k
d U V
U W A
U U A
A
(8)
2-80
i.e. the vector
1 m
d
is the vector containing the coefficients of the ( 1) m -th
predictor arranged in reverse order and scaled by the term
m
k
.
The second equation we can derive from (4) is
( ) ( )
( ) ( )
1 1 1 1
1 1 1 1
[ ] [0]
[0]
t t
r r
m m m m m
t t
r r r
m m m m m m
m k
k k
+ +
+
V A V d
V A V A
or
( )
( )
( )
1 1 1 1
1
1 1
[ ] [ ]
[0]
t t
r r
m m m m
m t
r r
m
m m
m m
k
V A V A
V A
(9)
where
( )
( )
1 1 1
1 1
[0]
[0]
t
r r
m m m
t
m m
V A
V A
(10)
In summary,
1. At the end of the (m-1)-th iteration, the available information to the LPC
encoder are
1 m
and the (m-1)-th order predictor
1 m
A
.
2. In the m-th iteration, the encoder computes
m
k
according to (9) and set
the highest order term in the m-th order predictor to (see Eqn. (1))
, m m m
a k
2-81
3. The other predictor coefficients are computed according to (1) and (8) as
, 1, 1,
1, 1,
; 1,2,..., 1
m k m k m k
m k m m m k
a a d
a k a k m
+
, (11)
where
1, m k
d
is the k-th component of the vector
1 m
d
in (8)
4. Update the term
m
and increase m by 1.
Exercise: Show that the term
m