Professional Documents
Culture Documents
Weiss
• In the first part, you have studied a good deal of background theory;
• digital signal processing (DSP) is based on theory that goes back more than 200
years to Norbert Gauss and others;
• DSP has seen a revolution due to the arrival of inexpensive computational devices
in the 1990;
• DSP has penetrated almost every aspect of life: Texas Instruments claims that
every 10 minutes you use one of their dedicated digital signal processors.
1
Signal Processing, 2004/05 — 1. Overview S. Weiss
DSP Applications
• Multimedia: compression and coding algorithms for audio, video, and still images;
acoustic echo control for teleconferencing;
• Biomedical: signal detection; source separation for ECG or EEG; digital hearing
aids;
2
Signal Processing, 2004/05 — 1. Overview S. Weiss
sampling
at RF or IF
ADC audio
video
DSP
data
DAC etc
3
Signal Processing, 2004/05 — 1. Overview S. Weiss
• Continuous time signal: x(t) with t ∈ R; example: analogue speech signal or any
other voltage signal;
4
Signal Processing, 2004/05 — 1. Overview S. Weiss
5
Signal Processing, 2004/05 — 1. Overview S. Weiss
with lag τ , complex conjugation operator {·}∗ and expectation operator E{·};
2
• power of x[n] is given by rxx[0]; if x[n] is zero mean, then the variance σxx = rxx[0];
6
Signal Processing, 2004/05 — 2. Conversion S. Weiss
• a sensor (actuator) can convert a real world signal to (from) a convenient voltage
signal:
analogue
2
pressure electrical
0
s (t )
−1
transducer −2
0 1 2 3 4 5 6 7 8 9 10
time t / [msec]
7
Signal Processing, 2004/05 — 2. Conversion S. Weiss
• Many real world signals yield small amplitudes (e.g. EEG order of 10−6v), and
amplification is required prior to further processing;
• generally such signals are corrupted by noise; we assume an additive noise model:
0.5
v(t)
0
−0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time t / [s]
v [ n]
s[n] y[n]
1.5 1.5
1 1
0.5 0.5
x(t)
y(t)
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time t / [s] time t / [s]
8
Signal Processing, 2004/05 — 2. Conversion S. Weiss
2 2 2
• from assumptions −→ σyy = σxx + σvv and Pyy (ejΩ) = Pxx(ejΩ) + Pvv (ejΩ).
9
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Signal-to-Noise Ratio
• The signal to noise ratio is a power ratio:
signal power
SNR = (5)
noise power
2 2
• for zero-mean signals: SNR = σsignal /σnoise ;
• the range of values to be measured may span several orders of magnitude (such as
the human hearing); therefore a logarithmic scale has been introduced:
2
σsignal σsignal
SNRdB = 10 · log10 2 = 20 · log10 [deciBel, dB] (6)
σnoise σnoise
2 2
• Examples: σsignal = 1, 000 · σnoise −→ SNRdB = 30 dB;
2 2
σsignal = 1, 000, 000 · σnoise −→ SNRdB = 60 dB.
10
Signal Processing, 2004/05 — 2. Conversion S. Weiss
• analogue signals, such as a speech signal, may require a conversion into the digital
domain for applying sophisticated source coding algorithms:
analogue digital analogue
AD C DAC
in processing out
11
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Analogue-to-Digital Conversion
• sampling: the selection of the sampling period Ts will affect the possible
representation of the frequency content of x(t) by x[n];
• signal conditioning: contains an anti-alias filter (AAF) and a gain in order to adjust
the amplitude of x(t) to the input range of the quantiser Q.
12
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Sampling
∞
P
• Sampling x[n] from a continuous x(t): xs(t) = x(t) · δ(t − nTs);
n=−∞
• remembering the Fourier series theorem and the duality property of the Fourier
transform, it is clear that the frequency domain of the sampled signal xs(t) has
been periodised, w.r.t. ωs = 2π/Ts:
∞
X
Xs(jω) = X(j(ω − nωs)) (7)
n=−∞
• hence
X(jω) , Xs(jω)
ω
−2 ω s −ωs 0 ωs 2ω s
13
Signal Processing, 2004/05 — 2. Conversion S. Weiss
ω
−2 ω s −ωs 0 ωmax ω s 2ω s
no aliasing X(jω) , Xs(jω)
ω
−2 ω s −ωs 0 ωmax ω s 2ω s
14
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Signal Conditioning
• if the signal’s content is not exactly known, we can guarantee this by conditioning
the input signal to the ADC by an anti-alias filter:
H(jω)
ω
−ωs − ω s /2 0 ω s /2 ωs
• this filter removes all frequency components above ωs/2, which would otherwise
alias in the sampling stage.
15
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Quantisation
q/2 q/2
q2
Z
1 1 3
σe2 = 2
e · p(e)de = e = (9)
−q/2 q 3 −q/2 12
16
Signal Processing, 2004/05 — 2. Conversion S. Weiss
• with (9) and q = 2V /2R, the signal-to-quantisation noise ratio (SQNR) is given by
V 2 · 12 · 22R−2 2R
SQNRdB = 10 · log10 2
= 10 · log 10 2 ≈ 6.02 · R [dB] (11)
3·V
17
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Digital-to-Analogue Conversion
• For the onversion from the digital to the analogue domain, a DAC comprises of:
R Ts x ( t) , x[nTs ]
signal conditioning
x[n] −1
x ( t) t
Q RCF
• the voltage signal at the output of Q−1 contains high frequency components, which
need to be removed:
desired spectrum
spectral repetitions X(jω) , Xs(jω)
ω
−2 ω s −ωs 0 ωs 2ω s
18
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Reconstruction Filter
• In order to reconstruct the analogue signal x(t), a lowpass filter is required to
remove undesired high frequency components;
• this lowpass filter interpolates between adjacent sample values and is therefore
termed reconstruction filter; requirement: stopband edge at ωs/2:
H(jω)
ω
−ωs − ω s /2 0 ω s /2 ωs
• with a perfect anti-alias filter and reconstruction filter, the concatenation of ADC
and DAC imposes no loss on the signal (Nyquist);
• loss occurs due to (i) non-ideal anit-alias and reconstruction filters and (ii)
quantisation.
19
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Realistic ADC/DAC
• High quality analogue filters for anti-alias and reconstruction filtering require high
order and low tolerances on elements;
• we will study later how part of the analogue processing can be performed in the
digital domain by oversampling and Σ − ∆ converters.
20
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Spectrum Estimation
• The discrete Fourier transform (DFT) has been discussed previously (see sections
11 and 12 in Prof. Rogers’ notes);
• if x is an N -element vector taken from a tap delay line (TDL) and T an N -point
DFT matrix, then from the transform vector X
X = T·x (12)
21
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
2
• Gaussian (or normal) PDF of a process x[n] ∈ N (µx, σxx ):
−(x−µx )2
1 2
p(x) = √ e 2σxx (13)
2πσxx
22
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
23
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• The PDF does not contain any information on how “correlated” successive samples
are;
2
• consider the following scenario with x[n] ∈ N (0, σxx ) being uncorrelated (successive
samples are entirely random):
2 2
N (0, σxx ) N (0, σyy )
24
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• such a model can be very useful in characterising the stochastic process y[n] by an
innovations filter[3].
25
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
3 1.4 0.5
2.5 0.4
1.2
2 0.3
1
1.5 0.2
|H(e )|
1 0.1
0.8
jΩ
x[n]
y[n]
0.5 0
0.6
0 −0.1
−1 −0.3
0.2
−1.5 −0.4
−2 0 −0.5
0 50 100 150 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 50 100 150
time n norm. angular freq. Ω/π time n
• the output will have Gaussian distribution, but the signal only changes smoothly:
neighbouring samples are “correlated”. We need a measure!
26
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Auto-Correlation Function I
• The correlation between a sample x[n] and a neighbouring value x[n − τ ] is given
by
N −1
1 X
rxx[τ ] = E{x[n] · x∗[n − τ ]} = lim x[n] · x∗[n − τ ] (16)
N →∞ N
n=0
x[n], x[n+50]
x[n], x[n+3]
0.4 0.4
0.2 0.2
0 0
−0.2 −0.2
−0.4 −0.4
−0.6 −0.6
0 50 100 150 0 50 100 150
time n time n
• the curves on the left look “similar”, the ones on the right “dissimilar”.
27
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Auto-Correlation Function II
• large values in the ACF indicate strong correlation, small values weak correlation;
28
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• If a signal has no self-similarity, i.e. it is “completely random”, the ACF takes the
following form:
rxx[τ]
• If we take the Fourier transform of rxx[τ ], we obtain a flat spectrum (or a lowpass
spectrum for the ACF on slide 28);
29
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• The power spectral density (PSD), Rxx(ejΩ), defines the spectrum of a random
signal:
X∞
Rxx(ejΩ) = rxx[τ ] ejΩτ (18)
τ =−∞
• PSD and ACF form a Fourier pair, rxx[τ ] ◦—• Rxx(ejΩ), therefore
Z π
1
rxx[τ ] = Rxx(ejΩ) e−jΩτ dΩ (19)
2π −π
30
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
PSD – Examples
31
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Cross-Correlation
• The cross-correlation function between two signal x[n] and y[n] is defined analogous
to (16):
N −1
1 X
rxy [τ ] = E{x[n] · y ∗[n − τ ]} = lim x[n] · y ∗[n − τ ] (21)
N →∞ N
n=0
∗
• note: ryx[τ ] = rxy [−τ ] and especially rxx[τ ] = rxx[−τ ]
i.e. the auto-correlation function is symmetric, while the cross-correlation function
is not;
32
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• Delay estimation. Imagine we send a random pulse x[n] and wait for the reflected
return signal y[n]:
x[n] n
ryx[τ]
y[n] τ
n
T T
33
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• Consider again filtering a random signal x[n] with a filter having an impulse
response h[n] (either MA, AR, or ARMA):
rxx[τ ] ryy [τ ]
∞
P
• relation between x[n] and y[n] is given by convolution: y[n] = h[ν] x[n − ν];
ν=−∞
• we are looking for the relations between rxx[τ ] and ryy [τ ] and between Rxx(ejΩ)
and Ryy (ejΩ).
34
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
∗
• note: rxy [τ ] = ryx [−τ ] = h∗[−τ ] ∗ rxx[τ ]
35
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• Going further:
N −1
1 X
ryy [τ ] = lim y[n] · y ∗[n − τ ] (28)
N →∞ N
n=0
N −1 ∞
1 X X
= lim y[n] h∗[ν] x∗[n − ν] (29)
N →∞ N
n=0 ν=−∞
∞ N −1
X 1 X
= h∗[ν] lim y[n] x∗[ν − n + τ ] (30)
N →∞ N
ν=−∞ n=0
∞
X
= h∗[ν] ryx[ν + τ ] = h∗[−τ ] ∗ ryx[τ ] (31)
ν=−∞
36
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
rxx[τ ] ryy [τ ]
• note for innovations filter (uncorrelated input x[n] ∈ N (0, 1)): Ryy [τ ] = |H(ejΩ)|2.
37
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Averaged Periodogram I
• The PSD is a deterministic quantity of a random process, while the periodogram is
random;
• method 1: take N -point DFT of x[n], and average over a neighbourhood of squared
Fourier coefficients;
|X(e jΩ )| 2 , |X(e jΩ )| 2
Ω
π
• method 2: calculate a number of periodograms and average across the the frequency
bins (Fourier coefficients).
38
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Averaged Periodogramme II
x[n]
| . |2 | . |2 | . |2
|X 0(e jΩ )| 2 |X 1(e jΩ )| 2 |X 2(e jΩ )| 2
averaging
|X(e jΩ )| 2
39
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
|X[k]|
x[n]
0 10
N
−1 0
0 20 40 60 80 n 0 5 10 15
k
• N = 27, discontinuties arise at the window edges, causing the main peak to “leak”:
1
10
|X[k]|
x[n]
0
5
N
−1 0
0 20 40 60 80 n 0 5 10 k
40
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Recall Windowing
• DFT applied to a segment of rectangularly windowed sinusoidal data:
magnitude / [dB]
40
time 20
n 0 leakage
−20
10 20 30 k 40
magnitude / [dB]
40
time 20
n 0
−20
10 20 30 k 40
• with windowing, the spectral leakage is reduced at the cost of a widened main lobe.
41
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
| . |2 | . |2 | . |2 | . |2 | . |2 | . |2
averaging
|X(e jΩ )| 2
42
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• Note the differences and correspondences between the PSP and the averaged
periodogram:
E{·}
x[n] −→ rxx[τ ]
|X(ejΩ)|2 −→ Rxx(ejΩ)
E{·}
• spectral estimation (approximations of the PSD) are usually based on the averaged
periodogram method outlined in slide 42.
43
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Summary
• We have defined PSD theoretically and the averaged periodogram as a
computational technique to estimate it;
• usually the longer the data sequence x[n] available, the more confident (accurate)
this estimate is;
• we will define a parametric method for spectrum estimation, trying to guess at the
innovations filter behind the signal at hand;
[1] B. Girod, R. Rabenstein, and A. Stenger. Signals and Systems. J. Wiley & Sons, 2001.
[Good explanation of auto- and cross-correlation, and the PSD]
[2] T. K. Moon and W. C. Stirling. Mathematical Methods and Algorithms for Signal Processing.
Prentice Hall, 1999.
[3] A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 3rd ed.,
1991.
[A classic text.]
44
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
45
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Impulse Response
• For an FIR filter, the impulse response h[n] (earlier stated as zero-state response,
i.e. the TDL is initialised to zero) is given by its filter coefficients:
y [n]
x [n]
h[n]
n n
0 0 N
• the impulse response can give direct clues about multipath propagation, or echos
and reverberation.
46
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Frequency Response
• Frequency response H(ejΩ): measures the gain of the system in the steady state:
• impulse and frequency response are a Fourier pair: h[n] ◦—• H(ejΩ).
0.6 1.5
0.4
|H(ejΩ)|
1
h[n]
0.2
◦—• 0.5
0
−0.2 0
0 5 10 15 20 25 0 0.2 0.4 0.6 0.8 1
time index n normalised angular freq. Ω / 2π
47
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
0.5
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalised angular freq. Ω / 2π
−20
−40
−60
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalised angular freq. Ω / 2π
48
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Group Delay
• from H(ejΩ) = |H(ejΩ)|ejΦ(Ω), note for the group delay g(Ω):
jΩ d
g(e ) = − Φ(Ω) (37)
dΩ
0 14
g(ejΩ) / samples
∠ H(ejΩ) /rad
−20 13
−40 12
−→
−60 11
−80 10
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
normalised angular freq. Ω / 2π normalised angular freq. Ω / 2π
• the group delay tells us by how long certain frequency components will be delayed
when propagating through the filter;
• note the difference between linear phase and non-linear phase systems.
49
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
z-Transform
• Reconsider FIR filter:
• the z-transform with z = ejΩ+α can capture the transient behaviour of a system;
50
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
P∞
• Assume a z-transform pair h[n] •—◦ H(z) = i=−∞ hi z −i
• time shift:
h[n − ∆] •—◦ H(z) z −∆ (40)
• complex conjugation:
∞
X
h∗[n] •—◦ H ∗(ejΩ) = h∗i z −i (41)
i=−∞
• time reverse: ∞
X
h[−n] •—◦ H(z −1) = hi z i (42)
i=−∞
51
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Transfer Function
• the roots βi of H(z) are called the zeros of the transfer function;
52
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
• If the impulse response only contains real-valued coefficients, the zeros βi must be
either real valued, or occur as complex conjugate pairs (as seen in slide52;
X(z) -
b0 -
1 − β0z −1 ··· -
1 − βN −2z −1 -
Y (z)
53
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
• For a minimum phase system, all zeros lie inside the unit circle;
• a maximum phase system has all zeros outside the unit circle;
∠ Hi(ejΩ) /rad
0
H1(z) −2
H2(z) •—◦
−4
H3(z) = H1(z)H2(z)
−6
0 0.2 0.4 0.6 0.8 1
normalised angular freq. Ω / 2π
54
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
• a linear phase system must have a z-transform of the form (including a possible
delay for causality)
−1
H(z) = Hmin. phase (z) Hmin. phase (z ) Πm(1 − ejφm z −1) (45)
• any zero of a linear phase system must have a reciprocially matching partner (unless
on the unit circle);
• linear phase iff impulse response is symmetric: recall that hmin. phase [n] ∗
hmin. phase[−n] will give an auto-correlation type result.
55
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Summary
• Note that certain properties of a system are best assessed in a specific domain:
either impulse response, frequency domain, or transfer function;
• phase information is often vital, e.g. if we need to invert the system (for equalisation
or control purposes);
• most FIR filter designs yield symmetric impulse response and hence are linear phase;
56
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
cutoff stopband
frequency transition edge
band width
57
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
• Four different basic filter types as defined by their magnitude response: (a) lowpass,
(b) highpass, (c) bandpass, and (d) bandstop filter:
(a) H (e j Ω) (b) H (e j Ω)
Ω Ω
0 π 0 π
(c) H (e j Ω) (d) H (e j Ω)
Ω Ω
0 π 0 π
58
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
• some windowing may be required in order to enforce finite support in the time
domain.
59
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
−20
0
0 5 10 15 20 −40
0.2
−60
0.1
h2[n]
0 −80
0 0.1 0.2 0.3 0.4 0.5
0 10 20 30 40 50 Ω / 2π
time index n
60
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
IIR Filtering
• Various ways to represent the difference equation as a flow graph:
L−1
X J−1
X
y[n] = al x[n − l] + bj y[n − j] (46)
l=0 j=1
x [n] x [n -1 ] x [n-2 ] x [n - L+1]
adder
multiplier a[J-1 ] a[2] a[1]
delay element
y [n - J +1] y [n-2 ] y [n -1 ]
61
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
IIR Stability
• transfer function:
• : note that H(z) can be spit into a cascade of FIR and recursive first-order sections;
1
• each recursive section 1−αz −1
must be stable; note geometric series
∞
1 X
i −i
−1
= α z for |a| < 1 (49)
1 − αz i=0
62
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
• the analogue design is transformed from the Laplace- to the z-domain by means of
the bilinear transform:
1 1 − z −1
s= −1
(50)
Ts 1 + z
• IIR filters generally achieve high quality filters with less coefficients than FIR.
∞
1 X
i −i
−1
= a z for |a| < 1 (51)
1 − az i=0
63
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
FIR/IIR Comparison
0 0 11.5 80
g(ejΩ) / [samples]
|H(ejΩ)| / [dB]
11.5 60
−20 −20
11.5 40
−40 −40
11.5 20
−60 −60
11.5 0
FIR quantised, B=8 bit IIR quantised, B=8 bit FIR quantised, B=8 bit IIR quantised, B=8 bit
20 20 11.5 150
0 0
11.5
g(ejΩ) / [samples]
|H(ejΩ)| / [dB]
100
−20 −20
11.5
−40 −40
50
11.5
−60 −60
64
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
Computational Complexity I
L−1
X J−1
X
y[n] = blx[n − l] + aj y[n − j] = bHxn + aHyn−1 (52)
l=0 j=1
65
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
Computational Complexity II
66
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
• Linear Phase. For FIR filters a symmetric impulse response is necessary and
sufficient for linear phase; IIR are never linear phase (although some yield an
approximation in the passband);
• Implementation. Both FIR and IIR are sensitive to fixed point implementation,
but IIR has larger dynamics in its filter coefficients and may even become unstable
when quantised.
67
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Adaptive Filtering
x[ n] d[ n]
w0 w1 w2 wL-1
e [ n]
y[ n]
• here the adjustment of the coefficients w[n] may be dependent on the environment,
and the coefficients may be variable over time.
68
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
x[ n] d[ n]
w0 w1 w2 wL-1
e [ n]
y[ n]
• Aim of an adaptive filter: the filter w[n] forms an output y[n] from an input
x[n], such that, when subtracted from the desired signal d[n], the reslting error is
minimised in a suitable sense.
69
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• noise cancellation
• prediction
v [ n] + x[n]
d[ n] d[ n]
x[ n] y[ n] e [ n] x[ n] y[ n] e [ n]
w
w
70
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
System Identification
x[ n] y[ n] e [ n]
w
• if the adaptive filter converges and e[n] is (ideally) zero, then the input-output
behaviour of c[n] and w[n] is identical;
71
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• if the error signal is minised, then w[n] represents the inverse system of c[n];
• if c[n] is non-minimum phase, we need to permit a delay ∆, such that c[n] ∗ w[n] ≈
δ[n − ∆].
72
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Prediction
• Due to delaying the input signal x[n], the desired signal d[n] is a “future version”
of the filter input:
d[ n]
x[ n] y[ n] e [ n]
w
• the adaptive filter should only be able to predict “periodic” or correlated components
with d[n];
• the error signal e[n] would in this case contain any remaining uncorrelated signal
components.
73
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• consider a noise corrupted signal d[n] (e.g. helicopter pilot’s microphone signal):
v [ n] + x[n]
d[ n]
x[ n] y[ n] e [ n]
w
• if we can obtain a reference signa x[n] of the noise, the adaptive filter should be
able to cancel the noise component from e[n], provided that the noise in x[n] and
d[n] is linearly related.
74
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• due to the reverberation times of a room, which the adaptive filter has to match,
this is a very computationally demanding task.
75
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
2
• In order to calulate an optimal filter wopt[n], the mean square error (MSE) E e [n]
is a good criterion to be minimised;
• with definitions: T
w = w0 w1 · · · wL−1 ;
T
xn = x[n] x[n − 1] · · · x[n − L + 1] ;
• (we could do the derivation for complex valued signals, but for simplicity here stick
with the real valued case.)
76
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• Mean Square Error formulation (note that the optimum filter wopt is deterministic!):
2 T 2
E e [n] = E (d[n] − w · xn)
2 T T
T
= E d [n] − 2w · E{d[n] · xn} + w · E xn · xn · w
= σd2 − 2wT · p + wT · R · w
with covariance matrix R = E xn · xT
n and correlation vector p = E{d[n] · xn};
2
• important: the expression for E e [n] is quadratic in the filter coefficients w;
hence optimisation should lead to a unique extremum!
77
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
N −1 N −1
1 X 1 X
rxx[τ ] = x[n] · x[n − τ ] rxd[τ ] = d[n] · x[n − τ ]
N n=0 N n=0
78
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
MSE Surface
2
• Quadratic in the filter coefficients w, the surface of the MSE ξ = E e [n] is a
hyperparabola in L + 1 dimensional hyperspace;
w1
w1,opt
w0
w0,opt
79
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
∂ 2 !
E e [n] = 0
∂w
∂ 2
E e [n] = 0 − 2p + 2R · w
∂w
• if R is invertible, then the optimum filter coefficients wopt are given by (Wiener-Hopf
equation):
wopt = R−1 · p
80
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
∂ 2 !
E e [n] = 0
∂w
∂ 2
E e [n] = 0 − 2p + 2R · w
∂w
• if R is invertible, then the optimum filter coefficients wopt are given by (Wiener-Hopf
equation):
wopt = R−1 · p
81
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Viability of Wiener-Hopf
82
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Iterative Solutions
• In order to bypass the problems of Wiener Hopf, we will utilise iterative approaches
to converge towards the optimal solution;
ξ
– Starting from an initial guess w[0],
gradient descent techniques follow
the negative gradient of the MSE large µ
cost function:
83
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• therefore:
w[n + 1] = w[n] + 2µ(p − Rw[n]) (58)
84
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• the gradient estimate is noise, but through itertion, on average the gradient estimate
points into the right direction.
85
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
x[ n] d[ n]
w0 w1 w2 wL-1
e [ n]
y[ n]
• Complexity: 2L multiply accumulate (L for scalar product, L for LMS update) per
sampling period.
86
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
LMS Convergence
• intuitive from MSE surface plot: small (large) µ −→ slow (fast) convergence with
high (low) final accuracy;
1
0 < µ < 2
(61)
Lσxx
2
where L is the filter length and σxx the variance of the input signal.
87
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
88
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• Convergence of error:
0
|e[n]| / dB
−100
−200
−300
0 100 200 300 400 500 600 700 800 900 1000
iterations n
• The error reaches Matlab’s floating point accuracy of approximately 10−16 — this
is numerically zero.
• after convergence, the filter coefficients match the impulse response of the unknown
system c[n].
89
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
1 1
W (z) = = (62)
C(z) 1 + 0.5z −1
1
P∞ n
• we can use the geometric series, 1−a = n=0 a for |a| < 1, to find an FIR
approximation of the inverse:
∞
X
W (z) = (−.5)nz −n = 1 − .5z −1 + .25z −2 − 0.125z −3 + · · · (63)
n=0
• For uncorrelated input, an adaptive filter of length L would identify the first L
elements of this geometric series.
90
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
1 2 −1 −2
W (z) = = = 2 − 4z + 8z − ··· (64)
C(z) 1 + 2z −1
∞
z X
W (z) = =z (−.5)nz n = z − .5z 1 + .25z 2 − · · · (65)
1 + 0.5z n=0
91
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
∞
z −2 X
W (z) = = z −2 (−.5)nz (66)
1 + 0.5z n=0
an adaptive filter can identify the 3 causal coefficients of the geometric series.
• rule of thumb: not knowing anything about thye phase properties of C(z),
practisioners choose a delay of L/2.
92
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• the inverse system would attempted to provide a very high (ideally infinite) gain at
fs/2 — this is very dangerous!
• inspect the covariance matrix R; for uncorrelated input u[n] to the channel, the
autocorrelation function of x[n] is given by
• try to establish what filter response the LMS algorithm converges to.
93
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• In a realistic scenario, a channel suffers from additive white Gaussia noise v[n],
such that
x[n] = c[n] ∗ u[n] + v[n] (69)
whereby v[n] is independent of u[n];
whereby Rcc is the covariance matrix of the noiseless channel output, and Rvv =
2
σvv I the covariance matrix of the AWGN.
94
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
2
λc,0 + σvv 0 0
2
H 2
0 λc,1 + σvv 0 0 H
Rxx = QΛQ
| {z } +σ vv I = Q
.. ... .. Q
Rcc 2
0 0 λc,L−1 + σvv
(72)
• simulate the previous example with channel noise: in the presence of channel
noise, the adaptive filter strikes a balance between inversion of C(z) and noise
amplification;
• the adaptive filter provides a minimum mean square error (MMSE) solution rather
than an inversion: the system inverse is “regularised” by the AWGN.
95
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
• Adaptive filters tuneable filter coefficients and can be applied in situation where
exactly desired response (highpass, lowpass, etc) is not known a priori;
[1] B. Widrow and S.D. Stearns Adaptive Signal Processing. Prentice Hall, 1985.
[Classical and easy to read text.]
[2] S. Haykin Adaptive Filter Theory. Prentice Hall, 1987, 1991, and 1996.
[A good mathematical treatment — strangely the 2nd edition is considered
better than the third.]
96
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
• Audio DSP: you may need to convert from professional audio (96kHz) down to HIFI
(e.g. digital audio tape, 48kHz) or “wideband” audio (teleconferencing, 16kHz);
• Software defined radio (SDR): you may need to create a digital IF signal (sampled
at, say, 75MHz) from a UMTS baseband signal (5MHz sampling rate);
Similarly, in an SDR receiver, you may have to extract a symbol stream at 5MHz
from a 75MHz IF signal.
97
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Multirate Operations
'$
-
anti-alias filter -
↓N -
&%
'$
-
↑N -
interpolation filter -
&%
98
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
−0.5
0 5 10 15 20 25 30 35 40
time index n
0.5
xd[m]
−0.5
0 1 2 3 4 5 6 7 8 9 10
time index m=n/4
99
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
• similar to sampling on slide 13, resampling “periodises” the spectrum with respect
to the new sampling rate ωs0 = ωs/N ; example for N = 4 (with unnormalised
angular frequencies):
X(jω)
ω
−ω s 0 ωs
Xs(jω)
ω
−4ω s −ω s 0 ωs 4 ωs
• note: if the original signal is not suitably bandlimited, aliasing may occur.
100
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Anti-Alias Filtering
• A digital anti-alias filter can be employed to filter the signal prior to decimation:
H(e jΩ)
Ω
−π −π /N 0 π /N π
• it is difficult to create a brick-wall type filter; however, in the digital domain we can
achieve a high quality filter design at a lower “cost” than in the analogue domain.
101
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
−0.5
0 1 2 3 4 5 6 7 8 9 10
time index m
0.5
xe[n]
−0.5
0 5 10 15 20 25 30 35 40
time index n
• note that our signal is now impulsive . . . our signal can be expected to contain
some high frequency components.
102
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
ω
−4ω s −ω s 0 ωs 4 ωs
Xe( jω)
ω
−ω s 0 ωs
103
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Ω
−π −π /N 0 π /N π
104
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Oversampled ADC
• Consider sampling an analogue signal at fs = N fN rather than fs = fN :
NTs R
signal conditioning
x ( t) x[m] x[n]
AAF Q AAF N
ω
0 πf N Nπ f N
• the transition band of the anti-alias filter can be much wider than in the non-
oversampled case; to compensate, we will postprocess the oversampled digital x[m]
by (much cheaper) digital operations.
105
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
signal
noise
Ω
−π −π /N 0 π /N π
• additionally, the digital AAF reduces the quantisation noise power (q 2/12) by a
factor 1/N (recall that the area under PSD is a measure for the power!).
106
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
• Assume that the original signal was quantised with a word length R;
• the quantisation noise power after the digital anti-alias filter has removed the
out-of-band noise for N = 4:
1 q2 (q/2)2
σq2 = = (73)
4 12 12
• if the quantisation step size is halved, we have gained an extra bit in resolution;
107
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
• to achieve 16 bit quality from a single bit quantiser, we would have to oversampled
by a factor 415;
• single bit converters exist, but use an additional technique, whereby the quantisation
noise is “pushed” towards higher frequencies and can be better suppressed by a
digital lowpass anti-alias filter.
108
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Sigma-Delta Converter
DAC
• The input sigma is passed through an integrator (lowpass filter), and the output is
converted with single bit accuracy, i.e. only retaining the sign;
• the output bit-stream is converted into a voltage signal by a DAC and subtracted
from the input.
109
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
z−1
1 bit DAC
• this means that the output contains the desired signal plus noise with a highpass
PSD.
110
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
0
PSD / [dB]
−10
−20
AWGN
−30 noise shaping
−40
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalised angular frequency Ω / π
• if we apply this in combination with oversampling, due to the reduced noise power
in the baseband, substantial resolution can be gained.
111
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Summary
112