Digital Audio Applications

Digital Audio Applications

• Some Practical Multimedia Digital Audio Applications:
‣ Digital Audio Synthesis — making some sounds

‣ Digital Audio Effects — changing sounds via some standard effects.

2
 Digital Audio Synthesis
• Several Approaches:
‣ Subtractive synthesis

‣ Additive synthesis

‣ FM (Frequency Modulation) Synthesis

‣ Sample-based synthesis

‣ Wavetable synthesis

‣ Granular Synthesis

‣ Physical Modeling

3
 Subtractive Synthesis
• Basic Idea:
‣ Subtract overtones from a sound via sound synthesis, characterized by the
application of an audio filter to an audio signal.
• Ex:
‣ Take the output of a sawtooth generator

‣ Use a low-pass filter to dampen its higher partials generates a more natural
approximation of a bowed string instrument than using a sawtooth generator
alone.

4
Subtractive Synthesis: A Human Example

• We can regard the way in which humans make noises as subtractive

synthesis:
‣ Oscillator — the vocal cords act as the sound source and

‣ Filter — the mouth and throat modify the sound.

- Saying or singing “ooh” and “aah” (at the same pitch.)

- Vocal chords are generating pretty much the same raw, rich in harmonic sound
Difference between the two comes from the filtering which we apply with the
mouth and throat.

- Change of mouth shape varies the cutoff frequency of the filter, so removing
(subtracting) some of the harmonics.

- The “aah” sound has most of the original harmonics still present
- The “ooh” sound has most of them removed (or to be more precise, reduced in
amplitude.)

5
 tive synthesis: Electronic Control
Subtractive synthesis: Electronic Control
elements:
c elements:
al: •Common
Three basicsource
elements:signals: square
al: waves,
Common pulsesource
waves, signals:
sawtoothsquarewaves
‣ Source signal:
waves,
and pulsewaves.
triangle waves, sawtooth waves
- Common source signals: square waves, 

andpulse
triangle
waves,
waves.
sawtooth(digital
waves andand triangle 

Modern synthesisers
waves.synthesisers (digital and
Modern
software) may include more complex
- Modern synthesizers
software) may include(digital
more and complex
software) 

waveforms or allow the upload
may include more complex waveforms or 

of
waveforms
arbitrary or allow
waveforms the upload of
allow the upload of arbitrary waveforms
arbitrary waveforms
‣ Filtering: the cut-off frequency and 

g: The cut-o↵ frequency and resonance
ng: ofTheresonance
cut-o↵ offrequency
the filter areand
controlled in 

resonance
the filter
order are controlled
to simulate in order
the natural timbre of a 

of the
to given filter
simulate are controlled in
the natural timbre of a order
instrument.
to simulate the natural timbre of a
given instrument.
‣ Amplitude Envelope:
given instrument.
- Further envelope control of signal amplitude (strictly: not subtractive synthesis
but frequently used). Also used with other synthesis techniques.
de Envelope: Further envelope control of signal amplitude
ude Envelope: Further envelope control 6
of signal amplitude
dditive synthesis
Basic Idea: Additive synthesis
Additive synthesis refers to the idea that complex tones can be
• Basic by
created Idea:
the summation, or addition, of simpler ones.
‣ Additive synthesis refers to the idea that complex tones can be created by the
summation, or addition, of simpler ones.
Frequency
- Frequency mixing
mixing is the
theessence
essence
 of
of additive
additive synthesis.
synthesis.
- Each of the frequency components 

Each(orof the frequency
partials) of a sound components
has its own 

(or amplitude
partials) envelope.
of a sound has its own
- This allows
amplitude envelope.
for independent behavior 

of these components.
This allowscan
- Sources forbeindependent
other forms of 

behaviour
synthesisoforthese components.
samples.

Sources can be other forms of

synthesis or samples.
3106 Chapter 5: Audio Synthesis Additive synthesis 20
7
 FM (Frequency Modulation) Synthesis

• Basic Idea: Timbre of a simple waveform is changed by frequency

modulating it with a frequency resulting in a more complex waveform —
Fundamentals of Multimedia, Chapter 6
different-sounding.
Fundamentals of Multimedia, cos
Chapter
(2 t) 6 cos (4 t)
• Used in popular 1980s Yamaha Synthesisers: DX7, Casio CZ ..... still in use
1.0 1.0
today cos (2 t) cos (4 t)
0.5 0.5
Magnitude

Magnitude
1.0 1.0
0.0 0.0
0.5 0.5
Magnitude

Magnitude
−0.5 −0.5
0.0 0.0
−1.0 −1.0
−0.5 0.0 0.2 0.4 0.6 0.8 1.0 −0.5 0.0 0.2 0.4 0.6 0.8 1.0
Time Time
−1.0 −1.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
(a)
Time (b)
Time
cos (cos (4 t)) cos (2 t + cos (4 t))
(a) (b)
1.0 1.0
cos (cos (4 t)) cos (2 t + cos (4 t))
0.5 0.5
Magnitude

Magnitude

1.0 1.0
0.0 0.0
0.5 0.5
Magnitude

Magnitude

−0.5 −0.5
0.0 0.0
−1.0
−0.5 −1.0
0.0 0.2 0.4 0.6 0.8 1.0 −0.5 0.0 0.2 0.4 0.6 0.8 1.0
Time 8 Time
 Sample-based synthesis
• Basic Ideas: Similar to subtractive synthesis or additive synthesis.
• The principal difference is that the seed waveforms are sampled sounds or
instruments instead of fundamental waveforms such as the saw waves of
subtractive synthesis or the sine waves of additive synthesis.
• Musical genres: Hip-hop, Trip-hop, Dance music, Garage, Jungle, Trance,
Modern Electronica invented due to samplers.
• Most music production now uses samplers.

9
 Sample-based synthesis: Comparison with
other Synthesis methods

• Advantages:
‣ Processing power requirements are much lower.

‣ Nuances of the sound models are contained in the pre-recorded samples

rather than calculated in real-time.

• Disadvantage: in order to include more detail, multiple samples might need

to be played back at once
‣ E.g. a trumpet might include a breath noise, a growl, and a looping
soundwave used for continuous play
‣ Reduces the polyphony as sample-based synthesizers rate their polyphony
based on the number of multi-samples that can be played back
simultaneously.

10
Sample-based synthesis Basics: Looping

• A sample-based synthesizer’s ability to reproduce the nuances of natural

instruments is determined primarily by its library of sampled sounds.
• Early days of Sampling (c. Late 1980s/Early 90s)
‣ Computer memory expensive:

- Samples had to be as short and as few as possible.

‣ This was achieved by looping a part of the sample

• Looping today:
‣ Looping still useful for

- Saving sample memory space — efficiency

- Looping audio material: Drum tracks, sound effects, etc.

11
Sample-based synthesis Basics: Looping

• Finding looping points:

‣ Simple idea: Find silence points (zero (amplitude) crossings) in sample.
‣ Alternative: Find portions in sample that have same audio content — pattern
matching.
• Pitch control:
‣ Speed or slow up sample to change pitch (realism to only a few semitones in
pitch change)
‣ Still need some sample across the range of the keyboard
‣ As memory became cheaper (and now with software based sample synthesis),
it became possible to use multisampling — looping still used in individual
samples.
• Finishing off the loop:
‣ Early Days: Use a volume envelope curve to make the sound fade away.
‣ Today: Include tail off sample in data — triggered by note off.

12
 Wavetable synthesis
• Similar to simple digital sine wave generation/additive synthesis but
extended at least two ways:
‣ Waveform lookup table contains samples for not just a single period of a sine
function but for a single period of a more general waveshape.
‣ Mechanisms exists for dynamically changing the waveshape as the musical
note evolves: generating a quasi-periodic function in time.

13
Wavetable synthesis: Advantages
• Well suited for synthesizing quasi-periodic musical tones because wavetable
synthesis can be as compact in storage requirements
‣ Amount of data being stored and used for this synthesis method is far less
than just the PCM sample of same sound.
‣ As general as additive synthesis but requires much less real-time
computation.
• Wavetable synthesis takes advantage of the quasiperiodic nature of the
waveform to remove redundancies and to reduce the data.

14
 Granular Synthesis
• ”All sound is an integration of grains, of elementary sonic particles, of sonic
quanta.” -Iannis Xenakis, Greek Composer (1971).
• Granular Synthesis
‣ Sound synthesis method that operates on the microsound time scale.

‣ Based on the same principles as sampling/wavetable synthesis but often

includes analog technology as well.
‣ Difference Samples are not used directly to make usual sounds:

- Split in small pieces of around 1 to 50 ms (milliseconds) in length, the grains.

- Multiple grains may be layered on top of each other all playing at different
speed, phase and volume.

15
Granular Synthesis: Making Sounds
• Sounds made by the generation of thousands of short sonic grains:
‣ Combined linearly to form large scale audio events

‣ 3 possible combinations:

- Quasi-synchronous granular synthesis

- Asynchronous granular synthesis
- Pitch/Tempo-synchronous granular synthesis
‣ The grains’ characteristics are also definable and when combined affect the
overall sound.

16
anular Synthesis: Making Sounds (Cont.)
A grain stream of equal duration grains, produces amplitude
Granular
modulation
Quasi-synchronous
Synthesis:
(see later) Making Sounds
grain durations less than 50 ms.
withsynthesis:
granular

• Quasi-synchronous
A grain stream of granular synthesis:
equal duration grains, produces amplitude
‣modulation (seeof later)
A grain stream with grain
equal duration durations
grains, less
produces than 50modulation
amplitude ms. with
grain durations less than 50 ms.

Grain streams with variable delay time between grains: the sum of
which‣ Grain
resembles
streams asynchronous
with variable delay granular synthesis.
time between grains: the sum of which
Grain streams with variable delay time between grains: the sum of
resembles asynchronous granular synthesis.
which resembles asynchronous granular synthesis.

17
Granular Synthesis: Making Sounds (Cont.)
Granular Synthesis: Making Sounds
Asynchronous granular synthesis:
• Asynchronous granular synthesis:
‣Grains
Grains are distributedstochastically
are distributed stochastically
with with no regularity.
no quasi quasi regularity.

CM3106 Chapter 5: Audio Synthesis Granular Synthesis 86

18
Granular Synthesis: Making Sounds
• Pitch/Tempo-synchronous granular synthesis:
‣ Preserve Pitch/Tempo while altering sample playback speed

‣ Overlapping grain envelopes designed to be synchronous with the frequency

of the grain waveform, thereby producing fewer audio artifacts.

19
 Physical Modeling
• Physical modeling synthesis
‣ The synthesis of sound by using a mathematical model: sets of equations and
algorithms to simulate a physical source of sound.
- Sound is generated using model parameters that describe the physical
materials used in the instrument and the user’s interaction with it

- For example, to model the sound of a drum, there would be a formula for how
striking the drumhead injects energy into a two dimensional membrane.

• Examples of physical modeling algorithms:

‣ Karplus-Strong strong synthesis (1971)

‣ Digital waveguide synthesis (1980s)

‣ Formant synthesis (1950s)

20
 Physical Modeling Example
• Karplus-Strong Algorithm
‣ Simple Algorithm: Makes a musical sound from noise
Physical Modelling Exampe
- Loops a short noise burst through a filtered delay line to simulate the sound of
a hammered or pluckedAlgorithm
Karplus-Strong string or some types of percussion.

- Feedback,Simple
Filtering and delay.
Algorithm: Makes a musical sound from noise
- Essentially subtractive
Loops a short noise burst
synthesis throughbased
technique a filtered
ondelay line to loop similar to
a feedback
that of a combsimulate
filter. the sound of a hammered or plucked string or
some types of percussion.

TD

Feedback, Filtering and delay.

Essentially subtractive synthesis technique based on a
feedback loop similar to that of a comb filter.
CM3106 Chapter 5: Audio Synthesis Physical Modelling 93
21
 Digital Audio Effects
• Digital audio effects may be applied:
‣ As part of the audio creation/synthesis stage — to be subsequently filtered,
(re)synthesized
‣ At the end of the audio chain — as part of the production/mastering phase.

‣ Effects can be applied in different orders and sometimes in a parallel audio

chain.
‣ The order of applying the same effects can have drastic differences in the
output audio.
‣ Selection of effects and the ordering is a matter for the sound you wish to
create. There is no absolute rule for the ordering.

22
 Effects Types
• Audio effects can be classified by the way process signals:
‣ Basic Filtering: Lowpass, High pass filter etc., Equalizer

‣ Time Varying Filters: Wah-wah, Phaser

‣ Delays: Vibrato, Chorus, Echo

‣ Modulators: Ring modulation, Tremolo, Vibrato

‣ Non-linear Processing: Compression, Limiters, Distortion, Exciters/Enhancers

‣ Spacial Effects: Reverb, Surround Sound

23
 Time-varying Filters
• Some common effects are realized by simply time varying a filter in a couple

Wah-wah Example
of different ways:
‣ Wah-wah: A bandpass filter with a time varying centre (resonant) frequency
The signal flow
and a small for a wah-wah
bandwidth. ismixed
Filtered signal as follows:
with direct signal.
direct-mix
x(n) + y(n)

Time
Varying wah-mix

BP
where BP is a time varying frequency bandpass filter.
‣ Phasing: A notch filter, that can be realized as set of cascading IIR filters,
again mixed with direct signal.
• A phaser is similarly implemented with a notch filter replaci
the bandpass filter.
• A variation is the M -fold wah-wah filter where M tap del
24
 Time Varying Filter Implementation:
State Variable Filter

• Time varying filter: independent control over the cut-off frequency and
damping factor of a filter.
• We can implement a State Variable Filter to solve this problem.

25
 State
The State Variable
Variable Filter Filter
yh (n) F1 yb (n) F1 yl (n)

x(n) + + + +

T T
1 Q1

T
1

T
‣ x(n): input signal
where:
‣ yl(n): lowpass signal
x(n) = input signal
‣ yb(n): bandpass signal
y
‣ yh(n): highpass signal l (n) = lowpass signal
yb(n) = bandpass signal
y (n) = highpass
26 signal
 TheState Variable Filter
The State Variable Filter Algorithm
State Variable Filter Algorithm
The algorithm difference equations are given by:
The algorithm difference equations are given by:
• The difference equations are given by:
 219

 yl (n) = F1yb(n) + yl (n 1) 219
y l (n) = F1 yb (n) + yl (n
yb(n) = F1yh(n) + yb(n 1) 1)


y yb(n)
(n) = F1yh(n)y+(nyb(n1) 1)Q y (n 1)
= x(n)

 h l 1 b


yh(n) = x(n) yl (n 1) Q1yb(n 1)
with tuning coefficients F1 and Q1 related to the cut-off frequency,
with tuning
with coefficients
tuning F1 andFQ1and
coefficients related
Q to the to
related cut-off
the frequency
cut-off fc and
frequency,
fc, and damping, d: 1 1
damping
fc, andd damping, d:
F1 = 2 sin(⇡fc/fs), and Q1 = 2d
F1 = 2 sin(⇡fc/fs), and Q1 = 2d
JJ
JJ
II
IIJ
JI
I
Bac
Back
Clos
Clos

27
 Delay Based Effects
• Many useful audio effects can be implemented using a delay structure:
‣ Sounds reflected of walls

‣ Vibrato, Flanging, Chorus and Echo are examples of delay effects

• Basic delay structure:

‣ We use FIR and IIR comb filters

‣ Combination of FIR and IIR gives the Universal Comb Filter

28
 FIR Comb Filter
• This simulates a single delay:
‣ The input signal is delayed by a given time duration τ

‣ The delayed (processed) signal is added to the input signal some amplitude
gain g
‣ The difference equation is simply: 

y(n) = x(n) + gx(n − M) with M = τ / fs
‣ The transfer function is:

H(z) = 1 + gz-M

29
FIR Comb Filter
y(n)
x(n) +
1

TM

x(n M)

30
 IIR Comb Filter
• This simulates a single delay:
‣ Simulates endless reflections at both ends of cylinder.

‣ We get an endless series of responses, y(n) to input, x(n).

‣ The input signal circulates in delay line (delay time τ ) that is fed back to the
input..
‣ Each time it is fed back it is attenuated by g.

‣ Input sometime scaled by c to compensate for high amplification of the

structure.
‣ The difference equation is simply:

y(n) = Cx(n) + gy(n−M) with M = τ / fs
‣ The transfer function is: 

H(z) = c / (1 − gz−M)

31
 IIR Comb Filter
y(n)
x(n) +
c

TM

y(n M)

32
 versal Comb Filter
Universal Comb Filter
combination of the FIR and IIR comb filters yields the Uni
b Filter:
• The combination of the FIR and IIR comb filters yields the Universal Comb
Filter:
sically this is an allpass filter with an M sample delay ope
‣ Basically this is an allpass filter with an M sample delay operator and an
d an additional
additional multiplier,
multiplier FF. FF.
BL

FF
x(n M) y(n)
x(n) + TM +

Parameters: FF = feedforward,
FB FB = feedbackward, BL = blend

33
Universal Comb Filter Parameters
234

Universal in that we can form any comb filter, an allpass or a delay:

• Universal in that we can form any comb filter, an allpass or a delay:

BL FB FF
FIR Comb 1 0 g
IIR Comb 1 g 0
Allpass a a 1
delay 0 0 1

JJ
II
J
I
Back
Close

34
Vibrato - A Simple Delay Based Effect

• Vibrato — Varying (modulating) the time delay periodically.

• If we vary the distance between an observer and a sound source (cf. Doppler
effect) we here a change in pitch.
• Implementation: A Delay line and a low frequency oscillator (LFO) to vary
Vibrato MATLAB Example (Cont.)
the delay.
• Only listen to the delay — no forward or backward feed.
The output from the above code is (red plot is original audio):
• Typical delay time = 5–10 Ms and LFO rate = 5–14Hz.
0.3
Vibrato First 500 Samples

0.2

0.1

−0.1

−0.2

−0.3

−0.4
0 50 100 150 200 35
250 300 350 400 450 500
 Comb Filter Delay Effects:
CombFlanger,
Filter Delay Effects:
Chorus, Slapback, Echo
Flanger, Chorus, Slapback, Echo
• A few
•Apopular effectseffects
few popular can becan
made with a
be made comb
with filter filter
a comb (FIR (FIR
or IIR) and some
or IIR)
modulation
and some modulation 240

• Flanger, Chorus,
• Flanger, Slapback,
Chorus, Echo Echo
Slapback, samesame
basicbasic
approach but but
approach different sound
different
sound outputs:
outputs:

Effect Delay Range (ms) Modulation

Resonator 0 . . . 20 None
Flanger 0 . . . 15 Sinusoidal (⇡ 1 Hz)
Chorus 10 . . . 25 Random
Slapback 25 . . . 50 None
Echo > 50 None
JJ
• Slapback (or doubling) — quick repetition of the sound, II
‣ Slapback (or doubling) — quick repetition of the sound
Flanging — continuously varying LFO of delay, J
‣ Flanging — continuously
Chorus — multiple varying
copies LFO of delay
of sound delayed by small random I
delays
‣ Chorus — multiple copies of sound delayed by small random delays Back
Close

36
langer MATLAB Example (Cont.)
Flanger
The output from the above code is (red plot is original audio):
Flanger and original Signal
1.5

0.5

−0.5

−1

−1.5
0 5 10 15
4
x 10

0.5

−0.5

−1
Original Audio
0 5 10 15
4
x 10

Click here to hear: original audio,

37 flanged audio.