Professional Documents
Culture Documents
• General introduction
• Communication by sound and voice
– Examples of communication situations
• Systems approach to communication
• Modeling and theory formation in research
1
M. Karjalainen
Information Transmission by Sound
2
M. Karjalainen
Communication by Speech
3
M. Karjalainen
Communication by Music
6
M. Karjalainen
Virtual Acoustic Reality
Input-output relationship
9
M. Karjalainen
A Systems Approach
A multi-level system
10
M. Karjalainen
Systemic Concepts
11
M. Karjalainen
Abstraction in Modeling and Theory Formation
Abstraction hierarchy
12
M. Karjalainen
Communication by Sound and Voice
contentware
Information Cognition
functionware
Analysis Synthesis
software
Signals Physics
hardware
13
M. Karjalainen
Chapter 2: Acoustics
1
M. Karjalainen
Chapter 2: Acoustics
2
M. Karjalainen
Vibrating systems
3
M. Karjalainen
Vibrating systems
4
M. Karjalainen
Resonance
Mass-spring
resonator
Helmholtz-
resonator
5
M. Karjalainen
Two-mass vibrating system
6
M. Karjalainen
Vibration modes of a string
7
M. Karjalainen
Wave propagation
Wave equation:
D’Alembert:
8
M. Karjalainen
Sound pressure, sound pressure level, decibel
9
M. Karjalainen
Wave phenomena: spherical wave
Spherical wave:
10
M. Karjalainen
Wave phenomena: planar wave
11
M. Karjalainen
Lowest resonance modes in a tube
12
M. Karjalainen
Spectral content of string vibration
13
M. Karjalainen
Bar and membrane modes
Bar
Membrane
14
M. Karjalainen
Reflection and refraction (bending)
15
M. Karjalainen
Diffraction
16
M. Karjalainen
Sound propagation paths in a room
17
M. Karjalainen
Sound field decay in a room
Tapiola-sali
18
M. Karjalainen
Sound field in a room, Computer simulation
19
M. Karjalainen
Sound field level in a reverberant room
20
M. Karjalainen
Modal behavior in a room
21
M. Karjalainen
Sound propagation by image source model
22
M. Karjalainen
Electroacoustics: Loudspeaker
Dynamic loudspeaker
principle driver structure enclosure
23
M. Karjalainen
Electroacoustics: Microphone
Condenser microphone
principle construction
24
M. Karjalainen
Chapter 3: Sound and Voice as Signals
1
M. Karjalainen
Sound and Voice as Signals
– Random signal:
Continues ...
2
M. Karjalainen
Sound and Voice as Signals
3
M. Karjalainen
Linear and time-invariant (LTI) systems
Convolution
Fourier analysis
5
M. Karjalainen
Signal processing algorithms
Fourier synthesis
6
M. Karjalainen
Decomposition of sawtooth waveform
7
M. Karjalainen
Spectrum analysis
Magnitude spectrum
Phase spectrum
Phase delay
Group delay
8
M. Karjalainen
Fourier analysis with windowing
• Rectangular window
• Hamming window
• Hann(ing) window
• Kaiser window
• Blackman (Blackman-Harris) window
9
M. Karjalainen
Spectrum analysis using Fourier analysis with windowing
Sine wave
Sine wave
windowed
synchronously
Sine wave
windowed non-
synchronously
Sine wave,
Hamming-
windowed
10
M. Karjalainen
Vowel spectra
11
M. Karjalainen
Time-frequency representations: Spectrogram
Word: /kaksi/
12
M. Karjalainen
Auto- and cross-correlation
Cross-correlation
Autocorrelation
13
M. Karjalainen
Cepstrum
14
M. Karjalainen
Digital signal processing: DSP systems
15
M. Karjalainen
Signal quantization: A/D conversion
17
M. Karjalainen
Digital filtering: FIR filters
18
M. Karjalainen
Digital filtering: IIR filters
19
M. Karjalainen
Linear prediction (AR-modeling)
Signal
Windowed LP-spectra
FFT-spectrum
20
M. Karjalainen
Neural networks
21
M. Karjalainen
Hidden Markov models (HMM)
22
M. Karjalainen
Audio reproduction: loudspeaker response
23
M. Karjalainen
Group delay response of a loudspeaker
24
M. Karjalainen
Reproduction quality: Distortion and SNR
25
M. Karjalainen
Response equalization
26
M. Karjalainen
Chapter 4: Speech and Music
• Speech communication
• Speech production:
– Speech production mechanism
– Vocal cords – phonation
– Vocal and nasal tract – articulation
– Units and notation of speech: vowels, consonants
– Prosody of speech
– Modeling of speech production
• Singing voice
• Speech processing: analysis, synthesis, coding, recognition
• Musical instruments as sound sources
• Music signal processing
– Sound synthesis techniques
– Physical modeling
– Digital audio vs. music
1
M. Karjalainen
Speech communication chain
2
M. Karjalainen
Speech production mechanism
3
M. Karjalainen
Phonation and articulation
• Concepts:
– Glottis (vocal cord opening)
– Voiced / unvoiced / combined
– Constriction
– Formant (and antiformant)
– Vowel / consonant
– Prosodic features
4
M. Karjalainen
Units and notation of speech – Phonetics
6
M. Karjalainen
Consonants (Finnish)
7
M. Karjalainen
Prosody (suprasegmental features)
• Intonation (intonaatio)
– Primarily by fundamental frequency trajectory
• Stress (paino)
– Primarily by intensity (loudness) of pronounciation
• Timing (ajoitus)
– Rhythmic pattern (primarily by segment durations)
8
M. Karjalainen
Modeling of speech production
9
M. Karjalainen
Circuit model (transmission-line model)
• Glottal oscillator
– Varying cross-section between vocal cords
• Vocal tract as a transmission line
– Two-directional wave propagation
• Lip radiation (acoustic load)
• Source = excitation
– (a) voiced = quasiperiodic excitation
– (b) unvoiced = noislike excitation
• Filter = vocal and nasal tract
11
M. Karjalainen
Glottal oscillation
12
M. Karjalainen
Formants (tract resonances)
13
M. Karjalainen
Vocal tract transfer functions: vowel /i/
14
M. Karjalainen
Radiation directivity of speech
Azimuth Elevation
15
M. Karjalainen
Singing voice
16
M. Karjalainen
Speech processing
• Speech analysis
– Feature analysis of speech signals
• Speech synthesis
– Typically synthesis from text
• Speech recognition
– From speech to text or commands
• Speech coding
– Compression for transmission or storage
• Speech enhancement
– Improving degraded speech signals
17
M. Karjalainen
Formant synthesis models
18
M. Karjalainen
Synthesis by waveform concatenation
19
M. Karjalainen
Text-to-speech synthesis
20
M. Karjalainen
Text-to-speech synthesis
21
M. Karjalainen
Speech coding
22
M. Karjalainen
Speech recognition
23
M. Karjalainen
Musical instrument sounds
• String instruments
– Plucked string instruments
– Struck string instruments
– Bowed string instrument
• Wind instruments
– Brass instruments
– Woodwind instruments
• Percussion instruments
– Drums etc.
24
M. Karjalainen
Modeling of musical instruments (string modeling)
• String model
– Two-dimensional waveguide (transmission line)
– Excitation (pluck) inserted to both delay lines
– Wave reflections at terminations modeled as filters
– Output is taken at bridge or pickup, sum of both lines
– The same model is applicable to wind instrument bores
(but there is a nonlinear oscillating feedback in them)
25
M. Karjalainen
Simplified string modeling
26
M. Karjalainen
Impulse response of a simple string model
27
M. Karjalainen
Body response modeling
Impulse
response
Magnitude
response
(low frequencies)
28
M. Karjalainen
Chapter 5: Structure and Function of Hearing
• Peripheral hearing
– External ear
– Middle ear
– Inner ear (cochlea)
• Basilar membrane
• Hair cells
• Auditory nerve
• Active cochlea and nonlinearities
• Higher levels of the auditory system
• Basic properties of human hearing
– Effective hearing area (level vs. frequency)
– Equal loudness curves
– Technical measures related to hearing
• Sound level and frequency weighting functions
1
M. Karjalainen
Approaches to hearing research
• Anatomy of hearing
– The structure of hearing organs is studied
• Physiology of hearing
– The (physiological) responses of hearing to physical
sound stimuli are studied
• Psychology of hearing
– Functional properties of auditory perception are studied
as subjects reactions to physical sound stimuli
2
M. Karjalainen
Peripheral hearing
3
M. Karjalainen
Schematic of peripheral hearing
4
M. Karjalainen
External ear and ear canal transmission
• Transfer functions
– Frontal sound source to the eardrum (solid line)
– Entrance of ear canal to the eardrum (dotted line)
• Ossicles
– Malleus (hammer-shaped bone)
– Incus (anvil-shaped bone)
– Stapes (stirrup-shaped bone)
6
M. Karjalainen
Animations of middle ear function
7
M. Karjalainen
Animations: University of Wisconsin http://www.neurophys.wisc.edu/~ychen/auditory/fs-auditory.html
Middle ear conduction and features
Cochlea linearized
9
M. Karjalainen
Cross-section of the cochlea
10
M. Karjalainen
Basilar membrane motion: traveling waves
11
M. Karjalainen
Animation of basilar membrane motion
12
M. Karjalainen
Basilar membrane response to a square-wave signal
13
M. Karjalainen
Hair cells
14
M. Karjalainen
Hair cells in the organ of Corti
15
M. Karjalainen
Stereocilia (= ’hair bundles’ of hair cells)
16
M. Karjalainen
Movement of the organ of Corti
17
M. Karjalainen
Movement and activation of hair cells
18
M. Karjalainen
Hair cells: neural conduction
19
M. Karjalainen
Activation and inhibition of hair cells
20
M. Karjalainen
Phase-locking and synchrony of neural firing
21
M. Karjalainen
Passive vs. active cochlea
23
M. Karjalainen
Poststimulus time histogram (PST)
24
M. Karjalainen
PST with steady-state sinusoidal excitation
25
M. Karjalainen
Firing rate saturation for a vowel excitation
26
M. Karjalainen
Tuning curves for constant firing level
27
M. Karjalainen
Effects of active cochlea
28
M. Karjalainen
Cochlear nonlinearity: Two-tone suppression
29
M. Karjalainen
Cochlear nonlinearity: Combination tones
30
M. Karjalainen
Central auditory system
• Higher-level functions
not known well.
• Cochlear nucleus has
specific cells such as
’chopper cells’ that do
temporal processing.
Spectral information is
recovered unsaturated.
• Binaural hearing starts
at superior olive level.
• Auditory cortex is the
center for processing
perceptions and
integrating the sound
scene.
• Interaction with other
senses (vision) strong.
31
M. Karjalainen
Dynamic range of hearing
Sound
level
’thermo-
meter’
6 dB steps
3 dB steps
1 dB steps
32
M. Karjalainen
Equal loudness curves and threshold of hearing
33
M. Karjalainen
Sound level and frequency weighting curves
34
M. Karjalainen
Recommended frequences and bands
• Recommended
frequences and
frequency bands
for measurements
and technical
applications:
• Octave = 2:1
• 1/2 octave
• 1/3 octave
35
M. Karjalainen
Filtered noise demo
• White noise
36
M. Karjalainen
Chapter 6: Fundamentals of Psychoacoustics
1
M. Karjalainen
Psychophysical experimentation
2
M. Karjalainen
Sound events: Stimulus signals
• Elementary sounds
– Sinusoidal tones
– Amplitude- and frequency-modulated tones
– Sinusoidal bursts
– Sine-wave sweeps, chirps, and warble tones
– Single impulses and pulses, pulse trains
– Noise (white, pink, uniform masking noise)
– Modulated noise, noise bursts
– Tone combinations (consisting of partials)
• Complex sounds
– Combination tones, noise, and pulses
– Speech sounds (natural, synthetic)
– Musical sounds (natural, synthetic)
– Reverberant sounds
– Environmental sounds (nature, man-made noise)
3
M. Karjalainen
Sound generation and experiment environment
• Reproduction techniques
– Natural acoustic sounds (repeatability
problems)
– Loudspeaker reproduction
– Headphone reproduction
• Reproduction environment
– Not critical in headphone reproduction
– Anechoic chamber (free field)
• Room effects minimized
• Not a natural environment
– Listening room
• Carefully designed, relatively normal
acoustics
– Reverberation chamber
• Special experiments with diffuse
sound field
4
M. Karjalainen
Psychophysical functions
5
M. Karjalainen
Experimental concepts: Thresholds
• Threshold values
– Absolute thresholds (e.g., threshold of hearing)
– Difference thresholds (just noticeable difference, JND)
6
M. Karjalainen
Experimental concepts
• Comparison of percepts
– Magnitude estimation
– Magnitude production
• Probe tone method
– Generation of a probe tone to make test tone
audible/noticeable
– Modulation, canceling, interference
• Classification and scaling of percepts
– Nominal scale (rough, sharp, reverberant, …)
– Ordinal scale (percepts have ordering)
– Interval scale (numeric scale, no zero point defined)
– Ratio scale (numeric scale, zero point defined)
• Multidimensional scaling
– Semantic differentials: low – high, dull – sharp, ...
7
M. Karjalainen
Psychoacoustic experiments
8
M. Karjalainen
Békésy audiometry
9
M. Karjalainen
Typical psychoacoustical test types
• AB test
– Set in preference order / select one
– AB hidden reference (one must be recognized)
• AB scale test
– As AB but assign numeric values for A and B
• ABC test
– A is fixed reference (anchor point) for assigning
values for B and C
• ABX test
– Which one, A or B, is equal to X ?
• TAFC (2AFC)
– Two alternative forced choice
11
M. Karjalainen
Frequency masking
12
M. Karjalainen
Frequency masking
13
M. Karjalainen
Frequency masking
14
M. Karjalainen
Frequency masking
15
M. Karjalainen
Frequency masking
16
M. Karjalainen
Frequency masking
17
M. Karjalainen
Temporal masking
18
M. Karjalainen
Temporal masking
• Beginning of postmasking
19
M. Karjalainen
Temporal masking
20
M. Karjalainen
Time-frequency masking
21
M. Karjalainen
Temporal masking
22
M. Karjalainen
Frequency selectivity of hearing
23
M. Karjalainen
Critical band experiment
24
M. Karjalainen
Critical band
25
M. Karjalainen
Critical band (Bark band) vs. frequency
26
M. Karjalainen
Critical band: 24 Bark bands (Zwicker)
27
M. Karjalainen
ERB band experiment
28
M. Karjalainen
Pitch scales
• Bark scale
or
Inverse function:
• ERB scale
Inverse :
29
M. Karjalainen
Logarithmic pitch scale
30
M. Karjalainen
Comparison of pitch scales
31
M. Karjalainen
Comparison of pitch scales
32
M. Karjalainen
Comparison of pitch scales
33
M. Karjalainen
Pitch
34
M. Karjalainen
Pitch phenomena
Cont’d from file 6a
1
M. Karjalainen
JND of frequency modulation
2
M. Karjalainen
Minumum duration of a tone for pitch percept
3
M. Karjalainen
JND pitch change vs. tone duration
4
M. Karjalainen
Pitch strength
5
M. Karjalainen
Pitch phenomena and theories
6
M. Karjalainen
Loudness
7
M. Karjalainen
Loudness of a sinusoidal tone
Loudness vs.
loudness level :
Power law:
More precisely:
8
M. Karjalainen
Partial loudness (by noise masking)
9
M. Karjalainen
Loudness example: two tones
10
M. Karjalainen
Loudness computation (Zwicker formulation)
• Loudness density
• Total loudness
11
M. Karjalainen
Loudness computation, examples
12
M. Karjalainen
Loudness graphically
13
M. Karjalainen
JND of loudness level
14
M. Karjalainen
JND of loudness level
15
M. Karjalainen
Modulation detection
16
M. Karjalainen
Loudness vs. duration
17
M. Karjalainen
Loudness formation temporally
18
M. Karjalainen
Timbre (perceived ’sound color’)
19
M. Karjalainen
Subjective duration
20
M. Karjalainen
Auditory Demonstrations 1
1 Cancelled harmonics
2-6 Critical bands by masking
7 C.B. by loudness comparison
8-11 The decibel scale
12-16 Filtered noise
17-18 Frequency response of the ear
19-20 Loudness scaling
21 Temporal integration
22 Asymmetry of masking by pulsed tones
23-25 Backward and forward masking
26 Pulsation threshold
21
M. Karjalainen
Auditory Demonstrations 2
23
M. Karjalainen
Chapter 7: Other psychoacoustic concepts
• Sharpness
– Spectral center of gravity
• Fluctuation strength
– Perception of slow modulations (beats)
• Impulsiveness
• Roughness
– Perception of fast modulations
• Tonality
– Periodic vs. random excitation
• Sensory pleasantness
• Psychoacoustic concepts and music
– Sensory consonance and dissonance
– Intervals, scales, and tunings
– Rhythm, tempo, bar, measure
• Perceptual organization of sound
1
M. Karjalainen
Sharpness
Bandpass noises:
2
M. Karjalainen
Computation of sharpness
3
M. Karjalainen
Fluctuation strength
1 Hz
4 Hz
16 Hz
4
M. Karjalainen
Fluctuation strength
5
M. Karjalainen
Fluctuation strength
6
M. Karjalainen
Impulsiveness
7
M. Karjalainen
Roughness
1 kHz+f
7 Hz
70 Hz
300 Hz
8
M. Karjalainen
Roughness
9
M. Karjalainen
Roughness
1 kHz+f
7 Hz
70 Hz
300 Hz
10
M. Karjalainen
Tonality
11
M. Karjalainen
Sensory pleasantness
12
M. Karjalainen
Sensory consonance and dissonance
13
M. Karjalainen
Consonance and dissonance of harmonic tones
14
M. Karjalainen
Examples of intervals
– 16/15 (dissonant)
– 40/27 (dissonant)
15
M. Karjalainen
Examples of intervals
Circularity of pitch
• Shepard effect
16
M. Karjalainen
Intervals, scales, tuning
18
M. Karjalainen
Temporal structures in music: Rhythm, tempo
19
M. Karjalainen
Perception of magnitude and phase spectrum
• Magnitude
– 1 dB deviation per critical band noticeable in direct comparison.
Even smaller deviations can be noticed by trained ”golden ears”
– Even ± 3...5 dB deviations are not easy to ”perceive” when there is
no immediate reference (except for well trained listeners)
– Magnitude response deviations = spectral coloration
• Phase and time differences
– The auditory system is relatively insensitive to phase (Helmholtz)
in general: magnitude spectrum more important than phase
spectrum, but sometimes phase is important
– Phase functions from Fourier analysis are circular and difficult to
analyze and interpret
– Group delay (phase derivative) is a relatively good perceptual
measure which describes the delay of modulation (not the carrier)
20
M. Karjalainen
Perception of phase: extreme cases
21
M. Karjalainen
Perceptual organization of sound
D D D
B B B
F F F
C C C
E E E
A A A
(a) (b)
Time Time
One stream Two streams
22
M. Karjalainen
Perceptual organization of sound
D
B
F
D
B
C F
C
E E
A A
Time Time
23
M. Karjalainen
Perceptual organization of sound
increasing segregation
tempo or of multiple
frequency streams
difference
time
TIMBRE/TEXTURE
24
M. Karjalainen
Perceptual organization of sound
20 10 5 3
20
15
always
separated
10
separated
or coherent
5
always
coherenti
0
0 50 100 150 200 250 300 400 500
Repetition period (msec)
25
M. Karjalainen
Auditory scene analysis
26
M. Karjalainen
Computational auditory scene analysis (CASA)
27
M. Karjalainen
Tilakuuleminen
Ville Pulkki
Akustiikan ja äänenkäsittelytekniikan laboratorio
Teknillinen korkeakoulu
Espoo, Suomi
http://www.acoustics.hut.fi/
Ville Pulkki@hut.fi
TKK, Akustiikan ja äänenkäsittelytekniikan laboratorio 26.3.2002
Ääni tilassa
Tilakuuleminen
Suuntakuulo
• Suuntakuulon tarkkuus
• Suuntakuulon teoria
Etäisyyskuulo
Tilan havaitseminen
Tilaän̈entoisto
HRTF:ien mittaaminen
Algazi
c et al.: http://interface.cipic.ucdavis.edu/
M.
c Karjalainen
Algazi
c et al.: http://interface.cipic.ucdavis.edu/
Algazi
c et al.: http://interface.cipic.ucdavis.edu/
0dB 0 0
0dB 0 0
M.
c Karjalainen
90°
80,7°
±9,2°
ϕ
179,3° 359°
180° 0°
±5,5° ±3,6°
281,6°
±10° Kuulotapahtuman suunta
Äänitapahtuman suunta
270°
M.
c Karjalainen
Äänitapahtuman
suunta
δ = 90ο
Kuulotapah-
tuman suunta
δ = 0ο 0ο δ = 0ο ο
ϕ = 0ο ±9ο ϕ = 180
M.
c Karjalainen
Lateralisaatiokokeet
a) b)
M.
c Karjalainen
Lateralisaatiokokeet, aikaviive
6
vasen aiemmin vasen myöh.
havaittu lateraalisijainti
4
oikea
2
0
vasen
2
6
-15000 -1000 -500 0 500 1000 15000
korvien välinen vaiheviive τph / μs
M.
c Karjalainen
Lateralisaatiokokeet, ominaisuuksia
Hyvät puolet:
• Voidaan vapaasti tuottaa mikä tahansa ITD-ILD yhdistelmä
• Perustulokset
Ongelmat:
• Epäluonnollisuus
• Pään sisälle lokalisointi
• Korkeiden taajuuksien toisto erilainen eri kuuntelukerroilla
Suuntakuulo
Vihjeet:
• Binauraaliset vihjeet
– Korvienvälinen aikaero
– Korvienvälinen voimakkuusero
• Monauraalinen spektri
• Pään kääntelyn vaikutus binauraalisiin vihjeisiin
• Heijastusten suppressio
Binauraaliset vihjeet
ITD:n taajuusriippuvuus
matalat taajuudet ~200 − ~1600 Hz korkeat taajuudet > ~1600 Hz
kantoaallon aikaviive verhokayran aikaviive
vasen
oikea
ITD ITD
ITD:n mallinnus
τ τ
τ τ
τ τ
τ τ
τ τ
oikeasta
vasemmasta korvasta korvasta
M.
c Karjalainen
ITD:n mallinnus
GTFB
IACC
ITD
spectrum
IACC
GTFB Composite
IACC
IACC
Ristikorrelaatio ERB-kanavilla
Band cross correlation functions
°
60° 40°
90
°
20 °
0 °
20 °
40 60°
90°
1
21 kHz
0.8
0.6 10 kHz
5
0.4
10
0.2
3 kHz
0 15
1.5 kHz
1 20
0.5 800 Hz
0 25
−0.5 200 Hz
−1 30
ITD:n taajuusriippuvuus
−3
x 10
1
0.5
ITD [ms]
−0.5
−1
18.2 90
12.4
8.5 60
5.7 30
3.9
2.6 0
1.7 −30
1.1
0.7 −60
0.4
0.2 −90
Direction [degree]
Frequency [kHz]
Binauraaliset vihjeet
dB dB
ILD:n mallinnus
GFTB LL CLL
Composite
LL loudness
CLL
level
spectrum
LL CLL
GFTB
LL ILD
ILD ILD
LL
spectrum
LL ILD
ILD:n taajuusriippuvuus
60
40
20
ILD [phon]
−20
−40
−60
18.2
12.4 90
8.5
5.7 60
3.9 30
2.6
1.7 0
1.1 −30
0.7
0.4 −60
0.2 −90
Direction [degree]
Frequency [kHz]
Sekaannuskartio
θcc
ääni-
lähde
φ
cc
sekaannuskartio
- karkea vihje
Kehon vaikutus
Korvalehden vaikutus
10 10
0 0
−10 −10
−20 −20
−30 −30
90 90
60 60
30 30
15 15
0 0
−15 −15
−30 3.9 5.7 8.5 12.4 18.2 −30 3.9 5.7 8.5 12.4 1
v [degr] 0.4 0.7 1.1 1.7 2.6 Elev [degr] 0.4 0.7 1.1 1.7 2.6
0.2 0.2
Frequency [kHz] Frequency [kHz]
1 2
Auditorinen spektri mediaanitasossa, suunnasta riippumaton osuus keski
voistettu pois.
10 10
0 0
−10 −10
−20 −20
−30 −30
90 90
60 60
30 30
15 15
0 0
−15 −15
−30 3.9 5.7 8.5 12.4 18.2 −30 3.9 5.7 8.5 12.4 1
v [degr] 0.4 0.7 1.1 1.7 2.6 Elev [degr] 0.4 0.7 1.1 1.7 2.6
0.2 0.2
Frequency [kHz] Frequency [kHz]
3 4
Auditorinen spektri mediaanitasossa, suunnasta riippumaton osuus keski
voistettu pois.
Vihjeiden luotettavuus
Jos vihjeet ovat ristiriitaisia:
• Signaalin spektri < ˜ 1000 Hz
– ITD yleensä vahvin
– ILD heikko, trading?
• Korkeammat taajuudet
– ITD ja ILD kumpikin vahvoja
– ILD voimakkaampi joskus
• Johdonmukaisempi vihje voittaa [Wightman]
• Voi syntyä useita havaintoja suunnasta
• Äänilähteen koko
• Individuaalisuus
Suuntakuulon fysiologia
Kalat
c 1998
Presedenssiefekti
Presedenssiefekti
ensimm. kuulotapahtuma
So ϕ
= 40o
ϕ
kaikukynnys
α=80o ϕ = 0o
kaiku
ϕ =-40o
ST 0 1 2ms 20 30 40 50ms
ST:n viive τph
Kaikujen havaitsemiskynnykset
40 ensimmäinen äänitapahtuma
dB ei enää erotettavissa
(ensiääni estetty) (≥ 6 henkeä)
tasoero LST - LSO
20
-40 peittokynnys
(1-2 henkilöä)
0 20 40 60 80 100 ms
ST:n viive
kuulotapahtuman etäisyys / m 6
4
viiden henk.
keskiarvo
2
0
0 2 4 6 8 10
äänilähteen etäisyys / m
M.
c Karjalainen
Etäisyyden havaitseminen
Vihjeet
• Äänekkyys
• Binauraaliset vihjeet
• Suoran äänen suhde kaiuntakenttään
• Spektri
Tilaäänen toistometodit
• Perinteinen toisto
– Monofonia
– Stereofonia
– Monikanava 2-D
– Monikanava 3-D
• Binauraalinen toisto
– Kuulokkeet
– Kaiuttimet, ristiinkuulumisen esto
Monofoninen toisto
Stereofoninen toisto
“Surround” toisto
3-D monikanavatoisto
Binauraalinen toisto
xm yˆ l yˆ r
Hc Hc
Hl Hr Hi Hi
yl yr yl yr
M.
c Karjalainen
Binauraalinen toisto
(a) (c)
Hl + Hr yˆl
Hl yl xl
xm Hi + Hc
mono binauraalinen stereo transau-
raalinen
Hl − Hr
Hr yr xr yˆr
− Hi − Hc −
(b) (d)
yl 1
yˆl yˆl Hi + Hc yl
Hi + Hc
binau- transau- transau- binau-
raalinen raalinen raalinen raalinen
1
yr yˆr yˆr Hi − Hc yr
− Hi − Hc − − −
M.
c Karjalainen
1
M. Karjalainen
Simple psychoacoustic modeling
2
M. Karjalainen
Auditory spectrum through FFT
3
M. Karjalainen
Examples of psychoacoustic spectra
• Auditory spectra
– Sinewave (400 Hz)
– White noise
4
M. Karjalainen
Examples of psychoacoustic spectra
5
M. Karjalainen
Mel frequency cepstral coefficients
• MFCC computation
– FFT, mel warping, logarithm, inverse cosine transform
6
M. Karjalainen
Filterbank auditory models
7
M. Karjalainen
Response of a filterbank model (Bark-bank)
8
M. Karjalainen
Gammatone filterbank
9
M. Karjalainen
Neural adaptation
10
M. Karjalainen
Temporal processing
11
M. Karjalainen
Responses
12
M. Karjalainen
Basilar membrane traveling wave model
13
M. Karjalainen
Meddis hair cell model
14
M. Karjalainen
Periodicity analysis (Meddis)
15
M. Karjalainen
Periodicity analysis example
16
M. Karjalainen
Auditory spectrum vs. auditory formant spectrum
17
M. Karjalainen
Auditory representation of speech
18
M. Karjalainen
Applications of auditory modeling
• Audio coding
– Psychoacoustic or perceptual models of masking
• Sound quality modeling
– Modeling of perceived differences
– Criteria for audio reproduction
– Binaural audio quality
• Speech recognition
– Advanced front-end models
• Advanced hearing aids
– Cochlear implants
19
M. Karjalainen
Chapter 10: Sound quality
• Effects of sound:
– Physical effects (generally meaningless)
– Physiological effects (hearing loss)
– Information and knowledge effects (communication)
– Esthetic and emotional effects (communication)
• Concept of quality in general:
– Quality as contrast to quantity (categorical
dissimilarity)
– Quality on scale low-Q vs. high-Q (measure of
preference)
• Speech intelligibility and quality
• Sound quality of concert halls and auditoria
• Sound quality in audio reproduction
• Noise quality
• Product sound quality
1
M. Karjalainen
Evaluation and measurement of sound quality
Computational models
4
M. Karjalainen
Speech quality: subjective measures and methods
5
M. Karjalainen
Speech quality: objective measures and methods
6
M. Karjalainen
MOS (mean opinion score)
8
M. Karjalainen
Modulation transfer function (2)
9
M. Karjalainen
Modulation transfer function (3)
10
M. Karjalainen
Modulation transfer function (4): STI
12
M. Karjalainen
STI vs. speech intelligibility
13
M. Karjalainen
RASTI vs. STI
14
M. Karjalainen
Percentage articulation loss of consonants (%ALcons)
• where
– r = distance of source and listener
– RT = reverberation time
– V = room volume
– Q = directivity of a sound source
– k = constant (for individual listener) = 1.5 ... 12.5 %
• %Alcons can also be estimated from room
measurements
• %Alcons up to 25...30% can be tolerated in
meaningful speech due to information
redundancy
15
M. Karjalainen
Sound quality in concert halls (and performing spaces)
• Intimacy or presence
• Reverberation (subjective)
• Spaciousness (apparent source width, listener envelopment)
• Clarity (separation of sounds and sources)
• Warmth (level and reverberation at low frequencies)
• Loudness
• Acoustic glare (walls should not reflect like mirrors)
• Brilliance (due to long reverberation at high frequences)
• Balance (how sound sources (instruments) are balanced)
• Blend (how instruments are mixed harmonically)
• Ensemble (how musicians can play together)
• Immediacy of response (from the hall back to musicians)
• Texture (how early reflections arrive to listeners)
• Freedom from echo (discrete echoes are highly undesirable)
• Dynamic range (useful range of playing levels)
• Extraneous effects on tonal quality (no extra sounds desired)
• Uniformity of sound (quality should be equal in all positions)
17
M. Karjalainen
Sound quality in concert halls: (2) objective measures
• Loudness
– Gmid (sound level at mid frequencies)
• Reverberation time
– RT60 (decay time of 60 dB for full hall)
– EDT (early decay time, 0–10 dB scaled to correspond to 60 dB)
• Clarity
– Early vs. late energy ratio C80 (empty hall)
• Spaciousness
– IACCearly (interaural cross-correlation, early)
– LFearly (lateral energy fraction, early)
• Envelopment
– IACClate and visual inspection of surface irregularity
• Intimacy
– ITDG (initial time delay)
• Warmth
– BR (base ratio, full hall)
• Stage support
– Early energy (20-100 ms), sound source on the stage 1m from the
microphone
18
M. Karjalainen
Objective sound quality in concert halls: definitions
• Base ratio
• Stage support
19
M. Karjalainen
Early vs. late ratios
Clearness
Centertime
20
M. Karjalainen
Audio sound quality
21
M. Karjalainen
Perception of audio reproduction
22
M. Karjalainen
Nonlinear distortion
• Nonlinear distortion
– In a nonlinear system a sine wave generates harmonics:
23
M. Karjalainen
Audio distortion mechanisms
• Schroeder
et al.:
• Karjalainen:
25
M. Karjalainen
PAQM (perceptual audio quality measure)
26
M. Karjalainen
Product sound quality
27
M. Karjalainen
Chapter 11: Technical audiology
1
M. Karjalainen
Hearing degradation I
2
M. Karjalainen
Hearing degradation II
• Medical classification of hearing impairments
– Conductive hearing loss (äänen johtumisvika)
• External and middle ear problems
• Attenuation of loudness
– Sensorineural hearing loss
• Inner ear and retrocochlear problems
• Attenuation or recruitment
• Tinnitus
– Central hearing loss
• Higher neural levels
• Problems in sound separation or speech analysis
• Problems in localization (spatial separation)
• Tinnitus
– Psychic hearing problems
• No clear physiological reason
3
M. Karjalainen
Hearing threshold change
4
M. Karjalainen
Audiometry
6
M. Karjalainen
Degrees of hearing impairment
7
M. Karjalainen
Other hearing impairment problems
8
M. Karjalainen
Ear drum impedance measurement
9
M. Karjalainen
Noise and causes of hearing loss
• Noise measurement
– A-weighted equivalent level
12
M. Karjalainen
Hearing protectors
13
M. Karjalainen
Hearing aid types
14
M. Karjalainen
Hearing aid response
15
M. Karjalainen
Hearing aid gain control
16
M. Karjalainen
Hearing aid AGC control
17
M. Karjalainen
Hearing aid output waveforms
18
M. Karjalainen
Other issues in hearing aids
• Directional microphones
• Binaural processing
• Noise cancellation
• Wind noise cancellation
• Feedback cancellation
• Speech enhancement
19
M. Karjalainen
Cochlear implants
20
M. Karjalainen
Cochlear implants II
21
M. Karjalainen