Recent CCRMA Research in Digital Audio Synthesis, Processing, and

Effects
Julius O. Smith III
CCRMA, Stanford University
DAFx-09, Keynote III, Como, Italy
September 4, 2009

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 1 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Overview

DAFx-09, Keynote III, Como, Italy 2 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators

In my DAFx06 review, we looked at

Early digital audio effects (delay lines, scanner vibrato, . . . )

Acoustic propagation models
Digital waveguide models (voice, strings, woodwinds, . . . )
Commuted synthesis (acoustic guitar, harpsichord, piano, . . . )
All my best accumulated sound examples!
That presentation is available online:

Spectral Delay Filters

http://ccrma.stanford.edu/~jos/pdf/DAFx06KeynoteII.pdf

Audio FFT Filter Banks

(sound examples via HTTP)

What can I talk about now?
New research results in the past three years?

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 3 / 54

Since Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments

Problem:
I havent done that much in the past three years!
Solution:
Summarize recent DAFx-related research at CCRMA
as a whole

New Oscillators
Spectral Delay Filters

Talk Design:

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

New results in the past year (three years is too much)

Developed at CCRMA
JOS involved as collaborator or adviser
(i.e., I know something about it!)

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 4 / 54

Since Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments

Problem:
I havent done that much in the past three years!
Solution:
Summarize recent DAFx-related research at CCRMA
as a whole

New Oscillators
Spectral Delay Filters

Talk Design:

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

New results in the past year (three years is too much)

Developed at CCRMA
JOS involved as collaborator or adviser
(i.e., I know something about it!)

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 4 / 54

Since Three Years Ago

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments

Problem:
I havent done that much in the past three years!
Solution:
Summarize recent DAFx-related research at CCRMA
as a whole

New Oscillators
Spectral Delay Filters

Talk Design:

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

New results in the past year (three years is too much)

Developed at CCRMA
JOS involved as collaborator or adviser
(i.e., I know something about it!)

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 4 / 54

DAFx-Related Research involving JOS at CCRMA, 2008-2009

Overview

Previously at DAFx06
Research Update
CCRMA
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

CCRMA building: The Knoll, Stanford University

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 5 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Multimodal Spring Reverb

Modeling

DAFx-09, Keynote III, Como, Italy 6 / 54

Accutronics Type 8 Spring Tank

Overview
Spring Reverb Models

Spring Tank
Impulse Response
Model
Sound Examples
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Top: Accutronics Type 8 Spring Tank

Bot: Single Spring Folded into a Compact Space
Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 7 / 54

Measured Single-Spring Impulse Response

Overview

spring element impulse response onset

Spring Reverb Models

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

response amplitude

Spring Tank
Impulse Response
Model
Sound Examples

1
0.5
0
0.5
1

New Oscillators

Spectral Delay Filters

Summary

200

250
level dB
0

10

20

30

40

50

Julius O. Smith III

150
time milliseconds

spring element impulse response onset spectrogram

frequency kHz

Faust to Flash Plugins

100

Audio FFT Filter Banks

Microphone Array

50

50

100
150
time milliseconds

200

60

DAFx-09, Keynote III, Como, Italy 8 / 54

Model Impulse Response

Overview

modeled spring element impulse response onset

Spring Reverb Models

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

response amplitude

Spring Tank
Impulse Response
Model
Sound Examples

1
0.5
0
0.5
1

New Oscillators

Spectral Delay Filters

Summary

400

500
level dB
0

10

20

30

40

50

Julius O. Smith III

300
time milliseconds

modeled spring element impulse response onset spectrogram

frequency kHz

Faust to Flash Plugins

200

Audio FFT Filter Banks

Microphone Array

100

50

100

150

200
250
300
time milliseconds

350

400

450

60

DAFx-09, Keynote III, Como, Italy 9 / 54

Model Impulse Response in Time and Frequency Domains

Spring Reverb Sound Examples

Overview
Spring Reverb Models

Spring Tank
Impulse Response
Model
Sound Examples

Dry Source Signal: (WAV) (MP3)

Measured Spring-Reverb Response: (WAV) (MP3)
Spring-Reverb Model Response: (WAV) (MP3)

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Submitted Paper: A Spring Reverb Model Employing Coupled

Torsional and Longitudinal Modes

New Oscillators
Spectral Delay Filters

Jonathan Abel, Dave Berners, Kyle Spratt, and Julius Smith

Audio FFT Filter Banks

Microphone Array

(in review)

Faust to Flash Plugins

Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 10 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Virtual Analog Circuits

DAFx-09, Keynote III, Como, Italy 11 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Digitizing Circuits in Real Time

Recent CCRMA/EE thesis by David Yeh:
Overview
Spring Reverb Models

Digital Implementation of Musical Distortion Circuits by

Analysis and Simulation June 2009

Virtual Analog Circuits

Real Time Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Summary

Julius O. Smith III

Analog Audio Circuits Real-Time Digital Audio Effects

Includes work of four past DAFx papers
Linear and nonlinear methods for digitizing circuits
Nonlinear methods similar to SPICE (implicit) but modified for
real time circuit-solving (semi-implicit) in discrete time
One method extends the K Method to
Nonlinear circuits, with automated application to netlists
Discrete-time nonlinear state-space formulation
(resolves issues with certain circuit types)
Instantaneous nonlinearities are precomputed as in K Method
Can be applied to transistor and vacuum-tube circuits
Accuracy limited primarily by the underlying device models
DAFx-09, Keynote III, Como, Italy 12 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Virtual Acoustic Guitar Models

DAFx-09, Keynote III, Como, Italy 13 / 54

Coupled Strings Analysis and Synthesis

Submitted paper (from pending CCRMA/CS thesis) by Nelson Lee:
Overview
Spring Reverb Models
Virtual Analog Circuits

Analysis and Synthesis of Coupled Vibrating Strings

Using a Hybrid Modal-Waveguide Synthesis Model

Acoustic Guitar Models

Coupled Strings
Sound Examples
Sound Examples

aki
(in review)
Nelson Lee, Julius Smith, and Vesa Valim

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Bank formulation, but replacing low-frequency

Similar to Balazs
partials by fourth-order resonators (instead of adding
second-order resonators to existing partials)

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 14 / 54

Virtual Acoustic Guitar Sound Examples of Individual Effects

From Nelson Lees thesis defense:
Overview
Spring Reverb Models
Virtual Analog Circuits

Original waveform: (WAV) (MP3)

Simple lossless, reflectively terminated digital waveguide (DWG):

Acoustic Guitar Models

Coupled Strings
Sound Examples
Sound Examples
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array

(WAV) (MP3)
Add loop filter: (WAV) (MP3)
Add interpolation filter: (WAV) (MP3)
Add excitation (ICMC07): (WAV) (MP3)
Add body response: (WAV) (MP3)
Add hybrid modal/waveguide model: (WAV) (MP3)
Exaggerate pitch glide due to tension modulation: (WAV) (MP3)

Faust to Flash Plugins

Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 15 / 54

Virtual Acoustic Guitar Sound Examples

More Nelson Lee examples:
Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models

Coupled Strings
Sound Examples
Sound Examples
Haptic Instruments
New Oscillators

Original 1: (WAV) (MP3)

Synthesized 1: (WAV) (MP3)
Original 2: (WAV) (MP3)
Synthesized 2: (WAV) (MP3)
Original 3: (WAV) (MP3)
Synthesized 3: (WAV) (MP3)

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Original 4: (WAV) (MP3)

Synthesized 4: (WAV) (MP3)
Original 5: (WAV) (MP3)
Synthesized 5: (WAV) (MP3)
Original 6: (WAV) (MP3)
Synthesized 6: (WAV) (MP3)
Synthesized Chord Demo: (WAV) (MP3)

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 16 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Haptic Feedback Control for

Virtual Instruments

DAFx-09, Keynote III, Como, Italy 17 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Haptic Virtual Musical Instruments

Overview

CCRMA/EE PhD student Ed Berdahl is working on

Spring Reverb Models

Haptic Feedback Control for Virtual Instruments

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Dissertation expected by the end of the year

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Goals:

Assist and/or augment gestures

Assist with accurate playing
Recent projects:
Haptically plucked virtual string
Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/HapticDrum/

DAFx-09, Keynote III, Como, Italy 18 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

New Digital Oscillator Algorithms

DAFx-09, Keynote III, Como, Italy 19 / 54

Sawtooth Waveforms via Differentiated Polynomials

Overview
Spring Reverb Models

Given

f (x) = xn + an1 xn1 + ... + a1 x + a0

Virtual Analog Circuits

Acoustic Guitar Models

then differentiating n 1 times gives

Haptic Instruments
New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example

f (n) (x) = n! x + (n 1)! an1 .

This first-order polynomial (a line segment) generates a sawtooth
waveform as x periodically traverses [1, 1].

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 20 / 54

Differentiated Polynomial Wave (DPW) Sawtooth Synthesis

1.

Digitally synthesize

Overview



 
nT
mod 1 1 ,
x(n) = f 2
P

Spring Reverb Models

Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments

where

New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters
Audio FFT Filter Banks

f (x) = xn + an1 xn1 + ... + a1 x + a0

T

= sampling interval (sec)

= desired period (sec)

n = sample number (integer)

2.

Apply n 1 first-order finite differences

xk+1 (n) = [xk (n) xk (n 1)]/(2T /P ) to get

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Xn1 (z) =

z 1

1
2T /P

n1

X(z)

DAFx-09, Keynote III, Como, Italy 21 / 54

Minimizing Aliasing
Overview

Initial waveform is given by sampling

Spring Reverb Models

f (x) = xn + an1 xn1 + ... + a1 x + a0 ,

Virtual Analog Circuits

x [1, 1).

Acoustic Guitar Models

Haptic Instruments

After n 1 derivatives, we get

New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

f (n) (x) = n! x + (n 1)! an1 .

For zero mean, set an1 = 0
The n 1 remaining degrees of freedom in f (x) can be used to

maximize flatness at the transition from x = 1 to x = 1

This smoothness minimizes aliasing in the synthesized sawtooth
We can set a0 = 0 because it has no effect on smoothness
This leaves n 2 coefficients to optimize

Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 22 / 54

First Several Maximum-Smoothness Examples

Overview

Results for polynomial orders up to n = 6:

Spring Reverb Models

Virtual Analog Circuits

f2 (x) = x2

Acoustic Guitar Models

Haptic Instruments
New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters
Audio FFT Filter Banks

f3 (x) = x3 x
f4 (x) = x4 2x2
10 3 7
f5 (x) = x x +
3
3
5

f6 (x) = x6 5x4 + 7x2

Submitted Paper: Alias-Suppressed Oscillators based on
Differentiated Polynomial Waveforms

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

aki,
Juhan Nam, Julius Smith, and Jonathan Abel
Vesa Valim
IEEE Transactions on Acoustics, Speech, and Language Processing
March 2010 (accepted for publication)
DAFx-09, Keynote III, Como, Italy 23 / 54

Comparison of First Four Polynomial Orders

Overview

Magnitude Spectra - windowed(200dB) saw case

Spring Reverb Models

50

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters

Magnitude (dB)

New Oscillators

-50

-100
saw1
saw2
saw3
saw4

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

-150
0

0.1

0.2
0.3
0.4
Normalized Frequency (cycles/sample)

0.5

DAFx-09, Keynote III, Como, Italy 24 / 54

Same Comparison over Log Frequency

Overview

Magnitude Spectra

Spring Reverb Models

50

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Spectral Delay Filters

Magnitude (dB)

New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example

saw1
saw2
saw3
saw4

-50

-100

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

-150
1e-05

0.0001
0.001
0.01
0.1
Normalized Frequency (cycles/sample)

DAFx-09, Keynote III, Como, Italy 25 / 54

Relation of Aliased Components to Masking Threshold

100
Overview

Masking curve

Spring Reverb Models

80

Virtual Analog Circuits

60

Haptic Instruments
New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters
Audio FFT Filter Banks

SPL [dB]

Acoustic Guitar Models

40
20
0
20
0

10

15

20

25

Bark

Spectrum of a sawtooth waveform over masking threshold

Microphone Array
Faust to Flash Plugins
Summary

F0 = 4.3 kHz, Fs = 44.1 kHz

3rd-order B-spline interpolation = 4th-order DPW
[Juhan Nam]

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 26 / 54

Sound Examples
Overview
Spring Reverb Models
Virtual Analog Circuits

Plain Digital Sawtooth

Differentiated-Parabolic-Wave Sawtooth
Doubly Differentiated Cubic-Wave Sawtooth

Acoustic Guitar Models

Haptic Instruments
New Oscillators

Sawtooth Synth
Diffd Polynomials
Aliasing Suppression
More Examples
Comparisons
Comparisons LogF
Aliasing Masked
Sound Example
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 27 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Spectral Delay Filters

DAFx-09, Keynote III, Como, Italy 28 / 54

Spectral Delay Filters

Overview
Spring Reverb Models

- A1 (z) - A2 (z) - . . . - AM (z) - Heq (z)

x(n)

- y(n)

Virtual Analog Circuits

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters

Spectral Delay Filters

Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Frequency (kHz)

Haptic Instruments

20

10

4
3
Time (ms)

Summary

See presentation later this morning (11:20 am):

Spectral Delay Filters with Feedback Delay and Time-Varying
Coefficients
aki,
Jonathan Abel, and Julius Smith
Jussi Pekonen, Vesa Valim
Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 29 / 54

Spectral Delay Filters

Overview

Recent Paper (published online):

Spring Reverb Models

Virtual Analog Circuits

Spectral Delay Filters

Acoustic Guitar Models

Haptic Instruments
New Oscillators

aki,
Jonathan Abel, and Julius Smith
Vesa Valim
Journal of the Audio Engineering Society

Spectral Delay Filters

Spectral Delay Filters

Spectral Delay Filters

July/August 2009

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 30 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Audio FFT Filter Banks

DAFx-09, Keynote III, Como, Italy 31 / 54

Octave Filter Bank Schematic

Complex Octave Filter Bank
Overview

Spring Reverb Models

Virtual Analog Circuits

0.8

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array

Filter Gain

Haptic Instruments

Band 1
Band 2
Band 3
Band 4

0.6

0.4

0.2

0
1

4
5
Spectral Sample Number

Simple octave filter bank for complex signals.

Faust to Flash Plugins

Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 32 / 54

Basic Idea
X(8)
x(1)

x1(1)

x(2)

x1(2)

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models

ifft(4)
x(3)

x1(3)

x(4)

x1(4)

Haptic Instruments
New Oscillators

fft(8)

Spectral Delay Filters

Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

x(5)

x2(1)
ifft(2)

x(6)

x2(2)

x(7)

x3(1)

x(8)

x4(1)
X(1)

FFT implementation of one frame of simple octave filter bank

Successive frames non-overlapping (rectangular window)

DAFx-09, Keynote III, Como, Italy 33 / 54

Interpolated Frequency Response

Interpolated128 Channel-Signal Magnitude-Spectra Overlaid
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Gain (dB)

Haptic Instruments

-40

-60

-80

Band 1
Band 2
Band 3
Band 4

-100
0

200

400
600
Spectral Sample Number

800

1000

Channel-frequency-response overlay for three-octave filter bank

Filter-bank driven by an impulse

Zero-padded FFT taken for each channel signal
Magnitude responses overlaid
DAFx-09, Keynote III, Como, Italy 34 / 54

Practical Octave Filter Bank

Interpolated Channel Signal Spectra
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Gain (dB)

Haptic Instruments

-40

-60

-80

-100
0

500

1000

1500
2000
2500
3000
Spectral Sample Number

3500

4000

Improve channel-filter impulse responses:

Rectangularly windowed sinusoids
Chebyshev-windowed sinusoids

DAFx-09, Keynote III, Como, Italy 35 / 54

Superposition of Channel Spectra after Critical Sampling

Interpolated Channel Signal Spectra after Aliased Reconstruction
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Gain (dB)

Haptic Instruments

-40

-60

-80

-100
0

500

1000

1500
2000
2500
3000
Spectral Sample Number

3500

4000

Each channel maximually downsampled

Transition bands alias heavily
Aliasing cancels in filter-bank sum

DAFx-09, Keynote III, Como, Italy 36 / 54

IFFT Band Allocation Including Transition Bands

Channel Signal Spectra Interpolated by 4
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Gain (dB)

Haptic Instruments

-40

-60

-80

-100
0

200

400
600
Spectral Sample Number

800

1000

As before but allocating wider overlapping IFFT bands:

Passband plus transitions fit inside IFFT

Passbands contiguous as before, but IFFTs overlap more
DAFx-09, Keynote III, Como, Italy 37 / 54

Aliasing Suppression
Channel Signal Spectra Interpolated by 4 from Aliased Bands
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Gain (dB)

Haptic Instruments

-40

-60

-80

-100
0

200

400
600
Spectral Sample Number

800

1000

Reconstruction after critical downsampling

Aliasing is reduced because now there is no aliasing of transition
bands

DAFx-09, Keynote III, Como, Italy 38 / 54

Real Octave Filter Bank

Channel Signal Spectra Interpolated by 4
Overview

Spring Reverb Models

Virtual Analog Circuits

-20

Acoustic Guitar Models

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes
Microphone Array

Gain (dB)

Haptic Instruments

-40

-60

-80

-100
0

200

400
600
Spectral Sample Number

800

1000

Approximate octave filter bank for real signals

Faust to Flash Plugins

Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 39 / 54

Notes on Audio FFT Filter Banks

Overview
Spring Reverb Models
Virtual Analog Circuits

Arbitrary nonuniform spectral partitions and/or overlap-add

decompositions are easily implemented, while preserving the
FFT speed advantage

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters

Extension to time-varying nonuniform filter banks is

straightforward

Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes

Come see the poster this afternoon!

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 40 / 54

Spectrally Matched Click Synthesis

Overview
Spring Reverb Models
Virtual Analog Circuits

While outside the 1-year limit, Matt Wrights poster this afternoon
(4:30 pm) Spectrally Matched Click Synthesis is another work
performed at least partially at CCRMA with JOS input.

Acoustic Guitar Models

Haptic Instruments

FIR filter design to achieve a minimum-duration click having a

New Oscillators
Spectral Delay Filters

desired magnitude spectrum

Applications:

Audio FFT Filter Banks

Octave Filter Bank

Basic Idea
Frequency Response
Practical Response
Critical Sampling
Widened IFFT Bands
Suppressed Aliasing
Real Signals
Notes

Incremental attack strength modification

Continuous gradual morphing between an input sound and
successively more impulsive/ percussive sounds

Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 41 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Acoustically Transparent and

Configurable Microphone Array

DAFx-09, Keynote III, Como, Italy 42 / 54

Microphone Array
Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array

Mic Array
Mic Array Paper
Faust to Flash Plugins
Summary

Adustable geometry (software calibrated)

Sixteen microphones (Countryman B6 Omni Lavalier):
2 mm diameter capsules
1 mm diameter flexible mounting wire
Acoustically transparent over most of the audio band
Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 43 / 54

Upcoming Paper
Overview
Spring Reverb Models

A Configurable Microphone Array with Acoustically Transparent

Omnidirectional Elements

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators

Jonathan Abel, Nicholas Bryan, Travis Skare, Patty Huang, Darius

Mostowfi, Miriam Kolar, and Julius Smith
AES-2009, New York

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array

Mic Array
Mic Array Paper
Faust to Flash Plugins
Summary

Julius O. Smith III

Current Application:
Recording and modeling acoustic properties of underground

galleries at pre-Inca archeological site Chavn de Huantar

in Peru

DAFx-09, Keynote III, Como, Italy 44 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Flash Audio Plugins and Faust to

ActionScript Conversion

DAFx-09, Keynote III, Como, Italy 45 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Faust to ActionScript Conversion

Overview
Spring Reverb Models
Virtual Analog Circuits

CCRMA/EE graduate student Travis Skare developed a Faust

architecture file for Flash browser plugins:

http://ccrma.stanford.edu/~travissk/faustflash/

Acoustic Guitar Models

Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Faust to Flash
Summary

Faust generates C++ as usual

Alchemy (by Adobe Labs) translates C++ to ActionScript
Several Faust examples successfully compiled:

pitch-shifter, freeverb, karplus, osc,

multibandfilter
Interesting points to note:
Flash version 10 needed for run-time sound processing
Delay from plugin controls to sound is about half a second
Thanks to Google for allowing Travis to release his code as free
software

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 46 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary

Julius O. Smith III

Summary

DAFx-09, Keynote III, Como, Italy 47 / 54

Summary
Overview
Spring Reverb Models

In summary, we took a quick look at some DAFx-Related Research

involving JOS at CCRMA in 2008-2009:

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments

Spring Reverb Modeling Jonathan Abel et al. new

propagation modes and calibration methods

New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks

Digitizing Analog Circuits in Real Time David Yeh

Automated K-Method for nonlinear analog circuits

Microphone Array
Faust to Flash Plugins
Summary

Summary

Coupled Strings Analysis and Synthesis Nelson Lee

Fourth-order modes for low partials, wavuide model for upper
partials; new analysis techniques

Haptic Virtual Instruments Ed Berdahl Real controllers (with

force feedback) for virtual instruments

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 48 / 54

Summary, continued
aki
et al.
Alias-Suppressed Virtual Analog Vesa Valim
Overview
Spring Reverb Models
Virtual Analog Circuits

differentiated higher-order polynomials suppress aliasing further

aki
2005)
than in the parabolic case (Valim

Acoustic Guitar Models

Haptic Instruments
New Oscillators

aki
et al. impulse
Spectral Delay Filters Vesa Valim
response synthesis

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

Audio FFT Filter Banks JOS arbitrary nonuniform filter

banks (spectral overlap-add decompositions) using overlapping
IFFTs for each subband that include transition bands

Summary

Summary

Microphone Array Jonathan Abel et al. Acoustically

transparent, configurable, software-calibrated microphone array
for sampling the 3D sound field

Faust to Flash Plugins Travis Skare Tools for making Flash

plugins from Faust source
Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 49 / 54

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary
Further Details on
Differentiated
Polynomial Synthesis

Julius O. Smith III

Further Details on Differentiated

Polynomial Synthesis

DAFx-09, Keynote III, Como, Italy 50 / 54

Maximum Smoothness Problem Formulation

Overview
Spring Reverb Models

For maximum wraparound smoothness, compute an ,

n [1, n 2], such that

Virtual Analog Circuits

Acoustic Guitar Models

f (k) (1) = f (k) (1)

Haptic Instruments
New Oscillators

for k = 0, 1, . . . , n 1.

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins

These equations yield an upper triangular system

Triangular matrix equations are easily back-solved
Solution gives the maximually flat coefficients for f (x)

Summary
Further Details on
Differentiated
Polynomial Synthesis

Max Smoothness
Even-Odd Parts
Even-Odd Properties
Simplified Solution

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 51 / 54

Even and Odd Polynomials

Overview

In general, f (x) is a sum of its even and odd parts:

Spring Reverb Models

f (x) = fe (x) + fo (x)

Virtual Analog Circuits

Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters
Audio FFT Filter Banks
Microphone Array

where

f (x) + f (x)
fe (x) = fe (x) =
2

f (x) f (x)
fo (x) = fo (x) =
2

Faust to Flash Plugins

Summary
Further Details on
Differentiated
Polynomial Synthesis

Max Smoothness
Even-Odd Parts
Even-Odd Properties
Simplified Solution

Even part fe (x) contains all even powers of x:

fe (x) = + a4 x4 + a2 x2 + a0
Odd part fo (x) contains all odd powers of x:

fo (x) = + a5 x5 + a3 x3 + a1 x
Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 52 / 54

Solution Properties of Even and Odd Polynomials

Overview

We have

f (x) = fe (x) + fo (x)

Spring Reverb Models

Virtual Analog Circuits
Acoustic Guitar Models

where

Haptic Instruments
New Oscillators

fe (x) =

f (x) + f (x)
2

fo (x) =

f (x) f (x)
2

Spectral Delay Filters

Audio FFT Filter Banks
Microphone Array
Faust to Flash Plugins
Summary
Further Details on
Differentiated
Polynomial Synthesis

Max Smoothness
Even-Odd Parts
Even-Odd Properties
Simplified Solution

Julius O. Smith III

Note that fe (1) = fe (1) smoothness constraint satisfied

spontaneously by even part

Since fo (1) = fo (1), we must have fo (1) = 0

sum of fo coefficients must be zero in odd part

DAFx-09, Keynote III, Como, Italy 53 / 54

Simplified Maximum Smoothness Solution

Overview
Spring Reverb Models
Virtual Analog Circuits
Acoustic Guitar Models
Haptic Instruments
New Oscillators
Spectral Delay Filters

If f (x) is an odd-order polynomial satisfying

f (k) (1) = f (k) (1) for k = 0, 1, . . . , n 1, then it continues

to satisfy those constraints when its even part is replaced by zero

Similarly, the odd part of an even-order polynomial f (x) may be

set to zero without affecting its wraparound smoothness

Audio FFT Filter Banks

Microphone Array
Faust to Flash Plugins

Thus, without loss of generality, the starting polynomial f (x) may

be taken as even or odd, according its order

Summary
Further Details on
Differentiated
Polynomial Synthesis

Max Smoothness
Even-Odd Parts
Even-Odd Properties
Simplified Solution

The derivative of an even polynomial is odd, and vice versa

Every other polynomial derivative has the sum-to-zero constraint
Upper triangular system is reduced by about half

Julius O. Smith III

DAFx-09, Keynote III, Como, Italy 54 / 54