You are on page 1of 7

Timbre as a structuring force in music

Stephen McAdams

Citation: Proc. Mtgs. Acoust. 19, 035050 (2013); doi: 10.1121/1.4799391


View online: https://doi.org/10.1121/1.4799391
View Table of Contents: http://asa.scitation.org/toc/pma/19/1
Published by the Acoustical Society of America

Articles you may be interested in


Timbre as a structuring force in music
The Journal of the Acoustical Society of America 133, 3448 (2013); 10.1121/1.4806102

The Timbre Toolbox: Extracting audio descriptors from musical signals


The Journal of the Acoustical Society of America 130, 2902 (2011); 10.1121/1.3642604

Multidimensional perceptual scaling of musical timbres


The Journal of the Acoustical Society of America 61, 1270 (1977); 10.1121/1.381428

The dependency of timbre on fundamental frequency


The Journal of the Acoustical Society of America 114, 2946 (2003); 10.1121/1.1618239

The Measurement of Timbre


The Journal of the Acoustical Society of America 8, 207 (1937); 10.1121/1.1901995

Perceptual effects of spectral modifications on musical timbres


The Journal of the Acoustical Society of America 63, 1493 (1978); 10.1121/1.381843
S. McAdams

Proceedings of Meetings on Acoustics


Volume 19, 2013 http://acousticalsociety.org/

ICA 2013 Montreal


Montreal, Canada
2 - 7 June 2013

Musical Acoustics
Session 3pMU: Perception and Orchestration Practice

3pMU1. Timbre as a structuring force in music


Stephen McAdams*​

​ *Corresponding author's address: Schulich School of Music, McGill University, 555 Sherbrooke Street West, Montreal, H3A 1E3,
QC, Canada, smc@music.mcgill.ca
Most of the music we enjoy uses the musical qualities of different instruments to create specific perceptual and emotional effects that
composers sculpt over time. Timbre is the auditory attribute that distinguishes different instruments. Research on timbre perception has
demonstrated that it is multifaceted and contributes in many ways to the perceptual organization of musical structures. The art of structuring
music with timbre is orchestration. A survey of orchestration treatises reveals the dearth of underlying theory, in sharp contrast to other
traditional areas such as harmony and counterpoint, which have long theoretical traditions. We seek to develop a theoretical ground for
orchestration practice starting with the structuring role that timbre can play in music. Many aspects of musical structuring are achieved by
auditory scene analysis, the perceptual processes that result in unified events, integrated streams of events, groups of events segmented into
phrases and sections, and larger-scale units extended over time that we call orchestral gestures. The roles that timbre plays in the manifestation
of these principles in orchestration practice will be considered as potential elements of a theory of orchestration. How such principles might be
incorporated into computer-aided orchestration systems and computer-aided orchestral rendering systems will also be examined.

Published by the Acoustical Society of America through the American Institute of Physics

© 2013 Acoustical Society of America [DOI: 10.1121/1.4799391]


Received 18 Jan 2013; published 2 Jun 2013
Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 1
S. McAdams

INTRODUCTION
Most music uses the musical qualities of different instruments to create specific perceptual and emotional effects
that composers sculpt over time. The auditory attribute that distinguishes different instruments is called timbre, and
research on timbre perception has demonstrated that it is multifaceted, can be linked to acoustic properties of
instrument sounds (Caclin et al., 2005, 2007; Chiasson et al., 2010; Giordano & McAdams, 2010; Marozeau et al.,
2003; McAdams, 1993; McAdams & Winsberg, 2000; McAdams et al., 1995; Peeters et al., 2011), and contributes
in many different ways to the perceptual organization of musical structures (McAdams, 2013). The art of structuring
music with timbre is orchestration.
A survey of orchestration treatises reveals the dearth of underlying theory. Their main focus is to provide
examples of scoring the most appropriate pitch and register for a given instrument and common combinations of
instruments. The presentation of this material most often resembles recipes with specific ingredients rather than
aspects of theory that would allow a practitioner to generalize beyond the given example to create new sonorities in
a principled way. This situation is in sharp contrast to the areas of harmony, musical form, voice leading, and
counterpoint, which have long theoretical traditions. But even these traditional musical phenomena can be strongly
affected by orchestration, and surprisingly the role of timbre in their perception is rarely addressed.
To begin to develop a theoretical ground for orchestration practice in Western art music, it would seem crucial
to start with the structuring role that timbre can play in music. In line with a recent trend toward the incorporation of
psychological principles and empirical research into what is now known as cognitive music theory, this ground
could be sought in principles that govern how timbre affects perceived musical structure. Many of these principles
can be derived from the theory of auditory scene analysis, which involves the perceptual processes by which musical
information is structured into: 1) unified events, 2) integrated streams of events, 3) groups of events segmented into
phrases and sections, and 4) larger-scale units extended over time that one might call orchestral gestures. Taken
together and linked to other musical phenomena, these areas of enquiry can contribute to theoretical considerations
of timbre as a form-bearing element in music (McAdams, 1989b, 1999; McAdams & Giordano, 2009; McAdams et
al, 2004; Poulin-Charronnat et al., 2004).

TIMBRE AND AUDITORY SCENE ANALYSIS


Of the myriad issues that could be addressed in orchestration practice, the most powerful techniques are those
that rely intuitively on processes related to auditory scene analysis, with particular focus on their interaction with
timbre perception. Four topics that address different processes will be considered: concurrent grouping, sequential
grouping, segmentational grouping, and the formation of orchestral gestures.

Concurrent Grouping
What combinations of instruments playing concurrently and what kinds of temporal structuring facilitate
blending from which new timbres emerge, as in the parallel melodies by different instruments in Ravel's Boléro? Or
conversely, what properties facilitate their segregation into distinct voices?
The creation of new timbres by orchestration necessarily depends on the degree to which the constituent sound
sources are fused or blended (Brant, 1971; Bregman, 1990; Erickson, 1975; Esling & Agon, 2010; Marin &
McAdams, 1991; McAdams, 1989a; McAdams & Bregman, 1979; Read, 2004). Sandell (1995) proposes three
perceptual goals in combining timbres concurrently; timbral heterogeneity: the instruments retain their individual
identities; timbral augmentation: the identity of one instrument is retained, but its timbre is embellished by the other;
and timbral emergence: all timbres fuse into a new sound entity and lose their individual identities (Kendall &
Carterette, 1993). Principles of concurrent auditory grouping (harmonicity, onset synchrony, parallel change in pitch
and dynamics) all contribute to the process of perceptual fusion (McAdams, 1984), but the actual acoustic properties
of the constituent sounds are also crucial, e.g., sounds with similar attacks and spectral properties are more likely to
blend (Sandell, 1995; Lembke & McAdams, 2011; Tardieu & McAdams, 2012). Perceptual data concerning timbral
blend have been incorporated into the computer-aided orchestration (CAO) environment Orchidée (Carpentier et al.,
2010). However, little music analysis has addressed how these effects are actually created in music and what their
formal and ornamental functions might be.

Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 2


S. McAdams

Sequential Grouping
How do relative similarities or differences in instrumental timbres favor their clear distinction as carriers of
melodic and rhythmic information in settings in which the voices move independently, as in a wind quintet, as
opposed to settings in which they are integrated into a common texture, as in a Barbershop quartet? To what extent
does the creation of orchestral layers, as in Berlioz's Symphonie Fantastique or Takemitsu’s How Slow the Wind,
depend on timbral similarities within layers and timbral differences between layers? Which instrumental timbres or
combinations are most likely to stand out in complex textures (creating foreground and background elements)?
Auditory streams carry melody and rhythm in music, and their perception depends on integrating the notes that
form them into the same stream (Bey & McAdams, 2003; McAdams & Bertoncini, 1997; McAdams & Bregman,
1979). Timbral continuity is an important factor in auditory stream formation and timbral discontinuity in the
segregation of notes into separate streams (Bregman & Pinker, 1978; Hartmann & Johnson, 1991; Singh &
Bregman, 1997; Wessel, 1979). The degree of segregation is directly related to the degree of similarity between
timbres (Bey & McAdams, 2003; Iverson, 1995). Timbre is also an important cue for following a voice that crosses
other voices (Culling & Darwin, 1993; McAdams & Bregman, 1979) or hearing out a given voice in a polyphonic
texture (Gregory, 1990; Janata et al., 2002). We have identified the acoustic cues involved in the perception of
timbral salience (Chon & McAdams, 2012a,b). Some music-theoretic work has applied streaming principles to
explain the rules of voice leading and polyphonic writing (Huron, 2001; Wright & Bregman, 1987), but with only
cursory consideration of timbre (Huron, 2001). How such effects depend on timbre and actually operate in orchestral
music has not been studied.

Segmentational Grouping
How are changes in instrumentation used to segment musical materials at specific points in time at the level of
motifs or themes, such as the antiphonal contrasts resulting from patterns alternating between the strings and winds
in the second movement of Beethoven's 9th Symphony? How do they work at the level of larger-scale musical
sections, as in the succession of sections of brass, woodwinds/strings, strings, and then woodwinds at the opening of
Sibelius' Finlandia?
Timbre is an important component in the perception of musical groupings at the segmentational level.
Discontinuities in timbre (instrument change) can provoke segmentation of longer sequences of notes into smaller
groups (Deliège, 1987) or of larger-scale sections delimited by significant changes in instrumentation and texture
(Deliège, 1989). Repeating timbral patterns that are learned over sufficient periods of time can also create
segmentation of sequences into smaller-scale timbral patterns (Tillmann & McAdams, 2004). Specific evaluation of
the role that timbre plays in such structuring in real pieces of music is exceedingly rare in the literature.

Orchestral Gestures
What techniques are used to create higher-level auditory images that are perceived as gestures with timbral
evolution, such as an orchestral swell or waning, a sudden “wall of sound” or “drop off”, or Klangfarbenmelodien
(timbre melodies, Schoenberg, 1911)? Each of these “orchestral gestures” is composed of multiple instruments, but
their perceptual cohesion gives rise to properties that emerge from the temporal shape of the whole and often creates
a dramatic emotional impact in concert with other musical parameters (Goodchild et al., 2011, 2012).
There is very little music-theoretical or perceptual research on the topic of orchestral gestures. However, such
gestures have been shown to contribute to peak emotional experiences in orchestral music (Guhn et al., 2007;
Sloboda, 1991). While some orchestration treatises mention certain of these gestures, a major concern in this topic is
the lack of a clear taxonomy of techniques and of a conceptual framework related to their musical function.

COMPUTER-AIDED ORCHESTRATION AND ORCHESTRAL RENDERING


For orchestral pedagogy, Denys Bouliane and Félix Baril at McGill University are developing a computer-aided
orchestral rendering package, the Digital Orchestra SIMulator (DOSIM). This kind of tool can provide orchestral
rendering of a score coded in the Finale notation software with remarkably compelling results. This kind of highly
interactive tool will be crucial in the future for developing reorchestrations to test specific hypotheses about the
various perceptual and cognitive phenomena researchers on orchestration may wish to study.

Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 3


S. McAdams

Carlos Agon and his team at IRCAM in Paris have developed the computer-aided orchestration environment
Orchidée (Carpentier et al., 2010), which allows composers to specify instrumentation and abstract orchestrational
goals and to receive different proposals for possible orchestrations. This environment has been extended by Eric
Daubresse and collaborators at the Haute Ecole de Musique de Genève with Orchis (Carpentier et al., 2012a,b) at
HEMG and by Agon's team at IRCAM with Philippe Esling's ATO-MS (Esling & Agon, 2010, in press; Esling et
al., 2010). These environments will be crucial for proposing reorchestrations to test specific hypotheses about the
role of timbral characteristics in the various orchestration effects. By integrating the perceptual and cognitive results
from psychological experimentation, perceptually relevant rules and principles can be embedded within these
systems.

CONCLUSIONS
A true theory of orchestration will have a major impact on composers, orchestrators/arrangers, and teachers, as
well as on people developing digital environments for such users. A clear theorizing of orchestral effects and their
relation to more traditional aspects of music theory will launch an area of enquiry that has just barely been touched
upon by the music theory community.

ACKNOWLEDGMENTS
This work is supported by the Canada Research Chairs program and the Canadian Natural Sciences and
Engineering Research Council.

REFERENCES
Bey, C. & McAdams, S. (2003). Post-recognition of interleaved melodies as an indirect measure of auditory stream formation, J.
Exp. Psychol.: Human Percep. Perf. 29, 267–279.
Brant, H. (1971). Orchestration. In J. Vinton (Ed.), Dictionary of Contemporary Music (pp. 538-546). New York: E. P. Dutton.
Bregman, A. S. (1990). Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: MIT Press.
Bregman, A. S. & Pinker, S. (1978). Auditory streaming and the building of timbre. Can. J. Psychol. 32, 19-31.
Caclin, A., McAdams, S., Smith, B. K., & Winsberg, S. (2005). Acoustic correlates of timbre space dimensions: A confirmatory
study using synthetic tones, J. Acoust. Soc. Am. 118, 471-482.
Caclin, A., Giard, M.-H., Smith, B., & McAdams, S. (2007). Interactive processing of timbre dimensions: A Garner interference
study, Brain Res. 1138, 159-170.
Carpentier, G., Cordero, V. & Daubresse, E. (2012a). Cartographier le timbre. Projets Orchis/Pléione à la Haute Ecole de
Musique de Genève, Dissonances 119, 48-63.
Carpentier, G., Daubresse, E., Garcia Victoria, M., Sakei, K. & Villanueva Carratero, F. (2012b). Automatic orchestration in
practice, Comput. Mus. J. 36, 24-42.
Carpentier, G., Tardieu, D., Harvey, J., Assayag, G. & Saint-James, E. (2010). Predicting timbre features of instrument sound
combinations: Application to automatic orchestration, J. New Mus. Res. 39, 47-61.
Chiasson, F., Traube, C., Smith, B. & McAdams, S. (2010). Koechlin's volume: Perception of auditory size among instrument
timbres from different families, Proc. 11th Int. Conf. Mus. Percep. Cog., Seattle, WA.
Chon, S.H. & McAdams, S. (2012a). Exploring timbre blending as a function of timbre saliency, Proc. Int. Conf. Mus. Percep.
Cog., Thessaloniki.
Chon, S. H. & McAdams, S. (2012b). Investigation of timbre saliency, the attention-capturing quality of timbre, Proc. Acoustics
2012, Hong Kong.
Culling, J. F. & Darwin, C. J. (1993). The role of timbre in the segregation of simultaneous voices with intersecting F0 contours,
Percept. Psychophys. 54, 303-309.
Deliège, I. (1987). Grouping conditions in listening to music: An approach to Lerdahl & Jackendoff's grouping preference rules,
Mus. Percep. 4, 325-360.
Deliège, I. (1989). A perceptual approach to contemporary musical forms, Contemp. Mus. Rev. 4, 213-230.
Erickson, R. (1975). Sound Structure in Music. Berkeley, CA: University of California Press.
Esling, P. & Agon, C. (2010). Composer les mélanges sonores avec les maquettes spectrales (Composing sound mixtures with
spectral mockups), Actes des Journées d’Informatique Musicale (Proceedings of the Computer Music Conference), Rennes.
Rennes: Université Rennes 2.
Esling, P. & Agon, C. (in press). Time series data mining and analysis, ACM Computing Surveys.
Esling, P., Carpentier, G. & Agon, C. (2010). Dynamic musical orchestration using genetic algorithms and a spectro-temporal
description of musical instruments. In C. Di Chio et al. (Eds.), Applications of Evolutionary Computation, LNCS 6025, 371-
380. Berlin: Springer Verlag.

Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 4


S. McAdams

Giordano, B. L. & McAdams, S. (2010). Sound source mechanics and musical timbre perception: Evidence from previous
studies, Mus. Percep. 28, 155-168.
Goodchild, M., Crellin, J., Wild, J., & McAdams, S. (2011). Construction and perception of 'peak experience' in music listening,
Proc. Soc. Mus. Percep. Cog., Rochester, NY.
Goodchild, M., Wild, J. & McAdams, S. (2012). The role of orchestral gestures on continuous ratings of emotional intensity,
Proc. Int. Conf. Mus. Percep. Cog., Thessaloniki.
Gregory, A. H. (1990). Listening to polyphonic music,. Psychol. Mus. 18, 163-170.
Guhn, M., Hamm, A., & Zentner, M. (2007). Physiological and music-acoustic correlates of the chill response, Mus. Percep. 24,
473-483.
Hartmann, W. M. & Johnson, D. (1991). Stream segregation and peripheral channeling, Mus. Percep. 9, 155-184.
Huron, D. (2001) Tone and voice: a derivation of the rules of voice-leading from perceptual principles, Mus. Percep. 19, 1-64.
Iverson, P. (1995). Auditory stream segregation by musical timbre: Effects of static and dynamic acoustic attributes, J. Exp.
Psychol.: Human Percep. Perf. 21, 751-763.
Janata, P., Tillmann, B., & Bharucha, J.J. (2002). Listening to polyphonic music recruits domain-general attention and working-
memory circuits, Cog. Affect. Beh. Neurosci. 2, 121-140.
Kendall, R. & Carterette, E. (1993). Identification and blend of timbres as a basis for orchestration, Contemp. Mus. Rev. 9, 51-
67.
Lembke, S.-A. & McAdams, S. (2011). The relevance of spectral shape to perceptual blend between wind instrument timbres.
Proc. Soc. Mus. Percep. Cog., Rochester, NY.
Marin, C. & McAdams, S. (1991) Segregation of concurrent sounds. II: Effects of spectral envelope tracing, frequency
modulation coherence, and frequency modulation width, J. Acoust. Soc. Am. 89, 341-351.
Marozeau, J., de Cheveigné, A., McAdams, S., & Winsberg, S. (2003). The dependency of timbre on fundamental frequency, J.
Acoust. Soc. Am. 114, 2946-2957.
McAdams, S. (1984) The auditory image: A metaphor for musical and psychological research on auditory organization. In W.R.
Crozier & A.J. Chapman (eds.), Cognitive Processes in the Perception of Art, North-Holland, Amsterdam, pp. 289-323.
McAdams, S. (1989a) Segregation of concurrent sounds. I: Effects of frequency modulation coherence, J. Acoust. Soc. Am. 86,
2148-2159.
McAdams, S. (1989b). Psychological constraints on form-bearing dimensions in music, Contemp. Mus. Rev. 4(1), 181-198.
McAdams, S. (1993). Recognition of sound sources and events. In S. McAdams & E. Bigand (Eds.), Thinking in Sound: The
Cognitive Psychology of Human Audition (pp. 146-198). Oxford: Oxford University Press.
McAdams, S. (1999). Perspectives on the contribution of timbre to musical structure, Comput. Mu. J. 23(2), 96-113.
McAdams, S. (2013). Musical timbre perception. In D. Deutsch (Ed.), The Psychology of Music, 3rd ed. (pp. 35-67), San Diego,
CA: Elsevier.
McAdams, S. & Bertoncini, J. (1997). Organization and discrimination of repeating sound sequences by newborn infants, J.
Acoust. Soc. Am.102, 2945-2953.
McAdams, S. & Bregman, A. S. (1979). Hearing musical streams, Comput. Mus. J. 3(4), 26-43.
McAdams, S. & Giordano, B. L. (2009). The perception of musical timbre. In S. Hallam, I. Cross & M. Thaut (Eds.), Oxford
Handbook of Music Psychology, Oxford University Press, New York, pp. 72-80.
McAdams, S. & Winsberg, S. (2000) Psychophysical quantification of individual differences in timbre perception. In A. Schick,
M. Meis & C. Reckhardt (eds.), Contributions to Psychological Acoustics: Results of the 8th Oldenburg Symposium on
Psychological Acoustics, Bis, Oldenburg, pp. 165-182.
McAdams, S., Vieillard, S., Houix, O., & Reynolds, R. (2004). Perception of musical similarity among contemporary thematic
materials in two instrumentations, Mus. Percep. 22, 207-237.
McAdams, S., Winsberg, S., Donnadieu, S., De Soete, G., & Krimphoff, J. (1995). Perceptual scaling of synthesized musical
timbres: Common dimensions, specificities, and latent subject classes, Psychol. Res. 58, 177-192.
Peeters, G., Giordano, B.L., Susini, P., Misdariis, N., & McAdams, S. (2011). The Timbre Toolbox: Extracting audio descriptors
from musical signals, J. Acoust. Soc. Am. 130, 2902-2016.
Poulin-Charronnat, B., Bigand, E., Lalitte, P., Madurell, F., Vieillard, S., & McAdams, S. (2004). Effects of a change in
instrumentation on the recognition of musical materials, Mus. Percep. 22, 239-263.
Read, G. (2004). Orchestral Combinations: The Science and Art of Instrumental Tone-Color. Blue Ridge Summit, PA:
Scarecrow Press.
Sandell, G. J. (1995). Roles for spectral centroid and other factors in determining "blended" instrument pairings in orchestration,
Mus. Percep. 13, 209-246.
Schoenberg, A. (1911/1978). Theory of Harmony. Original German publication, 1911 ed. University of California Press,
Berkeley, CA.
Singh, P. G. & Bregman, A. S. (1997). The influence of different timbre attributes on the perceptual segregation of complex-tone
sequences, J. Acoust. Soc. Am. 120, 1943-1952.
Sloboda, J. A. (1991). Music structure and emotional response: Some empirical findings, Psychol. Mus., 19, 110-120.
Tardieu, D. & McAdams, S. (2012). Perception of dyads of percussive and sustained instruments, Mus. Percep., 30, 117-128.
Tillmann, B. & McAdams, S. (2004). Implicit learning of musical timbre sequences: Statistical regularities confronted with
acoustical (dis)similarities, J. Exp. Psychol.: Learn. Mem. Cog. 30, 1131-1142.
Wessel, D. L. (1979). Timbre space as a musical control structure, Comput. Mus. J., 3(2), 45-52.

Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 5


S. McAdams

Wright, J. K., & Bregman, A. S. (1987). Auditory stream segregation and the control of dissonance in polyphonic music,
Contemp.Mus. Rev. 2(1), 63-92.

Proceedings of Meetings on Acoustics, Vol. 19, 035050 (2013) Page 6

You might also like