INCIDENTAL TIMBRES

Bachelor of Fine Arts, Graphic Design The University of Arts, Philadelphia, Pennsylvania. 2005

Submitted to Graduate Media Design in partial fulfillment of the requirements for the degree of Master of Fine Arts at Art Center College of Design. Pasadena, California. January 2012.

Authored by Salvador Orara Media Design Program, Graduate Media Design Candidate 2012

Salvador Orara 1

Lead Advisor: Philip Van Allen Writing Advisor: Shannon Herbert Thesis Committee: Elise Co and Mike Milley, with Casey Anderson Department Chair: Anne Burdick

META WORDS Auditory Display, Sonification, Sonic Interaction

MEGA THEMES Altered Perception, Affective Interaction, Productive Experiences, Immersive Intimacy

Salvador Orara 2

CONTENTS Anecdote Introduction 1 From Intentional to Incidental 2 Noisy Values 3 Context through Timbre 4 Incidental Timbres 5 Practice Works Cited

Salvador Orara 3

ANECDOTE Imagine yourself in a room full of people; some are sitting, standing, or walking about. Some have their eyes closed, others open. Swirling to and through you are semitonal blips, microtonal bleeps, backed by drones of static and hiss, mixed with dissolving pulses of Morse Code-like patterns sweeping in and out of focus. Pulling back, the sounds gather and coalesce into an assault of noise which invests your entire mind and body blanketing your skin with a tingling embrace. Then a moment of physical and mental clarity occurs. The noise has transcended rejection and approval, and has taken on a new meaning in itself. Stepping back in, your perceptions of the sounds change, there are messages, information, and data. Suddenly the sounds change, the timbre shifts, and the space is redefined. Everyone modifies their listening perspective, some climb stairs, others lay on the floor, some hug the walls, others continue to move about. Physical reactions: facial, full-body, stance, are tell-tale signs of the perception arc within the changes of frequency, amplitude, and tone. This is at the cusp of a moment, an instance when insurmountable layers of signals, data, and information or, the absence of becomes a tool to alter our perception of the material under observation and manipulation.

Our sonic future is thick with history and vastly undefined, what once was perceived as noise can become a signal what once stood as our signal can become our noise.

1 INTRODUCTION Listening is a complex sensory event involving pattern recognition unmatched by most modern computers. The human auditory system is capable of listening to an entire band on stage while focusing on the nuances of one guitar. This seemingly effortless, sensory system is complex, powerful, and flexible; and presents sound as a critical medium of exploration as a mode of information, experience, and interaction.

The interdisciplinary research of exploring sound in the development of objects and spaces has at least a 50 year history. A history which has been propelled by the ease with which technology has enabled sound to be implemented. This umbrella category of Auditory Display, thrives on an interdisciplinary

Salvador Orara 4

discourse ranging from artists, scientists, designers, and engineers. However, this diversity also leads to ungrounded theoretical models approaching it from one perspective would only allow for a single dimension and model of thought. The gray area in-between these various categories reveals a space where speculative approaches can be developed.

My research in the greater categories of sound and interaction have revealed an interest in developing idiosyncratic modes of productive experiences, affective interaction, altered perceptions, and immersive intimacy; all of which rest on a fine line of intricate immersion through the complex and enwrapping nature of sound. Following a model of inquiry focused on subjective, sublime, and affective values; this thesis is situated in a world that is increasingly technologically linked and globally mediated. Where we are inundated with quantitative functional models of display and interaction which leave no room for thought or meaning.

Incidental Timbres is a continuation along this line of inquiry of subjectivity, sublime, and affective values within and through technology. Incidental Timbres focuses on exploring ways of utilizing sound and interaction to break and re-connect our perception of the objects and infrastructures which define this complex world. Specifically, Incidental Timbres focuses on the concept of incidental sound as a means to expose the intimate possibilities we can have with our technologically expanding landscape.

1 FROM INTENTIONAL TO INCIDENTAL As the technological landscape expands we are discovering the characteristics of our electronic objects becoming quiet. The Sonification Handbook is the leading resource on the development and research of Auditory Displays and the many facets which create it. Sonification is a form of auditory display that uses a data-dependant generation of sound to communicate and transform data relations into perceived relations provided, that the transformation is systematic, objective, and reproducible. Sonification is then an intentional display specifically designed for an object and function. However, there is another dimension to our sonic experiences which is called the incidental. A more in-depth comparison between intentional and incidental sound is found in the Handbook:

Salvador Orara 5

Intentional sounds are purposely engineered to perform as an information display, and stand in contrast to incidental sounds, which are non-engineered sounds that occur as a consequence of the normal operation of a system (e.g., a car engine running). Incidental sounds may be quite informative (e.g., the sound of wind rushing past can indicate a cars speed), though this characteristic of incidental sounds is serendipitous rather than designed.

Sonification proclaims itself as a subjective layer or a sensationalized mode of perceiving information by ear. The distinction made between the two categories allows for speculation that the incidental sound is a critical path with potential applications. Incidental sound can be employed as a medium to design information and influence interaction taking it beyond the serendipitous and meaningless to the meaningful and purposeful. Incidental Timbres is a search for the ways of extracting incidental sound as means to develop a palette of material for design.

2 NOISY VALUES Incidental sound is synonymous to noise. Through this synonym Incidental Timbres breaks the mold of incidental sound away from the Handbook to the more speculative applications and characteristics of noise. Noise has the ability to transform our perception and presents itself as a complex method of communication. Noise is bridged into Incidental Timbres by The Sonification Handbook, where it is described that incidental sound is noise -- an unwanted by-product of events and actions taking place, and in the electrical sense, noise is the irregular fluctuations that accompany a transmitted electrical signal but are not part of it, and in musical terms -- pitch-less sound. However, in terms of information, the following definition allows for a more flexible approach: noise is the presence of many signals (more than one), or the lack thereof; both of which are practical states of information. This flexible practicality allows us to embrace the complexity of noise in different ways. I believe noise can be a self-sustaining communicative non-communicating art. Joseph Nechvatal discusses this notion in his book, Immersion Into Noise, where he proposes to entertain a non-communicating art of noise as an aesthetic act which communicates intricately. Incidental Timbres seeks to transform the seemingly meaninglessness of noise

Salvador Orara 6

into something meaningful -- something functional along the lines of being affective.

Noise is a supersensible disruption which can manifest in many different ways. Im interested in the affective ways which this disruption can function. My project The Apology You Deserve is a phonic noise dispenser intended to disrupt our perception of the private act of apologizing. The apology is a private event or message meant to be shared between two parties to alleviate tension, provide acknowledgment, and closure. The system transforms the affective act of an apology removing it from context and into a noisy by-product which disrupts and breaks the users perception of the apology itself. This places the user in a position to develop their own relationship arc with the apology. The post-supersensibile arc, in this example, is intended to follow a trajectory from incomprehension, comic relief, into a new intimate sensibility; revitalizing the users perception of the private act of apologizing into a new personal form of affective measurement. Incidental Timbres seeks to take the objective and functional intentions of activities and attempts to re-perceive it through subjectively affective force.

Noise can be a self-sustaining form of information. Im interested in ways which the disruptive nature of noise can transcend to subjective functions. Two projects, Tonalis Luminous and A Bow In C situates abstract noise in the forefront as a self-sustaining form of information not an unwanted by-product. Tonalis Luminous explores the idea of immersing any number of users into a cacophony of rhythmic tempos composed of metal and glass. What seems like noise at first gradually transitions into the mood of a system, ultimately defining the personality of the system through sound. This transition doesnt occur immediately. It requires the development of attention to details both at the macro and micro levels of perception -- a constant teetering at the cusp of comprehension and sensation. Immanuel Kant describes the sublime feeling as a kind of rapid alternation between the fear of the overwhelming and the peculiar pleasure of seeing that overwhelming overwhelmed. On the same line of inquiry, A Bow in C explores the idea of the aural sublime by immersing a user into dynamic states of sonic complexity. The simple mode of interaction afforded by the system is quickly subverted by the act of microperception. The macroperception is the larger objective view of noise itself -- the overwhelming. What Im interested in is the microperception where more affective results are achieved, Brian Massumi describes Felix Guattaris

Salvador Orara 7

view of microperceptions, microperception is not smaller perception; its a perception of a qualitatively different kind. Its something that is felt without registering consciously Microperception is this purely affective re-beginning of the world. When delving into the nuances of microperceptions, we become affectively invested because we have over come the overwhelming nature of the initial perception.

3 CONTEXT THROUGH TIMBRE When discussing sound we inheret terms from a world where sound thrives: music and art. According to the American National Standards Association, timbre is that attribute of auditory sensation in terms of which a listener can judge two sounds similarly presented and having the same loudness and pitch as dissimilar. In essence, its the way our ears differentiate an A being played on a piano than on a trumpet.

Luigi Russolos 1913 Futurist manifesto, The Art of Noises, embedded in a letter to the Futurist composer Francesco Balilla Pratella, outlines a framework of observing the world and its current techno-social condition by sound. He proposes a number of conclusions on how futurist musicians can harness the incidental sound found in the machines of his time as a "substitute for the limited variety of timbres that the orchestra possesses today. Russolo believed that listening to the incidental sound of our objective functional industrial landscape reveals a new sonic frontier capable of subjective functionality and affective applications. Engines, motors, and gears; the machines of Russolos time and the future technocultures to come, can harness these incidental timbres of our everyday tools and devices for artistic, speculative, and conventional means.

My project, Signals of Verbatim, specifically explored this notion of todays timbre, however, I was inspired by the history and development of Morse Code as a point of reference. A point in time when our modes of communication were at the early stages of mediation. The electrical telegraph, the mediator of that time, was a device to communicate the spoken word through a codified system of dots and dashes. The electrical telegraph and its intrinsic physical properties and technology, coupled with this codified system of writing, quickly became a noisy object. As it became evident that listening to the noisy by-

Salvador Orara 8

product, or the incidental sound of the device was just as effective as reading the dots and dashes. Like Russolo, this compelled the research and collection of the incidental sounds or the timbres of our contemporary communication devices: the smart phone. As scientific investigation is concerned, all the recordings were based on a systematic approach of treating every device exactly the same. Owners were requested to follow the same steps and perform the same tasks: text, email, load a website, get current geo-location, and watch a YouTube video. This systematic approach revealed the different timbres between each device, which are all performing the same task. Signals of Verbatim defines a context for Incidental Timbres: the immaterial infrastructure of the wireless network and the devices which mediate our relationship to and through it.

4 INCIDENTAL TIMBRES The methods by which our contemporary devices and our future technologies communicate with one another occurs in a layer of reality which is beyond our visual and audible means: the wireless network. We are not meant to be aware of this layer, nor are we meant to understand its workings its ever omnipresence is taken for granted. The wireless network is there, waiting for us to pick up a device and utilize it. Therefore, our only understanding of this infrastructure is based on our experiences at the surface of these devices which defines how, where, and when we have that access. Incidental Timbres materializes this intangible layer, making it physical, exposed, and liable to interpretation and manipulation.

This immaterial layer, in sonic form, is a supersensible disruptive noise capable of immediate overwhelming force. Incidental Timbres takes advantage of this disruptive force and seeks to exploit the human auditory system to alter our perception of the wireless network and the devices which mediate our relationship to and through it. The sublime arc of microperceptions is implemented to alter our perception of these devices from objective functional objects into more subjective affective artifacts.

4.1 INCIDENTAL TIMBRES FRAMEWORK The following diagram introduces the framework which the projects of Incidental Timbres are built upon. This framework outlines the specific details of how each theme manifests in a particular section: either as

Salvador Orara 9

value or context. This framework is specific to the development of the project Incidental Timbres, but the final outcome and its projects are wrapped in my overall interests and themes as a designer and artist.

4.2 PRINCIPLE PROJECTS Two principle projects have been formulated to manifest my interest in: productive experiences, affective interaction, altered perception, and immersive intimacy: The Timbre Array and Emergent Timbres.

4.2.1 THE TIMBRE ARRAY The Timbre Array is a spatial auditory interface to expose the timbre of a singular device and its relationship to the wireless network. It is composed of an array of 14 speakers of different sizes wired in parallel to spatialize the incidental timbre of the connection, transmissions, and internal actions of the device. Visitors are encouraged to control the timbre of the sound-space by manipulating the device. Manipulation will occur in different ways which have yet to be determined. However, the focus of the

Salvador Orara 10

project is to direct the attention of a singular devices timbre, and the possibilities of a dynamic spatial listening experience. Utilizing the familiar modes of interaction which are intrinsic to contemporary mobiles devices, The Timbre Array functions as a bridge to enable the exploration of microperceptions and the micro perceptions created by a single device. The Timbre Array is an immersive productive experience where the incidental timbre of a single device transitions from pitch-less sounds, into a functioning auditory interface. 4.2.2 EMERGENT TIMBRES Emergent Timbres explores the potential of utilizing the presence of many devices as they converse. With the development of a custom mobile web application, the devices transform into an organic animate object. Consisting of 8 stations equipped with on-board amplifiers and speakers; users can place their device on the mobile station, load the web application, and move freely about the space. In contrast to The Timbre Array, users are challenged to embrace a multitude of timbres like listening to an orchestra and are presented with the task of shifting their aural attention. To do so, they can move freely about the transforming space, follow a particular station, huddle, or spread out.

5 PRACTICE The research and values discovered over the course of two years has revealed an interest in the following areas: productive experiences, affective interaction, altered perception, immersive intimacy. My focus on sound has been to define and refine my interest in these areas as a means to explore speculative forms of interaction, experience, and information design. These values are specific to my projects which explore the use of sound and interaction however, they are values which can be applied to an array of mediums and contexts outside of sound. I believe that were entering an era where the need for more subjective qualitative values, and new perceptions of our world is in growing demand. We are moving beyond the objective functionality of technology and information, and were now focused on the affective, subjective, and sublime: the metaphysics of technology. Im interested in developing a practice for an era of media and design which has yet to, or may never surface. Where design research takes place on the fringes of society. Where speculation on the evolution of communication and information are combined with design, and interaction. Im interested in developing a mixed practice where these

Salvador Orara 11

speculative models of experimentation can inform conventional applications and live as artistic research in interaction design.

Salvador Orara 12

WORKS CITED Allen, Philip van. The New Ecology of Things. Limited. Media Design Program at Art Center College of Design, 2007. Print.

Barrass, Stephen and Vickers Paul. Sonification Design and Aesthetics Thomas Hermann, Andy Hunt, John G. Neuhoff. The Sonification Handbook. Logos Publishing House, 2011. Print

Brazil, Eoin and Fernstrom, Mikael. Navigation of Data. Thomas Hermann, Andy Hunt, John G. Neuhoff. The Sonification Handbook. Logos Publishing House, 2011. Print

Burnham, Douglas. Kant: Aesthetics. Internet Encyclopedia of Philosophy (2005): Web. 16 Jan. 2012.

Deleuze, Gilles, and Guattari, Felix. A Thousand Plateaus: Capitalism and Schizophrenia. 1st ed. Trans. Brian Massumi. University of Minnesota Press, 1987. Print.

Dunne, Anthony. Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. The MIT Press, 2006. Print.

Dyson, Frances. Sounding New Media: Immersion and Embodiment in the Arts and Culture. 1st ed. University of California Press, 2009. Print.

Hunt, Andy and Hermann, Thomas. Interactive Sonification Thomas Hermann, Andy Hunt, John G. Neuhoff. The Sonification Handbook. Logos Publishing House, 2011. Print

J.B Calvert, Receiving By Ear. The Electromagnetic Telegraph, 2008. Web 16 October 2011.

Salvador Orara 13

Laurel, Brenda. Designed Animism. Binder, Thomas, Jonas Lwgren, and Lone Malmborg. (Re) Searching the Digital Bauhaus. 1st ed. Springer, 2008. Print.

Martin, Sylvia. Futurism. Taschen, 2005. Print.

McVey, John. Introduction. Telegraphic Codes and Message Practices, 2011. Web. 16 October 2011.

McVey, John. Primitives. Telegraphic Codes and Message Practices, 2011. Web. 16 October 2011.

Nechvatal, Joseph. Immersion Into Noise. MPublishing, University of Michigan Library, 2011. Print.

Nees, Michael and Walker, Brucen N. Theory of Sonification. Thomas Hermann, Andy Hunt, John G.

Neuhoff. The Sonification Handbook. Logos Publishing House, 2011. Print

Rossing, Thomas D., Richard F. Moore, and Paul A. Wheeler. Science of Sound, The. 3rd ed. Addison Wesley, 2001. Print.

Shaviro, Steven. Bilinda Butcher. Kelly, Caleb, ed. Sound. 1st ed. The MIT Press, 2011. Print.

Virilio, Paul. The Information Bomb. Verso, 2006. Print.

Voegelin, Salome. Listening to Noise and Silence: Towards a Philosophy of Sound Art. Continuum, 2010. Print.

Wishart, Trevor. On Sonic Art. Routledge, 1996. Print.

Salvador Orara 14