Artificial Intelligence and Music

Author(s): C. Roads
Source: Computer Music Journal, Vol. 4, No. 2, Artificial Intelligence and Music Part 1
(Summer, 1980), pp. 13-25
Published by: The MIT Press
Stable URL: http://www.jstor.org/stable/3680079
Accessed: 14-07-2017 00:22 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted
digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about
JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://about.jstor.org/terms

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to Computer Music
Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
C. Roads
Artificial
Intelligence and
Music

Introduction chines. Some degree of musical intelligence would

be beneficial in each case.
These special issues on artificial intelligence (AI) In the following paragraphs some of the ties con-
and music mark a turning point in twentieth cen- necting AI with music will be surveyed. Due to
tury music. An increasing number of developments time limitations involved in preparing this issue of
point to a new and pregnant musical concept: artifi- the Computer Music Journal, a comprehensive
cial musical intelligence. This concept, taking on overview will not be attempted, but many pointers
a variety of forms and appearing in a range of appli- for further study will be provided.
cations, implies a new and possibly deeper way of
looking at music. In addition, it holds the potential
for transforming many musical activities and for Developments in Al
making new musical activities manifest.
AI offers more than simply a set of new para- AI appears so useful today because of the rapid
digms for music. There exists an increasing body of progress that has been made and the present fast
strategies and applications of AI methodology as a pace of the field. Important developments include
general problem-solving approach. AI can purport the creation of AI languages (e.g., LISP, Smalltalk,
to model something as intricate as cognitive pro- knowledge representation language (KRL), Prolog),
cessing precisely because of the power of this meth- the proliferation of search strategies and knowl-
odology. Thus one can use AI methodology without edge-representation schemes, the application of
attempting to model mental activity. expert problem-solving in limited knowledge
The articles in these issues address both these domains, and progress toward natural language
poles: projects working toward musical intelligence understanding. In addition, a number of interesting
(Rothgeb, Alphonce, Meehan, Rahn, Laske, Minsky, systems have been developed that model perceptual
and Fry) and applied AI methodology (Greussay, processing. Image understanding, for example, is a
Strawn, Smoliar). well-established branch of AI (McCarthy, et al.
The introduction of AI techniques addresses 1978). Planning is a part of any system that needs
many problems left at the present plateau of com- to interact in a complicated way with its world.
puter music. These include the apparent impasse of The AI fields of learning and inductive inference
music-analysis programs that are based on only ren- constitute a difficult set of problems for AI re-
dering an account of surface structure of composi- searchers. It is one thing to program a machine to
tions, the one-dimensional user interfaces of many understand something on the basis of preset seman-
digital sound synthesis systems, the outmoded tic categories, but it is quite another thing to pro-
composition programs, our inadequate understand- gram a machine to do as a child does, for example,
ing of cognitive processes involved in listening and to abstract and infer the meanings of phenomena
other musical tasks, and the rigid protocols im- on its own. A great deal of research has already
posed by unintelligent digital performance ma- been poured into all of these areas. Several good
surveys of the field of AL exist, including those by
@ 1980 C. Roads
Barr and Feigenbaum (1979), Winston (1977), and
Winston and Brown (1979) among others. Nilsson
Computer Music Journal, Vol. 4, No. 2, Summer 1980,
(1980) concentrates specifically on algorithmic
methodology generic to AI.

Roads 13

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
History composition; Mozart's Musikalisches Wiirfelspiel
and Pepys's Musarithmica Mirafica (ca. 1670) are
Although the term artificial intelligence is rela- others. A less famous early project was the machine
tively new, the concept goes back many centuries. for composition developed by the Dutch musical
It may come as a surprise to some to know that the inventor Dietrich Nicholas Winkel, who was also
ideas of musical automata and automatic music the real inventor of the metronome (Malzel is most
machines are also centuries old. A few significant often given the credit). In 1821 Winkel completed a
events will be mentioned here. One of the earliest device that he called the Componium. This ma-
reported automatic musical instrument designs was chine was displayed and aroused great interest in
for a wind instrument, prepared by Appolonius ofits time. The Componium was capable of producing
Perga (247-205 B.c.). Buchner (1978) reports that variations
in on a theme programmed into it.
the ninth century Leo the Philosopher made two Much later, the first musical application of the
automata settings that included singing birds. Bi- computer to composition was Lejaren Hiller's pi-
nary programmable carillons were developed by oneering work with algorithmic music in 1955 at
some Dutch musical engineers in the 1200s, 200the University of Illinois. Central to this work was
years before the first planetary computer (developed
the notion that music is an algorithmic process.
by Iranian astronomers in the fourteenth century). The foundations of algorithmic composition were
Up to 10,000 memory locations were available onlaid centuries before the advent of the Eckert-
these programmable carillons (Buchner 1978). In Mauchly-von Neumann style computer. The germ
the 1500s, no less a personage than Leonardo da idea of organizing musical compositions around a
Vinci designed and built a mechanical spinet piano set of systematic procedures contains within it the
and drum set. The construction and display of implication that these procedures could be made
working musical automata (particularly the work of automatic. The inverse notion, gaining ever more
Jacquet-Droz and son) fascinated eighteenth-cen- significance, is that these procedures and syntactic
tury society. Music boxes based around punched-structures can be recognized automatically. Indeed,
metal-disk technology were developed to grand one of the fundamental notions of any AI applica-
tion is that it can be characterized as rule-structured.
proportions (27 in) in the nineteenth century, ulti-
mately leading to the automated orchestra, the Certainly one of the major tasks of composition is
orchestrion. In 1877 both the electric phonograph creating a rule system (either explicitly or im-
(Edison) and the microphone (Berliner) were in- plicitly) for a piece. That is, besides specifying com-
vented, which allowed for storage of any kind of positional materials, each composer organizes a
sound information. By 1930 the mechanical age complex of rules (sometimes implicitly) for struc-
was over and the electronic age was well under way.turing them. Clearly creative composers do not
Storage and reproduction of recorded music was ac- simply execute a fixed set of instructions, inserting
complished by electromechanical means (audio new sound materials, for every piece. Of course
disks, audio tape, etc.). Electromechanical switch-music is not just rules; but rule specification is one
ing mechanisms led to the development of sequence- component of composition.
controlled calculators and ultimately to the devel- As mentioned, the inverse of relying on rule
opment of the purely electronic computer, the structures for compositional purposes lies in using
Eniac, in 1945. rules for guiding an analysis of music. Bronson's en-
coded score analysis (Bowles 1970) on an IBM
computer in 1949 is often cited as the first instance
Music and Rule Structures of work in this domain. However, IBM introduced
its first computer, the 701, in 1953, so it must be
Guido d'Arezzo's table-lookup procedure for gener- assumed that Bronson's work was actually car-
ating pitches from spoken text (d'Arezzo 1978) is ried out on a huge electromechanical sequence-
but one early example of a generative procedure forcontrolled calculator. Since then there have been

14 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
many efforts aimed at analyzing encoded musical description from the (encoded) musical score. The
scores. It has become apparent that these first-gen- program was proposed as a tool to be used to arrive
eration music-analysis programs can only go so far at an understanding of the cognitive activity of the
without knowing anything about what they are music listener, and in music theory as a means of
analyzing. By far the most prevalent techniques in explicitly describing pattern in music for purposes
these programs involve either labeling or tallying of style comparison. Simon and Sumner also sug-
various surface features of an encoded musical text. gested that it might in the future be used for
The idea of data-driven or knowledge-driven analy-experiments in music composition. Simon, of
sis was mostly absent in the early analysis course, made numerous references to cognitive pro-
programs. cesses in composition in his highly interesting
little volume The Sciences of the Artificial (1969).
The notation used in "Pattern in Music" was a
First Phase of Modern Research in AI and Music mixture of declarative and procedural statements
about a music fragment. Declarative statements
1968 marks the beginnings of modern research intowere simply facts about the music, for example,
AI and music. Two major papers were published
X11= 'C'
that year: "Pattern in Music" by Herbert Simon and
Richard Sumner at Carnegie-Mellon University, means "the first note is C." The procedural state-
and "Linguistics and the Computer Analysis of To-ments attempted to characterize each important
nal Harmony" by Terry Winograd, then at M.I.T. feature as the result of an operation which, if ap-
There were other indications of a changing aware-plied, would generate that feature. For example, the
ness. According to A. W. Slawson, A. Forte had statement

concluded that questions in music analysis would

increasingly become questions in AI (Forte 1967), xl = N4 (DIAT; x(i - ) 1)
for i = 2, 3, . . . 8
while Slawson himself suggested that Chomsky's
work on grammars might be a better generative means "each note after the first is four step
model for music than the popular Markov-chain above its predecessor on the diatonic scale"
model (Slawson 1968). By 1972 Moorer had com-and Sumner 1968, p. 231).
pleted a project in composition using heuristic
programming techniques from AI. In the following
Winograd
paragraphs, Sumner's, Simon's, Winograd's, and
Moorer's work will be discussed in a little more Winograd's harmony-analysis program was designed
detail. to do the kind of chord labeling involved in most
courses in the subject. Selections from Schubert
Sumner and Simon and Bach were used to test the program, which was
based on the principles of a systemic grammar. Sys-
"Pattern in Music" reported on a project whose pur-
temic grammar is a model developed in the 1960s
pose was to formalize musical patterns in tonal by M. A. K. Halliday and his associates at Univer-
music in terms of rhythm, melody, harmony, and sity College, London (Halliday 1973). Halliday's
form. (These categories were acknowledged as oftensystemic grammar was an abstraction and formal-
not distinct.) This pattern language was an exten-ization of the linguist Firth's category of a sys-
sion of a formalism used in information-processingtem-an abstract representation of a paradigm.
psychological experiments at Carnegie-Mellon. Si- This was interpreted by Halliday as a set of options
mon and Sumner designed a computer program for with an entry condition-a number of possibilities
translating pattern-language descriptions into out of which a choice has to be made if the stated
common musical notation. A second program was conditions of entry are satisfied, An example of
designed that automatically inducted the pattern Halliday's notation is shown in Fig. 1. Winograd's

Roads 15

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
Fig. 1. Halliday's notation.

a -- Meaning: if (a) then (x or y)

y
Y

a Meaning: if (a) then both (x or y) and (m or n)

mr

application of that notation is shown in Fig. 2. TERPE for real-time modeling of musical struc-
(Note that Winograd has added one other feature; tures. The language was used to generate Gregorian
see the terms ":*K* =major:-II." This notation is chant, medieval polyphony, Bach counterpoint, and
used to contextualize that particular production sonata-form examples. Sound generation was part
rule [cf. Winograd 1968, p. 46]). of the system.
Meaning in a systemic grammar can be encoded In 1973 J. Arveiller developed a program called
as a system of semantic options. In other words, SIM-SIM for the simulation of jazz improvisation
grammatical options are the realization of seman- (Arveiller, Battier, and Englert 1976). D. Baggi, at
tic options. In Winograd's Lisp program, semantic the University of California, Berkeley, wrote a se-
procedures were used to guide the parsing of the ries of programs (mostly in Lisp) for realizing the
music. This enabled the analysis program to avoid unfigured bass, and producing a complete score on a
many ungrammatical parsing paths; it also elimi- plotter (Buxton 1977).
nated ambiguous paths that might be grammatical One of the first uses of formal grammars for mod-
but not very meaningful in terms of an overall eling musical processes on computers was Lidov
analysis of a piece. and Gabura's study on melody writing (1973). In
1974, G. Rader published a paper that demonstrated
the use of stochastic grammars for the generative
Generative Modeling of Music modeling of rounds. Rader limited himself to very
simple music and used two grammars augmented
Generative modeling of music can be distinguished by various heuristic procedures: one for chord har-
from algorithmic composition on the basis of dif- mony and one for melody. The work of Lindblom
ferent goals. While algorithmic composition aims and Sundberg (1970) also fits into this category. In
at an aesthetically satisfying new composition, gen- 1977 W. Ulrich presented a paper to the Interna-
erative modeling of music is a means of proposing tional Joint Conference on Artificial Intelligence
and verifying a theory of an extant corpus of com- entitled "The Analysis and Synthesis of Jazz by
positions or the competence (Chomsky 1965) that Computer" (Ulrich 1977).
generated them.
Hiller's earliest work involved Markov-chain
simulations of traditional music, and in his excel-Systems for Recognition, Analysis, and
Understanding of Music
lent historical essay (1970) he reports on other early
attempts along those lines, including studies by P.
Barbaud toward generating tonal music and R. The early efforts of Simon, Sumner, Winograd, and
Zaripov's early work on modeling folk tunes. Moorer represent the first efforts to combine AI
concepts and techniques (termed, somewhat loose-
S. Smoliar (1971) developed a language called EU-

16 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
Fig. 2. Winograd's applica-
tion for music of Halli-
day's notation.

Simple
SComplete Simple
Type l Modulating
Implied Neapolitan
Altered
- Relative root Alteredtonic
Tonality

-- Unaltered - I
SMode:*K* = major:-II
s Root - III

IV

-V

VI

of Sonology
ly, heuristic programming) with musical applica-in Utrecht, started work
tions. In the early and mid-1970s aof number of pro-
music cognition based on informat
jects aimed at automatic recognition of music
psychology and generative grammar m
began to spring up; a number of them opedwill by A. be Newell,
cited H. Simon, G. Mi
here. Chomsky. In 1972 and 1973 Laske arri
A. Ashton (1971) and P. Knowlton ory (1972)
of what reported
a recognizer of music wo
on a minicomputer-based system in thatorder allowed
to model one the listening proce
to play keyboard music and have the 1975). This design included a sketch
computer
analyzer it
"remember" one's performance, displaying thatin would extract a set of s
common musical notation on a graphics and then display.
perform a sonological analys
mine possible
(This computerized effort was antedated by F. En- musical functions of t
object mechanical
gramelle's design of a clavier-oriented extracted. (Sonology is the rela
music-transcription system two and tween
one-halfthe functions
cen- of sounds in com
turies earlier [Leichtentritt 1934]).their
In 1774 acoustic
the representation.) Syntac
Berlin mechanician Hohlfeld actually succeeded
tic analysis would in then be called upon
object and
building a device capable of transcribing its possible functions into
performed
music into a piano-roll notation. structure.
This piano-roll no-
tation was then simply converted In intothe common
second part of Laske's 1975 article he de
musical notation.) A host of other fines the notions
systems of a musical robot and musical
have
been developed for computerized musicintelligence. A musical robot contains a sensory
transcrip-
tion, notably the Xerox PARC music/animation
pattern-recognition part, a particular grammar for
system described by T. Kaehler (1975).
music, and a general problem-solving part. This def-
Going beyond the idea of detecting inition
and corresponds
display- more or less to the state-of-the-
art model for
ing what is essential performance-control intelligent systems at the time. Of
informa-
tion (in contradistinction to audio special note however,iswas Laske's definition of mu-
information)
the notion of modeling the listening sical process
intelligence:with
"a robota is intelligent to the
computer. degree that it is capable of replacing a search
In the early 1970s O. Laske, then through
at theallInstitute
possibilities by plans developed on the

Roads 17

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
basis of musical-grammatical constraints" (Laske used to isolate the notes into distinct melody lines.
1975, p'art 2, p. 71). In other words, a music-recogni- First, islands of unambiguous melodic links were
tion system is intelligent to the degree that it distinguished (e.g., a very high line and a very low
replaces brute-force searching by knowledge-based line); then a search process was invoked that traced
understanding. In 1973 Laske reported on a project a reasonable melodic path. Next, the program took
called OBSERVER (Laske 1973a). The programming its results and prepared a script for L. Smith's mu-
in this project was carried out by Barry Truax. sic-manuscripting program. The output was thus a
Laske's other 1973 article (1973b) is a much more computer-typeset musical score (see Moorer's 1975
comprehensive treatment of one part of this sys- paper for an example).
tem, OBSPER. OBSPER was the fifth pass of the A more modest (and practical) system was de-
entire OBSERVER design. It was the function of scribed by N. Rowe in his 1975 paper, "Machine
OBSPER to characterize an input stream (sound Perception of Musical Rhythm." This system for in-
converted to symbolic form) in terms of a known telligent perception was implemented at the M.I.T.
"type."/ Experimental Music Studio. It enables a musician
Another interesting project from around this to play freely on an organ keyboard while the ma-
time was developed by an undergraduate student at chine infers a meter, its tempo, and note durations.
M.I.T. in 1973, J. Entwisle. Entwisle designed a An evaluation of the rhythmic "grammaticalness"
system incorporating a video camera, enabling a is also performed. The program, called HEAR,
computer to read visually a musical score written gathers events into tempo frames that Rowe lik-
in common musical notation (Entwisle 1973). ened to Minsky's concept of frames in Al (Minsky
A signal event in the computer music literature 1974).
was the publication of J. A. Moorer's dissertation in
1975. Briefly, Moorer's programs accepted continu-
ous digitized musical sound input and processed it Current Research into Al and Music
so as to isolate individual notes and rhythms in
two-part music. Moorer used band-pass filtering to Current research is taking a number of different di-
extract the individual harmonics of each instru- rections, many of which are covered in these
ment, after which AI methods (in particular, fairly special issues of Computer Music Journal. As Bo
elaborate list processing) were used to infer the cor- Alphonce and John Rothgeb emphasize, many proj-
rect musical transcription. ects in music analysis are in need of a music theory
These techniques warrant discussion in a little comprehensive enough to account for the wide
more detail. Moorer's automated solfeggio pro- range of musical behavior found in actual scores.
grams inferred notes by accumulating groups of Stephen Smoliar feels that the structure of AI lan-
harmonics without combinatorial searching (i.e., byguages like Lisp can provide a useful analogy to
Laske's definition, cited previously, the inference certain musical structures and furthermore that the
was done intelligently). By maintaining lists of oc- interactive and interpretive programming environ-
currences of various frequencies and eliminating ment so familiar to Al researchers could be a boon
redundant lists (harmonics of already occurring to music theorists. James Meehan puts forth the
notes), a list of regions of particular periodicities proposal that some major features of music could
(frequencies) was obtained. Values in this list were be characterized in terms of a Conceptual Depen-
used to set a filter that was scanned over each re- dency formalism, akin to Roger Schank's AI model
gion. This filter output indicated strong frequen- for natural language understanding. John Rahn's ar-
cies. This frequency data, together with data about ticle performs the important function of com-
their durations and amplitudes, was linked into an- paratively synthesizing Rothgeb's, Smoliar's, and
other list structure. At this point, intermediate- Meehan's papers. Computers offer a wide range of
level routines inferred which notes were present in representational and operational alternatives, and it
the music. Then further heuristic techniques were seems unlikely that any one system will be able to

18 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
incorporate all the important ideas of AI as applied
to music. Otto Laske's paper emphasizes human
cognitive activity as a major and indeed central
component of music theory. Marvin Minsky, in the
interview in this issue, suggests that the recent AI
scheme of a society-of-mind might be the best way
to model many musical and cognitive processes
effectively.
John Strawn has taken techniques from the disci-
pline of pattern recognition and has applied them
to the important tasks of data reduction (while
preserving essential features) in the analysis and
additive synthesis of sound. His technique finds
applications in a variety of musical situations, par-
ticularly where the power of additive synthesis is
required. Patrick Greussay has drawn from the AI
and computer science literature to provide general
ways of characterizing computer music processes
including improvisatory, interpretive, and algorith-
mic composition situations. Finally, Fry has devel-
oped a new minicomputer-based model for certain
forms of jazz improvisation that generates both
scores and high-quality sound output. While not
purporting to be a cognitive theory of what human
musicians do, it does bring into the open the dif-
ferent dimensions and levels of organization re-
quired for modeling even the more understood mu-
sical forms.
Further Directions for Al and Music
It is clear that antecedents to present-day interest
in AI and music have existed. But what directions
might this interest take in the near future? The
next few paragraphs attempt to account for these
directions, based on known proposals for future
work.
Representations for Music
As the cover of this issue demonstrates, Lisp may
be used effectively for interactive, graphics-based
musical activity. William Kornfeld (author of this
issue's "Machine Tongues" feature) has designed a musical structure in C. P. E. Bach's compositions.
graphic score editor, currently being implemented
This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
on the Lisp Machine. Other related projects at the
M.I.T. AI Laboratory include a new approach to the
generative modeling of music (in particular, jazz
compositions) by D. Levitt. Levitt is basing his de-
sign on a multilevel system of constraints in an
effort to build a common conceptual framework for
generation and analysis (Steele and Sussman 1978).
For sound output, a small digital music box has
been constructed and attached to a clavier and the
Lisp Machine. A project proposed by John Amuedo
involves automatically constructing concise de-
scriptions of the pitch/time structure of traditional
music. A second objective of this research is to ex-
pand these multiple, concise descriptions back into
common musical notation for realization. A key AI
concept informing this work is the notion of multi-
ple representations for music. Marvin Minsky's
participation in this project will be to apply his K-
lines (1979) and Frame theories (1974) to suggest
how musical descriptions may be recorded as "par-
tial mental states" and recalled to create musical
expectations.
Automatic Music-Transcription
Turning toward systems capable of perceiving mu-
sic, Piszczalski and Galler (1977) have developed a
minicomputer-based automatic music-transcription
system that uses successive (FFTs) to capture the
spectral content of acoustic musical instruments.
These spectral components may be plotted over
time or they may be passed to a program for pitch
detection. This pitch detection is linked with a pro-
gram that combines pitch and time information
into data structures for driving a music-plotting
program. Thus Piszczalski and Galler's system goes
completely from sound to score using a 16-bit
minicomputer.
Generative Theories
James Snell has been working for several years on a
project aimed at modeling the main principles of
Snell's goal is a program capable of taking a concise
Roads 19
semantic description (worked out by hand) of the the musical style, timing, instruments, and loud-
essential musical ideas and plans of specific pieces ness of the performance. A number of interesting
and having the program recompose the works ac- results might be obtained, including a better link-
cording to a musical grammar. age between signal processing and symbolic
In Snell's adaptation of Schenkerian theory, the processing, a more precise language for describing
composition is represented using a tree diagram, the timbral and event structures (morphologies) of
every furcation of which is a case under 12 genera- sound objects, and new notation systems for music.
tive rules, each having several parameters. The Automatic learning would be required for identify-
representation of each note includes, besides pitch ing style and performance characteristics of a piece.
and duration, harmonic status, local contrapuntal This project would draw from T. Mitchell's version
function, associations with inner voices, metric space concept-learning theory (1978). In this theory,
stress, relation to parallel motives, and position concepts are described by patterns that state the
within a rhythmic group. properties common to instances of the concept.
Aiming toward a somewhat different goal, Ste- Version spaces summarize information on the iden-
ven Haflich of the Yale School of Music is im- tity of the concept to be learned without relying on
plementing a computer-based theory that models extensive search techniques.
the competence (Chomsky 1965) manifest in the
musical structure of classical-era compositions, in
particular the piano sonatas of Mozart. Rather thanCognitive Science and Music
encode a given piece into a theoretical description
and then see if that description does indeed gen- A new effort aimed at a cognitive-science approach
erate that piece, Haflich's program will produce to certain musical areas is under way at the Center
myriad fragments of music that exemplify the char- for Music Experiment at the University of Califor-
acteristics of the tonal common-practice literature.nia, San Diego (UCSD) in La Jolla. One project,
undertaken by D. Deutsch and using computer
sound synthesis facilities on a large VAX computer
Grammars for Composition system, revolves around testing listeners' memory
capabilities for tones of complex spectral composi-
In the domain of original composition, computer- tion. Another project, headed by the cognitive
assisted systems based around various extended scientists D. Norman and D. Rumelhart, is based
forms of grammars have been designed by me on the view of music as a rich knowledge domain.
(Roads 1978), Holtzman (1979), and Jones (1980). Both linguistic and nonlinguistic schemes may be
Further work on compositional semantics and brought to bear on the problem of representing mu-
knowledge representations will greatly enhance thesic and music cognition. Another project will be
power and flexibility of these systems. undertaken using graphic representations for musi-
cal qualities such as timbre and phrasing based on
multidimensional perceptual space theories.
Intelligent Sound Analysis

A project that has been proposed at Stanford con- Intelligent Microprocessor Networks
centrates on intelligent analysis of musical sound
by computer. Using AI techniques, such a systemTaking a more direct, ad hoc approach to questions
would be capable of accepting a digitized perfor- in AI and music, a number of musicians, compos-
mance of a piece of music and building a coherent ers, and technicians affiliated with Mills College in
knowledge representation. This representation Oakland, California are developing a number of in-
would include not only information about what teresting musical contexts. Significant performance
score events were present, but also information onproblems are being confronted in a range of experi-

20 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
mental projects involving clusters of intercom- A main emphasis in AI today is on systems that
municating microprocessors. Each performance are driven by encoded knowledge. Clearly this kind
situation often requires a reprogramming and a new of system can only be taken as far as the (formal)
kind of interconnection among the machines. Mes- knowledge base available. As any music theorist
sages passed between the microprocessors include would attest, limits of formal musical knowledge
raw interrupts, acoustic signals, and symbolic pack- are readily apparent. Even if the base were avail-
ages of information concerning the current and able, effective representation schemes are still
proposed musical state. being explored. One key to overcoming the problem
An offshoot of this group is the League of Auto- of knowledge encoding is a machine capable of
matic Music Composers based in Berkeley, Califor- learning. But much work remains to be done in au-
nia. Their approach to artificial musical intel- tomatic learning theory.
ligence is broadly based on a cybernetic theory of A more fundamental set of limitations is inher-
mental activity that involves an aggregate of inter-ent in today's technology. Certain AI tasks of
limited scope are being accomplished today, but
acting parts. In this sense it resembles very closely
the current AI paradigm of a society of commu- others are bumping up against realities that will
nicating experts (Steels 1979). not be overcome in the near future. Simply stated,
today's hardware is not nearly powerful enough for
many AI applications. In his provocative essay
Musical Problem Solving Using a Society of Experts "The Role of Raw Power in Intelligence" (1976),
Hans Moravec of the Stanford Al Laboratory under-
L. Steels (1979) has described several instances in took a comparative analysis of the information-
which such AI notions as frames and communicat- processing capabilities of naturally intelligent
ing experts (programmed) can be applied to musical beings (squids, octopuses, dolphins, whales, ele-
situations. Whether Steels's notion of a conceptual phants, human beings) versus artificially intelligent
grammar using experts, Mitchell's version spaces, systems (computers). The gist of Moravec's analysis
Sussman's constraints, Schank's conceptual-depen- is that the information-processing power of an
dency networks, extended grammars, or a schemeadult human being surpasses that of a contempo-
perhaps drawing from Bobrow and Winograd's KRL rary computing system by a factor of one million.
will prove most effective as a music representation Thus it should be no surprise that the computa-
cannot be known at this time. It is likely that, as tionally expensive techniques like searching and
Halliday (1973) stresses, different representations data-base updating are major limitations to how far
may be appropriate for different functions they are AI can be taken at present. Thus although there are
to perform. Which musical features is one attempt- known brute-force solutions to many AI problems,
ing to characterize? What will one do with the current machinery makes their implementation
characterization? Is the implementation itself impractical.
effective on a particular machine? These are the For the present, reaching intermediate goals will
kinds of questions prompted by the application ofbe important to progress toward a larger goal. One
these newer AI concepts to music. might work with only simple forms of music, for
example, in developing representational schemes.
Another approach is to work toward a partial under-
Limitations standing of a more complex piece of music.
Analysis systems can be programmed to recover
One cannot expect miracles from this new surge of certain types of objects and relationships, while
interest in applying AI concepts and techniques to other more subtle (and more costly to ferret out)
music, just as one does not expect miracles from structures have to be ignored. The same applies for
similar endeavors in the most heavily researched AIlgenerative programs. Several such systems might be
field, natural language processing. devised that, instead of performing one comprehen-

Roads 21

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
sive analysis, perform a number of limited analyses been developed recently (Bobrow and Winograd
that may be interpreted by another program. 1977; 1979). This language, called KRL, incorpo-
rates such features as prototypes (similar to frames),
processing agendas (for hierarchical scheduling of
Prospects processes), flexible pattern-matching, variable
depth of processing, procedural attachment (traps
Current limitations are only one side of the story, and triggers), declarative knowledge (object repre-
however. The other side of the story is the steady sentations), and multiple descriptions (perspec-
rise in processing capability, the introduction of tives). To deal with a distributed multiprocessing
better software tools, and the development of richer environment, concurrent language extensions are
theoretical constructs. increasingly coming into play in Al applications
Prospects for computer hardware indicate higher (Kornfeld 1980).
and higher levels of integration and increasing Recent theoretical models for AI processing have
speed for integrated circuits. R. Noyce (1979) of In- stressed an underlying implementation built on
tel has conservatively predicted chips with 250,000 layers of distributed processors. An example of this
gates by 1991. Intel this year announced a micro- kind of system is Hewitt's Actor formalism model
processor chip set (the 432) with 140,000 gates per of computation (1976) and the Apiary (1979), which
chip. The massive memory requirements of truly subsumes a large number of interconnected VLSI
intelligent systems may be partially supplied by processing elements. The lexically scoped dialect of
chips like Bell Labs's experimental 11.5 Mbit bub- Lisp, called SCHEME, is also receiving a great deal
ble-memory device (Sheils, et al. 1980). Excluding of attention (Steele 1978).
such high-speed (< 10p-sec gate delays) but proba-
bly expensive technology like Josephson junction
circuits, the trend in increasing processing power Mechanical Performance
will depend more and more on distributed, multi-
processing computing systems. Both Noyce (1979) This discussion of the application of AI in the mu-
and Moravec (1976) predict that high-speed micro- sical world would not be complete without the
processors will become building blocks in compu- addressing of some societal implications. Thus
ter systems, much like flip-flops were in the 1960s. some brief reflections on the subject may be useful.
One major trend in very large-scale integration This is not the first time that the music world
(VLSI) today is precisely that of fabricating arrays of has confronted a technological challenge; as men-
microprocessors in a single package. tioned earlier, automated carillons were introduced
Software tools for AI are also being developed in the 1200s. More recently, the advent of the pho-
rapidly. One example is the Lisp Machine environ- nograph and the tape recorder have been introduced
ment, with a window-and-menu-oriented graphic into many situations where live musicians might
operating-system interface, a local network, and a have been employed. Present-day society already
sophisticated editing and debugging system. The lives with automated Muzak, lip-synced television
Smalltalk environment, running on the Alto or performances, and electronic instrument simula-
Note-taker machines at Xerox PARC, is another ex- tors (e.g., string synthesizers) quite apart from the
ample of a powerful programming environment use of any digital techniques. Moreover, the severe
(Ingalls 1977; Schoch 1979). A local network is also idiomatic constraints of the formula in most popu-
integral to these systems. As with the Lisp Ma- lar music have almost eliminated creative composi-
chine, the primary user interface to the Smalltalk tion, since the compositional rule structure and
system is a high-resolution bit-mapped display. A sound lexicon has become ossified. Thus one
powerful language specifically designed for repre- needn't point at digital instruments to find evi-
senting knowledge in a variety of ways has also dence of degraded musical quality. The use of

22 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
traditional string sections as "sweetener" in pop Conclusion
music is but one example.
Given intrinsically mechanical performance sit- The application of AI techniques to computer mu-
uations such as these, one can hardly be optimistic sic offers a number of benefits: intelligent instru-
about the introduction of more powerful musical ments (Mathews and Abbott 1980); deeper, multi-
tools into their midst. faceted representations for scores and sounds;
A central question revolves around the notion of intelligent musical data bases; singing and talking
mechanical performance itself. A popular miscon- input with singing and talking output; a better un-
ception is that whatever human beings do spon- derstanding of human musical cognition and
taneously is animated, while whatever machines do musical universals; new musical machines with ca-
is mechanical. Anyone who has had to sit throughpabilities
a beyond those of a single performer; more
lame jam session knows better. Improvisation can intelligent sound-analysis systems; performance
produce sublime musical moments, but it is also systems capable of intelligent response to musical
true that even the most spontaneously conceived sound; and new and interesting compositional rule
performance situation can be a license for some structures.

musicians to regress into hack idioms. Of course In every situation into which the concept of
my point is not to attack improvisation per se, sical intelligence is placed, new musical possi
rather, I feel it is important to recognize the ities become manifest. Results from the field of
robotlike aspects of any behavior, whether human should feed into research in musical intellige
or nonhuman in origin. Machines have no monop- and its applications, but musical work should
oly on mechanical musical performance. dependent on such results. Part of developing t
For those musicians who are interested in pro- ideas involves simply orienting current system
gramming musically intelligent systems, an inter- along these lines; the application of present tec
esting challenge will be to escape mechanistic niques could go a long way if they were applied
performance and achieve animated behavior. Ani- carefully and systematically.
mated musical behavior, as we know, is not simply
complicated in the sense of being maximally ran-
dom. It involves a multitiered memory-capable ofAcknowledgment
recalling or avoiding past events, a shifting focus of
attention, an adaptive sense of context, a knowl- I would like to thank Steven Haflich, Roger Ha
edge of boundaries and proportions, the ability to and David Levitt for their consultation on part
coordinate and synchronize events, and a sen- this paper.
sitivity to the task of transforming something
while maintaining its essential identity. A detailed
study of animated musical behavior would be an es-
say in itself. References
The key questions concerning AI and music in
society are traditional ones. For whom and to what d'Arezzo, G. 1978. "Micrologus." In Hucbald, Guido, and
end is the technology being used? Is it being used to John on Music, translated by W. Babb. New Haven: Yale
University Press.
enrich, advance, or extend the musical arts, or is it
Arveiller, J.; Battier, M.; and Englert, G. 1976. "Arveiller,
merely being used to provide a synthetic substitute
Jacques." In Un repertoire d'informatique musicale.
in a stereotyped musical situation? Technological
Paris: Universit6 Paris VIII, pp. 3-4.
change appears inevitable; if somehow the technol-Ashton, A. 1971. "Electronics, Music, and Computers."
ogy could be left exclusively in the hands of crea- UTEC-CSc-71-117. Salt Lake City: Computer Science
tive musicians and artists there would be much less Department, University of Utah.
cause for concern. Barr, A., and Feigenbaum, E. 1979. Handbook of Artificial

Roads 23

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
 Intelligence, 3 vols. Stanford: Stanford University Com- ligence." author's manuscript. Cambridge, Mas-
puter Science Department. sachusetts: M.I.T. AI Laboratory.
Bobrow, D., and Winograd, T. 1977. "An Overview of KRL: Laske, 0. 1973a. "Toward a Musical Intelligence System:
a Knowledge Representation Language." Cognitive Sci- OBSERVER." Numus- West 4(73): 11-16.
ence 1(1):3-46. Laske, 0. 1973b. "Musical Semantics: A Procedural Point
Bobrow, D., and Winograd, T. 1979. "KRL: Another Per- of View." In Proceedings International Conference on
spective." Cognitive Science 3(1): 29-42. the Semiotics of Music. Belgrade:
Bowles, E. 1970. "Musicke's Handmaiden: or Technology Laske, O. 1975. "Introduction to a Generative Theory of
in the Service of the Arts." In The Computer and Mu- Music." Sonological Reports No. 1b. Utrecht: Institute
sic, ed. H. Lincoln. Ithaca: Cornell University Press. of Sonology.
Buchner, A. 1978. Mechanical Musical Instruments,- Leichtentritt, H. 1934. "Mechanical Music in Olden
trans. by I. Urwin. Westport: Greenwood Press. Times." Musical Quarterly 20: 15-26.
Buxton, W., ed. 1977. "Baggi, D. L." In Computer Music Lidov, D., and Gabura, J. 1973. "A Melody Writing Al-
1976177: A Directory of Current Work. Ottawa: Cana- gorithm Using a Formal Language Model." Computer
dian Commission for UNESCO, pp. 105-106. Studies in the Humanities 4(3/4): 138-148.
Chomsky, N. 1965. Aspects of the Theory of Syntax. Lindblom, B., and Sundberg, J. 1970. "Towards a Genera-
Cambridge, Massachusetts: The MIT Press. tive Theory of Melody." Swedish Journal of Musicology
Entwisle, J. 1973. "Visual Perception and Analysis of Mu- 52:77-88.
sical Scores." Bachelor of Science thesis, Department of Mathews, M., and Abbott, C. 1980. "The Sequenti
Humanities, M.I.T. Drum." Computer Music Journal (to appear).
Forte, A. 1967. "Computer-implemented Analysis of Mu- McCarthy, J., et al. 1978. "Recent Research in Arti
sical Structure." In Computer Applications in Music, Intelligence and Foundations of Programming," ed
ed. G. Lefkoff. Morgantown, West Virginia: West Vir- Earnest. Stanford AI Laboratory Memo AIM-321.
ginia University. ford: Stanford University AI Laboratory.
Halliday, M. 1973. Explorations in the Function of Lan- Minsky, M. 1974. "A Framework for Representing
guage. London: Edward Arnold. edge." AI Memo No. 306. Cambridge, Massachus
Hewitt, C. 1976. "Viewing Control Structures as Patterns M.I.T.
of Passing Messages." AI Memo 410. Cambridge, Mas- Minsky, M. 1979. "K-lines: A Theory of Memory." Al
sachusetts: M.I.T. Memo No. 516. Cambridge, Massachusetts: M.I.T.
Hewitt, C. 1979. "The Apiary." lecture at Stanford Uni- Mitchell, T. 1978. "Version Spaces: An Approach to Con-
versity. December. cept Learning." STAN-CS-78-711. Stanford: Stanford
Hiller, L. 1970. "Music Composed with Computers: A University Computer Science Department.
Historical Survey." In The Computer and Music, ed. H.Moorer, J. A. 1972. "Music and Computer Composition."
Lincoln. Ithaca: Cornell University Press, pp. 42-96. In Communications of the Association for Computing
Holtzman, S. 1979. "GGDL-Generative Grammar Defi- Machinery 15(2, February): 104-113.
nitional Language." Edinburgh: University of EdinburghMoorer, J. A. 1975. "On the Segmentation and Analysis of
Department of Computer Science. Continuous Musical Sound." STAN-M-3. Palo Alto:
Ingalls, D. 1977. "The Smalltalk-76 Programming System: Stanford University Department of Music.
Design and Implementation." Palo Alto, California: Xe-Moravec, H. 1976. "The Role of Raw Power of Intel-
rox PARC. ligence." Stanford: Stanford University AI Laboratory.
Jones, K. 1980. "Stochastic Processes in Computer Mu- Nilsson, N. 1980. Principles of Artificial Intelligence.
sic." Proceedings: Computer Music in Britain. London: Palo Alto: Tioga Publishing Co.
Electro-acoustic Music Association, pp. 73-93. Noyce, N. 1979. "Hardware Prospects and Limitations."
Kaehler, T. 1975. "Some Music at Xerox PARC." In Pro- In The Computer Age: A Twenty-year View, ed. M.
ceedings of the Second Annual Music Computation Dertouzos and J. Moses. Cambridge, Massachusetts:
Conference, ed. J. Beauchamp and J. Melby. Urbana: The MIT Press, pp. 321-337.
Office of Continuing Education and Public Service inOrd-Hume, A. 1977. "2000 Years of Recorded Music."
Music, pp. 53-57. Studio Sound 19(6):52-54.
Knowlton, P. 1972. "Capture and Display of Keyboard Piszczalski, M. and Galler, B. 1977. "Automatic Music
Music." Datamation, May. Transcription." Computer Music Journal 1(4): 24-31.
Kornfeld, B. 1980. "Parallel Languages and Artificial Intel-
Rader, G. 1974. "A Method for Composing Simple Tradi-

24 Computer Music Journal

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms
 tional Music by Computer." Communications of the Smoliar, S. 1971. "A Parallel Processing Model of Musical
Association for Computing Machinery 17(11): 631- Structures." Report MAC TR-91. Cambridge, Mas-
638. sachusetts: M.I.T.
Roads, C. 1978. "Composing Grammars." In Proceedings Steele, G., and Sussman, G. 1978. "Constraints." AI
of the 1977 International Computer Music Conference. Memo No. 502. Cambridge, Massachusetts: M.I.T.
San Francisco: Computer Music Association. Steele, G. 1978. "Rabbit: A Compiler for SCHEME." Al-
Rowe, N. 1975. "Machine Perception of Musical TR-474. Cambridge, Massachusetts: M.I.T.
Rhythms." Bachelor of Science thesis, M.I.T. Steels, L. 1979. "Reasoning Modeled as a Society of Com-
Schank, R.; Lebowitz, M.; and Birnbaum, L. 1980. "An In- municating Experts." AI-TR-542. Cambridge, Mas-
tegrated Understander." American Journal of Computa- sachusetts: M.I.T.
tional Linguistics 6(1): 13-30. Ulrich, W. 1977. "The Analysis and Synthesis of Jazz by
Scheils, M., et al. 1980. "Machines that Think." News- Computer." Proceedings of the Fifth International Joint
week 45(26, 30 June): 50-56. Conference on Artificial Intelligence. Pittsburgh: Car-
Schoch, J. 1979. "An Overview of the Programming Lan- negie-Mellon University Department of Computer
guage Smalltalk-72." SIGPLAN Notices 14(9): 64-73. Science, pp. 865-872.
Simon, E. 1960. Mechanische Musikinstrumente friiherer Winograd, T. 1968. "Linguistics and the Computer Analy-
Zeiten und ihre Musik. Wiesbaden: Breitkopf und sis of Tonal Harmony." Journal of Music Theory
Hartel. 12(Spring): 2-49.
Simon, H., and Sumner, R. 1968. "Pattern in Music." In Winston, P. 1977. Artificial Intelligence. Reading, Mas-
Formal Representation of Human Judgement, ed. B. sachusetts: Addision-Wesley.
Kleinmutz. New York: John Wiley and Son, pp. 219- Winston, P., and Brown, R. 1979. Artificial Intelligence:
250. an M.I. T Perspective, 3 vols. Cambridge, Mas-
Simon, H. 1969. The Sciences of the Artificial. sachusetts: The MIT Press.
Cambridge, Massachusetts: The MIT Press. Woods, W. 1980. "Cascaded ATN Grammars." American
Slawson, A. W. 1968. "Review of Computer Applications Journal of Computational Linguistics 6(1): 1-12.
in Music." Journal of Music Theory 12(1): 105-111.

Roads 25

This content downloaded from 67.184.195.145 on Fri, 14 Jul 2017 00:22:47 UTC
All use subject to http://about.jstor.org/terms