Leonardo

The Markov Process as a Compositional Model: A Survey and Tutorial

Author(s): Charles Ames
Source: Leonardo, Vol. 22, No. 2 (1989), pp. 175-187
Published by: The MIT Press
Stable URL: http://www.jstor.org/stable/1575226 .
Accessed: 18/12/2014 21:12

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

The MIT Press and Leonardo are collaborating with JSTOR to digitize, preserve and extend access to
Leonardo.

http://www.jstor.org

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
 ABSTRACT

The Markov Process as a

Theauthor combines a survey
ofMarkov-based effortsinauto-
Compositional Model: matedcomposition witha tutorial
howvarious
demonstrating theo-
reticalproperties associated with
A Survey and Tutorial Markov processescanbeputto
practical use.Thehistorical back-
ground is tracedfromA.A.
Markov's formulation
original
CharlesAmes through tothepresent. Adigres-
sionintoMarkov-chain theory intro-
duces'waiting counts'and'station-
Theauthor's
aryprobabilities'.
Demonstration 4 forsoloclarinet
illustrateshowtheseproperties
affectthebehavior ofa melody
composed usingMarkov chains.
I n my previous Leonardo article, "Automated well Markovprocesses compare Thissimpleexample becomesa
Composition in Retrospect: 1956-1986", I described a va- to alternative methods in at- pointofdeparture forincreasingly
riety of approaches that have been used over the past 30 general ofthe
interpretations
taining these objectives. Markov process.Theinterpretation
years to generate musical compositions using computer of'states'is reevaluatedinthelight
programs [1]. One approach I mentioned only in passing ofrecentmusical effortsthatem-
was the Markov process, which I judged at the time to be SOME ployMarkov chainsofhigher-level
only of peripheral interest to users of composing programs. DEFINITIONS objectsandinthelightofother
Since then, several developments have shown me that the effortsthatincorporate relative
i,, c ;,in, tho
.r;ihi ,;l,nr;,,,
nc,i,
enthusiasm toward Markov processes is much greater than A Markov chain models the be- aLrIouuteiiiu .ii PuSSIlulcIlltl-
pretations.
Other
effortsexpand
I had estimated. Undoubtedly the strongest indication of havior of a sequence of events, Markov's definition
to em-
original
misjudgement on my part has been the phenomenal re- each of which can assume any brace'Nth-order' evolv-
transitions,
sponse to M and Jam Factoryover the past 2 years-the first one of a fixed range of states. ingtransition
matrices
andchains
time any effort at automated composition has met with the Changes in state between con- ofchains.Theremainder
article
ofthis
slightest commercial success-but these important pro- secutive events commonly are
secutive
events . contrasts
. Markovprocesses
common.ly withalternative
compositional
grams are only part of a general resurgence of interest in referred to as transitions. It is strategies.
Markovchains over the past decade. Several new approaches convenient to designate the
toward Markov chains have come to my attention since I state of the chain prior to a
wrote my retrospective, and I even found myself personally transition as the transition's
involved with them in 1986 when John Myhill hired me to source(equivalently, the chain's
implement some Markov processes that could gradually currentstate)and to designate the outcome as the transition's
evolve over the course of a musical work he was composing destination(the chain's future state). Consecutive transitions
[2]. link up so that the destination for one transition serves as
I choose to integrate the present survey of musical appli- the source for the next transition. The range of a Markov
cations with a tutorial because there are some significant chain is, appropriately, the collection of all possible states.
theoretical ramifications associated with Markov chains. As For musical applications, we can assume that this collection
with all mathematical formalisms, the intrinsic logic behind will alwaysbe finite.
Markov processes remains valid no matter what real-world In Markov'smodel, the behavior of a chain is distilled into
meaning the symbols (in this case the 'states' of the chain) a set of numbers called transitionprobabilities.
Each transition
might have. Indeed, the matrix representation devised by probability indicates the relative likelihood that the chain
mathematicians to describe how Markov chains behave will jump to a particular destination, assuming that it cur-
lends a deceptive simplicity; Markovprocesses can be shown rently resides at a particular source. Mathematicians conven-
to include many seemingly more complex methodologies as tionally organize these numbers into a transitionmatrix(or,
special cases. Although the mathematics is too formidable more formally, a matrixof transitionprobabilities).All transi-
to elaborate in a tutorial of this nature, a few examples will tion matrices have the form depicted in Matrix 1.
suffice to demonstrate how one can deduce the behavior of
a chain from its matrix [3]. The meanings associated with
each state of the chain depend, of course, on the applica- BACKGROUND
tion, and for examples of how Markovchains can be applied Markov chains were formulated in 1906 by the mathemati-
to music it will generally be best to go directly to the sources. cian Andrei Andreevich Markov (1856-1922), who first used
A constructive evaluation of a compositional procedure his model to distill tendencies of spelling in written Russian
ultimately must take into account the music produced; from the text of Aleksandr Pushkin's Eugene Onegin. The
however, different procedures have their different strengths dependency of consecutive letters in literary texts should be
and weaknesses. A composer can estimate the appropriate- clear from such basic rules of English spelling as 'always
ness of such a procedure by considering his or her musical
objectives and by determining how effective the procedure Charles Ames (composer), 68 Stevenson Boulevard, Eggertsville, NY 14226, U.S.A.
is at realizing these objectives. The last section of this article
Received 24 May 1988.
enumerates a number of objectives that have been recurrent
This article is dedicated to the memory of John Myhill.
among users of composing programs, and it discusses how

? 1989 SAST
Pergamon Press pic. Printedin Great Britain.
0024-094X/89 $3.00+0.00 LEONARDO, Vol. 22, No. 2, pp. 175-187, 1989 175

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions

Matrix1. Generalformof an n-statematrix.Thetransition
ofjumpingfromstatei to statej.
relativelikelihood
source
state 1 state 2
state 1 Pil, P1,2
state 2 P2,1 P2,2
state n Pn,1 Pn,2
follow Qwith U and 'Ibefore E except
after C'. Markov calculated his transi-
tion probabilities by tallying pairs of
consecutive letters (if Pushkin had writ-
ten in English, Markovwould have had
to compile 26 x 26 = 676 tallies); he
then divided these tallies into groups
based on first letters (e.g. one group of
tallies for pairs of letters beginning with
A, one for pairs beginning with B, etc.;
each one of these groups fills up one
row in the transition matrix); finally, he
'normalized' his transition proba-
bilities by dividing each individual tally
by the total of the tallies in the same
group. As is evident from Markov'sorig-
inal subject matter, his model is statisti-
calin nature-the processes it describes
need not necessarily be random. How-
ever, the fact that one would resort to
such a model suggests that the underly-
ing causes and effects are not well un-
derstood. At best, the model provides
only partial descriptions of nonrandom
behavior.
Markovchains are a primary topic of
study in the mathematical discipline of
stochastic processes, where they have
been used to model random processes,
such as the changing fortunes of a gam-
bler, the inventory of a commodity as it
is affected by continuing pressures
from supply and demand, or genetic
fluctuations under random matings
and mutations. Markovchains have also
played an important role in connection
with Claude Shannon's Information
Theory.
Since Information Theory was fre-
quently cited but inadequately ex-
plained in the music-theoretic writings
of the 1950s and 1960s, it is worthwhile
to provide a brief outline here. Shan-
non's 1948 article, "The Mathematical
Theory of Communication", proposes
a formula for measuring the 'informa-
tion content' of a 'message'-that is, of
a sequence of discrete symbols such as
a literary text [4]. According to these
formulas, the least predictable mes-
sages have the greatest information
content, i.e. information decreases as
176 Ames,The Markov Process as a Compositional Model: A Survey and Tutorial
This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM
Pij givesthe
probability
destination
state 3 state n
P1,3 P1,n
P2,3 P2,n
Pn,3 Pn,n
redundancy increases. It should be
pointed out that Shannon's formula
often clashes with our intuitive sense of
musical meaning, since he considers an
unpredictable message as formally
equivalent to a random message.
(Henceforth, I will enclose the terms
'information content' and 'redun-
dancy' in single quotes when Shan-
non's interpretation applies.)
Shannon models messages as Mar-
kov chains, with one state for each
different symbol in the message. Conse-
quently, when the 'information con-
tent' is high, the transition probabilities
will all be close to 1/n, n being the size
of Shannon's 'alphabet'. Conversely,
when 'redundancy' is high, transitions
will be strongly biased toward certain
patterns of succession. Using this
model, Shannon proceeds to analyze
mathematically how transmitting such
a message through a 'noisy channel'
(such as a telegraph wire) results in loss
of information and what safeguards can
be imposed to minimize this loss.
Musical analysts during the late
1940s and 1950s recognized both Mar-
kov matrices and Information Theory
as potential tools for distilling the
norms and deviations of musical styles.
Much initial interest in applying Infor-
mation Theory to music was sparked by
Harry Olson's Markov analyses of the
songs of Stephen Foster [5] and by
Richard Pinkerton's ScientificAmerican
article [6]. Allen McHose compiled ex-
tensive statistics on contrapuntal prac-
tices in Bach's chorale harmonizations
and used these statistics to deduce 'cor-
rect' and 'incorrect' practices [7]. In
Emotionand Meaningin Music,Leonard
Meyer, who was also an early proponent
of Markov-styleanalysis,cites the "Table
of Usual Root Progressions" from Wal-
ter Piston's Harmony (first published
1941) as "nothing more than a state-
ment of the system of [conditional]
probability which we know as tonal har-
mony" [8]. Meyer's later "Meaning in
Music and Information Theory" corre-
lates this attitude directly with Shan-
All use subject to JSTOR Terms and Conditions
non's ideas [9]. Attempts by Meyer and
others to apply Information Theory to
music have been much less rigorous
than Shannon's original formulation.
These attempts typically put the listen-
er's faculties for perceiving musical re-
lationships into the role of the 'noisy
channel'. They then draw inferences
concerning how much redundancy
may be removed from a musical mes-
sage before the message begins to lose
intelligibility.
Markov processes have also been
used directly in composing programs
that have employed both 'composed'
transition probabilities and probabili-
ties obtained through analysis of musi-
cal source material. Experiment 4 of
Lejaren Hiller and Leonard Isaacson's
1957 Illiac Suitestands as the first direct
use of Markov chains to compose mu-
sic. The Illiac computer selected con-
secutive intervals for each instrument
according to the criteria of harmony
(greatest weight to most consonant in-
tervals), proximity (greatest weight to
smallest intervals) and combinations of
the two [10]. The five "Strophe"move-
ments of Hiller and Baker's 1963 Com-
puter Cantata derive transition proba-
bilities from the "Putnum's Camp"
movement of Charles Ives's ThreePlaces
in NewEnglandand impose these proba-
bilities upon each of the following mu-
sical attributes: pitches, durations, dy-
namics, notes versus rests, and playing
styles. In describing their work, Hiller
and Baker cite Shannon's notion of
'information content', which Hiller
and Baker attempt to treat as a large-
scale musical attribute analogous to
tempo, key, thickness of texture, and so
on [11].
In 1959, Iannis Xenakis wrote three
works in which he used Markov chains
to control successions of large-scale
events: AnalogiqueA for string orches-
tra, AnalogiqueB for sinusoidal sounds,
and Syrmosfor 18 strings. Xenakis's ap-
proach differs strongly from that of
Hiller and his collaborators in that Xe-
nakis's processes emulate the behavior
of many simultaneous Markov chains.
The states of Xenakis's process are con-
structs, which Xenakis calls 'screens'.
Each screen constitutes a configuration
of one or more regions of 'musical
space' [12], in which some number of
elementary 'grains' of sound may
occur.
Although they have gained little en-
thusiasm from composers who do not
rely on automated decision-making,
Markovchains have been employed re-
currently in composing programs over
 [13] and with Petr Kotik's interactive the transition matrix are called the wait-
Matrix2. Athree-statetransition transition-matrix editor [14]. The "sys- ing probabilities(or fixed-stateprobabili-
matrix. tems approach to composition" devised ties),and they give the probabilities that
by Curtis Roads [15] can be shown to consecutive events in a chain will oc-
source destination
be based on Markovian principles-al- cupy the same state. Another way of
though this is not readily apparent- looking at this is to consider the waiting
A B C and the 'harmonic algorithm' de- count, that is, the number of times a
A 0.5 0.4 0.1 scribed by Laurie Spiegel [16] is single state occurs consecutively. When
B 0.4 0.5 0.1 actually a Markov process reminiscent the waiting probability is P, then the
C 0.4 0.4 0.2 of Piston's '"Tableof Usual Root Pro- average (or 'expected') waiting count
gressions". Intentional use of Markov Wis given by the formula:
processes has been made by R. C. Zar- 1
Table1. This'systemof equations' pov [17], Tracy Lind Peterson [18], W=
1-P
estimatestheoveralldistribution KevinJones [19], Thomas DeLio [20],
of statesthatwilloccurwheneach Sever Tipei, John Myhill, the group of When the waiting probabilities ap-
of manysimultaneous chainsun- Claudio Baffioni, Francesco Guerra
dertakesone of thetransitions proach zero, the rate of change grows,
and Laura Tedeschini Lalli [21], and and the waiting counts fall to a lower
describedby Matrix2.
Darrell Conklin [22]. Capabilities for limit of one state (reflecting the origi-
K' (A)= .5 x K(A) + .4x K(B) + .4x K(C) implementing Markov processes are nal arrival at a state). W grows to in-
K' (B)= .4 x K(A)+ .5 x K(B) + .4x K(C) among the features of the Hierarchical finity as P approaches unity, with the
K' (C)= .1 x K(A)+ .1 x K(B)+ .2 x K(C) Music SpecificationLanguage (HMSL) result that a chain evolves very slowly
environment for automated composi- when its waiting probabilities are large.
tion, which is being developed by Phil The greatest strength of Markov
Table 2. Given300 simultaneous Burk, LarryPolansky and David Rosen- chains is their capability for predicting
chains, initiallydividedequally be- boom [23]. Markovprocesses also have the immediate future based upon what
tween states A, B and C, repeated a close connection with the neural net has happened in the recent past. In an
applicationsof the system of equa- modeling of music currently being un- artistic endeavor such as music-which
tions shown in Table 1 willquickly dertaken by researchers such as Jam- depends heavily upon time as an organ-
result in an equilibriumstressing shed Bharucha [24], who has pointed izational reference-prediction of ex-
states A and B over state C.
out that the output from a neural net pectation (in the psychological sense)
States analysis is a transition matrix. is clearly a fundamental dynamic of ex-
By far the most ambitious use of Mar- perience. However, expectation is by
Transitions A B C kov chains in a composing program has no means the only factor upon which
0 100 100 100 been in two programs released in 1987 listeners base aesthetic judgments;
1 130 130 40 as commercial products: M by David another important basis forjudgement
2 133 133 34 Zicarelli, Joel Chadabe, John Offen- is the balance between compositional
hartz and Antony Widoff, and Jam Fac- elements. If these compositional ele-
3 133 133 34
toryby Zicarelli [25]. Both M and Jam ments operate as states of a Markov
4 133 133 34
Factoryare real-time compositional pro- chain, then it becomes possible to cal-
5 133 133 34 cessors, exploiting the fact that Markov culate a set of numbers called stationary
chains lend themselves well to efficient probabilities.This set of numbers sum-
implementation as computer pro- marizes the relative balances between
Table3. Whenall300 chainsare grams. Both programs have the capa- elements (or states) over the long term.
concentrated
initially instateA, bility to derive transition probabilities One can appreciate both the nature
derivedinTable2
the equilibrium from user-supplied musical excerpts. of stationary probabilities and the pro-
rapidlyassertsitselfhereas well. cedure for calculating them by consid-
ering a simple case study: Imagine 300
States
WAITING COUNTS AND Markov chains all running simultane-
STATIONARY ously in lockstep. Suppose that all 300
Transitions A B C chains share the transition probabilities
0 300 0 0 PROBABILITI'ES enumerated in Matrix 2. We want to
1 150 120 30 Two critical factors contributing to the estimate the overall distribution of
states.
2 135 132 33 long-term behavior of a Markov chain
are the average waiting counts, which Assume for the moment that 100
3 134 133 33
directly affect the rate of activity in the chains reside at the outset in state A,
4 133 133 34
chain, and the stationaryprobabilities, 100 begin in state B and 100 begin in
5 133 133 34
which predict long-term balances be- state C. Consider the first synchronized
tween the various states. These two transition. Of the 100 chains originally
in state A, the matrix tells us
properties should be taken into ac- residing
the past three decades. Ironically, some count whenever one tries to deduce the that approximately 50 will remain in
elegant instances where Markov chains behavior of a chain from its transition state A, 40 will jump to state B and 10
have been incorporated have been un- matrix. willjump to state C. Similarly, 40 of the
intentional. Such is the case with the The transition probabilities along 100 original B's would become A's, 50
INTERVAL feature of Gottfried Mi- to
the top-left bottom-right diagonal of would remain B's and 10 would become
chael Koenig's PROJECT2 program

Ames,The Markov Process as a Compositional Model: A Survey and Tutorial 177

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
 Matrix3. Durational
transitions
for Matrix5. Registral inDemonstration
transitions 4.
Demonstration4.
source destination
source destination F#4-F5
E3-Eb4 A3-Ab4 C#4-C5 Bb4-A5 D5-C#6 G5-F#6
2 3 5 9 E3-Eb4 0.66 0.17 0.08 0.05 0.03 0.01
2 0.875 0.042 0.042 0.041 A3-Ab4 0.11 0.66 0.11 0.06 0.03 0.02 0.01
3 0.062 0.813 0.062 0.062 C#4-C5 0.05 0.10 0.66 0.10 0.05 0.03 0.01
5 0.104 0.104 0.688 0.104 F#4-F5 0.03 0.05 0.09 0.66 0.09 0.05 0.03
9 0.187 0.188 0.187 0.438 Bb4-A5 0.01 0.03 0.05 0.10 0.66 0.10 0.05
D5-C#6 0.01 0.02 0.03 0.06 0.11 0.66 0.11
G5-F#6 0.01 0.03 0.05 0.08 0.17 0.66
Matrix4. Transitions ar-
controlling
inDemonstration
ticulations 4.

source destination
10% 17% 29% 50%
10% 0.91 0.02 0.02 0.05
L
E3-Ep4
I A3-A64 FP4-F5 B,4-A5
I G5-FP6
I
CP4-C5 D5-CO6

17% 0.02 0.91 0.05 0.02

Fig. 3. Stationary probabilities for registers (from Matrix 5). The stationary probabilities
29% 0.05 0.02 0.91 0.02 are nonuniform even though the waiting probabilities are equal. The central concentration
50% 0.02 0.05 0.02 0.91 indicates that the transition probabilities act on the whole to direct the music toward the
middle registers.

It is a mathematical fact that no mat- choose an average duration, expressed

n | l mately will settle down to an equilib-
2 3 5 9
,.~..--
1.
FiFg. Stationary probabilities dura-
for
tioIns(fromMatrix3). The varyingwidths
of the probabilitybarsreflectthe fact that
th<e statesaffect theirowntenures.
.10 .17 .29
,
l;.g' 9._.:. ,<,q : , c,at.r,,
rig. z. oLauonaiuy pruuoaDulues Ior
tions (from Matrix 4). The stationary prob-
abilities are uniform, since the flow
between states is symmetric and the wait-
ing probabilities are equal.
C's, while 40 of the 100 original C's
would become A's, 40 would become
B's and 20 would become C's. The net
result of the first transition therefore
would be to transform 100 A's, 100 B's
and 100 C's into approximately 130 A's,
130 B's and 40 C's.
For an arbitrary transition, if K(A),
K(B) and K(C) represent the number
of chains residing in states A, B and C
prior to the transition, while K'(A),
K'(B) and K'(C) represent these num-
bers after the transition, then K'(A),
K'(B) and K'(C) may be estimated by
the system of equations listed in Table
1. Table 2 shows what happens when
these equations are applied iteratively
to an original population of 100 A's,
100 B's and 100 C's, while Table 3 shows
what happens when the process starts
with 300 A's.
ter how it starts out, the system ulti-
rium determined by the stationary
probabilities: 44% A's, 44% B's and
12% C's. In general, the stationary
probabilities can be derived from any
transition matrix by setting up an ap-
propriate system of equations and by
applying it iterativelyuntil the numbers
settle into equilibrium. Although the
procedure is tedious to undertake by
.50 hand, it is straightforward to imple-
arucuia-
ment on a computer.
COMPOSITION WITH
MARKOV CHAINS: A
TUTORIAL EXAMPLE
My composition Demonstration4 illus-
trates some of the elementary consider-
ations involved in using Markov chains
to compose a piece of music. The piece
is one of a series of 11 didactic studies I
composed in 1983 and 1984 to demon-
strate techniques of musical composi-
tion by computer [26].
The composing program consists of
one short loop: each iteration of this
loop generates either a note or a rest,
and the iterations continue until the
end of the page has been reached (60
measures of 2/4 time). There is no
beginning or end and very little 'shap-
ing' of the music; what results is a snap-
shot of a process that could go on indef-
initely. At the beginning of an iteration,
the program uses transition matrices to
in sixteenths, and an articulation, ex-
pressed as a rest/play probability. This
rest/play probability is then used in a
random branch to determine whether
a note or a rest is to be composed next.
If the program decides to compose a
note, then it chooses a note duration
randomly around the average duration
[27] and uses transition matrices to se-
lect a degree and a register. If it decides
to compose a rest, then it chooses a rest
duration randomly around half the
average duration.
Average Duration
There are four average durations avail-
able to any note. Expressed in six-
teenths, these durations are 2, 3, 5 and
9 for notes (1, 1.5, 2.5 and 4.5 for rests).
Transitions are guided by the probabili-
ties shown in Matrix 3. One should in-
terpret Matrix 3 by noticing that transi-
tions to destinations other than the
source are uniformly likely, and that
the waiting probabilities go down as the
durations go up. This happens by de-
sign. Indeed, by multiplying the aver-
age note durations by their average
waiting counts, one sees that the period
of time over which any average dura-
tion holds sway will be approximately
two measures:
2 3
=16, =16,
1 - 0.875 1 - 0.813
5 9
=16, =16.
1 - 0.688 1 - 0.436
Bear in mind that the value 16 is an

178 Ames, The Markov Process as a Compositional Model: A Survey and Tutorial

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
 .

A
expectation and that it does not reflect a
r we
the shorter average durations for rests. - t-a- t- #- ^aa#-
t-t ? '^>a a- a# mi' s-'a , t-
#-
The actual periods of time will deviate 6/46 5/46 4/46 2/46 6/46 6/46 2/46 4/46 5/46 6/46

widely, since the actual (rather than A

tf
expected) waiting count and the note 'u
_m

durations both result from random 5/26 2/26 1/26 2/26 2/26 1/26 3/26 4/26 6/26
processes. Since every state has equal
a
access to every other, the relative em- sa --
t, , 'a
phasis placed on each given state is 4/23 1/23 2/23 2/23 1/23 4/23 4/23 5/23
entirely due to the state's waiting proba-
bility (see Fig. 1). 2/ .a
2/19 2/19 4/19 3/19 4/19
Articulation 2/19 2/19

Articulation is controlled through a

percentage indicating the probability 6118 #Is- I/1
that an iteration of the composing loop 6/18 6/18 2/18
2/18 1/18 1/18 2/18
will produce a rest instead of a note. iL_
- - -
HtL-
Transitions are guided by Matrix 4. No- o~ fo -
Pa - - -
u
tice that all four waiting probabilities in 6/36 2/36 2/36 2/36 6/36 6/36 2/36 2/36 2/36 6/36
Matrix 4 are 0.91. This means that any
A
of the four articulations will hold sway 6
si-
a -01# a 'k a
for an average of 11.11 notes and rests
2/18 1/18 1/18 2/18 6/18 6/18
before the next change occurs. The
transition probabilities off the diagonal Is-
219 219
are somewhat more interesting; they
favor the following cycle: 4/19 3/19 4/19 2/19 2/19 2/19 2/19
10% -- 50% -- 17% -- 29% -> 10%.
Figure 2 shows that the stationary prob- 5/23 4/
abilities are uniform. 5/23 4/23 4/23 1/23 2/23 2/23 1/23 4/23

Register Isa

My program treats registers as one-oc- ^ 6/26

As-
4/26 3/26 1/26 2/26 2/26 1/26
2/26
5/26
tave regions within which it places a -dPl
chromatic degree. There are seven reg- II- X AP Vw
ai-
s
,
v gL? -

-- -a . . . , a -
isters: E3-E 64, A3-Ab4, C#4-C5, F#4- k - . - - - - -

646 5/46
F5, Bb4-A5, D5-C#6 and G5-F#6. 6/46 4/46 2/46 6/46 6/46 2/46 4/46 5/46 6/46

Transition probabilities are shown in

Matrix 5. Since all seven waiting prob-
abilities for Matrix 5 are 0.66, the wait-
ing count for registers will be 3. Non-
waiting transitions favor jumps to
nearby registers. The stationary proba-
bilities are graphed in Fig. 3.
Fig. 4. (Matrix 6) Chromatic matrix for Demonstration4. Each entry gives the probability as-
sociated with a pair of two rising chromatic intervals-with middle degrees fixed arbitrari-
ly at B. Any triad containing chromatic identities receives a weight of 0, as do major triads,
minor triads, augmented triads and triads with two perfect consonances (e.g. C-F-Bb ) in
any inversion or voicing. The remaining triads receive relative weights from 1 to 6, depend-
ing on dissonance; the denominator of each fraction is a 'normalizing' value used to con-
vert these weights into transition probabilities.

Chromatic Degrees
The sequence of chromatic degrees is a
first-order Markov chain of displace-
ments through the chromatic scale, me -
n~~F7r7r
I .-?--
M2 .
._
ma
I .n
M3a
I
r
I I ---
TT
r

--
r r
iF
mb M
-
Mb
7H
I

mIr
--

which 'wrap-around' at the octave. The

chromatic matrix shown in Matrix 6
(see Fig. 4) details the transition prob-
abilities associated with each pair of
consecutive displacements. To the ex-
tent that it actively excludes 'un-
desirable' sonorities, Matrix 6 reflects
the INTERVALpitch-selection feature
of Koenig's PROJECT2composing pro-
gram; however, the graduated transi-
tion probabilities in Matrix 6 also pro-
mote 'desirable' sonorities, with the
objective of encouraging a consistently
dissonant style. These biases are re-
flected by the fact that the stationary
probabilities graphed in Fig. 5 favor
Fig. 5. Stationary probabilities for chromatic intervals generated by Matrix 6. Intervals are
indicated in diatonic scale steps, with the qualifications m (minor), M (major), and P (per-
fect). TT stands for 'tritone' (an augmented fourth or diminished fifth). The emphasis
upon dissonant intervals, especially the minor second, tritone and major seventh, reflects
the emphasis upon dissonant triads in Matrix 6.
dissonant intervals such as seconds, tri- frequently, while registers change fairly
tones and sevenths. often.
Figure 7 presents the final result.
Results Floating-point durations are approxi-
Figure 6 graphs average durations, ar- mated by rounding to the nearest six-
ticulations and registers. Notice that ar- teenth note and by accumulating the
ticulations change the least rapidly due residue into the next note or rest. When
to their high waiting probabilities. Aver- the duration of a rest rounds down to
age durations change somewhat more zero, it appears as a break in the slur.

Ames, The Markov Process as a C ompositional Model: A Survey and Tutorial 179

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
 I

AVERAGE
DURATION ^
a NNNE EW

U.DU -

0.29
ARTICULATION
0.17 . m . J I _I
!ir
07,~~~~~~~~~~~~~~~~~~~~~~~~~~~~ r
0.10 I l p-

G6

REGISTER

measure I I j I 15 1 I I 10 l I I I 115 I I 1 1 120 I

AVERAGE IF~ I 1l I i I-
DURATION
_ r L-UTi I__ I
0.50

0.29
ARTICULATION
0. 17

0.10 -

G6

REGISTER I

measure I i I | 125 1 | I | 130 1 | 1 | 135 1 | 1 140

AVERAGE
DURATION
3
2 I
I I
-I1 I --I
I

0.50

0.29
ARTICULATION
0.17
0.10

G6

REGISTER

E3~

measure I I I I 145 I I 50 1 1 55 | I| I | 60

Fig. 6. Graph of average durations, articulations and registers in Demonstration4, as guided by Matrices 3-5. Rates of activity in each
graph are determined by waiting probabilities; the relatively fluid motions in the register graph are due to the fact that Matrix 5 favors
transitions between 'nearby' states, while Matrices 3 and 4 encourage abrupt contrasts.

180 Ames, The Markov Process as a Compositional Model: A Survey and Tutorial

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
WHAT IS A STATE? Clarinet
4
Demonstration CharlesAMES
STRICTLYJ = 80
Formally the states assumed by a Mar-
kov chain can be anything at all, as long A ^eo, \ ^ - - _. b v !-
as the range of states is discrete [28]. In
practice, however, the nature of the
events that constitute the chain deter-
mines the range of states. In music,
these events might be notes or higher-
r
level objects, such as chords or phrases; ~E~Pr7 I?
in each case, the state of a chain would
be a way of describing one of these
16
events. More subtle interpretations are L a_<
also useful: in this section, I will discuss III--
--L

nJ -7lL
F7
II ^ bIl II /r-r it~~~~~~~~~~~~~~
1t?w:
I'

how states can be relative positions in

finite sequences; in later sections, I will
illustrate how states can be sequences ti^
"
^C^iJL^ ^^
of states or lower-order Markov chains
in their own right.
0e~Q1tr tot'e t?
When the events are simple notes,
the most common practice has been to
let the states be either a simple 'scalar' ;W<#S' t l"/JJIr7 r
attribute (e.g. duration alone or pitch
alone) or a 'vector' describing two or
more attributes simultaneously (e.g. Sw* 1PT^
41~~~~~~
f
?~~~~~~~l^'
ft 3e t

duration, pitch, dynamic and articula-

tion as components of a note). A more I~~t~ _ ~??
fi

subtle approach is to model modesof

i^nj^n,- -
behavior by using different states to
represent alternative inflections of one- te2S -%ft
a. j."^^. I -

and-the-same attribute. For example, A I

Kotik has represented the one-and-the-
same scale degree in ascending and SL- ijS--C 4, I
i^t
descending versions: the transition
probabilities for different versions i v_
w.
; 1. ^
1
,-?n-, J a
I
I
favor motion in the indicated direction I _ _ J

but also incorporate a background like- 0~~'~r~...,

- ~_ .' -
lihood of switching direction.
A common complaint against some
^- ?T' _. sjIi- 11
early musical applications of Markov ( Charles Ames 1984
chains has been their limited vertical
Fig. 7. A composition generated by transition Matrices 3-6. This is a 'snapshot' of a
sensitivity.This was especially evident in process that could go on indefinitely. Whatever long-term shapes are present in this music
the Illiac Suiteand the ComputerCantata, are due entirely to the slower-moving processes graphed in Fig. 6.
in which the various instrumental parts
were left to go along their independent
ways with no 'awareness' of what the
other parts were doing. However, many
other efforts have remedied this defect
by employing higher-level states. As dis-
cussed by Xenakis [29], a state can be a
'screen' describing all of the musical
activity at a particular instant of time
during the composition. States can be
Fig. 8. An event-relationdiagramrepresenting a procedural network for generating finite se-
chords, as in M and Jam Factory,or they
quences using the tokens a through m. This diagram is formally equivalent to Matrix 7.
can be polyphonic phrases, as discussed
byJones [30].
It is sometimes advantageous if the
states are ordinal positionswithin a se- ject appearing at different moments in to interpret the various matrix entries
quence. Consider the event-relationdia- time. Figure 8 is equivalent to Matrix 7. as follows:
gram [31] shown in Fig. 8, where the Matrix 7 is filled mostly with zeros 1. Entries on the top-left to bottom-
tokens a through m symbolize ordinal (omitted); to an information theorist it right diagonal indicate waiting prob-
elements. These elements might be would have high 'redundancy' and low abilities (elements d and k; all others
notes, chords or phrases; the main 'information content'. The fact that the are zero).
point is that different elements may process has a terminal state m allows us 2. Entries immediately to the right of
possibly represent the same musical ob- the waiting-probabilities diagonal

Ames, The Markov Process as a Compositional Model: A Survey and Tutorial 181

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
 STROPHE I (zeroth order)
i.- 2g
E
I
F1.
;A
gb.-
,p
f "1
STROPHE II (first order)
-
F'
~t- r i _ AF
mp
STROPHE III (second order)
ff
Fig. 9. Opening flute passages from Strophes I-III of Hiller and Baker's ComputerCantata,showing the differences in musical material
obtained through zeroth-, first- and second-order analysis of music by Ives. Copyright ? 1963 New Music Edition. Used by permission of
the publisher, Theodore Presser Company, Bryn Mawr, PA 19010, U.S.A.
indicate, for the most part, the prob-
abilities for advancing 'normally'
through the sequence (transitions
c -- d, e-> f, j -
3. Other entries to the right of the di-
agonal give probabilities for skip-
ping ahead (transitions
h - j, etc.) or for branching to alter-
native sequences
and b -> e).
4. Entries to the left of the diagonal give
probabilities for looping back (tran-
sitions 1 - h and
5. Since element m is a terminal state,
the transition probabilities in the
bottom are meaningless; the waiting
probability has been set to unity
simply for form's sake.
NTH-ORDER CHAINS
The discrimination of Markov model-
ing can be greatly enhanced by making
transition probabilities sensitive to two
or more previous states. The orderof a
Markov chain indicates the number of
past states taken into consideration:
events in a 'zeroth-order' chain are
independent of their predecessors;
events in a (standard) 'first-order'
chain are affected directly only by their
immediate predecessors; events in a
'second-order' chain are affected by
two predecessors, and so on.
Zeroth-order chains are described by
a one-dimensional probability distribu-
tion, while first-order chains require
the standard two-dimensional transi-
tion matrix. It is therefore logical to
extrapolate that each increment in
order should add another dimension to
the matrix, e.g. a three-dimensional
matrix for a second-order chain, or a
four-dimensional matrix for a third-
order chain. However, it can be shown
182 Ames,The Markov Process as a Compositional Model: A Survey and Tutorial
II 1,b
- -- I --P
mpp ,LrL=
fp
iff
\ r
7
mf,;mpJ PP fn f p ff pp f p f p f mppp p nffff '
tt |ziI J
if-:t7 / rn
Matrix7. Theevent-relation
diagramshowninFig.8 is equivalent
to thistransition
matrix.
k, etc.). source destination
a b c d e f g h i j k I m
e -> g and a 1.00
b 0.50 0.50
(transitions b -> c c 1.00
d 0.25 0.75
e 0.75 0.25
f 1.00
-> b).
g 1.00
h 0.60 0.40
i 1.00
1.00
k 0.33 0.67
0.50 0.25 0.25
m 1.00
that every Nth order process has an of various orders in their 1963 Computer
equivalent first-order formulation, Cantata [32], intending that these
which can be obtained by treating se- changing orders would be perceivable
quences of states as states in their own as large-scale changes in 'information
right. In addition, we have already seen context'. Figure 9 compares the open-
that, if the definition of a state takes ing flute passages from Strophes I-III;
into account modes of behavior, ordi- similar procedures were employed in-
nal positions and other relative proper- dependently to generate each of the
ties, it becomes a simple matter to in- remaining instrumental parts.
troduce higher-order sensitivity into a M and Jam Factory allow Markov
nominally first-order matrix. In those analysis and generation with up to four
instances where L-dimensional ma- orders of discrimination, and even this
trices indeed provide the most appro- liberal ceiling undoubtedly will be
priate representation, the fact that the raised as computers continue to gain in
theoretical number of cells in the ma- speed and memory. However, Baffioni
trix increases exponentially with Ncan and his collaborators suggest a point of
be somewhat daunting. However, it is diminishing returns: "if [the order] is
often possible for programmers to ef- taken very large . . . the sample paths
fect great savings in computer memory tend to reproduce long pieces of the
over this theoretical size, since only corpus from which the transition ma-
transitions of nonzero probability need trix was extracted ('low originality?')
be stored. and still the lack of general organiza-
Hiller and Baker used Markovchains
This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM
All use subject to JSTOR Terms and Conditions
Fig. 10. Parameter graph for an unfinished composition byJohn
Myhill. Each 2-minute 'section' is pieced together from many
short 'segments'; segment lengths are graphed logarithmically
from a minimum of 2 seconds to a maximum of 10. Separations
range from 0 to 1.
tion at [higher orders] will be clearly
recognized" [33].
Roads's PROCESS/ING program
implements Markov chains of ex-
tremely high order, although Roads's
program was not intentionally based on
Markov'smodel [34]. Roads used PRO-
CESS/ING to create several composi-
tions for tape alone; these included two
1975 pieces, prototype[35] and Plex (re-
vised 1982). The program was inspired
by the theory of 'finite automata', a
descendent of Information Theory.
Each one of 26 automata is given con-
trol over one of 26 attributes charac-
terizing a cloud of sonic 'grains' (e.g.
starting time, duration, mean fre-
quency, temporal density, registral
proximity of grains). The automata are
connected by a 26 x 26 'interconnec-
tion matrix'; thus, in general, the attri-
bute specified by an automaton during
the j+ 1st cloud depends both upon the
past history of the automaton itself and
upon the attributes specified by the re-
maining automata during the jth
cloud. The process as a whole therefore
may be regarded as a Markov chain in
which the number of possible states is
given by the number of ways in which
the attributes may be combined, while
the order depends upon how many pre-
vious clouds are taken into account by
the automata.
Darrell Conklin has devised an ap-
proach to Markov modeling that medi-
ates between multiple transition ma-
trices, each 'viewing' the music from a
different perspective, e.g. durations,
absolute pitches, chromatic degrees, in-
tervals between consecutive pitches in
the same voice, intervals between simul-
taneous pitches in different voices [36].
Conklin's approach embraces transi-
tions of arbitraryorder; he takes advan-
tage of the fact that a sequence of states
can be treated as a state in its own right
to express his transition matrices as lists
of productions of the form:
[note j- n,..., note j- 2, note j- 1]
-> [predictions for note j].
To select a production, Conklin's pro-
gram simply steps through the list until
it finds a production whose left side
matches the tail end of the music
generated so far. Because the produc-
tions are derived through analysis of
This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM
Time: 0'00" 2'00" 4'00" 6'00" 8'00" 10'00"
Active choirs: 1 2 4 2 1
Segment length Not applicable
Separation: Not applicable
musical sources, the following compli- second matrix in the second section,
cation results: and so on. It might mean using tran-
sition matrices to select certain 'un-
[When] models are being extracted
froman existingsequencea newprob- shaped' attributes but leaving the re-
lem arisesfor high-ordermodels.Most maining attributes to other selection
of the long [sequences]willneverhave methods. In a real-time compositional
been seen, and so there is no basison
which to make a prediction. ... A processor like MorJam Facto?y,it might
mean keeping the same transition ma-
compromisewhichworksverywellfor
text is to use a range of models, for trices but subjecting the results to trans-
example, all models from order 0 to formations (e.g. mapping chromatic
order 5 (Clearyand Witten, 1984). notes to diatonic notes), which are
Whenmaking[predictions]the order
5 model is firstchecked,if it has [pre-
more easily amenable to real-time ma-
dictionstheyare]used,if not the order nipulation. There are also ways in
4 model is checkedand so on [37]. which Markov'smodel can be extended
directly to accommodate long-term
Each of Conklin's viewpoints gener-
ates its own list of weighted predictions, shapes. One method is to allow the
so the program must include an arbiter
transition probabilities to evolve over
to mediate between lists and to estab- time; another is to implement the pro-
lish a final order of preferences. If the
cess of selecting transition matrices as a
Markov chain in its own right.
arbiter determines that there is no com-
mon ground between viewpoints, then
the program must backtrack and revise Evolving
Transition Matrices
An unfinished and untitled work by
an earlier note. Backtracking is an arti-
ficial intelligence technique that was John Myhill,
which I partially pro-
not envisioned in Markov'swholly left- grammed, is the first composition, to
my knowledge, to employ transition
to-right scenario, so in this respect the that evolve gradually un-
action of the arbiter takes Conklin's probabilities
der strict parametric control. Myhill's
program beyond the scope of this
article. piece was to be constructed from five
2-minute sections, for a total length of
10 minutes. The music was to be played
OF by four antiphonal choirs situated
TECHNIQUES around an auditorium in four loca-
CONTROL tions: on the stage, to the left side, to
The most straightforward approach to the right side, and in the balcony. Fig-
ure 10 shows how the antiphony was to
shaping Markov-generated music is to
select excerpts manually from different evolve over the length of the piece.
chains and to cut and paste these ex- Only one of the four choirs plays at a
time in the outermost sections, sections
cerpts, again manually, into a satisfac-
Markov chains, like all 1 and 5; the music trades off between
tory composite.
random processes, are inherently con- pairs of choirs in sections 2 and 4, and
all choirs are active continuously in sec-
traryand obstinate: over the short term
there is absolutely no guarantee that tion 3.
their behavior will conform to the prob- This much of the compositional
abilities set forth in the transition ma-
trix. Given this fact, it is reasonable to
include a degree of human interven- Matrix8. Thisone-choirtransition
tion in the process; the cut-and-paste matrixis derivedfromFig.1la
has been characteristic of Ko- whenthe separationis 0.75.
approach
tik's work, for this reason.
However, Kotik represents the ex-
ception more than the rule; developers source destination
of composing programs, for the most S L R B
part, seek automated methodologies S 0.25 0.25 0.50
that can accommodate both content
L 0.05 0.05
and form. This might mean simply in-
R 0.05 0.05
structing the program to use one ma-
trix in the first section of the piece, a B 0.50 0.25 0.25
Ames,The Markov Process as a Compositional Model: A Survey and Tutorial 183
All use subject to JSTOR Terms and Conditions
 Separation

Separation

s L REBS LRL] SB LSI RS LB RB

Fig. 1la. (left) Transition probabilities for one active choir. Choirs are abbreviated as follows: S-stage, L-left, R-right and
B-balcony. Values are derived for a given source state, destination state and separation parameter as follows: (1) Of the four vertically
stacked rectangles that together make up the graph as a whole, locate the rectangle with the given source state indicated on the left.
(2) Use the key given along the bottom to determine the shading for the given destination state; locate the appropriate destination region
within the source rectangle. (3) Locate the given separation parameter along the scale at the top of the graph and sketch a vertical line in
this position from the bottom of the source rectangle (zero) to the top of the rectangle (unity). (4) The length of the line segment cutting
across the destinationregionwill then indicate the transition probability for the given source, destination and separation.

Fig. 1Ilb. (right) Transition probabilities for two active choirs. Choirs are abbreviated as follows: LR-left and right, SB-stage and
balcony, LS-left and stage, RS-right and stage, LB-left and balcony, RB-right and balcony.

process actually was implemented as a
program. Control over the antiphonal
process is exerted through two high-
level parameters, both graphed in Fig-
ure 10: segment length and separation.
The segment length determines the pe-
riod of time separating transitions be-
tween choirs in sections 1 and 5 or
between pairs of choirs in sections 2
and 4; the program rounded each seg-
ment length to the nearest second and
accumulated the residue into the next
segment. The separation is a single
number that controls the antiphonal
intensity. When this number is small
(near zero), transitions occur between
adjacent choirs; when the number is
large (near unity), transitions occur
across the room or into the balcony,
which Myhill defined to be 'separated'
from all other choirs.
Myhill treated the one-choir and two-
choir sections separately. For the one-
choir sections, he devised a continuum
of transition matrices arranged along a
line from minimum to maximum sepa-
ration, and he represented this con-
tinuum with the graph shown in Fig.
1la; the graph in Fig. 1lb represents his
continuum of transition matrices for
the two-choir sections. Matrix 8 shows
what transition probabilities result for
a one-choir section when the separa-
tion in 0.75. To understand how the
upper row of probabilities was derived
(stage as source state), first locate the
'S' rectangle across the top of Figure
1la and then sketch in a vertical line for
0.75 separation. Notice that the upper
one-fourth cuts across the dark region
representing transitions from S to B
region. A similar procedure may be
used to derive transition probabilities
from Figure 1 b.
Chains of Chains
In their 1981 paper, Baffioni, Guerra
and Lalli proposed a hierarchical organi-
zation of Markov chains in which the
'low-level' transition matrices used to
select the musical details are selected
using 'median-level' transition matri-
ces; these 'median-level' matrices in
turn are selected by 'higher-level'
matrices, and so on, to arbitrary levels
of complexity. The approach makes
most sense when the lower-level chains
are 'highly redundant' from an Infor-
mation-Theoretic view and when they
are provided with terminal states to kick
the process back up into its higher-level
modes. Two of the examples presented
by Baffioni et al. illustrate thematic vari-
ation using low-level matrices with very
nonuniform probabilities to represent
particular themes. Two other examples
incorporate "some features of classical
tonal music", with matrices to simulate
"chord prolongation, cadences, and
modulation" [38].
Capabilities to implement chains of
chains have been included among the
many features of HMSL which Burk,
Polansky and Rosenboom have been
developing since 1985 [39]. Like the
M U S I C O MP computer-composition
'language' of the 1960s [40], HMSL is
a library of compositional procedures,
which a composer links together in a
main program of his or her own devis-
ing. In addition, HMSL provides a set
of standardized procedural 'objects',
which facilitate implementation of
hierarchic processes. Of specific rele-
vance to this article is the TSTRUC-
TURE object, which consists of the fol-
lowing: (1) a list of n subsidiary objects,

184 Ames,The Markov Process as a Compositional Model: A Survey and Tutorial

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
which may be musical-event generators,
lower-level TSTRUCTURES or other
types of HMSL objects; (2) an n x n
transition matrix; and (3) a user-
supplied procedure for controlling the
TSTRUCTURE's behavior. Whenever a
subsidiary object completes its task, the
TSTRUCTURE employs the behavior
procedure to determine which object it
should invoke next; the default beha-
vior uses the transition matrix to select
an object probabilistically; however,
HMSL's designers intentionally have
left their system open to alternative in-
terpretations of the matrix.
ALTERNATIVES TO
MARKOV CHAINS
Statistical Balances
Composers who work with computer
programs often have been concerned
more with medium-term balances be-
tween musical attributes than with
short-term note successions. Although
transition matrices exercise some influ-
ence over balances-through the sta-
tionary probabilities-such influence is
indirect at best. Unless a transition ma-
trix is 'redundant', one cannot expect
the stationary probabilities to become
manifest until quite a number of transi-
tions have gone by. For this reason,
composing programs generally resort
to methods other than Markov chains
when balances are important. The tra-
ditional mechanism for directly realiz-
ing balances is based on the mathe-
matical paradigm of random selection
without replacement [41], but I have
recently developed a much more
versatile technique called statistical
feedback [42].
Top-Down Grammars
Top-down grammars were introduced
to musical theorists through the publi-
cations of Fred Lehrdahl and RayJack-
endoff [43] and to the computer-music
community through the writings of
Curtis Roads [44], Steven Holtzman
[45] and KevinJones [46]. Arising out
of the linguistic theories of Noam
Chomsky, top-down grammars quickly
demonstrated great power in the for-
mal (non-linguistic) study of structures
and processes [47]. A formal grammar
consists of an archetype(or axiom) and a
set of productions.The archetype de-
scribes a structure in general terms,
while the productions provide the
means of deducing details of the struc-
ture from generalities. In the symbolic
This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM
formalism devised by Chomsky, an arbi-
trary structure can be represented as a
string of tokens, which can be either
terminalor nonterminal.Productions are
expressed as rewrite rules, that is, as
instructions for replacing nonterminal
tokens with subsidiarystrings (substruc-
tures). If one begins with an archetype
and applies productions repeatedly
until all nonterminal tokens have been
rewritten as terminal strings, then this
final product is said to be an instanceor
statementof the archetype.
Chomsky's formalism is sufficient in
theory, though not always in practice,
to embrace a wide variety of formalized
compositional approaches, including
gestalt hierarchies, fractal generation
and Markov chains. Unfortunately,
Markov chains have been grossly mis-
represented by mathematical linguists
who have dismissed them as a trivial
instance of a 'context-free' grammar.
Quite to the contrary, sensitivity to con-
text is an essential characteristic of any
system of conditional probability, espe-
cially Markov chains [48].
Chomsky and his colleagues make a
valid criticism, however, when they
point out that Markovchains are exclu-
sively left-to-rightprocesses predicating
all actions upon past behavior. Such
systems may be adequate for emulating
inanimate processes of nature, but they
are totally inadequate for modeling
planned activities, such as those in
which actions tend to be predicated
toward future goals-e.g. language or
musical composition.
Chomsky's
model, by contrast, is a top-downgen-
erative process in which the primary
origins and goals can be expressly
stated in the archetype, while the paths
connecting origins with goals (and con-
necting subsidiary origins and goals
encountered along the way) can be
worked out in detail through the pro-
ductions. Chomsky's model includes
Markov chains as a special 'context-
sensitive' case, and it offers much more.
Bottom-Up Processing
A bottom-upapproach to composition
begins with a kernel of primary mate-
rial, subjects this material to a variety of
transformations (e.g. inversion, retro-
grade, transposition) and pieces the re-
sults together into a piece of music.
Most prominent among the systematic
bottom-up procedures for composition
is the serial approach developed by
Schoenberg and extended by com-
posers such as Pierre Boulez, Karlheinz
Stockhausen and Milton Babbitt [49];
other examples include the Schillinger
Ames,The Markov Process as a Compositional Model: A Survey and Tutorial
All use subject to JSTOR Terms and Conditions
system of composition [50] and the
germ-cell procedures used in certain
pieces by Scriabin and Schoenberg
[51]. Some very diverse bottom-up ap-
proaches have been implemented in
composing programs by Emmanual
Ghent [52], William Buxton [53] and
myself [54].
To employ a bottom-up approach
one must assume a hierarchic division
of the whole into parts. This is incom-
patible with Markovchains as they were
originally formulated, since Markov's
original model recognizes neither hier-
archies nor even boundaries of any sort.
However, the proposal of Baffioni,
Guerra, and Lalli removes this incom-
patibility to some extent.
Artificial Intelligence
There has been some debate over the
past few years concerning which meth-
ods of automated composition should
legitimately be considered artificial in-
telligenceand which should not. Many
accept that in order for a program to be
called 'intelligent', it is sufficient that it
be able to undertake decisions on its
own; this has been the basis for the term
'intelligent instrument', which has
been used by Laurie Spiegel in refer-
ence to her Music Mouseprogram and
by Zicarelli and his colleagues in refer-
ence to M and Jam Factory.
I adamantly disagree. Any reader
who simply thumbs through a repu-
table introductory text on artificial in-
telligence [55] quicklywill discover that
AI has a well-established repertory of
techniques that bear little or no resem-
blance to Markovprocedures-aspects
of Conklin's approach excepted. (Mu-
sic Mousedoes not use Markov chains,
nor, for that matter, does it employ AI
techniques.) Indeed, Markov's ap-
proach is probably the most unintelli-
gent decision-making mechanism im-
aginable, for it is a veritable paradigm
of brute-force modeling. A Markov
chain treats each circumstance as a spe-
cial case; AI, by contrast, employs gen-
eralized criteria expressed either as ab-
solute rules or as relative preferences.
A Markov chain becomes committed
irrevocably to each decision as it is
made; AI programs are capable of back-
tracking if they come to an impasse. A
Markovchain regards a message (e.g. a
musical composition) as a linear se-
quence of events; AI regards a message
as a network of relationships. These
differences mean that where Markov-
chain programs are easy to implement
and quick to run, the code for an AI
program might well fill this journal,
185
 and the execution time for such a pro-
gram might run into many hours.
The results produced by AI pro-
grams (in conjunction with statistical
procedures, top-down grammars and/
or bottom-up processing) have much
greater integrity; when used to emulate
traditional music, Markovchains at best
have produced a garbled 'sense' of the
original style, while existing AI pro-
the analytic results will not be mean-
ingful.
As with any technique, the decision
of whether to employ transition ma-
trices depends on the compositional
requirements of the moment. If Mar-
kov's model is sufficient to handle the
problem at hand, then its ease and com-
putational efficiency make it a logical
way of doing things. If standard first-
simple objects into more elaborate ones-as con-
trasted with reductions, which simplify things.
random incident-an incident with unpredictable
outcomes. Probability theory circumvents the un-
certainty of isolated random incidents by predict-
ing distributions of outcomes from many similar
incidents taken together.
scalar-a number used to describe one attribute of
an object. (See also vector.)
stationary probabilities-a distribution predicting
how often each state in the range of a Markov chain
will appear over the long term.

grams for traditional music have pro-
duced results that are extremely similar
to the real thing.
CONCLUSIONS
By no means can Markovchains be said
to occupy the cutting edge of progress
in automated composition or analysis.
They have been around since the turn
of the century. They have been dis-
missed (sometimes unjustly) by mathe-
matical linguists and by AI researchers.
They have long been out of fashion in
the music-theory establishment. De-
spite all this, Markovchains continue to
be exploited by composers who work
with computer programs. This persist-
ent interest undoubtedly has to do with
the fact that Markov's model can be
adapted to accommodate just about
any discrete system of conditional prob-
ability one might think of using in a
composing program. In addition, Mar-
kov's representation is straightforward,
since a chain's behavioral character-
istics are described entirely by the tran-
sition matrix. This means that standard
Markov chains are almost trivial to im-
plement-to a large extent, the matrix
is the program.
From the earliest of Markov-based
composing programs, there have been
two equally legitimate waysof obtaining
transition probabilities. Either one can
'compose' probabilities to suit general-
ized compositional objectives, or one
can derive probabilities through analy-
sis of musical source material. In both
cases, it is critical to define the states of
the chain in an appropriate manner. If
one chooses to compose one's own
probabilities, then one should take into
account not only short-term transitions
but also the long-term trends described
by the stationary probabilities. (Al-
though stationary probabilities are te-
dious to calculate by hand, it is easy to
incorporate a feature for extracting sta-
tionary probabilities into a transition-
matrix editor.) If one chooses to derive
probabilities analytically, then one
should take care that the source mate-
rial is stylistically uniform; otherwise,
order matrices will not suffice, one transition-the progress in a Markov chain from
one state to its successor during two consecutive
might consider Nth-order matrices, events.
evolving matrices or chains of chains. transition probability-the probability of making a
Since transition matrices are rarely transition to a specified destination state, given a
used in isolation, perhaps a blend of specified source state.
approaches will be needed. If not, then transitionmatrix-a rectangular array detailing the
there are plenty of other techniques to transition probabilities obtained by enumerating
every pair of source states and destination states
choose from. from the range of a Markov chain and by determin-
ing the transition probability in each instance. The
Acknowledgments kth row of a transition matrix gives the probability
distribution for the state of the jth event in the
I am greatly indebted to the Kurzweil Foundation
for its continuing financial support and for the
chain under the assumption that thej-1 st position
resides in state k.
Macintosh computer upon which this article was
written. uniform distribution-the distribution of a ran-
dom process in which each outcome or state is
equally likely.
Glossary
vector-a set of numbers used to describe two or
antiphony-musical contrast realized through
physical separation of instruments in a perform-
more attributes of an object, e.g. a musical note
might be described by a vector consisting of the
ance environment. note's starting time, duration, pitch and dynamic.
continuous-a set is continuous, in a practical This is a mathematician's definition of 'vector';
sense, if every pair of distinct elements is connected physicists use the word more strictly. (See also
by intermediate elements. scalar.)
conditional waiting count-the expected number of times that
probability-random incidents
said to be conditional when their outcomes
are
are a state will directly succeed itself in a Markov chain.
influenced by external factors. (See also waitingprobability.)
discrete-a set is discrete if its elements can be waiting probability-the probability that the jth
represented by integers (e.g. item 1, item 2 .. .).
event of a Markov chain will reside in some state k,
given that the j-1 event also resides in state k.
st
event-relation diagram-a diagram showing
graphically how the various states of a Markov chain
References and Notes
progress between one another. Such diagrams are
most revealing when transition matrices are 1. Charles Ames, "Automated Composition in Ret-
'sparse', that is, when the great majority of transi- rospect: 1956-1986", Leonardo20, No. 2, 169-185
tion probabilities are zero.
(1987).
expectation-an average value predicted
basis of a probability distribution.
on the 2. The official jargon for such processes is 'time-
variant Markovchains'. Before his creative momen-
Markovprocess or Markovchain-a chain of states, tum succumbed during the struggle with cancer
that finally took his life in 1987, Myhill had worked
each determined by the outcome of a random inci-
dent. In addition, the following restrictions apply: out only the general form of a composition, the
(1) the range of states (i.e. the range of outcomes antiphonal process of which is discussed later in
available to the random incidents) is fixed, and (2) this article. Previously I had also had two direct
each random incident is conditioned personal encounters with musical Markov chains:
by the out-
come of its predecessor (or, equivalently, by the once in 1982 when Petr Kotik asked me to program
his interactive Markov-matrixeditor, and once in
preceding state).
1983 when I wrote my own short program to illus-
order of a Markov chain-the number of preced- trate basic Markov procedures for a class I was
ing states that directly influence a transition. Each teaching on automated composition.
increment in order adds an additional dimension
to the transition matrix. 3. Two favorite mathematical texts covering Mar-
kov chains are William Feller, An Introductionto
probability-a number indicating the likelihood
ProbabilityTheoryand ItsApplications,3rd Ed., 2 vols.
that a random incident will produce a specified
(New York:John Wiley & Sons, 1967) and Joseph
outcome. Probabilities range from zero (impossi-
bility) to unity (certainty), with a continuum of
L. Doob, Stochastic Processes (New York: John Wiley
& Sons, 1953). My primary source was Samuel
gradations in between. Karlin and Howard Taylor, A First Course in Stochas-
probabilitydistribution-a function that associates tic Processes (New York: Academic Press, 1975); Kar-
lin and Taylor assume intensive mathematical
a probability with every conceivable outcome of a
random incident. Distributions can be either dis- background.
crete or continuous, depending on the range of out- 4. Claude Shannon, "The Mathematical Theory of
comes. Probability distributions are mathemati- Communication", Bell Systems Technical Journal 27,
cally identical to statistical distributions, except
that the former describes the ideal distribution of
379-423 and 623-656 (1948). See also Colin
Cherry, On Human Communication, 3rd Ed. (Cam-
a future population, while the latter describes the
bridge, MA: MIT Press, 1978).
actual distribution of a population that already
exists. 5. Harry F. Olson, Music, Physics, and Engineering,
2nd Ed. (New York: Dover, 1967). A description of
production-an elementary operation within a Olson's work can be found in Charles Dodge and
generative process. Productions typically transform Thomas Jerse, "Composition with Computers",

186 Ames, The Markov Process as a Compositional Model: A Survey and Tutorial

This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM

All use subject to JSTOR Terms and Conditions
Chap. 8 in ComputerMusic: Synthesis,Composition,
and Performance
(New York:Schirmer Books, 1985).
6. Richard C. Pinkerton, "Information Theory and
Melody", ScientficAmerican194, 77-86 (1956).
7. Allen Irvine McHose, TheContrapuntalHarmonic
Techniqueof the 18th Century(East Norwalk, CT:
Appleton-Century-Crofts, 1947).
8. Leonard B. Meyer, Emotionand Meaningin Music Mills College, 1988).
(Chicago: University of Chicago Press, 1956).
9. Leonard B. Meyer, "Meaning in Music and In-
formation Theory", Chap. 1 in Music, theArts,and
Ideas(Chicago: University of Chicago Press, 1967).
10. Lejaren Hiller and Leonard Isaacson, Experi- MusicJournal11, No. 4 (1987) p. 13. More exhaus-
mental Music (New York: McGraw-Hill, 1959; re-
printed Greenwood Press, 1979).
11. Lejaren Hiller and Robert Baker, "Computer NY
Cantata:An Investigation of Compositional Proce-
dure", Perspectives of NewMusic3 (1964) p. 62.
12. Defined by coordinates of register, dynamics
and rate of activity.See Iannis Xenakis, "Markovian
Stochastic Music-Theory"and "MarkovianStochas-
tic Music-Applications", Chaps. 2 and 3 in Formal-
ized Music (Bloomington, IN: Indiana University
Press, 1971).
13. Gottfried Michael Koenig, "PROJECT2: A Pro-
gramme for Musical Composition", ElectronicMusic
Reportsl, No. 3 (1970) p. 1.
14. This program was first implemented on an
Apple II computer in 1982, with myself program-
ming to Kotik's specifications. Kotik learned the
name for what he wanted to do from me, but the
concept was entirely his own. He has since used the
program to compose numerous pieces, beginning
with his 1983 Solos and Incidental Harmonies.At
Kotik's request, I adapted the program in 1988 for
the Apple Macintosh, upgrading it in the process
to include capabilities for steady-state analysis and
for chains of chains. When during the spring of
1989 I faced the challenge of teaching automated
composition to non-programmers, the simplicity of
Kotik' program made it an ideal way of breaking
the ice.
15. Curtis Roads, "A Systems Approach to Com-
position" (Honors thesis, University of California
at San Diego, 1976).
16. Laurie Spiegel, "Sonic Set Theory: A Tonal
Music Theory for Computers", Proceedingsof the
SecondAnnual Symposiumon SmallComputers and the
Arts(1982).
17. A Russian about whose work I know only in-
directly. See R. C. Zarpov, Kibernetikai Muzyka
(Moscow: Nauk, 1971).
18. Peterson uses random walks, a process in which
the motions of a travelling subject (e.g. a drunk)
are described by transition matrices. See Tracy L.
Peterson, "Interactive Digital Composition", Pro-
ceedingsof the1978 InternationalComputer Music Con-
ference (San Francisco: Computer Music Associa-
tion, 1978) p. 167.
19. Kevin Jones, "Compositional Applications of
Stochastic Processes", ComputerMusicJournal5, No.
2 (1981), p. 45.
20. DeLio's Serenadefor piano (described in Ames
[1]) uses a simple transition process in conjunction
with many other techniques.
21. Claudio Baffioni, Francesco Guerra and Laura
Tedeschini Lalli, "Music and Aleatory Processes",
Proceedingsof the "5-Tage-Kurs" of the USP Mathe-
matisierung(BielefeldUniversity,1981) p. 272.
This content downloaded from 146.96.128.36 on Thu, 18 Dec 2014 21:12:09 PM
22. Darrell Conklin with John G. Cleary, "Model-
ing and Generating Music Using Multiple View-
points", Proceedingsof theFirst Workshopon Artificial
Intelligenceand Music (St. Paul, MN, 1988) p. 125.
23. Larry Polansky, Phil Burk, Robert Marsanyi,
Dorothy Hayes and Mitchel Gass, HierarchicalMusic
SpecificationLanguage Referenceand User Manual
(Oakland, CA: Center for Contemporary Music,
24. JamshedJ. Bharucha, "NeuralNet Modeling of
Music", Proceedingsof theFirst Workshopon Artificial
Intelligenceand Music (St. Paul, MN, 1988) p. 173.
25. David Zicarelli, M and Jam Factory,Computer
tive descriptions appear in the manuals provided
with both programs, available from Intelligent
Computer Music Systems, P.O. Box 8748, Albany,
12208, U.S.A. The M manual was written by
David Zicarelli andJoel Chadabe (1987). TheJam
Factorymanual was written by David Zicarelli, An-
tony Widoff andJoel Chadabe (also 1987).
26. Demonstrations7-11 are described in Charles
Ames, "Tutorialon Automated Composition", Pro-
ceedingsof the1987 InternationalComputerMusicCon-
ference (San Francisco: Computer Music Associa-
tion, 1987) p. 1.
27. Note durations are generated using John
Myhill's 'controlled' exponential distribution, lim-
iting the ratio between the maximum and min-
imum durations to 8. See Lejaren Hiller, Charles
Ames and Robert Franki, "Automated Composi-
tion: An Installation at the 1985 International Ex-
position in Tsukuba, Japan", Perspectivesof New
Music 23, No. 2 (1985) p. 215 n. 2.
28. If the range of states is not discrete, then the
transition probabilities cannot be represented as a
matrix. However, Markov's model may be general-
ized to encompass both discrete and continuous
ranges by replacing the transition matrix with a
conditional probability function of the form P(Xn,
X,1). For example, consider a chain of real num-
bers in which Xn follows a Gaussian distribution
whose mean is X,n. (such a chain simulates Brown-
ian motion).
29. Xenakis [12].
30. Jones [19].
31. This way of graphing event-relations comes to
me byway ofJones [ 19]; the notion of ordinal states
is the basis of the thematic procedures described in
Baffioni, Guerra and Lalli [21].
32. Hiller and Baker [11 ].
33. Baffioni, Guerra and Lalli [21].
34. Roads [15].
35. Curtis Roads, prototype(manuscript, University
of California at San Diego, 1975).
36. Conklin [22].
37. Conklin [22]. Conklin is referring to J. G.
Cleary and I. H. Witten, "DataCompression Using
Adaptive Coding and Partial String Matching",
IEEETransInformationTheory,IT-30:2 (1984) p. 306.
38. Baffioni, Guerra and Lalli [21].
39. Polansky et al. [23].
40. MUSICOMP is described by Hiller and Baker
[11].
41. Random selection without replacement has
been employed to compose numerous pieces, in-
cluding Herbert Brin's SonoriferousLoops, G. M.
Ames,The Markov Process as a Compositional Model: A Survey and Tutorial
All use subject to JSTOR Terms and Conditions
Koenig's Ubungfiir Klavierand Thomas DeLio's
Serenade;Ames [1] describes how all three of these
compositions were made.
42. The basic procedures for statistical feedback
are introduced in Charles Ames, "Two Pieces for
Amplified Guitar", INTERFACE.Journal of New
Music Research15, No. 1 (1986) p. 35; important
refinements are described in Charles Ames, "Con-
currence", INTERFACE17, No. 1 (1988) p. 3.
43. Fred Lehrdahl and RayJackendoff, A Generative
Theoryof Tonal Music (Cambridge, MA: MIT Press,
1983).
44. Curtis Roads, ComposingGrammars,2nd Ed.
(San Francisco: Computer Music Association,
1978).
45. Steven Holtzman, "A Generative Grammar
Definitional Language for Music", INTERFACE9,
No. 1 (1980) p. 51. Holtzman's synopsis of Roads
[44] is more readable than Roads's original treat-
ment.
46. Jones [19].
47. This includes linguistic structures and pro-
cesses but is not limited to them. See Maurice Gross
and Andre Lentin, IntroductiontoFormalGrammars,
translated from the French by Morris Salkoff (New
York:Springer-Verlag, 1970).
48. This error has been perpetuated by a variety of
writers, including Gross and Lentin [47], Roads
[44] and Holtzman [45]. At its heart is a mistaken
formulation of transition matrices as productions
of the form X -- aX (X non-terminal; a terminal)
when transition matrices should correctly be repre-
sented as setsof productions of the form aX -- abX
(X non-terminal; a, b terminal) where a stands for
the source state and b stands for the destination
state.
49. See George Perle, SerialComposition
and Atonal-
ity, 3rd Ed. (Berkeley: University of California
Press, 1972); and Charles Wourinen, SimpleCom-
position(New York:Longman, 1979).
50. An algorithmic approach to composition that
was very popular before World War II, but which is
all but forgotten today. SeeJoseph Schillinger, The
SchillingerSystemof Musical Composition(New York:
Carl Fischer, 1941).
51. Cf. "'Free' Atonality", Chap. 11 of Perle [49]
and Allen Forte, TheStructureof AtonalMusic (New
Haven, CT: Yale University Press, 1973).
52. Ames [1] describes some of Ghent's work and
cites Ghent's articles.
53. See Otto Laske, "Subscore Manipulation as a
Tool for Compositional and Sonic Design" Proceed-
ings of the 1980 InternationalComputerMusic Confer-
ence(San Francisco: Computer Music Association,
1980) p. 2, for a case study of Buxton's score-editing
utility being used to produce a composition.
54. See Charles Ames, "ProtocolMotivation, De-
sign, and Production of a Composition for Solo
Piano", INTERFACE11, No. 4, 213 (1982). See also
the descriptions of Demonstrations10 and 11 in
Ames [26].
55. E.g. Avron Barr, Paul Cohen and Edward
Feigenbaum, The Handbookof ArtificialIntelligence
(Reading, MA: Addison-Wesley, 1981). See also
CharlesAmes, "AIin Music",EncyclopediaofArtificial
Intelligence(New York:John Wiley & Sons, 1987) p.
638.
187