r e a di n g i m a g e s

Thissecondeditionof the landmarktextbookReadingImages

buildson its reputation
asthe irst systematic
and
comprehensive
accountof the grammarof visualdesign.
Drawingon an enormous
rangeof examples
from children,s
drawings
photo-journalism
to textbookil lustrations,
to fineart,
aswell asthree-dimensional
formssuchas sculpture
andtoys,
the authorsexamine
the waysin whichimagescommunicate
meantng.
Features
of this fully updatedsecondeditioninclude:
.
.
.
.

newmaterialon movingimagesand on colour

a discussion
of howimagesandtheir useshavechanged
t h r o u g ht i m e
websites
andweb-based
images
ideason the futureof visualcommunication.

ReadingImagesfocuses
on the structuresor'grammar'of visual
- and
design- colour,perspective,framing
andcomposition
provides
the readerwith an invaluable'tool-kit'for
reading
images,
whichmakesit a mustfor anyoneinterested
in
communication,
the mediaandthe arts.
GuntherKressis Professor
of Englishat the Instituteof
Education,
University
of London.Theo van Leeuwenhas
producerin the Netherlanos
workedas a film andtelevision
ano
Australiaandas Professor
in the Centrefor Language
&
Communication
Research
at CardiffUniversity.
He is currently
Deanat the Facultyof Humanities
and SocialSciences,
University
of Technology,
Sydney.
Theyhavebothpublished
widelyin the fieldsof language
andcommunication
studies.

Pr ai s ef o r t h e
f i r s t e di t i o n
'ReadingImagesis the mostimportantbookin visual
c o m m u n i c a t i os n
i n c eJ a c q u eB
s e r t i n ' s e m i o l o goyf
i n f o r m a t i ognr a p h i c sI t. i s b o t ht h o r o u g ha n dt h o u g h t provoking;
breakthrough.'
a remarkable
l(evinG. Barnhurst,SyracuseUniversity,USA
' F r e s ha n ds t i m u l a t i n gT.h es o c i o c e n t rai cp p r o a c ihs b y f a r
the mostpenetrating
approachto the subjectcurrently
available.'
PaulCobley,LondonGuildhallUniversity
'A usefultext for al I studentswho are involvedin areas
w h i c hr e l yo n b o t hl a n g u a gaen dv i s u a il m a g e fso r t h e i r
e x o r e s s i oann da r t i c u l a t i oonf i d e a s . '
Catriona Scott, M iddlesexUniversity
'Thisis the bestdetailedandsustained
development
of the
" s o c i a sl e m i o t i ca" p p r o a ctho t h ea n a l y s iosf v i s u a l sC. l e a r ,
informative
andtheoretical
ly developmental.'
Dr S. Cottle,Bath HE College
' E x c e l l e n-t w i d er a n g i n -g a c c e s s i b-l et u t o r s '" B i b l e " . '
Jan Mair, EdgeHill UniversityCollegeof
Higher Education
'Extremelyattractiveandwell laid out.Veryuseful
bibliography.'
Dr IVl.Brottman,East LondonUniversity
'Veryclearlywritten- it makesgoodconnections
between
usefulfor
differentareasof visualpractice- especially
studentsfrom a varietyof backgrounds
attempting
" m i x e d "c o u r s e w o r k . '
Amy Sargeant,PlymouthUniversity

r e a d i n gi m a g e - s
G U N T H E RK R E S S a n d
T H E 0 v a n L E E U W E N

T H E 6 R A M M A R
O F

V I S U A LD E S I 6 N

S E C O N DE D I T I O N

Hltiy"'l:s,g:",
LONDON AND NEW YORK

Firstoublished
1995
by Routledge
2 ParkSquare,
MiltonPark,Abingdon,0xon
0X144RN
published
in the USAandCanada
Simultaneously
by Routledge
270 Madison
Ave,NewYork,NY 10016
2006
Second
editionpublished
Reprinted
2007(twice),2008
Routledgeis an imprint of the Taylor & Francis Group,an informa business
@ 1996,2006GuntherKressandTheovanLeeuwen
Typesetin Bell Gothicby RefineCatch
Ltd, Bungay,Suffolk
Printedandboundin GreatBritainby
TJ InternationalLtd, Padstow,
Cornwall
All rightsreserved.
No partofthisbookmaybe reprinted
or reproduced
or
utilizedin anyformor byanyelectronic,
mechanical,
or othermeans/
now
photocopying
knownor hereafter
including
invented,
andrecording,
or in any
informationstorageor retrievalsystem,
withoutpermission
in writingfrom
the oublishers.
British Library Cataloguingin Publication Data
A catalogue
recordfor thisbookisavailable
fromthe BritishLibrary
Library of CongressCatalogingin Publication Data
l(ress.
GuntherR.
Reading
images: the grammarof visualdesign
/ Guntherl(ressandTheo
- 2nded.
vanLeeuwen.
0 .c m .
Includes
bibliographical
references
andindex.
1. Communication
in design. I. VanLeeuwen,Theo,I94T- II. Title.

Nl(1510.K64
2006
70I-dc22
(hbk)
ISBN10:0-415-31914-5
(pbk)
ISBN10:0-415-31915-3
I SBN10: 0-203-61972-2 Gbk)
ISBN13: 978-0415-3L9I4-0 (hbk)
ISBN13: 97844I5-3L915-7 (pbk)
(ebk)
ISBN13: 978-0-203-6L972-!

2006002242

CONTENTS
vii

P r e f a c teo t h e s e c o n e
dd i t i o n

ix

Prea
f c et o t h ef i r s t e d i t i o n

xi

Acknowledgements

I n t r o d u c t i otnh:eg r a m m aor f v i s u a l
design

16

T h es e m i o t i cl a n d s c a p e
l a: n g u a g ae n d
v i s u a cl o m m u n i c a t i o n

45

N a r r a t i v ree p r e s e n t a t i o ndse: s gi n i n g
s o c i a al c t i o n

79

C o n c e p t u ar e
l p r e s e n t a t i o ndse: s i g n i n g
s o c i a lc o n s t r u c t s

tt4

Representatio
a n di n t e t a c t i o n :
d e s i g n i n tgh e p o s i t i o no f t h e v i e w e r

t54

M o d a l i t yd: e s i g n i n m
g o d e l so f r e a l i t y

L75

T h em e a n i n go f c o m p o s i t i o n

2t5

M a t e r i a l i t ay n d m e a n i n g

239

T h et h i r d d i m e n s i o n

266

C o l o u r f utlh o u g h t s( a p o s t s c r i p t )

27t

References

287

Index

P r e fa c e t o t h e s e c o n de d i t i o n

amonga widegroupfrom
Thefirst editionof ReadingImageshashada positivereception
s nd real issues
t h e p r o f e s s i o nasn d d i s c i p l i n ewsh i c hh a v et o d e a lw i t h r e a l p r o b l e m a
i m a g e sT. h i s h a s g o n ea l o n gw i t h a b r o a d e ra g e n d ao f c o n c e r nw i t h ' m u l t i involving
. ed o n o t
m o d a l i t ya
' , r a p i d l yg r o w i n gr e a l i z a t i otnh a t r e p r e s e n t a t i iosna l w a y sm u l t i p l eW
a
definitive
approach,
like
settled
book
represents
anything
a
for
a
moment
this
think
that
'grammar'of images,
andat timeswe havebeenworriedby attemptsto treat it in that way.
We seeit as an earlyattempt/oneamongmanyothers,andwe wouldliketo seeit treated
l s
v e r ym u c ha s a r e s o u r cfeo r b e g i n n i ntgo m a k ei n r o a d si n t o u n d e r s t a n d i tnhge v i s u a a
r e p r e s e n t a t iaon dc o m m u n i c a t i o- ni n a s e m i o t i fca s h i o n- a n da l s oa s a r e s o u r cien t h e
d e v e l o p m eonftt h e o r i eas n d' g r a m m a r so' f v i s u acl o m m u n i c a t i oI n .t h a ts p i r i tw ew a n tt o
as our fully seriousand
we havewrittenheresimultaneously
stressthat we seeeverything
y e te n t i r e l yp r o v i s i o n sael n s e
o f t h i sf i e l d .
When we completedthe first editionof this book we were aware of a numberof
' o m i s s i o n s ' t-h i n g sw e f e l t s t i l l n e e d e d o i n g S
. o m eo f t h e s ew e h a v et a k e nu p i n o t h e r
otherswe havetried
a
for
instance
in
attempt
to
develop
theory
of multimodality;
ways,
our
been
the quite different
have
edition.
Foremost
among
these
to addressin this second
raisedby
been
constantly
The
has
and
first
of
these
issuesof the movingimage
of colour.
said
herecan
we
have
what
book,
rightly
so.
We
hope
that
and
thosewho haveusedthe
visual
to
approach
semiotic
field
moving
into
our
social
of
images
beginto integratethe
yet
a
l<ind
for
us
constituted
raised,
of
was
less
frequently
Theissue colour
communication.
with
a
theory
as
to
do
itself
as
much
with
the
issue
of
colour
to do
of theoreticaltest case,
s u c hm o r ew i d e l yc o n s i d e r e H
d .e r et,o o ,w e f e e lt h a t w e
o f m u l t i m o d asl o c i a sl e m i o t i cm
just
provided
In
additionwe haveaddeda
first
for
approach.
have
a
attempt
a different
o
f
r
o
m
a
n
d
w
e
b
s
i
t
e
s
d
,
o
m
a
i
n
s f v i s u a cl o m m u n i c a t i o n
CD-R0Ms
n u m b eo
r f n e we x a m p l e s
and are now of central
when
wrote
the
first
edition,
we
that had hardlybegunto develop
users
for many
of this book.
importance
0ne persistent
criticismof the first editionfrom a groupof readershasbeenthat the
b o o kw a s( t o o )l i n g u i s t i cT.h ef i r s tc o m m e nwt ew o u l dm a k ei st o s a yt h a tf o r u s ' f o r m a l i t y '
i n t h ed o m a i no f r e p r e s e n t a t i iosnn o t i n a n yw a yt h es a m ea s ' b e i n gl i n g u i s t i cS' .ot o s o m e
We alsothink
extentwe think that that criticismrestson that kind of misunderstanding.
have
aimedfor
certainly
and
formality.
We
betweenexplicitness
thatthere is a difference
(but
either
explicitness
that
not
latter.
Nor
do
we
think
always)for the
theformer,andoften
often all theselatter
creativity,imagination:
or formalityare the enemiesof innovation,
point
the
systemicfunctional
has
been
is
the
case
starting
rest on the former.It
that our
grammarof Englishdeveloped
we hadand haveattemptedto
by MichaelHalliday,though
focusedfeaturesas
useits generalsemioticaspectsratherthan its specificlinguistically
of
grounding
grammar.
done
at the beginning
had
As
Ferdinand
de
Saussure
for our
the
as
the
linguistics
part
do
not
see
semiotics;
but
we
last
linguistics
as
a
of
century,
we see
the

vil|

Preface to the second edition

disciplinethat can furnisha ready-mademodelfor the descriptionof semioticmodes

otherthan language.
Thenwe had thought,in our first attempt,that to showhow visual
c o m m u n i c a t i owno r k si n c o m p a r i s ot no l a n g u a gm
e i g h tb e h e l p f uiln u n d e r s t a n d i e
n igt h e r
and both - but that, too, was misunderstood
maybeas an attemptto imposelinguistic
categories
on the visual.We havethereforetried to refineand clarifythosesectionsof the
bookthat dealwith the relationbetweenlanguage
andvisualcommunication,
andto delete
or reformulatematerialwhichwe think mighthavegivenriseto thesemisunderstandings,
h o p e f u l lw
y i t h n o l o s so f c l a r i t yA
. c a r e f u rl e a d i n go f t h i s s e c o n d
e d i t i o no f o u r b o o kw i l l
show,we trust,that we are as concerned
to bring out the differences
betweenlanguage
a n d v i s u a lc o m m u n i c a t i oans w e a r e t h e c o n n e c t i o ntsh,e b r o a d e rs e m i o t i cp r i n c i p l e s
t h a t c o n n e c tn, o t j u s t l a n g u a g a
e n d i m a g e b, u t a l l t h e m u l t i p l em o d e si n m u l t i m o d a l
communication.
In our growingunderstanding
of this domain,reffectedin the reworkingof this book,
we owea debtof gratitudefor support,commentand critiqueto manymorepeoplethan
we can mentionor eventhanwe actuallyknow.But the namesof somefriends,colleagues,
students,
fellowresearchers
andcriticswho werenot alreadyacknowledged
in our preface
to the first editionhaveto be mentioned.
Amongtheseare CareyJewitt,Jim Gee,Ron
Scollon,Paul Mercer,Brian Street, Radan Martinec,Adam Jaworski,David Machin,
l(las Prytz,TealTriggs,AndrewBurn,Bob Ferguson,
PippaStein,DeniseNewfield,Len
Unsworth,LesleyLancasterand the many researchers
whosework has both givenus
confidence
and new ideas,and extendedour understanding
of this field - and of course,
and crucially,
we acknowledge
the supportfrom our publishers
and editorsat Routledge,
L o u i s aS e m l y eann dC h r i s t a b el (l i r k p a t r i c k .

P r e fa c e t o t h e f i r s t e d i t i o n

g r e wo u t o f d i s c u s s i o a
T h i sb o o l <
nb
s o u tv i s u a cl o m m u n i c a t i ownh i c hs p a n n ead p e r i o do f
sevenyears.Bothof us hadworkedon the analysisof verbaltexts,and increasingly
felt the
needof a betterunderstanding
of all the thingsthat 9o with the verbal:facialexpressions,
gestures,
images,music,and so on. Thiswas not only because
we wantedto analysethe
wholeof thetextsin whichthesesemioticmodesplaya vital roleratherthanjusttheverbal
part, but alsoto understand
language
can
of other languages
better.Just as a knowledge
0pennewperspectives
on one'sown language,
so a knowledge
of othersemioticmodescan
0pennewperspectives
on language.
I n 1 9 9 0 w e p u b l i s h ead f i r s t v e r s i o no f o u r i d e a so n v i s u a lc o m m u n i c a t i oR
n ,e a d i n g
Images,with DeakinUniversityPress.It waswrittenfor teachers,
andwe concentrated
on
c h i l d r e n 'dsr a w i n gas n ds c h o otle x t b o o ki l l u s t r a t i o nasl,t h o u g h
w e a l s oi n c l u d eedx a m p l e s
from the massmedia,suchas advertisements
and magazinelayout.Sincethen we have
expandedour researchto other fieldsof visualcommunication:
a muchwider rangeof
massmediamaterials;scientific(and other)diagrams,mapsand charts;and the visual
arts.We havealso madea beginning
with the studyof three-dimensional
communication:
sculpture,
children'stoys,architectureand everydaydesigned
objects.The presentbook
thereforeoffers a much more comprehensive
than the
theory of visualcommunication
e a r l i e rb o o k .
I n A u s t r a l i aa, n d i n c r e a s i n gel yl s e w h e roeu, r w o r k h a sb e e nu s e di n c o u r s e so n c o m municationand mediastudies,and as a methodology
for researchin areassuchas media
representation,
film studies,
children'sliteratureandthe useof illustrations
and layoutin
schooltextbooks.
The presentbook hasbenefitedgreatlyfrom the suggestions
and commentsof thosewho haveusedour work in theseways,and of our own undergraduate
and
postgraduate
students,
initiallyat the University
and MacquarieUniversity
of Technology
i n S y d n e yl ,a t e r a t t h e I n s t i t u t eo f E d u c a t i o na n d t h e L o n d o nC o l l e g eo f P r i n t i n gi n
London,andalsoat the TemasekPolytechnic
in Singapore.
W e b e g a no u r w o r k o n v i s u a lc o m m u n i c a t i oinn t h e s u p p o r t i v ea n d s t i m u l a t i n g
environmeo
n ft t h e N e w t o w n
S e m i o t i cC
s i r c l ei n S y d n e yd;i s c u s s i ow
n si t h o u r f r i e n d st ,h e
m e m b e r os f t h i s C i r c l eh, e l p e ds h a p eo u r i d e a si n m o r ew a y st h a nw e c a n a c k n o w l e d g e .
If any two peoplefrom that first periodwereto be singledout, it would be Jim Martin,
who gaveus meticulous,
detailed,extensive
and challenging
commentson severalof the
chaptersof the earlierbook,and FranChristie,who had urgedus to write it. But herewe
wouldalsoliketo makea specialmentionof Bob Hodge,whoseideasappearin this bookin
manyways/evenif not alwaysobviously
so.
0f thosewho usedour book in teachingand research,
and whosecommentson the
e a r l i e rb o o k h a v eh e l p e du s r e t h i n ka n d r e f i n eo u r i d e a s w
, e wouldlike to mention
the research
teamof the Disadvantaged
in
in particularRick
Programme
Sydney,
Schools
Iedema,SusanFeez,PeterWhite, RobertVeeland Sally Humphrey;Staffan Selander,

x .

Preface to the first edition

throughwhoseCentrefor TextbookResearch
in Hdrnosand
our work cameto betakenup
by researchers
in the field of textbookresearchin Swedenand severalother European
countries;the membersof the 'Languageand Science'researchteam at the Institute
o f E d u c a t i o nI s, a b e lM a r t i n s J, o n 0 g b o r na n d l ( i e r a nM c G i l l i c u d d yP;h i l i pB e l l ; B a s i l
B e r n s t e i nP;a u lG i l l e na n dT e u nv a n D i j k .
T h r e ew r i t e r si n f f u e n c eodu r i d e a si n d i f f e r e nat n d f u n d a m e n t awl a y s . O n ei s R o l a n d
B a r t h e sA. l t h o u g h
w e s e eo u rw o r ka s g o i n gb e y o n h
d i ss e m i n awl r i t i n go n v i s u asl e m i o t i c s
in severalways,he remainsa stronginspiration.
Thereis not a subjectin semioticson
w h i c hB a r t h e sh a sn o t w r i t t e no r i g i n a l l ay n d i n s p i r i n g l H
y .e h a sp r o v i d efdo r u s a m o d e l
of what semioticscan be, in the rangeof his interests,
in the depthof his work, and in
h i se n g a g e m ew
n ti t h t h e s o c i a a
l n dc u l t u r aw
l o r l d .E q u a l l ys i g n i f i c a nf ot r u s i s M i c h a e l
H a l l i d a yH. i s v i e w o f l a n g u a g a
e s a s o c i a ls e m i o t i ca, n d t h e w i d e r i m p l i c a t i o nosf h i s
gaveusthe meansto go beyondthe structuralistapproachof 1960sParisSchool
theories,
s e m i o t i c sa,n d o u r w o r k i s e v e r y w h e rien f f u e n c ebdy h i s i d e a s T
. h e nt h e r e i s R u d o l f
Arnheim.The morewe readhis work,the morewe realizethat most of what we haveto
say hasalreadybeensaid by him, often betterthan we havedoneit, albeit it usuallyin
commentaries
worksof art ratherthan in the form of a moregeneraltheory.
on individual
: w o u l dl i k et o c l a i mh i ma s a g r e a t
H e i s c o m m o n lay s s o c i a t ewdi t h G e s t a lpt s y c h o l o gwy e
s o c i asl e m i o t i c i a n .
W ew o u l dl i k et o t h a no u r e d i t o rJ, u l i aH a l l ,f o r h e re n c o u r a g e m eanntd i n v a l u a b lhee l p
i n p r o d u c i ntgh i s b o o k .J i l l B r e w s t ear n d L a u r aL o p e z - B o n i lw
l ae r ei n v o l v e d
in various
s t a g e so f t h e b o o k t; h e i re n c o u r a g e m eanntdh e l pm a d et h ew o r kp o s s i b laen de n j o y a b l e .

Acltnowledgements

P l a t e3 J o s h u aS m i t h b yW i l l i a mD o b e l l1, 9 4 3 , @D A C S2 0 0 4 .
Plate5 Cossacks
by Vassilyl(andinsl<y,
1910-1911.@ ADAGP,Parisand DACS,London
2 0 0 4 .P h o t o g r a p h@
y T a t eL
, o n d o n2 0 0 5 .
P l a t e6 H i s t o r i cC o l o u r sb y C o l i nP o o l er,e p r o d u c ebdy k i n d p e r m i s s i oonf P h o t o W o r d
S y n d i c a t i oLnt d .
P l a t e7 P a l g r a vceo l o u rs c h e m ree p r o d u c ewdi t h p e r m i s s i oonf P a l g r a v M
e acmillan.
1.1+1.13 'My bath' from Baby's First Book by B. Lewis,illustratedby H. Wooley,
c o p y r i g h@
t L a d y b i r dB o o k s1 t d . , 1 9 5 0 .
L 2 B i r d i n t r e e f r o m 0 n M y W a l l gb y D i c k B r u n a ,1 9 8 8 .I l l u s t r a t i o nD i c k B r u n a@
M e r c i sb 4 1 9 7 2 .
, v e ro f N e w s w e eA
1 . 4 M a g a z i n ceo v e rw i t h n a t u r a l i s t ipch o t o g r a p h
co
k ,p r i l 9 , 2 0 0 4 @
2 0 0 4 N e w s w e e kI n, c . P h o t o g r a p b
h y l ( a r i m S a h i b - A F P - G e t It m
a
g
e s .R e p r i n t e db y
y
p e r mi s si o n .
photograph,
1.5 Magazine
12,200I
coverwith conceptual
November
coverof Newsweek,
O 2 0 0 1 N e w s w e eIkn,c .R e p r i n t ebdy p e r m i s s i o n .
1.6 Imagefrom 'lnteractivePhysics'.
Courtesyof MSC Software.
2.5 Communication
modelf rom Watson,
J. and Hill, A. (1980) A Dictionaryof Communicationand MediaStudiegLondon,Arnold,p.I43. Reproduced
by permission
of Hodder
H e a d l i nPeL C .
2 . 6 T w o C o m m u n i c a t i oMno d e l sf r o m W a t s o nJ, . a n d H i l l , A . ( 1 9 8 0 ) A D i c t i o n a r yo f
Communication
of
and MediaStudieqLondon,Arnoldp.I47. Reproduced
by permission
H o d d eH
r e a d l i nPeL C .
2.IO Beatthe Whiteswith the Red Wedgeby El Lissitzky,I9I9-20.@ DACS2004.
( 1 8 7 8 - 1 9 3 5 )" S u p r e m a t i sCt o m p o s i t i o R
2 . 1 1 l ( a s i m i rM a l e v i c h
n :e dS q u a r ea n d B l a c k
) 2 0 0 4 ,D i g i t a il m a g eT, h e
S q u a r e "1, 9 1 4 ,N e wY o r k ,M u s e u mo f M o d e r nA r t ( M o l V l A@
Museum
o f M o d e r nA r t , N e wY o r l < / S c a lFal ,o r e n c e .
2.I7 Gulf War Diagram,SydneyMorning Herald,14 February,1991 reproducedby
permissionof SydneyMorning Herald.
2.18 Speechcircuitfrom Saussure's
F.de Saussure,
Coursein GeneralLinguistics,I9T4,
t r a n s l a t ebdy R o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
h Co.

xii .

Acknowledgements

2 . 2 0 V i t t e la d v e r t i s e m e nr e
t sp r o d u c ebdy k i n dp e r m i s s i oonf N e s t l 6G r o u p .
2.22 Communication
Model from Watson,J. and Hill, A. (1980) A Dictionaryof
Communication
and MediaStudieELondon,Arnoldp.I47. Reproduced
by permission
of
H o d d eH
r e a d l i nPeL C .
2.23 Arctictundrasystem/fi1.7.5, p.1,72f rom Sale,C.,Friedman,
B. and Wilson, G. )ur
ChangingWorld,Book1, PearsonEducationAustralia.Reproduced
by permission
of the
publisher.
2.24 Communication
Modelfrom Watson,J. and Hill, A. (1980) A Dictionaryof Communicationand MediaStudies,
London,Arnoldp.54. Reproduced
by permission
of Hodder
H e a d l i nP
eL C .
3 . 1 G u i d ei n t e r f a c fer o m ' D a n g e r o uCs r e a t u r e s ' , 7 9 9S
4 .c r e e n s h or et p r i n t e d
by permiss i o nf r o m M i c r o s o fC
t orporation.
3 . 2 S e k o n da d v e r t i s e m ernetp r o d u c ebdy k i n dp e r m i s s i oonf S e k o n d a / T i mPer o d u c t s .
3.3 Sourcesof signsfrom Eco,U. (1976) A theoryof semiotics,Bloomington,
Indiana
U n i v e r s i tPy r e s sp, . 1 7 7 .R e p r o d u c ebdy p e r m i s s i oonf t h e p u b l i s h e r .
3.5 Semanticfield diagramfrom Eco, U. (I976) A theoryof semiotics,Bloomington,
IndianaUniversityPress,p.78.Reproduced
of the publisher.
by permission
3 . 7 N e t w o r kf r o m S h a r p l e sM, . a n d P e m b e r t o n , ' R e p r e s e nwt irni tgi n g e
: x t e r n arle p r e s e n tationsand the writing process'inN. Williamsand P. Holt,eds Computers
and Writing.
Reproduced
by kind permission
of IntellectLtd,www.intellectbooks.com
3.13 Resortwear,AustralianWomen'sWeekly,December
1987.@ AustralianWomen's
Weekly/ACP
Reproduced
Syndication.
with permission.
3.19 Electricalcircuitdiagramfrom J. Hill, 1980, IntroductoryPhysics.Reproduced
by
p e r m i s s i oonf T a y l o r& F r a n c i G
s roup.
3 . 2 0 T h e p l a c eo f l i n g u i s t i cosn t h e m a p o f k n o w l e d gfer o m H a l l i d a yM
, . A . l ( .( 1 9 7 8 )
Languageas Social Semiotic,London,Arnold. Reproduced
permission
by
of Hodder
H e a d l i nPeL C .
3.21 'Womenat work', Fig.III-6, p.29,from Pictographs
and Graphs:How to Makeand
Use Themby RudolfModleyand DynoLowenstein.
@ 1952 by Harper& Brothers.
Copyr i g h tr e n e w e 1
d 9 8 0 b y P e t e rM . M o d l e ya n d M a r i o nE . S c h i l l i n gR. e p r i n t ebdy p e r m i s s i o n
o f H a r p e r C o l l i nPsu b l i s h e rI n
sc.
3.28 'Fun with fungi',SydneyMorningHerald,18 June1992, reproduced
by permission
of SydneyMorning Herald.
4 . 2 A T Ms c r e e nr e p r o d u c ebdy k i n dp e r m i s s i oonf N a t i o n aAl u s t r a l i aB a n k .
4.4 The murder of Dr Chang,SydneyMorning Herald,5 July 1991, reproducedby
permissionof SydneyMorning Herald.

Acknowledgements

xiii

4 . 5 P l a y s t a t i owne b s i t er e p r o d u c ebdy k i n dp e r m i s s i oonf S o n yC o m p u t eEr n t e r t a i n m e n t

E u r o p eL t d .
4 . 7 N e wl o o kF o r dM o n d e of r o mw w w . f o r d . c o . u
r ekp r o d u c ebdy l < i n dp e r m i s s i oonf F o r d .
4 . 8 F i e s t a ' R o c ks o l i d ' w e b s i tree p r o d u c ebdy k i n dp e r m i s s i oonf 0 g i l v yG r o u pH o l d i n g s
Ltd and Ford.
4.16 'PrisonGuard'by DannyLyon,1969,fromConversations
with the dead.@ Danny
p
e
r
m
i
s
s
i
o
n
.
L y o n . M a g n uP
mh o t o sR. e p r o d u c ewdi t h
by permission
4.19 Gulfwar map,SydneyMorningHerald,22January1991,reproduced
of SydneyMorning Herald.
by
4.20 An increasein tourism,SydneyMorningHerald,22 January1991, reproduced
permissionof SydneyMorning Herald.
4.22 DeIail from a fourteenth-century
Spanishnativity,from Rudolf Arnheim,Art
and Visual Perception: A Psychologyof the Creative Eye. The New Version.@ I974
T h e R e g e n tos f t h e U n i v e r s i toyf C a l i f o r n i aR. e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf
CaliforniP
a ress.
de Saussure,
5.1 Speechcircuitfrom Saussure's
Coursein GeneralLinguistics,7974,F.
t r a n s l a t ebdy R o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
h Co.
5.2 Schematized
speechcircuit from Saussure's
Coursein GeneralLinguistics,1974,
F . d eS a u s s u r e , t r a n s l abt e
h Co.
yd
R o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
Tim l(oster,ICN,
5.6 Card-players(Van Doesburg,1916-17) photograph/picture:
R i j s w i j k / A m s t e r d aRme.p r o d u c ewdi t h p e r m i s s i oonf I n s t i t u tC o l l e c t i N
e ederland.
DenHaag.
5.7 Composition
9 (YanDoesburg,
I9I7). Collection
of the Gemeentemuseum
R e p r o d u c ewdi t h p e r m i s s i o n .
5 . 8 C o l o u rP r o j e cfto r t h e S c h r o d eR
r e s i d e n c( G
e e r r i tR i e t v e l d1,9 2 3 - 4 )@ D A C S2 0 0 5 .
5 . 9 P h o t o g r a pohf t h e S c h r o d eR
r e s i d e n cGee, r r i tR i e t v e l d@, D A C S2 0 0 4 .
Mountainsby
Investigates:
5.11 @ OxfordUniversityPressfrom The YoungGeographer
T e r r yJ e n n i n g(s0 U P ,1 9 8 6 ) ,r e p r i n t ebdy p e r m i s s i oonf 0 x f o r dU n i v e r s i tPy r e s s .
5.12 Drawing
b y N e w t o nB. y p e r m i s s i oonf t h eW a r d e na n d F e l l o w sN, e wC o l l e g e , O x f o r d ,
a n dT h eB o d l e i a n
L i b r a r yU
, n i v e r s i toyf 0 x f o r d .M S 3 6 1 ,v o l .2 , f o l . 4 5 V .
5.13 Drawingof Stretton'sexperiment,
figure 8.1 (p.141) from The Eye and Brain:
of 0xford
Psychology
by permission
of Seeing5/eby RichardGregory,1998,
reproduced
of Princeton
UniversityPress.Gregory,Richard,Eyeand Brain.Reprinted
by permission
U n i v e r s i tPy r e s s .
AustralianWomen'sWeekly,November
1987.@ AustralianWomen's
6.2 Gold-diggers,
W e e k l y i A CS
Py n d i c a t i o R
n .e p r o d u c ewdi t h p e r m i s s i o n .

xiv .

Acknowledgements

6 . 3 S o n yl Vild d l eE a s t w e b s i tree p r o d u c ebdy k i n dp e r m i s s i oonf S o n yG u l f F Z E .

f r o dmeW
A0D)i c t i o n a r y o f
6.9 Gerbner'scommunicationm
l atson,J.andHill,A.(198
and MediaStudies,London,Arnold.Reproduced
Communication
by permission
of Hodder
H e a d l i nPeL C .
h y B a r o n ,C a m e r aP r e s s ,
6 . 1 0 R o y a lc o u p l e .H . M . T h e Q u e e n ' w
s e d d i n gp, h o t o g r a p b
p
h
o
t
o
g
r
a
p
h
L o n d o nH. . M .T h eQ u e e a
P
r
i
n
c
e
n
d
P
h
i
l
i
p
,
b
y
H
.
R
.
H .P r i n c eA n d r e wC, a m e r a
n
P r e s sL, o n d o nR
. e p r o d u c ewdi t h p e r m i s s i o n .
6.12 Buddhistpaintingfrom RudolfArnheim,Art and VisualPerception:
A Psychology
of the CreativeEye.TheNew Version.@1974
The Regents
of the University
of California.
R e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf C a l i f o r n i P
a ress.
6.13 'Going on holiday'from Prosser,R. (2000) Leisure,Recreationand Tourism,
L o n d o nC, o l l i n sE d u c a t i o n aRl .e p r i n t ebdy p e r m i s s i oonf H a r p e r C o l l i nPsu b l i s h e rLst d @
R .P r o s s e r , 2 0 0 0 .
n o d e lf r o m W a t s o n J, . a n d H i l l , A . ( 1 9 8 0 )
6 . 1 4 A n d e r s c he t a l . ' s c o m m u n i c a t i om
A Dictionary of Communicationand Media Studies,London,Arnold. Reproducedby
p e r m i s s i oonf H o d d e H
r e a d l i nP
e LC.
6.16 Verticaltriptychfrom the websiteof 0xford Universityreproduced
by kind permissionof 0xford University.
6 . 2 3 S c r e e n s h of rt o m C D - R 0 M ' 3 D B o d y A d v e n t u r e 'l,( n o w l e d g A
e d v e n t u r e1, 9 9 3
provided
Adventure,
courtesyof l(nowledge
Inc.
7.1 Roy Lichtenstein,
Big Painting,1965. @ The Estateof Roy Lichtenstein/DACS
2004.
8.I Jacob and the Angel Qacob Epstein,1940) O The Estate of Jacob Epstein/Tate,
L o n d o n2 0 0 5 .I m a g es u p p l i ebdy a n dr e p r o d u c ebdy p e r m i s s i oonf G r a n a d T
a V.
8.2 Peoplein the Wind (l(ennethArmitage,1952). Reproduction
courtesyof l(enneth
ArmitageEstate.Photography
@ Tate,London2005.
8 . 5 W o m a nb y J o a nM i r 6 , I 9 7 O@ S u c c e s s M
i oi r o ,D A C S2 0 0 4 .
8 . 6 L e s H e u r e s d e s T r a c e s ( H o u r o f t h e T r a c e l b y A l b e r t o G i a c1o9m3e0t@
t i ,A D A G P ,
Parisand DACS,London2004. Photography
@ Tate,London2005.
8.7 Jacoband the Angel Qacob Epstein,1940) O The Estateof Jacob Epstein/Tate,
L o n d o n2 0 0 5 .I m a g es u p p l i ebdy a n dr e p r o d u c ebdy p e r m i s s i oonf G r a n a d T
a V.
8 . 8 P l a y m o b 'i fl a m i l ys e t ' a n d ' e t h n i cf a m i l y ' f r o mP l a y m o b icl a t a l o g u eR. e p r o d u c ebdy
k i n dp e r m i s s i oonf P l a y m o b iUl l <L t d .
Figureby HenryMoore,1938.Illustratedon p. 250;hasbeenreproduced
8.9 Recumbent
b y p e r m i s s i oonf t h e H e n r yM o o r eF o u n d a t i o P
n .h o t o g r a p h@
y T a t eL
, o n d o n2 0 0 5 .

Acknowledgements . xu

B . I 0 l ( i n g a n d Q u e e nb y H e n r yl V l o o r e1,9 5 2 - 3 . l l l u s t r a t e d
o n p . 2 5 3 ; h a sb e e nr e p r o d u c e db y p e r m i s s i oonf t h e H e n r yM o o r eF o u n d a t i o P
, o n d o n2 0 0 5 .
n .h o t o g r a p h@
yTateL
8.11 Churchof SantaMaria DellaSpinafrom RudolfArnheim,Art and VisualPerception: A Psychologyof the CreativeEye. The New Version.@ 7974 The Regentsof the
U n i v e r s i toyf C a l i f o r n i aR. e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf C a l i f o r n i P
a ress.
8 . 1 2 C o n n e c t eadn d d i s c o n n e c t enda r r a t i v ep r o c e s sp/p . 8 3 - 8 4 f r o m S . G o o d m a na n d
D. Graddol,RedesigningEnglish- new texts, new identities,London,Routledge,1997.
R e p r o d u c ebdy p e r m i s s i oonf t h e p u b l i s h e r .
8.13 Overshoulder
Inc.
shotin computergame,DeltaForce.Image
courtesyof NovaLogic
@ 2 0 0 4 .A l l r i g h t sr e s e r v e d .
8.14 Dynamicinterpersonal
relationsin the openingsceneol The Big Sleep(Howard
Hawks,1947),pp.9I-92 from S. Goodmanand D. Graddol,Redesigning
English- new
texts,newidentities,
London,Routledge,
ofthe publisher.
l997. Reproduced
by permission

r fvisual
I n t r o d u c t i o nt :h e g r a m m a o
design
. e h e s i t a t e od v e rt h i s t i t l e .
T h es u b t i t l eo f t h i s b o o ki s ' t h e g r a m m a ro f v i s u a ld e s i g n 'W
E x t e n s i o nosf t h e t e r m ' g r a m m a r ' o f t e ns u g g e s t ' r u l e sI'n. b o o k sw i t h t i t l e s l i k e T h e
aboutthe rulesof continuity;
Productionone learns,for instance,
Grammarof Television
' . h a tw e
f rrot mt h e' a m a t e u r W
l < n o w i nt h
g e s er u l e si st h e nw h a ts e t st h e' p r o f e s s i o n a l ' a p a
wish to expressis a little different.In our view,most accountsof visualsemioticshave
' w o r d s ' - w h a t l i n g u i s tcsa l l
c o n c e n t r a t eodn w h a tm i g h tb e r e g a r d eads t h e e q u i v a l e notf
'lexis'- ratherthan 'grammar',andthenon the 'denotative'
the 'iconoand'connotative',
, e i n d i v i d u aple o p l e ,
g r a p h i c a l ' a n d ' i c o n o l o g i c a l ' s i g n i f i coaf nt hceee l e m e n tisn i m a g e st h
, yc o n t r a s t , w e
p l a c e sa n dt h i n g s( i n c l u d i nagb s t r a c t ' t h i n g sd' )e p i c t etdh e r e I. n t h i sb o o k b
on 'grammar' and on syntax,on the way in which theseelementsare
will concentrate
howwords
eescribe
c o m b i n e di n t o m e a n i n g f uwl h o l e sJ. u s t a s g r a m m a r so f l a n g u a g d
' g r a m m a r ' w i l dl e s C r i bt e
h ew a y i n
C o m b i nien C l a u s esse, n t e n c eaSn dt e x t s s, o o u r v i s u a l
'statements'of
e l e m e n t-s p e o p l ep, l a c e sa n dt h i n g s- c o m b i n ei n v i s u a l
w h i c hd e p i c t e d
greateror lessercomplexity
andextension.
to
by comparison
We are by no meansthe first to dealwith this subject.Nevertheless,
or
the studyof visual'lexis',the studyof visual'grammar'hasbeenrelativelyneglected,
from the point of view of art history,or of the
dealt with from a differentperspective,
or with a
of perception,
or the psychology
formal,aestheticdescriptionof composition,
can be usedto attract
the way composition
focuson morepragmaticmatters,for instance
as
e.g.in suchappliedenvironments
the viewer'sattentionto onethingratherthananother,
and in many placesand many
All theseare validapproaches,
advertisingor packaging.
wayswe havemadeuseof the insightsof peoplewritingfrom thesedifferentperspectives.
Yet the resulthas beenthat, despitethe very largeamountof work doneon images,not
are
in theway imageelements
of regularities
muchattentionhasbeenpaidto the meanings
ways.It is this
used- in short,to their grammar- at leastnot in explicitor systematic
andcapturein our book.We intendto
that we seek,aboveall,to describe
focuson meaning
p r o v i d eu s a b l ed e s c r i p t i o nosf m a j o rc o m p o s i t i o nsatlr u c t u r ew
s h i c hh a v eb e c o m e s t a b s t h e c o u r s eo f t h e h i s t o r yo f W e s t e r nv i s u a ls e m i o t i c sa,n d t o
l i s h e da s c o n v e n t i o ni n
image-makers.
analysehowtheyare usedto producemeaningby contemporary
of linguistic
l r a m m a r ' i s t r u ea l s oo f t h e m a i n s t r e a m
W h a t w eh a v es a i da b o u t v i s u a' g
y e e ns t u d i e di n
g r a m m a rg
: r a m m a rh a s b e e n ,a n d r e m a i n s , ' f o r m a lI' t. h a s g e n e r a l l b
f r o m m e a n i n gH. o w e v et[h e l i n g u i s tas n dt h e s c h o ool f l i n g u i s t itch o u g h ft r o m
isolation
o ri ck ho af eHl a l l i d a y w h i c h w e d r a w p a r t oofu r i n s p i r a t i o n - l i n g u i s t s f o l l o w i n g t h e wM
for encoding
havetaken issuewith this view,and see grammaticalforms as resources
i n t e r p r e t a t i o nosf e x p e r i e n caen d f o r m s o f s o c i a l( i n t e r ) a c t i o nB. e n j a m i nL e e W h o r f
. w h a th ec a l l e d' S t a n d t o l a n g u a g ef rso m d i f f e r e nct u l t u r e sI n
a r g u e dt h e p o i n ti n r e l a t i o n
'morning',
'winter','September',
'summer',
terms like
ard AverageEuropean'languages,
' n o o n ' , ' s u n s eat 'r e c o d e da s n o u n sa/ s t h o u g ht h e yw e r et h i n g s H
. e n c et h e s el a n g u a g e s

Introduction

make it possibleto interprettime as something

you can count/use/save/etc. In Hopi,a
N o r t h A m e r i c a nI n d i a nl a n g u a g et h, i s i s n o t p o s s i b l eT.i m e c a n o n l y b e e x p r e s s eads
'subjective
you haveto sav
duration-feeling'.
Youcannotsay'at noon',or'threesummers,.
, 950.
s o m e t h i nlgi k e' w h i l et h es u m m epr h a s ei s o c c u r r i n g( ,W h o r f 1
T h ec r i t i c a ll i n g u i s tos f t h e E a s tA n g l i aS c h o o lw, i t h w h o mo n eo f u s w a sc o n n e c t e d ,
haveshownthat suchdifferentinterpretations
of experience
can alsobe encoded
usingthe
resources
of the samelanguage,
on the basisof differentideological
positions.
TonyTrew
( 1 9 7 9 : 1 0 6 - 7 ) h a sd e s c r i b ehdo w ,w h e nt h e H a r a r ep o l i c e- i n w h a t w a s i n 1 9 7 5 s t i l l
- fired into a crowdof unarmedpeopleand shotthirteenof them,the
Rhodesia
Rhodesia
Heraldwrote,'A politicalclashhas led to death and iniury,,whilethe TanzanianDaily
N e w sw r o f e , ' R h o d e s i aw
' sh i t e s u p r e m a t i spt o l i c e. . . o p e n e df i r e a n d k i l l e dt h i r t e e n
unarmedAfricans.'In otherwords,the politicalviewsof newspapers
are not onlyencoded
t h r o u g hd i f f e r e nvt o c a b u l a r i e( so f t h e w e l l - k n o w 'nt e r r o r i s t ' v s' f r e e d o mf i g h t e r ,t y p e ) ,
but also throughdifferentgrammaticalstructures;that is, throughthe choicebetween
' )r a v e r b( ' k i l l ' ) ,w h i c hf o r i t s g r a m m a t i c a l
c o d i n ga n e v e n ta s a n o u n( ' d e a t h ' , ' i n j u r y o
c o m p l e t i orne q u i r eas n a c t i v es u b j e c(t' p o l i c e 'a) n da n o b j e c t( , u n a r m eA
df r i c a n s , ) .
Grammargoesbeyondformal rulesof correctness.
It is a meansof representing
p a t t e r n so f e x p e r i e n c e . . . . l te n a b l e h
s u m a nb e i n g st o b u i l da m e n t a lp i c t u r eo f
reality,to makesenseof their experience
of what goeson aroundthem and inside
them.
( H a l l i d a y , 1 9 8t50:t )
T h e s a m ei s t r u e f o r t h e ' g r a m m a ro f v i s u a ld e s i g n 'L. i k e l i n g u i s t i sc t r u c t u r e sv,i s u a l
structurespointto particularinterpretations
of experience
andformsof socialinteraction.
To somedegreethesecan also be expressed
linguistically.
Meaningsbelongto culture,
ratherthanto specificsemioticmodes.And the way meanings
are mappedacrossdifferent
semiotim
c o d e st ,h ew a ys o m et h i n g sc a n , f o ri n s t a n c eb,e ' s a i d ' e i t h evri s u a l l yo r v e r b a l l y ,
o t h e r so n l yv i s u a l l ya,g a i no t h e r so n l yv e r b a l l yi s, a l s oc u l t u r a l l ay n dh i s t o r i c a l lsyp e c i f i cI n.
the courseof this book we will constantlyelaborateand exemplifythis point.But even
whenwe can express
what seemto be the samemeanings
in eitherimage-form
or writing
or speech,
they will be realizeddifferently.For instance,
what is expressed
in language
throughthe choicebetweendifferentword classesand clausestructures,may,in visual
communication,
beexpressed
throughthe choicebetween
differentusesof colouror differe n tc o m p o s i t i o nsatlr u c t u r e sA.n dt h i sw i l l a f f e c tm e a n i n gE. x p r e s s i nsgo m e t h i nvge r b a l l y
or visuallm
y a k e sa d i f f e r e n c e .
As for otherresonances
of the term 'grammar'(grammar,asa set of rulesonehasto
obey if one is to speakor write in'correct', sociallyacceptable
ways),linguistsoften
protestthat they are merelydescribing
what peopledo,and that othersinsiston turning
d e s c r i p t i o ni nst or u l e s B
. u t o f c o u r s et o d e s c r i bies t o b e i n v o l v e idn p r o d u c i nkgn o w l e d g e
whichotherswill transformfrom the descriptive
intothe normative,
for instancein educat i o n .W h e na s e m i o t i cm o d ep l a y sa d o m i n a nrto l e i n p u b l i cc o m m u n i c a t i oint s, u s ew i l l
inevitablybe constrainedby rules,rulesenforcedthrougheducation,for instance,
and

I ntroduction

t h r o u g ha l l k i n d so f w r i t t e na n d u n w r i t t e ns o c i a sl a n c t i o n s0.n l y a s m a l le l i t eo f e x p e r i s e c e s s a troy k e e p
r u l e sr e m a i n n
m e n t e r iss a l l o w e dt o b r e a kt h e r u l e s- a f t e ra l l ,b r e a k i n g
i
s
c o m i n gt o b e l e s s
v
i
s
u
a
c
l
o
m
m
u
n
i
c
a
t
i
o
n
p
o
s
s
i
b
i
l
i
t
y
t
h
a
t
o f c h a n g eW
. ebelieve
o p e nt h e
d
o m a i n so f p u b l i c
i
n
t
h
e
c
r
u
c
i
a
l
m
o
r
e
m
o
r
e
a
n
d
a n d l e s st h e d o m a i no f s p e c i a l i s tasn, d
t o m o r ef o r m a l /
a
n
d
r
u
l
e
s
,
a
n
d
m
o
r
e
n
e
w
,
c o m m u n i c a t i oInn.e v i t a b l tyh i s w i l l l e a dt o
s
o c i a ls a n c t i o n s .
t
o
a
t
t
r
a
c
t
b
e
g
i
n
n o r m a t i v et e a c h i n gN
. o t b e i n g ' v i s u a l l Iyi t e r a t e ' w i l l
' V i s u a ll i t e r a c y ' w i lbl e g i nt o b ea m a t t e ro f s u r v i v ael ,s p e c i a l il n
y t h ew o r k p l a c e .
We are well awarethat work suchas ourscan or will helppavethe way for developthe relativefreedomwhich
as constraining
mentsof this kind.Thiscan be seennegatively,
o
f a c e r t a i nm a r g i n a l i z a t i o n
e
x
p
e
n
s
e
a
t
t
h
e
v i s u acl o m m u n i c a t i oh na ss of a r e n j o y e da,l b e i t
greateraccess
to a wider
people
positively,
more
as allowing
by comparison
to writing;or
rulesof
the
Teaching
of
reativity.
in
way
rangeof visualskills.Nor doesit haveto stand the
and
elsewhere,
and
literature
in
language
writinghasnot meanttheendof creativeusesof
j
u
s
t
g
r
a
m
m
a
r
c
r
e
a
t
ively
a
s
t
h
e
Y
e
t
,
t e a c h i n vgi s u a sl k i l l sw i l l n o t s p e l tl h e e n do f t h e a r t s .
writing
grammar
when
use
we
same
employedby poetsand novelistsis, in the end,the
by
l e t t e r sm
, e m o sa n d r e p o r t ss, o t h e ' g r a m m a ro f v i s u a ld e s i g n ' c r e a t i v eel ym p l o y e d
images
producing
layouts,
grammar
attractive
we needwhen
artistsis,in the end,the same
a n dd i a g r a mfso r o u rc o u r s eh a n d o u t sr e, p o r t sb, r o c h u r ecso, m m u n i q u 6asn,ds o o n '
It is worth askingherewhat a linguisticgrammaris a grammarof.fhe conventional
' E n g l i s h ' o r ' D u t c h ' o r ' F r e n c h ' - t hr uel e st h a t
a n s w eirs t o s a yt h a t i t i s a g r a m m aor f
' , u t c ha s ' D u t c h 'a, n ds oo n .A s l i g h t l yl e s sc o n v e n t i o naanl s w e r
h s ' E n g l i s hD
d e f i n eE n g l i s a
w o u l db e t o s a yt h a t a g r a m m a ri s a n i n v e n t o roy f e l e m e n tasn d r u l e su n d e r l y i ncgu l t u r e s h o r t h a ntde r mf o r s o m e s p e c i f i fco r m so f v e r b a cl o m m u n i c a t i o n . ' U n d e r l y i n g ' ihsear e
m
o
h a r e d r eo r l e s sb y m e m b e rosf a
t h i n gm o r ed i f f u s ea n dc o m p l e xm, o r el i k e ' k n o w l e d gs e
g r o u pe, x p l i c i t layn di m p l i c i t l yT' .h i sb r i n g si n s u b t l em a t t e r so f w h a tl < n o w l e d igsea n dh o w
i t i s h e l da n de x p r e s s eadn, da b o v ea l l t h e s o c i aql u e s t i oonf w h a ta ' g r o u p ' i s .T h a tm a k e s
and practices
oneof the knowledges
of grammarvery mucha socialquestion,
definitions
s h a r e db y g r o u p so f p e o p l e .
We might now ask,'What is our "visual grammar" agraffimar of?'First of all we
w o u l ds a yt h a t i t d e s c r i b eass o c i a rl e s o u r coef a p a r t i c u l a9r r o u pi,t s e x p l i c iat n di m p l i c i t
we
and its usesin the practicesof that group.Then,second,
aboutthis resource/
knowledge
encompass
can
that
grammart
a
term
we need
because
wouldsaythat it is a quitegeneral
o i l p a i n t i n ga s w e l l a s m a g a z i n lea y o u tt,h e c o m i cs t r i pa s w e l l a s t h e s c i e n t i f idci a g r a m .
and bearingin mindour socialdefinitionof grammar,
Drawingthesetwo pointstogether,
l Y a m m aorf c o n t e m p o r a rvyi s u a l
w e w o u l ds a y t h a t ' o u r ' g r a m m a r i s a q u i t eg e n e r a g
gn
e dp r a c d e s i g ni n ' W e s t e r n ' c u l t u r easn, a c c o u not f t h e e x p l i c iat n d i m p l i c i tl < n o w l e d a
c
u
l
t
u
r
e-specific
a
u
n
d
e
r
l
y
i
n
g
r
u
l
e
s
co
, n s i s t i nogf t h e e l e m e n tasn d
t i c e sa r o u n da r e s o u r c e
a
s
o
c i aol n e ,
q
u
i
t
e
d
e
f
i
n
i
t
i
o
n
o
u
r
m
a
d
e
deliberately
n .eh a v e
f o r mo f v i s u acl o m m u n i c a t i oW
p
r
a
c
t
i
c
e
s
?
'
a
n
d
f
r
omthere
g
r
o
u
p
?
i
t
s
W h a ta r e
b e g i n n i nwgi t h t h e q u e s t i o n ' W h ai st t h e
says,
which
approach
an
adopting
ratherthan
the grammarat issue,
attemptingto describe
' H e r e i so u r g y a m m a r ; d o t h e p r a c t i c e s a n d l < n o w tl e
hd
i sggersooufp c o n f o r m t o i t o r n o t ? '
from
to visualtext-objects
In the bookwe have,by and large,confinedour examples
, W e s t e r n ' c u l t u raens da s s u m etdh a t t h i sg e n e r a l i z a t i o
hn
a ss o m ev a l i d i t ya s i t p o i n t st o a

Introduction

communicational
situationwith a longhistorythat hasevolvedoverthe pastfivecenturies
or so/alongside
writing (quitedespitethe differences
betweenEuropeanlanguages),
as a
'languageof visualdesign'.
Its boundaries
are not thoseof nation-states,
althoughthere
are,andverymuchso,cultural/regional
variations.
Rather/
this visualresource
hasspread,
alwaysinteracting
with the specificities
of locality,whereverglobalWesterncultureis the
d o m i n a nct u l t u r e .
This means,first of all, that jt is not a'universal'grammar.
Visuallanguageis not despite
a s s u m p t i o tnost h e c o n t r a r y- t r a n s p a r e nat n du n i v e r s a lul yn d e r s t o o idt ;i s c u l t u r ally specific.We hopeour work will continueto providesomeideasand conceptsfor the
studyof visualcommunication
in non-Western
formsof visualcommunication.
To givethe
mostobviousexample,
Westernvisualcommunication
is deeplyaffectedby our convention
o f w r i t i n gf r o m l e f t t o r i g h t ( i n c h a p t e r6 w e w i l l d i s c u s tsh i s m o r ef u l l y ) .T h e w r i t i n g
directionsof culturesvary:from rightto left or from leftto right,from top to bottomor in
circularfashionfrom the centreto the outside.Consequently
differentvaluesandmeanings
are attachedto suchkeydimensions
of visualspace.Thesevaluations
and meanings
exert
their inffuencebeyondwriting, and inform the meaningsaccordedto differentcompositionalpatterns,
the amountof usemadeof them,andso on.In otherwords,we assume
t h a t t h e e l e m e n t s ,u c ha s ' c e n t r e ,o r , m a r g i n ,',t o p , o r ' b o t t o m , ,w i l l p l a y a r o l e i n t h e
visualsemioticsof any culture,but with meaningsand valuesthat are likelyto differ
d e p e n d i nognt h a t c u l t u r e ' hs i s t o r i eosf u s eo f v i s u asl p a c ew, r i t i n gi n c l u d e dT.h e' u n i v e r s a l ,
aspectof meaningliesin semioticprinciples
and processes,
the culture-specific
aspectlies
in their applicationoverhistory,and in specificinstances
of use.Herewe merelywant to
signalthat our investigations
havebeenrestricted,by and large,to Westernvisualcomm u n i c a t i o nE.v e nt h o u g ho t h e r sh a v eb e g u nt o e x t e n dt h e a p p l i c a t i o nosf t h e p r i n c i p l eosf
this grammar,
we makeno specificclaimsfor the application
of our ideasto othercultures.
Within Westernvisualdesign,however,
we believethat our theoryappliesto all formsof
v i s u a cl o m m u n i c a t i oW
n .e h o p et h a t t h e w i d er a n g eo f e x a m p l ews e u s ei n t h e b o o kw i l l
c o n v i n cree a d e ros f t h i sp r o p o s i t i o n .
0 u r s t r e s so nt h e u n i t yo f W e s t e r vni s u acl o m m u n i c a t i o
dn
o e sn o te x c l u dteh ep o s s i b i l i t y
o f r e g i o n aal n ds o c i avl a r i a t i o nT.h eu n i t yo f W e s t e r d
n e s i g ni s n o ts o m ei n t r i n s i fce a t u r eo f
v i s u a l i t yb,u t d e r i v efsr o m a l o n gh i s t o r yo f c u l t u r acl o n n e c t i oann di n t e r c h a n gaes,w e l la s
now from the globalpowerof the Westernmassmediaand cultureindustriesand their
technologies.
In manypartsof the world,Westernvisualcommunication
existssideby side
with localforms.Westernformsmight be used,for instance,
in certaindomainsof public
c o m m u n i c a t i osnu,c ha s p u b l i cn o t i c e ss,i t e so f p u b l i ct r a n s p o r t h
, e p r e s sa, d v e r t i s i nagn, d
t h ev i s u aal r t s ,a sw e l la s i n s o m e w h amt o r e ' p r i v a t e , d o m a i innst ,h e h o m ea, n di n m a r k e t s
a n ds h o p s , f ol rn s t a n c e . O f ttehner e l a t i o ni s h i e r a r c h i c a l , w o
i t nh ef o r mo v e r l a i od n a n o t h e r
( s e eS c o l l o na n dS c o l l o n2, 0 0 3 ;l ( r e s s2, o o 3 ) ,a n do f t e n- a s i n a d v e r t i s i n g ,
f o r i n s t a n c-e
the two are mutuallytransformed
andfused.WhereWesternvisualcommunication
begins
t0 exertpressure
on localforms,therearetransitionalstagesin whichthe formsof thetwo
c u l t u r e sm i x i n p a r t i c u l aw
r a y s I. n l o o k i n ga t a d v e r t i s e m e ni nt sE n g l i s h - l a n g u amgaeg a zinesfrom the Philippines,
for instance,
we were struck by the way in which entirely
conventional
Westerniconographical
elements
wereintegratedinto designsfollowingthe

Introduction

r u l e so f a l o c a lv i s u asl e m i o t i cI n. a d v e r t i s e m e notnst h e I V I T R
i n H o n gK o n gs, o m ea d v e r t i s e m e n tcso n f o r mt o t h e ' E a s t e r n ' d i r e c t i o n a loi ttyh,e r st o t h e W e s t e r ny,e to t h e r sm i x t h e
two. As with the Filipinoadvertisements,
can be 'Western',
discourses
and iconography
'East',
mixedin variouswayswith thoseof the
can/at the sametime,
whilecolourschemes
(as it is, differbe distinctlynon-Western.
The situationthereis in any casecomplicated
ently,in Japan)by the fact that directionalityin the writing systemhas becomecomplicatedin severalways:by the adoption,in certaincontexts,
and
of 'Western'directionality
t h e R o m a na l p h a b eat l o n g s i dteh e c o n t i n u e u
d s eo f t h e m o r et r a d i t i o n adl i r e c t i o n a l i t i e s
andformsof writing.And as economic(andnow oftencultural)poweris re-weighted,
the
trend can go in both or more directions:
the influence
of Asianforms of visualdesignis
b e c o m i nm
g o r ea n d m o r ep r e s e nitn t h e ' W e s t ' S
. u p e r i m p o s oe nd a l l t h i sa r et h e i n c r e a s i n g l yp r o m i n e ndt i a s p o r i c o m m u n i t i e- s o f G r e e k sL, e b a n e s T
e ,u r k so, f m a n yg r o u p so f
t h e I n d i a ns u b c o n t i n e not f, n e w a n d o l d e r C h i n e s e
c o m m u n i t i e(sf o r i n s t a n c eH, o n g
l(ongChinese
aroundthe PacificRim) whichseemingly
affectonlythe membersof this
diaspora,
andyet in realityare havingdeepinfluences
well beyondthem.
W i t h i nE u r o p ei n
, c r e a s i nrge g i o n a l i tcyo u n t e r b a l a n ci ne cs r e a s i nggl o b a l i z a t i oSno. l o n g
asthe European
nationsand regionsstill retaindifferentwaysof life anda differentethos,
t h e y w i l l u s et h e ' g r a m m a ro f v i s u a ld e s i g n ' d i s t i n c t lIyt . i s e a s yf,o r e x a m p l et ,o f i n d
examplesof the contrastinguse of the left and right in the compositionof pagesand
imagesin the Britishmedia.It is harderto findsuchexamples
the Greekor
in,for instance,
the Spanishor the Italian media.as studentsfrom thesecountrieshaveassuredus and
demonstrated
in their worl<- aftertryingto do the assignments
we hadsetthemat home
d u r i n gt h e i r h o l i d a y si n. t h e c o u r s eo f o u r b o o kw e w i l l g i v es o m ee x a m p l eosf t h i s ,f o r
instancein connection
However,we
with newspaper
layoutin differentEuropean
countries.
a r e n o t a b l et o d o m o r et h a nt o u c ho n t h e s u b j e c ta; n dt h e i s s u eo f d i f f e r e n t ' d i a l e c t s ' a n d
' i n f f e c t i o n s ' n e et d
o sb ee x p l o r e m
d o r ef u l l y i n t h ef u t u r e .
In any case,the unity of languages
is a socialconstruct,a productof theoryand of
s o c i aa
l n dc u l t u r a lh i s t o r i e sW. h e nt h e b o r d e r so f ( a ) l a n g u a gaer e n o t p o l i c e db y a c a d emies,and when languages
are not homogenized
by educationsystemsand massmedia,
p e o p l eq u i t ef r e e l yc o m b i n e l e m e n tfsr o m t h e l a n g u a g et h
s e yk n o wt o m a k et h e m s e l v e s
understood
the
M.i x e dl a n g u a g e(s' p i d g i n s 'd) e v e l o pi n t h i sw a y ,a n d i n t i m e c a n b e c o m e
l a n g u a g eo f n e w g e n e r a t i o n(s' c r e o l e s ' )V. i s u a lc o m m u n i c a t i o n o
, t subject o such
policing,has developed
beena
more freely than language,
but there has nevertheless
d o m i n a nlta n g u a g e , ' s p o k e n ' ad ne dv e l o p ei n
d c e n t r e os f h i g hc u l t u r ea, l o n g s i dlee s sh i g h l y
v a l u e dr e g i o n aal n ds o c i a vl a r i a n t s( e . 9 . ' f o l ka r t ' ) . T h ed o m i n a nvt i s u a l a n g u a gies n o w
c o n t r o l l ebdyt h eg l o b a cl u l t u r a l / t e c h n o l o gei cmapl i r e o
s f t h e m a s sm e d i aw, h i c hd i s s e m i n ate the examples
set by exemplary
designers
the spreadof imagebanksand
and,Ihrough
c o m p u t e r - i m a g i tnegc h n o l o g ye,x e r ta ' n o r m a l i z i n gr'a t h e rt h a n e x p l i c i t l y' n o r m a t i v e '
i n f f u e n coen v i s u a cl o m m u n i c a t i oanc r o s st h ew o r l d .M u c ha s i t i s t h e p r i m a r ya i m o f t h i s
b o o kt o d e s c r i bteh e c u r r e n st t a t eo f t h e ' g r a m m a ro f v i s u a d
' , e w i l l a l s od i s c u s s
l e s i g nw
t h e b r o a dh i s t o r i c a ls, o c i a la n d c u l t u r a lc o n d i t i o ntsh a t m a k ea n d r e m a k et h e v i s u a l
'language'.

I ntroduction

A S O C I A LS E M I O T I CT H E O R YO F R E P R E S E N T A T I O N
0ur work on visualrepresentation
is setwithinthetheoretical
frameworkof 'socialsemiotics'.It is importantthereforeto placeit in the contextof the way'semiotics'has
developed
during,roughly,
the past75 years.In Europe,
threeschoolsof semiotics
appliedideasfrom
t h e d o m a i no f l i n g u i s t i ctso n o n - l i n g u i s tm
i co d e so f c o m m u n i c a t i oTnh. e f i r s t w a s t h e
P r a g u eS c h o ool f t h e 1 9 3 0 sa n de a r l y1 9 4 0 s I. t d e v e l o p et h
d ew o r ko f R u s s i a F
normalists
by providing
it with a linguistib
c asisN
. o t i o n ss u c ha s ' f o r e g r o u n d i nw
g ,e r ea p p l i e dt o
l a n g u a g(ee . 9t.h e ' f o r e g r o u n d i nfgo'r,a r t i s t i cp u r p o s eosf/ p h o n o l o g i coarl s y n t a c t ifco r m s
t h r o u g h' d e v i a t i o n ' f r o m
standard
f o r m s f, o r a r t i s t i cp u r p o s e sa)s w e l l a s t o t h e s t u d yo f
a r t ( M u k a r o v s k yt )h,e a t r e( H o n z l )c, i n e m a( J a k o b s o na)n dc o s t u m e
( B o g a t y r e vE
) .a c ho f
t h e s es e m i o t iscy s t e mcso u l df u l f i l t h es a m ec o m m u n i c a t i fvuen c t i o n(st h e ' r e f e r e n t i aaln, d
t h e ' p o e t i c ' f u n c t i o n sT)h. es e c o n d
w a st h e p a r i sS c h o ool f t h e 1 9 6 0 sa n d 1 9 7 0 s w
, hich
a p p l i e di d e a sf r o m d e s a u s s u r e
a n d o t h e r I i n g u i s ttso p a i n t i n g( S c h e f e r )p, h o t o g r a p h y
( B a r t h e sL, i n d e l < e n fsa) s, h i o n( B a r t h e s )c, i n e m a( M e t z ) ,m u s i c( N a t t i e z )c, o m i cs t r i p s
( F r e s n a u l t - D e r u e lel e
t c) .,T h ei d e a sd e v e l o p ebdy t h i s S c h o oal r e s t i l l t a u g h ti n c o u n t l e s s
coursesof mediastudies,
art anddesign,
etc.,oftenunderthe heading,semiology,,
despite
the fact that they are at the sametime regardedas havingbeenovertakenby postp a r o l e ,t ;h e ' s i g n i f i e r ,
structuralism
E .v e r y w h e rset u d e n tasr e l e a r n i n ag b o u t ' l a n g u e ' a n, d
'signified',''arbitrary'
,motivated,
,icons,,
,indexes,
andthe
and
signs;
and'symbols,(these
t e r m sc o m ef r o m t h e w o r k o f t h e A m e r i c a np h i l o s o p h e
a rn ds e m i o t i c i aCnh a r l e sS a n d e r s
Peirce,but are often incorporated
in the frameworkof 'semiology,),
and so on. Generally
this happens
withoutstudentsbeinggivena senseof, or accessto, alternativetheoriesof
s e m i o t i c(so r o f l i n g u i s t i c sW
) .ew i l l c o m p a r e
a n dc o n t r a stth i s k i n do f s e m i o t i cws i t h o u r
o w na p p r o a c hi n, t h i s i n t r o d u c t i oans w e l la s e l s e w h e ri en t h e b o o k .T h i st h i r d ,s t i l lf f e d g l i n g ,m o v e m e ni tn w h i c hi n s i g h t sf r o m l i n g u i s t i chsa v eb e e na p p l i e dt o o t h e rm o d e so f
representation
hastwo sources/
bothdrawingon the ideasof MichaelHalliday,
onegrowi n g o u t o f t h e ' c r i t i c a l L i n g u i s t i c s ' oaf g r o u po f p e o p l ew o r k i n gi n t h e 1 9 7 0 sa t t h e
U n i v e r s i toyf E a s tA n g l i a l,e a d i n g
t o t h e o u t l i n eo f a t h e o r yt h a t m i g h te n c o m p a sost h e r
semioticmodes(Hodgeand l(ress),the other,in the later 1990s,as a development
of
H a l l i d a y asny s t e m i c - f u n c t i olni nagl u i s t i cbsy a n u m b e or f s c h o l a r isn A u s t r a l i ai n
, semioti c a l l y o r i e n t e ds t u d i e so f l i t e r a t u r e( T h r e a d g o l dT,h i b a u l t )v, i s u a ls e m i o t i c s( 0 , T o o l e ,
o u r s e l v e sa )n dm u s i c( v a nL e e u w e n ) .
. u r b o o ki s a b o u ts i g n sT h ek e yn o t i o ni n a n ys e m i o t i ciss t h e ' s i g n ' 0
o r ,a s w e w o u r o
r a t h e rp u t i t , a b o u ts i g n - m a k i nW
g .e w i l l b e d i s c u s s i nf o
g r m s( ' s i g n i f i e r ss' )u c ha s c o l o u r ,
perspective
and line,as well as the way in whichtheseformsare usedto realizemeanings
( ' s i g n i f i e d si 'n) t h e m a k i n go f s i g n sB
. u t o u r c o n c e p t i oonf t h es i g nd i f f e r ss o m e w h af rt o m
that of 'semiology',
andwe wishthereforeto comparethe two viewsexplicitly.
In doingso
we usethe term 'semiology'to referto the way in whichthe ParisSchoolsemioticsis
g e n e r a l ltya u g h ti n t h e A n g l o - S a x owno r l d t, h r o u g ht h e m e d i a t i o on f i n f f u e n t i at el x t b o o k s
s u c ha s t h e s e r i e so f m e d i as t u d i e st e x t b o o k e
s d i t e db y J o h n F i s k e( F i s k ea n d H a r t l e y ,
r 9 7 9 ; D y e r , r 9 B 2F; i s l < e , r 9 B 2
H;a r i l e y , 1 9 B 20;' s u l l i v a ne t a \ . , 1 9 9 3 )I.n d o i n gt h i sw e d o
not seekto repudiatethosewho went beforeus.We seea continuitybetweentheir work

I ntroduction

that of the
whichechoes
andours,as shouldbeclearfrom our maintitle, ReadingImages,
(Fiskeand Hartley,I979).
first volumein Fiske'sseries,ReadingTelevision
.e
o f w h a tw e u n d e r s t a nbdy ' s i g n - m a k i n gT' h
W e w o u l dl i k et o b e g i nw i t h a n e x a m p l e
boy.Sitting on his father'slap,he
drawingin figure 0.1 was made by a three-year-old
t a l k e da b o u t h ed r a w i n ga s h ew a sd o i n gi t : ' D o y o uw a n tt o w a t c hm e ?I ' l l m a k ea c a r . . .
gottwowheels...andtwowheelsattheback...andtwowheelshere...that'safunny
w h e e l . . . . ' W h e nh e h a df i n i s h e dh ,e s a i d , ' T h i iss a c a r . ' T h i sw a st h e f i r s tt i m eh e h a d
Howwasthis a car?0f coursehe had
nameda drawing,andat first the namewaspuzzling.
p r o v i d e tdh e k e y h i m s e l f : ' H e r e 'as w h e e l . A
' c a r ,f o r h i m ,w a s d e f i n e db y t h e c r i t e r i a l
focusedon this aspect.What he
of 'havingwheels',and his representation
characteristic
t o c h o o s feo r t h r e e s ' h. e e l sa r ea p l a u s i b lcer i t e r i o n
r e p r e s e n t ewda s ,i n f a c t , ' w h e e l n e sW
year-olds,
and the wheel'saction/on toy cars as on reai cars/is a readilynoticedand
interestin carswas,for him,most
feature.In otherwords,thisthree-year-old's
describable
. h e e l si ,n t u r n ,a r e m o s t
p l a u s i b lcyo n d e n s ei nd t oa n de x p r e s s eads a n i n t e r e sitn w h e e l sW
of
and because
plausiblyrepresented
of their visualappearance
by circles,both because
s c t i o no f ' g o i n gr o u n d
t h e c i r c u l a rm o t i o no f t h e h a n di n d r a w i n g / r e p r e s e ntthi negw h e e l ' a
a n dr o u n d ' .
in whichthe mal<ers
as a process
To gatherthis up for a moment,weseerepresentation
objector entity,
some
of
a
representation
seek
to
make
child
or
adult,
signs,
whether
of
, t t h e p o i n to f
w h e t h e rp h y s i c aol r s e m i o t i ca, n d i n w h i c ht h e i r i n t e r e s itn t h e o b j e c t a
s
o
c
i aal n dp s y c h o c
u
l
t
u
r
a
l
,
o
f
t
h
e
o
u
t
o
n
e
a
,
r
i
s
i
n
g
i
s
a
c
o
m
p
l
e
x
r
e
p
r
e
s
e
n
t
a
t
i
o
n
,
m a k i n gt h e
w
h i c ht h e s i g n i
n
c
o
n
t
e
x
t
s
p
e
c
i
f
i
b
y
t
h
e
f
o
c
u
s
e
d
s
i
g
n
m
a
k
e
a
r
n
,
d
h
i
s
t
o
r
y
o
f
t
h
e
logical
o
f
w
h
a
ti s s e e na st h e
p
r
o
d
u
c
e
s
t
h
e
s
e
l
e
c
t
i
o
n
s
o
u
r
c
e
o
f
i
s
t
h
e
s
i
g
n
T
.
h
a
t
'
i
n
t
e
r
e
s
t
'
t
h
e
maker
as
adequately
regarded
is
then
aspect
criterial
and
this
aspect
of
the
object,
criterial
'wholeobject'
the
given
it
is
never
words,
In
other
context.
object
in
a
of
the
representative
whichare represented.
but onlyeverits criterialaspects
at the moment
in what seemsto the sign-maker,
represented
aspects
are
Thesecriterial
p
l a u s i b lree p r e a
n
d
p
l
a
u
s
i
b
l
e
m
o
s
t
a
p
t
t
h
e
f
a
s
h
i
o
n
a
,
n
d
m
o
s
t
a
p
t
a
n
d
t hge,
of sign-mal<in
(
e
.
g
.
trhsu s ' h a v e ' a
p
a
i
n
t
i
n
g
,
S
i
g
n
m
a
k
e
s
p
e
e
c
h
)
.
L
e
g
o
b
l
o
c
k
s
,
d
r
a
w
i
n
g
,
s e n t a t i o n aml o d e

co
O

child
Fig 0.1 Drawingby a three-year-old

Introduction

meaning/
the signified,
whichtheywishto express,
andthenexpressit throughthe semiotic
mode(s)that make(s)availablethe subjectively
felt, mostplausible,
mostapt form,as the
s i g n i f i eTr .h i sm e a n s t h aitn s o c i asl e m i o t i c s t hsei g ni s n o t t h ep r e - e x i s t i nc g
o n j u n c t i oonf a
signifieranda signified,
a ready-made
signto be recognized,
chosenandusedas it is,in the
waythat signsare usuallythoughtto be 'availablefor use'in ,semiology,.
Ratherwe focus
on the processof sign-making,
in which the signifier(the form) and the signified(the
meaning)are relativelyindependent
of eachotheruntil they are broughttogetherby the
sign-maker
in a newlymadesign.To put it in a differentway,usingthe examplejust above,
the processof sign-makingis the processof the constitutionof a sign/metaphor
in two
s t e p s : ' ac a r i s ( m o s tl i k e )w h e e l s ' a n d ' w h e ealrse ( m o s tl i k e )c i r c l e s , .
Puttingit in our terms:the sign-maker's
interestat this momentof sign-making
has
s e t t l e do n ' w h e e l n e s s ' atsh e c r i t e r i a lf e a t u r eo f ' c a r ' . H e c o n s t r u c t sb,y a p r o c e s o
sf
'wheel'is aptlyrepresented
analogy,
two metaphors/signs:
first,the signified
by the signifier
' c i r c l e ' t om a k et h e m o t i v a t e d
s i g n ' w h e e ls' ;e c o n dt h
, e s i g n i f i e d ' c a ri s' a p t l yr e p r e s e n t e d
by the signifier'many wheels'to makethe motivatedsign 'car'. The resultingsign,the
drawingglossed'this
is a car', is thusa motivatedsignin that eachconjunction
of signifier
and signifiedis an apt, motivatedconjunctionof the form which best reoresents
that
w h i c hi s t o b e m e a n tT. h i ss i g ni s t h u st h e r e s u l to f a d o u b l em e t a p h o r ipcr o c e sisn w h i c h
analogyis the constitutive
principle.
Analogy,inturn,is a process
of classification:
x is like
y ( i n c r i t e r i aw
l a y s )W
. h i c hm e t a p h o r(sa n d , ' b e h i n d ' t hmee t a p h o rw
s ,h i c hc l a s s i f i c a t i o n s )
carrythe day and passinto the semioticsystemas conventional,
and thenas naturalized,
andthenas'natural',neutralclassifications,
is governed
by socialrelationsof power.Like
adults,childrenare engagedin the construction
of metaphors.
Unlikeadults,they are,on
the one hand,lessconstrictedby cultureand its already-existing
and usuallyinvisible
metaphors,
but, on the otherhand,usuallyin a positionof lesspower,so that their metaphorsare lesslikelyto carrythe day.
It followsthat we seesignsas motivated- not as arbitrary-conjunctionsof signifiers
( f o r m s )a n ds i g n i f i e d(sm e a n i n g sI)n. ' s e m i o l o g y ' m o t i v a t i iosnu s u a l l yn o t r e l a t e dt o
the
act of sign-making
as it is in our approach,but definedin termsof an intrinsicrelation
b e t w e etnh e s i g n i f i ear n dt h e s i g n i f i e dI t. i s h e r et h a t p e i r c e ' s ' i c o n ' , , i n d eaxn,d ' s y m b o l ,
m a k et h e i r a p p e a r a n cien/c o r p o r a t ei d
n t o ' s e m i o l o g y ' ian w a y w h i c hi n f a c t c o n t r a d i c t s
s o m eo f t h e k e yi d e a si n P e i r c e 'sse m i o t i c sT.h e ' i c o n 'i s t h e s i g ni n w h i c h, t h es i g n i f i e r signifiedrelationshipis one of resemblance,
likeness'(Dyer,r9g2: r24) - i.e. objective
likeness,
ratherthananalogymotivatedby'interest',establishes
the relation.The,index,is
the sign in which'thereis a sequential
or causalrelationbetweensignifierand signified,
(Dyer,1982: 125);that is,a logicof inference,
ratherthananalogymotivatedby ,interest,.
Thethird term in the triad,'symbol',by contrast,is relatedto signproduction,
as it'rests
on convention,
or "contract" ' (Dyer,r9g2:125), but this veryfact makesit ,arbitrary,,
'unmotivated'/
a caseof meaningby decreeratherthan of activesign-making.
I n o u r v i e ws i g n sa r e n e v e a
r r b i t r a r ya, n d ' m o t i v a t i o n ' s h o ubl e
df o r m u l a t e idn r e l a t i o n
to the sign-maker
andthe contextin whichthe signis produced,
and not in isolationfrom
the act of producing
analogies
andclassifications.
Sign-makers
usetheformstheyconsider
apt for the expression
of their meaning,
in anymediumin whichtheycanmakesigns.when

I ntroduction

childrentreat a cardboardbox as a pirate ship,they do so becausethey considerthe

of the meaningthey havein mind
materialform (box)an apt mediumfor the expression
(pirateship),and because
of the criterialaspectsof pirateships(conof their conception
s f sign-making
A.l l
t a i n m e n tm
, o b i l i t ye, t c . ) .L a n g u a g ies n o e x c e p t i otno t h i s p r o c e s o
y a n n e ri n t h e e x p r e s s i oonf m e a n i n g .
l i n g u i s t ifco r m i s u s e di n a m e d i a t e dn,o n - a r b i t r a rm
yearsthereis bothmoreand lessfreedomof expresFor childrenin their early,pre-school
sion:more,because
theyhavenot yet learnedto confinethe makingof signsto the culturconventhey are unawareof established
ally and sociallyfacilitatedmedia,and because
they do not have
in the makingof signs;less,because
tionsand relativelyunconstrained
boy,
suchrich culturalsemioticresources
availableas do adults.So whena three-year-old
by not havingthe
labouring
to climba steephill,says,'Thisis a heavyhill',heis constrained
of
Thesameis the casewith the resources
word'steep'asan availablesemioticresource.
syntacticandtextualforms.
' H e a v y 'i,n ' h e a v yh i l l ' ,i s ,h o w e v ear ,m o t i v a t esdi g n t: h ec h i l dh a sf o c u s e o
dnparticular
a s p e c tos f c l i m b i n ga h i l l ( i t t a k e sa l o t o f e n e r g yi;t i s e x h a u s t i n ga)n d u s e sa n a v a i l a b l e
Theadultwho correctsby
form whichhe seesas apt for the expression
of thesemeanings.
offering'steep'('Yes,it's a verysteephill') is,from the child'spointof view,not so much
whichhe hadgivento'heavy'
for the precisemeaning
offeringan alternative
as a synonym
in that context.Boththe childand the parentmakeuseof 'what is available';it happens
on this is to missthe central
that differentthingsare available
to each.But to concentrate
a s p e cot f s i g n - m a k i negs, p e c i a l tl yh a t o f c h i l d r e n . ' A v a i l a b i l i tsyn' o t t h e i s s u eC. h i l d r e n ,
Theyare not'acquired',but made
likeadults,maketheir own resources
of representation.
by the individual
sign-maker.
o t h et e r m s ' l a n g u e '
I n ' s e m i o l o g yc'o, u n t l e ssst u d e n tasc r o s tsh ew o r l da r e i n t r o d u c et d
for instance,
as'the abstractpotentialof a language
and'parole',with 'langue'explained,
system. . . the sharedlanguagesystemout of which we make our particular,possibly
(O'sullivanet a|.,1983:t27) or,in our terms,as a systemof available
statements'
unique,
ds:
, dw i t h ' p a r o l ed' e f i n e a
f o r m sa l r e a d yc o u p l e tdo a v a i l a b lm
e e a n i n gas n
of the potentialof langue.. . .
an individualutterance
that is a particularrealization
By extension
we can arguethat the total systemof televisionand film conventions
andpractices
constitutesalangue,
andthe waytheyare realizedin eachprogramme
or filma parole.
( 0 ' S u l i v a ne t a l . ,l 9 B 3 i I 2 7 )
W e c l e a r l yw o r kw i t h s i m i l a rn o t i o n sw, i t h ' a v a i l a b lfeo r m s 'a n d' a v a i l a b lcel a s s i f i c a t i o n s '
( ' l a n g u e 'a) n di n d i v i d u aalc t so f s i g n - m a k i n(g' p a r o l e ' )a,n dw ea g r e et h a ts u c hn o t i o n cs a n
But for usthe ideaof'potenusefullybe extended
to semioticmodesotherthan language.
tial' (whatyoucan meanand howyoucan'say' it, in whatevermedium)is not limitedby a
e e a n i n g s ' c o u p lw
, n dw e w o u l dl i k et o u s ea
s y s t e mo f ' a v a i l a b l m
e idt h ' a v a i l a b lfeo r m s ' a
a semiotic'potential'is definedby the semioticresources
slightlylessabstractformulation:
availableto a specificindividualin a specificsocialcontext.0f course,a descriptionof
and manycontexts.
of manyspeakers
semioticpotentialcan amalgamate
the resources

10

Introduction

B u t t h e r e s u l t i n 'gl a n g u e(' t h e l a n g u eo f ' E n g l i s h ' o ro f ' w e s t e r nv i s u a d

l e s i g n ,i)s i n t h e
end an artefactof analysis.
What exists,and is thereforemorecrucialfor understanding
representation
andcommunication,
arethe resources
available
to realpeoplein realsocial
c o n t e x t sA.n d i f w e c o n s t r u cat ' l a n g u e 'a, m e a n i n g
p o t e n t i aflo r ' W e s t e r nv i s u a d
l esign,,
then it is no moreand no lessthan a tool which can serveto describea varietyof signmakingpractices,
within boundaries
drawnby the analyst.It followsthat we would not
d r a wt h e l i n eb e t w e e n ' l a n g u e ' a n d ' p a r o l es' ahsa r p l ya s i t i s u s u a l l yd o n e .D e s c r i b i nag
' l a n g u e ' i sd e s c r i b i nag
s p e c i f i cs e t o f s e m i o t i cr e s o u r c easv a i l a b l e
for communicative
a c t i o nt o a s p e c i f isco c i agl r o u p .
Hereare someantecedents
of the car drawing.Figure0.2 is a drawingmadeby the
s a m ec h i l ds, o m et e n m o n t h se a r l i e rI .t s c i r c u l a rm o t i o ni s e x p r e s s i voef t h e c h i l d ' se x u b e r a n t ,e n t h u s i a s tai cn de n e r g e t iacc t i o n si n m a k i n gt h e d r a w i n gI.n f i g u r e0 . 3 ,m a d ea b o u t
three monthslater,the circular motion has becomemore regular.The exuberance
and
e n e r g ya r es t i l l t h e r eb, u t t h ed r a w i n gh a sa c q u i r e m
d o r er e g u l a r i t ym, o r ei n t e r e si tn s h a p e :
' c i r c u l a rm o t i o n 'i s b e g i n n i n g
t o t u r n i n t o' c i r c l e ' I. n o t h e rw o r d st,h e m e a n i n gosf f i g u r e

Fig 0.2 Drawingby a two-year-old

chitd

I ntroduction

II

Fig0,3 Drawingbyatwo-yeaholdchild

yetwith significant
figure0.3 gathersup,
continuity:
0.2 persistin figure0.3,transformed,
of figure0.2, andthentransformsandextendsthem.
so to speak,the meanings
sheet,onecircleto
showsa seriesof circles,eachdrawnon a separate
Figure0.4,finally,
eachsheet.The movementfrom figure 0.2 to figure 0.4 is clear enough,as is the conwork doneby the childovera periodof fourteenmonths(figure
ceptualandtransformative
the drawingsshowhowthe child
0.4 datesfrom the sameperiodas figure0.1). Together
suchan
to him,andwhy circlesseemed
representational
resources
available
developedthe
physicality
of the motionof figure0.2 persisted
energetic
apt choiceto him:the expressive,
sothat the circularmotionremained
resource/
this representational
as the childdeveloped
part of the meaningof circle/wheel.
was addedas well:the transformation
But something
from the
of the child'ssubjectivity,
resources
wasalsoa transformation
of representational
physicalandexpressive
in the act of representing'circular
expressed
disposition
emotional,
in the act of representexpressed
andcognitivedisposition
motion'tothe moreconceptual
i n ga ' c a r ' .
, k et h e i r ' o w n ' r e p r e s e n t a t i orneaslo u r c easn, dd o s oa s
C h i l d r e nl i,k ea l l s i g n - m a k e rms a

12 .

Introduction

Fig 0.4 Drawingby a three-year.old

child

part of a constantproductionof signs,in which previouslyproducedsignsbecomethe

signifier-material
to be transformedinto new signs.This processrestson lhe interestof
productivestancetowardssign-making
sign-makers.
Thistransformative,
is at the same
- a notionfor whichtherewas little
time a transformation
of the sign-makers'subjectivity
place in a 'semiology'which describedthe relationbetweensignifiersand signifiedsas
restingon inference
or objectiveresemblance,
or on the decrees
of the social'contract'.
We haveusedchildren'sdrawingsas our examplebecause
we believethat the producp r o v i d etsh e b e s tm o d efl o r t h i n k i n ga b o u ts i g n - m a k i nIgt .a p p l i e s
t i o n o f s i g n sb y c h i l d r e n
also to fully socializedand acculturatedhumans,with the exceptionof the effectsof
'convention'.
As maturemembersof a culturewe haveavailablethe culturallyproduced
semioticresources
andconstraints
which
of our societies,
andareawareof the conventions
are sociallyimposedon our making of signs.However,
as we havesuggested,
in our
approachadult sign-makers,
too,are guidedby interest,by that complexcondensation
of
c u l t u r a la n d s o c i a lh i s t o r i e a
s n d o f a w a r e n e sosf p r e s e nct o n t i n g e n c i e s . ' M a t u r e ' s i g n makersproducesignsout ol that interest,alwaysas transformations
of existingsemiotic
materials,
thereforealwaysin somewaynewlymade,andalwaysas motivatedconjunctions
of meaningandform.Theeffectof convention
is to placethe pressure
of constantlimitations of conformityon sign-making;
that is, the way signifiershavebeencombinedwith
signifieds
in the historyof the culture,actsas a constantlypresentconstrainton how far
one might movein combiningsignifierswith signifieds.
doesnot negatenew
Convention
making;it attemptsto limit andconstrainthe semioticscopeof the combinations.
T h i s , t h e ni s, o u r p o s i t i o n
v i s - d - v i s ' E u r o p e a n ' s e m i o lwohgeyr: ed e S a u s s u rhea d( b e e n
assumed
to have)saidthatthe relationof signifierandsignifiedin the signis arbitraryand
conventional,
we wouldsaythat the relationis alwaysmotivatedand conventional.
Where
placedsemioticweightand powerwith the social,we wishto assertthe
he had seemingly
yetalsothe constantpresence
effectsof the transformative
roleof individual
agents,
of the
social:in the historicalshapingof the resources,
in the individualagent'ssocialhistory,in

Introduction

in whichrepresenin the effectof the environment

of presentconventions,
the recognition
along
actionof individuals,
happen.Yet it is the transformative
tationandcommunication
possible
and mal<es
the resources,
whichconstantlyreshapes
the contoursof socialgivens,
of socialsubjects.
the self-making
insightsof sociallyorientedtheoriesof languageis
0ne of the now taken-for-granted
ofthis variation
the variationof language
with the variationof socialcontext.Theaccounts
form x relatesto socialcontexty') to determindiffer,rangingfrom correlation('language
dy s o c i a a
l c t o r sy o r i n s o c i a cl o n t e x yt ' ) . A s o c i a l
a t i o n( ' l a n g u a gfeo r m x i s p r o d u c e b
s e m i o t i ac p p r o a ctha k e st h e l a t t e rv i e wa, l o n gt h ef o l l o w i n gl i n e s .
( 1 ) C o m m u n i c a t i or n
s a k et h e i r m e s s a g em
s a x i m a l l yu n d e r e q u i r e st h a t p a r t i c i p a n tm
which
standablein a particularcontext.Theythereforechooseforms of expression
t o t h e rp a r t i c i p a n t s . 0tnh e o t h e rh a n d ,
t h e yb e l i e v e
t o b e m a x i m a l l tyr a n s p a r e nt o
markedby power
whichare inevitably
communication
takesnlacein socialstructures
the notionof'maximal
differences,
andthis affectshoweachparticipantunderstands
understanding'.
Participantsin positionsof powercan forceother participantsinto
is theregreatereffortsof interpretation,
andtheir notionof'maximalunderstanding'
that
fore differentfrom that of participantswho do their bestto producemessages
s ho,
w i l l r e q u i r ea m i n i m a le f f o r t o f i n t e r p r e t a t i oonr, f r o m t h a t o f p a r t i c i p a n tw
that are
system,producemessages
throughlack of commandof the representational
h a r d e rt o i n t e r p r e(te . g .c h i l d r e nl e
, a r n e r os f a f o r e i g nl a n g u a g eT) .h eo t h e rp a r t i c i or refuse
pantsmaytheneithermakethe effort requiredto interpretthesemessages
to do so,whetherin a schoolor in a railwaystationin a foreigncountry.
( 2 ) Representation
of what
chooseforms for the expression
requiresthat sign-makers
t h e yh a v ei n m i n df,o r m sw h i c ht h e ys e ea s m o s ta p t a n dp l a u s i b lien t h eg i v e nc o n t e x t .
The examplesaboveinstantiatethis: circlesto standfor wheels,and wheelsto stand
for cars; heavyto stand for significanteffort, and significanteffort to stand for
climbinga steeps/ope.Speakersof a foreign languageuseexactlythe samestrategy.
of what
Theychoosethe nearest,most plausibleform they knowfor the expression
t h e yh a v ei n m i n d T
. h er e q u i r e m e not sf c o m m u n i c a t i oanr e n o d i f f e r e nitn m o r eu s u a l
at the
circumstances,
they are simply lessapparent.The interestof sign-makers,
of the
momentof makingthe sign,leadsthemto choosean aspector bundleof aspects
whatthey
as beingcriterial,at that moment,for representing
objectto be represented
the most apt form for lts
want to represent,
and then choosethe most plausible,
withinwhich
representation.
Thisappliesalsoto the interestof the socialinstitutions
and thereit takesthe form of the (historiesof) conventions
messages
are produced,
andconstraints.

APPLICATIONS
of our work,but our
In the previous
sectionwe havefocusedon thetheoreticalbackground
andpractical.We seekto developa
Theyare alsodescriptive
aimsare not just theoretical.

14 .

Introduction

descriptive
frameworkthat can be usedas a tool for visualanalysis.
Sucha tool will have
its usefor practicalas well as analyticaland criticalpurposes.
To givesomeexamples
of
the former,educationalists
everywhere
havebecomeawareof the increasing
roleof visual
c o m m u n i c a t i oi nnl e a r n i n m
g a t e r i a losf v a r i o u ks i n d sa, n dt h e ya r ea s k i n gt h e m s e l v e
wsh a t
kind of maps,charts,diagrams,picturesand forms of layoutwill be most effectivefor
learning.
To answerthis questionthey needa language
for speakingaboutthe formsand
m e a n i n gosf t h e s ev i s u a l e a r n i n m
g a t e r i a l sW. i t h i nt h e m e d i av, i s u adl e s i g ni s l e s sa n dl e s s
the provinceof specialists
who had generally
seenlittle needfor methodicaland analytically explicitapproaches,
and had reliedinsteadon creativesensibilities
honedthrough
- as is the case.for
experience.
But wheremediaformsare relativelyrecentlyintroduced
example,
with advertising
in EasternEuropeandpartsof Asia- thereis no suchresistance
to combiningsystematic
analysisand practice.And with the advanceof easyto usesoftwarefor desktoppublishing,
the productionof diagramsand charts,imagemanipulation,
etc.,visualdesignbecomeslessof a specialistactivity,somethingmany peoplewill do
alongside
otheractivities.
Thishasalreadyledto rapidgrowthin the numberof coursesin
t h i s a r e a- a n d d e s i g n i nsgu c hc o u r s e rse q u i r e m
s o r eo f a n a n a l y t i c agl r a s po f p r i n c i p l e s
than learningon the job by exampleandosmosis.
Last,and maybeat bottomat the root of
m u c ho f t h i sc h a n g ei s, ' g l o b a l i z a t i o n
w'h, i c h- m a y b en e a r l yp a r a d o x i c a -l l yd e m a n dtsh a t
the culturalspecificities
of semiotic,
social,epistemological
andrhetoricaleffectsof visual
communication
mustbe understood
everywhere,
sincesemioticentitiesf rom anywnere
now
appearandare'consumed'
everywhere.
A n a l y s i n vgi s u a lc o m m u n i c a t i oi sn, o r s h o u l db e ,a n i m p o r t a n p
t art of the 'critical'
d i s c i p l i n eAs l.t h o u g hi n t h i s b o o kw e f o c u so n d i s p l a y i ntgh e r e g u l a r i t i eosf v i s u a lc o m munication,
ratherthan its ('interested',
i.e.political/ideological)
uses,we seeimagesof
whateverkind as entirelywithinthe realmof the realizations
and instantiations
of ideology/as means- always- for the articulationof ideological
positions.
Theplainfact of the
matteris that neitherpowernor its usehasdisappeared.
It hasonlybecomemoredifficult
to locateand to trace.In that contextthere is an absoluteneedin democratictermsfor
makingavailablethe meansof understanding
the articulationsof poweranywhere,
in any
f o r m .T h es t i l l g r o w i n ge n t e r p r i soef ' c r i t i c a ld i s c o u r saen a l y s i s ' s e e tkos s h o wh o w l a n guageis usedto conveypowerandstatusin contemporary
socialinteraction,
and howthe
(linguistic)
apparently
neutral,purelyinformative
textswhichemergein newspaper
reportpublications,
ing, government
socialsciencereports,and so on, realize,articulateand
'discourses'
disseminate
as ideologicalpositionsjust as much as do texts which more
explicitlyeditorializeor propagandize.
To do so we needto be ableto'read betweenthe
lines',in orderto get a senseof what discursive/ideological
position,what'interest',may
havegivenrise to a particulartext, and maybeto glimpseat leastthe possibilityof an
alternativeview.It is this kind of readingfor which critical discourseanalysisseeksto
providethe waysand means.So far, however,
criticaldiscourse
analysishas mostlybeen
confinedto language,
realizedas verbaltexts,or to verbalparts of textswhich also use
othersemioticmodesto realizemeaning.
Weseeour bookas a contribution
to a broadened
criticaldiscourse
analysis,
and we hopethat our examples
will demonstrate
its potential
for this kindof work.

Introduction

. \5

' oaf n yI < i n d fsr,o m w o r k so f a r t t o e n t i r e l yo r d i n 0 u r e x a m p l eisn c l u d e ' t e x t - o b j e c t sm

ary, banal artefactssuchas maps,charts,pagesof differentkinds,includingthose of
of their keyrolein the history
websites,
etc.We haveincludedworl<sof art notjust because
l esign',
, n c ei n t h e f o r m a t i o no f t h e ' g r a m m a ro f v i s u a d
o f c o n v e n t i o nasn dc o n s t r a i n ths e
p
o
t
e
n
t
k
i
n
d sa, n d
i d e o l o g i cp
ao
l s i t i o nosf c o m p l eax n d
b u ta l s ob e c a u steh e yt,o o ,a r t i c u l a t e
t h e yt,o o ,s h o u l db e a p p r o a c h ef rdo mt h e p o i n to f v i e wo f s o c i acl r i t i q u e .
As is perhapsalreadyobviousfrom what we havesaid so far, we believethat visual
c o d e sf ,u l f i l st h r e em a j o rf u n c t i o n sT.o u s eH a l l i d a y 'tse r m se, v e r y
d e s i g nl i,k ea l l s e m i o t i m
h eo r l da r o u n d
s e m i o t i fcu l f i l sb o t ha n ' i d e a t i o n a l ' f u n c t i oanf,u n c t i o no f r e p r e s e n t i n g ' t w
'
i
n
t
e
r
p
e
r
s
o
n
a
l
'
f
u
n
c
t
i
o
n
,
a f u n c t i o no f e n a c t i n sgo c i a il n t e r a c t i o nass
a n di n s i d eu s ' a n da n
l df
s o c i a rl e l a t i o n sA.l l m e s s a geen t i t i e s t e x t s a l s oa t t e m p t o p r e s e nat c o h e r e n t ' w o r o
osf
e
l
e
m
e
n
t
t h e t e x t ' ,w h a t H a l l i d a yc a l l st h e ' t e x t u a l ' f u n c t i o n a w o r l di n w h i c ha l l t h e
Whether
with its relevantenvironment.
andwhichitselfcoheres
the text cohereinternally,
producean advertisement
or play a pieceof music,we are
we engagein conversation,
s i m u l t a n e o u scloym m u n i c a t i ndgo, i n gs o m e t h i ntgo , o r f o r ,o r w i t h ,o t h e r si n t h e h e r ea n d
the readerof a magazine
newswith a f riend;persuading
nowof a socialcontext(swapping
someaspectof the world
an audience)and representing
entertaining
to buy something;
'out there',be it in concreteor abstractterms (the contentof a film we haveseen;the
q u a l i t i eo
s f t h e a d v e r t i s epdr o d u c ta; m o o do r m e l a n c h o sl ye n t i m e notr e x u b e r a netn e r g y
t x to r c o m m u n i c o n v e y em
d u s i c a l l ya) ,n dw e b i n dt h e s ea c t i v i t i etso g e t h eirn a c o h e r e nt e
cativeevent.The structureof our book reffectsthis. Chapters2 and 3 deal with the
patternsof representation
which the 'grammarof visual design'makesavailable,and
visually.Chapters4 and 5 dealwith the
hencewith the wayswe can encodeexperience
and hence
available,
patternsof interactionwhichthe'grammarof visualdesign'mal<es
and with the
with the thingswe can do to, or for,eachotherwith visualcommunication,
'texts'which
this entails.Chapter6
relationsbetweenthe makersand viewersof visual
andcommunicative
dealswith the 'textual'f unction,with the way in whichrepresentations
a c t sc o h e r ei n t om e a n i n g f uwlh o l e sC. h a p t e7r d e a l sw i t h t h e m a t e r i a l i toy f v i s u a sl i g n s
(ink,
paint,brushstrokes,
etc.) and the materialswe make
the tools we makethem with
etc.);these,too, contributeto the meaningof
them on (paper,canvas/computerScreens/
chaptersintothe domainofthree-dimensional
the previous
visualtexts.Chapter8 extends
visualsand movingimages.Again we assumethat there is somethinglil<ea Western
'grammao
r f t h r e e - d i m e n s i ovniaslu adl e s i g n a
' , s e to f a v a i l a b lfeo r m sa n d m e a n i n guss e d
c o d e l so,r i n c h i l d r e n ' s
i n s c u l p t u raesw e l la s ,f o r i n s t a n c ei n, t h r e e - d i m e n s i osncaile n t i f im
toys- anda Western'grammarof the movingimage'.
someof the broaderthemeswe havetouchedon in
by discussing
Wewill begin,however,
t h i si n t r o d u c t i o n .

I T h e s e m i o t i cl a n ds c a p e :
languaga
e n d v i s u a lc o m m u n i c a t i o n
In the earlyyearsof schooling,
childrenare constantly
encouraged
to produceimages,
and
to illustratetheir writtenwork.Teachers
commenton theseillustrations
as muchas they
do on the writtenpart of thetext,thoughperhapsnot quitein the samevein:unlikewriting,
i l l u s t r a t i o nasr en o t ' c o r r e c t e d ' n osru b j e c t et do d e t a i l e cdr i t i c i s m( ' t h i sn e e d m
s o r ew o r k , ,
'not clear','spelling!','poor
e x p r e s s i o na'n, d s o o n ) . T h e ya r e s e e na s s e l f - e x p r e s s i o n ,
- as something
ratherthan as communication
which the childrencan do arreaoy,spontaneously,
ratherthan as something
theyhaveto betaught.
By the time childrenare beyondtheir first two yearsof secondary
schooling,
illustrationshavelargelydisappeared
from their ownwork.Fromhereon,in a somewhatcontradictorydevelopment,
writing increases
in importanceand frequencyand imagesbecome
s p e c i a l i z eTdh. i s i s m a d em o r ep r o b l e m a t ibcy t h e f a c t so f t h e p r e s e npt e r i o di,n w h i c h
w r i t i n ga n d i m a g ea r e i n a n i n c r e a s i n gul yn s t a b l er e l a t i o nW
. e m i g h tc h a r a c t e r i zteh e
situationof saytwentyor thirty yearsago in thisway:textsproduced
for the earlyyearsof
schoolingwere richly illustrated,but towardsthe later yearsof primaryschoolimages
beganto givewayto a greaterandgreaterproportionof writtentext.In as muchas images
continued,
they had becomerepresentations
with a technicalfunction,maps,diagramsor
photographs
illustratinga particularlandformor estuaryor settlementtype in a geographytextbook,for instance.
Thuschildren'sown productionof imageswas channelled
in
- awayfrom 'expression'and
the directionof specialization
towardstechnicality.
In other
words,imagesdid not disappear,
but theybecamespecialized
in their function.
In manyways the situationin schoolremainsmuchthe same,with two profoundly
i m p o r t a n tp r o v i s o s . 0 nt h e o n e h a n da l l s c h o o sl u b j e c t sn o w m a k em u c h m o r e u s eo f
images,particularlyso in the yearsof secondaryschooling.
In many of thesesubjects,
c e r t a i n l yi n t h e m o r et e c h n i c a l / s c i e n tsi fui cb j e c tssu c ha s ( i n E n g l a n dS
) c i e n c eI n, f o r m a tion Technology
or Geography,
imageshave becomethe major meansof representing
c u r r i c u l acr o n t e n tI.n t h e m o r eh u m a n i s t iscu b j e c t-s f o r e x a m p l eH, i s t o r yE
, n g l i s ha n d
R e l i g i o uSs t u d i e s- i m a g e sv a r y i n t h e i r f u n c t i o nb e t w e e ni l l u s t r a t i o nd,e c o r a t i o a
nn d
information.
Thistrendcontinues,
and it is the casefor worksheets,
in textbooksand in CDR 0 M s . 0 nt h e o t h e rh a n dt,h e r ei s n o t e a c h i n g
or'instruction'in
t h e ( n e w )r o l eo f i m a g e s
(thoughin England,in the schoolsubjectInformationTechnology,
there is teachingin
desktoppublishing).
Most importantly,
assessment
continues
to be basedon writingas the
m a j o rm o d e S
. t u d e n tas r e c a l l e du p o nt o m a k ed r a w l n g isn S c i e n c eG, e o g r a p hayn d H i s tory; but, as before,thesedrawingstend not to be the subjectof the teacher,sattention,
s n t h e c h i l d r e n 'w
i u d g i n gb y t h e i r ( w r i t t e n )c o m m e n t o
s o r k . I n o t h e rw o r d s ,m a t e r i a l s
providedforchildrenmakeintenserepresentational
useof images;in materialsdemanded
from children- in variousformsof assessment
particularly- writingremainsthe expected
a n dd o m i n a nm
t ode.
Outsideschool,however,
imagesplay an ever-increasing
role,and not just in textsfor

The semiotic landscape . 1,7

c h i l d r e nW
. h e t h e irn t h e p r i n to r e l e c t r o n im
c e d i aw
, h e t h eirn n e w s p a p e rmsa, g a z i n eC
s ,D R 0 M s o r w e b s i t e sw, h e t h e ra s p u b l i cr e l a t i o n sm a t e r i a l sa, d v e r t i s e m e not sr a s i n f o r mationalmaterialsof all kinds,mosttextsnowinvolvea complexinterplayof writtentext,
imagesand othergraphicor soundelements,
designed
as coherent(oftenat the first level
v i s u a lr a t h e rt h a n v e r b a l )e n t i t i e sb y m e a n so f l a y o u t B
g ulti. u t t h e s k i l lo f p r o d u c i nm
modaltexts of this kind,howevercentralits role in contemporary
society,is not taught
i n s c h o o l sT.o p u t t h i s p o i n th a r s h l yi n
, t e r m so f t h i se s s e n t i anle wc o m m u n i c a t i oanb i l i t y ,
this new'visualliteracy',institutionaleducation,
underthe pressure
of often reactionary
politicaldemands,
produces
illiterates.
0f course,writing is itselfa form of visualcommunication.
Indeed,and paradoxically,
t h e s i g no f t h e f u l l y l i t e r a t es o c i a lp e r s o ni s t h e a b i l i t yt o t r e a t w r i t i n gc o m p l e t e lays a
v i s u a lm e d i u m- f o r i n s t a n c eb y n o t m o v i n go n e ' sl i p s a n d n o t v o c a l i z i nw
g h e no n e i s
r e a d i n gn,o te v e n' s u b v o c a l i z i n(ga's i l e n t ' s p e a k i n
ag
l o u di n t h e h e a d 't,o b r i n go u tt h ef u l l
paradoxof this activity).Readers
who movetheir lipswhenreading,
who subvocalize,
are
r e g a r d eadsc u l t u r a l lay n di n t elle c t u al yl t a i n t e db y h a v i n gt o t a k er e c o u r steo t h ec u l t u r a l l y
l e s sv a l u e dm o d eo f s p o l < el n
a n g u a gwe h e nr e a d i n g
v i s u a sl c r i p t T
. h i s ' o l d ' v i s u al li t e r a c y ,
writing,hasfor centuriesnow beenoneof the mostessentialachievements
and valuesof
Westernculture,and one of the most essentialgoalsof education,
so muchso that one
majorandheavilyvalue-laden
distinctionmadeby Westerncultureshasbeenthat between
(oralandprimitive)cultures.Nowonderthat the move
literate(advanced)
andnon-literate
towardsa newliteracy,
basedon imagesandvisualdesign,
cancometo be seenas a threat,
a s i g no f t h e d e c l i n e
o f c u l t u r ea, n dh e n c ea p a r t i c u l a r lpy o t e n st y m b oal n d r a l l y i n gp o i n t
for conservative
andevenreactionary
socialgroupings.
The fadingout of certainkindsof texts by and for children,then, is not a straightf o r w a r dd i s v a l u a t i oonf v i s u a lc o m m u n i c a t i obnu, t a v a l u a t i o nw h i c h g i v e sp a r t i c u l a r
p r o m i n e n ct oe o n ek i n do f v i s u acl o m m u n i c a t i ownr,i t i n ga, n dt o o n ek i n do f v i s u a l i t e r a c y ,
t h e ' o l d ' v i s u a l i t e r a c yO. t h e rv i s u a cl o m m u n i c a t i oi sne i t h e rt r e a t e da s t h e d o m a i no f a
v e r ys m a l le l i t eo f s p e c i a l i s tosr, d i s v a l u eads a p o s s i b lfeo r m o f e x p r e s s i of onr a r t i c u l a t e ,
r e a s o n ecdo m m u n i c a t i osne/e na sa ' c h i l d i s h ' s t a goen eg r o w so u t o f .T h i si s n o ta v a l u a t i o n
of language
as suchovervisualcommunication,
because
evennowthe structures,
meanings
a n d v a r i e t i e so f s p o k e nl a n g u a g e
a r e l a r g e l ym i s u n d e r s t o oadn, d c e r t a i n l yn o t h i g h l y
valuedrn their varietyin the educationsystem(with someexceptions,
suchas in formal
' d e b a t i n g 'o) r i n p u b l i c
f o r u m so f p o w e r .
To sumup:the opposition
to the emergence
of the visuafas a full meansof representat i o n i s n o t b a s e do n a n o p p o s i t i otno t h e v i s u a al s s u c hb, u t o n a n o p p o s i t i oinn s i t u a t i o n s
whereit formsan alternative
to writingand canthereforebe seenas a potentialthreatto
the presentdominance
of verballiteracyamongelitegroups.
In this bookwetal<ea freshlool<at the questionof the visual.Wewantto treatformsof
communicatio
en
m p l o y i nigm a g e as ss e r i o u s a
l ys l i n g u i s t ifco r m sh a v eb e e nW
. e h a v ec o m e
to this positionbecauseof the now overwhelming
evidenceof the importanceof visual
c o m m u n i c a t i oann, dt h e n o w p r o b l e m a t iacb s e n coef t h e m e a n sf o r t a l k i n ga n dt h i n k i n g
a b o u tw h a ti s a c t u a l l yc o m m u n i c a t ebdy i m a g e a
s n db y v i s u a dl e s i g nI .n d o i n gs o , w eh a v e
to moveawayfrom the positionwhich RolandBarthestook in his 1964 essay'Rhetoricof

I8

Thesemiotic landscape

to Elements
as in the introduction
the image'(1977i32-51).ln thisessay(andelsewhere,
of Semiology;Barthes,1967d, he arguedthat the meaningof images(and of other
on,
semioticcodes,like dress,food,etc.) is alwaysrelatedto and,in a sense,dependent
too opento a variety
verbaltext. By themselves,
imagesare,hethought,too 'polysemous',
of possiblemeanings.
To arriveat a definitemeaning,languagemustcometo the rescue.
V i s u am
l e a n i n igst o o i n d e f i n i t ei t;i s a ' f f o a t i n gc h a i no f s i g n i f i e d sH' .e n c eB, a r t h e s a i d , ' i n
are developed
intended
to fx the ffoatingchainof signieverysocietyvarioustechniques
is one
fiedsin sucha way asto counterthe terrorof uncertainsigns;the linguisticmessage
betweenan image-textrelationin which
of thesetechniques'
i977'.39). He distinguished
the verbaltext extendsthemeaningof the image,or viceversa,as is the case,for example,
withthespeech
b a l l o o n si n c o m i cs t r i p s a, n d a n i m a g e - t e xrte l a t i o ni n w h i c ht h e v e r b a l
t e x t e l a b o r a t e s t h e i m a g e , o r v i c e v el nr st hae. f o r m e r c a s e , w h i c h h e c a l lreedl a y , n e w a n d
In the lattercase,the samemeandifferentmeanings
are addedto completethe message.
ingsare restatedin a different(e.g.more definiteand precise)way,as is the case,for
0f the
what is shownin a photograph.
example,
whena captionidentifiesand/orinterprets
t w o ,e l a b o r a t i oi nsd o m i n a n tR. e l a ys,a i dB a r t h e si s, ' m o r er a r e ' .H ed i s t i n g u i s htewdot y p e s
of elaboration,
onein whichthe verbaltext comesfirst,sothat the imageformsan illustra'
tionof it, andonein whichthe imagecomesfirst,sothat the text formsa moredefiniteand
p r e c i s ree s t a t e m e n
o tr ' f i x i n g ' o fi t ( a r e l a t i o nh ec a l l sa n c h o r a g e ) .
verygradual),Barthesargued,
Beforeapproximately
1600 (thetransitionis,of c0urse,
' i l l u s t r a t i o n ' w adso m i n a n tI .m a g e se l a b o r a t etde x t s ,m o r es p e c i f i c a ltl hy e f o u n d i n tge x t s
o f t h e c u l t u r e- m y t h o l o gtyh, e B i b l et,h e ' h o l yw r i t ' o f t h e c u l t u r e- t e x t st,h e r e f o r ew,i t h
to befamiliar.Thisrelation,in whichverbaltextsformeda
whichviewerscouldbeassumed
the dominanttexts in a
and in which imagesdisseminated
sourceof authorityin society,
p a r t i c u l a rm o d et o p a r t i c u l a rg r o u p sw i t h i ns o c i e t yg,r a d u a l l yc h a n g e tdo o n e i n w h i c h
images,
becamethe sourceof authority.In the eraof science,
nature,ratherthandiscourse,
beganto functionas'thebookof nature',as'windowson theworld',
evermorenaturalistic,
to'load the image,burdenandverbaltext servedto identifyandinterpret,
as'observation',
, n imagination'.
i n gi t w i t h a c u l t u r ea, m o r a l a
T h i sp o s i t i o n
d o e se x p l a i ne l e m e n tosf c o m m u n i c a t i oAnn. y o n eo f t h e i m a g e - t e xrte l a may at timesbe dominant,althoughwe feelthat todaythereis a
tions Barthesdescribes
m o v ea w a yf r o m ' a n c h o r a g eC' .o m p a r ef ,o r e x a m p l et ,h e ' c l a s s i c ' d o c u m e n t afri yl m i n
whichthe vieweris first confrontedwith'imagesof nature',then with the authoritative
'current
voiceof a narratorwho identifiesand interpretsthe images,with the modern
verbaldisaffairs'item,in whichthe vieweris first confrontedwith the anchorperson's
n ,i t h t h e
c o u r s ea n d ,e i t h e rs i m u l t a n e o u sol ry f o l l o w i n go n f r o m t h e v e r b a li n t r o d u c t i o w
' i m a g e so f n a t u r e ' t h a ti l l u s t r a t ee,x e m p l i fayn d a u t h e n t i c a t h
e e d i s c o u r s eB.u t B a r t h e s '
accountmissesan importantpoint:the visualcomponentof a text is an independently
andstructuredmessage,
connected
with the verbaltext,but in no way dependent
organized
o n i t - a n ds i m i l a r l tyh e o t h e rw a ya r o u n d .
One importantdifferencebetweenthe accountwe developin this book and that of
e a r l i e rs e m i o t i c i a ni s o u r u s eo f w o r k i n l i n g u i s t i tch e o r i e sa n d d e s c r i p t i o nTsh. i si s a
difficultargumentto make,but worth makingclearly.We think that this bookwouldnot

The semiotic landscape

havebeenpossible
withoutthe achievements
yet we do not,in the way some
of linguistics,
criticsof our approachhavesuggested,
seeour approachas a linguisticone.So what have
we usedfrom linguistics,
and how havewe usedit? And,equally,what havewe not used
from linguistics?
To start with the latterquestion,
we havenot importedthe theoriesand
m e t h o d o l o g io
e fs l i n g u i s t i cdsi r e c t l yi n t o t h e d o m a i no f t h e v i s u a la, s h a sb e e nd o n eb y
othersworkingin thisfield.For instance,
we do not makea separation
of syntax,semantics
a n d p r a g m a t i cisn t h e d o m a i no f t h e v i s u a lw
; e d o n o t l o o kf o r ( t h e a n a l o g u eosf) s e n tences,clauses,nouns,verbs,and so on, in images.We take the viewthat languageand
v i s u a lc o m m u n i c a t i ocna n b o t h b e u s e dt o r e a l i z et h e ' s a m e ' f u n d a m e n t as ly s t e m so f
meaningthat constituteour cultures,but that eachdoesso by meansof its own specific
forms,doesso differently,
and independently.
To givean example,
the distinctionbetween'subjective'and 'objective'meanings
has
playedan importantrole in Westerncultureeversincethe physicalsciences
beganto
developin the sixteenthcentury.Thisdistinctioncan be realized(that is, givenconcrete,
m a t e r i ael x p r e s s i ohne, n c em a d ep e r c e i v a bal en dc o m m u n i c a b lw
e i)t h l i n g u i s t iacs w e l l a s
visualmeans.Theterms'subjective'
and 'objective'canthereforebe appliedto both:they
belongto the meaningpotentialof a cultureand its society.But the way the distinctionis
realizedin language
is quitedifferentfrom the way it is realizedin images.For example,
in
l a n g u a gaen i d e ac a nb e r e a l i z e sd u b j e c t i v ebl yy u s i n ga ' m e n t a lp r o c e svse r b 'l i k eb e l i e v e
in the first person(e.9. We believethat there is a grammar of image); or objectively
throughthe absence
of sucha form (e.9.Thereis a grammarof image). Visualrepresentation,too,can realizebothsubjectivity,
throughthe presence
of a perspectival
angle,and
o b j e c t i v i t tyh, r o u g hi t s a b s e n c ea, p o i n tw h i c hw i l l b e d i s c u s s em
d o r ef u l l y i n c h a p t e 4
r.
M e n t a lp r o c e scsl a u s eas n d n o m i n a l i z a t i oanr e u n i q u et o l a n g u a g eP.e r s p e c t i vi seu n i q u e
to images.But the kindsof meanrngexpressed
are from the samebroaddomainin each
case;andthe forms,differentastheyare/weredeveloped
in the sameperiod,in response
to
t h e s a m ec u l t u r a lc h a n g e sB. o t h l a n g u a g ae n d v i s u a lc o m m u n i c a t i oenx p r e s m
s eanings
belonging
to and structuredby culturesin the onesociety;the semioticprocesses,
though
n o t t h e s e m i o t i cm e a n sa/ r e b r o a d l ys i m i l a r a; n dt h i s r e s u l t si n a c o n s i d e r a bdl e g r e eo f
congruence
between
the two.
At the sametime,however,
eachmediumhas its own possibilities
and limitationsof
meaning.
Not everything
that can be realizedin language
can alsobe realizedby meansof
images,
or viceversa.As well as a broadculturalcongruence,
difference
thereis significant
between
the two (andothersemioticmodes,of course).In a language
suchas Englishone
needsto usea verbin orderto makea full utterance(believe,rs);and language
hasto use
(a grammar of images,believe,wd.But
namesto refer to whateveris to be represented
language
doesnot haveor needanglesof visionto achieveperspective,
nor doesit haveor
needspatialdispositions
of elements
to achieve
the meanings
of syntacticrelations:
images
haveand needboth.The meaningpotentialsof the two modesare neitherfully conflated
nor entirelyopposed.
We differfrom thosewho seethe meaningof language
as inherentin
the forms and the meaningof imagesas derivedfrom the context,or the meaningsof
l a n g u a gaes ' c o n s c i o uasn' dt h e m e a n i n gosf i m a g e a
s s' u n c o n s c i o r . l s ' .
To returnto the first of our two questions What havewe usedfrom linguistics,
and

19

20 .

The semiotic landscape

an
how havewe usedit? - perhapsthe mostsignificantborrowingis our overallapproach,
'attitude'whichassumes
images,like language,
will
for representation,
that,as a resource
n .ec a l l
d i s p l a yr e g ul a r i t i e sw, h i c hc a nb e m a d et h es u b j e cot f r e l a t i v e lfyo r m a ld e s c r i p t i oW
l ye,
t h i sa ' g r a m m a r ' t od r a wa t t e n t i o tno c u l t u r a l l yp r o d u c erde g u l a r i t yM. o r es p e c i f i c a l w
featureswhichwe takento be generalto all human
haveborrowed'semioticorientations',
we thinkihat the distinctionbetween
meaning-making,
irrespective
of mode.For instance,
' o b j e c t i v i t y ' a n d ' s u b j e c t i v i t ya' i sg e n e r acl u l t u r a l / s e m i o ti si cs u ew h i c hc a n b e r e a l i z e d
l i n g u i s t i c a la
l ys w e l l a s v i s u a l l yt ,h o u g hd i f f e r e n t l ys o ,a s w e h a v es a i d . 0 r ,a s a n o t h e r
instance,
we havetaken MichaelHalliday'ssocialsemioticapproachto languageas a
ratherthanas
model,as a sourcefor thinkingaboutgeneralsocialandsemioticprocesses/
a mine for categories
to apply in the descriptionof images.His modelwith its three
the modelworkswell
functionsis a startingpointfor our accountof images,not because
(whichit does,to an extent),but because
it workswell asa sourcefor thinking
for language
a b o u ta l l m o d e so f r e p r e s e n t a t i o n .
M a y b em o s tt o t h e p o i n ti s t h i s :o u r a p p r o a c tho c o m m u n i c a t i osnt a r t sf r o m a s o c i a l
photographers,
expressed
by speakers,
writers,printmakers,
base.In our viewthe meanings
paintersand sculptorsare first and foremostsocialmeanings/
eventhoughwe
designers,
are
Giventhat societies
acknowledge
the effectand importanceof individualdifferences.
interests,
not homogeneous,
but composed
of groupswith varying,andoftencontradictory,
producedby individuals
incongruities
andclashes
will reffectthe differences,
the messages
oftenthe case,that the
which characterize
sociallife. It is likely,and in our experience
so that
showthesesocialdifferences,
differentmodesthroughwhichtextsare constructed
in a multimodaltext usingimagesandwritingthe writing maycarry oneset of meanings
for instance,
it may be that the verbal
and the imagescarry another.
In an advertisement,
overtlysexiststereotypes.
Given
the visualtext encodes
text is studiously'non-sexist',while
the still prevalentsenseaboutthe meaningof images,it is possibleto pretendthat the
m e a n i n cg a r r i e di n t h e i m a g ei s t h e r eo n l y ' i n t h e e y eo f t h e b e h o l d e rs' ,o m e t h i ntgh a t i t
d eanings.
w o u l dn o t b e p o s s i b lteo a s s e rat b o u tv e r b a l l yr e a l i z e m
drawnfrom very manydomains,and
Our examplesin this bookare quitedeliberately
from differenthistoricalperiods.We hopethat our ideaswill helpanyoneinterestedin
but alsothe struccommunication
to seein imagesnot onlythe aestheticand expressive,
l n d c o m m u n i c a t i vdei m e n s i o nW
t u r e ds o c i a l p
, o l i t i c aa
s .e w i l l d r a w e x a m p l efsr o m t h e
kindsof textswhichare alreadyfully basedon the newvisualliteracyand playa dominant
websites
andso on.
rolein anypublicsphere,
magazine
articles,advertisements,
textbooks,
Thisis not because
we wantto promotethesetextsasa kindof modelwhichshouldreplace
otherkindsof texts,but because
their rolein the livesof childrenandadultsis so important
that we simplycannotaffordto leavethe abilityto ihink andtalk aboutthem (and,indeed,
We havea particularinterestin the placeof
to producethem)to a handfulof specialists.
t h ev i s u a il n t h e l i v e so f c h i l d r e na,n dw e h o p et o s h o wt h a t c h i l d r e vn e r ye a r l yo n ,a n dw i t h
v e r yl i t t l eh e l p( d e s p i tael l t h ee n c o u r a g e m e ndte) ,v e l oap s u r p r i s i nagb i l i t yt o u s ee l e m e n t s
of the visual 'grammar'- an ability which,we feel, shouldbe understoodbetter and
developed
furlher,ratherthan beingcut off prematurelyas is, too often,the caseat
presena
t ;n da n a b i l i t yt h a t s h o u l da l s ob ea v a i l a b lteo a d u l t s .

The semiotic landscape

27

A N U N C O N V E N T I O N AHL I S T O R YO F W R I T I N G
T h ed o m i n a n coef t h e v e r b a lw
, r i t t e nm e d i u mo v e ro t h e rv i s u a m
l e d i ai s f i r m l yc o d e da n d
b u t t r e s s ei d
n c o n v e n t i o nhails t o r i eos f w r i t i n g T
. h e s eg o s o m e t h i nlgi k et h i s .L a n g u a gien
i t s s p o k e fno r m i s a n a t u r a pl h e n o m e n ocno,m m o nt o a l l h u m a ng r o u p sW
. r i t i n gh
, owever,
i st h ea c h i e v e m eonfto n l ys o m e( h i s t o r i c a l lbyy, f a r t h e m i n o r i t yo f ) c u l t u r e sA. t a p a r t i c u lar stagein the historyof certaincultures,theredeveloped
the needto makerecordsof
transactions
power.
of variouskinds,associated
usuallywith trade,religionor (governing)
Theserecordswere initiallyhighlyiconic;that is, the relationbetweenthe objectto be
recordedand the formsand meansof recordingwas closeand transparent.
For instance,
the numberof notchesin a stickwouldrepresent
the numberof objectsstoredor tradedor
owed.The representation
of the objectwould usuallyalso be transparent:a wavy line
e v e n t u a l lbye c a m et h e C h i n e s ied e o g r a m
f o r ' w a t e r ' ;t h e h i e r o g l y p hi m
c a g eo f t h e o x ' s
h e a dw hi c hi n i t i ally ' s t o o df o r ' ' o x 'e v e n t u a
l yl b e c a mteh el e t t e ra l e p h( N ) ,a l p h a( a ) ,a .T h i s
exampleillustrates
what in thesehistoriesis regarded
asthe rarestof all achievements,the
i n v e n t i oonf a l p h a b e t iwc r i t i n g .
A l p h a b e t iwc r i t i n gd e v e l o p eidt ,s e e m cs l e a ro, u t o f i c o n i ci,m a g e - b a s secdr i p t sI.n t h e s e
originalscriptforms,an objectwas initiallyrepresented
by an imageof that object.Over
t i m e ,i n t h e u s eo f t h e s c r i p tb y d i f f e r e ngt r o u p ss,p e a k i ndgi f f e r e nlta n g u a g e s , t ihme a g eo f
the objectcameto standfor the nameof the objectandthenfor its initialletter.Aleph,'ox'
in Egyptianhieroglyphics,
after centuriesof traveland constanttransformation
through
the culturesand languages
of the easternMediterranean,
becamethe letter alpha,and
eventually
the lettera in the Romanalphabet.Clearlythis was a process
whereeachstep
involvedconsiderable
abstraction,
so muchso that,seemingly,
alphabetic
writinghasbeen
i n v e n t eodn l yo n c ei n t h e h i s t o r yo f h u m a nc u l t u r e sA. l l p r e s e nat l p h a b e t iscc r i p t sf,r o m
I n d i at o t h e M i d d l eE a s tt o E u r o p ea, r e d e v e l o p m e not fst h a t i n i t i a ls t e pf r o m E g y p t i a n
( o r p o s s i b l yS u m e r i a n i)c o n i ch i e r o g l y p h ri ce p r e s e n t a t i ot on t h e P h o e n i c i aanl p h a b e t ,
and from there westwardto the Greek-speaking
world, and eastwardto the Indian
s u b c o n t i n e notr, , i n t h e r e g i o no f i t s o r i g i n ,d e v e l o p i nign t o t h e A r a b i cv e r s i o no f t h e
alphabet.
This is indeedan impressive
culturalhistory,impressive
enoughto havestoodas the
acceptedhistoricalaccountof the achievement
writing,unquestioned
for
of (alphabetic)
centuries.
Withinthis account,all cultureswith formsof visualrepresentation
that are not
directlyconnected
to language
aretreatedascultureswithoutwriting.However,
it is worth
investigating
this history,and in particularthe crucialstepfrom visualrepresentation
to
t h e l i n kw i t h l a n g u a g e
a , l i t t l e m o r ec l o s e l yP. r i o rt o t h i s s t e p( i n r e a l i t ya d e v e l o p m e n t
s p a n n i nm
g i l l e n n i at )h e r ew e r et w o s e p a r a taen di n d e p e n d em
n to d e so f r e p r e s e n t a t i o n . 0 n e
was language-as-speech;
the other,the visualimage,or visualmarks.Eachserveda particular set of purposes
suchas the construction
of historiesand myths,the recordingof
genealogies
andtransactions,
andthe recording
andmeasurement
of objects.In thecaseof
'took over'the other,as a means
somecultures,however,
the oneform of representation
- onecouldsay,reduced
- to
of recording;
that is,visualrepresentation
becamespecialized
functionas a meansof the visualrepresentation
perhapsin highlyorganized
and
of speech,

22

The semiotic landscaPe

takenover,by the verbalas

At this pointthe visualwassubsumed,
societies.
bureaucratized
p
o
s
a sn dp o t e n t i a lfso r
p
u
b
l
i
c
uses, sibilitie
i t s m e a n so f r e c o r d i n gC.o n s e q u e nittlsyf o r m e r
away'
declinedandwithered
disappeared,
representation
independent
did not occur.Herethe visual
this development
however,
In the caseof othercUltures,
of this abound:from
Instances
alongwith the verbalmeansof representation.
continued,
to
(sensorily
of
the tactilemode representation)
the oneextremeof the Inca quipustrings
a
manner
in
encode,
and carvings.These
AustrallanAboriginaldrawings,sand-paintings
'translation'of, verballanguage,
meaningsof the
not at all directlydependentontor a
with
connected
are
They
in visualform.
culturewhichare deemedto be bestrepresented
are
instance,
for
or sand-paintings,
with them,sothat wall-paintings
or language
language,
journeyst
so
and
myths,
ancestor
features,
of geographical
by verbalrecounts
accompanied
the
other
over
one
priority
of the
in thesecasesthere is no questionof the
on. However,
to the verbalas its form of
mode,and the visual has certainlynot becomesubsumed
representation.
the historyof two wordswhichin a sense
to consider
it is interesting
In this connection
with Westernnotionsof literacy,the words grammar and syntax' Gramare synonymous
mar derivesfrom the Greekgrammatike('the art of readingand writing','qrammar',
'letter','alphabet'),
grammatikosCliter'alphabet'),'
relatedwordSweregramma('Sign',
of thingsin the
state
Thisetymologyrecordsthe
ate,,,(primary)teacher','grammarian').
m
e
a
n
i n g ' s i g na' ,s i n
H e l l e n i s t ipce r i o d( f r o ma p p r o x i m a t e3l y0 0 8 c ) ;i n e a r l i e rt i m e st h e
the verb
'paintedor drawn[etc.] mark'wasthe primarymeaning.
for
example,
In Homer,
to
comes
it
fr6m
there
and
in', as in engraving/
grapheinstill means'Scratch','scratch
'painting'.Syntaxis,in pre-Hellenistic
'drawing',
meant
times,
mean both'writing'and
'Contract','Wage','OrganizatiOn','System','battle
fOr inStanCe,
with Syntagmat
fOrmatiOn',
t
h
e Hellenistic
' c o n t i n g e notf t r o o p s ' , ' c o n s t i t u t i(oonf a s t a t e ) ' , ' b o ook r t r e a t i s e 0
i
n
n
l
y
'.
'grammatical
construc(among
its othermeanings)
perioddoessyntaxiscometo mean
'concentrate
'arrange
battleformations'and
again/meansboth
tion'. The verbsyntasso,
( o n e ' st h o u g h t s ) ' , ' o r g a n i z e ' , ' w r i t e ' , ' c o m p o s e ' .
W h i l ew ed o n o tw i s ht o p l a c et o o m u c he m p h a s iosn e t y m o l o gny e, v e r t h e l et shseh i s t o r y
of thesetwo wordswhichare so crucialto our notionsof literacypointsto formsof social
'markings'on the other'Together
organizationandorder,on the one hand,and to visual
of the modeof imagesand the
organization
they indicatethe initiallyquite independent
historyof the word grammar
At the sametime,the subsequent
modeof verballanguage.
b r i n g so u t c l e a r l yt h e s u b o r d i n a t i oonf t h e v i s u a lm e d i u mt o t h e m e d i u mo f v e r b a l a n are,from the
guage.Cultureswhichstill retainthe full useof bothmediaof representation
underdeveloped,
ooint of view of'literate cultures',regardedas illiterate,impoverished,
than that overtlyand
when in fact they havea richer arrayof meansof representation
pointed
out earlier,literate
we
as
availableto literatecultures.Nevertheless,
consciously
be it that they
writing,
otherthan
culturesdo makeuseof meansof visualcommunication
by childrenor
expression
replicasof realityor as a meansof individual
areseenas uncoded
to
or
of,
accessible
artists.In otherwords,they are not treatedas eitherthe expressions
meansof readingbasedon,articulated,rationalandsocialmeanings'
historyof writing is onethat treatsthe comingtogetherof visual
0ur unconventional

Thesemiotic landscape

23

a n dv e r b a rl e p r e s e n t a t i ao sno n l yo n ep o s s i b i l i tayn/ do n e f/ u r t h e r m o r e
t h, a t b r i n g sw i t h i t
n o t j u s tt h o s eb e n e f i tosf w r i t i n gw h i c ha r ew e lI e n o u g hu n d e r s t o o bd u, t a l s ot h e n e g a t i v e
aspectsincurredin the lossof an independent
form of representation,
the diminutionof
point
modesof expression
and representation.
Fromthat
of viewculturessuchas Australi a n A b o r i g i n acl u l t u r e sa r e s e e na s h a v i n gb o t h m o d e so f r e p r e s e n t a t i ot h
ne
: v i s u a l( o r
p e r h a pa
s w h o l es e t o f v i s u a fl o r m so f r e p r e s e n t a t i oann) d t h e v e r b a lT
p
. h e o i n to f t h i s
l n e o f u n d e r m i n i nt h
h i s t o r yi s n o t o n l yt h e p o l i t i c a o
g e n o t i o no f i l l i t e r a t ec u l t u r e ' ( o r
' m e r e l yo r a l c u l t u r e ' )b, u t a l s ot h e a t t e m p t o s e et o w h a te x t e ntth e c o n v e n t i o nhails t o r y
b l i n d su st o t h ef a c t sa n du s e so f v i s u acl o m m u n i c a t i oi nns o - c a l l eldi t e r a t ec u l t u r e s .
I n t h i s b o o kw e d e v e l o p
t h e h y p o t h e sti hs a t i n a l i t e r a t ec u l t u r et h e v i s u a lm e a n so f
c o m m u n i c a t i oanr e r a t i o n a le x p r e s s i o nosf c u l t u r a lm e a n i n g sa,m e n a b l et o r a t i o n a l
accountsand analysis.
The problemwhichwe face is that literatecultureshavesystematicallysuppressed
meansof analysisof the visualformsof representation,
so that thereis
not, at the moment,an established
theoreticalframeworkwithin which visualforms of
reoresentation
can be discussed.

T H E ' O L D ' A N DT H E ' N E W ' V I S U A LL I T E R A C YI N B O O K SF O RT H E V E R Y

YOUNG
S o f a r w e h a v ed i s t i n g u i s h tewdo k i n d so f v i s u a l i t e r a c yo: n ei n w h i c hv i s u acl o m m u n i c a tion hasbeenmadesubservient
to language
and in whichimageshavecometo be regarded
a s u n s t r u c t u r erde p l i c a o
s f r e a l i t y( t h e ' o l dv i s u a l i t e r a c y ' , i no u r t e r m s ) a
; n da n o t h e ri n
w h i c h ( s p o k e n l)a n g u a g e x i s t ss i d eb y s i d ew i t h , a n d i n d e p e n d e o
n ft , f o r m s o f v i s u a l
representation
which are openlystructured,ratherthan viewedas more or lessfaithful
d u p l i c a t eosf r e a l i t y( t h e ' n e w ' ,i n o u r t e r m s ) W
. e h a v el o o k e da t t h e s ea s h i s t o r i c aal n d
culturalalternatives.
But they also existside by side,at leastin contemporary
Western
c u l t u r ea, n dw e s u g g e st h
t a t w e a r e i n t h e m i d d l eo f a s h i f ti n v a l u a t i o a
n n du s e sf r o mt h e
o n em o d et o t h e o t h e rf,r o mt h e ' o l d ' t o t h e ' n e w ' v i s u al li t e r a c yi n, m a n yi m p o r t a nst o c i a l
contexts.The exampleswe will now discusssuggestthat the very first bookschildren
e n c o u n t emr a ya l r e a d yi n t r o d u cteh e mt o p a r t i c u l akr i n d so f v i s u a l i t e r a c y .
Figure1.1 showsa typicaltwo-pagespreadfrom Baby'sFirst Book,a bookwhich,on
i t s i n s i d ec o v e rd, e c l a r etsh a t ' t h e t e x t a n d i l l u s t r a t i o ntsh,o u g ho v e r s i m p tl e
o grown-ups,
w i l l s a t i s f yt h e i rt i . e .t h e t o d d l e r s 'cl r a v i n gfso r t h e r e p e t i t i o on f w h a tt h e ya l r e a d yk n o w ,
andwill helpthemassociate
thewordswith the objects'.Whenwewrotethefirst versionof
t h i s c h a p t e ri ,n 1 9 8 9 , i t w a s s t i l l w i d e l yd i s t r i b u t e da,n d t o d a y i t i s a l r e a d ym a k i n ga
c o m e b a caks a n o b j e c o
t f nostalgia.
Figure1.2 showsa typicalpagefrom Dicl<Bruna's0n My Walk.Thisbookis oneof a
setof four,the othersbeingIn My Home,In My ToyCupboardand0n the Farm.lt consists
of eightpagesand,with the exception
of the front and backcovers,
the pagescontainno
wordswhatsoever.
Comparedto the pictureof the bird in the tree,the pictureof the bath is realistic,
detailedand complex.If we were to analyseit into its components,
if we were to try

Thesemiotic landscape

{l-

' ---,,.-,

i . - _y *

i'1d
-,J

\,

f: i,;: i: i

-"i

i i'lii !ii:l

f-Tt il
i

i;*iir*o**

3
r

!-:* I i

!,-,,t E

"r

i'i:u **:,::1

Fig l.l

My bath(from Bahls Fitst Boo&Ladybird)

, e m i g h te n c o u n t eprr o b l e m sA.r e
a n d i d e n t i f ya l l t h e d i f f e r e net l e m e n tosf t h i s p i c t u r ew
the ripplesin the water to be countedas components?
Are the shadows,
cast by the tub
and towel?And if we wereto try and identifythe relationsbetweenthesecomponents,
what wouldwe haveto say,for example,about the relationbetweenthe duck and the
soap?We ask thesequestionsbecausethey are the kindsof questionswith which one
might start if one wantedto show that imagesare structuredmessages,
amenable
to constituentanalysis.Isn't the structureherethat of the cultural object'bathroom',
r a t h e rt h a no n ei m p o s ebdy t h ec o n v e n t i o no sf a v i s u acl o d e ?I s n ' tt h i sp i c t u r eu n p r o b l e m providedone knowswhat bathroomslook
atically,transparently
readable(recognizable),
like?
T h i si s t h e l i n eP a r i sS c h o osl e m i o t i c i a nssu c ha s R o l a n dB a r t h e sa n d C h r i s t i a nM e t z
t o o k i n t h e 1 9 6 0 sC
. o m m e n t i nogn p h o t o g r a p hBya, r t h e s a i d :
In orderto movefrom the realityto its photograph
it is in no way necessary
to divide
up this realityinto unitsandto constitutetheseunitsas signs,substantially
different
f r o m t h e o b j e c t t h e yc o m m u n i c a t e . . .C. e r t a i n l y , t hi e
m a g ei s n o t t h er e a l i t yb u t a t
leastit is its perfectanalogonand it is exactlythis analogicalperfectionwhich,to
commonsense,
definesthe photograph.
Thuscan be seenthe specialstatusof the
photographicimage:.it is a messagewithouta code.
( B a r t h e s1. 9 7 7 : l - 7 )

The semiotic landscape . 25

fig f.Z Bird in tree (Bruna,t988)

A n d h ee x t e n dtsh i sa r g u m e nt to o t h e rp i c t o r i am
l o d e sa, l b e i tw i t h a q u a l i f i c a t i o n :
Are there other messages
without a code?At first sight,yes:precisely
the whole
rangeof analogicalreproductions
of reality- drawings,paintings,
cinema,theatre.
However,
each of those messages
developsin an immediateand obviousway a
s u p p l e m e n t am
r ye s s a g e
. . . w h i c h i s w h a t i s c o m m o n l yc a l l e dt h e s t y l e of the
reoroduction.
( B a r t h e s1,9 7 7 : 1 7 )
The pictureof the bird in the tree,on the otherhand,is muchlessnaturalistic,
muchless
d e t a i l e ad n dm u c hs i m p l etrh a nt h ep i c t u r eo f t h eb a t h r o o mI t. i s s t y l i z e ad n dc o n v e n t i o n a l ,
a n d q u i t ec l e a r l ya ' c o d e d ' i m a g e .N o d e p t h ,n o s h a d o w sn,o s u b t l en u a n c e o
sf colour:
e v e r y t h i ni g
s p l a i na n db o l da n ds i m p l eA. n dt h es t r u c t u r o
e f t h e i m a g ew, i t h i t s o n ec e n t r a l
a n df o u r m a r g i n ailm a g e sd,o e sn o t i m i t a t ea n y t h i n g
i n t h e r e a lw o r l d .I t i s a c o n v e n t i o n a l
visualarrangement,
basedon a visualcode.As a resultthe components
of the wholestand
out as separate,
distinctunits,and the picturewouldseemquiteamenable
to constituent
a n a l y s i sT. h i s i s n o t j u s t a m a t t e ro f s t y l e :t h e s t r u c t u r eo f t h i s p i c t u r ec o u l da l s o b e
realizedin more detailedstyles.Bruna'sbook datesfrom 1953, well beforethe era of
c o m p u t e' ri m a g i n gb' ,u tt h ep i c t u r eo f t h e b i r di n t h et r e ec o u l dh a v eb e e nc o m p o s ewdi t h a
c o m p u t ear ,l i g n i n g
r e a d y - m a dsei m p l ei c o n si n a c o m p o s i t i o ncaol n f i g u r a t i o- ni t i s i n f a c t
q u i t es i m i l a tro t h e c o m p u t e r - d r a w
d inn n e ri n v i t a t i o inn f i q u r e1 . 3 .

The semiotic landscape

irrulNer{1\'
@

fig f.: Computer-drawn

dinnerinyitation

by
the pictureof the bathroomis part of a two-pagelayout,and accompanied
Second,
imposingmeaningon the image,turningit
words.Languagecomesfirst, authoritatively
into a typicalinstanceof a bathroomby meansof the genericlabel'Bath'.As a resultthe
picturecould be replacedby other imagesof bathroomswithout much lossof meaning
( o n ev e r b a lt e x t , m a n y i m a g e sm
is general,
, a n y p o s s i b l ei l l u s t r a t i o n sH
) .e r el a n g u a g e
world of images.Thusthe
bestowingsimilarityand order on the diverse,heterogeneous
t h ew o r l da s i t
p
r
e
s
e
n
t
s
,
book
o n t h e o n eh a n d a, n ' u n c o d e dn' ,a t u r a l i s t irce p r e s e n t a t i (o' n
i s ' - e m p i r i c a lf ,a c t u a l ,s p e c i f i ca) n d ,o n t h e o t h e rh a n d ,a s p e c i f i ca,u t h o r i t a t i v epl yr e . ew i l l s h o wl a t e rt h a t ,c o n t r a r y
n 'a t u r a l i s t ipci c t u r eW
s c r i b ew
d a yo f r e a d i n tgh i s ' u n c o d e d
pictures
of this kindare alsostructured,
to what Barthesand othersarguedin the 1960s,
whetherthey are photographs,
drawings,paintingsor other kinds of pictures.For the
as such,that
moment,however,
the importantpointis that theyare not usuallyinterpreted
suppressed
andnot
of imagesof this kindis,in our society,
awareness
of the structuredness
o a r to f t c o m m o n
sense'.
impose
In Dick Bruna's0n My Walk,by contrast,thereare no wordsto authoritatively
meaning
o n t h e i m a g ea, n dt h e i m a g ei s n o l o n g e ra n i l l u s t r a t i o nt h: e i m a g ec a r r i e st h e
c o u l da l l
s h o r e a dt h i sb o o kw i t h t h e i rc h i l d r e n
m e a n i n g , t hweo r d sc o m es e c o n dP. a r e n tw
(one image,manyverbaltexts).
tell a differentstory,couldevenusedifferentlanguages

The semiotic landscape

The world of '0ne image,many differentverbaltexts' ('commentariesi)

imposesa new
mode of control over meaning,and turns the image,formerlya recordof natureor
a
p l a y g r o u nfdo r c h i l d r e na n d a r t i s t s i, n t o a m o r ep o w e r f u b
l , u t a l s om o r er i g o r o u s lcyo n t r o l l e da n d c o d i f i e dp u b l i cl a n g u a g ew,h i l ei t g i v e sl a n g u a g ef o, r m e r l yc l o s e l yp o l i c e d
in
m a n ys o c i a il n s t i t u t i o nas ,m o r ep r i v a t ea n d l e s sc o n t r o l l e db,u t a l s ol e s sp o w e r f u l ,
s
t
a
t
u
s.
The'readings'which
parentsproducewhenthey read0n My Walkwith their childrenmay
all be different,yet thesedifferentreadings
will necessarily
havecommonelements,
deriving from their commonbasis- the elementsincludedin the image,and the way
these
e l e m e n tasr ec o m p o s i t i o nl yabl r o u g htto g e t h e r .
Whateverstoryparentswill tell aboutthe pagewith the bird in thetree,it will necessarily haveto be a story that createsa relationbetween,
for instance,
birdsand aeroplanes
( n a t u r ea n dt e c h n o l o g ya)n d b i r d sa n dc a t s( p r e y
a n dp r e d a t o r )I .t w i l l a l s oh a v et o b e a
storyin whichthe bird,safelyin itstree,isthe centralcharacter,literally
andfiguratively.
In
how manywayscan catsand birdsbe related?Not that many,at leastnot if one
assumes
that bookslike 0n My Walkserveto introducechildrento the world aroundthem,rather
e o r l d so f f a n t a s i eas n d u t o p i a sC
t h a nt o t h e p o s s i b lw
. a t sc a n ' h u n t , , ' t o r t u r e.,k, i l l , a n d
'eat' birds.Birds
can 'escape'cats or fail to do so. Thereare not that manystoriesto
choose
f r o m . 0 nt h e o t h e rh a n d p
, a r e n t sa n dt h e i rc h i l d r e nc a nc h o o s teh e o r d e ri n w h i c h
theywant to dealwith the variouselements:
the pageis'non-linear,.
It doesnot imposea
sequential
structure.And theycanchoosewhetherto tell the storyof the bird andthe cat
as a politicaistory,a story of powerfulpredatorscomingfrom anothercontinentand
n a t i v eb i r d sk i l l e da n dt h r e a t e n ewdi t h e x t i n c t i o n
( a sm i g h tb e d o n ef,o r i n s t a n c ei n, A u s tralia),or as a storythat legitimizes
the survivalof the fittest.Thestoryof the bird andthe
a e r o p l a n se i,m i l a r l ym, a y b e t o l d f r o m a n e n v i r o n m e n t a lpi soti n to f v i e w o
, r a s a s t o r yo f
e v o l u t i o n a rtyr i u m p h sa n d h u m a nt e c h n o l o g i c a
p lr o g r e s sE. v e nw h e r es u c hd i s c o u r s e s
are not explicitlyinvoked,
they will still communicate
themsejves
to childrenthrouqhthe
parents'attitudes
towardsthe characters
andthe actions.
Not only the elementson the individualpages,but alsothe pagesthemselves
must be
broughtin relationto eachother.The book as a whole must be readableas a conerent
sequence.
Thisis promptedby the title (0n My Walb aswell as by the pictureon the front
cover,which showsall the elements
together.
We haveinvestigated
this a little further in
connection
with anotherbool<in the Bruna series,0nthe Farm.This book containsthe
followingcentralpictures:house,
farmer,cat,dog,appletree,rooster,
lamb,cow.Listingthe
w a y si n w h i c ht h e s ep i c t u r e cs a np l a u s i b lbye l i n k e dt o e a c ho t h e rw, e f o u n dt h a t s o m e( e . g .
t h e a p p l et r e ea n dt h e h o u s ec) a n o n l yb e l i n k e di n s p a t i a ll,o c a t i v e
t e r m s( e . g t. h e a p p l e
t r e ei s n e x t o t h eh o u s e )O. t h e r s( e . gt.h ea n i m a l a
s n dt h eh o u s ec) a nb er e l a t e db y v e r b so f
'dwelling'(e.9.
the cow liesunderthe appletree) or by theverbsof 'motion,(e.g.the cat
c l i m b su p t h e a p p l et r e e ) .T w oo f t h e a n i m a l s( t h ec a t a n dt h e d o g )c a nr e l a t et o t h e o t h e r
animalsand to eachotherby meansof antagonistic
or co-operative
actions(e.g.the dog
barksat the cow;the dog leadsthe sheep).only the farmercan relateto all the other
elementsin an agentiveway. He can buy them,own them,buildthem,grow them,keep
them,raisethem,harvestthem,shearthem,slaughterthem,and so on. In otherwords,
whateverway the parentsreadthesepictures,they will, in the end,haveto dealwith the

The semiotic landscape

themeof spatialorder,the themeof socialinteraction(projectedon to animals)and the

with the theme
themeof humanmasteryovernature(aswell as,via the marginalpictures,
pre-selected
by the
of procreation),
andtheywill haveto do all this in termsof the elements
book.An analysisof the way the elementscan be opposedto each other showsthat,
parentsmayconstruct,
whateverthe classifications
theywill not be ableto avoidengaging
'untamednature','domesticated/cultivated
with the Westernculturaldistinctions
between
n a t u r e ' a n d ' h u m a nt e c h n o l o g yA' .n d t h e y w i l l a l s o h a v et o r e c o g n i zteh e d i s t i n c t i o n
flora andfauna,and betweenpets,farm animalsandwild
betweenanimateand inanimate,
animals.
It shouldbe noted,however,
that everypagein the book (and in Bruna'sotherbooks)
classifications,
that forms
containsat leastonerelationthat doesnot easilyfit the received
is the relationbetween
a rabbit
somewhat
of a challengeandapuzzle.What,for example,
parents
and a basketof ffowers?A beetleanda fence?Suchvisualenigmascanchalienge
a n dc h i l d r e tno e x e r c i steh e i ri m a g i n a t i otno, i n c l u d ei n t h e i rt h i n k i n ge l e m e n ttsh a t d o n o t
to allowthe
easilyfit in with the traditionalorder of things,to toleratesomeambiguity,
i n c l u s i oonf t h e ' o t h e r 'i n t h e i rc o n s t r u c t i oonf t h ew o r l d .
Thetwo books,then,are very differentin their stancetowardsthe image.The Bruna
essentialized
and idealized
representations,
and provides
stancepresents
highlyprocessed/
parentsand,later,childrenwith the opportunity
to talk aboutthe imagesin wayswhichare
specificdiscourses
to theserelatively
or seemappropriate
to them,to applyspecificvalues,
lessprocessed,
morenaturalistic
The Ladybirdstancepresents
ostensibly
abstractimages.
parents(and,lateron,children)with a specificverbally
visualrepresentations
andprovides
from the
realizedway of readingthe image.The Ladybirdbook is openand interactive
perspective
of writing;the Bruna
of the image,and authoritarianfrom the perspective
b o o kw o r k si n t h e o p p o s i t e
w a y .T h ec l o s u r ei n t h e B r u n ab o o k l i e si n t h e l i m i t sw h i c h
openreadings-these
selection,form
andstructureof the imagesimposeon the apparently
e n t e rt h e d i s c o u r s ewsh i c ha r e a l r e a d y ' i n ' t h es o c i a l i z epda r e n t ss,o t h a t t h e w h o l eo, n c e
a sn d ' n a t u r a l ' t op a r e n t sa n d
o r a l l yt r a n s m i t t etdo t h e c h i l d r e nw, i l l a p p e a rs p o n t a n e o u
c h i l d r e na l i k e .A r e t h e yn o t ,a f t e ra l l , m e r e l ye n g a g e idn a n i n n o c e nrte a d i n go f ' w h a t i s
two differentformsof socialcontrol
there'in the pictures?Thusthe two booksreoresent
, erhaps
o v e rm e a n i n g . 0 nies o p e n l ya n de x p l i c i t l lyo c a t e di n t h e t e x t i t s e l f t; h e o t h e rl i e s p
more covertlyand implicitly,in the way the book presentsitself lessas a text than as
that will
an organizedresourcefor makingtexts,jointly wiih the parentaldiscourses
inevitably
enterthe text aswell.
part of Bruna'sbooks,of the publictext
Thesediscourses,
however,
are not themselves
meantto transcend
Insteadtheyare relegated
their diversity.
to the realmof the private,of
'lifestyles'where
they do not threatenthe orderof the largersocialworld.Thereis never
j u s t ' h e t e r o g l o s s(im
a 'a n ym e a n i n g sn)o, re v e rj u s t ' h o m o g l o s s i(ao'n ea u t h o r i t a t i vm
ee a n i n g ) .I n s t e a tdh e r ei sa r o l ed i s t r i b u t i oanm o n gt h ed i f f e r e nst e m i o t i cas ,r o l ed i s t r i b u t i oinn
which somesemioticsare givena great deal of socialpower,but at the price of being
s u b j e c t etdo g r e a t e ri n s t i t u t i o n a( la n dt e c h n o l o g i c acl o
) n t r o lw
, h i l eo t h e r sa r e a l l o w e d
power.Today,we seemto
relativefreedomfrom control,but pay for this with diminished
movetowardsa decreaseof control over language(e.9.the greatervarietyof accents

Thesemiotic landscaPe '

29

i n c r e a s i npgr o b l e misn e n f o r c i nngo r m a t i vsep e l l i n ga) ,n d

a l l o w e do n t h e p u b l i cm e d i a , t h e
andcontroloverthe visual(e.g.the useof imagebanks
in
codification
increase
an
towards
be drawnfor the constructionof visualtexts,and,
can
images
ready-made
which
from
imaging
technology).
generally,
computer
effect
of
the
also.Compare,for
can be found elsewhere
meaning
control
over
forms
of
The two
a
u
t
h
o
r
i
t
a
t
i
v
e
'
v
o i oc feG o d 'n a r r a t o r
w
h
i
c
h
a
n
i
n
f
i
l
m
d
o
c
u
m
e
n
t
a
r
y
c
l
a
s
s
i
c
e x a m p l et h, e
m
o
d
e r n' d i r e c tc i n e m a '
m
o
r
e
t
o
t
h
e
r
e
a
l
i
t
y
o
f
r
e
c
o
r
d
e
d
i
m
a
g
e
s
i
n
t
e
r
p
r
e
t
s
e x p l a i nas n d
n f i m a g e sa n d i n t h e
d o c u m e n t a riyn, w h i c hc o n t r o lo v e rm e a n i n gl i e si n t h e s e l e c t i o o
0r thinkof the
edited
together'
images
are
these
in
ways
which
noticeable
hardly
sometimes
'whatthetext says'
'cultural
on
analysing
emphasis
an
studies',
of
in
the
field
way in which,
s a dt h es a m et e x t ' ,
i s g r a d u a l lbye i n gr e p l a c ebdy a n e m p h a s losn ' h o wd i f f e r e nat u d i e n c er e
by diverting
which,
of
interpretation
freedom
the
apparent
words,
on
in
an emphasis/ other
on this
imposes
text
the
which
limitations
the
allows
the
itself,
text
attentionawayfrom
'freedomof reading'to remaininvisible,
all the moreefficacious
and therefore,perhaps,
a n dp o w e r f u l .
I n t h i sc o n n e c t i ot h
n eb a c k g r o u nodf t h e B r u n ab o o ki s w o r t hb r i e fm e n t l o nI t. w a sf i r s t
p r i n t e di n 1 9 5 3 ,i n A m s t e r d a ma,n dr e p r i n t e m
d a n yt i m e si n i t s c o u n t r yo f o r i g i nT. h ef i r s t
p
e r h a pn
s o a c c i d e n tU. n l i k et h e B r i t i s ht,h e
i
s
l
a
g
p
r
i
n
t
i
n
g
T
h
e
t
i
m
e
w
a
s
1
9
7
8
.
in
British
that their countrydid not havea
recognized
century,
Dutchhad,early in the twentieth
'commonculture',but wasdividedintogroupscharacterized
by differentandoftenoppos(
l
i
t
e
r
a
l
l
y
'
c
o
l
u
m
n
s
'
,
'
p
i
l
l
a
r
s
'
)
,
D
u
t c hh a v ec a l l e dt h e m .D u t c h
t
h
e
a
s
i n g l d e o l o g i ezsu, i l e n
7920sa systemin which
late
in
the
from
its
inception
had
for example,
broadcasting,
(i.e.'groups
a particularviewof
towards
groups
orientated
differentlevensbeschouwelijke
to the size
according
time
air
were
allotted
which
organizations
life,) ran broadcasting
o
n
t
e
l e v i s i o nb,e
a
n
d
l
a
t
e
r
r
a
d
i
o
o
n
w
o
u
l
d
,
s
a
m
e
e
v
e
n
t
s
t
h
e
p .h u s
o f t h e i r m e m b e r s h iT
other
most
positions,
while
ideological
discursive
of
from a variety different
interpreted
o
r
g
a
n
i
z
ations
g
o
v
e
r
n
m
e
n
t
r
u
n
,
b
r
o
a
d
c
a
s
t
i
n
g
u
s
u
a
l
l
y
E u r o p e acno u n t r i ehs a d c e n t r a l i z e d ,
populawhole
context,the
this
in
to
reach,
For
message
a
message.
with oneauthoritative
and,aswe
constructions
to a varietyof culturaland ideological
tion,it hadto beadaptable
the
Perhaps
that.
exactly
to
achieve
medium
visual
haveseen,the Brunabool<susethe
now,
is
there
that
shows
and
Australia
Brltaln
like
of the seriesin countries
belatedsuccess
' c o m m o nC u l i n t h e s ec o u n t r i etso o ,a n i n c r e a s i nagw a r e n e st hs a t t h e yn o l o n g e rh a v ea
divided,and
ture', andthat, instead,they havebecomecomplex,diverseand discursively
(although
went
system
zuilen
Dutch
the
thereforein needof newformsof communication
i n d e c l i n fer o mt h e 1 9 6 0 so n w a r d s ) .
g e t w e e lna n g u a gaen di m a g e , w h i cwhe s u g g e s t
T h ec h a n g l ndgi s t r i b u t i o nosf m e a n i n b
in the SovietUnionof the
experiments
various
by
is now in full flow,was foreshadowed
and Bakhtinwrote of
Voloshinov
lil<e
scholars
literary
earlyI920s. While linguistsand
' h e t e r o g l o s s icco' ,n s t r u c t i v i a
'
m
u
l
t
i
a
c
c
e
n
t
u
a
t
'
s tr t i s t s
a
n
d
l a n g u a gaes s o c i a l l yd i v i d e d ,
natr
e
j
e
c
ted
E
i
s
e
n
s
t
e
i
n
,
l
i
k
e
f
i
l
m
m
a
l
<
e
r
s
a
n
d
R
o
d
c
h
e
n
k
o
,
l i k e M a l e v i c hE, l L i s s i t z kay n d
new,
communicating
of
capable
language,
vlsual
uralismanclbeganto elaboratea new
moreabstractand
Then,as now,imagesbecamemorestylized,
ideasvisually.
revolutionary
with language,
compared
explicitly
was
visual
language
more obviouslycoded:the new

30

The semiotic landscape

with hieroglyphic
writing,with the stylizedmasksof kabukitheatre.Then,as noWvisual
was alsoseenas transparent:
coloursand shapeswerethoughtto havea
communication
d i r e c t ,u n m e d i a t e d , ' p s y c h o l o g i c a l ' i map ancotn, - s e m i o tci ca p a c i t yf o r s t i r r i n gt h e e m o was to be removedfrom the
tions of the 'masses'.Then,as now,visualcommunication
sphereof art, to becomepart of the morepowerfuland more publicsphereof industrial
production,
design,architecture.
Thissemioticrevolutionwas alliedto the
of typography,
s n df i l m sh a da p r o p a g a n d i s tpi u
c r p o s -e t h e y
p o l i t i c arl e v o l u t i o nc:o n s t r u c t i v i ps o
t s t e ra
s o u g htto h e l pb r i n ga b o u ta c u l t u r a rl e v o l u t i o na,n dt h e yh a dt o g e tt h e i rm e s s a gaec r o s s
to a socialla
y n d l i n g u i s t i c a lhl ye t e r o g e n e opuosp u l a t i o nT.h ev i s u a lt,h o u g h t o b e a b l et o
produce
a n e m o t i v ei m m e d i a cw
y ,a st o b et h e m e d i u mt h a t c o u l da c h i e v teh i s .I n t h e e n d ,
. a sc r u s h e b
d y S t a l i n0. l d t h e n e ws e m i o t i o
c r d e rf a i l e dt o e s t a b l i siht s e l fp e r m a n e n t Il yt w
fashioned
centralistandrepressive
controlovermeaning(andwith it a returnto naturalist,
'bourgeois'arp
t )r e v a i l eodv e rc o n t r o b
a a t c o u l da l l o w
l y m e a n so f a f o r mo f p r o p a g a n dt h
pluralism
t . h i st i m e- t h o u g h w i t h v e r yd i f f e r e npt o l i t a n d i d e o l o g i c caol h e s i otno c o e x i s T
i c a l ,s o c i a lt,e c h n o l o g i caanl de c o n o m icco n d i t i o n-si t m a yn o tf a i l .
bookscan
in our discussion
of thetwo children's
Thesemioticshiftswe haveexemolified
to stylbe observed
elsewhere,
too. The shift from 'uncoded'naturalisticrepresentations
which
on the coversof newsmagazines,
ized,conceptual
imagescan be seen,for instance,
- photographs
recordingevents,or
usedto be dominatedby documentaryphotographs
p o r t r a y i nnge w s w o r t hpye o p l e . O c c a s i o ntahlilsys t i l lh a p p e n as s, i n f i g u r e1 . 4 ,b u t i n c r e a s inglythe photographs
on magazinecoversare contrivedand posed,usingconventional
newsworthy
events,
symbolsto illustratethe essence
of an issue,ratherthan documenting
as in figure1.5,wherea padlockand a UnitedStatesflag,againsta neutralbackground,
i l l u s t r a t et h e i s s u eo f t i g h t e n e d
b o r d e rc o n t r o l .U n l i k et h e p i c t u r eo f t h e ' B i r d i n T r e e '
( f i g u r e1 . 2 ) , t h e s a
c a g e sb,u t t h e ym i g h ta s w e l lb e d r a w i n g s .
e r es t i l l p h o t o g r a p hiim
As an exampleof the changingrelationbetweenlanguageand image,consideran
e x t r a c tf r o m a S c i e n c eC D - R O Mf o r t h e l o w e ry e a r so f h i g hs c h o o l( f i g u r e1 . 6 ) . H e r e
l a n g u a ghea sh e r eb e e nd i s p l a c ebdy t h ev i s u a al s d e c i s i v eal ys i n t h e B r u n ab o o k .I n s t e a d
o f t h e m a j o r m e d i u mo f i n f o r m a t i o nw,i t h t h e v i s u a la s ' i l l u s t r a t i o n 'i ,t h a s b e c o m ea
n .w o
m e d i u mf o r c o m m e not r l a b e l l i n gw,i t ht h ev i s u aal st h ec e n t r asl o u r c eo f i n f o r m a t i o T
q u e s t i o nnse e da s k i n go: n ei s t h e q u e s t i o o
n f i m p l i c i ct h a n g eisn n o t i o n sa n d p r a c t i c eos f
reading,and of readingsciencein particular;the other is the questionof changesin the
constitutionof what is represented
here,scienceitself.The students/viewers/users
of
the science
C D - R 0 Ma r e n o l o n g e ra d d r e s s evdi a t h e h i e r a r c h i c a lcl yo m p l e xs t r u c t u r e s
and its needto
of scientificwriting,with its specificdemandsfor cognitiveprocessing,
'translate'verba
f ol r m st o t h e i r t h r e e - d i m e n s i o o
n ravl i s u a le q u i v a l e n (t sa s o n t h e p a g e
r e p r o d u c eidn f i g u r e1 . 7 ) .T h e ya r e a d d r e s s elda r g e l yi n t h e v i s u a lm o d e a
, n d e i t h e ra s
' s c i e n t i s t s ' w huon d e r s t a nadb s t r a c t i ofnr o mt h ee m p i r i c a l lrye a l ,o r a s p e o p l ef o c u s i nogn
t h e e m p i r i c a l l rye a lw i t h t h e i n t e n t i o n
t o u n d e r s t a ntdh e r e g u l a r i t i el sy i n g ' b e h i n d ' t h a t
reality.In otherwords,eventhoughthe visualmodemightseemto providedirect access
t o t h ew o r l d ,i t i s a s a m e n a b lteo r e a l i z i ntgh e o r e t i c aplo s i t i o nass i s t h ev e r b a l .
M o r e c o m p l e xi s t h e q u e s t i o n
w h e t h e ri ,n t h i s d r a m a t i cs h i f t f r o m t h e v e r b a lt o t h e
visual,the veryconstitutionof the schoolsubjectScienceis undergoing
a transformation.

The semiotic landscape . 3l

fiq f,l Magazinecoverwith naturalisticphotograph(Newsweek,lgApril2004)

Can everything
that was communicable
in the formationof scientificwriting be said in
t h e s ev i s u a l l cy o n s t r u c t ef d
o r m s ?C o n v e r s eal yr ,et h e r ep o s s i b i l i t i e
o sf s c i e n t i f icco m m u n i c a t i o ni n t h ev i s u aw
l h i c hw e r en o t a v a i l a b lien t h e m o d eo f w r i t i n g ?A n dw h i c ho f t h e s ei s
a m o r ea p t m e d i u mf o r s c i e n t i f itch e o r y ?W i l l s c i e n t i f itch e o r i e sc h a n g ea s t h e f o r m o f
expression
shiftsfrom the writtento the visualmode?We cannottal<eup thesequestions
here,but if we are to makeourselves
conscious
of the far-reachingimplications
of these
changes
in the semioticlandscape,
theyneedat leastto be asked.
ch
I m p l i c i ti n t h i si s a c e n t r aql u e s t i o n , w h i n
e e dtso b ep u to p e n l ya,n dd e b a t esde r i o u s l y :
is the movefrom the verbalto the visuala lossor a gain?0ur answerat this stagein our
t h i n k i n gi s m u l t i p l eT. h e r ea r e l o s s e sa,n dt h e r ea r e g a i n s . 0 u ra r g u m e ntth r o u g h o ut th i s
bookis that differentsemioticmodes- the visual,the verbal,the gestural,
etc.- eachhave
p
o
t
e
n
t
i
a
l
i
t
i
e
s
their
a n dt h e i r l i m i t a t i o n A
s . m o v ef r o m a c e n t r arl e l i a n c o
e n o n em o d et o a
centralrelianceon anotherwill thereforeinevitablyhaveeffectsin both directions.But
that is notthe endof the story.We alsohaveto consider
what is represented.
It maybethat
visualrepresentation
is moreapt to the stuff of sciencethan language
everwas/or even
t h a t a s c i e n cw
e h i c hi s c o n s t r u c t evdi s u a l l yw i l l b e a d i f f e r e nkt i n do f s c i e n c eT.h ew o r l d
represente
v ids u a l l o
y n t h e s c r e e nosf t h e ' n e wm e d i a ' i sa d i f f e r e n t lcyo n s t r u c t ewdo r l dt o
that whichhad beenrepresented
on the denselyprintedpagesof the print mediaof some

The semiotic landscape

fi9 f.S Magazinecoverwith conceptualphotograph(ltewsweek,l2November

200I)

andfor meaning-making
thirty or forty yearsago.Theresources
it offersfor understanding
differfrom thoseof the world represented
in language,
andso do the citizensit produces.
Theseare far-reachingquestionsand they can only be answeredby considering
the
i n t e r c o n n e c t i obnest w e e tnh e c h a n g i n p
g o l i t i c a le, c o n o m i ac n d c u l t u r a lc o n d i t i o ngsa t h e r e d u p u n d e rt h e l a b e lo f g l o b a l i z a t i oann d t h e n e w p o s s i b i l i t i ef so r r e p r e s e n t a t i o n
We havebarelyhintedat these
affordedby the newmediaof production
anddissemination.
k i n d so f q u e s t i o ni sn o u r d i s c u s s i oonf t h e B r u n aa n d L a d y b i r db o o k sC
. o u l di t b et h ec a s e
that informationis now so vast,so complex,that perhapsit, hasto be handledvisually,
because
the verbalis no longeradequate?
M e r en o s t a l g i am/ e r es o c i aal n dc u l t u r a rl e g r e t so r p e s s i m i scma n n o ht e l ph e r eW
. e ,a l l
of us, haveour particularstandpoints
and our particularvaluescarriedforwardfrom
yesterdayor from the day beforeyesterday.
The first most importantchallengeis to
u n d e r s t a nt h
d i ss h i f t ,i n a l l o f i t s d e t a i la, n di n a l l o f i t s m e a n i n gF.r o mt h a t u n d e r s t a n d i n g ,
we can hopeto beginthe task of constructing
adequate
newvaluesystems.
T os u m m a r i z e :
(1) Visualcommunication
is alwayscoded.It seemstransparentonly because
we know
the codealready,
at leastimplicitly- but withoutknowingwhat it is we know,without

The semiotic landscape

fin f.O Contemporary

scienceCD-ROM

(e)

fig f.Z Earlytwentieth-century

sciencetextbook(McKenzie,Ig3g)

havingthe meansfor talkingaboutwhat it is we do whenwe readan image.A glance

at the 'stylized'artsof otherculturesshouldteachusthat the mythof transparency
is
i n d e e da m y t h .W e m a ye x p e r i e n ct hee s ea r t s a s ' d e c o r a t i v e , , , e x o t i c , , ' m y s t e roiro u s ,

The semiotic landscape

'beautiful',but we cannotunderstand
them as communication/
as formsof 'writing'
unlesswe are,or become,
membersof thesecultures.
(2) Societies
tendto developexplicitwaysfor talkingonlyaboutthosesemioticresources
w h i c ht h e yv a l u em o s th i g h l ya, n dw h i c hp l a yt h e m o s ti m p o r t a nrt o l e i n c o n t r o l l i n g
the commonunderstandings
theyneedin orderto function.Untilnow,language,
especiallywrittenlanguage,
hasbeenthe mosthighlyvalued,
the mostfrequentlyanalysed,
policedmodein our society.
the mostprescriptively
taughtandthe mostmeticulously
If, as we haveargued,this is now changingin favour of more multiplemeansof
representation,
with a strongemphasison the visual,then educationalists
needto
r e t h i n kw h a t w i l l n e e dt o b e i n c l u d e di n t h e c u r r i c u l ao f ' l i t e r a c y 'w, h a t s h o u l db e
t a u g h tu n d e ri t s h e a d i n g
i n s c h o o l sa,n dc o n s i d et rh e n e wa n ds t i l l c h a n g i n pg l a c eo f
writingas a modewithinthesenewarrangements.
If schoolsare to equipstudentsadequately
for the newsemioticorder,if they are not to
producepeopleunableto usethe new resources
of representation
activelyand effectively,
t h e nt h e o l d b o u n d a r i ebse t w e etnh e m o d eo f w r i t i n go n t h e o n eh a n da, n dt h e ' v i s u aal r t s '
on the other,needto be redrawn.Theformerhad traditionallybeenthat form of literacy
withoutwhich peoplecouldnot adequately
functionas citizensor as workers;the latter
had beeneithera marginalsubjectfor the speciallygifted,or a subjectwith limitedand
s p e c i a l i z eadp p l i c a t i o nas s, i n ' t e c h n i c adl r a w i n g 'T. h e n e w l yd e f i n e da r e aw i l l h a v et o
involve
t h e t e c h n o l o g i eosf t h e ' n e ws c r e e n s ' -t h e e l e c t r o n itce c h n o l o g i eosf i n f o r m a t i o n
a n d c o m m u n i c a t i ocne, n t r a n
l o wt o t h e s e m i o t i cl a n d s c a p B
e .u t a b o v ea l l , s u c ha c u r r i c u l u mi s c r u c i a l l yd e p e n d e notn h a v i n gt h e m e a n so f a n a l y s i st h, e m e a n sf o r t a l k i n ga b o u t
the'new lileracy',aboutwhat it is we do whenwe produceand readimages.As Iedema
(I99 4 : 64) notes,in the'post-Fordist' workpIace,
W o r k e r sm u s tb e m u l t i - s k i l l e d
a ,r t i c u l a t ea n d ' s e l f - s t e e r i n .g.' . t T h e y Jn e g o t i a t e
t h e i rj o b sa s m e m b e rosf ' q u a l i t yc i r c l e s ' a n cdo n s u l t a t i vceo m m i t t e eTs h
. i sr e q u i r e s
that workersare not merelycapableoI doingtheir work, but also that they are
capableof talkingandthinkingabouttheir work and its effectiveness.
(l(ress,2000;l(ressandvan Leeuwen,200L)
Elsewhere
we havetalkedof the needfor the
introductionof the conceptof design,both as a categorywith generalsignificance
in
r e p r e s e n t a t iaon dc o m m u n i c a t i oann,da s a c r u c i a cl a t e g o rfyo r d e v e l o p i nt h
g ec u r r i c u l a
o f i n s t i t u t i o n a l i z e d u c a t i o nw,h e t h e ri n t h e t r a d i t i o n asl c h o o ol r o t h e rf o r m a ls i t e so f
learning.
Thisis implicitalsoin the description
we havegivenearlierin this chapterof the
newformsof reading.This is not the placeto developthat point,thoughit is essential
to
draw attentionto its unavoidable
significance
as part of the urgentneedfor developing
a d e q u a tw
e a y sa n dt a l k i n ga b o u t h ev i s u a l .

Thesemiotic landscape

T H E S E M I O T I CL A N D S C A P E
in the context
Theplaceof visualcommunication
in a givensocietycanonlybe understood
public
in that
available
communication
of, on the onehand,therangeof formsor modesof
'the
semiotic
to
this
as
We refer
societyand,on the otherhand,their usesand valuations.
Thefeaturesof a
landscape'.
Themetaphoris worth exploringa little,as is its etymology.
(
a
g
r
o
u
p
b
u
i
l
d
i
n
g
so)n l ym a k es e n s e
, w o o da
of
l a n d s c a p e f i e l da
, c l u m po f t r e e sa, h o u s ea,
('waste
its
development
in the contextof their wholeenvironment
and of the historyof
ogn l ya s
h
a
s
m
e
a
n
i
n
l a n d ' h a sm e a n i nogn l yi n t h a t c o n t e x a
t ,s h a s ' f i e l d ' o r ' t r a c k ' ; ' v i l l a g e '
group
part
same
way,
land).
In
the
a
of buildings
that is
of a historyof waysof workingthe
p a r t i c u l a fr e a t u r e sa n d m o d e so f c o m m u n i c a t i osnh o u l db e s e e ni n t h e h i s t o r yo f t h e i r
which
develooment.
and in the environmentof all the other modesof communication
years
fifty
surroundthem.The useof the visualmodeis not the samenow as it was even
is
it
not
the
ago in Westernsocieties;
it is not the samefrom onesocietyto another;and
group
samefrom onesocial
or institutionto another.
Eachfeatureof a landscape
as a whole,andeach
hasits history,as doesthe landscape
is
is subjectto constantremaking.It is herethat the etymologyof the word landscape
a
r e v e a l i n gT.o t h e c a s u a b
l e h o l d ear l a n d s c a psei m p l yi s , a n d m a y e v e nh a v e t i m e l e s s
a p p e a r a n c( 'et h et i m e l e sbse a u t yo f t h e E n g l i s ho,r S p a n i s hc ,o u n t r y s i d eY
' )e. ti t i s i n f a c t a
productof socialactionand of a socialhistory,of humanwork on the land,on nature:
-scape,withits relationto shapein Englishandschaffen(both'to work'and'to create')in
G e r m a ni n, d i c a t et sh i s .A n dt h i sa p p l i eas l s ot o t h e' s e m i o t i lca n d s c a p eM' .e t a p h o r iecx c u r s i o n so f t h i sk i n dc a nb es t r e t c h etdo o f a r ; h o w e v e r , wwei l l a l l o wo u r s e l v eosn eo t h e rp o i n t
Landscapes
are the result,not just of humansocialwork, but alsoof the
of comparison.
characteristics
of the land itself.Theflat landby the riveris mostsuitablefor the grazing
or forestry.At the sametime,
of cattleor the growingof wheat;the hillsides
for vineyards
valuesof a culturemaydetermine
whichof the potentialusesof the land
the characteristic
are realized,whetherthe hillsidesare usedfor vineyardsor forestry,for example.And
c u l t u r avl a l u e sm a ye v e ni n d u c ep e o p l e
t o g o a g a i n stth e g r a i no f t h e l a n dt,o u s et h es t e e p
h i l l s i d ef o r g r o w i n gr i c e ,f o r e x a m p l ew, h i c ho p p o s etsh e ' n a t u r a lp o t e n t i a l ' o ft h e l a n d
a l m o st o t h el i m i t .
andpotentialareshapedbothby the intrinsiccharacteristics
Semioticmodes,
similarly,
i t i e so f t h e m e d i u ma n d b y t h e r e q u i r e m e n thsi,s t o r i eas n d v a l u e so f s o c i e t i easn d t h e i r
c u l t u r e sT.h ec h a r a c t e r i s t iocfst h e m e d i u mo f a i r a r en o tt h e s a m ea st h o s eo f t h e m e d i u m
of stone,andthe potentialities
of the speech
organsare notthe sameasthoseof the human
h a n dN
. e v e r t h e l e s s , c u l t au nr adsl o c i avl a l u a t i o nasn ds t r u c t u r esst r o n g l ay f f e c t t h eu s e so f
hashad
writtenlanguage
thesepotentialities.
It is not an accidentthat in Westernsocieties
p
l
a
c
e
l o d eh a s
w h i c hi t h a sh a df o r t h e l a s tt h r e eo r f o u r m i l l e n n i aa,n dt h a t t h ev i s u am
the
in writing.Western
in effect becomesubservient
to language,
as its modeof expression
l i n g u i s t itch e o r i e h
s a v em o r eo r l e s sn a t u r a l i z et dh e v i e wt h a t t h e u s eo f a i r a n dt h e v o c a l
But evenspeechis, in the
organsis the natural,inevitablesemioticmeansof expression.
e n d ,c u l t u r a lW
. e a r e n o t b i o l o g i c a l lpyr e d i s p o s et od u s es p e e c ha s o u r m a j o r m o d eo f
of
Thepartsof the bodythat we callthe tspeech
organs'arean adaptation
communication.

ta

The semiotic landscape

p h y s i c aol r g a n si n i t i a l l yd e v e l o p et d
o p r e v e nht u m a n sf r o m c h o k i n gw h i l eb r e a t h i n a
gn d
eating.Whenthe needarises,we cananddo useothermeansof expression,
as in the highly
articulated evelopmen
o tf g e s t u r ei n s i g n l a n g u a g eas n
, d a l s o i n t h e a t r i c am
l i m ea n d
certain Easternforms of ballet.And, while theseare at presentrestrictedto relatively
m a r g i n adl o m a i n sw, h o i s t o s a yt h a t t h i sw i l l a l w a y sr e m a i ns o i n t h ef u t u r ed e v e l o p m e n t
o f h u m a n k i n dI?t i s s a l u t a r yt o c o n s i d ehr o wo t h e rc u l t u r e s ' r a n k ' m o d eosf c o m m u n i c a tion, and to bring that knowledgeinto the mainstreamof 'Western'thinking(see,for
instance,
Finnegan,
2002).
T h e n e wr e a l i t i e o
s f t h e s e m i o t i cl a n d s c a paer e b r o u g h a
t b o u tb y s o c i a lc, u l t u r a a
l nd
e c o n o m i cf a c t o r s :b y t h e i n t e n s i f i c a t i oonf l i n g u i s t i ca n d c u l t u r a ld i v e r s i t yw i t h i nt h e
boundaries
of nationstates;by the weakening
of theseboundaries
withinsocieties,
dueto
m u l t i c u l t u r a l i seml e, c t r o n im
c e d i ao f c o m m u n i c a t i ot e
n c, h n o l o g i o
e fst r a n s p o rat n dg l o b a l
e c o n o m idce v e l o p m e nG
t sl.o b afl l o w so f c a p i t a a
l n d i n f o r m a t i oonf a l l k i n d so, f c o m m o d i t i e s ,a n d o f p e o p l ed, i s s o l v ne o t o n l yc u l t u r a a
l n d p o l i t i c a bl o u n d a r i ebsu t a l s os e m i o t i c
boundaries.
Thisis alreadybeginning
to havethe mostfar-reaching
effectson the characteristico
s f E n g l i s h( a n dE n g l i s h e g
s )l o b a l l ya,n de v e nw i t h i nn a t i o n abl o u n d a r i e s .
T h ep l a c eu, s ef,u n c t i o na n dv a l u a t i o on f l a n g u a gien p u b l i cc o m m u n i c a t i oi snc h a n g i n g .
I t i s m o v i n gf r o m i t s l o r m e ru, n c h a l l e n g e
r odl ea s t h em o d eo f c o m m u n i c a t i o n ,at or o l ea s
one modeamongothers,to the function,for instance,
of beinga modefor comment,for
r a t i f i c a t i o no,r f o r l a b e l l i n ga,l b e i tm o r es o i n s o m ed o m a i n st h a n i n o t h e r sa, n d m o r e
r a p i d l yi n s o m ea r e a st h a ni n o t h e r sA. l t h o u g h
t h i si s a r e l a t i v e lnye wp h e n o m e n oi nnp u b l i c
c o m m u n i c a t i ocnh,il d r e nd o i t q ui t e ' n a t u r al yl ' i n t h e i rt e x t - m a k i n g .
N e w w a y so f t h i n k i n ga r e n e e d e di n t h i s f i e l d .H e r ew e u s e /o n c em o r e ,c h i l d r e n , s
representation
as a metaphorto suggestsomedirections.The drawingsreproducedin
f i g u r e1 . 8 w e r em a d eb y a f i v e - y e a r - o bl do y . 0 n a s u m m e rS u n d a ya f t e r n o o nw, h i l eh i s
parentswereentertaining
friends,the childtook a small,squarenotepadfrom nearthe
t e l e p h o naen dd r e wa p i c t u r eo n e a c ho f s i x p a g e sH. i s f a t h e rh a dn o t n o t i c e d
t h i s u n t i lh e
c a m ea c r o s sh i m i n t h eh a l lo f t h e i rh o u s ew, h e r et h ec h i l dw a sp u t t i n gt h ec a r d s' i n o r d e r ' ,
as shownin figure1.8. Askedwhat he was doing,the child'saccountwas as follows:for
picturesI and2 together'Meandthe dog are in life,so they'rein the correctorder,;on
picture3
s a n d4 ' T h ef f y i n gb o m bi s i n t h e a i r a n dt h e p l a n ei s i n t h e a i r ,s o t h e y , r ei n t h e
correctorder';andon 5 and6'The patternsare in the correctorder,.
The wholeprocess/
involvingsign-making,
representation
and classification,
had proc e e d e dt h r o u g ht h e v i s u a lm e d i u mI.t w a s o n l y w h e nt h e p a r e n tc a m ea l o n gw i t h h i s
questionthat the childwasforcedto usewords.Themetaphoric
processes
of sign-making,
the actsof representation
quite
and classification,
eachinvolving
complexanalogies,
took
place in the visual mode.Language,
as speech,enteredwhen communication
with the
parentbecamenecessary.
Speech
wasthe modeusedfor'ratifying'andfor describing
what
hadtakenplacewithoutit.
Sometwo weekslater,at the end of the summerterm of his primaryschool,the child
b r o u g h ht o m es o m eo f h i se x e r c i sbeo o k sA. m o n gt h e s ew a st h e p a g es h o w ni n f i g u r e1 . 9 .
Clearly,
herethe taskwasoneof classification,
and it hadbeenundertaken
at school,prior
t o t h e m a k i n ga n d o r d e r i n go f t h e d r a w i n g si n f i g u r e1 . 8 ,a t h o m e A
. w h o l es e q u e n coef

The semiotic landscape

37

4,,.

/ t t p
l*gL

\a'r
" fu

uu^'

. . M EA N O
T H E D O GA R EI N L I F E ,S OT H E Y ' R EI N T H E C O R R E CO
TRDER"

. . T H EF L Y I N GB O M B
I S I N T H EA I R A N DT H E P L A N ET SI N T H EA I R ,S OT H E Y ' R EI N T H E C O R R E CO
TRDER''

Y-

/U,N
r,zN!,1

, , T H EP A T T E R N S
A R EI I { T H E C O R R E CO
TRDER''

Fig 1.S Six drawingsby a five-year-old

boy

and developsemioticactivitiesis thus involved,

of production,
transformation
a sequence
ment,movingfrom the initialtask of joiningimagesof the sameobjects- a classificatory,
cognitive,
conceptual,
semioticand manualtask - to that of producingcomplexand diss i m i l a ri m a g e sa, n d f i n d i n gl i k e n e sisn t h e m ( o r i m p o s i n g
l i k e n e sosn t h e m )t h r o u g ha n
intermediary
task of abstractionand generalization.
If we think aboutthis periodof two
w e e k st,h e c h i l d ' sp r o d u c t i oonf s i g n si n v o l v e ad s e r i e so f d i s t i n c st e m i o t i cm o d e sa, n d o f
translations
betweensuchmodes.Firstthe teachersnokewith the childrenaboutthe task
(mode:language
as speech);
thensheintroduced
the bookand showedthemwhat was at
i s s u e( m o d e : 3 Dp h y s i c aol b j e c ta, n dv i s u a m
l o d e )t;h e nt h e c h i l d r e nu s e dt h e i rp e n c i ltso
: anuaa
d r a wt h e c o n n e c t i nlgi n e s( m o d e m
l c t i o na n d v i s u a lm o d eo f d r a w i n g )t;h e nt h e
on their
comments
teacherengaged
the childrenin spokendiscussion,
and madeevaluative

38

The semiotic landscape

15

{ /-

\-

1/s.

.\\L
ll APt
Drowo linet6 jbin thethingswhich

fig f.e Schoolexercise

bookof afive-year-otd
boy

work.Thiswasfollowedby a long periodof 'silence',a fortnightor so whennothingwas

s e e no r h e a r db, u t w h e n , w a
e s s u m e , t hsee r i e o
s f t r a n s f o r m a t i vaec t so f t h e c h i l dc o n t i n u e d
'internally','mentally'.
F i n a l l yt h e i n t e r n aal c t i v i t yb e c a m ev i s i b l el,i t e r a l l yt ,h r o u g ht h e
child'sunpromptedproductionof the drawings,his unpromptedclassificatoryactivity
( s p a t i a l lsyh o w na) n dh i ss p o k e n
c o m m e n t a riyn r e s p o n st e
o h i sf a t h e r , q
s uestion.
0 f c o u r s ew, h i l ea l l t h i s t o o k p l a c et h e c h i l d ,a s d o a l l o f u s ,w o u l dn o d o u b th a v e
experienced
constantly
shiftingaffective,
emotionalstates.Hemighthavebeenenthused
by
thetask in the classandpraisedby theteacherfor hissuccess;
he mighthavehada difficult
t i m ew i t h h i sf r i e n d si n t h e p l a y g r o u n o
d r, a t h o m ea, n ds o o n ,a n da l l o f t h i sw o u l dh a v e
i n f f u e n c ehdo wh e ' r e a d ' t h ea c t i v i t ya n dh o wi t w a s ' t a k e nu p ,b y h i m .I f w e s e ei t l i k et h i s ,

The semiotic landscape

In other
it makesit impossible
to think of affect and cognitionas distinct,as separable.
as
words,here- asalways- the affectiveaspects
arealwaysonewith,andact continuously
a ' m o d a l i t yo' n ,c o g n i t i vsee m i o t i p
crocesses.
made
In part in response
to the representational,
semioticand cognitiveresources
availableby the teacher,
and her demandsmadein the class,thoughafterwardsprompted
modes(including,
by hisown interests,
the childuseda seriesof differentrepresentational
productive
representations')
sequence
of semioticactivof course,'internal
in a constantly
i t i e s .S o m eh a p p e n ewdi t h i nt h e s a m em o d e( l i n k i n gt h e i m a g e o
s f t h e o b j e c tb y a l i n ef,o r
instance),
sometook placeby a shift acrossmodes(theshiftfrom the spatiallyperformed
transformaare constantly
classification
to the spol<en
commentary
on it). Suchprocesses
(our namefor
tive (thenamewe usefor suchorocesses
withinonemode)andtransductive
such processes
acrossmodes).All these,we assume/haveeffectson 'inner resources',
whichconstantlyreshape(transform)the subjectivity
of the child.
seemed
As we haveindicated,
the visual,actionalandspatialmodes,ratherthanspeech/
Speechwas usedfor communito be the centralrepresentational
andcognitiveresources.
It maywell
cationwith adults,as a meansfor translation,
for commenlandfor ratification.
b e t h a t t h e c o m p l e x i t i er e
s a l i z e di n t h e s i x i m a g e sa n d t h e i r c l a s s i f i c a t i owne r ei n i t i a l l y
but that the
beyondthe child'scapacityof spokenexpression,
conception
andformulation,
v i s u am
l o d eo f f e r e dh i ms e m i o t i a
c n dc o g n i t i v ree s o u r c ewsh i c hw e r en o t a v a i l a b lteo h i m
throughthe
in the verbalmode.Howeverl
onceexpressed
in the visualmode,onceclassified
visual/spatia
ml o d e ,t h e m e a n i n gw
s h i c ht h e c h i l d h a d p r o d u c e db e c a m ea v a i l a b l ea s
externalized,
objectiveexpression;
this in turn may havemadethem differentlyavailable
processes
for verbalexpression,
for the verbalratificationof semiotic,affective/cognitive
that had alreadytaken place.
- betweena
- transduction
This incessantprocessof 'translation',or 'transcoding'
of
understanding
rangeof semioticmodesrepresents,
we suggest,
a better,
a moreadequate
is not
here,language
representation
andcommunication.
In the examplewe havediscussed
the sameis eitherthe casealready,
at the centre.In manyareasof publiccommunication
c o d e so f r e p r e s e n o r r a p i d l yc o m i n gt o b et h e c a s eA. n dc l e a r l yi t m a t t e r sw h i c hs e m i o t i m
t a t i o na n dc o m m u n i c a t i oa nr ed o m i n a n m
t , o s tf r e q u e n m
t , o s tv a l u e di n t h e p u b l i cd o m a i n s
in whichwe act.
F i g u r e1 . 1 0 c o m e sf r o m w o r k d o n eb y t w o ( 1 3 - y e a r - o l ds)t u d e n t isn S c i e n c ei n, t h e
canbe asked.Thefirst is,What
earlyyearsof secondary
schoolin England.
Twoquestions
andthe second:
of science?,
is the effectof the modeof representation
on the epistemology
'Do differentmodesof representation
facilitate,or rule out, differentaccountsof natural
of
phenomena?'
To answerboth,we needto comparetwo differingmodalrepresentations
'the same'issue.
T h i st i m eo u rq u e s t i oins n o t ' W h a ti st h es t a t u so f w r i t t e nl a n g u a gien t h e s et e x t s ?b' u t
s t e r m so f e p i s t e m o l o gi n,
r a t h e r ' W h a ti s t h e e f f e c to f t h e d i f f e r e nm
t o d a lr e a l i z a t i o ni n
If we comparefigure1.10 with just one
termsof the students'perspective
on knowledge?'
- the task'to write a storyof the journeyof
examplefrom anotherexercise
for assessment
a r e db l o o dc e l la r o u n dt h e b o d y ' - w e c a ns e et h e r u d i m e n tosf t h a t d i f f e r e n c eH.e r ei s a
briefextractfrom onesuch'story',in this casewrittenin the genreof'diary':

Thesemiotic landscape

y"

sr\4oKrNG
I
I
\

Drooogoes

o'oo'"""

1Yi{1'9T
the hean
In aneites

/-

/
\

EilI
rroNoxrpE
I

\ '""un"
Y i n

[;;I

\ ,, ,o,'"0
i o t

[;;;]

rhe head
pumps rhe
\ blood around
ooav
\v.
,,/

1,
,/
,/ ..
ne

I ruruesI
I
l----.-.-- | eLooo| -=>
*
serslxysenr,om /a-=

n;ii,;:a:"
//
the lunos

---a

/
,/

broodsetsl

I HEARrI

t;;]

the blood rerums

to the hean th@sh
rhe verns

I contains

F;;;l
cELLs
I,

I maKe

| *",r. u.*o I
|

cELis

',1""Yrfl"
l"*tto I
intestines

F'--I
I
f

I ,*"

+
{"on,u,n"
I

|-;I

'I

..
oo@ conrarns

"n"ro"
maoJin

lnNrraooresl

|;;;I

G;t
..
DroxrDE w;;
I
I

t--_l

'

' productof
respiration

fiq f.fO Conceptmapr'bloodcirculation'

DearDiary,I havejust leftthe heart.I hadto comefrom thetop ofthe rightchamber

of the heart( Rightatrium)andsqueeze
my waythroughto the Rightventriclewhere
the heartbeatgot stronge4
and I left the heart.
Dear Diary,I am currentlyin the lungs,it is terriblycrampedin hereas the
capillaries
aretiny andthereare millionsof us.We havejust droppedoff oxygenand
w e p i c k e du p s o m ec a r b o nd i o x i d e .
DearDiary,we haveenteredthe Iiverwherewe hada thoroughwash.
DearDiary,we havejust left the kidneywherewe droppedoff somewaterwhich
w i l l b et u r n e di n t ou r i n e .
Dear Diary,I havefinishedmy journeyaroundthe bodyby stoppingoff at the
heart.
In the diary,thefundamental
principle,
organizational
or logic,isthat of 'sequence
in time,,
andthefundamental
principleis that of actionor eventor,lessfrequently,
representational
statesof affairs.Objectsare relatedto other objectsby actionsrepresented
by verbs('we
haveenteredthe liver','wehavejust left the kidney').Theactionsand eventsthemselves
are arrangedin temporalsequence/
mirroringthat of the imagined
eventsastheyhappened
in the world. In the conceptmap,on the other hand,the fundamentalorganizational
principleis that of a conceptualorder,realizedby the spatial arrangements
of the

The semiotic landscape

4t

individualconcepts.
Hereobjectsare related,not by actions,but by hierarchy,
by signific a n c ed e r i v i n fgr o m r e l a t i o no f ' p r i o r i t y o
' f v a r i o u sk i n d s .
All theseexamples
revealwhat has in fact alwaysbeenthe case:language,
whetherin
speechor writing,hasalwaysexistedas just onemodein the ensemble
of modesinvolved
in the productionof texts,spokenor written.A spokentext is neverjust verbal,but also
visual,combiningwith modessuchas facial expression,
gesture,postureand otherforms
of self-presentation.
A written text, similarly,involvesmore than language:it is written
on something,
on somematerial(paper,wood,vellum,stone,metal,rock,etc.) and it is
written with something(gold,ink, (en)gravings,
dots of paint,etc.);with lettersformed
as typesof font, influenced
pragmaticand other considerby aesthetic,psychologicai,
ations;and with layout imposedon the materialsubstance,
whetheron the page,the
computerscreenor a polishedbrassplaque.Yet the multimodalityof writtentexts has,
by and large,beenignored,whetherin educational
contexts,in linguistictheorizingor in
p o p u l a rc o m m o ns e n s eT. o d a yi,n t h e a g e o f ' m u l t i m e d i ai' t, c a n s u d d e n lby e p e r c e i v e d
again.
W e c a n s u m m a r i zteh i s d i s c u s s i oi n t h e f o r m a s e t o f h y p o t h e s e(sa:) h u m a ns o c i e t i e s
(b) eachmodehas,inherently,
usea varietyof modesof representation;
differentrepre(c) each mode has
sentationalpotentials,different potentialsfor meaning-making;
specificsocialvaluationin particularsocialcontexts;(d) differentpotentialsfor meaningm a k i n gm a y i m p l yd i f f e r e npt o t e n t i a lfso r t h e f o r m a t i o no f s u b j e c t i v i t i e(se;) i n d i v i d u a l s
usea rangeof representational
modes,and thereforehaveavailablea rangeof meansof
(f) the differentmodes
meaning-making,
eachaffectingthe formationof their subjectivity;
of representation
are not held discretely,
separately,
strongly
as
boundedautonomous
d o m a i n isn t h e b r a i no, r a sa u t o n o m o ucso m m u n i c a t i o nr a
e ls o u r c ei ns c u l t u r en, o ra r et h e y
(g) affectiveaspectsof
deployed
discretely,
eitherin representation
or in communication;
humanbeingsand practicesare not discretefrom othercognitiveactivity,and therefore
(h) each
neverseparateor absentfrom representational
and communicative
behaviour;
modeof representation
hasa continuously
evolvinghistory,in whichits semanticreachcan
contractor expandor moveinto differentareasof socialuseas a resultof the usesto
w h i c hi t i s p u t .
None of these hypotheses
would, we imagine,attract significantdisagreement,
e s p e c i a lw
l yh e np u t s i n g l yJ.o i n t l yt h e yr e p r e s e natc h a l l e n gteo t h ee x i s t i n cgo m m o ns e n s e
on the relationsbetweenlanguage
andthoughtand in mainstream
theoriesand practices
i n a l l a r e a so f p u b l i cc o m m u n i c a t i o T
nh
. i s i s a c r u c i a lf e a t u r eo f t h e n e w s e m i o t i c
landscape.

A N O T EO N A S O C I A LS E M I O T I CT H E O R YO F C O M M U N I C A T I O N
i n o r d e rt o f u n c t i o na s a f u l l s y s t e mo f c o m m u n i c a t i ot h
ne
, v i s u a ll,i k ea l l s e m i o t i m
c odes,
hasto serveseveralrepresentational
andcommunicational
We
adopted
requirements. have
t h e t h e o r e t i c anl o t i o no f ' m e t a f u n c t i o n ' f r o m
t h e w o r k o f M i c h a e lH a l l i d a yf o r t h i s

42 .

The semiotic landscape

purpose.
interpersonal
Thethreemetafunctions
whichhe positsare the ideational,the
and
t h e t e x t u a l . l nt h e f o r m i n w h i c hw e g l o s st h e m h e r et h e ya p p l yt o a l l s e m i o t i cm o d e s ,
andare not specificto speechor writing.
The ideationalmetafunction
by
aspectsof the worldas it is experienced
Any semioticmodehasto be ableto represent
h u m a n sI n
. o t h e rw o r d si,t h a st o b ea b l et o r e p r e s e notb j e c t a
s n dt h e i rr e l a t i o nisn a w o r l d
outsidethe representational
system.Thatworld may of coursebe,and mostfrequentlyis,
alreadysemiotical
ly represented.
In doingso,semioticmodesofferan arrayof choices,
of differentwaysin whichobjects,
Twoobjectsmay
can be represented.
andtheir relationsto otherobjectsandto processes,
as involvedin a processof interactionwhichcouldbe visuallyrealizedby
be represented
vectors:

Fiq l.tl

vector

But objectscanalsorelatedin otherways,for instancein termsof a classification.

They
wouldbe connected,
not by a vectorbut,for instance,
by a'tree'structure:

rlg f.fZ Treestructure

precisely
In chapters2 and3 we will investigate
whichideationalchoicesare available
f o r v i s u asl i g n - m a k i ni n
g t h i sw a y .
The interpersonalmetafunction
Any semioticmode has to be able to projectthe relationsbetweenthe producerof a
(complex)sign,andthe receiver/reproducer
of that sign.Thatis,anymodehasto beableto
representa particularsocialrelationbetweenthe producer,
the viewerand the object
represented.
As in the caseof the ideationalmetafunction,
modesoffer an array of choicesfor
r e p r e s e n t i nd gi f f e r e n t ' i n t e r p e r s o nr ealla' t i o n s ,o m eo f w h i c hw i l l b ef a v o u r e d
i n o n ef o r m
(say,in the naturalisticimage),othersin another(say,in the
of visual representation

The semiotic landscape . 43

d i a g r a m )A. d e p i c t epde r s o nm a yb es h o w na sa d d r e s s i nvgi e w e rds i r e c t l yb,y l o o k i n ga t t h e

camera.This conveysa senseof interactionbetweenthe depictedpersonand the viewer.
andthisconveys
But a depictedpersonmayalsobeshownasturnedawayfrom the viewer,
the represented
scrutinize
viewer
to
It
allows
the
the absenceof a senseof interaction.
in
a
display
case.
characters
as thoughtheywerespecimens
bothin terms
choices,
theseandotherinterpersonal
In chapters4 and5 we will discuss
featuresthat
visual
of
the
in
terms
represented,
and
that canbe
of the kindsof interactions
realizetheseinteractions.
The textual metafunction
of signswhichcohere
Any semioticmodehasto havethe capacityto form fexfs,complexes
with the contextin and for whichtheywere
both internallywith eachotherand externally
differentcomavailable:
produced.
Here,too,visualgrammarmakesa rangeof resources
In figure
positionalarrangements
to allow the realizationof differenttextualmeanings.
t
h
1 . 1 ,f o r e x a m p l et h, e t e x t i s o n t h e l e f t a n dt h e p i c t u r eo n t h e r i g h t .C h a n g i n g e l a y o u t
(figure1.13) would completelyalter the relationbetweenwritten text and imageand
the meaningof thewhole.Theimage,ratherthanthewrittentext,wouldnowserveas point
o f d e p a r t u r ea,s ' a n c h o r ' f o rt h e m e s s a g eI n. c h a p t e r6 w e w i l l d i s c u s s u c hl e f t - r i g h t
r e l a t i o n s h ia
pn
s do t h e rc o m p o s i t i o nrael s o u r c e s .

: e . ,
'

" , i" : , " .j t- ' "i .. , '

.j

eli*hr

i ir*,r* lY:.y ** il:

h*{* r*. 1 qr:

I*

fig f.fl

Alteredlayoutof tigurel.I (left-right reversal)

**lj

44 .

The semiotic landscape

Ourfocusis on the description

of theseideational,
interpersonal
andtextualresources
as they are realizedin the visualmode.We recognizethat in doingthis work we are
engagedin morethan 'meredescription',
and participateourselves
in the reshaping
of the
s e m i o t i cl a n d s c a paen; dw e r e a l i z ea l s ot h a t t h i s i s a h i g h l yp o l i t i c ael n t e r p r i s e .

etations:
2 N a r r a t i v er e p r e s n
d e s i g n i n sg o c i a a
l ction
INTRODUCTION
T h e p i c t u r e ss h o w ni n f i g u r e2 . 1 a r e t a k e nf r o m a n A u s t r a l i a np r i m a r y - s c h osool c i a l
s e t r a d i t i o n atle c h n o l o goyf t h e
s t u d i e tse x t b o o l(<Q a k l e ye t a 1 . , 1 9 8 5 ) . 0 n er e p r e s e n t h
A u s t r a l i aA
n b o r i g i n etsh,e o t h e rt h e s u p e r i otre c h n o l o goyf t h o s ew h o i n v a d etdh e i rt e r r i that was capableof changingthe face of the earth.
tory ('The Britishhad a technology
were
and their weapons
Theirtoolswereableto work fasterthanthoseof the Aborigines
(an axe,a basketanda wooden
Theformerhasthreemainelements
muchmorepowerful').
s w o r d ) t, h e l a t t e rf o u r ( t h e ' B r i t i s h ' ,a s t h e y a r e c a l l e di n t h e c a p t i o nt,h e i r g u n s t, h e
A b o r i g i n easn dt h e l a n d s c a p eB) .u t t h e t w o p i c t u r e sd i f f e rn o t o n l yi n w h a te a c hi n c l u d e s
the right
the usersof the technology,
excludes
and excludes(the left picture,for instance,
pictureincludesthem),they differ also in structure:they relatetheir elementsto each
againsta
The elementsof the left pictureare arrangedsymmetrically,
other differently.
placed
in
size,
equal
as
are
represented
and
sword
axe,
basket
wooden
neutralbackground:
horizontal
the
way
towards
in
same
the
from
each
other
and
oriented
at equaldistance
andthe verticalaxes,sothat the pictureas a wholecreatesa relationof similaritybetween
The picturesays,as it were,that this axe,this basketandthis wooden
the threeelements.

Fig2.1TheBritishusedguns(0akleyeta/.,1985)

Narrative representati ons

swordall belongto the sameoverarching

category(a category,
incidentally,
which is only
implied,andwhichconflates
the notionof 'tools,andthe notionof 'weaoons,).
Theright picturerepresents
technology
in action.Wherethe left pictureis impersonal,
l . h e r et h e l e f t p i c t u r ei s s t a t i c t, h i s p i c t u r ei s d y n a m i cW
t h i s p i c t u r ei s p e r s o n aW
. here
t h e l e f t p i c t u r ei s d r y a n d c o n c e p t u atlh, i s p i c t u r ei s d r a m a t i c I. t r e l a t e st h e B r i t i s h
a n dt h e A b o r i g i n etsh r o u g ha t r a n s a c t i o n a
s cl h e m ai n w h i c ht h e B r i i i s hp l a yt h e r o l eo f
'Actor',the oneswho the deed,
do
andthe Aborigines
the role of 'Goal,,the onesto whom
the deedis done-the British stalktheAborigines,
onecouldsay.It alsorelatesthe lands c a p et o t h e B r i t i s ha n dt h e A b o r i g i n ei sn a ' l o c a t i v e ' w a y( t h e B r i t i s ha n dt h e A b o r i g i n e s
are in lhe landscape),
and the gun to the British in an 'instrumental'wav (the British
stalk the Aborigineswith their gun).
Theserelationscan be transformedinto linguisticform,as we havejust done,but the
pointis that heretheyare realizedby visualmeans.Thetransactional
relationbetween
the
B r i t i s ha n dt h eA b o r i g i n ei s r e a l i z e bd y t h ev e c t o r t h alti n k s t h e mn,a m e l y t hoeb l i q u el i n e s
formedby the glancesandoutstretched
armsof the Britishand by their guns.Thelocative
relationis realizedby overlapping,
by the gradientsof focus,the degreesof coloursaturation and so on,whichcreatethe contrastbetweenforegroundand background.
And the
i n s t r u m e n t rael l a t i o ni s r e a l i z e b
d yt h e g e s t u r e
o f h o l d i n gw, h e r et h e o b j e c th e l di s a t o o l .
The importantpointat this stageis not the detailof the analysis,
but the observation
that the semioticmodesof writing and visualcommunication
eachhavetheir own quite
p a r t i c u l a rm e a n so f r e a l i z i n gw h a t m a y b e q u i t e s i m i l a rs e m a n t i cr e l a t i o n sW
. hat in
I a n g u a gies r e a l i z e bd y w o r d so f t h ec a t e g o r y ' a c t i ovne r b si's v i s u a l l rye a l i z e b
d ye l e m e n t s
that can be formallydefinedas vectors.
What in languageis realizedby locativeprepositions is visuallyrealizedby the formal characteristics
that createthe contrastbetween
foregroundand background.
This is not to saythat all the relationsthat can be realized
l i n g u i s t i c a lcl ya na l s ob e r e a l i z e vdi s u a l l -y o r v i c ev e r s at,h a t a l l t h e r e l a t i o ntsh a t c a nb e
realized
v i s u a l l yc a na l s ob e r e a l i z e dl i n g u i s t i c a l lRy a
. t h e ra, g i v e nc u l t u r eh a sa r a n g eo f
g e n e r a pl ,o s s i b lree l a t i o nw
s h i c hi s n o tt i e dt o e x p r e s s i oi n a n yp a r t i c u l asr e m i o t i cm o d e ,
although
s o m er e l a t i o ncsa no n l yb er e a l i z evdi s u a l l a
y n do t h e r so n l yl i n g u i s t i c a lol yr ,s o m e
m o r ee a s i l yv i s u a l l ya n d o t h e r sm o r ee a s i l yl i n g u i s t i c a l lTyh. i sd i s t r i b u t i oonf r e a l i z a t i o n
possibilitie
a sc r o s tsh es e m i o t i m
c o d e si s i t s e l fd e t e r m i n ehdi s t o r i c a l layn ds o c i a l l a
y sw e l l
as by the inherentpotentialities
and limitationsof a semioticmode.
To returnto thetwo picturesin figure2.I,they canbesaidto represent
an aspectof the
experiential
world,technology.
But throughthe differentdesignpatternsselectedin each,
t h r o u g ht h e m a n n e ri n w h i c he a c hb r i n g si t s i n d i v i d u aell e m e n ttso g e t h e irn t oa c o h e r e n t
and meaningful
whole,they represent
the technologyof Aborigines
very differentlyfrom
the technologyof the British.Nothingabout Aboriginaltechnologynecessitates
it to be
represented
as a static,conceptual
taxonomy,
nor is thereanythingintrinsicabout British
technology
that requiresit to be represented
in a personalized
anddramatized
way.British
technologyis just as capableof beingrepresented
by a classificatory
schemeas is Abor i g i n atl e c h n o l o gayn, d i t i s j u s t a s p o s s i b lteo t e l l d r a m a t i cs t o r i e so f A b o r i g i n esst a l k i n g
their invaders
with spearsor woodenswordsas it is to tell suchstoriesaboutthe British.
evenif the former kind of story doesnot usuallyform part of mainstreamAustralian

Narrativerepresentations' 47

throughtwo distinctdisvisually,
is mediated,
history.In otherwords,the representation
'
k
n
o
w
n o h i s t o r y 'a; n dt h a t o f
c o u r s e st h: a t o f a n t h r o p o l o gf oy r A b o r i g i n apl e o p l e w h o
historyfor the whites who are not subjectsof anthropology.
imaginea reversalof thoserelations.Imagineon the left a catalogueof Britishtools
a n dw e a p o n sa ,n do nt h e r i g h ta p i c t u r ei n w h i c hA b o r i g i n epso i n t h e i rw o o d e ns w o r d sa t a
du
. d d e n lay r e p r e s e n t a t ioofnc o l o n i z a t i oanst h e
s m a l lg r o u po f B r i t i s hi n t h e b a c k g r o u nS
of
(
'
p
r
i
m
i
t
i
v
e
'o) r d e ro f t h i n g st o t h e d y n a m i cu n f o l d i n g
t r a n s i t i o nf r o m a f i x e d ,s t a b l e
t
e
c
h
n
oW
e
s
t
'
s
h i s t o r yi s c h a n g e idn t os o m e t h i nlgi k et h e r e v e n goef t h e ' p r i m i t i v e ' o tnh e
in
an
apocalyptic
set safely
logicalorder.This may be suitablefor a fictionfilm, perhaps,
Australia.
textbookin contemporary
future,but not for a primary-school
patternand the transactional
Thetwo designpatternsin figure2.I,the classificatory
pattern,are only two of severalpossiblepatterns.In the courseof this chapterwe will
introduceothers,andtry to givean overviewof the visualstructuresthat can realizeways
nor on the questionof recogniis not on depiction,
theworld.0uremphasis
of representing
or pixelsas
of pencilmarksor brushstrol<es
tion,on how we cometo seeconfigurations
m
e
p i c t u r e so f t r e e s ,o r o n h o w p i c t u r e so f t r e e sm a y c o n n o t eo r s y m b o l i z e a n i n gasn d
Thisaspectof the pictorialhasalready
valuesoverandabovewhatthey literallyrepresent.
r e c e i v ead g o o dd e a lo f a t t e n t i o ni n t h e w r i t i n g so f p h i l o s o p h e(res . g 'G o o d m a n1,9 6 9 ;
H e r m e r e n1, 9 6 9 ) , s e m i o t i c i a n(se . g . E c o , I 9 7 6 a ; B a r t h e s ,I 9 7 7 ) , m e d i a a n a l y s t s
( W i l l i a m s o n1,9 7 8 ) a n d a r t h i s t o r i a n (se . g .P a n o f s k yI 9, 7 O ) .A t t h i s s t a g eo f o u r w o r k
we havelittleto addto what hasbeensaidin theseareas.
on visualcommunication
o ,n t h eo t h e rh a n dh, a s i, n o u ro p i n i o nb,e e nd e a l tw i t h
T h eq u e s t i oonf v i s u asl t r u c t u r i n g
the
Visualstructuringhas either beentreatedas simplyreproducing
lesssatisfactorily.
prostructuresof reality(e.g.Metz,I974a,7974b), ratherthan as creatingmeaningful
n f o r m a lt e r m so n l y ( e ' 9 .
p o s i t i o nbs y m e a n so f v i s u a ls y n t a xo, r i t h a s b e e nd i s c u s s ei d
A r n h e i m1, 9 7 4 t 1 9 8 2 ,w h o i n h i s a c t u a la n a l y s eosf f e r sm a n yi n s i g h t so n t h e s e m a n t i c
d i m e n s i oonf v i s u a ls t r u c t u r i n g ) . 0 uer x a m p l eo f t h e r e p r e s e n t a t i o nf A b o r i g i n aal n d
satisfiesus.
has,we hope,madeclearwhy neitherof theseapproaches
Britishtechnology
the structuresof'reality'.0n the contrary,they
Visualstructuresdo not simplyreproduce
produceimagesof realitywhichare boundup with the interestsof the socialinstitutions
d n d r e a d .T h e ya r e i d e o l o g i c aVl .i s u a l
w i t h i nw h i c ht h e i m a g e sa r e p r o d u c e dc,i r c u l a t e a
structuresare nevermerelyformal:theyhavea deeplyimportantsemanticdimension.

PARTICIPANTS
as the abilityof semioticsystemsto
In chaptert we definedthe ideationalmetafunction
systemor in
represent
objectsand their relationsin a world outsidethe representational
to the presentchapterwe tried to
the semioticsystemsof a culture.In the introduction
of broadlythe
showhow two designpatternscan producetwo differentrepresentations
'
o
b
j
e
c
t
s
'
o r ' e l e m e n t s ' ww
e i l l ,f r o m n o wo n ,u s et h e
s a m ea s p e cot f t h e w o r l d .I n s t e a do f
'represented
'particip
participants'.
Thishastwo advantages:
anfs'er,moreprecisely,
term
'participantrn something';and it draws
it pointsto the relationalcharacteristicof

48 .

Narrative representations

attentionto the fact that therearetwo typesof participantinvolvedin everysemioticact,

interactiveparticipantsand represented
participants.The former are the participantsin
- the participants
the act of communication
who speakand listenor write and read,make
imagesor viewthem,whereasthe latter are the participantswho constitutethe subject
matter of the communication;
that is, the people,placesand things(includingabstract
'things') represented
in and by the speechor writing or image,the participantsabout
w h o mo r w h i c hw e a r es p e a k i nogr w r i t i n go r p r o d u c i nigm a g e s .
Thesituationis of coursemorecomplexthan this,for the real interactive
participants,
the real lmage-producers
and -viewers,
cannotbe takento be identicalwith the'implied,
p r o d u c ewr h o' s i l e n t l yi n s t r u c tus s ,t h r o u g ht h ed e s i g no f t h ew h o l e (' c h a t m a nr,9 7 B : r 4 B )
and the 'implied'viewer.It may alsobe that the producers
and/orviewersare themselves
explicitlyrepresented
in the image,causingthe two categories
to shadeinto eachother,
complexities
which havebeenstudiedextensively
in the field of literarynarratology(e.g.
Iser,l97B; Bal,1985; Rimmon-l(enan,l-9B3).
we will returnto thesein the nextchapter;
f o r t h e p u r p o s eosf t h i sc h a p t etrh e b a s i cd i s t i n c t i ow
nill suffice.
In the caseof abstractvisualssuchas diagrams,it doesnot seemtoo difficult to
determine
who or what the represented
participantsare. Shannonand Weaver'sfamous
' c o m m u n i c a t i omno d e l (' f i g u r e
2 . 2 ) ,f o r i n s t a n c ei s, m a d eu p o f b o x e sa n da r r o w s( S h a n non and Weaver,
1949). The boxesrepresentparticipants(peopleand/orthings,the dist i n c t i o nb e i n gb l u r r e db y o b j e c t i f y i nlga b e l sl i k e ' i n f o r m a t i osno u r c e ' a n d ' d e s t i n a t i o n ' ) ;
the arrowsrepresent
the processes
that relatethem.If we wantedto translatethis into
languagw
e ,e c o u l ds a yt h a t t h e b o x e sa r e l i k en o u n st,h e a r r o w sl i k ev e r b s( e . g . ' s e n do,r
'transmit'),andthat,together,
theyform clauses(e.g.'aninformationsourcesendsIinformationlto transmitter').
In the caseof moredetailednaturalisticimages,
however,
it maybedifficult,evenfutile,
to try and identifythe represented
participants.
Take'TheBritishusedguns',for instance
( f i g u r e2 . 1 ) .D ow e i n c l u d e
t h e h a t sa n dk e r c h i e fwso r nb y t h et w o m e n ?E v e r ys i n g l et r e e ,
everysingleoneof the rocksstrewnaboutin the foreground?
Theanalogywith language
losesits relevancehere.In language,
wordssuchas man/ gun, tree,rocky groundabstract
awayf rom detailsof this kind.In naturalisticimagesthis doesnot happen.
Theyare'worth
a t h o u s a nw
d ords'.

Fig2.2 ShannonandWeayer'scornmunicationmodel

Nar rati verepresentati ons

49

Yet we think that naturalisticimagescan be analysedinto participantsand processes

ways
muchin the samewayas diagrams.
Therearetwo different,but in the endcompatible,
o f a r g u i n gt h i s .T h ef i r s t i s t h e w a y o f f o r m a la r t t h e o r y( e . 9 .A r n h e i m1, 9 7 4 , 1 9 8 2 ) . f h e
l a n g u a g oe f t h i s k i n d o f t h e o r yi s , f o r t h e m o s t p a r t ,f o r m a l i s t i ca, n d g r o u n d e di n t h e
p s y c h o l o goyf p e r c e p t i o n
P.a r t i c i p a n tasr e c a l l e d ' v o l u m e s ' o r ' m a s s eesa' ,c hw i t h a d i s t i n c t ' w e i g h t ' o r ' g r a v i t a t i o npaull l ' .P r o c e s s e
a sr e c a l l e d ' v e c t o r s ' o r ' t e n s i o n s ' o r
'dynamicforces'.But,andthis is what mattersfor the purposeof identifyingparticipants,
t h e s e ' v o l u m e s ' a rpee r c e i v eads d i s t i n c e
t n t i t i e sw h i c ha r e s a l i e n (t ' h e a v y 't)o d i f f e r e n t
degreesbecause
of their differentsizes,shapes,
colour,and so on. Thusthe two men in
figure 2.1 stand out as a distinctentity becauseof the tonal contrastbetweentheir
snthe
s i l h o u e t t easn dt h e l i g h to f t h e f i r e .A n d w h a t i s m o r e w
, e r e c o g n i zteh e i r s h a p e o
b a s i so f v i s u a sl c h e m anso t u n l i k et h o s et h a t a r e r e a l i z e di n d i a g r a m sA. r t i s t sh a v el o n g
l e a r n e tdh e i rc r a f tb y r e d u c i ntgh e v i s i b l ew o r l dt o s i m p l eg e o m e t r ifco r m s( s e eG o m b r i c h ,
1 9 6 0 ) .A c c o r d i n g
t o A r n h e i m( 1 9 7 4 : c h . 4 ) , c h i l d r e nl e a r nt o d r a w i n t h e s a m ew a y ,
b u i l d i n gu p a r e p e r t o i r eo f b a s i cf o r m s ,a n d t h e n ,g r a d u a l l y' f,u s i n gt h e p a r t s ' .I f t h e
perceptionof picturesdoes indeedoperateon the basisof the sameprinciplesas the
p r o d u c t i oonf p i c t u r e sw, e m i g h tg r a s pt h e o v e r a lsl t r u c t u r eo f a p i c t u r el i l < e ' T h e
British
u s e dg u n s ' a c c o r d i ntgo a s c h e m at h a t i s n o t s o d i f f e r e n ft r o m t h a t o f S h a n n o na n d
Weaver's
communication
model,as shownin figure2.3.
T h es e c o n dw a y o f i d e n t i f y i npga r t i c i p a n tiss t h a t o f f u n c t i o n asl e m i o t i ct h e o r y( s e e
H a l l i d a y1,9 7 8 ,1 9 8 5 ) .T h ec o n c e p t u a lp p a r a t uosf t h i s k i n do f t h e o r yh a s ,s o f a r ,b e e n

'd\
O

Fig 2.3 Schematicreductionof tigure2.1,showingvector

50 .

Narrative reoresentations

appliedonlyto language,
the mostfrequentlyand methodically
analysed
semioticsystem.
It is orientedtowardsthe semanticf unctionsratherthantowardsthe formsof the particiR e c i p i e n t ' r a t htehra nt e r m sl i k e ' v o l u m e ' a n d
p a n t sI.t u s e st e r m sl i k e ' A c t o r ' , ' G o a l ' a n' d
' m a s s 'Y
. e tt h et w o a p p r o a c h easr e c o m p a t i b l e
T.h em o s ts a l i e n t ' v o l u m ei sn'' T h e B r i t i s h
u s e d g u n s ' a r e n o t o n l y p e r c e p t u a l l y m o s t c o n s p i c u o u s , t h e y a l s o p l a y t hr oelm
e so s t c r u c i a
in the grammaticalstructurethat constitutes
the meaningof the picture:the two men(the
participantfrom whichthe vectoremanates)havethe role of Actor, andthe Aborigines
(the participantat which the vector points)havethe role of Goal in a structurethat
represents
their relationas a Transactio4as somethingdoneby an Actor /o a Goal.The
s a m et e r m s( ' A c t o r ' , ' G o a l ' , ' T r a n s a c t i oanr'e) u s e di n f u n c t i o n alli n g u i s t i cT
s .h i si s p o s s i b l eb e c a u steh e y a r e s e m a n t i c - f u n c t i o nr a lt,h e rt h a n f o r m a l ,t e r m s . 0 u ru s eo f t h e s e
termsdoesnot implythat imagesand diagramswork in the sameway as language;
only
that they can 'say' (someof ) the samethingsas language- in verydifferentways:what in
languageis realizedby meansof syntacticconfigurations
of certainclassesof nounsand
certainclassesof verbsis visuallyrealized,madeperceivable
and communicable,
by the
v e c t o r i arl e l a t i o n b
setween
v o l u m e sI .n A r n h e i m ' w
s o r d s , ' W es h a l ld i s t i n g u i sbhe t w e e n
volumesandvectors,betweenbeingandacting'(1982:1,54).
Thetransactional
structureis not the only kind of structurethat can be realizedvisually.We havealreadydiscussed
an exampleof a classificatory
structure(a subjectwhich
w e w i l l t a k eu p i n m o r ed e t a i li n a l a t e rs e c t i o ns; e ep p . 7 9 - 8 7 ) . l n t h e p i c t u r ei n f i g u r e
2.4,takenfrom the samesocialstudiestextbookas figure2.1 (Oakleyet al., 1985),the
s t r u c t u r ei s ' a n a l y t i c a lH
' . e r et h e p a r t i c i p a n thsa v et h e r o l e sn o t o f ' A c t o r ' a n d' G o a l 'b u t
'
C
a
r
r
i
e
r
'
a
n
d
'
A
t
t
r
i
b
u t eT' .h i sp i c t u r ei s n o t a b o u ts o m e t h i nw
of
g h i c hp a r t i c i p a n tasr e
doingto otherparticipants,
but abouttheway participantsfit togethertomakeup a larger
whole.It hasthe structureof a map.Just as in mapsa largerparticipant,the 'Carrier',
represents
the 'whole'(say,Australia),anda numberof otherparticipants,
the 'Possessive
'parts'
(say,
Attributes',represent
the
the statesof Australia),so the Antarcticexplorer
functionsas'Carrier',andthe balaclava,
thewindprooftop,thefur mittens,etc.functionas
'Possessive
Attributes',asthe partsthat makeup the whole.Theclosestlinguistictranslation here- werewe to attemptone- wouldnot be an actionclauselike'The Britishpoint
their gunsat the Aborigines',
but a 'possessive
attributive'clauselike tTheoutfit of the
Antarcticexplorerconsistsof a balaclava,
a windprooftop,fur mittens. . . [etc.]'
We can now look at the abundance
of detail in naturalisticimagesin a newway.The
naturalisticimage,whateverelseit maybe about,is alwaysalsoaboutdetail.It Contains
a
'analytical'processes.
multitudeof embedded
It may,at the most salientlevel,say,'The
B r i t i s hp o i n t h e i rg u n sa t i h e A b o r i g i n e sb'u, t i t w i l l a l s o a, t l e s si m m e d i a t ecl yo n s p i c u o u s
l e v e l ss,a yt h i n g sl i k e' T h em e n ' so u t f i t sc o n s i sot f h a t s ,k e r c h i e f.s. . [ e t c . ] ' a n d ' T h et r e e s
ah
l r a s e (st w o m e nw i t h h a t sa n d k e r h a v ec l u m p so f l e a v e s . ' I nl a n g u a g ep,r e p o s i t i o n p
(two
chiefs)and subordinate
clauses
men,wearinghats and kerchiefs)fulfil the same
functionof addingdetailat a'secondary'or evenmoredeeplyembedded
level.
E m b e d d i ncga na l s oo c c u ri n d i a g r a m sT.a k et h e ' c o m m u n i c a t i m
o no d e li' n f i g u r e 2 . 5 , a
modeldrawnup,not by two telecommunication
engineers,
as in the caseof Shannonand
W e a v e r( f i g u r e2 . 2 ) , b u t b y t w o s o c i o l o g i s tR
s ,i l e ya n d R i l e y( 1 9 5 9 ) .A s a w h o l e t, h e

Narrati verepresentations

llr-t

"r.

r11'

5l

rdqqv *i3-:

i::t !rf1*{},:x :iq l';!* *a $$:!jh*

t1]!i1*:a1

;
;'.'.:. .i:";:

'r

. trjs':h; F"!,..,,;i

.1:;;'r.: :-;-.,i. !

i
n

:l;
i

i1

.,*.J

Fig 2.4 Antarcticexplorer(oakleyet at,,LgPjs)

Fig 2.5 C0mmunication

model(from Watsonand Hill,1980:143)

d i a g r a mi s ' a n a l y t i c a l 'i;t i s a k i n d o f a b s t r a c m
t a p .I t s h o w st h a t t h e ' o v e r - a l sl o c i a l
s y s t e m ' c o n s i sot sf ' l a r g e rs o c i a sl t r u c t u r e s ' w h i ci n
h t u r n c o n s i sot f ' p r i m a r yg r o u p s 'I.t
a l s o f e a t u r e st w o i n d i v i d u a l s , ' C('' C o m m u n i c a t o ra' )n d ' R ' ( ' R e c i p i e n t ' )T.h e s ea r e
to,
depictedas half in, half out of the 'larger socialstructures',
and they are connected
. m b e d d ewdi t h i nt h i s a n a l y t i c asl t r u c t u r ei s a
t h o u g hn o t p a r t o f , t h e ' p r i m a r yg r o u p s 'E

Narrativerepresentati ons

Fig 2.5 Twocommunication

models(trom Watsonand Hill,1980:147)

transactiona
s tl r u c t u r et :h e ' l a r g e rs o c i a sl t r u c t u r e sa/n d t h e i n d i v i d u a 'l C
s ' a n d' R ' a r e
represented
as involvedin an activeprocess
of communication,
realizedby vectors.
W h e nw e l o o ka t ' T h eB r i t i s hu s e dg u n s (/ i n f i g u r e2 . 1 ) a sa t r a n s a c t i o nsatlr u c t u r e , t h e
two menform oneparticipant:togethertheyhavethe roleof 'Actor'.Whenwe lookat the
two menasan 'analytical'structure,
theyformtwo distinctparticipants,
linkedby the lines
formedby the handof the manon the right andthe gunof the man on the left. Diagrams
a l l o wf u r t h e rp o s s i b i l i t i eass,c a n b e s e e ni n t h e t w o c o m m u n i c a t i omno d e l si n f i g u r e2 . 6 ,
b o t hd r a w nb y S c h r a m m( 1 9 5 4 ) ,a s o c i apl s y c h o l o g iwsrt i t i n ga b o u tm a s sc o m m u n i c a t i o n .
In the first model,'source'and 'encoder'are separate
entities,conjoinedby a line,just as
. e w i l l a r g u el a t e rt h a t l i n e sw i t h o u ta r r o w
a r e t h e t w o m e ni n ' T h e B r i t i s hu s e dg u n s ' W
h e a d sr e a l i z ea p a r t i c u l akr i n do f ' a n a l v t i c asl 't r u c t u r e :

Fig 2.7 Conjoiningparticipants

I n t h e s e c o n dm o d e l , ' s o u r c e ' a n' edn c o d e ra' r e compounded,

weldedtogetheryet still
d i s t i n cct o m p o n e not sf t h ew h o l e :

Fig 2.8 Compounding

of participants

ions
Narrative representat

53

eh
r 'e.
T h et h i r d p o s s i b i l i twy o u l db e a c o m p l e t e
f u s i o nb e t w e e \ns o L r r c e ' a n d ' e n c o d T
s h a p eo f R i l e ya n d R i l e y ' s ' o v e r aslol c i asl y s t e m(/f i g u r e2 . 5 ) c a nb e i n t e r p r e t eads s u c ha
fusion- a fusionof two circlesand a box.Apparentlyparticipants
can losetheir separate
identityto differentdegrees.
Whenthey are conjoined,
the act of connecting
the process,
t h e m ,i s s t i l le x p l i c i tr,e a l i z e b
d y a l i n eW
. h e nt h e ya r ec o m p o u n d et dh ,e i ri d e n t i t i erse m a i n
distinct,but thereis no longeran explicitlyexpressed
process
to connectthem.Whenthey
are fused,eventheir separateidentitieshavedisappeared.
In speechand writing,with
somewhatdifferentmeans- for instance,
stressand intonation- we can movefrom, say,
('is'),'
Thebird is black,whichhastwo distinctparticipants
process
aswell as a connecting
to the blackbird,whichhasblack andbird still as differentwords,but removesthe process,'
to the blackbird,in whichtwo wordshavebeenfusedto becomeonesemanticentity/noun.
Eachsuccessive
stepfurtherobscures
the act of predication,the
explicitact of bringingthe
s g e t h eur ,n t i lt h es t r u c t u r e
t w op a r t i c i p a nt o
i s n o l o n g e r ' a n a l y t i c anlo' ,l o n g ear n a l y s eodr
a n a l y s a b lW
e .e m a k et h e p o i n ta t s o m el e n g t hb e c a u soef t h e ( i d e o l o g i c asl i)g n i f i c a n coef
t h i ss e m i o t i rce s o u r cien c o n f i g u r i nt g
herepresente
wdo r l d .

Fig 2.9 Fusionof participants

A s w i t h m a n yo t h e rk i n d so f d i a g r a mt,h e c o m m u n i c a t i omno d e l sw e h a v eu s e dt o
illustratethis sectionare explainedor paraphrased
in the writtentextsthat accompany
them.But by no meanseverything
in the
that is expressed
in the diagramsis alsoexpressed
written texts.Not all of the meanings
verbally.The
conveyed
visuallyare also conveyed
m e a n i n gosf t h ev i s u asl h a p e st h, eb o x e a
s n dc i r c l e sa n dt r i a n g l etsh a t g i v et h ep a r t i c i p a n t s
t h e i r v o l u m ef,o r e x a m p l ea, r e a l m o s ta l w a y sl e f t u n e x p l a i n e d . 0 l dceor ,m m o ns e n s eo r
t h e o r e t i c anl o t i o n ss, u c ha s ' i l l u s t r a t i o n('i m a g e s ' i l l u s t r a t i n g ' v e r bt eaxl t s )o r ' e x p l a n ation' (words'explaining'diagrams)are no longeran adequateaccountof the relations
between
wordsand pictures,hereas in otherinstances.
Why,in figure2.6, is the 'signal'a
'source'
'encoder'
circle,the
a rectangle,
the
a triangle?Why,in figure2.5, is the 'primary
g r o u p ' ar e c t a n g lw
e ,h i l et h e ' l a r g e sr o c i asl t r u c t u r e ' a nt dh e i n d i v i d u a l s ' C ' a n d ' R ' a r e
circlesW
? h yd o S h a n n o an n dW e a v e (r f i g u r e2 . 2 ) p r e f e ra n g u l a r i t w
y ,h i l eR i l e ya n d R i l e y
prefercurvature (figure2.5)?
Therecanbe littledoubtthat suchchoicesarechargedwith meaning.
Basicgeometrical
shapes
havealwaysbeena sourceof fascination,
awe.0ur scientificageis
evenof religious
no exception.
Circles,squaresand triangleshavebeenregardedas pure/quasi-scientific
' a t o m s ' o ft h e v i s i b l ew o r l d ,a ' p u r e m a n i f e s t a t i o n
o f t h e e l e m e n t st 'h, e ' u n i v e r s a l - a s - t h e mathematical',
as Mondriansaid (quotedin Jaff6, 1967: 54-5). And they have been
thoughtto havethe powerto directlyaffect our nervoussystem,for instanceby the
c o n s t r u c t i v ia
s tr t i s t G a b o : ' T h ee m o t i o n aflo r c eo f a n a b s o l u t e
s h a p ei s u n i o u ea n d n o t

ons
Narrati verepresentati

theyelateand make
exultandshapesdepress,
replaceable
by any othermeans.. . . Shapes
desperate'(quotedin Nash,I974i 54). As we are here primarilyconcernedwith the
this subjectfalls somewhatoutsideour main concern;it
relationsbetweenparticipants,
(mythical)significance
of
study.But giventhe semioticand ideological
deserves
a separate
theseaspects,we will at least indicatethe issueswith which such a study might be
concerned.
arethe elements
of the mechWesternsociety,
squares
In contemporary
and rectangles
Theydominate
the shapeof
anical,technological
order,of theworldof humanconstruction.
our roads.Theydominatethe shapeof manyof the objectswe use
our cities,our buildings,
in daily life, includingour pictures,which nowadaysrarelyhavea roundor oval frame,
thoughotherperiodswerehappyto usetheseto framemoreintimateportraitsin particurectangularshapes
lar. Unlikecircles,which are self-contained,
completein themselves,
patterns:
theyform the modules,
the
canbestacked,
alignedwith eachotherin geometrical
buildingblockswith whichwe constructour world,and they are thereforethe dominant
c h o i c eo f b u i l d e ras n de n g i n e e rasn, do f t h o s ew h ot h i n kl i k eb u i l d e ras n de n g i n e e r s . Ianr t ,
abstractionists,
artistsfor whomart hasto be,aboveall,
theyarethe choiceof geometrical
w r o t ei n t h e 1 9 2 0 s ,
r a t i o n a lA. s M o n d r i a n
I n a l l f i e l d sl i f eg r o w si n c r e a s i n gal yb s t r a c t w h i liet r e m a i n rse a l .M o r ea n dm o r et h e
tensingof form
naturalpower.In fashionwe seea characteristic
machinedisplaces
of colour,signifying
the departuref rom the natural.
and intensification
In moderndancesteps(boston,tango,etc.)the sametensingis seen:the curved
lineof the old dance(waltzetc.)hasyieldedto the straightline,andeachmovement
- signifyingthe searchfor equiis immediately
neutralized
by a counter-movement
with its (natural)rule
librium.)ur social/ifeshowsthis too: autocracy,
imperialism
o f p o w e ri ,s a b o u t o f a l l i f i t h a sn o t f a l l e na l r e a d y a n dy i e l d st o t h e ( s p i r i t u a l )
oowerof law.
the newspiritcomesstronglyforwardin logic,scienceandreligion.The
Likewise
i m p a r t i n go f v e i l e dw i s d o my i e l d st o t h e w i s d o mo f p u r e r e a s o na; n d k n o w l e d g e
The old religion,with its mysteriesand dogmas,is
showsincreasingexactness.
i n c r e a s i n gt lhyr u s ta s i d eb y a c l e a rr e l a t i o n s hti op t h e u n i v e r s a l .
( q u o t e di n J a f f 6 ,1 9 6 7 : 6 4 )
s n d t o e x p r e stsh e
G l o s s e isn ' d i c t i o n a r i eos f v i s u a ls y m b o l s ' a n ds i m i l a rp u b l i c a t i o nt e
shapesin termsof intrinsic,abstractqualities,
but theypointin the
meaningof geometrical
straightsamedirection.Accordingto Dondis(1973:,44),thesquarerepresents'honesty,
n e s sa n dw o r k m a n l i km
e e a n i n ga' ;c c o r d i ntgo T h o m p s oann d D a v e n p o r(t1 9 8 2 : 1 1 0 ) ,i t
'represents
the worldanddenotesorder'.
C i r c l e sa r e g l o s s e dv e r y d i f f e r e n t l yi n s u c h d i c t i o n a r i e sa,s d e n o t i n g ' e n d l e s s n e s s ,
w a r m t hp
, r o t e c t i o n( 'D o n d i sI,9 7 3 : 4 4 ) , o r a s ' t h et r a d i t i o n asl y m b ool f e t e r n i t ya n dt h e
h e a v e n s( T
' h o m p s oann d D a v e n p o r t , I 9 S 2 : 1 1 0B) .u t s u c hd e s c r i p t i o nc sa nb e m u l t i p l i e d
or,to put it in our
endlessly:
the moreabstractthe sign,the greaterits semanticextension;
terms,the greaterits potentialrangeof usesas a signifierin signs.We needto lookfor the

Narrativerepresentations. 55

betweensquare
principles
oppositions
and for the fundamental
that unitethesemeanings,
doesnot exist.
and circle,betweenthe angularand the curved.In nature,squareness
the curve'madeit difficultto represent
Mondrianadmittedthat hismethodof 'abstracting
that
the curve;you can understand
nature:'1npaintinga tree,I progressively
abstracted
i ndJ a f f 6 ,1 9 6 7 : 1 2 0 ) .C i r c l e sa n dc u r v e df o r m sg e n e r v e r yl i t t l e" t r e e " r e m a i n e d ' ( q u o t e
with an organicand naturalorder,with the world of
ally are the elementswe associate
with themderivefrom
as may be associated
organicnature- and suchmysticalmeanings
e o r l d ,o r w i t h t h e w o r l d o f
e i t h t h e i n o r g a n i cc,r y s t a l l i nw
t h i s .A n g u l a r i t yw e a s s o c i a tw
andthereforea worldwe can/at least
technology,
whichis a worldwe havemadeourselves,
in principle,understand
fully and rationally.The world of organicnaturels not of our
making,and will alwaysretain an elementof mystery.Curvedforms are thereforethe
d o m i n a nct h o i c eo f p e o p l ew h o t h i n k i n t e r m so f o r g a n i cg r o w t hr a t h e rt h a n m e c h a n i c a l
construction,
in termsof what is naturalratherthan in termsof what is artificial.In art, it
i st h ec h o i c eo f w h a ti s s o m e t i m ecsa l l e d' b i o m o r p h iacb s t r a c t i o n a l i s mt h' -ec u r v e sb, l o b s
a n db u l g e si n t h e p a i n t i n gosf H a n sA r p o r t h e s c u l p t u r eosf H e n r yM o o r e .
The valuesattachedto thesepolesof meaning- that is, the actual signsproduced
w i t h t h e s i g n i f i e r os f t h e ' t e c h n o l o g i c a l ' a ntdh e ' n a t u r a l ' - d o , o f c o u r s ed, i f f e r .T h e
or
positively,
as a sourceof powerand progress/
squarecan connotethe 'technological'
n e g a t i v e lay s, a s o u r c eo f o p p r e s s i ownh i c h ,l i t e r a l l ya n d f i g u r a t i v e l y , ' b o xuess i n ' . I n
R i l e ya n d R i l e y ' sc o m m u n i c a t i omno d e l( f i g u r e2 . 5 ) s o c i e t yi s r e p r e s e n t eads a n a t u r a l
d .u t t h e ' p r i m a r yg r o u p s ' a r e
o r d e ro, r g a n i c a l leyv o l v e d
r a t h e rt h a nh u m a n l yc o n s t r u c t e B
bias in favour of modern
depictedas rectangles.
Perhapsthis betraysan unconscious
u r b a ns o c i e t ya, v i e wi n w h i C ht h e s m a l l ,c l o s e - k n ci to m m u n i t iyn w h i c he v e r y o nken o w s
and the 'larger social structure'as
everythingabout everyoneis seenas oppressive,
l i b e r a t i n gp,r o v i d i ntgh e i n d i v i d u awl i t h a n o n y m i t ya,n dt h e r e b yw i t h a u t o n o m ya n d s e l f . o r R i l e ya n d R i l e y i, n d i v i d u a lhsa i l f r o m ' p r i m a r y
c o n t a i n m e nct ,h o i c ea n d f r e e d o m F
g r o u p s (' a n da r e s t i l l c o n n e c t etdo t h e m ,a l b e i tt e n u o u s l yb) ,u t t h e n g o t h e i r o w n w a y ,
l e a v i n gt h e ' p r i m a r yg r o u p s ' b e h i n da ,n d m o v i n gf r e e l yi n a n d o u t o f t h e ' l a r g e rs o c i a l
structures',in a sociallymobileworld.This exampleshowsthat diagrams,rationaland
also
visuallythat are not necessarily
scientificas they may seem/can conveymeanings
conveyed
verbally.
l ,c h n o l o g i c a l
T h et r i a n g l ei s a n g u l a rl ,i k et h e s q u a r e- a n e l e m e not f t h e m e c h a n i c at e
of a)
o r d e rB
. u t ,u n l i k et h e s q u a r et h/ et r i a n g l ee, s p e c i a lw
l yh e nt i l t e d ,i s a ( f u s e ds t r u c t u r e
pointat things.Themeanings
participantanda vector,because
it canconveydirectionality,
i t a t t r a c t sa r e t h e r e f o r el e s s l i l < e ' q u a l i t i eos f b e i n g ' t h a n l i l < ep r o c e s s eas s/ i n t h e
well-known
r e v o l u t i o n a rpyo s t e rb y E l L i s s i t z k y( f i g u r e2 . 1 0 ) ,i n w h i c ht h e r e v o l u t i o n ,
represented
by a red triangle,is an active,dynamicforce,wedgingitself into the inert,
s e l f - c o n t a i n e' odr,g a n i cs' o c i e t o
y f W h i t eR u s s i a .
s .h et r i a n g l e isn f i g u r e
of procesT
I n d i a g r a m st r,i a n g l ecsa ns i m i l a r l yi n t r o d u cae s e n s e
2 . 6 , f o ri n s t a n c ce o, u l db e s e e na s ' f o c u s i n g ' o r ' a i m i n g ' t h e ' m e s s(awgeeh' a v ea l r e a d y
l r o c e s s easn dw e w i l l
i n d i c a t ew
d h y i t i s s o d i f f i c u l t o g i v ev e r b a tl r a n s c o d i n gosf v i s u a p
, s s eosf t h em e a n i n gosf t r i a n g l e isn v i s u a l
. o ts u r p r i s i n g gl yl o
d i s c u stsh i sm o r ef u l l yl a t e r ) N
dictionariesreflectthis dynamicquality.Trianglesare 'a symbolof generativepower'

Narrative representations

&i,'..i.,

Fig2.LO BeattheWhiteswiththeRed ryedge(ElLissitzky,lglg-20)(fromilash,1974)

( T h o m p s oann d D a v e n p o r1t ,9 8 2 : 1 1 0 ) ,a n d r e p r e s e n' at c t i o nc, o n f f i c t ,e n s i o n(' D o n d i s ,

I973:44).
The meanings
of the basicgeometrical
shapes,
then,are motivatedin two ways.First,
they derivefrom the propertiesof the shapesort rather,from the valuesgivento these
propertiesin specificsocialand culturalcontexts.The straightline,for instance,
means
what it literallyis:'straight'.This'straightness'may
thenbe usedto carry anyoneof a vast
rangeof meanings
compatible
with that.It maybe positively
valuedin onecontext(e.g.the
'straightand narrowpath',or Mondrian'sassociation
of straightness
with the'spiritual
p o w e ro f l a w ' ) ,l e s sp o s i t i v e liyn a n o t h e (r e . 9 . ' t h es t r a i g h m
t a n ' a so p p o s etdo t h e ' f u n n y
man',or 'straight'as opposed
to 'gay').Theproducers
of an imagehavetheir interestsin
makingthe visualsign,and this makesthe meaningof the imagequite specificfor the
producer;it coloursin and makesspecificthe abstractmeaningsthat derivefrom the
inherentproperties
of the shapesand from the historiesof their culturaluses.Rectangles
can be stacked- and,again,this may be positivelyvaluedin one context,say in urban
planning,or in geometricabstractionism;
and lesspositivelyin another,say in counterc u l t u r e tsh a t d r e a mo f l i v i n gi n g e o d e s idco m e so, r i n b i o m o r p h iacb s t r a c t i o n i s m .
Second,these meaningsderivefrom the commonqualitieswe may detect in such
o b j e c t si n o u r e n v i r o n m e n
a ts w o u l db e c i r c u l a ro r r e c t a n g u l awr h e na b s t r a c t etdo t h e i r
underlying
basicshape,and from the valuesattachedto thesequalitiesin differentsocial

Narrativerepresentations. 57

contexts.
Thesun,the moon,the bellyof the pregnantwoman,are curved.Theskyscraper,
desk,the expensive
briefcase,
Suchcommonqualitiesas we
the executive
are rectangular.
(
s
a
y
,
'
n
a
t
u
r
e
'
s
) i l l e v i d e n t lbye
g
r
o
u
p
s
m a ys e ei n t h e s e
of objects
c y c l e s ' a n d ' m apl eo w e r 'w
groupings
of objectsfrom
readandvalueddifferentlyin differentsocialcontexts andthe
so as to obtainthe
which we derivetheserneanings
are likelyto be made selectively,
q
u
a
l
i
t
i
e
s
common
sought.
F i n a l l yo, u r a r g u m e nst u g g e s ttsh a t t h e s e m i o t i ca e n e s ios f d i a g r a m sl i e s ,n o t j u s t i n
of our
in the abstractart movements
technicaldrawing,but also in art, and specifically
n
a
turalr
e
d
u
c
t
i
o
n
w
s
h
i
c
h
c e n t u r yw, h i c hn o l o n g efri l l o u t a n dc o r p o r e a l i zt hees c h e m a t i c
r
i
g
h
t
,
u
n
d
e
r
l i ea l l
i s t i c a r t i s t sh a v eu s e df o r c e n t u r i e sa,n d w h i c h ,i f t h e G e s t a l t i s tasr e
visualreoresentation.
From the basicshapesother geometricalshapescan be derived:square/circle and
l yr v e r t i c a l ley l o n g a t et do d i f f e r e ndt e g r e e sa ;n dt h es q u a r e , t h e
t r i a n g l ec a nb eh o r i z o n t a l o
shapes
triangleandall elongated
canbetilted,eithertowardsthe right or towardsthe left.
Verticalelongationcreatesa more pronounced
distinctionbetweentop and bottom,and
(what is most importhencea biastowardshierarchy,
andtowards'opposition'generally
goes
to
on top,what is lessimportantor dominantis relegated
ant or otherwisedominant
a shapeto leantowardsthe kindof structurein
the bottom).Horizontalelongation
causes
p
o
s
i
t
i
o
n
e
d
p
r
e
s e n t eads ' G i v e n 'a, s i n f o r m a t i otnh a t i s a l r e a d y
o nt h e l e f ti s
w h i c hw h a t i s
e /h i l ew h a t i s
f a m i l i a rt o t h e r e a d e ra n d s e r v e sa s a ' d e p a r t u r ep o i n t ' f o r t h e m e s s a g w
yet
positioned
presented
knownto the reader,
on the right is
as'New',as informationnot
his or her specialattention.Theshapeof Schramm's'fieldof experiand hencedeserving
'signal'
suggests
that these
in the seconddiagramof figure 2.6, for instance,
ence'and
participantsare,at leastpotentially,
endowed
with suchan informationstructure.Tilting,
In Malevich'sSupremacist
finally,createsobliquelinesand hencea senseof vectoriality.
are represented
RedSquareand BlackSquare(fi9ure2.11)the participants
Composition:
y o r es i m i l a tro
a s s q u a r e sB.u t b e c a u steh e r e ds q u a r ei s t i l t e d t, h e p a i n t i n gi s s t r u c t u r a l lm
to
El Lissitzky'sBeat the Whiteswith the Red Wedge(figure2.10) than,for instance,
M o n d r i a n ' cs o m p o s i t i o nosf r e d ,y e l l o wa n d b l u es q u a r e si :t i s a b o u td y n a m i c ' a c t i o n ,
w h e r e aM
s o n d r i a n 'cso m p o s i t i o n
a sr ea b o u ta s t a b l eo r d e ra, ' s e a r c hf o r e q u i l i b r i u m 't-h e
largeblacksquare.t
redsquareseemsto moveawayfrom the oppressively
of visualand verbal
Finally,it is importantto stressthe essentialinterchangeability
p a r t i c i p a n ti sn d i a g r a m sa,n d ,i n d e e di n
, m a n yo t h e rv i s u a gl e n r e sA. l t h o u g ht h e p r o c e s s e s
and structuresin diagramsare alwaysvisual,the participantswhichthey relateto each
with
abstractshapes,
othermay be of differentkinds:pictures,naturalisticor schematic;
o r w i t h o u tv e r b a l a b e l sw; o r d se, i t h e re n c l o s eodr n o t e n c l o s eidn b o x e so r o t h e rs h a p e s ;
letters;and so on.Thesamethingcan be seenin pagelayout:the participantsare heterogeneous.
Theycanbeverbal(headlines,
blocksof copy,etc.),but the semioticmeanswhich
bring them togetherinto a coherentsemanticstructureare alwaysvisual.The key to
of the visualsemiotic
understanding
suchtextsthereforeliesaboveall in an understanding
elementsinto a coherentwhole,into a
meanswhichare usedto weldtheseheterogeneous
however,
text.Visualstructuresrelatevisualelements
to eachother;thesevisualelements,
- a word as a visualelement,
a blockof writtentext as
maythemselves
be heterogeneous

Narrati verepresentati ons

Fig 2.ll

SuprematistComposition:Red Sqaareand BlackSquare(Kasimir Matevich,I9l4) (from Nash,1974)

Narrative representations . 59

a visualelement,an imageas a visualelement,a numberor an equationas a visual

element.

N A R R A T I V EP R O C E S S E S
to
as doingsomething
by a vector,theyare represented
Whenparticipantsare connected
and
patterns
in
l(ress
narrativewe
will
such
vectorial
call
or for eachother.Fromhereon
van Leeuwen(1990) we usedthe term 'presentational'-andcontrastthemto conceptual
patternsrepresentparticipantsin termsof
patterns(seefi9ure2.12).Whereconceptual
and moreor
in otherwords,in termsof their generalized
their class,structureor meaning/
present
actionsand
patterns
unfolding
serve
to
narrative
lessstableandtimelessessence/
processes
transitoryspatialarrangements.
events,
of change,
of a vector:narrative
The hallmarkof a narrativevisual'proposition'is the presence
pictures,
thesevectorsare
never
do.
In
structures
structuresalwayshaveone,conceptual
quite
diagonalline,
strong,
often
a
that form an obliqueline,
formedby depictedelements
'
T
h
e
(
i
n
g
u
n
s
g
u
n
s
'
o
u
t
s
t
r
e
t c h eadr m so f
a
n
d
t
h
e
f i g u r e2 . 1 ) ,w h e r et h e
B r i t i s hu s e d
asin
l
i
m
b
so r t o o l s ' i n
o
r
b
y
b
o
d
i
e
s
t h e B r i t i s hf o r m s u c ha l i n e .T h ev e c t o r sm a y b e f o r m e d
linesof
diagonal
into
elements
action',but thereare manyotherwaysto turn represented
a
vector,
is
also
picture
for
instance,
space,
action.A road runningdiagonallyacrossthe
s
u c ha s
i
m
a
g
e
p
r
o
c
e
s
s
I
n
a
b
s
t
r
a
c
t
of'driving'.
a n dt h e c a r d r i v i n go n i t a n ' A c t o r ' i n t h e
graphic
for
instance,
elements
are realizedby abstract
diagrams,narrativeprocesses
l i n e sw i t h a n e x p l i c i ti n d i c a t oor f d i r e c t i o n a l i tuys, u a l l ya n a r r o w h e a dS.u c hf e a t u r e so f
mustalwaysbe presentif the structureis to realizea narrativerepresentadirectionality
r f d i r e c t i o n a l i tfyo r m a p a r t i c u l a rk i n d o f
t i o n : c o n n e c t i n lgi n e sw i t h o u ta n i n d i c a t o o
o ' , ' i sc o n j o i n etdo ' , ' i s r e l a t e d
a n a l y t i c aslt r u c t u r ea,n dm e a ns o m e t h i nlgi k e' i s c o n n e c t et d
t0'.
T h e ' A c t o r ' i st h e p a r t i c i p a nf tr o mw h o mo r w h i c ht h e v e c t o rd e p a r t sa, n dw h i c hm a y
be fusedwith the vector to different degrees.In Beat the Whites with the Red Wedge
(figure2.10),for instance,
the redtriangleis both participantand vector,and represents
i n ( o r ' b e a t i n ga' ,st h et i t l e h a si t ) . I n f i g u r e2 . I 3 ,
b o t ht h e ' w e d g ea' n dt h e a c t o f w e d g i n g
, A c t o r sa r er e a l i z e d
t h) e
t a k e nf r o ma f a c t u a cl h i l d r e n 'bso o ka b o u tF r a n c e( B e n d e r , 1 9 B B
'
M
'
G
u
l
f
(
t
h
e
S t r e a mv' e c t o ri s r e d ,t h e i s t r a l 'v e c t o rb l u e )a, n d
b y t h ec o l o u ro f t h e a r r o w s
(they
just
are realizedby
thin lines),and the narrativeprocesses
by their volume
are not

r
Reoresentational
structures

I
I

I
I

-----*t
L conceptuar

Fig 2.12 Main types of visual representationalstructure

classificatorv

ilfl;:f::

60

Narrativerepresentatrons

E n g l i s hC h a n n e l

Biscay

ffi
{

Mediterranean

,'-.,',]::'
i:,,i",:,:i,rll!'t'
Gulf Stream
ftrtistrat
LEso+,f*,

il

Fig 2.13 culf Streamand Mistrat(Bender,t988)

t h e i rv e c t o r i a l i tIyn. S h a n n o an n dw e a v e r ' si 9 4 9 ) c o m m u n i c a t i omno d e (l f i g u r e2 . D , o n

the otherhand,Actor and process
are realizedby separate
visualelements,the
Actor by a
b o x ( ' i n f o r m a t i osno u r c e ' )t h
, e p r o c e sbs y a n a r r o w .P i c t u r e sl i k e ' T h eB r i t i s hu s e dq u n s '
( i n f i g u r e2 . 1 ) o c c u p yp,e r h a p sa,n i n t e r m e d i apt eo s i t i o n .
In the caseof 'realist'images,
the contextusuallymakesclearwhat kind of actionthe
v e c t o r sr e p r e s e n t . ' T hBer i t i s hu s e dg u n s /( i n f i g u r e2 . 1 ) c a n b e t r a n s l a t e-d s h o u l dw e
wishor needto do so- notso muchwith 'used',asthe captionhasit, aswith something
like
' T h e B r i t i s hs t a l kt h e A b o r i g i n e s
w i t h t h e i rg u n s ' . O t h epro s s i b i l i t i e sx i s t b, u t i h e f i e l di s
limited.Thevectorsin abstractpicturesare moredifficultto transcode.One
first needsto
formulatewhat they,literally,formally,do. Thiscan then helpto circumscribe
the field of
possiblereadings.The trianglein El Lissitzky'sBeat the whites with the Redwedge
(figure2.10) literallywedges
itselfintothewhitecircle.Thisopensup a perhapslarge,but
b y n o m e a n si n f i n i t er,a n g eo f p o s s i b l e
r e a d i n g st h: e t r i a n g l ec a n b e s a i dt o , p i e r c e ,o r
'infiltrate'or'destabilize'the
c i r c l eI.n f a c t i t d o e sa l l t h e s et h i n g s- t h e p r o c e srse p r e s e n t s
a field of possiblemeanings.
In the caseof Malevich,sSuprematistComposition:Red
Squareand Black Square(frgure2.7I), the red squareliterally tilts away from the black
squareaboveit. Howcanwe transcode
this?Doesthe redsquare'ffeefrom,or 'pointaway

Narrativerepresentations. 6I

f r o m ' t h e b l a c ks q u a r e ?I s i t ' e j e c t e df r o m ' o r ' d i s c a r d e b

d y ' i t ? A l l t h e s er e a d i n gas r e
l e g i t i m a t eT.h e p o i n ti s t h a t e a c hr e a d i n gw i l l h a v et h e s m a l lr e d s q u a r ea s t h e m o b i l e
participant,howevermuch it is dominatedby the big, heavyblacksquareiand the black
s q u a r ea s s t a t i c ,m o t i o n l e sasn d m o n o l i t h i cT.h e p i c t u r et e l l st h e s t o r yo f a n ' u n d e r l i n g '
e s c a p i nfgr o m ,o r b e i n ge x p e l l efdr o m ,t h e m o n o l i t h ipc o w e ro f t h e b l a c ks q u a r eW
. hoor
w h a tt h i s u n d e r l i nogr t h e s ef o r c e sa r et h e r e a d e rw
s i l l p r o d u c ei n t h e p r o c e sosf r e a d i n g ,
, a l < ec e r t a i n
a l t h o u g ht h e s h a p e sa n d c o l o u r sw i l l p o i n tt h e m i n a c e r t a i nd i r e c t i o nm
r e a d i n gm
s o r ep l a u s i b l e
m, o r el i k et h o s eo f t h e p r o d u c e rt' sh a no t h e r s .
I n t h e c a s eo f d i a g r a m ist i s a l s od i f f i c u l t o s a yi n w o r d sj u s tw h a t k i n do f a c t i o nt h e
v e c t o r sr e p r e s e nTt .h ec o m m o ns e n s e
v i e wo f t h e f u n c t i o no f i m a g e sa s ' i l l u s t r a t i o n ' a n d
' e x p l a n a t i o n ' w o ubl d
et h a t t h e a c c o m p a n y i n
ve
s h a t i s n o t m a d ec l e a r
g r b atl e x te x p l a i nw
v i s u a l l yB. u t u s u a l l tyh i s i s n o t s o .U s u a l l tyh e p r o c e siss r e p r e s e n t eodn l yv i s u a l l ya,n dt h e
writtentext eitherdoesnot paraphrase
contradictory
or evenmisleadit at all, or provides
i n g g l o s s e sT.h e S h a n n o n
a n d W e a v e cr o m m u n i c a t i omno d e lh a sb e e n ' q u o t e d ' a nvde r b a l l ye x p l a i n eidn m a n yb o o k sa n da r t i c l e s , f oirn s t a n cien W a t s o na n d H i l l ' sD i c t i o n a royf
and MediaStudies(1980),whichhasa 250-wordentryaboutthe model,
Communication
(I972).The
and in a ScientificAmericanarticleby Pierce,a telecommunications
engineer
first givesthe followingindications
of the meaningof the vectors:the authorssaythat the
model'can be appliedto any informationtransfersystem'and they call it a'process
c e n t r e dm o d e l (' 1 9 8 0 ; I 4 9 ,o u r i t a l i c s -) t w o r a t h e ro b l i q u er e f e r e n c eosn, ea b r o a dg l o s s
( ' p r o c e s s 't)h, e o t h e ro n l y s l i g h t l ym o r e s p e c i f i c( ' t r a n s f e r ' )P. i e r c e ' se x p l a n a t i o nasr e
c o n t r a d i c t o r y . 0t hneo n eh a n d h, ec a l l st h e m o d e al ' s y s t e m(' a sd o W a t s o na n d H i l l i n t h e
dictionary)and paraphrases
it in termsof an analyticalratherthan a narrativestructure:
'The systemconsists
of an informationsource,a transmitter,
a communication
channel,a
n o i s es o u r c ea, r e c e i v ear n da m e s s a gdee s t i n a t i o n0 '9 7 2 ' . 3 2 , o u r i t a l i c s -) n o t et h a t h e
listsonlythe participants,
not the processes.
0n the otherhand,whenhe givesan example,
he paraphrases
the vectorsby meansof activeverbs:
A humanbeingmay type a message
consistingof the lettersand spaceson the
keyboard
of a teletypewriter.
servesas a transmitterthat encodes
Theteletypewriter
e a c hc h a r a c t earsa s e q u e n coef e l e c t r i c aplu l s e sw, h i c hm a yb e ' o n 'o r ' o f f ' , ' c u r r e n t '
or'no current'.Theseelectricalpulsesare transmittedbya pair of wiresto another
teletypewriter
that actsas a receiverandprintsout the lettersandspaces.
( P i e r c eI ,9 7 2 13 3 , o u r i t a l i c s )
Thesearejust two examples
whereothersare possible.
Themeaningpotentialof diagrammaticvectorsis broad,abstractand hencedifficultto put into words.Theaccompanying
textstendto be muchmoreexplicitaboutparticipants,
aboutthingsexistingin space(or
represented
as thoughthey are),than aboutprocesses/
eventsand actions.Scientificand
put most of their
writing,and manyforms of expositorywriting generally,
bureaucratic
meaningin the nounsratherthan in the verbs.Verbs,in theseformsof language,
remain
r e s t r i c t etdo a r e l a t i v e lsym a l ls e to f ( l o g i c a l c) o n n e c t o r(s' i s ' , ' h a s ' , ' l e a dtso ' , ' c a u s e s ' ,
' g e n e r a t e s ' , ' d e v e l ionpt o
s ' ,a n ds o o n ) ,a l m o s ta s t h o u g ht h e yw e r e ' f u n c t i o w
n o r d s ' l,i k e

Narrative representat
i ons

articlesand pronounst
ralherthan 'contentwords/.In his book Factual Writing (1985:
4 0 ) , M a r t i ng i v e sa n e x a m p l e :
Thereis little doubtthat televisioncoverage
of a domesticslaughtering
operation,
in a government
conducted
approved
abattoir,whichinvolved
the slaughterof lambs,
calvesandswine,wouldgenerate
a gooddealof publicrevulsion
and protest.
In this extract most of the specificactionsappearin nounIorm, and somehavebeen
nominalized;
that is,turnedinto nounsfrom prior full clausalforms ('doubt','coverage',
' s l a u g h t e r i nogp e r a t i o n ' , ' s l a u g h t e r ' , ' r e v u l s i o n ' , ' p r oI n
t eastte' )x. t o f t h i r t y - s e v ewno r d s ,
thereare onlytwo mainverbs('is' and'generate'),
bothverygeneral.Doingsand happeningshavebeenturnedintothings.Thedynamicsof actionhasbeenchangedintoa staticof
relations.
Diagrams
do something
similar.Theyrepresent
eventswhichtakeplaceovertime
as spatialconfigurationg
and so turn 'process'into 'system'- or into something
ambiguouslyin between,
something
that canbecalledeither'system-centred'or'process-centred'.
I n t h i s r e s p e cdt i a g r a m as r e a k i nt o c e r t a i nf o r m so f n o m i n a l i z i nwgr i t i n gw
, h i l en a t u r a l istic images,
with their humanparticipants
andtheir moreconcrete/
specificprocesses,
are
more akin to story-writing.Like many naturalisticimages,storiesare about humanor
animatebeingsandthe thingstheydo,and in storiesmuchmoremeaningis put intoverbs
than in mostnon-narrative
formsof writing.
Becausetheir meaningis so abstract and general,vectorscan representfundamentallydifferentprocesses
as thoughtheywerethe same(for instance,'humans
typing
lettersand spaceson a keyboard'and'teletypewriters
transmittingelectricalpulses').
Diagramsof the Shannonand Weavertype can imposetwo modelsof interpretation
on
one situationor perhapsone modelon many,'herethe two modelsare 'transport'and
'transformation'.
Figure2.2 represents
what is goingon eitheras transport,movement
from one placeto another,or as the more or lesscausallydeterminedtransformation
from one thing into another.And becauseone sign,the arrow,can representboth,the
two meaningsoften becomeconffated:movement,
transportrs transformation;
mobility
rs the causeof, and conditionfor,change,growth,evolution,progress.
The Shannonand
Weavermodel,for instance,
represents
communication
as transport,as movinginformation from one placeto another,but it also and at the sametime represents
communication as the transformationof messagesinto signals,of 'letters and spaces'into
' e l e c t r i c apl u l s e s ' .
grammar(our diagramin
The arrowsin the 'systemnetworks'of systemic-functional
f i g u r e2 . I 2 i s s u c ha ' s y s t e mn e t w o r k 'a) r e u s u a l l tyr a n s c o d ebdy ' c h o o s eo' r ' s e l e c t (' e . g .
'
').
" C o n c e p t u a ls" e l e c t "sC l a s s i f i c a t i o n a"l "A, n a l y t i c a l "o r " S y m b o l i c a l " B u t v i s u a l l y
the processis, again,a combinationof transportand transformation.
And when such
networksare turnedinto computerprograms/
as indeedthey havebeen,the visualmetap h o r b e c o m eas r e a l i t yi n w h i c ht h e r ea r e n o t p e o p l e ' c h o o s i n g ' b e t w e e n ' o p t i obnust ' ,
pulsestransported
to pointsat whicha changeof stateoccurs.At that pointthe schematic
reductionof one semioticreality has turned into a blueprintfor another,new semiotic
reality,and peoplewill havebeenreducedto the role of'source' and 'destination'in an

Narrati verepresentations

63

just as they are in Shannonand Weaver's

autonomized
and exteriorized
semioticprocess,
c o m m u n i c a t i omno d e l .
Differentkindsof narrativeprocesscan be distinguished
on the basisof the kindsof
vectorandthe numberand kindof participants
involved,
I A c t i o np r o c e s s e s
TheActor is the participantfrom whichthe vectoremanates,
or whichitself,in wholeor in
p a r t ,f o r m st h e v e c t o r( a sw i t h t h e t r i a n g l ei n f i g u r e2 . 1 0 ) .I n i m a g e tsh e ya r e o f t e na l s o
t h e m o s t s a l i e n tp a r t i c i p a n t st h, r o u g hs i z e ,p l a c ei n t h e c o m p o s i t i o nc ,o n t r a s a
t gainst
b a c k g r o u n cd o, l o u rs a t u r a t i o n
f
o
c
u
s
,
t
h
r
o
u g ht h e
and
o r c o n s p i c u o u s n esshsa, r p n e sosf
' p s y c h o l o g i csaal l i e n c ew' h i c hc e r t a i np a r t i c i p a n t(se . 9 t. h e h u m a nf i g u r ea n d ,e v e nm o r e
so,the humanface)havefor viewers.
In figure2.l,for instance,
the Britishare largerthan
t h e A b o r i g i n eas n, dp l a c e di n t h e f o r e g r o u n d
I n. t h e S h a n n o a
n n dW e a v ecr o m m u n i c a t i o n
m o d e l( f i g u r e2 . 2 ) , t h e ' i n f o r m a t i o sno u r c ea' n d t h e ' n o i s es o u r c ea' r e A c t o r s( w e w i l l
commenton their differentpositionin the diagram,and on the differentdirectionality
of
t h e i ra r r o w si,n c h a p t e6r ) :
, i s p a r t i c i p a nits u s u a l l ya n
W h e n i m a g e so r d i a g r a m sh a v eo n l y o n e p a r t i c i p a n t h
Actor.Theresultingstructurewe call non-transactional.The
actionin a non-transactional
p r o c e s hs a s n o ' G o a l ' , i s n o t ' d o n e t o ' o r ' a i m e d a t / a n y o n eo r a n y t h i n gT. h e n o n (the
transactional
actionprocess
is thereforeanalogous
verbin language
to the intransitive
processes
verbthat doesnot take an object).The
in figure 2.I3 are non-transactional:
the waterof the Gulf Streamdoesnot movesomething,
it just moves;andthe wind of the
just
Mistraldoesnot blowsomething,it blows.Thisvisualrepresentation
is akinto theway
processes
meteorological
are represented
in English;it rains,orit snows.As Hallidayhas
pointed
o u t ( 1 9 8 5 :1 0 2 ) ,o t h e rl a n g u a g edso n o tn e c e s s a r d
i lo
y t h i s .I n o n eC h i n e sdei a l e c t ,
for instance,
one has to say somethinglike'the sky is droppingwater'; in otherwords,
process.
raininghasto be represented
as a transactive
In figure2.15,thegestureof the old
gesture
manformsa vector,but hedoesnot
at leastnot so far
towardsanyoneor anything,
as we can seein this picture.As a result,the vieweris left to imaginewho or what he may
b e c o m m u n i c a t i nwgi t h .I s h e a l r e a d yi n t o u c hw i t h w h a t l i e sb e y o n dl i f e ?I s t h a t w h y t h e
youngboy looksat him with suchconcentrated
fascination?

Fig2.t4Actors

Narrative representati ons

Fig2.I5 NewYork,1955(RobertFrank)

At othertimes,thereis onlya vectorand a Goal(figure2.16).TheGoalis the participantat whomor whichthe vectoris directed,henceit is alsothe participantto whomor
w h i c ht h e a c t i o ni s d o n eo, r a t w h o mo r w h i c ht h ea c t i o ni s a i m e d .
R e p r e s e n t a t i oonf as c t i o n w
s h i c hi n c l u d e
o n l yt h e G o a lw ew i l l c a l l E v e n t ss:o m e t h i nigs
h a p p e n i nt o
g someonb
eu
, tw e c a n n ost e ew h oo r w h a tm a k e si t h a p p e nF. i g u r e2 . 1 7s h o w s
a diagramwhich appearedin the SydneyMorning Heraldduringthe first Gulf War.A
vectorrepresents
the actionof movingtowardsthetownof l(hafji,andthe Goalisthetown
of l(hafji itself,represented
by a blackdot. But nothingrepresents
the war planeswhich
are movingtowardsl(hafji.Closelyrelatedis the casein which just a small part of the
Actor ls visible,a hand,or a foot, so that the Actor becomes
anonymous.
In both cases
there rs in fact an Actor,as in figure 2.I7, but the Actor is either deletedfrom the
r e p r e s e n t a t ioornm a d ea n o n y m o uas v/ i s u aal n a l o g u ep ,e r h a p so,f ' p a s s i vaeg e n td e l e t i o n ' ,
a l i n g u i s t ifco r m o f r e p r e s e n t a t itohna t p l a y sa n i m p o r t a nrt o l ei n c r i t i c a ll i n g u i s t i casn d
c r i t i c a ld i s c o u r saen a l y s i sa,s w h e na n e w s p a p ehre a d l i n sea y s , ' F i f t e e n
Demonstrators
Shot in Riots',therebyomittingto mentionthat they were shot by police(frew, I979:
eTff).
Whena narrativevisualproposition
hastwo participants,
oneis the Actor,the otherthe
Goal.fheActor in sucha transactionalprocess
is not so muchthe participantwhichmoves
(as in the non-transactional
process)as the participantwhich instigates
the movement,
process
and if we hadto givea verbalparaphrase
of a transactional
we wouldprobablyuse

Fig2.16Goat

Nar rat i verePresentati ons

65

Fig 2.17 Gulf War Diagnn (SydneyMoming Herald,I4 Fehruary1991)

or'send'insteadof Iintransian object(e.g.'transport'

a transitiveverb,a verbthat tal<es
tivel'move').
arethe
In'The Britishusedguns/(figure2.1),thetwo menarethe Actor,theAborigines
by
formed
vectors
process:
are
also
there
second
transactional
Goal.Thereis, in fact, a
a
constitute
so
and
the
fire,
to
Aborigines
the
from
the
heads
of
linesthat can be drawn
p r o c e s si n w h i c ht h e A b o r i g i n easr e t h e A c t o r a n d t h e f i r e t h e G o a l . ' T h eA b o r i g i n e s
s u r r o u ntdh ef i r e ' ,o n ec o u l dt r a n s c o d eB.u tt h e p o i n ti s n o tt o f i n da v e r b ael q u i v a l e nt th;e
as Actor and the fire as Goal.
point is to establishthat the Aborigines
are represented
is a
this process
background,
placement
the
further
towards
of its smaller sizeand
Because
, m i n o rp r o c e s se, ,m b e d d eidn t h e m a j o rp r o c e s sT.h ew h o l ec o u l db e t r a n s c o d eads ' T h e
s ,h os u r r o u ntdh ef i r e ' .
B r i t i s hs t a l kt h e A b o r i g i n ew
possible
to arguethat structuressuchasthe ShannonandWeaverdiagram(figure
it is
2.2) havebeenaffectedby the fact that Westernculturegivessuchcentralityto language
and nouns/
that the structureof English,wlth its lexicaldistinctionof verbs/processes
vectors/processes
as
arrows
So
semiotic
schema.
as
a
model
for
a
objects,may haveacted
translationfrom
may be a more or lessunconscious
and boxesas participants/nouns

Narrati verepresentati
ons

language
intothe visual.However,
it is importanthereto insiston the distinctorganization
of the two modes.The visualstructureof arrowsand boxesconveysa strongsenseof
' i m p a c t i n go' r ' t a r g e t i n gw
' , h i c hi s q u i t ea b s e nitn t h ev e r b atlr a n s l a t i o nwsh i c hc o m em o s t
procedure
immediately
to mind.Thevisualstructureforegrounds
oversubstantive
content,
the act of impacting'overwhat makesthe impact,more or lessin the way that, for
instance,
marketingexpertsare oftenmoreconcerned
aboutstrategies
for reachingconsumersthan about the goodsand servicesthat shouldreachthem, or that pedagogic
expertsare more concerned
about the format of classroominteractionthan about the
contentof lessons.

i--._:--*
-\\-

------'--"

. z-: - ,/a--

Fig2.l8 Speechcircuit(deSaussurelgT4tlgf6D

S o m et r a n s a c t i o n satl r u c t u r easr e b i d i r e c t i o n aela, c hp a r t i c i p a npt l a y i n gn o wt h e r o l e

of Actor,nowthe roleof Goal,as for instancein de Saussure's
well-known'speechcircuit'
(
f
i
g
u
r
e
diagram
2 . 1 8 ) i n w h i c h ' A ' a n d ' B ' a r e n o ws p e a k enr ,o w l i s t e n eIrt. i s n o t a l w a y s
c l e a rw h e t h ebr i d i r e c t i o ntarla n s a c t i o nasr e r e p r e s e n t eads o c c u r r i n sgi m u l t a n e o u solryi n
succession,
althoughthereis a tendency
to useonearrowwith two headsto signifysimultaneity,and two arrowspointingin differentdirectionsto signifysequentiality.
In figure
2.I8,for instance,
sequentiality
is realizedby two separatedottedlines(line and arrowhead,connection
anddirectionality,
are separate
elements
in this diagram).

Fig 2.19 Simultane0us

and sequentialbidirectionality

We will refer to the participantsin suchstructuresas Interactors.to indicatetheir

d o u b l er o l e .

Narrativerepresentations. 67

2 Reactionalprocesses
Whenthevectoris formedby an eyeline,
bythe directionof the glanceof oneor moreof the
participants,
process
represented
the
andwe will speaknot of Actors,but of
is reactional,
Reacters,
Reacter,
the participantwho doesthe
and not of Goals,bul of Phenomena.The
w i t h v i s i b l ee y e s
l o o k i n gm
, u s tn e c e s s a r ibl ye h u m a no, r a h u m a n - l i kaen i m a l a c r e a t u r e
pupils,
The
Phenomenon
maybeformed
that havedistinct
andcapableof facialexpression.
participant,
participant
which
Reacter
is looking,or
eitherby another
the
the
at whomor
p
r
o
p
o
s
i
t
i
o
n
,
s
t
r
u
c
t
u
r
e
I
.
n
f
i
g
u r e2 . ! 5 , f o r
b y a w h o l ev i s u a l
f o r e x a m p l ea, t r a n s a c t i o n a l
gesture
young
boy is the
instance,
whilethe
the old man and his
form the Phenomenon,
in
a transis
Actor
Reacter.
In figure2.20,an advertisement
for mineralwater,the man
('The
process
might
one
actionalaction
in which the water is Goal
man drinl<swater',
represented
the two
transcode):
the wholeangleof hisbodyformsa strongvectorbetween
participants.
of a reacthe Phenomenon
Thisprocess('Man drinkswater')thenbecomes
tional structurein whichthe womanis Reacter a vectotformedby the directionof her
glanceandthe angleof her left arm,leadsfrom herto the drinl<ing
man.Shereactsto his
(the
precise
actionwith a smileof approval
natureof reactionsis colouredin by facial
expression).
Themanas doer,thewomanas faithfuladmirerof hisactions,is a distribution
(I976), is verycomof roleswhich,as Goffmanhasshowninhis GenderAdvertisements
(
b
u
t
m o n i n a d v e r t i s e m e n t s n o t o n l v i n a d v e r t i s e m e n t s ) : ' W hae nm a n a n d a w o m a n

T..-::

lfffi

j
tjesi?;s

i
i"*r'-*,tfiil.,
;T .*.*"'*a{&,:s
es-i*$-ir-;:,i.i}fri
f;w$$$ryrrwr-i;r*;Fi{i
O

Fig2.2O Vittel advertisement(/Verlde4 5 Decenber 1987)

N arrat i ve representati ons

c o l l a b o r a tien a n u n d e r t a k i n tgh, e m a n i s l i k e l yt o p e r f o r mt h e e x e c u t i vreo l e ' ( G o f f m a n ,

7976:3D.
Like actions,reactionscan be transactionalor non-transactional.
In the latter case
t h e r ei s n o P h e n o m e n oans,i st h ec a s ew i t ht h e l o o ko f t h eo l dm a ni n f i g u r e2 . 1 5 .I t i st h e n
l e f tt o t h e v i e w e tro i m a g i n w
e h a t h e o r s h ei s t h i n k i n ga b o u to r l o o k i n ga t , a n dt h i s c a n
createa powerfulsenseof empathyor identification
participants.
with the represented
Sometimesphotographers
or picture editors crop photosback to close-upsof nontransactionalReacters
who look bored,or animated,or puzzled,at something
we cannot
see.This can becomea sourceof representational
manipulation.
A caption may,for
instance,
suggestwhat the Reacteris lookingat, but, needless
to say,it neednot be what
t h e R e a c t ewr a sa c t u a l l yl o o k i n ga t w h e nt h e p i c t u r ew a st a k e n .S t u a r tH a l l ( 1 9 8 2 )h a s
d e s c r i b ehdo w t h i sk i n do f m a n i p u l a t i oi sn u s e di n p r e s sp h o t o g r a p hosf p o l i t i c i a n s .
3 S p e e c hp r o c e s sa n d m e n t a lp r o c e s s
A s p e c i akl i n d o f v e c t o rc a n b e o b s e r v eidn c o m i cs t r i p s t: h e o b l i q u ep r o t r u s i o nosf t h e
t h o u g h tb a l l o o n a
s n dd i a l o g u e
b a l l o o ntsh a t c o n n e cdt r a w i n g o
s f s p e a k e rosr t h i n k e r st o
t h e i rs p e e c h
o r t h o u g h tU
. n t i l r e c e n t l tyh e yw e r ec o n f i n e d
t o t h e c o m i cs t r i p s a
, lthough
therehave,of course,
processes
alsobeenspeech
in medievalart,for instancein the form of
r i b b o n se m a n a t i n fgr o m t h e s p e a k e r 'm
s o u t h .T o d a yt h e y i n c r e a s i n gcl yr o p u p i n o t h e r
contexts,
too; for instance,
in connection
with quotesin schooltextbooksor on the screens
of automaticbanktellers.Liketransactional
reactions,
theseprocesses
connecta human
(or animate)beingwith 'content',but wherein transactional
reactionsit isthe contentof a
p e r c e p t i o inn,t h e c a s eo f t h o u g h b
t u b b l eas n ds i m i l a rd e v i c eist i s t h e c o n t e not f a n i n n e r
mentalprocess(thought,fear,etc.),and in the caseof speechvectorsthe contentof the
s p e e c hH. a l l i d a y( 1 9 8 5 :2 2 7 f f . )c a l s t h i s k i n do f s t r u c t u r e ' p r o j e c t i vTeh' .e P h e n o m e n o n
o f t h e t r a n s a c t i o n aRle a c t i o na,n dt h e c o n t e not f t h e d i a l o g u e
b a l l o o no r t h o u g h b
t alloon
are not represented
directly,but mediatedthrougha Reacter,
a'Senser'(in the caseof a
t h o u g hb
t a l l o o no) r a ' S p e a k e r(' i n t h ec a s eo f t h e d i a l o g u b
ea l l o o n ) .
processes
4 Conversion
The Shannonand Weavercommunication
model (figure2.2) forms a chain of transa c t i o n apl r o c e s s eTsh. i sc h a i n i n rge s u l t si n a t h i r d k i n do f p a r t i c i p a nat ,p a r t i c i p a nwt h i c h
is the Goalwith respectto oneparticipantandthe Actor with respectto another.
Shannon
a n dW e a v e r ' s ' t r a n s m i t t e rs' iusc ha p a r t i c i p a nat ,c t i n ga s G o a lw i t h r e s p e ct to t h e ' i n f o r m a t i o ns o u r c e ' a nadsA c t o rw i t h r e s p e ct to t h e ' r e c e i v e rW
' . ew i l l c a l lt h i sk i n do f o a r t i c i panta Relay(our useof thisterm obviously
differsfrom that of Barthesll977l, who uses
it to denotean image-textrelationin which the text extends,rather than elaborates
[ ' a n c h o r s ' tJh, e v i s u a li n f o r m a t i o n R
) .e l a y d
s o n o t j u s t p a s so n / i n u n c h a n g ef d
o r m ,w h a t
they receive;they alwaysalsotransformit - for instance,
in the caseof communication
models,
from 'lettersandspaces'into'electricalpulses',
or,as in figure2.23,from 'grasses,
sedges
andffowers'into'urine,droppings
anddeadcarcasses'.

Narrat i verepresentatIons

Fi92.2l Retay

process
r o d e lr e p r e s e nct so m m u n i c a t i oans a c h a i n e d
W h i l et h e S h a n n o an n dW e a v em
with a beginningand an end,henceas an agentiveprocess/a processset into motionby an
as a cycle,
communication
suchasthat shownin figure2.22,represent
Actor,othermodels,
a n d i n t h a t c a s ea l l t h e p a r t i c i p a n tasr e R e l a y sa,n da g e n c yi s m o r ew e a l < lsyi g n i f i e dT'h i s
l ll l a C o n v e r s i o n p r o c e s s , i s e s p e c i a l l y c o m m o n i n r e p r e s e n k i n d o fp r o c e s s , w h i c h w e wci a
representafood chaindiagramsor diagrammatic
tationsof naturalevents;for instance,
. u t ,a s f i g u r e2 . 2 2s h o w si,t c a na l s ob e a p p l i e dt o h u m a n
t i o n so f t h e h y d r o l o g i c cayl c l e B
asthoughit wasa
is represented
(inter)action,
human(inter)actlon
andwhenthis happens
naturalprocess.

("..")
/o""oo")

/en"oo")

Interpreter

Interpreter

t"'o0"7

t*"?

F i g 2 . 2 2C o m m u n i c a t i o n m o d e l ( f r o m w a t s o n a n d H i l l , l 9 S 0 : 1 4 7 )

\=*"0"7

70 .

Narrative representations

!. il

t'i*

i'atitilt

ilr*r****rrs

l?,i

..-..-..-..-..._.,"_.-,-,".-.-.-..,"_..-_,1

.si:i]#*+'d*ihan.
i!

*;t

!:i3t*IeS
! / " -

r+' i:i)3::r!i,,p,1i*::*
!:ji-1{ta,!1jir!:ll

&

'q]wsG

i,i:|,$irn iltiii$i #\4 AiSSal

l;trlf,i! lr: :14v* riii4 qr$ii!{
ia!t*.i 1+!s Y*rf$ ls.P.**sl
n.!,i. :-3iirriis s:!*,Sr#!!{q

ilillf"
ff"

i*'-'q * x l ' :

---------"----l
i----"
I Ptr{.*s$*s$fs l3'.: i
i-----,_.--.--.---.:-i

*"'***uusll*.

., 'lri&#4:S!ilqt:l
'' :'cq$e{ &6aS:$

Fig 2.23 Arctictundrasystem(Saleetal.,1980)

5 G e o m e t r i c asl y m b o l i s m
s .h e r ei s
F i g u r e2 . 2 4 ,a n o t h e r ' c o m m u n i c a t im
oo
n d e l /d/ o e sn o t i n c l u d ea n y p a r t i c i p a n t T
o n l ya v e c t o ri,n d i c a t i ndgi r e c t i o n a l i b
t yy m e a n so f a n ' i n f i n i t y ' s i g nr ,a t h e rt h a nb y m e a n s
of an arrowhead.Dance'sdiagramis in fact not so much a communication
modelas a
'metadiagram'which
in orderto discuss
showsusa process
in isolation,
why helicalvectors
are moresuitablefor the representation
of communication
straightor
than,for instance,

Fig 2.24 Communication

model(from Dance,1967)

Narrative representations . 7l

c u r v e da r r o w sD
. a n c ed o e st h i sb y p o i n t i n g
croperties
a t t h es y m b o l im
c e a n i n gosf i n t r i n s i p
o f t h e h e l i xA
. c c o r d i ntgo D a n c et h
, es h a p eo f t h e h e l i x :
c o m b i n etsh e d e s i r a b lfee a t u r e o
s f t h e s t r a i g h lt i n ea n do f t h e c i r c l ew
, h i l ea v o i d i n g
t h e w e a k n e sosf e i t h e r . . . . l t g i v e st e s t i m o n tyo t h e c o n c e ptth a t c o m m u n i c a t i o n ,
whilemovingforward,is at the sametimecomingbackuponitselfandbeingaffected
by its pastbehaviour,
affectedby
for the comingcurveof the helixis fundamentally
t h e c u r v ef r o mw h i c hi t e m e r q e s .
(Dance,1967)
Imagesof this kind usepictorialor abstractpatternsas processes
whosemeanings
are
c o n s t i t u t ebdyt h e i rs y m b o l ivca l u e sa,n ds oe x t e n d
t h ev e c t o r i avl o c a b u l a rbyy d r a w i n go u r
a t t e n t i o tno p o s s i b i l i t i eb se y o n tdh e d i a g o n aal c t i o nl i n eo r t h es i m p l ea r r o w :c o i l ss, p i r a l s ,
heIixes.
Variantsof the arrow may affectthe meaningof the processin narrativediagrams.
A
partakesof the symbolicvalueof the circle,so that the process
curvedarrow,for instance,
is represented
as'natural'and'organic'(seefigure2.22).Vecforsmayalsolrc attenuated,
b y t h e u s eo f d o t t e dl i n e sb, y m a k i n gt h e a r r o w h e asdm a l l e ro r l e s sc o n s p i c u o ui ns o t h e r
w a y so/ r b y p l a c i n gi t i n t h e m i d d l er,a t h e tr h a na t t h ef r o n to f t h e l i n ew
, h i c hd i m i n i s h et hse
senseof impacting'and 'targeting',and causesthe meaningof the vectorto movein the
o f m e r ec o n n e c t i v i (t vs e ef i q u r e2 . 2 5 ) .
direction

Attenuatedvectors

Fig 2.25 Attenuatedvectors

The vectorialrelationmay also be amplified,by meansof bolderarrows(seefigure

2.26),whichperhaps
suggest
a certaindensityof 'traffic',as in figure2.17,the diagramof
i

A m p l i l i e d v !c t o r

Ampliliedvector

i nd icati ng

i nd icatj ng

cenSlty

frequency

Fig 2.26 Amplitied vectors indicating density and frequency

72 .

Narrative representations

the attackon l(hafji,or by the useof a numberof arrows,whichmaysuggest

the f requency
o r m u l t i p l i c i tw
y i t h w h i c ht h e p r o c e sosc c u r s .
In imagea
s s i m i l a re f f e c tc a nb e a c h i e v ebdy m a k i n gt h e d i a g o n aal c t i o nl i n e sm o r eo r
lessconspicuoust
moreor lessdominantin the composition
as a whoie.
6 Circumstances
A s w e i n d i c a t eidn o u r d i s c u s s i oonf ' T h e B r i t i s hu s e dg u n s ' ( f i g u r 2
e . I ) , n a r r a t i v ei m a g e s
may containsecondary
participants,
participantsrelatedto the main participants,
not by
meansof vectors,but in otherways.Wewill referto theseparticipants,
followingHalliday
(1985), as Circumstances.They
are participantswhichcould be left out without affecting
the basicproposition
realizedby the narrativepattern,eventhoughtheir deletionwouldof
courseentaila lossof information.
LocativeCircumstances
relateotherparticipants
to a specificparticipantwe will call
Setting.Thisrequiresa contrastbetweenforegroundand background,
whichcan be realizedin oneor moreof the followingways:(1) the participantsin the foregroundoverlap,
and hencepartiallyobscurethe Setting;(2) the Settingis drawnor paintedin lessdetail
(or, in the caseof photography,
has softer focus),'G) the Settingis more muted and
desaturated
in colour,with the variouscoloursall tendingtowardsthe samehue,usuallythe
(4) the Settingis darkerthanthe foreground,
blueof distance;
or lighter,
sothat it acquires
'overexposed',
an
ethereallook.Theseformalfeaturescan occurin variouscombinations,
-'more-or-less,,
andtheyare all gradients
ratherthan 'either-0r,,features.
As we will discussin moredetailin chapter5, settingshaveimportance
for the realization of visual modality.The Settingsthemselves
can of coursebe read as embedded
('Thelandscape
analyticalprocesses
consistsof grass,treesand rocks,).
Thetoolsusedin actionprocesses
are oftenrepresented
as Circumstances
of Means.If
this is the case,thereis no clearvectorbetween
the tool and its user.Thetoolsthemselves
may,of course/
constitutethe vectorswhichrealizethe actionprocesses/
aswith the gunsin
' T h eB r i t i s hu s e dg u n s (' i n
f i g u r e2 . r ) , a n dt h e yn e e dn o tb eo b j e c t sF. o ri n s t a n c e , wweo u l d
i n t e r p r e t t hgee s t u r e
o f t h e o l d m a ni n f i g u r e2 . 1 5a s a n o n - t r a n s a c t i o n
aa
c tl i o n( ' T h eo l d
man addresses
an unseenparticipant'),and his handsas a circumstance
of Means('The
old manaddresses
participantwlthhishands,).
an unseen
Figure2.27 showsa penguinwith her baby.Thereis againno vectorto relatethe two.
Yetthe penguinand her babyclearlyform two distinctparticipants:
this is a pictureof a
penguinwith ababy.In sucha casewe will interpretthe relationas a Circumstance
of
AccompanimeA
n ts. t h e p i c t u r ec o n t a i n n
s o v e c t o ra n dd i s p l a ytsh e p e n g u i n
m o r eo r l e s s
frontally,againsta de-emphasized
background,
we interpretit as'analytical,,
asthe kindof
picturemorelikelyto illustratea text givingdescriptive
information
aboutpenguins
thana
storyaboutwhat penguins
do.

Narrativerepresentations' 73

Fig 2.27 Penguinwith baby(oakleyefal.,1985)

SUMMARY
F i g u r e2 . 2 8 s u m m a r i z et sh e d i s t i n c t i o nwse h a v ei n t r o d u c eidn t h i s s e c t i o n .F' o l l o w i n g
H a l l i d a y( 1 9 8 5 ) ,w e h a v ec a l l e dp r o c e s s et hsa t c a nt a k ea w h o l ev i s u a l( o r v e r b a l )p r o p osition as their 'object' proiective,and the othersnon-proiective.The squarebrackets
a rl
t r a n s a c t i o n a lc t i o ni s e i t h e ru n i d i r e c t i o n o
i n d i c a t ea s i n g l ec h o i c ef;o r i n s t a n c e , ' a
p
r
i
n
c
i
p
a
l
in
w
a
y
s
b i d i r e c t i o n a l ' . 0 ucrl a i m i s t h a t t h e ' c h o i c e s ' i nf i g u r e2 . 2 8 c h a r tt h e
'
d
o
i
n
g
'
a
n
d
o
f
i
n
w h i c h i m a g e sc a n r e p r e s e nt th e w o r l d ' n a r r a t i v e l y ' -t h a t i s , t e r m s
' h a p p e nn ig ' .

74 ,

Narrativerepresentations

Action -----N
lon-lrolective
{

-----N

Transactional

Non-transactional

Unidirectional

Bidirectional

( Non-transactional)reaction

^**,*{

-*l
,a"a"rra,

P.oj(tiw-{

Mental process
VerbaI process

.u"rr",'""
*f

Non agentive: Conversion

structures

Setting

L Cirauartunaar{

lvleans
Accompaniment

Fig 2.28 Narrativestructuresin visualc0mmunication

REALIZATIONS
UnidirectionaI transactional
action
Bi di rectionaI transacti onaI
action
Non-transact
i ona I acti on

Actor

Goal

Interactors

TransactionaI reacti on
Non-transacti onaI reacti on

A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
depictedelement,
or an arrowlconnects
two
p a r t i c i p a n tas n
, A c t o ra n da G o a l .
A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
depictedelement,
or a double-he
adedarrow,
connects
two Interactors.
A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
depictedelement,
or an arrowtemanates
from
a participant,
the Actor,but doesnot pointat
anyotherparticipant.
Theactiveparticipantin an actionprocessis
the participantfrom whichthe vector
emanates
or whichis fusedwith the vector.
participantin an actionprocess
Thepassive
participant
is the
at whichthe vectoris
di rected.
Theparticipantsin a transactional
action
process
wherethe vectorcouldbe saidto
emanatef rom,andbe directedat, both
participants.
An eyeline
vectorconnects
two participants,
a
Reacteranda Phenomenon.
An eyeline
vectoremanates
from a participant,
the Reacter,
but doesnot pointat another
participant.

Narrativerepresentati ons

Reacter
Phenomenon

Conversion

Mental process

Senser
Verbal process

Sayer
Utterance
Qailinn

Means

Accompaniment

Theactiveparticipantin a reactionprocessis
the participantwhoselookcreatesthe eyeline.
participantin a (transactional)
Thepassive
r e a c t i o ni s t h e p a r t i c i p a nat t w h i c ht h ee y e l i n e
is directed;in otherwords,the participant
look.
whichformsthe objectof the Reacter's
Thesameterm is usedfor the participant
( v e r b aoi r n o n - v e r b ael )n c l o s ebdy a ' t h o u g h t
b u b bl e ' .
in whicha participant,
the Relay,is
A process
the Goalof oneactionandthe Actor of
Thisinvolves
a chanqeof statein the
another.
participant.
r
A v e c t o rf o r m e db y a ' t h o u g h tb u b b l e ' o a
s i m i l a rc o n v e n t i o ndael v i c ec o n n e c ttsw o
p a r t i c i p a n ttsh,e S e n s ear n dt h e P h e n o m e n o n .
T h ep a r t i c i p a nf rt o mw h o mt h e ' t h o u g h t
bubble'vectoremanates.
A vectorformedby the arrow-likeprotrusion
b a l l o o no' r s i m i l a rd e v i c e
of a'dialogue
a Sayerandan
two participants,
connects
Utterance.
from whom
Theparticipantin a verbalprocess
t h e ' d i a l o g ubea l o o n 'e m a n a t e s .
T h e( v e r b a lp) a r t i c i p a netn c l o s eidn t h e
' d i a l o g u be a l l o o n ' .
is recognizable
TheSettingof a process
the participantsin the foreground
because
overlapandhencepartiallyobscureit; because
it is oftendrawnor paintedin lessdetail,or,in
hasa softerfocus;
the caseof photography,
of contrastsin coloursaturation
and because
between
and overalldarkness
or lightness
f o r e g r o u nadn db a c k g r o u n d .
is formedby the tool
The Meansof a process
with whichthe actionis executed.
It usuallyalsoformsthe vector.
A n A c c o m p a n i m eins ta p a r t i c i p a ni tn a
narrativestructurewhichhasno vectorial
andcannot
relationwith otherparticipants
be interpreted
as a SymbolicAttribute(see
c h a p t e3r ) .

76 .

Narrative representations

V I S U A LS T R U C T U R E S
A N D L I N G U I S T I CS T R U C T U R E S

We havedrawnattentionto the fact that,whilebothvisualstructures

andverbalstructures
canbe usedto express
meanings
drawnfrom a commonculturalsource,
the two modesare
n o t s i m p l ya l t e r n a t i vm
e e a n so f r e p r e s e n t i n g ' t sh ae m et h i n g ' .I t i s e a s yt o o v e r e m p h a s i z e
e i t h e rt h e s i m i l a r i t o
y r t h e d i f f e r e n cbee t w e etnh e t w o m o d e s . 0 n lay d e t a i l e d
comparison
canbringout howin somerespects
theyrealizesimilartypesof meaning,
thoughin different
ways,while in other,perhapsmost respectsthey representthe world quite differently,
allowingthe development
of the differentepistemologies
we discussed
in the previous
chapter.In this brief final sectionwe wishto explorethis in somedetailwith respectto
narrativevisualstructures.
By comparison
to the structures
we will discuss
in chapter3,narrativevisualstructures
are comparatively
easyto'translate';though,as we will see/therecertainlyis no one-too n ec o r r e s p o n d e nLciek.e ' n o n - t r a n s a c t i oancatli o n s ' , ' o n e - p a r t i c i pma an t e r i apl r o c e s s e s '
( Halliday,1985: 103ff.) represent
eventsas thoughthey bearno relationto, and haveno
consequences
for, participantsotherthan the Actor.In Many peoplemigrated onecannot
add a secondparticipantto this clauseand say,for instance,Many peoplemigratedtheir
relatives,althoughone can of courseadd circumstances:
Many peoplemigrated to Aus(Halliday,
tralia. And,like 'transactionalactions','two-participantmaterialprocesses'
1985: 103ff.) involvetwo participants(e.9.MigrantsinvadedAustralid. But linguistic
'Events'and
v i s u a 'lE v e n t s ' a r eq u i t ed i f f e r e n tL. i n g u i s t i E
c v e n t sh a v ep r o c e s s et hsa t a r e
'happenings'which
cannothavean Actor,as in Many of my relativesdied.In the caseof
visualEvents,the Actor is left out, but couldhavebeenused.Theyare the equivalent
of
passives
with agentdeletion,of clauseslike Many of my relativeswerekilled, ratherthan
of clauseslike Many of my relativesdied.Toshowsomeonedying ii is necessary
to show
someone
beingkilled,or to showsomeone
performingan actionthat represents
hisor her
death.Also,whilein Englishmanyprocesses
cantakea third participant,
the 'Beneficiary'
(traditionally'indirect
object'in,e.9.,Mary gavehim theboob,in imagesthe possibility
of
sucha third participantdoesnot exist.What is a Beneficiary
in Englishbecomes
a Goalin
images('shemessage-sends
him' insteadof 'shesendshim a message').
0n the other hand, Englishlacksthe visual mode'sstructuraldevicesto represent
e v e n t sa s c y c l i c a(l a l t h o u g h
l i n g u i s t ipca r t i c i p a n tcsa nh a v ea d o u b l er o l ei n E n g l i s -h f o r
instance,
in exampleslike He madethemdo it,wherethemis Goalof makedo as well as
A c t o r o f d o ;c f . H a l l i d a y1,9 8 5 : 1 5 3 ) . N o r i s t h e r ea n ' i n t e r a c t i o n a l ' p r o c eTs o
s .r e a l i z e
what we havecalled'lnteractors',Englishwouldhaveto makeuseof reffexive
pronouns.
Consider,
for instance,the problemof trying to 'translate'figure 2.18, de Saussure,s
' s p e e c hc i r c u i t ' d i a g r a mi,n t o E n g l i s hA. s i n g l ev i s u a lp r o c e s s
i n d i c a t es o m e t h i n fgo r
w h i c h i,n E n g l i s hw, e n e e da t l e a s ft o u r c l a u s e s : ' A
s p e a ktso B ' , ' B s p e a ktso A , , ' A l i s t e n s
'A and B communicate
to B','B listensto A'. Howcan onerenderthis in onec{ause?
with
'A'
'B'to
eachother'? But that causes and
losetheir separateidentity,transforminga
reciprocal,
bidirectional
transactionintoa jointlyauthorednon-transactional
action.
W h a t w e h a v ec a l l e dt h e ' n o n - t r a n s a c t i o nr ae la c t i o n ' i si n s o m ew a y sa k i n t o w h a t
H a l l i d a y c a l l s t bh e h a v i o u r a l p r o c e s s ( 1 9 8 5 : I 2 B ) , a p r o c e s s t y p e w h i c h c a n t a k e o n l y o n

Narrativerepresentations. 77

participant(whomustbe human)andservesto realizea restrictedfieldof action,the field

o f ' p h y s i o l o g i c aaln d p s y c h o l o g i cbael h a v i n g('1 9 8 5 : 1 2 8 ) . B u t t h e m e a n i n gosf v i s u a l
'non-transactional
reactions'form
a morerestricted
field,tied up astheyarewith onekind
o f b e h a v i o u- r l o o k i n g .
-that is,mentalandverbalprocesses
- playan importantpart in
Projectiveprocesses
E n g l i s ha,n di t i s p o s s i b lteo d i s t i n g u i sahn u m b eor f d i f f e r e ntty p e so f e a c ho n t h e b a s i so f
s /r i n s t a n c e ,
f o r m a lg r a m m a t i c ac lr i t e r i a( H a l l i d a y1,9 8 5 :1 0 6 f f . ,1 2 9 ) .M e n t a p
l r o c e s s ef o
includeprocesses
of perception('see','hear',etc.),processes
of affection('like','fear',
'wish'e
, t c . )a n dp r o c e s s eosf c o g n i t i o n
( ' k n o w ' , ' t h i n k ' , ' b e l i e veet c' ,. ) .E a c hh a sa S e n s e r ,
t h e p e r s o nw h o d o e st h e s e e i n go, r l i k i n g ,o r k n o w i n g( t h i s h a s t o b e a p e r s o no/ r a
p a r t i c i p a nrte p r e s e n t eads h u m a n )a, n d a P h e n o m e n osno, m e o n oe r s o m e t h i nsge e no/ r
l i k e d ,o r k n o w n ,b y t h e S e n s e rP. h e n o m e nm
a a y b e r e a l i z e db y p a r t i c i p a n tos r w h o l e
structures,just as in the caseof images.In Manypeoplewant to migrate to Australia,the
apart from
clausefo migrate to Australiais Phenomenon;
this sets mentalprocesses
a c t i o n sa n dt r a n s a c t i o nwsh, i c hc a n n o ht a v ea c l a u s ea s G o a l .B u tw h a tw e h a v ec a l e dt h e
'transactional
reaction'can,if onewishesor needsto, be relatedto only a subsetof the
p e r c e p t i opnr o c e s sb ,e c a u sneo n - v i s u aPlh e n o m e ncaa n n odt i r e c t l yb e r e a l i z e idn t h ev i s u a l
s e m i o t i cM
. e n t a lp r o c e s s ef os r m ,a s w e h a v es h o w no, n l ya m i n o rc a t e g o r yi n t h e v i s u a l
for makingthe strong
semiotic;as far as we cansee,thereare no structuralvisualdevices
d i s t i n c t i obne t w e e n ' c o g n i t i o n ' a' a
n fdf e c t i o np' r o c e s s e
t hsa t h a sc o m et o c h a r a c t e r i zt hee
setof
ideationalresources
of English.
Thecinema,however,
hasdeveloped
afairlyextensive
projectiveconventions
for realizingdifferentkindsof mentalprocesses
suchas memories,
d r e a m sh, a l l u c i n a t i o nasn,ds o o n .
In English,verbalprocesses
differ from mentalprocesses
in that they do not needa
human'Sayer'(onecan say Thedocumentsaid that..., but not Thedocumentthought
t h a t . . . ) . 0 n t h e o t h e rh a n d ,l i k em e n t a p
l r o c e s s et sh,e yc a nt a k ew h o l ec l a u s eas s t h e i r
object,andthis in two differentways- in theform of Reported
SpeechGs in He saidtthatl
he had no ided and in the form of QuotedSpeech(e.sinHe said'l haveno idea).fhere
s e e m st o b e n o d i r e c t s, t r u c t u r awl a y o f e x p r e s s i nt hgi s k i n d o f d i f f e r e n cvei s u a l l y . ' D i a l o g u eb a l l o o n sa' l w a y sq u o t e .
We haveidentified
in images:location,means
onlythreedifferenttypesof circumstance
a n da c c o m p a n i m e n t . A l l t h reexei s ti n E n g l i s h( H a l l i d a y , 7 9 8 5 : 1 , 3 7 1bfu. )t ,t h e r et h e ya r e
b y n o m e a n tsh eo n l yt y p e sE
. nglisa
h l l o w sa l l k i n d so f i n f o r m a t i otno b ea d d e dt o t h e b a s i c
narrativepropositionconveyed
and the participants
by the process('What happened?')
( ' W h oo r w h a tw a si n v o l v e d ? 'i )n;f o r m a t i oanb o u t i m e( ' W h e nd i d i t h a p p e n ? ' ; ' H olw
ong
d i d i t l a s t ? ' )a; b o u tp u r p o s (e' W h a td i d i t h a p p e n
f o r ? ' ) ;c a u s e( ' W h yd i d i t h a p p e n ?a' )n d
s o0 n .
Thefollowingtablegivesan overview
betweenlinguistic
of someof the correspondences
andvisualnarrativeorocesses:

Narrat ive representati ons

Table2.7 Nafiatiue processin languageand yisual communication

VisuaI narrati ve processes

L i nquisti c narrative cIauses

Non-transactional
action
Unidirectional
transactional
action
Event
Bidirectionaltransactional
action
Non-transactional
reaction
reaction
Transactional
Mentalprocess
Verbalprocess

(Actor)materialprocess
('action')
0ne-participant
Two-participant
materialprocess
Passive
transactional
clausewith agentdeletion
(fieldof looking)
process
Behavioural
(visualonly)
perception
Mentalprocess:
(cognition
Mentalprocess
andaffection)
(quotation)
Verbalprocess
(affection
Verbalprocess
)

Conversion

suchas thesecan highlightwhichwaysof representing

Comparisons
the world can be
realized
y n dw h i c h( m o r eo r l e s s )i n b o t hw a y sT. h i s i,n t u r n ,i s
l i n g u i s t i c a lw
l yh, i c hv i s u a l l a
for analysingrepresentation
usefulas a background
in multimodaltexts:photographs
and
their captions,diagramsand their verbalglosses,
If, for
storiesand their illustrations.
instance,
a diagramshowsan arrowemanatingfrom a participantlabelled'environment'
a n dd i r e c t e a
d t a n o t h epr a r t i c i p a nl ta b e l l e'dm e s s a g e ' , t h ae 'nl i t e r a l ' t r a n s l a t i owno u l db e
'the environment
acts uponthe message/.
If the accompanying
text saysthat the'comg i t h f a c t o r s ( o r s t i m u l i )f r o m t h e e n v i r o n m e n t '
m u n i c a t i o np r o c e s s ' i s ' i n t e r a c t i n w
( W a t s o na n d H i l l ,1 9 8 0 : 1 4 ) ,i t ' m i s t r a n s l a t e sa' n, d t h e m i s t r a n s l a t i oi sn n o t d u et o t h e
l i m i t a t i o nosf e i t h e rE n g l i s h
o r v i s u acl o m m u n i c a t i oAn .l i t e r a 'lt r a n s l a t i o n ' o' tf h es o u r c e
sendsa message
to the receiver',
on the otherhand,is not possible:
the spatialrepresentaideaschanges
tion of verballyconceived
the ideasthemselves,
and viceversa.In the next
c h a p t ew
r e w i l l s h o wf u r t h e re x a m p l eosf t h i sp r o b l e m .
Notes
1 We are gratefulto the Danisha r t h i s t o r i a nL i s e M a r k f o r p o i n t i n go u t t h a t , i n t h e
'on its side',i.e.tilted by 90
p r e v i o ues d i t i o no f t h i s b o o k ,t h i s paintingwas reproduced
degrees.
2 Wewouldliketo thank BenteFogedMadsenfor someusefulcomments
that haveledto
improvements
of this diagram.

3 C o n c e p t u ar le p r e s e n t a t i o n s :
d e s i g n i n sg o c i a lc o n s t r u c t s
CLASSIFICATIO
NAL PROCESSES
In the previouschapterwe notedthat visualstructuresof representation
can either be
presenting
narrative,
unfoldingactionsand events,processes
of change,
transitoryspatial
arrangements,
participantsin termsof their moregeneralized
or conceptual,
representing
and moreor lessstableandtimelessessence/
in termsof class,or structureor meaning.
It is to the lattercategoryof representational
structuresthat we nowturn, beginning
processes.
with classificational
processes
Classificational
relateparticipantsto eachother
i n t e r m so f a ' l < i n do f ' r e l a t i o na, t a x o n o m ya: t l e a s to n es e t o f p a r t i c i p a n tws i l l p l a yt h e
role of Subordinafeswithrespectto at leastoneother participant,the Superordinafe.
We
havealreadycomeacrossan examplein the left-handpictureof figure2.1,wherethethree
- the axe,the basketandthe woodensword- wererepresented
participants
as 'species'
of
t h e s a m e ' g e n u sa' ,s a l l b e l o n g i ntgo t h e s a m eo v e r a r c h i ncga t e g o r yI n
. t h i se x a m p l e
the
overarching
categorywas not shownor named.Thestructurewasa CovertTaxonomy,
a
taxonomyin whichthe Superordinate
is inferredfrom suchsimilaritiesas the viewermay
perceive
to existbetween
the Subordinates,
or only indicatedin the accompanying
text,as
i n f i g u r e3 . 1 .
0 n ev i s u acl h a r a c t e r i s tiiscc r u c i ailn t h er e a l i z a t i oonf c o v e rtta x o n o m i etsh:ep r o p o s e d
e q u i v a l e n cbee t w e etnh e S u b o r d i n a t ei ssv i s u a l l yr e a l i z e db y a s y m m e t r i c ac lo m p o s i t i o n .
The Subordinates
are placedat equaldistancefrom eachother,giventhe samesizeand
the same orientationtowardsthe horizontaland vertical axes.To realizethe stable,
timelessnatureof the classification,
the participantsare often shownin a more or less
objective,
decontextualized
way.The backgroundis plain and neutral.Depthis reduced
or absent.The angle is frontal and objective.And frequentlythere are words inside
the picturespace.Thesefeatureswill be discussed
in a later chapterunderthe heading
'modality'.
pn
r o c e s s ed so n o t ,o f c o u r s es,i m p l yr e f f e c t ' r e a l ' , ' n a t u r a l ' c l a s s i f i c a t i o n s .
Classificatio
Forparticipants
to beput togetherin a syntagmwhichestablishes
means
the classification
judged
that theywere
to be members
of the sameclass,andto be readassuch.'Naturalizat i o n ' i s n o t n a t u r a lw
, h e t h e ri n i m a g e so r i n l a n g u a g eT.h e o r d e r i n gi n t h e i m a g ei t s e l f
producesthe relations.This makesit possiblefor the producerof an imageto classify
b a s k e t sa n d w e a p o n s( f r g u r e2 . I ) , o r z o o l o g i s t w
s ,i l d l i f ep h o t o g r a p h ear sn d A b o r i g i n a l
(figure3.1) as beingof the sameorder.
storytellers
Coverttaxonomiesare often usedin advertisements,
wherethe photographs
may,for
instance,
showarrangements
of bottlesthat representthe varietyof productsmarketed
undera brandname,or arrangements
of differentpeoplewho all usethe sameproduct.
Figure3.2 is a pagefrom Cosmopolitan
magazine.
What do thesewatcheshavein com,
mon?Theyall belongto the sameXposerange.

Conceptual representations

rig f.f cuide intertace(Microsoft,1994)

O t h e rt a x o n o m i essh o wa h i g h e d
r e g r e eo f ( e x p l i c i to) r d e r i n gi n c l u d ea S u p e r o r d i n a t e .
Theyrepresentand namethe Superordinate
within somekind of tree structure.In that
structurethe orientationis vertical,and the Superordinate
is placedaboveor belowthe
s a y b e r e a l i z e vd e r b a l l yv,i s u a l l yo,r b o t h
S u b o r d i n a t eass,i n f i g u r e3 . 2 . f h e p a r t i c i p a n tm
verballyandvisually,
but the processis alwaysvisual.
Taxonomies
do not haveto be represented
by formaldiagramswith simplelines;they
may be realizedin more realistfashion,for instance,
by an actualtree in a'family tree'.
0 v e r tt a x o n o m i easr e u s u a l l y ' c h a i n e ds 'o, t h a t t h e ' i n t e r m e d i a t e ' p a r t i c i p a (net s. 9 .t h e
' i n o r g a n i sc u b s t a n c e s ' a n d ' o r g asnui cb s t a n c e s ' o
f i fg u r e3 . 3 ) w i l l b e S u p e r o r d i n awt ei t h
respectto someof the other participants,
and Subordinate
with respectto others.To
indicatethis we will coin the term Interordinate.
In otherwords,overttaxonomieshave
levels,and participantsat the samelevelare represented
as being,in somesense,'ofthe
s a m ek i n d ' .
Treestructures,however,
are not only usedto realize'kind of' relations.'Reporting
d i a g r a m s 's, h o w i n gt h e h i e r a r c h i c aslt r u c t u r eo f c o m p a n i eas n d o t h e r o r g a n i z a t i o n s ,
and genealogical
or evolutionary
trees,usethe samestructure.This meansthat visual
grammarconflates,
or at leastrepresents
as verycloselyrelated,what would,in language,
be expressed
by differentmeans.Conceptualclassification
is represented
by the same

Conceptual
representations. 8),

r F r r ' " & p * { f ' s A r , { $ c d F v . " . 1 ' ; . . s { ;fi i y s

.{.r.'",;s :r ; *tr i:?}J;1s1f.a.6+r I I r"5i1'a+lsr

fig f,Z Xposerange (Cosmopafifau,November2O0l: g4)

82 .

Conceptual representations

Sourcesof siqns
I

Inorganic substances

Natural

Manufactuied

organic substances

Extraterrestrial

Terrcstrial

Sp!echlesscreatures

Homo sapiens

I
organisms

componetrts

components

ofgdisms

fis f.: Sourcesof signs(Eco,l976b:177)

structureas socialhierarchy;
as similarto
that is, the more generalidea is represented
greaterpower.As VirginiaWoolfhassaid,'General
ideas.'
ideasare alwaysalsoGenerals'
Furfher,hierarchies
of concepts
and hierarchies
of socialpowerare represented
as similar
is represented
as
to genealogies.
In otherwords,the identityof an individual(or a species)
b e i n g ' s u b o r d i n a t e 'ittos ' o r i g i n s ' o r ' a n c e s t o ri sn 't h e s a m ew a y a s s p e c i f i c o n c e p tasr e
subordinated
and loweremployees
or localbodies
to moregeneraland abstractconcepts,
to managers
or centralorgans.
Diagrammatic
of the tree may
tree structurescan take differentforms.The branches
l r o b l i q u es, t r a i g h to r c u r v e da, n d s o o n ( s e ef i g u r e3 . 4 ) . O b l i q u eb r a n c h e s
b e p a r a l l eo

tre structure with

parallel branches

-\

tree slructure wilh

cuded bianches

/n\
AA AA
AAAA

piSf.l Treestructureswith parallel(straight)0bliqueand curyedbranchet

Conceptualrepresentations. 83

so that moreof
abstractsomewhatlessfrom the shapeof the treethan parallel branches,
the symbolicmeaningof the tree can be preserved.
Hencethey are commonin contexts
e rsi n
w h e r ea s e n s e
o f ' g e n e r a t i o n ' a n d ' g r o w t hc' iosn n o t e da,sf o r i n s t a n cien g e n e a l o g i o
whichpositsthat'surface
the diagramsusedin'generativegramrnar',
a form of linguistics
structures'.
structures'aregenerated
from (possiblyinnateand maybeuniversal)'deep
The contrastbetweenstraightand curvedbranchesis perhapssimilarto that between
participantsrepresented
as boxesand participantsrepresentedas circles or ovals
( s e ec h a p t e 2r ) , a c o n t r a sbt e t w e etnh e ' m e c h a n i c a l ' a n d ' t e c h n o l o g i c a lt'haen'dn a t u r a l '
a n d' o r g a n i c ' .
are obliquethe
Manytreediagramsare inverted('bottom-up'),andwhenthe branches
with
overallshapewill tend towardsthat of a pyramid.Such structuresare concerned
hierarchyand hierarchicaldifferenceratherthan with clarity about levels:a readingof
are inverted.
Sometimes
levelsis possible,
but not readilyfacilitated.Not all trees,however,
the specificor,in the caseof genealogies
and evolutionary
trees,the present,is placedon
t h et o p ,d e p i c t i n gf o, r i n s t a n c e
h ,u m a n k i nadst h ep i n n a c loef e v o l u t i orna t h e tr h a na sb e i n g
foreverdominated
by its lowlyorigins.
participantsin termsof their placein a
Althoughclassificational
structuresrepresent
themdo not always
staticorderithe verballabelsand explanations
whichmayaccompany
do so.Theterm 'reportingdiagram',for instance,
usesan activeprocess('report')rather
t h a na s t a t i co n es u c ha s ' i s s u b o r d i n a t o
e ' . G e n e a l o g iaensde v o l u t i o n a tr rye e ss, i m i l a r l y ,
m a y b e g l o s s e db y v e r b sl i k e ' g e n e r a t e s ' , ' e v o l vienst o ' , ' b e g e t s a
' ,n d s o o n . V i s u a l l y ,
can
however,
a hierarchicalorder is signified,a system.Thusthe visual representation
b l u r t h e b o u n d a r i ebse t w e e n
t h e d y n a m i ca n d t h e s t a t i c .I s t h e d y n a m i ci n r e a l i t yt h e
instantiationor enactmentof an underlyingsystem?0r is the static the systemization
and objectification
of a dynamicand ever-changing
reality?Suchquestionsbecomedifbetweenthe schemaand the
ficult to answerin a modewhich haserasedthe boundaries
b lu e n r i n t .
A s i m i l a rb l u r r i n gm a yo c c u rb e t w e e a
n n a l y t i c a(l' p a r t o f ' ) a n dc l a s s i f i c a t i o n(a' kl i n d
o f ' ) p r o c e s s eTsh. e ' s t a c kd i a g r a mi'n f i g u r e3 . 5 c o u l db e c a l l e dc l a s s i f i c a t i o naanl ,d i s i n
fact so renderedin the accompanying
text:'to the Latin word /mus/ correspondtwo
d i f f e r e ntth i n g sw h i c hw e s h a l lc a l l x , a n dx r ' ( E c o ,r 9 7 6 b : 7 8 ) .I n o t h e rw o r d sa, ' m o u s e '
i s a k i n do f ' m u s ' .B u t i t c a na l s ob e s e e na s a n a l y t i c a0l .n ec a na l s os a yt h a t t h e m e a n i n g
'mouse'is part of the meaningof 'mus'.And,mostimportantly,
Ecohaschosenthe form of
the analyticaI diagram.

Mous
Mus
Rat

fis f.l Semantictield diagram(Eco,l976b:78)

84 .

Conceptual representations

Finally,classificational
diagramsmay be rotatedthroughninetydegreesso that their
main orientationis alongthe horizontalaxis.Theyhavethen the orientationtypical of
narrativediagrams,
and hencea dynamicconnotation.
But they retainthe structureof the
classificational
diagram.Theystill representthe relationbetweenthe participantsas a
system.Features
from differenttypesof structureare abstracted
andrecombined
to create
patternsthat are ambiguously
in betweenthe dynamicand the conceptual.
When such
diagramsacquirearrows- as, for example,in systemnetworks(seefigure 3.8) - they
becomein fact dynamicand narrative.Yettheystill movefrom the generalto the specific,
in contrast(for example)to flowcharts.
Taxonomies
The one
and ffowchartsclearlyprovidetwo differentkindsof knowledge.
represents
the world in termsof a hierarchical
order.Its main concernis the rankingof
phenomena
from the perspective
of a singleunifyingterm,be it that of the originof things,
generalization,
the most generalizing
or that of the highestpower.The other describes
the world in termsof an activelypursuedprocesswith a clear beginning
and an end (or
' i n p u t ' a n d' o u t p u t ' , ' s o u r c e ' a n' d e s t i n a t i o n ' , ' r am
wa t e r i a l s ' a n'df i n i s h e d
p r o d u c t ' )I.t
progression
hasa sequential
and is goal-oriented.
And,as we havenotedalready,system
networkssuchas we usein this bookattemptto combinethe two perspectives.
R e c e n t layn o t h e kr i n do f d i a g r a mh a sb e g u nt o g a i na s c e n d a n -c et h e ' n e t w o r k 'N
. etworksseekto showthe multipleinterconnections
betweenparticipants.
Any participantin
a network('node')can form an entry-pointfrom which its environment
can be explored,
and the vectorsor lines('links') betweentheseparticipantscan take on manydifferent
v a l u e st ,h e v a l u eo f s i g n i f i c a t i o(n' a m e a n sb ' ) , o f c o m b i n a t i o(n' a g o e sw i t h b ' ) , o f c o m position('a containsb'): the essence
of the link between
two participantsis that theyare,
in somesense,nextto eachother,or closeto eachother,associated
with eachother.To
demonstrate
with a linguisticexample,
the difference
a taxonomywouldshow,for instance,
a hierarchy
of words,a 'ffowchart',for instance,
a way of generating
a clauseby followinga
precisesequence
of instructions',
anda networkmightshowlhe collocation
of words- the
otherwordswith which any givenword typicallycombines/
regardless
of the structural
relationsbetween
the words(seefigure3.6).
Figure3.7 showsa'linear' (ffowchart)and'non-linear'(network)representation
from
program.Thenetwork,its authorssay/allows
an articleon a'writer's assistant'computer
t h e w r i t e r ' t o f o r m i d e a si n t o a n a s s o c i a t i vnee t w o r k ' ( S h a r p l easn d P e m b e r t o nI 9, 9 2 i
2 2 ) . T h ep r i n c i p l e
b e h i n ds u c hn e t w o r kcsl e a r l yr e l a t e tso t h e i d e ao f t h e ' n o n - l i n e a r ' t e x t
t o w h i c hw e h a v ea l l u d e di n o u rd i s c u s s i oonf D i c kB r u n a ' s0 n M y W a l k( f i g u r e1 . 2 )a n dt o
whichwe will returnin greaterdetail in chapter6. In discussing
a pagefrom Brunawe
stressed
that suchpageson the onehandprovidethe readerwith manychoices,
manypaths
to follow,but on the other hand tend to obscurethe fact that the range of choices
is ultimatelypre-designed
and limited.As such,networksare, in the end,just as much
modelledon formsof socialorganization
as taxonomies
and ffowcharts.
Thetaxonomyis
modelledon a static,hierarchical
organizationin which everythinghas its pre-ordained
placein a grandschemeunifiedby a singlesourceof authority.Theflowchartis modelled
prescribed,
on the principleof authoritatively
structured,goal-oriented
activity.The network is modelledon a form of socialorganization
whichis a vastlabyrinthof intersecting

Concept uaI representati ons

affection

Pleasure

physical

psychological

pleasure

pleasure

(etc.)

(D

displeasure

\ (etc

r--+---r

contentment

joy

delight

fis f,O Taxonomy,flowchart

and network

localrelationsin whicheachnodeis relatedin manydifferentwaysto othernodesin its

immediateenvironment,
but in which it is difficult,if not impossible,toform a coherent
viewof the whole.Perhapsit is not accidental
that this kind of networkis comingto the
fore in an age of increasingsocialfragmentationand regionalization.
But regionsare
n e v e r t h e l ecsosn n e c t etdo t h e w h o l ea n dt h e n e t w o r km o d e lm a y o b s c u r teh e g l o b a l i z i n g
t e n d e n c i ew
s ,h i c ha r e a l s oa n d s i m u l t a n e o u sal tyw o r k i n c o n t e m p o r a rsyo c i e t yT. h i s i s
recognizedby those who pioneerthe applicationof this mode of representation,
for
instancein connection
with hypertexts,
whichare alsonetworks:
Readers
who browsehypertextnetworksbecome'lost',unawareof wherethey are
in relationto the document,and thus unableto achievea senseof text, i.e. an

86

Concept uaI representati ons

lrtii:ri !4 iJl iXf;r?;i

il

fig f.Z Network(from Sharplesand Pemberton,1992)

to startingat
Thisis analogous
of the document.
feelfor the substance
intellectual
f r o m o n e s i m i l a ra s s o c i a t i vweo r d
a p a r t i c u l a rw o r d i n a t h e s a u r uasn d p a s s i n g
to another.Soon it is likelythat the readerwould be examininga word that is
differentfrom the originalword,with no notionof any meaningto the
completely
that hasjust takenplace.
derivation
(Ghaouiet al.,1992: 7I0)
of networkguru l(evinl(elly,as reportedby Jim McClellanin the
In the pronouncements
of the networkbecomeevenclearer:
)bserver,thepoliticalimplications
Its sure,regularorbits
The imageof the atom . . . stoodfor powerand knowledge.
s p a c e rse p r e s e n t e d ' l a w - a b i dsionlga rs y s t e mos f e n e r g y ' u n d ecre n t r a l
a n dd e f i n e d
direction.In contrastthe Net - a tangleof apparentdisorder was an icon of no
(
o
r
a l l b e g i n n i nagl,l c e n t r ea, l l e n d ) .I t w a st h e e m b l e m
b e g i n n i n ng o, c e n t r en, o e n d
the banner
of the complexlogicof natureand computers;
of our newunderstanding
had
beenthe
The atom
organizedthemselves.
of systemswhich,in somesenses,
(
t
h
e
w
o
u
l db e t h e
A t o m A g e ) ,h e c o n c l u d e db ,u t t h e N e t
i c o no f t h e 2 6 i h c e n t u r y
of the comingnetworkcultureof the 2lst.
archetype
(McClellan,0bserverLife Magazine,
1994,p. 62)
21 September
s re
i n f i g u r e3 . 8 w e s u m m a r i zteh e d i s t i n c t i o nwse h a v em a d ei n t h i s s e c t i o nN. e t w o r k a
T
n o t i n c l u d e di n t h e s u m m a r ya s t h e y a r e ' a n a l y t i c a l ' r a t h etrh a n ' c l a s s i f i c a t i o n a l h' .e
d i f f e r e n cbee t w e e n ' s i n g l e - ' a n d ' m u l t i - l e v e l l emda' i rsk e db y t h e a b s e n coer p r e s e n coef
'lnterordinates'.
abovehas,we hope,madeit clearthat we seethesedistinc0ur discussion
tionsas toolswith whichto describevisualstructuresratherthan that specific,concrete
anduniquelyin termsof anyoneof
exhaustively
alwaysbedescribed
visualscannecessarily
o u rc a t e g o n e s .

Conceptua I representat i ons

Covert taxonomy

Overttaxonomy
-

Single-levelled

Ntulti-leve|ed

fig f.g Chssificationat

imagestructures

REALIZATIONS
Coverttaxonomy

Single-leveIIed overt taxonomy

M ulti- |eve| |ed overt taxonomy

A s e to f p a r t i c i p a n t(s, s u b o r d i n a t e si s, )
d i s t r i b u t esdy m m e t r i c a lal yc r o s tsh e p i c t u r e
space,at equaldistancefrom eachother,
equalin size,and orientedtowardsthe
verticaland horizontalaxesin the same
way.
A participant('Superordinate,)
is
c0nnected
to two or moreotherparticipants
( ' S u b o r d i n a t e tsh' )r o u g ha t r e es t r u c t u r e
with two levelsonly.
A p a r t i c i p a n(t' S u p e r o r d i n a t e
i s, )
connected
to otherparticipants
througha
tree structurewith morethantwo levels.
Theparticipants
whichoccupyintermediate
levelsare interordinates,
whilethosewhich
o c c u p tyh e l o w e slte v e (l i f t h e
S u p e r o r d i n aitseo n t o p ) o r t h e h i g h e slte v e l
(if the Superordinate
is at the bottom)are
Subordinates.

A N A L Y T I C A LP R O C E S S E S
Analytical processesrelate participantsin terms of a part-whole
structure.They
involvetwo kindsof participants:oneCarrier(thewhole) and any
numberof possessive
Attributes(the parts).We havealreadygivenan exampleof an analytical
structure,the
pictureof the Antarcticexplorer(figure2.4), which
analysedthe explorerin terms of
his'outfit'. Fashionshots,too, are analytical.Like the pictureof
the Antarcticexplorer,
they clearlydisplaythe parts of an 'outfit', and label both the
Carrier('easy-wearing,
inexpensive
cottonsteamedwith the rightaccessories'seefigure3.9) andthe possessive
Attributes('Laura Ashleytrenchcoat,Stuart Memberysweater,Benettonjodhpurs,),
albeit in a caption, rather than insidethe picturespace,and in the
ffowery languageof
fashion
w r i t i n gw h i c hh a sb e e nd e s c r i b esdow e l l b y B a r t h e s( r 9 6 7 b ) .

a7

ntations
I represe
Conceptua

C0ttons(yogue,November1987)
fis f.S Easy-wearing

Differentas they may seemat first sight,mapshavethe samestructure:there is a

Attributes,for instance'the
and there are Possessive
Carrier,for instance'Australia',
picture
spaceor in a legendor
statesof Australia',and bothare labelled,eitherinsidethe
of what seemsto be the samecarrler'
caption.Mapsmay providequitedistinctanalyses
altitude,etc.,whileothers
featuressuchas waterways/
Somemapsfocuson geographical
s .n a l y s i sa l w a y si n v o l v esse l e c t i o nS.o m e
l n d p o l i t i c a bl o u n d a r i eA
c o n c e n t r a toen s o c i a a
of the Carrierare singledout as criterialin the givencontext
attributesor characteristics
and irrelevant'
treatedas non-essential
orlgeneralltwhileothersare ignored,
Thedifferencebetweenthe mapand the fashionshot liesnot in their ideationalbut in
structures- for instance,in their modality(seechapter5)' Many
their interpersonal
analyticalvisualshavelow modality,from the naturalisticpoint of view.Too much life, o u l dd i s t r a c ft r o m t h e i ra n a l y t i c apl u r p o s e0' n l yt h e e s s e n t i a l
l i k e n e stso, o m u c hd e t a i lw
Attributesare shown/and for this reasondrawingsof various
featuresof the possessive
or highlydetailedartwork'
are oftenpreferredoverphotographs
of schematization
degrees
of llghi
of depthis reducedor absent,as is the detailedrepresentation
The representation
to a
restricted
is
Colour,if it is usedat all,
and of subtletonal distinctions.
and shadow,
as
participants,
such
to distinguish
reducedpalette,or usedconventionallyfor instance,
sketched
only
or
groups/or landforms.Backgroundis left out,
differentsocioeconomic
Attributesare labelled.Notethat the arrowsin figure2.4 do
in lighly.And the Possessive
but a relationof identitybetweena verbaland a visual
not realizenarrativeprocesses

ConceptuaI representat i ons

realizationof the samePossessive

particularlyposedphotoAttribute.Yet photographs,
graphs,can alsobe analytical,as in the caseof fashionshots,or of advertisements
which
g i v ea d e t a i l e d e p i c t i oonf t h ea d v e r t i s epdr o d u c to, r o f p r e s s ' m u g s h o t s p
' oof l i t i c i a nasn d
persons.
othernewsworthy
The schoolsocialstudiestextbookfrom whichwe took figure
2.4 shows- on the pageoppositeto that showingthe drawingin figure2.4 -the picture
r e p r o d u c ei n
d f i g u r e3 . 1 0 .
It is an analyticalpicture;there is neithera vector (narrativeprocess)nor compositionalsymmetryand/oratree structure(classificational
process).ltservesto identify
a Carrierand to allow viewersto scrutinizethis Carrier'sPossessive
Attributes.Some
minordegreeof loweredmodalityis alsopresent:
the background
is plain,andthefact that
the pictureis posedaddsfurther artificiality.Yet,althoughit is analytical,its purposeis
more interactionaland emotivethan representational.
The interactionalsystemof the
gazedominates:
participantsdirectlyaddresses
the gazeof represented
the viewersand
s o e s t a b l i s h easn i m a g i n a r rye l a t i o nw i t h t h e m ,w h i l em o r es c h e m a t iacn a l y t i c apl i c t u r e s
i n v i t ei m p e r s o n adl ,e t a c h e sdc r u t i n yA. s i m i l a ra r g u m e nct a n b e m a d ea b o u ta d v e r t i s e mentsshowingthe advertisedproduct- the overall impressionof an abundanceof
p a r t s( o r i n g r e d i e n tosr, v a r i e t i e o
s f t h e p r o d u c t )o, r t h e a l l u r i n gs e n s o r q
y u a l i t yo f t h e
a d v e r t i s e pd r o d u c ta s a w h o l e( t h e s t r e a m l i n esdh e e no f t h e c a r t h e v i v i dc o l o u ra n d

fiq f.fO Sir DouglasMawson(0akleyetal.,1985)

9O .

rePresentations
Conceptual

over more dispassionate

of a cannedsoup)take precedence
textureof the ingredients
Attributes. Persuasionis foregrounded,instructionand
scrutiny of the Possessive
3.10
expositioa
n r e b a c k g r o u n d eTdh. e t e x t b o o ki n c l u d e sb o t h f i g u r e 2 . 4 a n d f i g u r e
but
it seeksnot onlyto teachchildrenobjectivefactsaboutAntarcticexploration
because
then
this
hero.Until recentlyr
alsoto makethememotionallyidentifywith an adventurous
their readers
Advancedtextbooksaddressed
in the later yearsof education.
decreased
as
as'no longerneedingpictures',as havingbeenweanedoff everydaynaturalism,and
learning
higher
attitudethat characterized
havingacquiredthe abstractand impersonal
the tendencyis now for the
and higherart appreciationin Westernculture.However,
to enterthe textbooksof lateryears,as well throughimageand through
interpersonal
of
writing.In the languageof the seniorsecondaryschooltextbook,as in the languaqe
impersonal
passive
voicehad beennormal,realizinga more
scienceand bureaucracy,the
form of address:
anddetached
mapsandstreetdirectories.
from topographic
Theseoutlinesmaybe abstracted
a
language,'you'was
textbook,asalsoin advertising
of the primary-school
In the language
andthe readerwasalwaysdirectlyaddressed:
keyword,
how wouldyou
Hereis a pictureof Antarctica.If you landedherein a spaceship,
d e s c r i bteh i sP l a c e ?
many domainswhich formerly
Today,the personaland the informalenter increasingly
andformalmodesof address,verballyaswelI asvisually'
by impersonal
werecharacterized
remainalmost as objectiveand detachedas traditional
0f course,some photographs
of
and,for instance,
diagramsand maps.This is still true of manyscientificphotographs
background
their top-downangleensuresthe absenceof depthand
aerialphotographs:
on the heightfrom whichthey are taken,detail is moreor lessremoved.
and,depending
with
B u t , a s i n o u r p r i m a r y - s c h otoelx t b o o ke x a m p l et /h e yw i l l n o w o f t e nb e c o m b i n e d
scientists
presentations
of
less formal pictures,even in the PowerPointconference
( R o w l e y - J o l i v2e0t ,0 Q .
Abstract art may also be analytical.The structureof Theo van Doesburg'sPure
Painting,for instance(figure3.11), is like the structureof a map' It analysesreality
of differentsizeand
Attributes,highlyabstractones:rectangles
in terms of possessive
Attributes.It leavesit
colour.But it doesnot labeleitherthe Carrieror the Possessive
up to the viewerto do so,and as a resultthe paintingcan be readin manydifferentways'
as the map of the moderncity (greenfor recreational
We couldseeit, quite concretely,
areas,yellowfor housing,red for industrialareas,and so on) figure3.12 ls in fact a
map of a city,and looksquitesimilar,if we ignorethe writing.we couldalsoseeit, more
(greenfor contemplation,
as a map of desirablehumanqualitiesor activities
abstractly,
point is,
red for passion,etc.), and of the spacethey shouldoccupyin our lives.The
and
readings,
boththesethings,and manymore.It is opento many
the paintingrepresents
unleashed
be
cannot
that constitutesits powerto shapereality,a powerwhich,however,

i ons
Conceptual representat

I
I

fig f.ff

(from Jaff6,1967)
PurePainting(Theovan Doesburg,1920)

until the boxesare givenconcretereference,

so that the schemacan be turned into a
bIu e p r i n t .
Lowerednaturalisticmodality,however,
of analytic
is not a definingcharacteristic
v i s u a l sM
. o d a l i t yf o r m s a s e p a r a t e( i n t e r a c t i o n asl )y s t e mw h i c h i s p r e s e n it n v i s u a l s
simultaneously
with the kindsof structurewe aredescribing
in thischapter.
At mostwe can
saythat, in specificsocialcontexts(cartography,
educationat differentlevels,advertising
fashion),there is a tendencyfor certain modalitychoicesto go togetherwith certain
representational
of the
choices,certain kindsof processes.
The definingcharacteristic
analyticalprocess
is in fact a'default'one.It liesin the absence
of vectorsandthe absence
of compositional
symmetryand/ortree structures,
of the features
and also in the absence
t h a t m a r k t h e s y m b o l i cp r o c e s s ewse w i l l d e s c r i b e
i n t h e n e x ts e c t i o nT. h e r ea r e m o r e
positiveand specificcharacteristics,
but these pertainto specifickinds of analytical
process.
As a whole,the analyticalprocessis the usual,the 'unmarked',
andthereforealso
- a visual'this is'.
the most elementaryoption in the visualsystemof representation

91

Concept uaI representati ons

92

WGww

wrc
w
ffi

ffiWffi
iWuffiffiffiW
ffiffiffiffiffi

Wwffiffiffiffiffi}rcry]I 1

13r::irf;*y*

ffi*u.,."

W**.,,'",,.,",,o

I'.'.**.*q4::.:'i.ni'i,"rnl.r1

ffi81*.o**o-*.*0"*, n.,''-,"'

m
t:

fig Z.tZ Map of the centralbusiness

district of Melbourne(Paskand Bryant,1982)

Experimental
studiesof the productionof drawingswould seemto bear this out: the
principaltask that drawersmustmasteris the representation
of objectsin termsof their
l u r p o s ef o r w h i c h n o n - s p e c i a l i s t s
m i n i m a ld e f i n i n gc h a r a c t e r i s t i casn, d t h e p r i n c i p a p
actually use drawing in everydaylife is the productionof pragmaticallymotivated
'descriptions',
sketchesof localities,clothesand hairstyles,mechanicaldevices,and so
on, as well as the productionof doodles,
which often are also analytical(van Sommers,
I9B4 234ff.).
1 Unstructuredanalyticalprocesses
areunstructured;that
is,theyshowusthe Possessive
Someanalyticalprocesses
Attributes
of the Carrier,but not the Carrieritself,theyshowusthe parts,but not the way the parts
fit togetherto makeup a whole.Thepagereproduced
in figure3.).3,Iorinstance,
is a kind
of unstructured
map,accuratein scale,but without any visualindicationof the location
of the Possessive
Attributesrelativeto eachother.We seethe parts of the garment,but
n o t t h e w a y t h e ya r e t o b e a s s e m b l e-d a l t h o u g hv e r b a l a b e l s( ' c o a tf r o n t ' , ' b i k i n ti o p
back',etc.)providean indication.
Thepagedoesalsoincludea photograph
of the finished
garment.This photograph,however,
would have to be seenas a separateanalytical
structure,
with highernaturalisticmodalitythan the patternitself.The magazineusesthe

representations. 93
Conceptual

' ' ' ' - ' - ' .

" '
" l
' .
t " . " :
" '::
_.
:1 :" :.- .;. I :::

fiq f.ff

Escape(Austnlian Women'sWeekl!,December1987)

samestrategyas the socialstudiestextbool<:

onepictureto enticeyou/andanotherto give
youa m0reobjectiveanalysisof the Carrier.
E s p e c i a lw
l yh e nt h e C a r r i e ri s a b s t r a c t ,h e r em a yn o t b e a s i n g l ep r i n c i p l feo r t h ew a y
i n w h i c hP o s s e s s iAv tet r i b u t e s h o u l db ea s s e m b l eTdh. eC a r r i e cr a n n o bt ev i s u a l i z ei d
nan
assembled
an
state,andanyarrangement
of the Possessive
Attributesis thereforepossible:
u n s t r u c t u r eadn a l y t i c apl r o c e siss l i k ea m o r eo r l e s su n o r d e r eldi s t .T h i sc a n b e s e e nf,o r
i n s t a n c ei n, a d v e r t i s e m e nwths i c hd i s p l a ya l l t h e p a r t st h a t m a k eu p t h ee n g i n e
of acar,or

Conceptual representati ons

in orderto impressthe viewer

the full rangeof productsmanufactured
by an advertiser,
w i t h t h e i rs h e ear b u n d a n c e .
2 Temporalanalyticalprocesses
so far represent
the thingstheyrepresent
as objects.They
Theexamples
we havediscussed
whichseemsto
focuson spatiality.But thereis alsothe categoryof the timeline,a process
positionbetween
Timelines
involve
occupyan intermediate
the narrativeandthe analytical.
the temporaldimension,
and this suggestsnarrative.Yet these linesare not vectorial
theyanalyseit into
and,ratherthan representing
historyas a gradualunfoldingof events,
stageswhichcanthenbetreatedas
successive
stageswith fixedandstablecharacteristics,
thoughtheywerethings.Thetypicalevolutiontimeline,showinga seriesof figuresstarting
with an ape 0n the left and endingwith homo sapienson the right,would be analytical
in this sense,'saying':'the evolutionof humankindconsistsof the "Ape" stage,the
stage(etc.)',with 'the evolutionof humankind'
"Apeman"stage,the "Australopithecus"
Attributes.
as Carrier,
andthe stagesas Possessive
isthat theyare realizedby
Theessential
characteristic
of temporalanalyticalprocesses
t i m e l i n e st :h e p a r t i c i p a n t (ss o m e t i m ews h o l es t r u c t u r e s' s, c e n e s 'a) r e a r r a n g e do n a n
a c t u a lo r i m a g i n a rlyi n e u, s u a l l yh o r i z o n t asl ,o m e t i m evse r t i c a lT. h et i m e l i n em a yb et o p o graphical,
the participantsin the rightsequence,
drawnto scale,or topological,
assembling
b u t n o td r a w i n gt h et i m e i n t e r v a l s ' t soc a l e ' .
e l l k i n d so f g e o m e t r i c a
s yl m b o l i s m .
T i m e l i n ense e dn o tb es t r a i g h ta, n dt h e ym a yi n v o l v a
F i g u r e3 . I 4 , t a k e nf r o m a S w e d i s jhu n i o rh i g h - s c h o h
o il s t o r yt e x t b o o ki,s a p a r t i c u l a r l y
interesting
example.Heretime doesnot progresslinearly,but takesturns,goesdownhill,
movestowardsthe viewer,and changescolou[ from a crystallineblueto a drab brown
( t h ec o l o u rc o d ee m p l o y e bd y t h e s e r i e so f f o u r b o o k so f w h i c ht h e b o o kc o n t a i n i ntgh i s
i l l u s t r a t i o ins p a r t ,h a sb l u ea s t h e c o l o u ro f h i s t o r ya n d b r o w na s t h e c o l o u ro f s o c i a l
studies).The resultis a ratherethnocentric
and patriarchalstorywhichcouldbe saidto
modernand egalitarianwelfarestate,a deeplongreveal,behindthe surfaceof Sweden's
ingfor the primitiveuncertainlife of the nomadichunter.
When'M an' first appeared
on the
horizonof time,sothisstorygoes,hewasa hunteranda makerof tools.Hethenacquireda
wife,but she,the womanof 'homoerectus',was not yet 'erectus'herselfand,whereher
manfixedhisgazeuponthe future,shelookedbacktowardsthe past.Then'Man'invented
fire - a maleachievement,
for womanis not shownto be part of it, not evenas an admiring
onlooker.Next,he beganto coverhis bodyand begota child.But at this point,the point
wherehistorytakesa backwardturn, a curiousinversionoccurred.It was now he who
hewho lookedbackwards
crouched
and shewhowas'erectLrs',
and shewhofixedhergaze
upona futurewhich,however,
was no longerthere,as time hadalreadyturnedbackwards.
B u ta l l w a sn o t l o s t .T h eu n i o no f t h et w o p r o v i d eadv e r y ' e r e c tv' ,e r ym a l ea n dv e r yN o r d i c
homosapiensas the outcomeof evolution,
as'our'ancestor,hereplacedin the centreof
the composition
andcloseto the viewer.
ThisstoryaboutSwedishidentityis told by visual
meansonly,and it illustrates
a text whichis told,not as Swedishhistory,but as a historyof
t h e w o r l d .T h et w i s t i n ga n dt u r n i n go f t h et i m e l i n ei n f i g u r e3 . 1 4 d o e so, f c o u r s eg, i v ei t a

ConceptuaI representations

l*

.H

- *ffi

ffiffiffi
@

-&ssa*lffiqqr

Wqiftir*r4!!&ltsi!&

q *aw
rs4
* * #4+'*F;
jk;s.4*
$d{.1
" #&
sS
ffi;6htu"tutrF,

@
*{"!

iisq!{i!r4ejd;ii*ii&etd6
* a iR
s- L*i*lw*:&:*a'drd#W#q@'S
dts
4'. 'h
eFe4;"
.:q3?@ 4aq"e.!!@?:

dlb
bFr

"i

{i;!i!{saii,iid.istd;.'l

!M

'

,r: i

r' _,.. r"

.:

:r'

rig f.f+ Evolution(ohman,1989)

d y n a m i cq u a l i t ya n dr e m i n d o
s f t h e h e l i xi n f i g u r e2 . 2 4 . T h ec a t e g o r i eosf v i s u a lg r a m m a r
do not haveclear-cutedges,
andspecificrepresentations
canmergetwo or morestructures
- for instance,
the narrativeandthe analytical.

3 Exhaustiveand inclusiveanalyticalprocesses
(Spatial)structuredanalyticalprocesses
canbe exhaustive,that
is,theycan exhaustively
represent
the Possessive
Attributesof a Carrier,so that all of the Carrieris accounted
for,
all of its spacetakenup by Possessive
Attributes.To put it in anotherway,in an exhaustive
analyticalstructurethere is assembly:
the Possessive
Attributesare joinedtogetherto
makeup a complexshape.Structuredanalyticalprocesses
can also be inclusive,
that is,
they can showus only someof the Possessive
Attributesof a Carrier,leavingmuch of
the Carrierunaccounted
for, as blank spacenot taken up by Possessive
Attributes.The
c i r c l e so n t h e l e f t a n dr i g h to f S c h r a m m 'tsh i r dc o m m u n i c a t i omno d e l( f i g u r e 2 . 2 2f)o r m
e x h a u s t i vaen a l y t i c asl t r u c t u r e se,m b e d d eidn t h e l a r g e rn a r r a t i v se t r u c t u r et :h e ' s e n d e r '
andthe 'Receiver'are hereanalysed
as beingmadeup out of the samethreecomponents
( ' E n c o d e r ' , ' l n t e r p r e t e r ' a' D
ne
d c o d e r 'a) ,s s e m b l ei nd d i f f e r e nwt a y sa, n de v e r yp a r t o f t h e
spaceof the circlesis coveredby Possessive
Attributes.Thepointis not,of course,
that the
a n a l y s i iss i n f a c t e x h a u s t i v e . ' S e n d e r s ' a n d ' R e c e im
ve
a rysw,e l l h a v eo t h e rc o m p o n e n t s
besidethesethree.The point is that in the analysisthe world is treatedas thoughit is
exhaustively
represented,
as thoughthe Carriershavethesemajor components
and no

96 .

Conceptual representations

others.Just as we haveto assumethat a map showingthe statesof Australiashowsus

a//the statesof Australia,anda pictureof the outfit of an Antarcticexplorera//the parts
of
of this outfit, so we haveto assumethat this diagramshowsus all the components
t h e ' S e n d e r ' a nt dh e ' R e c e i v e r ' . c0 of u r s e , ' e n c o d i n g ' , ' i n t e r p r e t i n g ' a n d ' d e c o d i n g ' m i g h
However,
Schrammhas represented
be seenas activitiesrather than as components.
as though'Senders'and 'Receivers'
them hereas thoughthey were parts of a machine,
As sooftenin diagrams,
doingshavebeen
havethese
elements,
or consistoI theseelements.
turnedintothings,spatialized
andobjectified.
Attributesdo not exhaustively
divide
In inclusive
the Possessive
analyticalstructures,
andsotakeup part of its
up the spaceof the Carrier.
Theyarecontainedwithinthe Carrier,
Thisstructurealsoentailsthe
space,but not all - otherpartsare left blank,unanalysed.
possibilityof partial inclusionand exclusionIn figure 3.15, A and B are Possessive
Attribute
Attributesof the Carrierformedby the largestcircle,C is a Partial Possessive
(in part belongingto the Carrier,in part outsideit), and D is excluded,hencenot a
Possessive
Attribute.In other words,where exhaustivestructuresare formed by the
weldingtogetherof Possessive
Attributes,inclusivestructuresare formedby the full or
partial overlappingof the participants.And the structure is recursive'.
a Possessive
Attributes,as in the casewith B in
Attributecan becomethe Carrierof other Possessive
fi9ure3.15: it is a Possessive
Attributewith respectto the largercircle,but a Carrierwith
a t w o r k i n R i l e ya n d R i l e y ' sc o m m u n i c a t i omno d e l
r e s p e ct to E . W e c a ns e et h i ss t r u c t u r e
(figure2.5). It doesnot tell us that 'over-allsocialsystems'exhaustively
consistof two
'larger socialstructures'and 'larger socialstructures'of two and not more than two
' p r i m a r yg r o u p s 'b, u t t h a t ' o v e r - a lsl o c i a sl y s t e m s ' c o n t a(i bn e s i d teh e m a n yo t h e rt h i n g s
t h e y m a y a l s o c o n t a i n )a t l e a s tt w o ' l a r g e r s o c i a ls t r u c t u r e sa' ,n d t h a t ' l a r g e r s o c i a l
s t r u c t u r e s ' c o n t aai tn l e a s t w o ' p r i m a r yg r o u p s 'I.n d i v i d u a sl su c ha s ' C ' a n d ' R ' a r e
partial Possessive
Attributes of the CarrierlAttribute'larger social structure' and
f u l l y P o s s e s s iA
ve
t t r i b u t e so f t h e C a r r i e r ' o v e r - asl lo c i a sl y s t e m 'T. o g i v ea g l o s so n t h i s
s t r u c t u r ea, c c o r d i n g
t o R i l e ya n d R i l e y i, n d i v i d u a lcsa n n o te s c a p et h e ' o v e r - a l ls o c i a l

fi9 f.fS Inclusiveanalyticalstructures

representations' 97
Conceptual

of'largersocial
from the constraints
but theycan/at leastin part,freethemselves
system//
structures'.
Manymapshavea similarstructure.A mapof a state0r nationwhichshowscitiesand
towns,for instance,is not read as meaningthat everycity and everytown has been
osn
se
i n c l u d e d , o r t h a t t h e s t a t e o r n a t i o n c o n t aPi n
os s i v e A t t r i b u t e s o t h e r t h a n c i t i e s a n d
(the
citiesandtownsof that state
major)
some
are
that these
towns.It is readas meaning
to you,the
and relevance
interest
of
and
towns
cities
the
theseare
or nationor,ralher,that
e
xhaustive
w
i
t
h
i
n
e
m
b
e
d
d
e
d
h
e
r
e
a
r
e
s
t
r
u
c
t
u
r
e
s
p
a
r
t
i
c
u
l
a
r
I
n
c
l
u
s
i
v
e
map.
r e a d e ro f t h i s
b
o
u
n
d
a r i et sh a t
t
h
e
w
i
t
h
p
r
i
m
a
r
i
l
y
c
o
n
c
e
r
n
e
d
b
e
m
a
y
s t r u c t u r e st h
: e mapas a whole
(Europe
Australia
into
nations,
Attributes
into
Possessive
dividethe Carrier
exhaustively
are Carriersin embedded
Attributesthemselves
into states,etc.),but these Possessive
rivers,lakes,etC'
cities,
as
major
such
Attributes
inclusive
structureswith Possessive
4 C o n j o i n e da n d c o m p o u n d e de x h a u s t i v se t r u c t u r e s
by a line
Attributesare eitherconnected,
structures,Possessive
In conjoinedexhaustive
Attributes
a layoutof the Possessive
or disengaged,by
lackinga featureof directionality,
The latter is the case,for
them,yet clearlyshowshow they fit together.
whichseparates
of the Possessive
instance,of the 'pie chart' in figure 3.16, wherethe disengagement
the rifts that divide
Attributesacquiressymbolicvalue,showing,literallyand figuratively,
l n t o p o f f i 9 u r e2 . 6 , w h e r e
t h e n a t i o nT. h ef o r m e ri s t h e c a s ei n t h e c o m m u n i c a t i omno d e o
. u c hc o n n e c t i nlgi n e sm a y b e p u r e l ya b s t r a c t
a l l t h e p a r t i c i p a n tasr e c o n n e c t ebdy l i n e sS
as in figure2.6,or havea featureof conductivity:
connectors,

*&{x*v*

xs",4

&*hi*q*s*
*?YE

ryffi* Srs$lti*eal
*{*%
tl*ar*igncd1S%

fig f,fe Australiarthe segmenls(Builetin,lo January1989)

98

Conceptual representations

Conductors
are connectors
that alsorepresent
a Possessive
Attribute,a physicalentity
- for instance,wiring,
a pipeline,
a road,a railwaytrack- andtheymayalsobeabstract,as
i n s o m ec o m m u n i c a t i omno d e l sR. e a l i z ebdy a d o u b l el i n e c, o n d u c t o risn d i c a t a
e potential
for dynamicinteractionbetweenthe Possessive
Attributesthey connect.As such,they
are both participantand process,connectedelementand connector,
compounded
and
conjoining.
In compounded
structuresthe Possessive
Attributesare weldedtogether,
whileat the
s a m et i m e r e t a i n i ntgh e i rd i s t i n c it d e n t i t i e sT.h i si s a s m u c ht h e c a s ei n s i m p l ep i ec h a r t s
as in a technicaldrawing,for instance,
the drawingof a machinefor crushingore, in
f i g u r e3 . 1 8 .

5 Topographicaland topologicalprocesses
When analyticalstructures are topographicalthey are read as accuratelyrepresenting
the physicalspatialrelationsandthe relativelocationof the Possessive
Attributes.AII of

fig f.fZ Connectors

and conductors

,:#qr
; i:,:

lr:!1i:
1:i;gli

fiS f.fg Fourth.stage

crusher(Merritt,1984)

rePresentations' 99
Conceptual

sofar maybetopographical:
we havediscussed
thetypesof structuredanalyticalprocesses
scalingdown the
as
accurately
figure
3.18
in
picture
ore
crusher
the
of
the
we read
just aswe readtopographical
and relativelocationof the partsof the machine,
dimensions
of a lake,and its distance
dimensions
the
example,
down,
for
scaling
as
accurately
maps
(mountains,
the boundariesof the
from
and
rivers)
Attributes
Possessive
other
from
Carrier.
the
representing
are readas accurately
Whenanalyticalstructuresaretopological,they
' l o g i c a l ' r e l a t i o nbse t w e e n
p a r t i c i p a n t tsh, e w a y i n w h i c hp a r t i c i p a n tasr e c o n n e c t etdo
e a c ho t h e r( w h e t h etrh e y h a v ec o m m o nb o u n d a r i eos r, a r e p a r t i a l l yo r w h o l l yi n c l u d e d
etc.),but not the actualphysicalsize
theyare connected,
in eachother,in whichsequence
structures,
of the participantsor their distancefrom eachotherorl in the caseof inclusive
is
instance,
for
diagram,
circuit
electrical
An
the
Carrier.
of
from the boundaries
'at
'above'
and
topological(figure3.19). It doesnot signify,say,that lampsa and b are
the
between
distance
the
down
scale
accurately
not
it
does
and
battery,
the right of,the
conbatteryand a lamp a, or betweenthe two lamps.But it doessignifythat they are
n e c t e di n t h i sp a r t i c u l asr e q u e n cjeu,s ta s e v o l u t i otni m e l i n essi g n i f yt h a t ' A p e ' , ' A p e m a n ' ,
'Australopithecus',
in historyin the ordershown'
etc.madetheir first appearance
in mapsof urban
as,for instance,
topological,
or
be
topographical
either
Maps,too,may
are als0
previous
section
in
the
discussed
we
networks
digital
The
transportsystems.
In other
co-location'
adjacency,
on
based
are
as
they
mapsr
abstract
topologicaldiagrams,
Schramm's
or
topographical.
topological
either
be
can
too,
diagrams,
words,abstract
the
c o m m u n i c a t i omno d e l( f i g u r e2 . 6 ) , f o ri n s t a n c ei s, t o p o l o g i c a l . ldt o e sn o t t e l l u st h a t
, S o u r c ei ,s o n t h e l e f t ,o r t o t h ew e s t o
, f t h e ' E n c o d e ri' t; t e l l su so n l yt h a t t h e p a r t i c i p a n t s
the
a r ec o n n e c t eidn t h i ss e q u e n cTeh. i si s n o tt o s a yt h a t t h e p l a c e m e notf t h e ' S o u r c e ' t o
the
'Encoder'
not
from
derives,
significance
its
that
only
not
significant,
is
left of the
(
s
e
e
H
a
l
l
i
d
ay's
6
)
.
c
h
a
p
t
e
r
d
i
a
g
r
a
m
o
f
t
h
e
s
t
r
u
c
t
u
r
e
t
e
x
t
u
a
l
f
r
o
m
t
h
e
i d e a t i o n a bl ,u t
of
other
fields
to
relations
their
and
structures
linguistic
of
diagramof the'nature
t
o
p
o
g
r
a
phy
p
i
e
c
e
o
f
a
b
s
t
r
a
c
t
i
n
t
r
i
c
a
t
e
i
s
a
n
h
a
n
d
,
o
n
t
h
e
o
t
h
e
r
s c h o l a r s h i p| 1' - 9 7 8 : 1 0 ) ,
( s e ef i g u r e3 . 2 0 ) . T h e d i a g r a mu s e sd i s t a n c et o i n d i c a t eh o w c l o s et h e v a r i o u sk i n d s
studyare from what ls hereseenas the centraland most importantform of
of language
yetfinely
study,the studyof'languageas system'.It usesdistancein a figurative,
language
'
l
a
n
g
u
a
g
e
a ss y s t e mt /h a nt h es t u d yo f
w,a y : ' p h o n e t i cfso' ,r e x a m p l ei s, c l o s etro
calibrated
takes
the studyof language
way:
in
same
the
size
relative
it
uses
And
dialectsandregisters.
m
e
dieval
A
s
i
n
o
f
s
c
h
o
l
a
r
s
h
i
p
'
t
o
g
e
t
h
e
r
.
o
t
h
e
r
'
f
i
e
l
d
s
g
r
e
a
t
e
r
a
l
l
t
h
e
t
h
a
n
area
up a much

fig:.rf

Electriccircuitdiagram(Hi11,1980)

i00

Conceptua I representati ons

logic and
mathematics

communications
engineering

languagevarieties:
dialect

graphlc
torn: grammar and

language
pathology

language

language

typology

universals

I
tt

I
l
t

--l---------1-'
V
culture

internalization
productionand

aphasia etc

,;)
( psycho
linguist

?
human
biology

fiS f.ZO The placeof linguisticson the map ot knowledge(Halliday,lgTg)

m a p s /t h e c a r t o g r a p h e r / s ' h o mt e
o w n ' i s b o t h e x a g g e r a t eidn s i z e a n d r e p r e s e n t e d
i n t e r n a l lm
y o r ea c c u r a t e layn dw i t h m o r ed e t a i tl h a nt h e s u r r o u n d i n q ' c o u n t r v s i d e , .
6 Dimensionaland quantitativetopography
Liketopographical
visuals,chartsare drawnto scale.Thescale,however,
is based,not on
the physicaldimensions
of the participants,
but on the quantityor frequency
of aggregates
of participants
that aretakento be identical.pie charts,for instance(seee.g.figure3.ro,
dividea Carrier(the populationof Australia)into components,
Possessive
Attributesthat
are in fact aggregates
of participantsanalysed
as beingthe samein somerespect,
and it
tellsus,notthat'Achievers'arefoundnextto, andto the southeast
of,,Adapters,,
but that
the numberof 'Achievers'
standsto the numberof 'Adapters'as the sizeof the possessive
'Achievers'
Attribute labelled
standsto the size of the Possessive
Attribute labelled
'Adapters'.
Quantityis translatedinto relativesize- althoughit is,of course,
alsopossible
quantitywith quantity,as in figure3.21,a now perhapsratherold-fashioned
to represent

Conceptual representat i ons

101

WOMEN AT WORK
1890

l900

r910

1920

I9 3 0

r940
l9 5 0

figf.zf

ttti
titti
AAAAAAAA
AAAAAAAA
AAAAAAAAAAA
AAA
AIAAATIITII
AAAA
AA
AAAAA
AAAAAA
AA

women atwork(ModleyandLowenstein,1952)

examplewhich usesOtto von Neurath's'picture language'Isotype(see e.g. Lupton,

1989).
they showus one
Pie chartsand bar charts (seefigure 3.23) are one-dimensionali
(abstract)Carrier( in the caseof figure 3.16, the concreteCarrier'Australia' is a metonym
Attributes.Their
for the abstractCarrier'the populationof Australia')and its Possessive
structureis quitesimilarto that of stackdiagrams(seefigure3.5). Theyare exhaustive,
c o m p o u n d eadn a l y t i c asl t r u c t u r e sT. h e d i f f e r e n c el i e s o n l y i n t h e i r p e c u l i a rk i n d o f
bothof whichare quantitative.
accuracy,
abstraction
and peculiarkindof topographical
7 Spatio-temporalanalyticalstructures
comTwo-dimensional
charts createa conjunctionbetweena sef of such (exhaustive,
quantitatively
pounded,
analyticalstructuresand a timeline,
abstractand topographical)
In the caseof linegraphs
for the sal<e
of comparative
analysisalongan orderedtimescale.

r02

ConceptuaI representati ons

this can result in the quasi-vectorial,

quasi-narrative
structuretypifiedby temperature
charts,profit charts,companygrowthcharts,etc. (seefigure3.22). In otherwords,the
s u b s t i t u t i oonf a l i n ef o r d i s c r e t e n t i t i e sc r e a t e s o m e t h i nlgi k ea d y n a m i cp r o c e s(sw i t h
m e a n i n gssu c ha s ' c h a n g e ' , ' g r o w ' , ' d e c r e aes tec' ., ) ,a n d b a c k g r o u n dosr e v e ne r a s e tsh e
a n a l y t i c aslt r u c t u r etsh a t u n d e r l i teh e g r a p hs, ot h a t t h e g r a p hn o l o n g e sr u g g e s t sh a t ' l n
ef AIDS") consisteo
1 9 B Bt h e C a r r i e r( " i n c i d e n c o
d f 5 0 0 , 0 0 0 P o s s e s s i vAet t r i b u t e s
("AI DS cases")and in 1990 the Carrierconsisted
of 800,000 Possessive
Attributesand
. . . (etc.)',but that'The Actor ("the epidemic")acts("spreadsrapidly,'),.
To complicate
matters/one-dimensional
structurescan be representedas though they were twod i m e n s i o n a ln, ds o p r o d u c ae s e n s e
o f p r o g r e sosr d e c l i n ed,e p e n d i nognt h eo r d e ri n w h i c h
plotted
the Possessive
Attributesare
alongthe horizontalaxis - an order not givenby
relationo
s f c o n j o i n e d n eosrs c o m p o u n d n e sosr ,o f i n c l u s i o o
n r e x c l u s i o nb,u t s o l e l yb y
quantity.Thebar chart in figure3.23 caneasilybe (mis)taken
as suggesting,
notthat more
peoplelive in ffatsthan in largehouses,
but that people,as time progresses/
increasingly
livein ffats.
S o m e t h i nqgu i t es i m i l a rc a nh a p p e ni n l a n g u a g eH.o d g ea n d l ( r e s s( 1 9 7 8 ,1 9 9 3 )d r e w
attentionto the effectsof the transformationof nominalization,
which turns clauses
(reportsof events)such as peoplelearnedinto nominals(namesof objects)such as
people'slearningwhichmay then becomeactors in newevents(the new learningspread.
Martin et al. described
this processin historytextbooks.
Theycommented
that suchtransf o r m a t i o n fsa v o u rt h e ' a n t h r o p o m o r p hm
i ce t a p h o o
r f b i r t h ,g r o w t ha n d d e a t h ,( 1 9 8 8 :
1 5 7 ) a n da l l o wh i s t o r i a ntso a n a l y s e
h i s t o r yi n s u c c e s s i vp e r i o d si n w h i c hs i m i l a rt h i n g s
go on. In otherwords,despitetheir apparentstatusas active,dynamicprocesses,
these
structuresstill serveto establish
stable,conceptual
orders.

&iilstflis
th*"'1rutrl}rlln
$a rliJl#

Wrrrtd

"qtxl$

l$

rig rZz c lobal epidemic(Baflet i n, 26 Junetgg})

ConceptuaI rep resentati ons

ct

34
f2
3A
28
26

t t

Leavrn9a space
betweeneach baf
is optional

- 2 2

=
' 1 "8
o
u ! 4

^
6
2
0
houses

fig\ZZ

Smal

Semi-

houses

detached

RoWterface

Villa
units

Typesof dwelling(Bindonand Williams,1988)

is possible,
but nol

risl.z+ Dynamization
of analyticalprocesses

cannotoccur when the

analyticalprocesses
This dynamizationof two-dimensional
Attributesthen remaindiscrete,
is vertical(seefigure3.24).The Possessive
arrangement
This is
and the graphsas a wholesuggesta stableorderratherthan a dynamicprocess.
b o r n e o u t , f o r i n s t a n c e , b y t h e f a c t t h a t o n e c a n inno' t h
' f ei l sl p a c e ot hf e C a r r i e r .
that therehasbeen,in Westerncultures,a shiftfrom a focuson the
It is our impression
w i t h ' W h a t i s t h e s t a t eo f
v e r t i c atl o a f o c u so n t h e h o r i z o n t aal , c h a n g ef r o m a c o n c e r n
' W h e r ea r e w e ? ' t o ' W h e raer ew eg o i n g ? ' a n'dH a v ew e p r o g r e s s eodr a r ew e
a f fa i r s ? ' a n d
i n d e c l i n e ?T' h i si s b o r n eo u t a l s ob y t h e f a c t t h a t s c r i p t sw h i c hw e r et r a d i t i o n a l lwy r i t t e n
to bewrittenfrom left to right.
top-downare nowbeginning
I n f i g u r e3 . 2 5 , w es u m m a r i zteh e d i s t i n c t i o ni sn t r o d u c eidn t h i s s e c t i o nN. o t et h a t t h e
' n dt h e s u p e r s c r i p t ' Tm' e a n s ' t h e nI'n. o t h e rw o r d s i,f t h e r ei s a
superscript'm
l ' e a n s ' i fa
y u s tb eq u a n t i t a t i v e .
t o p o g r a p h i ctai m
l e l i n et h
, e nt h et o p o g r a p hm

103

I04

ConceptuaI representati ons

-{
,-***r*,
Anarvticar |

|
L
""tt""'-Lrnrr.u.,u."o

fiq f.ZS Analyticimagestructures

REALIZATIONS
Unstructured anaIyt i caI process

Temporalanalytical process

Exhaustiveanalytical process

Dimensional topographical
accuracy
Quanti tat i ve topographi caI
accuracy

Topologicalaccuracy

Abstraction

An unordered
setof participants('Possessive
Attributes')is interpreted
as the setof parts
of a wholewhichitselfis not represented.
A setof participants('Possessive
Attributes,)
i s o r d e r e dl i n e a r l o
y n a ( h o r i z o n t aolr
vertical)timelineand interpreted
as the setof
successive
stagesof a temporallyunfolding
process.
A p a r t i c i p a n(t' C a r r i e r 'i)s d e p i c t e a
dsm a d e
up of a numberof parts('Possessive
Attributes')andthe structureis interpreted
as
showina
g l l t h e p a r t sf r o mw h i c ht h ew h o l ei s
m a d eu p .
TheCarrierandthe Possessive
Attributesof
an analyticalprocessare drawnto scale.
Thesizeof the Possessive
Attributesin an
analyticalprocess
accurately
represents
the
numberor someotherquantitative
attribute
of the Possessive
Attributes.
TheCarrierandthe Possessive
Attributesof
an analyticalprocess
are not drawnto scale,
but the way theyare interconnected
is drawn
accurately.
Theparticipantsin an analyticalprocess
may
be concrete.

I representati ons
Conceptua

t05

S Y M B O L I CP R O C E S S E S
are aboutwhata participantmeansor rs.Eithertherearetwo particiSymbolicprocesses
-the
participantwhosemeaningor identityis established
in the relation,theCarrier,
pants
the meaningor identityitself, the SymbolicAttribute
andthe participantwhichrepresents
- or there is only one participant,the Carrier,and in that casethe symbolicmeaningis
we will call
below.Theformertypeof process
established
in anotherway,to be described
SymbolicAttributive; the latter, Symbolic Suggestive.
c sh i c hc a n r e a l i z et h e
A r t h i s t o r i a nhsa v ec h a r t e dt h e f o r m a lp i c t o r i acl h a r a c t e r i s t iw
y e r m e r e n1,9 6 9 ) . S y m b o l i ca t t r i b u t e sa r e
S y m b o l i cA t t r i b u t er e l a t i o n( s e ee s p e c i a l lH
characteristics.
o b j e c tw
s i t h o n eo r m o r eo f t h ef o l l o w i n g
(1) Theyare madesalientin the representation
by
in oneway or another;for instance,
s idz et,h r o u g hb e i n ge s p e c i laylw e lI
b e i n gp l a c e di n t h ef o r e g r o u ntdh,r o u g he x a g g e r a t e
l i t ,t h r o u g hb e i n gr e p r e s e n t ei nde s p e c i a l fl iyn ed e t a i lo r s h a r pf o c u so, r t h r o u g ht h e i r
conspicuou
co
s l o u ro r t o n e .
(2) Theyare pointedat by meansof a gesturewhichcannotbe interpreted
as an action
otherthan the actionof 'pointingout the symbolicattributeto the viewer'- herewe
c a ni n c l u d e
a l s ot h ea r r o w sw h i c hc a nc o n n e cvt i s u a rl e a l i z a t i o nosf p a r t i c i p a n twsi t h
verbalrealizations
of the sameparticipant,or viceversa/as in figure2.4, for these
a l s oe s t a b l i sah r e l a t i o no f i d e n t i t yt h r o u g h' p o i n t i n g ' .
( 3 ) T h e yl o o ko u t o f p l a c ei n t h ew h o l ei,n s o m ew a y .
( 4 ) T h e ya r ec o n v e n t i o n aal lsys o c i a t ewdi t h s y m b o l ivca l u e s .
s e r ev e r yc o m m o ni n t h e M i d d l eA g e sa n dt h e R e n a i s s a n c e :
S u c hc o n v e n t i o nsayl m b o l w
s e e ,f o r i n s t a n c ef i,g u r e3 . 2 7 , w h e r et h e a p p l e ,l o o k i n gs o m e w h aot u t o f p l a c ei n S t
J e r o m e ' ss t u d y ,s y m b o l i z etsh e F a l l , T e m p t a t i o n , 0 r i g i n aSl i n , a n d s o b r i n g st h e s e
i m m e d i a t etloy m i n df o r t h ev i e w e o
r f t h e p a i n t i n gT. ot a k ea m o d e r ne x a m p l et h, es c i e n t i s t
d e p i c t e di n f i g u r e3 . 2 8 i s c l e a r l yn o t d o i n ga n y t h i n gw i t h , o r t o , t h e f u n g iw h i c h a r e
. i sp o s i t i o ni n
d i s p l a y eidn t h ef o r e g r o u nadn do f w h i c hh e i s h o l d i n go n ei n h i sr i g h th a n d H
relationto the fungiseemscontrivedand posed.Thefungifunctionhereas the Attributes
his identityas an experton fungi.
that establish
H u m a np a r t i c i p a n t isn S y m b o l i cA t t r i b u t i v ep r o c e s s euss u a l l yp o s ef o r t h e v i e w e r ,
ratherthan beingshownas involvedin someaction.This doesnot meanthat they are
look at the viewer,
portrayedfront-onand at eyelevel,or that they necessarily
necessarily

Symbolic
5tructures

figl.ZO Typesof symbolicimagestructure

106

ConceptuaI representat i ons

ligl.Zt

St Jeromein his Stud! (Jan van Eyck,1434) (from Hermeren,1969)

eventhoughall of thesewill oftenbe the case.It meansthat theytake up a posturewhich

cannotbe interpreted
as narrative:theyjust sit or standthere,for no reasonotherthanto
d i s p l a tyh e m s e l v et os t h ev i e w e r .
SymbolicSuggestive
Processes
haveonly one participant,the Carrier.Theycannotbe
interpreted
as analytical,becausein this kind of imagedetailtendsto be de-emphasized
i n f a v o u ro f w h a tc o u l db ec a l l e d' m o o d 'o r ' a t m o s p h e r eT'h. i sc a nb e r e a l i z e idn a n u m b e r
of ways:thecoloursmay all blendtogether,
into a hazyblue,for instance,
or a soft golden
glow;the focusmay be soft;or the lightingmay be extreme,rendering
the participantsas
o u t l i n e os r s i l h o u e t t eIst .i s t h i sw h i c hl e n d sS y m b o l i S
c u g g e s t i vpei c t u r e tsh e i rg e n e r i c i t y ,
their qualityof depictingnot a specificmomentbut a generalized
essence.
rhe way in
w h i c h t h eb l u r r i n go f d e t a i lo c c u r s t h e lne n d s s y m b o l i c v a l u et h
t oe C a r r i e-ra s o f t g o l d e n
glow,for instance,
wouldconferon the Carrierall the valuesassociated
with softness,
and
w i t h g o l d ,a s i n t h e B u s h e l lasd v e r t i s e m ernet p r o d u c eidn p l a t e2 . A s a r e s u l tS y m b o l i c
processes
Suggestive
representmeaningand identityas comingfrom within,as deriving
from qualitiesof the Carrierthemselves,
whereasSymbolicAttributiveprocesses
represent
meaningand identityas beingconferredto the Carrier.
E x p r e s s i o n ilsatn d s c a p e( se . 9 .N o l d e ' sT r o p i c a S
l u n ,s e ep l a t e 1 ) a l s o d i m i n i s ht h e
detailof the representation,
in favourof overallcoloureffectsevolvinga strongmood,and
imbuing
t h e C a r r i e r( ' a u t u m ne v e n i n gw
' ) i t hs y m b o l im
c eanings.

Conceptual representations '

I07

!Ets, \.eiEre.B{i{'q"\ b**+'u;

&g .&eeg,g;,
*le}dru
{Fsx{e"';-a$

. tTi"'*li-*X

ffi
O

f in l.zg Fun with fungi (s/dney M oming Henld,lS June 1992)

EMBEDDING
se
, n t e n 6 ecsa n b e s i m p l e( c o n s i s t i nogf o n l yo n ec l a u s e / p r o c e so sr )c o m p l e x
In language
with or subordinated
coordinated
(containing
eachwith their ownprocess/
severalclauses,
how
to eachother).Pictures,too, can be simpleor complex.We havealreadydiscussed
minor,
a second,
and the fire constitutes
in figure2.1 the relationbetweenthe Aborigines
to the 'major' process,which is constitutedby
transactionalprocess/subordinated
and how participantssuchas the
the relationbetweenthe British and the Aborigines;
with a Carrier
be readas analyticalstructures,
can themselves
Britishor the landscape
(e.g.the landscape)
Attributes(e.g.rocks and trees).which of these
and Possessive
s t r u c t u r easr em a J o ar n dw h i c ha r em i n o ri s ,i n v i s u a l sd,e t e r m i n ebdy t h e r e l a t i v se i z ea n d
c o n s p i c u o u s no
e fstsh e e l e m e n t sT.h ec o v e ro f t h e b o o kf r o m w h i c hw e t o o k ' T h e B r i t i s h
. e c a nr e c o g n l z e
o f t h i sk i n do f c o m p l e x i t(yf i T u r e 3 . 2 9 )W
u s e dg u n s ' p r o v i d ea sn e x a m p l e
in this picture:
four differentprocesses
( 1 ) A c l a s s i f i c a t i o nparlo c e sisn w h i c ht h e f i v ec h i l d r e na r e S u b o r d i n a t eo sf t h e c l a s so f
'youngAustralians"in what we havecalleda covert Taxonomy.
They are shown
f al s h i o ni n
, a c i r c l ew h i c h
a g a i n sat n e u t r abl a c k g r o u nadn da r r a n g e idn a s y m m e t r i c a

ConceptuaI representations

108

fig f.Zf Our Societ! antt Others(0akteyet at.,Lg}5)

is closed(and reiterated)by the globe.(we will discussthe meaningof circularity

m o r ef u l l y i n c h a p t e6r . )
( 2 ) A n u m b eor f a n a l y t i c apl r o c e s s ei nsw h i c he a c hc h i l di s C a r r i e irn r e l a t i o n
to a number
o f ( p r o t o t y p i c a l , ' e s s e n t i aPlo' )s s e s s i A
v et t r i b u t e s( s k i nc o l o u rc, o l o u ra n d k i n d o f
hair,colourof eyes,itemsof clothing)- attributeswhich createvisualconceptsof
t h e i rd i f f e r e net t h n i c i t i e s .
3 ) A s y m b o l iact t r i b u t i v pe r o c e sisn w h i c ht h eg l o b ei s a c o n v e n t i o nsayl m b owl i t h s t r o n g
associations.
It is placedin theforeground
of the picture,andtwo of the childrentouch
it with a gesturewhichcannoteasilybe interpreted
as an actionotherthan onethat
drawsthe viewer'sattentionto the globe.Thechildrenarethusshownto be part of the
worlda
, microcosm
o f t h ew o r l di n A u s t r a l i a .
processes
G ) Transactional
in whichthreeofthe childrenhaveputtheirarmsroundother
c h i l d r e nT.h eb o yo n t h e l e f tw o u l ds e e mt o b et h e m a j o rA c t o rh e r ew
, h i l et h et w o g i r l s

ConceptuaI representati ons

r09

are Goalwith respectto his action.Thetwo girls,who havetheir arms roundeach

way.Theyarewhatwe havecalled
other,on the otherhand,relatein a morereciprocal
Interactors.
F r o m t h i s m u l t i p l ea n a l y s i w
s e c a n s e et h a t t h e p i c t u r e( e v e nw h e nw e d i s r e g a r tdh e
i n t e r a c t i vaen d c o m p o s i t i o n satl r u c t u r ew
s h i c ha r e a l s op r e s e n ta, n d w h i c hw i l l b e d i s p
l , u l t i d i m e n s i o ns at rlu c t u r e .
c u s s e idn t h e n e x t h r e ec h a p t e r sf )o r m sa o w e r f u m
processes,
of its sheer
Turningto the relationbetween
we mightsaythat because
these
o tf t h e c h i l d r e ni s d o m i n a n w
t , h i l et h e
s i z ea n dc o n s p i c u o u s ntehsesc i r c u l a ra r r a n g e m e n
and eachof them
analyticalprocesses
are embedded
in it: five childrenare co-classified,
can be further analysedin terms of Possessive
Attributes.In other words,the major
in gender
despitethe difference
message
is that thesechildrenbelongto the samecategory,
different.As far as the
and ethnicity;the minor messageis that they are, nevertheless,
weo u l da r g u et h a t t h ew h i t el i n e so n
t r a n s a c t i o na nl ds y m b o l ipc r o c e s s e
a sr ec o n c e r n e d , w
the boy'stracksuitform stronganddominantvectors.
P e r h a ptsh e ' t r a n s a c t i o n aplr' o c e sosf w h i c ht h i sb o yi s A c t o ra n dt h es y m b o l ipc r o c e s s
process/
andalso
of whichhe is Carrierweighas heavilyin the scaleasthe classificational
girls
processes
are
in the foreground
Thetransactional
form majorprocesses.
of whichthe
b o t hA c t o ra n dG o a l a, n dt h e s y m b o l ipc r o c e sosf w h i c ht h e g i r l o n t h e r i g h ti s t h e C a r r i e r ,
lessconspicuous
as minor processes.
however,
are considerably
and might be interpreted
andas Carriersof the symbolicvalue
In otherwords,as Actorsof the gestureof solidarity,
g e w o r l d ' ,t h e ' e t h n i c 'g i r l s h a v e ,i n t h i s r e p r e s e n t a t i oan m
, uchless
o f ' r e p r e s e n t i nt h
significantroleto playthanthe whiteAustralianboyin histracksuit.

C O N C E P T U ASLT R U C T U R EISN L A N G U A G E
structuresare realizedin
Thereare somepointsof contactbetweenthe way conceptual
l a n g u a gaen di n i m a g e sT.h ec o m p a r i s owno u l dh a v et o b e m a d ew i t ht h e k i n d so f l i n g u i s t i c
s t r u c t u r eH
s a l l i d a yc a l l s ' r e l a t i o n a l ' a n d ' e x i s t e n t i a l ' p r o c e(ssseeesH a l l i d a y1, 9 8 5 :
the
imagesthat theyrepresent
1l-2ff.).Thesehaveat leastthis in commonwith conceptual
world in termsof moreor lesspermanent
statesof affairsor generaltruths,ratherthan in
of relational
maincategories
termsof actionsor mentalprocesses.
Hallidayrecognizestwo
process,
the Attributiveand the Identifyingprocess.Themeaningof an Attributiveprocess
clausecan be schematically
describedas 'a is an attributeof x'. The attribute'a' is
thensimplycalledAttribute,andthe participantwhoseattributeit is, isthe Carrier-we
have borrowedtheseterms in our analysisof images.In Somepeopleare racist,for
andracistis
instance,
somepeopleis Carrier,are is the Relational(Attributive)process/
the Attribute.Attributive processescan be Intensive- that is, they can be about what a
-that is,theycan be
Carrier is,as in the examplejust given;they can be Circumstantial
(
e
.
9
.
i n T h e i rh o m ew a si n H o C h iM i n h
a b o u t ' w h e r e ' o r ' w h e n ' o r ' w hw
a ti t h ' a C a r r i e irs
and in
an Attributive[Circumstantial]Relationalprocess,
City, theirhomeis Carrier,razas
- that is, they can be about
Ho Chi Minh City an Attribute); and they can be Possessive

110

Concept uaI representati ons

whata Carrierhas(e.9.in I didn'towna thingin the world,I is Carrier,owntheAttributive

IPossessive]
process,
Relational
anda thingin the world,the Attribute).
Themeaning
of an ldentifyingRelationalprocessclause
canbeschematically
described
a s ' a i s t h e i d e n t i t yo f x ' . T h ei d e n t i t y ' a 'i s t h e nt e r m e dt h e V a l u ea n dt h e p a r t i c i p a n t ' x '
whoseidentity it is, is called the Token.In Rev Peter Nyanginguis the Minister of the
Uniting Church at Ernabella, Rev Peter Nyanginguis Token, rs is the Identifying
Relationalprocess/and the Minister of the Uniting Churchat Ernabellais the Value.The
Value,then,tendsto be a'status' or 'function'or 'meaning,whichservesto identifythe
Token,and the Tokenis the nameor somedescriptionof the holderof the status,or of
t h e o c c u p a not f t h e f u n c t i o no, r o f t h e s i g nw h i c hh a st h e m e a n i n gP. i c t u r ec a p t i o n a
sre
oftenidentifying
clauses,
with a reference
to the pictureas Tokenandthe meaningof the
Pictureas Value(for instance,This is oysterfarmingin Palm Islana.ldentifyingand
Attributiveclauses
can be distinguished
from eachotherby meansof the reversibility
test:
in Identifyingclausesthe orderof the participantscan be reversedG.g. TheMinisterof
the UnitingChurchat Ernabellais RevPeter Nyanginguis just as acceptableas RevPeter
Nyanginguis the Minister of the Uniting Churchat Ernabelld, whereasin Attributive
clausesthis is not the case(Racistare somepeopleis much more unusualthan Some
peopleare racis).
E x i s t e n t i acll a u s e sf i, n a l l y s, i m p l ys t a t et h a t ' s o m e t h i n g
e x i s t s ,T. h e yh a v eo n l y o n e
participant,the Existent- that is,the participantwhoseexistence
the structureaffirms.
Existentsmayeitherbe EventsG.g.in It's terribleweatheAor EntitiesG.g.in Therewas
a public librarfl. The presenceof a 'dummy subject'(there or referentless
i/) is the
p r i n c i p ai ld e n t i f y i ncgh a r a c t e r i s toi cf e x i s t e n t i ac l a u s e(ss e eH a l l i d a y1,9 8 5 :1 3 0 ) .
VisualClassificational
and Analyticalstructurescouldthereforebe saidto be akin to,
respectively,
Intensiveand Possessive
Attributiveclauses,
and SymbolicAttributivestructurescouldbe seenas akin to Identifyingclauses,
andthereis perhapsalsosomeaffinity
betweenSymbolicSuggestive
structuresand Existentialclauses.But the differences
are
greaterthan the similaritiesand,especially
in the areaof Classificational
and Analytical
images,
the visualsemiotichasa rangeof structuraldeviceswhichhaveno equivalentin
language:
the difference
between
visualconceptualization
and linguisticconceptualization
is evidentlyquite large.All the more importantto havea vocabulary
for expressing
what
can be doneand is donewith eachin concretetextsthat combinethe two semiotics,
texts
suchas 'The 0verland'(figure3.30),a schoolprojectby an eight-year-old
boy.Looking
first at the words,the structureof the verbaltext on the left pageis Possessive
Attribute
throughout.Thereis a Carrier('The Overland'),
and thereare six Possessive
Attributes
('fivefirst classcarriages','three
secondclasscarriages','twodiningcar,s,,,twoengines,
and'one clubcar','onemotor rail cart'). Theprocessitselfis elided.
The picture,too, is concerned
with the relationof parts to a whole.It has all the
hallmarksof an exhaustive,
structuredanalyticalprocess:
the pictureis front-onand eye
level,there is no Setting,and the differentPossessive
Attributes(signs,windows,wheels,
s u s p e n s i oann d r a i l s )a r e s h o w nc l e a r l yb, u t w i t h o u tu n n e c e s s adr ye t a i l :i t i s r e l a t i v e l y
'abstract'.Therelationbetween
pictureandtext is not oneof illustration.
Theoicturedoes
not duplicatethe text, it doesnot representvisuallywhat has alreadybeenrepresented

ConceptuaI representati ons

111

rig s.lo'The overtand'

l i n g u i s t i c a l lN
y .o r i s t h e r ea r e l a t i o no f ' a n c h o r a g e( 'B a r t h e s1, 9 7 7 ) i n w h i c ht h e t e x t
elaborates
the informationgivenin the picturewithout providingnew information.
It is
true that both text and pictureare about part-wholerelationsbut this doesnot mean
theyduplicateeachother,because
in the text the Carrieris the train as a whole,whereasin
thepicturetheCarrie
r n e o / t h e c a r r i a g oe fst h e t r a i n . T h e c h i l d h a s t a k e n a t o p i c , t h e
so
'0verlandExpress',
andtreatedit verballyandvisuallyin sucha waythat eachpart of the
text supplements
the otherpart.
Let us analysethe secondpage.Thewordsare as follows:
I t g o e sf r o mA D E L A I D
t oEM e l b o u r nrea i l A u s t r a l i a N
n a t i o n arlo o t .
Herewe havefirst of all a non-transactional
action,with an Actor ('it') and an Action
( ' g o e s ' )a, s w e l l a s t w o C i r c u m s t a n c o
e fs p l a c e( ' f r o m A d e l a i d ea, n d ' t o M e l b o u r n e , ) ,
Therefollowsan identification
of the route,with the Tokenand the Processelided('rail
AustralianNationalroot'):the childobjectifies
the action'going,,turns it into a thing,a
' r o u t e ' ,b y m e a n so f a n i d e n t i f y i n g
r e l a t i o n apl r o c e sisn w h i c ht h e ' r o o t ' i s V a l u eF
. inally
the route is describedin more detail,througha visualanalyticalprocessin which the
r e l e v a nst e c t i o no f A u s t r a l i ai s t h e c a r r i e ra, n d A d e l a i d eM, e l b o u r naen d t h e ' r o u t e ' t h e

r72

Conceptu a I rep resen tat i ons

Possessive
Attributes,in a processwhich usesmany of the structuraldevicesof the
analyticalvisualto showprecisely
howthe Possessive
Attributesare spatially related,how
they'fit together'.
' M y A d v e n t u r e ' ( f i g u3r e
. 3 1 )i s a n o t h esr c h o opl r o j e c t , w r i t t ebny a c h i l df r o mt h es a m e
schooland yearas the authorof'The 0verland'.Most of the verbalprocesses
are transactionaland non-transactional
actionsin whichthe narratoris Actor: he walks,findsa
c a v e /f i n d ss o m en a i l s ,a n d s o o n . H e i s n o t d e s c r i b i nsgo m e t h i n b
g u t t e l l i n ga s t o r y ,
narratina
g p a r t i c u l aer v e n t :
N o t l o n ga g o I w e n tf o r a j o u r n e yd o w nt o t h e b e a c hI. w a l k e da l l a l o n gt h e b e a c h
until I founda cave.That cavewas big.Thentherewas somewood but I couldn,t
m a k ea r a f t w i t h o u tn a i l sJ. u s tt h e nI f o u n ds o m en a i l si n t h e c a v e I. s a i dt o m y s e l f
'Somebodymusthavebeenhere'. I got piece
So
a
of woodand startedmakingthe
raft. I useda pieceof woodfor a. . . .
S t i l l ,t h e r ei s a l s oa c o n c e p t u a
e l e m e n it n. t h i s p a r t o f t h e s t o r yt h ew r i t e ri s p r e o c c u p i e d
with the materialsneeded
for buildinga raft. Thereis a hiddenattributiveprocess,
somet h i n gl i k e ' t h em a t e r i a lfso r a r a f t a r ew o o da n dn a i l s 'b, u t i t i s t r a n s f o r m eidn t ot h e s t o r y
o f t h ef i n d i n go f t h e s em a t e r i a l (sw h i c he m e r g em y s t e r i o u silnya b i gc a v e )A
. s i n t h es t o r y

,MyAdventure'
@ rigr.3l

ConceptuaI representations

1r3

that opensthe bool<of Genesis,

with conceptualizing
which is alsoa narrativeconcerned
the elements
that makeup a whole,the world,the writer alternatesactionaland trans('then
('l foundsomenails')with moreconceptual
processes
actionalprocesses
existential
t h e r ew a ss o m ew o o d ' ) .
and it is
Theverbaltext stopsabruptly,in the middleof the Circumstance
of Purpose,
continued,
not on the followingpage,but by the picture,whichshowswhatthesematerials
wereusedfor: the partsof a raft. It is an Unstructured
Analyticalprocess,
with the sail,
and the raft itself (alreadycompletewith mast and rudder),as Possessive
Attributes.
Theseare,in the child'sconceptualization,
the partsfrom whicha raft is made.However,
t h e r ei s a l s oa s t r o n gn a r r a t i v e l e m e nitn t h i s p i c t u r ei:t h a sa S e t t i n g( t h es h o r e l i n teh, e
cave) and it doesnot use the frontal,eye-levelperspective
typical of most analytical
i m a g e s( a n da l r e a d ys u c c e s s f u lul ys e db y t h e a u t h o ro f ' T h e O v e r l a n d b
' )u t a h i g ha n d
o b l i q u ea n g l eI.n b o t ht h e p i c t u r ea n dt h e w o r d st,h ew r i t e rb l e n d tsh e c o n c e p t u a ln dt h e
narrativeor, rafher,narrativizes
the conceptual.
In the verbaltext he doesthis through
c o n j u n c t i o n( 'st h e n ' , ' j u st th e n ' e
, t c . )t,h r o u g ht h e u s eo f t h e p a s t e n s ea n dt h r o u g ht h e u s e
of transactionaland non-transactional
with the narratoras a (firstAction processes
person)Actor.In the picturehe doesit throughthe use of a Settingand a narrative
p e r s p e c t i vFei.n a l l yt ,h e w o r d sa n dt h e p i c t u r ea g a i nc o m p l e m e neta c ho t h e rT. h ep i c t u r e
doesnot il lustratethe storybut continues
it.
It is clear,we hope,that childrenactivelyexperimentwith the representational
Their
resources
of word and image,and with the ways in which they can be combined.
d r a w i n gas r e n o tj u s t i l l u s t r a t i o nosf a v e r b a tl e x t ,n o tj u s t ' c r e a t i v e m b e l l i s h m e nt h
t 'e
;y
a r e p a r t o f a ' m u l t i m o d a l l y ' c o n c e i vt e dx t ,a s e m i o t i ci n t e r p l a iyn w h i c he a c hm o d et,h e
verbalandthe visual,is givena definedandequalroleto play.

4 R e p r e s e n t a t i oa nn d i n t e r a c t i o n :
d e s i g n i n tgh e p o s i t i o no f t h e v i e w e r
In the previous
chapterwe discussed
visualresources
for the representation
of interactions
a n d c o n c e p t u ar e
l l a t i o nb
s e t w e e tnh e p e o p l ep, l a c e sa n d t h i n g sd e p i c t e di n i m a g e sB. u t
v i s u acl o m m u n i c a t i oanl s oh a sr e s o u r c ef os r c o n s t i t u t i nagn dm a i n t a i n i nagn o t h e kr i n do f
interaction,
the interaction
between
the producerandthe viewerof the image.Anotherway
o f s a y i n gt h i s i s t h a t i m a g e s( a n do t h e rk i n d so f v i s u a l )i n v o l v e
t w o k i n d so f p a r t i c i p a n t s ,
participants(the people,theplacesandthingsdepictedin images)andinterrepresented
active participanfs(the peoplewho communicatewith each other through images,the
producersand viewersof images),and three kinds of relations:(1) relationsbetween
participants;(2) relationsbetweeninteractiveand represented
participants
represented
(the interactiveparticipants'attitudestowardsthe represented
participants);and (3)
participants(thethingsinteractive
participantsdo to or for
relationsbetweeninteractive
eachotherthroughimages).
Interactiveparticipantsare thereforereal peoplewho produceand make senseof
imagesin thecontextof socialinstitutions
which,to differentdegrees
andin differentways,
r e g u l a t ew h a t m a y b e ' s a i d ' w i t h i m a g e sh, o w i t s h o u l db e s a i d ,a n d h o w i t s h o u l db e
interpreted.
In somecasesthe interactionis direct and immediate.Producerand viewer
know eachother and are involvedin face-to-faceinteraction,
as whenwe make photographsof eachotherto keepin walletsor pin on pinboards,
or draw mapsto giveeach
otherdirections,
or diagramsto explainideasto eachother.But in manycasesthereis no
immediateand direct involvement.
The produceris absentfor the viewer.and the viewer
producer.
is absentfor the
Thinkof photographs
in magazines.
Who is the producer?The
photographer
who took the shot?Theassistant
who processed
and printedit? Theagency
picture
who selectedand distributedit? The
editorwho choseit? The layoutartist who
c r o p p e di t a n d d e t e r m i n eidt s s i z ea n d p o s i t i o no n t h e p a g e ?M o s tv i e w e r sw i l l n o t o n l y
nevermeetall thesecontributors
faceto face,but alsohaveonly
to the productionprocess
ahazya
, n dp e r h a pds i s t o r t e d
a n dg l a m o r i z e di d, e ao f t h e p r o d u c t i opnr o c e s s ebse h i n dt h e
image.All they haveis the pictureitself,as it appearsin the magazine.
And producers,
similarly,
can neverreallyknowtheir vastand absentaudiences,
and must,instead,
create
a mentalimageof 'the' viewersand 'the' way viewersmakesenseof their pictures.In
particieverydayface-to-face
communication
it is easyenoughto distinguish
interactive
pantsfrom represented
participants:
thereis alwaysan image-producer
anda viewer(who,
depending
on the situation,mayswaproleswith the producer,
addto the scribbled
ffoorplan
(for instance,
or diagram,for instance),
participants
andthentherearethe represented
the
peopleon the quicksketchof the dinnertablearrangement,
or the landmarks
on the handdrawnmap),andthesemay,of course/includethe producerand/orthe viewerthemselves.
Producerand viewerare physicallypresent.The participantsthey representneednot be.
But when there is a disjunctionbetweenthe contextof productionand the contextof
produceris not physically
present,
reception,the
andthevieweris alonewith the imageand

Rep resentati on and i nteract i on

115

- an illuminating
of billboard
exception
is the caseof the 'defacement'
cannotreciprocate
'respond'to
the initial 'turn' or statementof the
when graffiti artists
advertisements,
image.
S o m e t h i nsgi m i l a ro c c u r si n w r i t i n g .W r i t e r s t, o o ,a r e n o t u s u a l l yp h y s i c a l lpyr e s e n t
whentheir wordsare read,and must addresstheir readersin the guiseof represented
participants,
too, are alonewith the
evenwhenthey write in the first person.Readers,
writtenword,and cannotusuallybecomewriters in turn. Literarytheorists(e.9.Booth,
'real'
1 9 6 1 ; C h a t m a n!,9 7 8 ) h a v ea d d r e s s etdh i s p r o b l e mb y d i s t i n g u i s h i nbge t w e e n
a n d ' i m p l i e d ' a u t h oar sn,db e t w e e n ' r e a l ' a n d ' i m p l ireeda'd e r sT.h e ' i m p l i eadu t h o r i' s a
d i s e m b o d i evdo i c e o
, r e v e n ' a s e t o f i m p l i c i tn o r m sr a t h e rt h a n a s p e a k eor r a v o i c e '
( R i m m o n - l ( e n a n , t 9 8 3 : 8 7 ) : ' hoer,b e t t e ri,t h a sn o v o i c en, o d i r e c tm e a n so f c o m m u n i cating,but instructsus silently,
throughthe designof the whole,with al I the voices,by all
' i m p l i e dr e a d e r ' ,
t h e m e a n si t h a s c h o s e nt o l e t u s l e a r n '( C h a t m a nI,9 7 8 : 1 4 8 ) . T h e
' p r e f e r r erde a d i n g
p o s i t i o ne' ,t c . s, i m i l a r l yi s, ' a n i m a g eo f a c e r t a i nc o m p e t e n cber o u g htto
I9B3'.
within the text' (Rimmon-l(enan,
the text and a structuringof suchcompetence
1 1 8 ) :t h e t e x t s e l e c t a
s ' m o d e l r e a d e r ' t h r o u giht s ' c h o i c eo f a s p e c i f i cl i n g u i s t icco d e a,
c e r t a i nl i t e r a r ys t y l e ' a n db y p r e s u p p o s i n gs'pae c i f iecn c y c l o p e dciocm p e t e n c e ' ot hnep a r t
in the text itself.
of the reader(Eco,I97g:7). Thiswe can know.0f this we haveevidence
Realauthorsand real readerswe cannotultimatelyknow.This bracketingout of real
authorsand real readerscarriesthe risk of forgettingthat texts,literaryand artistictexts
in
as muchas massmediatexts,are producedin the contextof real socialinstitutions,
o r d e rt o p l a ya v e r yr e a l r o l e i n s o c i a l i f e- i n o r d e rt o d o c e r t a i nt h i n g st o o r f o r t h e i r
attitudestowardsaspectsof sociallife andtowards
readers,
and in orderto communicate
awareof this
peoplewho participatein them,whetherauthorsand readersareconsciously
mustwork within moreor
or not. Producers,
if theywant to seetheir work disseminated,
lessrigidlydefinedconventions,
and adhereto the moreor Iessrigidlydefinedvaluesand
b e l i e fo
s f t h es o c i ailn s t i t u t i ow
n i t h i nw h i c ht h e i rw o r ki s p r o d u c eadn dc i r c u l a t e dR. e a d e r s
intentionsand thesevaluesand attitudes
will at least recognizethesecommunicative
for what they are, evenif they do not ultimatelyacceptthem as their own valuesand
of what is meantwhile refusingthe speaker's
beliefs.Theycan 'recognizethe substance
i n t e r p r e t a t i oanns da s s e s s m e n(tS
s 'c a n n e l1l ,9 9 4 :1 1 ) .
Howeverimportantand real this disjunctionbetweenthe contextof productionand
the contextof reception,
the two do haveelementsin common:the imageitself,and a
n n du n d e r s t a n d i n g ,
k n o w l e d goef t h e c o m m u n i c a t i vr ees o u r c et hs a t a l l o wi t s a r t i c u l a t i o a
It
in images.
andsocialrelationscanbeencoded
a knowledge
of the waysocialinteractions
of the viewerdifferin a
is oftensaidthatthe knowledge
of the producerandthe knowledge
s e l la s t h e ' r e c e i v i n g '
f u n d a m e n t ar el s p e c t h
: e f o r m e ri s a c t i v ea, l l o w i n gt h e ' s e n d i n g ' aw
' m e s s a g e sP' .r o d u c e r s
o f ' m e s s a g e st h' ;e l a t t e ri s p a s s i v ea ,l l o w i n go n l yt h e ' r e c e i v i n g ' o f
areableto'write'aswellas'read',viewersareableonlyto'read'.Uptoapointthisistrue,
activity,so that
at leastin the sensethat the productionof imagesis still a specialized
'write' moreffuentlyand eloquently,
than viewers.But we
producers
and morefrequently,
meanings
explicitwill showthat the interactive
hopeour attemptsto makethat knowledge
and viewers.
sharedby producers
are visuallyencodedin waysthat rest on competencies

116

Representation and i nteracti on

T h ea r t i c u l a t i o a
nn du n d e r s t a n d i n
og
f sociam
l e a n i n gisn i m a g e sd e r i v e fsr o m t h e v i s u a l
articulationof socialmeanings
in face-to-face
interaction,
the spatialpositionsallocated
to differentkindsof socialactorsin interaction(whethertheyare seatedor standing,
side
by side or facingeach other frontally,etc.). In this sensethe interactivedimensionof
i m a g e si s t h e ' w r i t i n g ' o fw h a t i s u s u a l l yc a l l e d' n o n - v e r b ac lo m m u n i c a t i o a
n ' l,a n g u a g e '
sharedby producers
andviewersalike.
Thedisjunction
betweenthe contextof productionandthe contextof receptionhasyet
anothereffect: it causessocial relationsto be representedrather than enacted.Because
the producersare absentfrom the placewherethe actualcommunicative
transactionis
c o m p l e t e fdr,o mt h e l o c u so f r e c e p t i o n
t h, e yc a n n o st a y ' l ' o t h e rt h a nt h r o u g ha s u b s t i t u t e
'l'. Evenwhenthe viewerreceives
an imageof the 'real author' or a contributorto the
production
process the presenter
programme,
in a television
the painterin a self-portrait,
t h e o w n e ro f t h e c o m p a n y( o r t h e w o r k e ri n t h e c e n t u r i e s - odl di s t i l l e r yi)n a n a d v e r t i s e ment - that imageis only an image,a doubleof the 'real author',a representation,
d e t a c h efdr o m h i so r h e ra c t u a lb o d yA. n dt h e ' r e a la u t h o r sm
' a ya l s os p e a ki n t h eg u i s eo f
someone
else,of a 'character',
aswhen,insteadof the ownerof a company,
it is UncleSam,
or a larger-than-life
walkingandtalkingteddybear,who addresses
us in an advertisement.
This dimensionof representation
is anotherone which has beenstudiedextensively
in
literarytheory (e.9.Genette,7972). The relationbetweenproducerand viewer,too, is
represented
ratherthan enacted.In face-to-face
we must respondto a
communication
friendlysmilewith a friendlysmile,to an arrogantstarewith a deferentialloweringof the
e y e sa, n ds u c ho b l i g a t i o ncsa n n o et a s i l yb e a v o i d e w
d i t h o u ta p p e a r i nigm p o l i t eu, n f r i e n d l y
or impudent.
Whenimagesconfrontus with friendlysmilesor arrogantstares,we are not
obligedto respond,
eventhoughwe do recognize
howwe areaddressed.
Therelationis only
represented.
We are imaginarilyratherthan reallyput in the positionof the friend,the
customer,
the lay personwho must deferto the expert.And whetheror not we identify
w i t h t h a t p o s i t i o nw i l l d e p e n do t h e rf a c t o r s- o n o u r r e a l r e l a t i o nt o t h e p r o d u c eor r
the institutionhe or sherepresents,
and on our real relationto the otherswho form part
of the contextof reception.
All the same,whetheror not we identifywith the way we are
addressed,
we do understand
how we are addressed,
we do understand
because
the way
imagesrepresent
socialinteractions
and socialrelations.
It is the business
of this chapter
t o t r y a n dm a k et h o s eu n d e r s t a n d i negxsp l i c i t .

T H E I M A G EA C T A N D T H E G A Z E
In the previouschapterswe showedtwo picturesof an Antarcticexplorer,
takenfrom the
Australianprimary-school
socialstudiestextbook)ur Societyand )thers (}akley et al.,
1 9 8 5 ) . F i g u r e3 . 1 0 w a s a p h o t o g r a p ihn w h i c ht h e A u s t r a l i a nA n t a r c t i ce x p l o r e rS i r
D o u g l aM
s a w s o nl o o k e d i r e c t l ya t t h ev i e w e rT.h es c h e m a t iacn d' g e n e r a l i z e d ' e x p l oi rne r
figure 2.4, on the other hand,did not look at the viewer.The two imagesare in fact
positioned
sideby side,the photoon the left page,the drawingon the right.Together,
they
combine
t w o d i f f e r e nct o m m u n i c a t i fvuen c t i o n sT.h ep h o t os e e k sa b o v ea l l t o b r i n qa b o u t

Representati on and i nteract io n

t17

an imaginaryrelationbetweenthe represented
explorerand the childrenfor whom the
book is written,a relationperhapsof admirationfor, and identification
with, a national
h e r oA
. n dt h i s m e a n sa l s ot h a t t h e i m a g e - p r o d u c(et hr ei n s t i t u t i o on f e d u c a t i o n p
au
l blishi n g )a d d r e s s et hse c h i l d r e ni n t h e v o i c eo f t h e n a t i o n ahl e r oa n d m a k e st h a t n a t i o n ahl e r o
a n ' e d u c a t i o n a l ' v o i cTeh.ed r a w i n go, n t h e o t h e rh a n d s, e e k sf ,i r s t o f a l l ,t o b e r e a da s a
pieceof objective,
factualinformation,
and in this way aimsto set into motionthe actual
process
of learning.
There is, then, a fundamentaldifferencebetweenpicturesfrom which represented
participantslook directlyat the viewer'seyes,and picturesin whichthis is not the case.
participantslool<at the viewer,
Whenrepresented
vectors,
formedby participants'eyelines,
c o n n e ctth e p a r t i c i p a n tws i t h t h e v i e w e rC. o n t a c ti s e s t a b l i s h eedv, e ni f i t i s o n l yo n a n
imaginarylevel.In additiontheremaybe a furthervector,formedby a gesturein the same
d i r e c t i o na,s i n f i g u r e4 . 1 .
Thisvisualconfiguration
hastwo relatedfunctions.In the first placeit createsa visual
form of direct address.It acl<nowledges
the viewersexplicitly,addressing
them with a
p l a c ei t c o n s t i t u t easn ' i m a g ea c t ' .T h ep r o d u c eur s e st h e i m a g e
v i s u a'ly o u ' .I n t h es e c o n d

fiq e.f Recruitmentp0ster(AtfredLeete,I9l4) (ImperialWar Museum)

118 .

Representation and interaction

to do something
to the viewer.
It is for this reasonthat we havecalledthis kind of imagea
' d e m a n d f' ,o l l o w i n gH a l l i d a y( 1 9 8 5 ) :t h e p a r t i c i p a n t 'gsa z e( a n dt h e g e s t u r ei f, p r e s e n t )
demandssomethingfrom the viewer,demandsthat the viewerenter into somekind of
imaginaryrelationwith him or her.Exactlywhat kindof relationis thensignifiedby other
participants.Theymay
means,for instanceby the facial expression
of the represented
relation
smile,in whichcasethe vieweris askedto enterinto a
of socialaffinitywith them;
theymaystareat the viewerwith colddisdain,in whichcasethe vieweris askedto relateto
poutat the viewer,
them,perhaps,
as an inferiorrelatesto a superior;
theymay seductively
gestures.
in whichcasethe vieweris askedto desirethem.Thesameappliesto
A handcan
pointat the viewer,in a visual'Hey,you there,I meanyou',or invitethe viewerto come
gesture,
closer,
or holdthe viewerat baywith a defensive
as if to say,'Stayawayfrom me'.
In eachcasethe imagewantssomething
from the viewers- wantsthemto do something
(comecloser,
stayat a distance)or to form a pseudo-social
bondof a particularkindwith
participant.
the represented
And in doingthis,imagesdefineto someextentwho the viewer
participant,etc.),and in that way excludeother
is (e.9.male,inferiorto the represented
viewers.
I n t h e h i s t o r yo f a r t ,t h i sl o o kw a sa s i g n i f i c a ni nt n o v a t i oA
n l. t h o u g hi n I t a l i a np a i n t i n g
s m a l lf i g u r e sa m o n gt h e b y s t a n d e o
r sf t h e C r u c i f i x i oann do t h e rb i b l i c asl c e n ecsa nb es e e n
to look at the viewerfrom the fourteenthcenturyonwards,the 'demand'picturecomes
i n t o i t s o w n i n t h e f i f t e e n t hc e n t u r yA. c c o r d i n tgo P a n o f s k(y1 9 5 3 : 1 9 0 ) ,i t o r i g i n a t eidn
and Jan van Eyckwas the first to useit in Man in a Red Turban(1433),
self-portraits,
whichis regardedby mostart historiansas a self-portrait.
In 1.433Jan van Eyck made one of the great discoveries
in portraiture.In the
portraitof a 'M an in a Redfurban', completed
in October21 of that year,the glance
picture
with an
of the sitteris turnedout of the
andsharplyfocusedon the beholder
with
its
slightly
air of skepticismintensifiedby the expression
of the thin mouth
compressed
corners.For the first time the sitter seeksto establishdirect contact
with the spectator.
andscrutinized
by a wakefulintelligence.
. . . We feelobserved
( P a n o f s k1
y ,9 5 3 : 7 9 8 )
0 t h e r st r a c ei t b a c kf u r t h e rA. c c o r d i ntgo B e l t i n g( 1 9 9 0 :5 7 ) , ' t h es u g g e s t i oonf r e c i p r o c i t y
purpose.
By the
between
the viewerandthe persondepictedin the image'hada devotional
m
o
n
k
s
V
i
r
g
i
n
a
n dh e r
thirteenth
century,
i n t h e i rc e l l s ' h a db e f o r et h e i re y e si m a g e o
sf the
praying
crucifiedson,sothat whilereading,
andsleeping,
theycouldlookuponthemandbe
l o o k e du p o n w i t ht h ee y e so f c o m p a s s i o n ' ( o iut ra l i c s ) .
t sh o l o o ka t t h ev i e w ear r eu s u a l l h
R e p r e s e n tpeadr t i c i p a n w
y u m a n( o r a n i m a l )b, u t n o t
headlights
always:the
for instance,
and
of a car canbe drawnas eyeslookingat the viewer,
on the screenof oneautomaticbankteller,a creaturewhosecombinedheadand bodyhas
t h e b o x - l i k es h a p eo f a m a c h i n es,m i l e sa t t h e v i e w e rh, o l d i n go u t h i s h a n di n a n i n v i t i n g
gesture,
thus'demanding'
a friendlyrelationbetween
the machineand its user(figure4.2).
point
is,
The
whethertheyare humanor not,by beingrepresented
as lookingat the viewer,
theyare represented
as human,anthropomorphized
to somedegree.

Representat i on and interact i on

I19

riq a.z ATMscreen

Otherpicturesaddressus indirectly.Herethe vieweris not object,but subjectof the

participantis the objectof the viewer'sdispassionate
look,and the represented
scrutiny.
N o c o n t a c it s m a d eT
. h ev i e w e r ' rso l ei s t h a t o f a n i n v i s i b loen l o o k eAr .l l i m a g e w
s h i c hd o
n o t c o n t a i nh u m a no r q u a s i - h u m apna r t i c i p a n tlso o k i n gd i r e c t l ya t t h e v i e w e ra r e o f t h i s
k i n d . F o r t h i s r e a s o n w e h a v e , a g a i n f o lHl oaw
l l i nd ga y ( 1 9 8 5 ) , c a l l e d t h i s k i inmdaogf e a n
' o f f e r ' - i t ' o f f e r s ' t h e r e p r e s e n t epda r t i c i p a n ttso t h e v i e w e ra s i t e m so f i n f o r m a t i o n ,
objecto
s f c o n t e m p l a t i oinm, p e r s o n a lal ys,t h o u g ht h e yw e r es p e c i m e ni nsa d i s p l a cy a s e .
participantsare,in a given
It is alwaysinteresting
to studywhichkindsof represented
context,depictedas demanding
an imaginarysocialresponse
of somel<indfrom the viewer,
andwhichare not.In OurSocietyand0thers(Oal<ley
et al.,).985),theAustralianprimaryschooltextbookfrom whichwe drewmanyof our keyexamplesin the first versionof this
b o o k ,i m m i g r a nfta m i l i e s m i l ea t t h e v i e w e rH. o w e v etrh, e h u m a np a r t i c i p a n tisn p i c t u r e s
f r o m t h e s ei m m i g r a n t s ' c o u n t r ioef so r i g i nd o n o t l o o ka t t h e v i e w e rn,o t e v e ni n c l o s e - u p
portraits,as,for instance,
in the portraitof an Italiangrandmother
who stayedbehind.In
the chapteron Aborigines,
by contrast,hardlyanyof the Aboriginalparticipantslookat the
viewer.
TheAboriginalpoetOodgeroo
Noonuccal,
referredto in the bookas'l(ath Walker',
a n dd e p i c t e d
i n c l o s e - u ipn t h e l a s t i l l u s t r a t i oonf t h a t c h a p t e ri s, t h e o n l ye x c e p t i o(ns e e
f i g u r e4 . 3 b e l o w )H
. e re x p r e s s i ohne, r m a k e - u ph,e rh a i r s t y l e
a n dd r e s sh a r d l yd i s t i n g u i s h
herfrom non-Aboriginal
womenof herage.At mosther skin is somewhatdarker,but even
t h a t i s n o t v e r y p r o n o u n c ei d
n t h e b l a c k - a n d - w h ist he o t .O t h e rA b o r i g i n apl e o p l ei n t h e
chapterare much more cleariydepictedas'other', and even if they do, occasionally,
look directlyat the viewer,
the
they do so from a longdistance,
whichgreatlydiminishes
i m p a c to f t h e i r l o o k ,o r a r e f i g u r e si n t h e b a c l < g r o u nl od o, k i n gb l a n k l ya n d m o r eo r l e s s

120

Representat i on and i nteract io n

fig l.f OodgerooNoonuccal(0akleyetal.,1985)

, t h i sp r i m a r y - s c h otoelx t o f t h e c a m e r a . A b o r i g i npael o p l ei n
a c c i d e n t a lilny t h e d i r e c t i o n
for the pupilto enterintoan
not assubjects
book,aredepictedas objectsof contemplation,
imaginarysocialrelationwith.Immigrants,
by contrast,at leastoncetheyare in Australia,
are portrayedas peoplewith whom the pupilsshouldengagemore directly,and in a
friendlyway,as equals.
The choicebetween'offer' and 'demand',which must be madewheneverpeopleare
depicted,is not only usedto suggestdifferentrelationswith different'others',to make
viewersengagewith someand remaindetachedfrom others;it can also characterize
pictorialgenres.In somecontexts- for instance,
and the posed
televisionnewsreading
t
h
e
'
d
e
m
a
n
d
'
p
i
c
t
u
r
e
p
r
e
f
e
r
r
e
d
:
of
t h e s ec o n t e x trse q u i r ea s e n s e
is
m a g a z i npeh o t o g r a p h
and role modelsthey
connection
between
the viewersandthe authorityfigures,celebrities
featurefilm and televisiondramaand scientific
depict.In othercontexts- for example,
-the'offer' is preferred:
the
herea realor imaginarybarrieris erectedbetween
illustration
participantsand the viewers,a senseof disengagement,
in whichthe viewer
represented
participants
do not knowtheyare beinglooked
musthavethe illusionthat the represented
participants
mustpretendthattheyare not beingwatched.
at, andin whichthe represented
may in anotherbe a startlingmistakeor
And what in onecontextis accepted
convention
a n i n n o v a t i veex p e r i m e nFt .i l mt h e o r i s t (se . 9 .A l l e n ,1 9 7 7 ; W o l l e n t l g 9 2 )h a v eh a i l e dt h e
figure,but in television
look at the cameraas a daring,Brechtian,'self-reflexive'-style
n e w s r e a d i tnhge l o o ka t t h ec a m e r ai s c o m m o n p l a caen d ,w ew o u l dt h i n k ,n o te x a c t l y ' s e l f who looks at the camerain a
reflexive'- at least for the oresenters:
an interviewee
way. Not everyonemay
televisionnewsprogrammebreaksthe rulesin an unacceptable

Rep resent at i on and i nteract i on

addressthe viewerdirectly.some may only be lookedat, othersmay themselves

be the
b e a r e r so f t h e l o o k .T h e r ei s a n i s s u eo f c o m m u n i c a t i vpeo w e ro r ' e n t i t l e m e n t(' S a c k s ,
1992) involvedin this,not only in pictures,but also in everyday
face-to-face
communication,for instancein interactions
betweenmenanowomen:
As he answers
the girl'slaststatementhe beginstalkingand reaches
the pointwhere
normallyhe would look away,but insteadhe is still staringat her.This makesher
uncomfortable,
because
sheis forcedeitherto lock eyeswith him,or to look away
f r o m h i mw h i l eh e i s t a l k i n q I. f h e c o n t i n u et so t a l k a n ds t a r ew h i l es h ed e f l e c ths e r
eyes,it puts her into the 'shy'category,
whichsheresents.If shebo/dlylockseyes
w i t h h i m ,h e h a sf o r c e dh e ri n t oa ' l o v e r , sg a z e , , w h i cshh ea l s or e s e n t s .
( M o r r i s1, 9 7 7 i 7 6 )
Diagrams,
mapsandchartsare mostoftenfoundin contextsthatoffera kindof knowledge
which,in Westernculture,hastraditionallybeenvaluedhighly- objective,
dispassionate
l<nowledge,
ostensibly
f reeof emotiveinvolvement
andsubjectivity.
Hencethe'demand'has
beenrare in thesevisualgenres.But therewerecontextsin whichthe two formsof address
were combined.Schooltextbooksof the kind we usedas data whenwe wrote the first
editionof this book,for instance,constructeda progression
from 'demand,to 'offer,
pictures,and this not only in the courseof a chapter,as in the chapteron Antarctic
exploration
b ,u t a l s oi n t h e c o u r s eo f a w h o l eb o o ko r s e r i e so f b o o k sa n d ,i n d e e di ,n t h e
c o u r s eo f e d u c a t i oans a w h o l e- i l l u s t r a t i o nt hs a t s e r v e dt o i n v o l v e
s t u d e n tesm o t i v e liyn
the subjectmatterthengraduallydroppedout as higherlevelsofeducationwerereached.
I n s e n i o hr i g h - s c h otoelx t b o o kw
s ef o u n d' d e m a n d ' p i c t u r e
a st m o s ti n t h ec a r t o o nw
s hich,
in almostapologeticfashion,soughtto alleviatethe seriousness
of the text from time to
t i m e ,a s i n a c a r t o o ni n a g e o g r a p ht ye x t b o o k( B i n d o na n d w i l l i a m s ,r g B B )w h e r ea g i r l
lookeddespondently
at the viewer,with the words 'what doeshypothesis
mean?,in a
dialogueballoonemanalingfrom her mouth.In the contextof education,
the'demand,
p i c t u r ep l a y e da n a m b i v a l e nr o
t l e . 0 nt h e o n eh a n d i,t w a sn o t a h i g h l yv a l u e df o r m ,b u t a
f o r m d e e m e ds u i t a b l eo n l y f o r b e g i n n e r a
s , f o r m o n e g r e w o u t o f a s o n e c l i m b e dt h e
educational
ladder;on the otherhand,it playedan indispensable
role in educational
strategy: objectiveknowledgehad to be built, apparenily,upon a foundationof emotive
involvement,
of identification
with celebratorymythologies,
for instance.
Thisfoundation
wasthen,gradually,
repressed,
for if it was not repressed,
the l<nowledge
built on it could
n o t b e s e e nt o b e o b j e c t i v e . O u t s itdhee s p h e r eo f e d u c a t i o nt h, e v a l u eo f t h e , d e m a n d ,
picturedepended
on the assumed
educational
levelof the reader.When,for instance,
the
m a s sm e d i a( o r a u t o m a t itce l l e rm a c h i n e sb)e g a nt o u s e' d e m a n dp' i c t u r e st h, o s ee d u c a t e d
i n t h e l i n g u i s t iac n dv i s u a lg e n r e so f o b j e c t i v ke n o w l e d gaen d i m p e r s o n a ld d r e sw
s ould
h a v ef e l t p a t r o n i z e d , ' a d d r e s bs e ldo wt h e i r c l a s s 'T. h o s en o t s o e d u c a t e (do r t h o s ew h o
contested
the valueof suchan education)
wouldhavefelt that communication
hadbecome
moreeffective(andmorefun) thanwasthe casein the eraof more formalandimpersonal
p u b l i cc o m m u n i c a t i oAns. w e a l r e a d yd i s c u s s ei d
n t h e p r e v i o ucsh a p t e r st h, i s s i t u a t i o ni s
n o wc h a n g i nagn d ,w i t ht h eg r a d u adl i s a p p e a r a nocfet h es e m i o t idc i s t i n c t i o tnh, ec l a s sa n d

121

Renresentati on and i nteract i on

it supported(andthe differentvaluesand attitudesthat wereassociated

age distinctions
w i t h t h e s ed i s t i n c t i o n sa )l s ob e g a nt o e r o d e .
I t i s p o s s i b lteo r e l a t et h e m e a n i n gcso n v e y ebdy ' d e m a n d s ' a n d ' o f f e r s 'tthoe l i n g u i s t i c
r i r e c t l yr,e a l i z i n g
s d d r e stsh ev i e w e d
s y s t e mo f p e r s o nA. s w e h a v es e e n , ' d e m a npdi'c t u r e a
'l'.
'l'
a visual'you'.But this is not matchedby a visual The is absentin picturesQr,rather,
The 'demand'picturethereforeremindsmore of
hidingbehinda he/she/they.
objectified,
Where'yoLl'S'abOundbut
and instruCtions,
advertisements
the languageOf,for instance,
'l's'and'you's'are likely
'I'5' are rare,than,say,of the language
of personalletters,where
t o b e e q u a l l yc o m m o n . ' R e apl r o d u c e r s ' c a n n roet f e rt o t h e m s e l v edsi r e c t l yT. h e ym u s t
s p e a ki m p e r s o n a l lays, t r a d i t i o n a l liyn b u r e a u c r a t iacn d s c i e n t i f i lca n g u a g ew,h e r e ' I ' s '
Thepublic,on the otherhand,is addressed
were,and in manycasesstill arelalsorepressed.
'demands'derives
between'offers'and
directly.And yet,as we haveseen/the distinctiOn
e i n t e r tso f i n d w a y so f s a y i n g ' l ' i n t h e s e l f h i s t o r i c al yl f r o m a t t e m p t so f R e n a i s s a npc a
and status of independent
portraits which expressed
their new-foundself-confidence
artistsratherthan humblecraftsmen'
of'offer' and 'demand'can alsobe relatedto anotherkeyconceptin
But the concepts
we havetakenthe termsf rom Halliday's
that of the'speechact'.As mentioned,
linguistics,
f u n c t i o n s ' ah
s e c a l l st h e mi n h i s I n t r o h c t s( o r ' s p e e c h
d e s c r i p t i oonf f o u r b a s i cs p e e c a
ductionto FunctionalGrammar,1985).Eachof thesespeechacts,saysHalliday,is part
( a l t e r n a t i v es)o c i a l
o f a n i n t e r a c t i o n da yl a d a, n d h a si t s ' e x p e c t e d ' a nidt s ' d i s c r e t i o n a r y '
'offer
in which
that is,form a statement,
information',
Thusspeechactscan (1)
response.
soughtis'agreement',althoughthe statementmay of coursealso be
casethe response
(e.g. Wouldyou like a drink), in
they can (2) 'offer goods-and-services'
contradicted;
althoughthe offermayalsobe rejected;
is'acceptance',
response
whichcasethe expected
t h e yc a n ( 3 ) ' d e m a n di n f o r m a t i o nt'h, a t i s ,f o r m a q u e s t i o ni n, w h i c hc a s et h e e x p e c t e d
for instanceby
althoughthe listenermayalsodisclaimthe question,
is an answer,
response
saying1 don'tknowor I can'ttellyou that; andG),they can'demandgoods-and-services',
is for the
that is, constltutesomekind of command,in whichcasethe expectedresponse
mayof course
what he or shehasbeenaskedto do,althoughlisteners
listenerto undertake
alsorefuseto do so.
'mood',that is, by syntactic
Thesespeechactsare realizedby the linguisticsystemof
permutations
permutations,
of the orderof the subjectandthe finiteelementof the verbal
'offer of
tenseandmodality).The
group(i.e.theelementof the verbalgroupthat expresses
mood,in whichthefinite
is realizedbythe indicative
for instance,
information'(statement),
from Vogue"
elementfollowsthe subject,as in this sentence
Women(subject)cannot(finite) live by diamondsalone.

' D e m a n c f i ni ngf o r m a t i o n ' , t h e q u e s t i o n , i s r e a l i z e d b y t h e i n t e r r o g a t i v e m o o d , w h i c h h a s t h

' p o l a rq u e s t i o n( t' h ek i n do f q u e s t i otno w h i c h
s u b j e cfto l l o w i n tgh ef i n i t ei n t h ec a s eo f t h e
'WH-question'
'no'),
'yes'
jectfollowed
a WH-sub
and,in the caseof a
or
onecanjust answer
by the finite,or the finiteelementfollowedby the subject:

Representati on and i nteract i on

Can(finite) women(subject)live by diamondsalone?

Who (wn-subjecil could (finite) live by diamondsalone?
What could(finile) women(subject)live by?
T h e i m p e r a t i vm
e o o d /t h e ' c o m m a n d ,h, a s n o s u b j e c a
t t a l l a n d ,w h e nt h e p o l a r i t yi s
positive,
no finiteeither:
Don't (finite) live by diamondsalone
Live by diamondsalone
Thespeechact of'offeringgoods-and-services',
finally,is not realizedby a permutation
of
subjectand finite,but by variousidioms(e.g.Hereyou ard,by questions,
in conjunction
with specificmentalprocessverbsG.g. Do you wanta drink) or by commands(Havea
drinb and,indeed,
in variousotherways.
Thereare many subtypesof thesefour basicl<indsof speechact. Theyare realized
t h r o u g hs p e c i f icco m b i n a t i o nosf a d d i t i o n al il n g u i s t ifce a t u r e sA.' p r e d i c t i o nf' o
, rexample,
i s a n ' o f f e ro f i n f o r m a t i o n ' w i tf hu t u r et e n s ea n de i t h e rs e c o n do- r t h i r d - p e r s osnu b j e c to,r
first-personsubjectwitlr a'non-volitional'verb(e.g. Youwill live by diamondsalone,or I
. ' p r o m i s e ' i sa n ' o f f e r o f i n f o r m a t i o n ' w i tfhu t u r et e n s ea, f i r s t - p e r s o n
w i l l d i ey o u n g ) A
subjectand a volitionalverb G.g. I will buyyou diamondd.lt wouldtake us too far to
d i s c u stsh e s ei n d e t a i l T
. h ep o i n tt o r e m e m b ei rs t h a t i n l a n g u a gteh e r ea r e a f e w ' c o r e '
typesanda verylargenumberof furthertypeswhichare constructed
out of the coretypes.
s 'demandp
T h es a m ei s t r u e i n t h e c a s eo f i m a g e sA. v i s u a 'l i n v i t a t i o n , i a
, i c t u r ew i t h a
b e c k o n i nhga n da n d a s m i l i n ge x p r e s s i o an ;v i s u a l' s u m m o n sa' ,' d e m a n d ' p i c t u r ew i t h
a beckoninghand and an unsmilingexpression,'a
visual 'warning',a 'demand'picture
with a raisedforefingeranda sternexpression;
andso on.
Despite
t h e s eb r o a ds i m i l a r i t i e si t, w o u l ds e e mt h a t ' i m a g ea c t s ,d o n o t w o r k i n t h e
sameway as speechacts.Whenimages'effer',theyprimarilyoffer information.
0f course,
an image,sayan advertisingimage,may showsomeone
offeringsomething
to the viewer,
andthis offer may in fact be a real offer,whichcan be obtainedby writingto an address
specifiedin the advertisement.
But if there is suchan 'offer of goods-and-services'
in
images,it musttake the form of an 'offer of information'.It must be represented.Itcannol
beenacteddirectly.
when images'demand',they demand,one could say,the 'goods-and-services'
that
r e a l i z ea p a r t i c u l a sr o c i a lr e l a t i o n . 0 fc o u r s ea, n i m a g ec o u l ds h o wa g e s t u r eo f p u z z l e m e n t ,a ' s i l e n t ' q u e s t i o nb,u t t h e e x a m p l ei s s o m e w h acto n t r i v e a
d n d w o u l dn e e dv e r b a l
reinforcement,
or reinforcement
by a conventional
visualsign,for instance,
a question
mark.Thereis no imageact for everyspeech
act. But this neednot be so forever.
Although
l a n g u a gaen d i m a g ed o h a v et h e i rs p e c i f iac f f o r d a n c e s , w hcaatnb e , s a i da, n d ' d o n ew
, ith
i m a g e s( a n dw i t h l a n g u a g ed)o e sn o t o n l yd e p e n do n t h e i n t r i n s i ca n d u n i v e r s aclh a r a c t e r i s t i c so f t h e s em o d e so f c o m m u n i c a t i obnu, t a l s oo n h i s t o r i c a l layn dc u l t u r a l l sy p e c i f i c

123

rlq

Representation and interaction

social needs.It is quite possibleto extendthe semanticreachand use of imagesinto

ef l a n g u a g e
d o m a i nw
s h i c hf o r m e r l yw e r et h e e x c l u s i vper o v i n c o
a ,s i s a l r e a d yd o n eo, n a
s h e r ep e o p l ea r eu n l i k e ltyo h a v ea n yg i v e nl a n g u a gien c o m m o n( f o r
s m a l ls c a l ei ,n p l a c e w
example,
international
airports).

S I Z E O F F R A M EA N D S O C I A LD I S T A N C E
Thereis a seconddimension
to the interactive
meanings
of images,relatedto the'sizeof
frame',to the choicebetweenclose-up,
mediumshot and long shot,and so on. Just as
image-producers,
participants,
in depictinghumanor quasi-human
mustchooseto make
them look at the vieweror not,so they mustalso,and at the sametime,chooseto depict
themas closeto or far awayfrom the viewer- andthis appliesto the depictionof objects
also.And,lil<ethe choicebetweenthe 'offer' andthe 'demand',the choiceof distancecan
participantsand viewers.In handbool<s
suggestdifferentrelationsbetweenrepresented
a b o u tf i l m a n dt e l e v i s i opnr o d u c t i o ns ,i z eo f f r a m ei s i n v a r i a b ldye f i n e di n r e l a t i o nt o t h e
h u m a nb o d y E
. v e nt h o u g hd i s t a n c ies ,s t r i c t l ys p e a k i n g
a ,c o n t i n u u mt h, e' l a n g u a goef f i l m
and television'
has imposeda set of distinctcut-offpointson this continuum,in the same
way as languages
imposecut-offpointson the continuumof vowelswe can produce.
Thus
t h e c l o s es h o t( o r ' c l o s e - u p 's)h o w sh e a da n ds h o u l d e rosf t h e s u b j e c ta, n dt h e v e r yc l o s e
s h o t( ' e x t r e m e
c l o s e - u p ' , ' b icgl o s e - u p 'a) n y t h i n gl e s st h a nt h a t .T h e m e d i u mc l o s es h o t
cuts off the subjectapproximately
at the waist,the mediumshot approximately
at the
k n e e sT. h e m e d i u ml o n gs h o ts h o w st h e f u l l f i g u r e .I n t h e l o n gs h o tt h e h u m a nf i g u r e
o c c u p i easb o u th a l ft h e h e i g h ot f t h ef r a m ea, n dt h ev e r yl o n gs h o ti s a n y t h i n 'gw i d e r ' t h a n
that. Stylisticvariantsare possible,
but theyare alwaysseenand talkedabout intermsof
peopletalk of 'tight closeshots'or'tight framing',
this system,
as whenfilm andtelevision
o r a b o u t h ea m o u n o
t f ' h e a d r o o mi n
' a p i c t u r e( i . e .s p a c eb e t w e etnh et o p o f t h e h e a da n d
t h e u p p e fr r a m el i n e ) .
In everydayinteraction,
socialrelationsdeterminethe distance(literallyand figuratively)we keepfrom one another.EdwardHall (e.9.1966: 170-20) hasshownthat we
c a r r yw i t h u s a s e t o f i n v i s i b l b
e o u n d a r i ebse y o n dw h i c hw e a l l o wo n l yc e r t a i nk i n d so f
peopleto come.Thelocationof theseinvisibleboundaries
is determined
by configurations
- by whetheror not a certaindistanceallowsusto smellor touch
of sensorypotentialities
the otherperson/for instance,
and by how muchof the otherpersonwe can seewith our
p e r i p h e r a( sl i x t y - d e g r evei )s i o n . ' C l o spee r s o n adli s t a n c ei s' t h ed i s t a n caet w h i c h ' o n ec a n
peoplewho havean
holdor graspthe otherperson'and
thereforealsothe distancebetween
intimaterelationwith eachother.Non-intimates
cannotcomethiscloseand,if theydo so,it
personaldistance'is the distancethat
will be experienced
as an act of aggression.'Far
'extendsfrom a pointthat is just outsideeasytouching
distanceby one personto a point
wheretwo peoplecan touchfingersif they bothextendtheir arms',the distanceat which
'subjectsof personalinterestsand involvements
are discussed'.'Close
socialdistance'
b e g i n sj u s t o u t s i d et h i s r a n g ea n d i s t h e d i s t a n c e
at which'impersonb
au
l s i n e sosc c u r s ' .
'Far sociad
l i s t a n c e ' i s ' t h ed i s t a n c teo w h i c hp e o p l em o v ew h e ns o m e b o dsya y s" S t a n d

Representation and interaction

a w a ys o I c a n l o o ka t y o u " ' - ' b u s i n e s sa n ds o c i a il n t e r a c t i ocno n d u c t eadt t h i sd i s t a n c e

h a sa m o r ef o r m a la n d i m p e r s o n ac lh a r a c t etrh a n i n t h e c l o s ep h a s e ' . ' P u b l idci s t a n c e ' ,
finally,is anythingfurther than that,'the distancebetweenpeoplewho are and are to
r e m a i ns t r a n g e r sT' .h e s ej u d g e m e n tasp p l y o
, f c o u r s ew
, i t h i na p a r t i c u l a rc u l t u r e a, n d
H a l l c i t e sm a n ye x a m p l eos f t h e m i s u n d e r s t a n d i nwghsi c hc a n a r i s ef r o m i n t e r c u l t u r a l
differences
in the interpretation
of distance.
With thesedifferences
correspond
differentfieldsof vision.At intimatedistance,
says
Hall (1964),we seethe face or headonly.At closepersonaldistancewe take in the head
andthe shoulders.
At far personaldistancewe seethe otherpersonfrom the waist up.At
closesocialdistancewe seethe wholefigure.At far socialdistance
we seethe wholefigure
'with spacearoundit'.
And at publicdistancewe can seethe torsoof at leastfour or five
p e o p l eI .t i s c l e a rt h a tt h e s ef i e l d so f v i s i o nc o r r e s p o ncdl o s e ltyo t h et r a d i t i o n adl e f i n i t i o n s
of sizeof frame in film and television;in otherwords,that the visualsystemof size of
frame derivesfrom the'proxemics',
as Hall calls it, of everydayface-to-face
interaction.
Hall is aware of this and in fact acl<nowledges
the inffuenceof the work of Grosser,
a p o r t r a i tp a i n t e ro,n h i s i d e a sA. c c o r d i n tgo G r o s s e(rq u o t e di n H a l l ,I 9 6 6 i 7 I - Z ) , a t a
d i s t a n c eo f m o r et h a n 1 3 f e e t ( 4 m ) , p e o p l ea r e s e e n ' a sh a v i n gl i t t l e c o n n e c t i ow
nith
o u r s e l v e sa'n, dh e n c e ' t h e
p a i n t e cr a nl o o ka t h i sm o d e al s i f h ew e r ea t r e ei n a l a n d s c a p e
, nt h eo t h e rh a n d i,st h e' p o r t r a i t
o r a n a p p l ei n a s t i l l l i f e ' .F o u rt o e i g h tf e e t( r . 2 5 - 2 . 5 m ) o
distance':
t h e p a i n t e ri s n e a re n o u g hs o t h a t h i s e y e sh a v en o t r o u b l ei n u n d e r s t a n d i n
t hge
sitter'ssolid forms,yet he is far enoughaway so that the foreshortening
of the
f o r m sp r e s e n tnso r e a lp r o b l e mH. e r ea t t h e n o r m a d
l i s t a n c oef s o c i a il n t i m a c ya n d
easyconversation,
the sitter'ssoul beginsto appear.. . . Nearerthan three feet
[ 9 0 c m Jw
, i t h i nt o u c h i n g
distance
t h, es o u li s f a r t o o m u c hi n e v i d e n cf eo r a n ys o r to f
disinterested
observation.
- w h e t h etrh i s i s t h e m o r e
T h ed i s t a n c epse o p l ek e e pt,h e n ,d e p e n o
d n t h e i rs o c i a rl e l a t i o n
p e r m a n e nkt i n d o f s o c i a lr e l a t i o no n w h i c h H a l l m a i n l yc o n c e n t r a t e( st h e d i s t i n c t i o n
betweenintimates,friends,acquaintances,
strangers,
etc.) or the kind of socialrelation
that lastsfor the durationof a socialinteraction
andis determined
by the context(someone
in the audience
givenby an acquaintance
of a speech
or relativewouldnevertheless
stayat
p u b l i cd i s t a n c et h, e d i s t a n c o
e f t h e ' s t r a n g e r ' )B. u t t h e s ed i s t a n c easl s o a
, n da t t h e s a m e
time,determinehow muchof the otherpersonis in our field of vision- just as doesthe
f r a m i n go f a p e r s o ni n a p o r t r a i to r f i l m s h o t .
L i k et h e ' d e m a n dp' i c t u r et,h e c l o s e - ucpa m et o t h e f o r e i n t h e R e n a i s s a n cRei .n g b o m
( I 9 6 5 : 4 8 ) a r g u e st h a t i t h a si t s o r i g i ni n d e v o t i o n apli c t u r e sw, h e r ei t s e r v e dt o p r o v i d e
'the
" n e a r - n e s ss" o d e a r t o t h e G o d - s e e k i ndge v o u t ,I.n I t a l i a n a n d D u t c hp a i n t i n g s
o f t h e e a r l ys i x t e e n t che n t u r yi t a c q u i r e d
a ' d r a m a t i c ' f u n c t i o na,l l o w i n g' t h e s u b t l e sot f
e m o t i o n arle l a t i o n s h i w
p si t h a m i n i m u mo f d r a m a t i cs c e n e r y( ,p .4 B ) .
Thepeoplewe seein imagesarefor the mostpart strangers.
It is true that we seesome
o f t h e m ( p o l i t i c i a n fsi ,l m a n dt e l e v i s i osnt a r s s, p o r t sh e r o e se,t c . )a g o o dd e a lm o r et h a n

L25

L26

Representation and i nteract i on

others,but this kind ol lamiliaritydoesnot of itselfdetermine

whethertheywill be shown
in closeshot or mediumshot or long shot.The relationbetweenthe humanparticipants
represented
in imagesand the vieweris onceagain an imaginaryrelation.Peopleare
portrayedas thoughtheyare friends,or as thoughtheyare strangers.
Imagesallow us to
i m a g i n a r i cl yo m ea s c l o s et o p u b l i cf i g u r e sa s i f t h e yw e r eo u r f r i e n d sa n dn e i g h b o u -r so r
t o l o o ka t p e o p l el i k eo u r s e l v eass s t r a n g e r s , ' o t h e rIsn' t. h e p r i m a r y - s c h osool c i asl t u d i e s
textbookfrom whichwe havequotedseveralexamples,three
Aboriginalboysare shownin
longshot,occupying
onlyabouta quarterof the heightof the 'portrait'formatframe.The
c a p t i o nr e a d s , ' T h e s ep e o p l el i v e a t R e d f e r na, s u b u r bo f S y d n e y . ' T h e ya r e s h o w n
as'trees
impersonally,
as strangerswith whomwe do not needto becomeacquaintances,
in a landscape'.
Althoughtheydo lookat the viewer,
theydo sof rom sucha distancethat it
barelyaffectsus. Indeed,they are so small that we can hardlydistinguish
their facial
words.The caption,
features.'Theirsoul doesnot yet beginto appear',to useGrosser's
l da v eb e e n ,
significantg
l yi ,v e st h e mn o n a m e ;i n f a c t ,w h e r et h e m o r ef r i e n d l y ' b o y s ' c o u h
t h eq u i t ef o r m a l' p e o p l e ' h abse e nu s e d .
The portrait of the Aboriginap
l o e t O o d g e r o oN o o n u c c a(l f i g u r e 4 . 3 ) , a l r e a d y
mentioned
in the previoussectionof this chapter,is a tight closeshot.Sheis depictedin a
p e r s o n awl a y .I f t h i s w a s a l l w e c o u l ds e eo f h e r i n r e a l i t yw
, e w o u l db e c l o s ee n o u g ht o
a chapteron
touch her.As mentioned,
the sectionin which the photo occursconcludes
A b o r i g i n ei sn w h i c hn o o t h e rA b o r i g i nsem i l e sa t t h e v i e w e ri n t h i sw a y . 0 n eo f h e rp o e m s
i s q u o t e d : ' D a r ka n d w h i t eu p o nc o m m o ng r o u n d lI n c l u b a n d o f f i c ea n d s o c i a lr o u n d !
Y o u r st h e f e e lo f a f r i e n d l yl a n d l T h eg r i p o f t h e h a n d '( O a k l e ye t a l . , 1 9 8 5 : 1 6 4 ) .B u t
N o o n u c c a lm
' s e s s a gies n o t b o r n eo u t b y t h e w a y ' d a r k a n dw h i t e ' a r ep o r t r a y e di n t h e
chaoter.
Patternsof distancecan becomeconventionalin visual genres.In current affairs
television,for example,'voices'of differentstatus are habituallyframed differently:
t h e c a m e r a ' m o v eisn f o r b i g g e rc l o s e - u posf s u b j e c t w
s h o a r e r e v e a l i ntgh e i r f e e l i n g s ,
- the
whereasthe set-upfor the "expert" is usuallythe sameas that for the interviewer
'nominated'
'statused
(their
participants'tendto be
breastpocketshot'. Both kindsof
n a m e sa p p e a ro n t h e s c r e e ni n s u p e r i m p o s ec d
a p t i o n s a) n d ' h a v et h e i r c o n t r i b u t i o n s
(
B
r
u
n
s
d
o
n
, i s t a n c ei s
f r a m e da n d s u m m e du p '
a n d M o r l e yl,9 7 B i 6 5 ) . l n o t h e rw o r d s d
usedto signifyrespectfor authoritiesof variouskinds,on televisionas in face-to-face
interaction.
I n d i a g r a m tsh e h u m a nf i g u r ei s a l m o s ta l w a y ss h o w ni n m e d i u ml o n go r l o n gs h o tif he werea tree in a landscape'.
The picturesin figure4.4 illustrateda
objectively,'as
front-pagenewspaper
The diagramsshowexactly
story about a murdercasein Sydney.
what happened,
from an objectifyingand impersonaldistance(and from a high angle).
The close-upphotosaccompanytestimoniesby former patientsof the victim, but are
represented
as also'friends'of we readersof the SydneyMorning Herald,andthereforeas
peoplewhoserelationwith the victim we shouldidentifywith. As in the chapteron
Antarcticexploration,
are
the personalandthe impersonal,
the emotiveandthe detached,
combined.
participants,
So far we havediscussed
socialdistancein relationto human-represented

Representation and interaction

. I2-l

}II5 PATIENTS

fig l.+ The murder of Dr Chang(SydneyMotning Heratd,sJuly799L)

, e s y s t e mo f s o c i adl i s t a n c cea na p p l ya l s o
b u t u n l i k et h e s y s t e mo f ' o f f e r ' a n d' d e m a n dt' h
to the representation
of objectsand of the environment.
As sizeof frame is traditionally
definedin terms of specificsectionsof the humanbody,beginning
studentsof film and
television
are oftenat a lossas to whichtermsto usefor describing
shotsof objectsand
landscapes.
Thescaleof sevensizesof frameseems
Thereare no clear-cut
too fine-grained.
equivalents
for the shoulder,
the waist,the knees.And objectscome in many different
shapesand sizes.We wouldnevertheless
suggestthat at leastthreesignificantdistances
can be distinguished,
andthat thereare correspondences
between
thesedistances
and our
everydayexperience
of objectsand the environment,'
in otherwords,that size of frame
c a na l s os u g g e ssto c i arl e l a t i o nbse t w e etnh ev i e w e a
r n do b j e c t sb,u i l d i n gasn dl a n d s c a p e s .
At closedistance,
we wouldsuggest,
the objectis shownas if the vieweris engaged
with
it as if he or she is usingthe machine,readingthe book or the map,preparingor eating
t h e f o o d .U n l e s st h e o b j e c ti s v e r ys m a l l ,i t i s s h o w no n l y i n p a r t ,a n d o f t e nt h e p i c t u r e
includes
the user'shand,or a tool - for instance,
a knifescrapingthe soft margarinein an
a d v e r t i s e m e nFti.l m a n d t e l e v i s i o n ' c u t a w a y s ' ( ' o v e r s h o u l doef rosb' )j e c t si,n w h i c ht h e
o b j e c t s h o w na r e i n t e g r a t ei dn t oa n a c t i o nt h r o u g ht h ee d i t i n gu, s e t h i sd i s t a n c e . Amt i d d l e
distance,
the objectis shownin full, but withoutmuchspacearoundit. It is represented
as

r2a

Rep resentation and i nteracti on

w i t h i nt h e v i e w e r ' sr e a c h b
, u t n o t a s a c t u a l l yu s e d T
. h i st y p e o f p i c t u r ei s c o m m o ni n
productis shownin full, but from a fairly closerange,and a
advertising:
the advertised
At long
steepangle,as if the viewerstandsjust in front of thetableon whichit is displayed.
the viewerandthe object.Theobjectis there
distancethereis an invisiblebarrierbetween
f o r o u r c o n t e m p l a t i oonn l y ,o u t o f r e a c h a, s i f o n d i s p l a yi n a s h o pw i n d o wo r m u s e u m
of the EuropeanPlayStationwebsite,in figure 4.5, usesboth
exhibit.The screenshot
puttingthe closeshoton the right,as the
significantly
middledistanceand closedistance,
' N e w ' ( s e ec h a p t e6r ) .
of buildings
Thesamekind of distinctions
can be madewith respectto representations
aboutto enterit, in
and landscapes.
We can seea buildingfrom the distanceof someone
w h i c hc a s ew e w i l l n o t s e et h e w h o l eo f t h e b u i l d i n ga,s i s ( a g a i n )o f t e nt h e c a s ei n f i l m
shotsin whichthe buildingis relatedto someaction.Wecanalsoseeit from the distanceof
s o m e o nw
e h o j u s t i d e n t i f i e idt a s h i so r h e rd e s t i n a t i o an n, d i s s u r v e y i nigt f o r a m o m e n t ,
g n dl e a v eo u t
o n l yt h eb u i l d i n a
b e f o r em o v i n g
t o w a r d si t . I n t h a tc a s et h ef r a m ew i l l i n c l u d e
environment.0r
we canseeit, soto speak,from behindthe gatesthat keep
the surrounding
the publicat a respectfuldistancefrom the palace/or the fortress,or the nuclearreactor,
a n d i n t h a t c a s et h e r e p r e s e n t a t iw
on
i l l i n c l u d ea l s ot h e s p a c ea r o u n dt h e b u i l d i n gL. a n d with a foreground
scapes/
too, can be seenfrom within;from a kind of middledistance,
perhaps,
that the vieweris imaginarilylocatedwithin the landscape,
objectsuggesting,
but stoppingfor a moment,as if to take stockof what is ahead;or from a longdistance,
f r o m t h e a i r ,p e r h a p so,r f r o m a ' l o o k o u t ' p o s i t i o na,p l a c en o t i t s e l fi n t h e l a n d s c a pbeu t
illustrationsin
in manyof the photographic
affordingan overviewof it, as,for instance,
geography
textbooks.

fig a.S Playstationwebsite(http://eu.playstation.c0m/europe-select.jhtml)

Representati on and interacti on

r29

t a y i n w h i c hs o c i a l
W e w i l l e n d w i t h s o m eb r i e f c o m m e n t so n t h e v e r y d i f f e r e n w
i n t h e E n g l i s hl a n g u a gm
e a i n l yt h r o u g hp e r m u t a t i o ni ns t h e f o r m a l i t y
d i s t a n c ies r e a l i z e d
spokenperhaps
is a kindof personallanguage,
of style(seeJoos,i-967).Intimatelanguage
o n l yb yt h e m e m b e rosf a c o u p l eo r f a m i l yo, r b y a g r o u po f s c h o oflr i e n d sT. h es p e a l < eor fs
, a m e sw h i c h
h a v es p e c i anl a m e sf o r e a c ho t h e r n
s u c ha ' l a n g u a g eo f i n t i m a t e s ' o f t e n
. n dt h e l a n g u a giet s e l fi s m i n i m al yl a r t i c u l a t e da: h a l f - w o r di s
o u t s i d e rdso n o t g e tt o u s eA
voicequality,
eyecontact,intonation,
eachother.Facialexpressions,
enoughto understand
and peoplewho are in an intimaterelationwith eachother
etc.carrymostof the meaning,
language
in thlsway.'Personal/
conveyed
becomefinelyattunedto the readingof meanings
. o n - v e r b ae lx p r e s s i osnt i l l c a r r i e s
i s c a s u a lw, i t h a g o o dd e a lo f c o l l o q u i a l i samn ds l a n g N
' S o c i a ll a n g u a g e ' ,
m u c ho f t h e m e a n i n gb u t n o t s o m u c ht h a t ' h a l f a w o r d i s e n o u g h ' .
t h o u g hs t i l l c o l l o q u i aal ,l r e a d yb e g i ntso i n t r o d u cae h i n to f f o r m a l i t yA. n d t h e r ei s ,i n t h i s
k i n do f s i t u a t i o nl ,e s ss h a r i n go f i n f o r m a t i oann d a s s u m p t i o nTsh. e l a n g u a gnee e d st o b e
m o r e a r t i c u l a t em
, o r e v e r b a l l ye x p l i c i ts, o t h a t n o n - v e r b aelx p r e s s i oins n o l o n g e ra s
i m p o r t a nat s i n i n t i m a t ea n d p e r s o n aslt y l e .P u b l i cl a n g u a g fei ,n a l l yi,s t h e l a n g u a gues e d
s o n o l o g i cl i:s t e n e rns o l o n g e r
e e c o m em
i n m o r eo r l e s sf o r m a la d d r e s sH. e r el a n g u a g b
p a r t i c i p a ta
e s t h e yd o i n t h e o t h e rs t y l e so f s p e e c hS. p e e c hi s n o l o n g e ri m p r o v i s e bd u, t
a n do t h e r
t h o u g h to u t i n a d v a n c ep,e r h a p es v e nf u l l y o r p a r t i a l l yw r i t t e no u t .I n t o n a t i o n
to controlas syntax
expression
becomeas formal,as muchsubjected
formsof non-verbal
a n d w o r d u s a g e .S p e e c hm u s t b e f u l l y e x p l i c i t ,m e a n i n g sf u l l y a r t i c u l a t e dv e r b a l l y .
y u s tb ee m p l o y e dW. r i t e r s
a rseo u t o f p l a c ea n da m o r ef o r m a lv o c a b u l a rm
Colloquialism
evenwhenwe are
even
intimates,
us
as
friends
or
use
styles
address
these
to
can of course
j
u
s
t
p
e
o
p
l
e
a
r
e
a n dw i l l r e m a i n
g
i
v
e
p
i
c
t
u
r
e
s
w
h
o
,
i
n
r
e
a
l
i
t
y
,
o
f
u
s
a
s
c
a
n
c
l
o
s
e
u
p
s
not,
w
h
i
c
h
w
e a r ea d d r e s s e d
w
i
t
h
c
h
u
m
m
y
i
n
f
o
r
m
a
l
i
t
y
t
o
u
s
t
h
i
n
k
o
f
t
h
e
c
o
l
l
o
q
u
i
a
l
,
strangers
in manyadvertisements.

A N D T H E S U B J E C T I V EI M A G E
PERSPECTIVE
parThereis yet anotherway in whichimagesbringaboutrelationsbetweenrepresented
c
h
oice
p
e
r
s
p
e
c
t
i
v
e
.
n
o
t
o
n
l
y
t
h
e
i
n
v
o
l
v
e
s
a
n
i
m
a
g
e
P
r
o
d
u
c
i
n
g
a
n
d
t
h
e
v
i
e
w
e
r
:
ticipants
'offer'
'demand'
and
at
but
also,
of
frame,
a
certain
size
selection
of
and
and
the
between
p
o
s
s
i
b
i
l
i
t
oyf
i
m
p
l
i
e
t
s
h
e
v
i
e
w
'
a
,
n
d
t
h
i
s
a
'
p
o
i
n
t
o
f
s
e
l
e
c
t
i
o
o
n
f
a
n
a
n
g
l
e
,
t h es a m et i m e ,t h e
By
otherwise.
participants,
human
or
represented
subjective
attitudes
towards
expressing
do not meanthattheseattitudesare alwaysindividualand
attitudes',we
saying'subjective
attitudes.But theyare always
will
see
they
are
oftensociallydetermined
that
unique.We
a
n
d
u
n
i
q u eT. h es y s t e mo f p e r s p e c t i v e
i
n
d
i
v
i
d
u
a
l
a
s
t
h
o
u
g
h
t
h
e
y
w
e
r
e
s
u
b
j
e
c
t
i
v
e
,
encoded
a
i
n
R
e
n
a
i
s
s
a
n
c
ep,e r i o di n w h i c hi n d i v i d u a l i t y
t
h
e
d
e
v
e
l
o
p
e
d
w h i c hr e a l i z e s ' a t t i t u d e ' w a s
v
a
l
u
e
s
a
,
n
d
i
t
d
e
v
e
l
o p epdr e c i s e tl yo a l l o wi m a g e s
b
e
c
a
m
i
e
m
p
o
r
t
a
n
s
t
o
c
i
a
l
a n ds u b j e c t i v i t y
while thesewere the
points
of
view.
Paradoxically,
informed
by
subjective
to become
g
e
o
m
e
t
r
i
c
f
o
u n d a t i o na ,c o n s t r u c p
e
r
s
p
e
c
t
i
v
o
n
a
n
i
m
p
e
r
s
o
n
a
l
,
r
e
s
t
s
e
n
c
o
d
e
d
,
meanings
'
r
e
c
o
r
d
i
n
g
'
i
m
a
g
e
s
q
u
a
s
i
m
e
c
h
a
n
i
c
a
l
r
e
a
l
i
t
yS. o c i a l l d
y etermined
o
f
o
f
i
s
w
y
a
t i o nw h i c h
'studiesof nature',faithful
presented
as
and
in
way,
be
naturalized,
could,
this
viewpoints

130

Representationandinteraction

c o p i e so f e m p i r i c arl e a l i t y . 0 n l yr e c e n t l yh a s i t b e c o m ep o s s i b l a
e g a i nt o s e et h a t p e r s p e c t i v ei s a l s o ' a d a r i n ga b s t r a c t i o n('H a u s e r1, 9 6 2 : 6 9 ) , a n dt o d i s c u s si t s s e m i o t i c
effects,for instance,
in film theory(e.g.Comolli,I97I).
Pre-Renaissance
forms,frescoeson the wall of a churchnave,for example,
or mosaics
in the domedroof of a church,did not haveperspective
to positionthe viewer.Viewersof
suchworkswerepositioned,
not by the internalstructureof the work,but by the structure
of its environments,
both the immediateenvironment
of the church,its proximityto the
altar,for instance,
and the wider socialenvironments.
In otherwords,the syntaxof the
objectdepended
for its completion,
its closure,
not on a particularrelationwith the viewer
but on a particularrelationwith its surroundings,
andthe pointof viewwasthe positionthe
v i e w ea
r c t u a l l yt o o k u p i n r e l a t i o nt o t h e i m a g e : ' T hw
e o r l di n t h e p i c t u r ew a se x p e r i e n c e d
a s a d i r e c tc o n t i n u a t i oonf t h e o b s e r v e r o
' sw ns p a c e(' A r n h e i m , ! 9 7 4 : 2 7 4 .A s a r e s u l t ,
the viewerhad a certainfreedomin relationto the object,a degreeof what, today,we
wouldcall 'interactiveuse'of the text,albeitin the contextof a highlyconstrained
social
order.Fromthe Renaissance
onwards,
visualcomposition
becamedominated
by the system
of perspective,
with its single,centralizedviewpoint.The work becamean autonomous
object,detachedfrom its surroundings,
movable,producedfor an impersonalmarket,
ratherthan for specificlocations.
A frame beganto separatethe represented
world from
the physicalspacein whichthe imagewasviewed:at the time perspective
was developed,
picturesbeganto bef ramedprecisely
to createthis division,
to markoff the imagefrom its
environment,
and turn it into a kind of 'windowon the world,.At the sametime,images
b e c a m em o r ed e p e n d e notn t h e v i e w e rf o r t h e i r c o m p l e t i o nt h, e i r c l o s u r ea, n d v i e w e r s
becamemore distancedfrom the concretesocialorder in whichthe world had formerly
beenembedded:
they now hadto learnto internalize
greater
the socialorder.Thisyieldecl
freedomwith respectto the immediate,
concretesocialcontext,but diminished
freedomin
r e l a t i o nt o t h e w o r k .A p a r a l l ecl a n b e m a d ew i t h t h e d e v e l o p m e nwt sh i c h ,m o r eo r l e s s
s i m u l t a n e o u st loyo, k p l a c ei n m u s i c( s e es h e p h e r d , r 9 7 7 ) . l nm e d i e v aml o d e sb, a s e da s
theywereon the pentatonic,
any noteof the scalecouldstandin onlyintervallicrelationto
a n yo t h e rn o t e .H e n c ea n yn o t ec o u l dp r o v i d e
a s e n s eo f r e s o l u t i o no,f c l o s u r eI .n t h e n e w
diatonicmusica strict hierarchywas established
betweenthe fundamentals,
so that any
melody,
whateverthe harmonicprogressions
it traversed,
hadto return,ultimately,
to the
samepredetermined
note,the'tonic',in the keyof whichthe piecewasscripted.Thenotes
in musicthus relateto the keycentrein the samefixedway in whichviewersrelateto the
perspectival
centreof the visualwork.
Thereare,then,sincethe Renaissance,
two kindsof imagesin westerncultures:subjectiveand objectiveimages,
(andhencewith a'built-in,
imageswith kentral) perspective
) e r s p e c t i v(ea n dh e n c ew i t h o u ta ' b u i l t - i n ,
p o i n to f v i e w )a n d i m a g e sw i t h o u t( c e n t r a l p
point of view).In subjectiveimagesthe viewercan seewhat there is to seeonly from a
particularpointof view.In objectiveimages,
the imagerevealseverything
thereis to know
(or that the imageproducedhasjudgedto be so) aboutthe represented
participants,
even
if,to do so,it is necessary
to violatethe lawsof naturalistic
depictionor indeed,
the lawsof
nature.Thehistoryof art hasmanystrikingexamples
of this- for example,
the sculptures
o f w i n g e db u l l sa n d l i o n sw h i c hf f a n k e d
t h e d o o r so f A s s y r i a tne m p l e sf :r o mt h e s i d et h e s e

Rep resentati on and i nteract io n

I31

so as to
had four movinglegs,and from the front two stationarylegs,five altogether,
provide,from everyside,a view from which no essentialparts were missing.Modern
technicaldrawingsmay still showwhat we knowaboutthe participantsthey represent,
at them in
what is objectively
there,ratherthan what we wouldseeif we were lool<ing
reality,ratherthan what is subjectively
there.If we were,in reality,to seethe front of the
we wouldnot at the same
is (a square),
cubein figure4.6 the way we knowit 'objectively'
t i m eb ea b l et o s e et h et o p a n dt h es i d e I. t i s a n i m p o s s i bpl ei c t u r e( o r a p o s s i b lpei c t u r eo f
a highlyirregularhexahedron,
ratherthan a cube)from the pointof viewof what we can
assemblyinstructionsfor a pieceof
see in reality.Yet in many contexts(for instance,
0 .b j e c t i v ei m a g e st ,h e n ,
f u r n i t u r e )a n ' o b j e c t i v ep' i c t u r el i k et h i s i s e n t i r e l ya c c e p t a b l e
of who or whereor
disregardthe viewer.Theysay,as it were,'l am this way,regardless
w h e ny o ua r e . '
perspectival
imagehasbeenselectedfor
By contrast,
the pointof viewof the subjective,
the viewer.As a resultthere is a kind of symmetrybetweenthe way the image-producer
participants,
alsoreiate
relatesto the represented
andthe waythe viewermust,willy-nilly,
s ,u t a l s o
t o t h e m .T h e p o i n to f v i e wi s i m p o s e d
n o t o n l yo n t h e r e p r e s e n t epda r t i c i p a n t b
in the originalsenseof
on the viewer,
andthe viewer's'subjectivity'is thereforesubjective
t h e w o r d ,t h e s e n s eo f ' b e i n gs u b j e c t etdo s o m e t h i nogr s o m e o n eI'n. a s h o r te s s a yo n
t a sc o m m e n t eodn t h i s :
C h i n e saer t , B e r t o l tB r e c h h
Theydo not liketo see
As we know,the Chinese
do not usethe art of perspective.
e v e r y t h i nfgr o m a s i n g l ep o i n to f v i e w .C h i n e s ceo m p o s i t i otnh u s l a c k st h e c o m p u l s i o nt o w h i c h w e h a v e b e c o m ea l t o g e t h ear c c u s t o m e .d. . a n d r e j e c t st h e
subjugation
of the observer.
( B r e c h t\,9 6 7 : 2 7 8 - 9 )
n a p e r i o di n w h i c h
T h es y s t e mo f p e r s p e c t i vi sef u n d a m e n t a lnl ya t u r a l i s t i cI t. d e v e l o p ei d
(which
wasalso/and
divine
order
a
the worldof naturewas no longerseenas manifesting
order
and
ultimately
meaningless
at the sametime,a socialorder),but as an autonomous

fig l.o Frontal-isometric

cube

132

Representation and i nteracti on

w h o s el a w sa l s og o v e r n etdh e c o n d u c o
t f p e o p l eI.t w a s e x p l i c i t l yg r o u n d e d
in the new
scientificspirit,legitimized
by the authorityof scientificobservation
andthe physicallaws
of nature.Thenewmusic,similarly,
wasconstructed
as congruent,
not with a (divineand)
s o c i aol r d e rb, u t w i t h t h e p h y s i c al la w so f s o u n d .
In the late nineteenth
century,after centuriesof hegemony,
both systemscame into
, e l v e - t o nme u s i c )a sw e l la s i n t h e p o p u l a ar r t s .F i l m ,f o r
c r i s i si,n t h e h i g ha r t s ( C u b i s mt w
e x a m p l es,t i l lu s e sp e r s p e c t i viaml a g e sb,u t ,i n a n e a r - C u b ifsat s h i o np,r o v i d em
s u l t i p l ea n d
c o n s t a n t lsyh i f t i n gv i e w p o i n tisn i t s e d i t i n gM
. o d e r nt e l e v i s i o n
e ,s p e c i a l li yn p r o g r a m m e s
not basedon the modelof film, suchas newsprogrammes,
has gonea stepfurther,and
c h a l l e n g epse r s p e c t i va el s ow i t h i nt h e i m a g eA. n e w s r e a d m
e ra yh a v eb, e h i n dh i mo r h e ro, n
t h e l e f t a v e r b atl e x t ,a n do n t h e r i g h ta c h r o m a - k e y e
mdo v i n gp i c t u r eo n t h e w a l l ( a w a l l
whichis in fact a kindof two-dimensional
screenon whichto projecta'layout',andin f ront
of whichto positionthe newsreader).
Modernmagazine
andwebsitelayoutsform another
c a t e g o ro
y f v i s u aw
l o r k sw h i c ha r e n o l o n g e b
r a s e ds o l e l yo n t h e c o m p o s i t i o nparl i n c i p l e s
of perspective.
0f course,theystill containmanyperspectival
imagesbut thesehavebeen
subordinated
to a structurethat can no longerbe saidto be perspectival.
Two examples
m a yi l l u s t r a t teh i s .
T h ep i c t u r eo n t h e F o r dM o n d e ow e b s i t e( f i g u r e4 . 7 ) i s n a t u r a l i s t i W
c . h a tw e o b s e r v e
herecouldalsobe observed
in reality.Therecouldbea car positioned
in thisway,in front of
t h i s p a r t i c u l acr o u p l ea n dt h i s p a r t i c u l a br u i l d i n gA. s a r e s u l to f t h e a n g l ea n dt h e s o c i a l
d i s t a n c (ea l o w - a n g l e ' l o nsgh o t 'w
, i t ht h ec a r i n t h ef o r e g r o u n dv)i,e w e ras r et h e nm a d et o
r e l a t et o t h e r e p r e s e n t epda r t i c i p a n ti sn a c e r t a i nw a y .T h e ya r em a d et o ' l o o k u pt o ' t h e m ,
andtheyare madeto seethemas if theynoticethe car andthe stylishcouplefrom across
the street,with envy.In the pictureon the FordFiestapage(figure4.8),on the otherhand,
the viewer,ratherthan beingpositionedin the naturalworld,is confrontedwith a world

[]]e*i#'l*tltrN"l**:S.
lans*c{rr*{rillr}s}st

ffi

hle$*-l.xlk

Msmd*rail

fig a.Z NewJookFordMondeo(www.ford.co.uMe/mondeo)

p*.l""r"*"w[i6'ffi-

!il

T}}* Mwd*a
h$$ I
f&r|1, d{e.tldattre d6r{Ss
lsp.?iratin!
s** t*vd;$
s? {ra*qpsr*ship
sd
k{hsol*q?.
e *!*
fa*il1 of reglx*r *Sei
sltrlr*dlrS
pd$oeas#
&ed t!61
s{ticrs*art clf isatcl* }rs
C4t doirrlesr.
Aid
v'tUblte f+-ffinFree;$4
treq*$l*ene P?*tq$io^
Sf!18a" t|:r f$d
l4eed*6 i5 oee *{ tiie
6tf<!t pl*ei
t$ be.

and interaction . I33

Representation

qffi

ir;r luvrr,6g
{..m*u'gr**t
t,a'st
i:r'{.pt$*+itr6q}'r.}
'i...' P

xali_.

neaixr|.oc***rffi
al,*,

t'

..1-.e3{*

;:...+:

fig n.A FordFiestaRockSolid(http://www.f0rd.co,uk)

which openly presentsitself as a semioticconstruct,mixing perspectivaland nonperspectival

elements
in sucha wayasto givethe appearance
of a continuumof formsfrom
t h e r e p r e s e n t a t i o nt a
o l t h e s i g n i f i c a t i o n awl h
, i l e t h e v i s u a la s a w h o l e r e m a i n sn o n n a t u r a l i s t i ct h: e c a r ,o n t h i s p a g ec, a n n o tb e s a i dt o b e ' b e h i n d ' t h eL a s V e g a sr o a ds i g n
o r ' b e h i n d ' t h ew o r d' r o c ks o l i d 'i n t h e w a y t h a t t h e c o u p l ea n dt h e b u i l d i n gi n f i g u r e4 . 7
c a n b e s a i dt o b e b e h i n dt h e F o r dM o n d e o . ' l nf r o n t o f ' a n d ' b e h i n dl'o s et h e i r i d e a t i o n a l
d i m e n s i oa
nn
, d b e c o m tee x t u a lp r i n c i p l eosn l y T
. h et w o p a g e st h u se x e m p l i fay s h i f tf r o m
the dominance
of natureto the dominance
of the
of signification,
andfrom the dominance
- in a way very similarto that whichwe
perceptual
to the dominance
of the conceptual
observed
in chapterl whenwe comparedBaby'sFirst Book(figure1.1) to Dicl<Bruna's
0n My Walk and 0n TheFarm (figureI.2).

I N V O L V E M E N TA N D T H E H O R I Z O N T A A
L NGLE
W h e nw e p r o l o n gt h e c o n v e r g i npga r a l l e lfso r m e db y t h ew a l l so f t h e h o u s e isn f i g u r e4 . 9 ,
they cometogetherin two vanishingpoints.Both pointsare locatedoutsidethe vertical

I34

Representation and interaction

rig e.r Aborigines(oakleyet a1.,L985)

g o i n t sa l l o w u s t o
b o u n d a r i eosf t h e i m a g ea/ s s h o w ni n f i g u r e4 . 1 0 . T h e s ev a n i s h i n p
reconstructwhat we can seeevenwithout the aid of geometricalprojection:the scene
has not situatedhimself
hasbeenphotographed
from an obliqueangle.The photographer
themfrom the side.
or herselfin front of the Aborigines,
but hasphotographed
Figure 4.10 shows how the positionfrom which the photo was taken can be
pointsin sucha way that they meetto
reconstructed
by droppinglinesfrom the vanishing
f o r ma 9 0 oa n g l eo n t h e l i n ed r a w nt h r o u g ht h ec l o s e sct o r n e ro f t h e c o t t a g e sF.i g u r e4 . 1 1
showsthe scenefrom above.Theline(ab) represents
the frontalplaneof the subjectof the
photograph:
is alsothe
which,as it happens,
the lineformedby the front of the cottages,
l i n ea l o n gw h i c ht h e A b o r i g i n easr e l i n e du p .T h e l i n e( c d ) r e p r e s e n t h
s e f r o n t a lp l a n e
(and henceof the viewer).Had thesetwo linesbeenparallelto one
of the photographer
another,
the horizontalanglewouldhavebeenfrontal- in otherwords,the photographer
would havebeenpositionedin front of the Aboriginesand their cottages,facingthem.
Instead,the two linesdiverge:the angle is oblique.The photographer
has not aligned
but viewedthem 'from the
himselfor herselfwith the subject,not facedthe Aborigines,
s i d ei n
l es'.
Horizontalangle,then,is a functionof the relationbetweenthe frontal planeof the
participants.
image-producer
Thetwo can either
andthe frontal planeof the represented
be parallel,alignedwith oneanother,
or form an angle,divergefrom oneanother.

Representati on and i nteract i on

pointsof'Aborigines'(figure4.9)
fiq e.fO Schematicdrawing:vanishing

*"-9oot>5

r^
(zDr
- !>46>Cn-(--e-I

fiq +.ff Schematicdrawing:topyiew0f'Aborigines'(figure4.9)

T h ei m a g ec a nh a v ee i t h e ra f r o n t a lo r a n o b l i q u ep o i n to f v i e w I. t s h o u l db e n o t e dt h a t
t h i si s n o ts t r i c t l ya n e i t h e r / odr i s t i n c t i o n
T.h e r ea r ed e g r e eosf o b l i q u e n e sasn,dw e w i l l ,i n
f a c t , s p e a ko f a f r o n t a la n g l es o l o n ga s t h e v a n i s h i n pg o i n t ( s )s t i l l f a l l ( s ) w i t h i nt h e
v e r t i c abl o u n d a r i eosf t h e i m a g e( t h e ym a yf a l l o u t s i d teh e h o r i z o n t abl o u n d a r i e s ) .

t35

136 .

Reoresentation and interaction

F i g u r e4 . I 2 h a s a f r o n t a la n g l e A
. s s h o w ni n f i g u r e4 . I 3 , t h e r e i s o n l y o n e m a j o r
, n d i t l i e si n s i d et h e v e r t i c a bl o u n d a r i eosf t h e i m a g eF. i g u r e4 . 1 4 s h o w s
v a n i s h i npgo i n t a
h o w t h ef r o n t a lp l a n eo f t h ep h o t o g r a p h(el ri n ea b )a n dt h ef r o n t a lp l a n eo f t h e r e p r e s e n t e d
participants(line cd) run parallel- that is, if one only considersone set of represented
participants,
the teachers,
the blackboardand the readingchart.Thefrontal planeof the
A b o r i g i n acl h i l d r e n( l i n ee f ) m a k e sa n a n g l eo f n i n e t yd e g r e ew
s i t h t h e f r o n t a lp l a n eo f
the teachersand with the frontal planeof the photographer.
TheAboriginalchildrenhave
b e e np h o t o g r a p h e
f rdo m a v e r yo b l i q u e
angle.
The differencebetweenthe obliqueand the frontal angle is the differencebetween
detachment
and involvement.
The horizontalangleencodes
whetherthe image-producer
( a n dh e n c ew, i l l y - n i l l yt h, e v i e w e r )i s ' i n v o l v e d ' w i t h
t h e r e p r e s e n t epda r t i c i p a n tosr n o t .
Thefrontal anglesays,as it were,'What you seehereis part of our world,something
we
are involved
with.' Theobliqueanglesays,'Whatyouseehereis nol part of our world;it is
theirworld,somethingweare not involved
with.' Theproducers
of thesetwo photographs
have,perhapsunconsciously,
alignedthemselves
with the whiteteachersandtheir teaching
tools,but not with the Aborigines.
The teachersare shownas 'part of our world',the
'other'.
Aboriginesas
And as viewerswe haveno choicebut to seetheserepresented

fig l.fe Aboriginalchildfenat scho0l(Oakleyetal.,1985)

Representation and interaction

=
lI
-. I I

--_\

ffi

\d

point 0f'Aboriginalchildrenat school'(figure4.12)

fig e.ff Schematicdrawing:vanishing

- z - - - @ - - -: g
'--Jr

M0 l
iolloi
plEl
l:l

l^ I

&
()

fiO C,f+ Schematicdrawing:topviewof'Aboriginalchildrenat school'(figure4.12)

participantsas they havebeendepicted.

We are addressed
as viewersfor whom'involvement' takestheseparticularvalues.In reality,they might not - we might be Aboriginal
viewers,for example.It is onethingfor the viewerto be limitedby what the photograph
shows(and to understandwhat this means/for exampleexclusion,in the case of an
A b o r i g i n avli e w e r )i;t i s a n o t h e trh i n gt o a c t u a l l yi d e n t i f yw i t h t h e v i e w p o i net n c o d e d
in
the photo.We can acceptor reject,but eitherway we first needto understand
what is
meant.
The primary-school
socialstudiestextbook)ur Societyand Othersprovidesa further

L37

r38

Representation and i nteracti on

i l l u s t r a t i o nA. s h o to f t h e N e wS o u t hW a l e sP a r l i a m e nHt o u s ei n S y d n e yi s f r o n t a l a
, nd
t a k e nf r o ma l o wa n g l eA. s h o to f t h ec h u r c hi st a k e nf r o ma n o b l i q u e
a n ds o m e w h ahti g h e r
a n g l eT. h ef o r m e ri l l u s t r a t eas s e c t i o na b o u tS y d n e yi n t h e c h a p t e r ' W h aits a C i t y ? ' t; h e
l a t t e ra, s e c t i o n
a b o u ta M a o r if a m i l yi n t h ec h a p t eor n i m m i g r a n t sR.e l i g i oins d e p i c t e a
ds
s o m e t h i nw
g h i c h ,i n t h e c o n t e xot f p r i m a r y - s c h osool c i a sl t u d i e sd,o e sn o t b e l o n gt o ' o u r
s o c i e t y ' -t h e b o o kc o n t a i n s t a t e m e n tlsi k e , ' T h eB r i t i s hb e l i e v eidn o n e G o d '( n o t et h e
p a s t e n s e a) n dq u e s t i o nl si k e , ' D oy o ut h i n ka c h u r c ho r a c e m e t e riys l i k ea s a c r e ds i t e ? I' t
fostersa detached,
outsider's
attitudetowardsthe Christianreligion.
n f h u m a n s( a n da n i m a l s ) , ' i n v o l v e m eanntd' ' d e t a c h m e ncta' n i n t e r a c t
I n t h ed e p i c t i o o
t a yb e
w i t h ' d e m a n d ' a n d ' o f f e r ' icno m p l e w
x a y sT
. h eb o d yo f a r e p r e s e n t epda r t i c i p a nm
a n g l e d a w a y f r o m t h e p l a ntehoefv i e w e r , w h i l e h i s o r h e r haenaddl o r g a zm
eaybeturned
towardsit (seee.g.figure4.24 below)- or vice versa.The resultis a doublemessage:
'althoughI am not part of your world,I nevertheless
makecontactwith you,from my
p
e
r
s
o
n
o w n ,d i f f e r e nw
t o r l d ' ;o r ' a l t h o u g ht h i s
i s p a r t o f o u r w o r l d ,s 0 m e o nlei l < ey o ua n d
me,we nevertheless
offer his or her imageto you as an objectfor dispassionate
reflect i o n . ' T h el a t t e ri s t h ec a s ef,o r e x a m p l ei n, a n i l l u s t r a t i ofnr o m a D u t c hj u n i o rh i g h - s c h o o l
g e o g r a p htye x t b o o k( B o l se / a l . , l , 9 9 6 i 2 1 ) . I n a s e c t i o ne n t i t l e d' D e D e r d eW e r e l di n
onzesfraat'('TheThirdWorldin our Street'),two picturesare shownsideby side.0n the
left we seethreeolderwomen,their headscarves
an emblemof their statusas immigrants.
T h e ya r e p h o t o g r a p h e
f rdo m a n o b l i q u e
a n g l eh, e n c ea s ' n o tp a r t o f o u r w o r l d ' a n di n l o n g
shot,henceas 'others','strangers'.
a blonde
0n the right we see,left in the foreground,
g i r l ,c l e a r l ym e a n t o b e t a k e na s D u t c hw
, i t h a b l a c kf r i e n d w
, h o h a sh i sa r m a r o u n dh e r .
The angleis a gooddeal morefrontalthan that of the shotof the threewomen,and the
s h o ti s a c l o s e - u ps:h ei s s h o w na s l i k e' u s ' ,D u t c hh i g h - s c h o sotl u d e n t sa,n d f r o m ' c l o s e
p e r s o n a l ' d i s t a n cBeu. t s h ed o e sn o t m a k ec o n t a c w
t i t h t h e v i e w e r sS. h ed o e sn o t i n v i t e
the viewersto identifywith her,and with her relationship
to a black man. Instead,the
vieweris invitedto contemplate
her relationship
detachedly,
to ponderthe fact that some
p e o p l el i k e ' u s ' h a v er e l a t i o n s h i pwsi t h b l a c kp e o p l eb, u t n o t , i t i s i m p l i c i t l ys u g g e s t e d ,
'we'viewero
s u r s e l v eS
s .h ei s a p h e n o m e n ot on b e o b s e r v e dn ,o t a p e r s o na d d r e s s i nt h
ge
viewer.
E q u a l l yc o m p l e xa n d a m b i v a l e nits t h e b a c k v i e w . O n eo f t h e a u t h o r sa, t a g e 2 1 ,
(figure4.15) and,
photographed
park,just outsideBrussels
his parentsin a snow-covered
perhapsmoreimportantly,
picture
pin
pinboard
it wasthis
hechoseto
on the
of hisstudent
roomin Amsterdam,
ratherthanoneof the other,morefrontalpictureshe hadtakenon the
sameday.At the time,his feelingsfor his parentswerecomplex.Deepattachmentmixed
w i t h o n l y h a l f - u n d e r s t o odde s i r et o d i s t a n c eh i m s e l f r o m t h e w o r l d i n w h i c h h e w a s
b r o u g hu
t p . P e r h a ptsh e p i c t u r ec r y s t a l l i z et dh e s ec o n f u s eedm o t i o nfso r h i m . 0 nt h e o n e
h a n d i, t s h o w e h
d i sp a r e n t tsu r n i n gt h e i rb a c ko n h i m ,w a l k i n ga w a yf r o m h i m ( a r e v e r s a l ,
of course,of the actualsituation);on the other hand,it showedthis gestureof'turning
o n e ' sb a c k ' ,i n a s e n s e , ' f r o n t a l l iyn' ,a m a x i m a l l y ' c o n f r o n t i n g ' w B
a yu.t t o e x p o s e
one's
b a c kt o s o m e o n ies a l s ot o m a k eo n e s e lvf u l n e r a b l a
en
, dt h i s i m p l i e sa m e a s u r o
ef t r u s t ,
despitethe abandonment
whichthe gesturealso signifies.Perhapsthe picturereminded
passage
him of a
from a Dutchnovelhe likedat the time:

and interaction . I39

Representation

i,ri...,'ff:i

fiq e.fl Photographof author'sparents,1968

m u c hI l o v et h a t m a n ' ,h et h i n k s ,
Through
t h ew i n d o wh es e e st h e mw a l ka w a y . ' H o w
that. . . . His motherhaslinked
andhow impossible
he hasmadeit for meto express
He
armswith him.With hesitantstepsshewalksbesidehim on the frozenpavement.
keepslookingat themuntiltheyturn the corner,nearthe tall featheredpoplars.
(Wolker1
s .9 6 5 : 6 I )
H o w i s ' i n v o l v e m e nrte' a l i z e di n l a n g u a g eP
? e r h a ptsh e s y s t e mo f p o s s e s s i vper o n o u n s
c o m e sc l o s e stto r e a l i z i n gt h e k i n d so f m e a n i n gw
s e h a v ed i s c u s s ehde r e .B u t t h e t w o
systems,
the visualsystemof horizontalangleand the linguisticsystemof possessive
pronouns/
differ in manyways.Involvement,
as we haveseen,is alwaysplural,amatterof
'mine'and'his/herlits';
to 'us' andwhatto
between
what belongs
a matterof distinguishing
'ourness'
'them'.And,whilein language
and'theirness',
of
onecannoteasilyhavedegrees
Finally,thereis
in imagessuchgradationis an intrinsicpart of the systemof involvement.
i s' ,a s w e h a v es e e n ,
n o ' y o u r s 'i n t h e s y s t e mo f h o r i z o n t aal n g l eT. h ev i s u a 'l y o u - r e l a t i o n
puts a barrierbetweenthe
realizedby the systemof'offer'and 'demand'.Perspective

140

Representat i on and i nteract i on

viewerand the represented

participants,
evenin the caseof a frontal angle:the viewer
looksat the represented
participantsand has an attitudetowardsthem, but doesnot
i m a g i n a r ie
l yn g a g w
e i t ht h e m .

P O W E RA N D V E R T I C A LA N G L E
Textbooks
of film appreciation
neverfail to mentioncameraheightas an imporuanr
means
o f e x p r e s s i oi n c i n e m a t o g r a p A
h yh. i g ha n g l ei,t i s s a i d m
, a k e st h e s u b j e clto o ks m a l la n d
i n s i g n i f i c a natl,o wa n g l em a k e si t l o o ki m p o s i nagn da w e s o m e : ' L oawn g l e sg e n e r a l lgyi v e
a n i m p r e s s i oonf s u p e r i o r i tey x, a l t a t i o a
n n dt r i u m p h. . . h i g ha n g l e st e n dt o d i m i n i s h
the
i n d i v i d u a l , tfof a t t e nh i m m o r a l l yb y r e d u c i nhgi mt o g r o u n dl e v e l , t or e n d e h
r i ma sc a u g h t
(M
i n a n i n s u r m o u n t a bdl e t e r m i n i s m
' a r t i n ,1 9 6 8 : 3 7 - B ) .B u t t h i s l e a v e tsh e v i e w e ro u t
of the picture.We would rather say it in a somewhatdifferentway: if a represented
participantis seenfrom a highangle,then
participants
the relationbetween
the interactive
(the producerof the image,and hencealsothe viewer)and the represented
participants
is depictedas one in which the interactiveparticipanthas poweroverthe represented
participant- the represented
participantis seenfrom the point of view of power.If the
participantis seenfrom a low angle,thenthe relationbetween
represented
the interactive
and represented
participantsis depictedas one in whichthe represented
participanthas
poweroverthe interactive
participant.
If, finally,the pictureis at eyelevel,thenthe pointof
viewis oneof equalityandthereis no powerdifferenceinvolved.
Thisis,again,a matterof degree.
participantcantowerhighaboveus or
A represented
l o o kd o w no n u s e v e rs o s l i g h t l yI .n m a n yo f t h e i l l u s t r a t i o ni sn s c h o otle x t b o o kw
s e look
d o w nr a t h e rs t e e p loy n p e o p l -e w o r k e r si n t h eh a l l ;c h i l d r e n
i n a s c h o oyl a r d .I n s u c hb o o k s
the socialworld liesat the feetof the viewer,
soto speak:knowledge
is power.
Themodelsin
magazine
advertisements
peopleandcelebrities
andfeatures,
andnewsworthy
in magazine
articles,on the otherhand,generallylook downon the viewer:thesemodelsare depicted
as exercising
symbolicpoweroverus.As shownin figure4.5, productsadvertisedin the
advertisements
maybe photographed
bothfrom a lowangle,as havingsymbolicpowerover
us,and from a high angle,as beingwithin reachand at the commandof the viewer.The
photograph
reproduced
in figure4.16 showsa guardin the 'deathrow'sectionof a prison
in Texas.The angle is low,to make him look powerful.But what makesthis picture
extraordinary
is that not the guard,but the horseis closestto the viewer,
andthat it is not
the guard,but the horse,whoseeverymovementis commandedby this guard,who is
l o o k i n ga t t h ev i e w e rw. h a t c a nt h i sh o r s e' d e m a n d ' f r o m
u s ?T h e r e i nl i e st h e m y s t e r ya n d
the forceof this picture.Empathywith a fate of beingsubjugated
to the powerrepresented
by the guard?0r with a fate of suffering?
How is powerrealizedin language?
Herewe need,again,to rememberthe difference
betweenface-to-facecommunication
and mediatedcommunication.
In the classroom.
for example,powerwill manifestitself first of all in the relationbetweenteacherand
p u p i l .T h i s ,a s c a t e P o y n t o n
h a ss h o w n( 1 9 8 5 :c h . 6 ) , i s i n t h e m a i nr e a l i z e tdh r o u g ht h e
difference
between
the linguisticformsthat may be usedby the teachers
andthe linguistic

Representation and i nteract io n

rig e,fO Prisonguard(DannyLyon,1969)

r42

Representati on and i nteracti on

formsthat maybe usedby the pupils;in otherwords,througha lackof reciprocity

between
the choicesavailableto eachparty in the interaction.
The teachermay usefirst names
in addressing
the pupils;the pupilsmay not usefirst namesin addressing
teachers.
The
teachermay useimperatives
to 'demandgoods-and-services';
the pupilswouldhaveto use
politeforms,for instance,
questions.
This lack of reciprocityhas its effecton everylevel
phonology,
grammart
of language:
vocabulary,
discourse,
and on ideational,
interpersonal,
as well as textualmeanings.
If there is, in face-to-face
communication,
any questionof
powerrelationsbetweenrepresented
participants
andthe pupils,thenthis resultsfrom the
powerrelationbetween
the teacherandthe pupils.
To someextentthis is the casein writingalso,and not just becausein writing- as in
generally- the absenceof the writer causes/
mediatedcommunication
from the start, a
fundamentallack of reciprocity(you cannottalk backto the writer), but also because
the writer andthe readerare oftenunequalin a numberof otherways.Thereadermay be
pronounyou, whilethe writer hides
addressed
directly,by meansof the second-person
behindimpersonalforms. Mental processes
may be attributedto the readerwhile the
writer's mentalprocesses
are neverreferredto. Imperatives
may be used,as modulated
processes
predicatedof the reader(you can,you should,you need,etc.),while suchforms
are not usedof the writer.Hereare someexamples
of textsin whichpoweris encodedin
this way- the first from a Revlonadvertisement,
the secondlrom OurSocietyand 0thers
(Oaklee
y t al.,I9B5):
yourskin.
Wrinkles.Theydon'tstart whereyouthinktheydo.Theystart underneath
That'swhy Anti-AgingDailyMoisturizergoesbeyondmeresurfacetreatment.
Whenyoustudyplacesandpeopleyouneedto havea way of keeping
the information
youcollect.One
wayof doingthis isto takenotesfrom the bookswhichyouread.You
c a n n ow
t r i t ed o w na l l t h et h i n g sy o ur e a da s t h i sw o u l dm e a nw r i t i n go u t t h ew h o l e
book.Notesare a shortway of recording
the mostimportantinformation.
In the first text the writer doesnot directlyreferto himselfor herself,but writesas an
('Theystart underneath
yourskin').
impersonal
authority,in termsof relationalprocesses
('you'),
The readeris referredto directly
and the writer not only knowswhat the reader
thinks('whereyouthink they do'), but alsothat the reader'sthoughtsare misguided:
the
authorityof the writer is firmly basedon the reader'signorance.
In the secondtext,too,
the writer doesnot directlyreferto himselfor herself,but writes impersonally,
in terms
('Notesare a short way of recordingthe most importantinforof relationalprocesses
e si t hw h i c ht h er e a d e r
m a t i o n ' ) , w h itl h
e er e a d e irsa d d r e s s eddi r e c t l y( ' y o u ' )T. h ep r o c e s s w
is associated
are modulatedin variousways('you need','yor.r
cannot').In both casesthe
lackof reciprocity
whichrealizespoweris encodedin the text itself.
B u t t h i so m n i s c i e nk tn o w l e d goef t h e r e a d e r ' m
s i n d t, h i sd i r e c tp o s t u l a t i oonf w h a tt h e
r e a d e rn e e d s( m u s td o , s h o u l dt h i n k ,w i l l f e e l ,a n d s o o n ) a n d t h i s l a c k o f r e c i p r o c i t y
between
the writerandthe readeror the speaker
andhearer,
cannotbe realizedin the same
powerof an image-producer
way in images.
In images,the
must,as it were,be transferred

Representati on and i nleract i on

- the powerof the advertiser

participants
on to oneor morerepresented
on the model,the
powerof the producerof the textbool<
on the ordinarypeoplerepresented
in the textbook.
The nearestequivalent
in speechwouldbe the useof evaluative
We might,for
adjectives.
example,transcode
a picturein whichwe lookdownon factoryworkersor refugees
as'the
humbleworkerl or'the downtroddenrefugees'.lnthe issueof the Australian Women's
Weeklyfrom which we tool<many of our originalexamples,
this kind of transcoding
o c c u r sa n u m b e ro f t i m e s .T h e m a g a z i n ceo n t a i n sp h o t o g r a p hosf a b e j e w e l l eQ
dueen
- bothtakenfrom a low angle.0nthe coverthe
Elizabeth,
andthe actor MichaelDouglas
r e l e v a nat r t i c l e sa r e a n n o u n c ebdy t h e f o l l o w i n gl i n e s : ' D n z z l r r u- eT h e Q u e e n 'jse w e l s ' ;
'FASctNATIN
- GM i c h a e lD o u g l a s ' " F a t aA
l t t r a c t i e n " ' . B u t t h e r e r e m a i n sa v e r y b i g
difference.
What in the imageis an attitudetowardsthe represented
participants
becomes
in language
participants:
a characteristic
of the represented
it objectifies
the attitude.

NARRATIVIZATIONOF THE SUBJECTIVEIMAGE

In manycasesthereis no immediately
apparentmotivationfor pointof view(andfor size
of frame).Theanglemay be highand frontal,and so conveypoweroverand involvement
participants,but the precisenatureof the relationof powerand
with the represented
i n v o l v e m e instn o tg i v e nT. h u sa h i g h - a n g p
l ei c t u r eo f w o r k e r si n a f a c t o r yc o u l db es a i dt o
betakenf romtheviewpointof a supervisor
in an elevated
office,with a windowoverlooking
the factory,but this remainsa metaphor.
We do not seethe office in the picture.Other
p o s s i b i l i t i emsi g h ta l s os e r v et o m a k ec o n c r e t teh e r e l a t i o no f p o w e ra n d i n v o l v e m e nI nt .
othercasesthe (imaginary)viewerintrudesin the pictureto a greateror lesserdegree.
In an advertising
campaign
that ranat thetimewe worl<ed
on thefirst versionof this book,
t h i s w a s d o n eb y i n c l u d i n tgh e h a n d so f t h e i m a g i n a r vy i e w e ri n t h e f o r e g r o u nodf t h e
p i c t u r eT. h e s ec o u l dt h e nb e m a l eo r f e m a l ea, n dg r o o m e di n d i f f e r e nw
t a y s- t h e yc o u l d
weardrivinggloves,
expensive
rings,andso on.In figure4.).7theycreatedthe viewpointof
a couple.
I n f i l m s t h e s e q u e n c i nogf i m a g e sc a n f u l f i l t h i s f u n c t i o nT
. he shotof the factory,
showingthe workersfrom a highangle,canbe preceded
by a low-angle
shotof the elevated
office,with a supervisor
behindthe windowlookingdownat the workers.In suchcasesthe
text narrativizes
the pointof viewand imposesa fictionalviewerbetween
the represented
participants.
andthe interactive
But evenwhentheir originsare not shown,viewpoints
can
a l w a y sb e r e l a t e dt o c o n c r e t e
s i t u a t i o n s . 0 ncea n ,a n d p e r h a p s h o u l da, l w a y sa s k , ' W h o
couldseethis sceneinthisway?','Wherewouldonehaveto beto seethissceneinthisway,
andwhat sort of personwouldonehaveto beto occupythat space?'

O B J E C T I V EI M A G E S
Scientificand technicalpictures,suchas diagrams,mapsand charts,usuallyencodean
objectiveattitude.This tendsto be done in one of two ways:by a directlyfrontal or

t43

Representati on and i nteract i on

t44

oXlfkata
SMt*g td*d'

'rWhYdon'twe
takl nnine?"
cil

:iti;r:1.,\ !!l.til::ts:l.ii'

r !i r ;; :r j :,i r : ri: : x j. ljl !..1.

; i'; i. i! r jl i 1!;1.

(ivewldea,Novembel1987)
fiq +.f Z Sterlinqadvertisement

p e r p e n d i c u ltaorp - d o w na n g l e S
. u c ha n g l e sd o s u g g e svt i e w e rp o s i t i o n sb,u t s p e c i aal n d
privilegedones,which neutralizethe distortionsthat usuallycome with perspective,
whena
itself.To illustratethis with a simpleexample,
they neutralizeperspective
because
degree
of
distortion
perspectivally
length,
and
the
not
equal
its
sides
are
of
drawn
cubeis
d e p e n dosn t h e a n g l eo, n t h e e n c o d evdi e w e rp o s i t i o nT. h ec u b ed o e sn o t l o o k ' a sw e k n o w
r o s i t i o nB. u t
i t i s ' ,w i t h a l l i t s s i d e so f e q u a l e n g t hb, u t ' a s w e s e ei t ' , f r o m a p a r t i c u l a p
fsl a t ,w i t h a l l
c
u
b
e
a
p
p
e
a
r
a
n
d
t
h
e
d
i
s
a
p
p
e
a
r
s
,
t
h
e
t
h
i
r
d
d
i
m
e
n
s
i
o
n
i
n
f
r
o
n
t
fromdirectly
and its
its sidesof equallength.Fromabove,exactlythe sameeffectoccurs.Perspective
been
neutralized:
effect
have
attitudinizinq

rig +.fe Cuheseenfrom an angle,frontallyandfrom above

o
z
E

.s

"[,Ove"n*f]lp'
'ullflavour
o[ a trulvxtisfring
lovca cupof
granulatcd
coflce...
Co{Icc.
Bushclls
iVlastcr
Roast

ftate Z Bushellsadyertisemenl(Woman'sWeekt!,1987)

Plalet Joshua Smith (William Dobell,1943)(Art callery of New South Wales)

etat. I Patick White(LouisKahan,1963)(Art Galleryot NewSouthWales)

gl
t.

f aI
3
a .rl

s*,,

* ,
J ':
a

re
:w
:

:
Q

etatez PalgraYe
colourscheme

C t a t eI

Colourfulthoughls (transparency)

Representation and interaction

- 145

Frontalandtop-downangles,however,
are not objectivein entirelythe sameway.The
f r o n t a l a n g l e i s t h e a n g l e o f m a x i m u mi n v o l v e m e nItt. i s o r i e n t e dt o w a r d sa c t i o n .
The picturesof the Antarcticexplorer(figure2.4 couldbe transcoded
as 'Theseare the
c l o t h e ys o us h o u l dw e a ra n dt h i si st h ew a yy o us h o u l dw e a rt h e mi f y o uw a n tt o e x p l o r teh e
A n t a r c t i c . ' T h fer o n t a la n g l ei s t h e a n g l eo f ' t h i s i s h o w i t w o r k s ' , ' t h i si s h o wy o uu s ei t ' ,
'this is how you do it'. Thetop-down
angle,on the otherhand,is the angleof maximum
power.It is orientatedtowards'theoretical',objectiveknowledge.
It contemplates
the
world from a god-likepointof view,puts it at yourfeet,ratherthan within reachof your
hands.Abstractdiagramscan sometimes
be readin bothways.A communication
model,
for instance(e.9.figure 2.2), can be read as a map ('top-down',a schema,a'theory
o f c o m m u n i c a t i o n ' : ' t hi issw h a t c o m m u n i c a t i ol ono k sl i k e ,f r o m t h e p o i n to f v i e wo f a
d i s i n t e r e s t eodb s e r v e r ' o
) ,r a s a f r o n t a lv i e w ,a b l u e p r i n ta, ' p r a c t i c a l m a n u a lo f c o m m u n i c a t i o n( ''t h i si s w h a ty o ud o w h e ny o uc o m m u n i c a t e- ' )a n dt h i s i s p e r h a posn eo f t h e
sources
of its socialpower.
A third objective
v i e w p o i n t ,h e c r o s s - s e c t i oann, d t h e , X - r a y , v i e w ,s h o u l da l s o b e
considered:
its objectivityderivesfrom the fact that it doesnot stopat appearances,
but
probesbeyondthe surface,to deeper,
more hiddenlevels.In Westerncultureit is almost
e x c l u s i v eul ys e di n d i a g r a m sa,l t h o u g h
o n ec a ns o m e t i m easl s oo b s e r veex p e r i m e nwt si t h i t
i n c h i l d r e n 'dsr a w i n g s .
Nof all diagrams/
mapsandcharts,however,
arecompletely
Theverticalangle
objective.
o ft h e G u l W
f a r m a p i n f i g u r4e. 7 9 , i sh i g h , b u t n o t c o m p l e t e l y t o p - d o w n , a n d i t s h o r i z o n t a l

!t'dairy
tranrport planor drop
'
csttcr' botnbsofi
lslandln oreoaradonfsr

US plana dcrtroy
28 anks,26 othCr
wtdder thr.i
lItiugry Dlrcar
and tiici rrnmrniricn
"*dchi

fig l.ff

lEri

cuff War map (s/dr e! Morning Hetatd,22Januaryt99I)

{urer

r46

Representati on and i nteract i on

a n g l ei s o b l i q u ec,a u s i n g
u st o l o o ka t t h et h e a t r eo f w a r f r o m t h e s i d e l i n eisn, a r e l a t i v e l y
w e f i n ds i m i l a ra n g l e s( s e ef i g u r e
d e t a c h ew
d a y .I n b o o k sa b o u ts c i e n cfeo r y o u n gc h i l d r e n
5 . 1 1 ,f o r e x a m p l e )t ,h e i r o b l i q u e n e spse r h a p s u g g e s t i nt gh a t t h e y a r e n o t ( o r n o t y e t
quite)meantas 'howto do it' pictures.
to givea sense
Elements
of perspective
mayalsobeaddedto graphsandcross-sections,
visuals.Havingfirst been
of reality,of physicalexistence,
to abstract,two-dimensional
world of people,placesand things,they
abstractedfrom the concrete,
three-dimensional
way,as neryhuman-made
l<inds
of thingsand
are now restoredto it, but in a transformed
places.Thuswe can see- for instance,
in lavishlyproducedannualcompanyreportst h r e e - d i m e n s i obnaarl g r a p h sl ,o o k i n gl i k es k y s c r a p eor sr m o n o l i t h sa,g a i n sat b a c k g r o u n d
o f c l e a na n d s m o o t hh i l l si n f f a t ,p r i m a r yc o l o u rI.n f i g u r e4 . 2 0 g r a p h sb e c o m ea s e t t i n g
threefor action:tourists movethrough the abstractlyrepresented,
but nevertheless
d i m e n s i o n awl o
, r l d o f t h e i n t e r n a t i o n taol u r i s tb u s i n e sjsu,s t a s m a y a l s ob e t h e c a s ei n
t e l e v i s i onne w sg r a p h i c s , w h ear e
f u r t h e rs e n s eo f r e a l i t ym a yb e g i v e nt o s u c hp i c t u r e sb y
m e a n so f a n i m a t i o n .
meaningof these
The additionof perspective
adds nothingto the representational
l e a n i n g Isn. a l l t h e s ee x a m p l et sh e
d i a g r a m sm, a p sa n dc h a r t sb; u t i t d o e sa d da t t i t u d i n am
angleis high,explicitlyattitudinalizing
the objectivestanceof the god-liketop-downview,
that moderntool of the production
andoftennarrativizing
it as the viewfrom the satellite,
ao
l w e rT. h eh o r i z o n t aal n g l eo, n t h e o t h e r
o f v i s u a kl n o w l e d gaen ds y m b ool f i n f o r m a t i o n p
handm
, a yv a r y :w i t h t h e ' i n c r e a s ien t o u r i s m ' w ea r e d i r e c t l yi n v o l v e dt h; e e v e n t so f t h e
, e w a t c h ' f r o mt h e s i d e l i n e sa' s, b y s t a n d e r s .
G u l f W a r ( f i g u r e4 . 1 9 ) ,o n t h e o t h e rh a n d w
not so much in the contextswherethis new
This processof attitudinalization
happens,

oun ?ounttlt sttLLAR ?Al(Gs (lrr

gEBRiltGtNG
ym"
tT tN*
AND\ryHO'LL

fig e.ZO An increasein tourism (Sydne! Moming Heratd,23 January]-ggll

Represe
ntati on and interacti on

r47

but in the
visual knowledgeis producedand this new informationalpowerexercised,
hereconceptual
in popularized
form,and celebrated:
contextsin which it is disseminated
a n d s c h e m a t i icm a g e sa r e d r e s s e d
u p i n t h e c l o t h e so f v i s u a lr e a l i t ya, n d l i t e r a l l ya n d
f ig u r a t i v e l y ' a n i m a t e d ' .
To conclude
t h i s s e c t i o nw e a d d s o m en o t e so n d i f f e r e n tl,e s s ' s u b j e c t i v e ' k i n dosf
sofar,
perspective.
we havebeendiscussing
If, in centralperspective,
the kindof perspective
is seenfrom the front and at eyelevel,the sides,top and bottomwill be hidden
something
r s i n d r a w i n g1 o f f i g u r e4 . 2 I . I f t h e s a m ec u b ei s s e e n
f r o m v i e w A c u b ew o u l da p p e a a
f r o m a n o b l i q u e a n g l e , o nt he eo sf i d e s w i l l c o m e i n t o v i e w , b u t t h e o t hreerm
w ai l il n h i d d e n .
I f t h e a n g l ei s h i g h s, ot h a t w e l o o kd o w no n t h e c u b et,h e t o p w i l l a l s oc o m ei n t ov i e wa, s
r e a s q u a r eI .t w i l l b e
i n d r a w i n g4 o f f i g u r e4 . 2 I . B u t i n t h i sc a s et h e f r o n tw i l l n o l o n g e b
p
a
r
a
l
l
e
l
s
p
e
r
s
p
e
c t i vceo n v e r gteo w a r d s
i
m
a
g
e
T
h
e
h
o
r
i
z
o
n
t
a
l
i
n
i
n
c
e
n
t
r
a
l
distorted.
an
p
o
i
n
t
s
p
a
r
a
l
l
e
l
s
,
a
l
t
h
ough
t h i s i s o f t e nl e s s
m
o
r
e
v
a
n
i
s
h
i
n
g
a
n
d
s
o
t
h
e
v
e
r
t
i
c
a
l
o n eo r
do
great,
is
often'corrected'
distortion
as
vertical
are
not
and
as
vertical
obvious,
distances
so
i n d r a w i n gas n dp a i n t i n g s .
D r a w i n g2 i n f i g u r e4 . 2 I t o n t h e o t h e r h a n d ,i s a n e x a m p l eo f ' f r o n t a l - i s o m e t r i c '
yetwe canseethe sideandthe top.
perspective.
Herethe front of the cubeis not distorted,
p
a
r
a
l
l
e
l
s
v a n i s h i npgo i n t .F r o n t a l - i s o m e t r i c
h
o
r
i
z
o
n
t
a
l
d
o
n
o
t
c
o
n
v
e
r
g
t
e
o
w
a
r
d
s
a
A n dt h e
participants,on
perspective
is basedon the 'objective'dimensions
of the represented
to us. For this
how
they
appear
we
know
rather
than
on
what
thesedimensions
to be,
perspective
whereit is important
reasonfrontal-isometric
is usedin technicaldrawings,
objectsfrom the drawing.In
to be able to measurethe dimensions
of the represented
and
perspective,
yet,
betweeninvolvement
is
not,
as
a
choice
frontal-isometric
then,there
t
e
r
m
s
o
f
d e t a c h m e nItt. i s t h e a n a l o g yi n v i s u a l
t h e ' i m p e r s o n a l i t y ' c h a r a c t e r i sotfi c
l
a
n
g
u
a
g
e
.
scientific
T h ep e r s p e c t i vues e di n d r a w i n g3 o f f i 9 u r e4 . 2 7 i s c a l l e da n g u l a r - i s o m e tpr iecr s p e c as a square.But the
tive. Herethe front is distorted,the squareno longerrepresented
parallels
no
end
to
spacein this kind
converge.
There
is
horizontaland vertical
do not
perspective
was used,
perspective
Angular-isometric
of
it stretcheson indefinitely.
period
this
artists
of
woodcuts
for example,in eighteenth-century
Japanese
Japanese
point
looked
at the
They
high
angle.
view,
as
a
relatively
alwayschosean oblique
of
as well
point
meditative
a
of
view,
from
without
involvement,
a
detached
world
a senseof
from
distance.

rfl
Q

(3) cube in angular-isonetricpersp!ctive;

fig +.Zf (1) Cubeseenfrom the front; (2) cube in frontal-isometricperspective;
(4) cubeseenfrom an angleIn centralperspective

148 .

Representation and interaction

EtsBEE
Q

fig +.ZZ Detailfiom a fourt!enth-century

Spanishnativity(from Arnheim,1974)

T h i sb r i e fs u r v e yd o e sn o t e x h a u stth e p o s s i b i l i t i eIsn. m e d i e v aalr t ' i n v e r t e dp e r s p e c tive'wassometimes

used(seefigure4.22).Thisallowsbothsidesof an objectto be seen
perspectival
and causes
the
vectorsto divergeratherthan converge.
It can oftenbe found
i n c h i l d r e n 'dsr a w i n g s( y o u n gc h i l d r e na l s ot e n dt o d r a wt h e w o r l da s t h e yk n o wi t t o b e ,
ratherthan as they see it) and, in recenttimes,has beentaken up by painterssuchas
Picasso
and Braque,
who lookedfor moreobjective
waysof representing
theworld,regarding the simpleviewpointof centralperspective
as one-sided
and restrictive,
and viewing
r e a l i t ya s m u l t i f a c e t e da , c o m p l e xw h o l eo f o f t e n i n c o m p a t i b laen d m u t u a l l yc l a s h i n g
viewpoints.
In this way,as Arnheimnotes(1974: I32),'Ihey makethe contradictions
of
w h i c hM a r x i s t s o e a kv i s u a l ' .

A SUMMARY

F i g u r e4 . 2 3 s u m m a r i z et h
s e m a i nk i n d so f i n t e r a c t i vm
g e h a v ed i s c u s s ei n
d this
e e a n i nw
chapter.It shouldbe remembered
that theseare'simultaneous
systems/(as indicatedby
thecurlybrackets):anyimagemusteitherbea'demand'oran'offer'andselectacerta

REALIZATIONS
Demand
0ffer
Intimate/personal
Social
Impersonal
Involvement
Detachment
Viewerpower
Equality
Rep resented p art i cipant p ower

gazeat the viewer

absence
of gazeat the viewer
c l o s es h o t
m e d i u ms h o t
l o n gs h o t
frontalangle
o b l i q u ea n g l e
h i g ha n g l e
eye-leve
a ln g l e
l o wa n g l e

Representation and interaction

Contact ___________>l

. I49

T Demand

L 0""'
Intimate/personal
Social distance

Social
Impersonal

Intefactive
,

meanrngs

Involvement

L Detachment

viewerpower

subjectivity
fAuitude----rl

-+f

I
L

objectivitv

---1

Equaritv

- Representation
power
|.- Action orientatlon
Knowredge orientation

fig C.Zf Interactivemeaningsin images

sizeof frame and selectacertainattitude.In the nextfew sectionswe will discuss

some
examplesat greaterlength,to showhow the systemsof 'contact,,'socialdistance,
and
'attitude'interact
to create more complexand subtle relationsbetweenreoresented
and interactive
participants.

TWOPORTRAITS
A N D T W O C H I L D R E N ' SD R A W I N G S
Rembrandt's
famousSelf-portraitwith Saskiadatesfrom 1634.JohnBerger(1972: 111)
calls it'an advertisement
of the sitter,sgoodfortune,prestigeand wealth,and,he adds,
'like all such
advertisements
it is heartless'.
Yet,from the pointof viewof the interactive
meanings
we havediscussed
in this chapteri
the paintingis perhapsa little morecomptex
t h a n B e r g e r 'rse m a r k s u g g e s t . 0tnh e o n eh a n d i,t i s a ' d e m a n dp, i c t u r e- R e m b r a n d t
and
Saskiasmileat the viewer,Rembrandt
perhapsa little moreeffusively
and invitinglythan
Saskia:heevenraiseshisglassin a gesturedirectedat the viewer.0n
the otherhand,hehas
s h o w nh i m s e l fa n d S a s k i af r o m b e h i n da, n d f r o m w h a t H a l l w o u l dc a l l ' c l o s es o c i a l ,
distance,
with Saskiaa little furtherawayfrom the viewerthan Rembrandt
- her headis
c o n s i d e r a bsl ym a l l e rt h a n R e m b r a n d te' sv e nt h o u g hs h ei s s i t t i n go n h i s l a p a n d s h o u l d
therefore,
strictlyspeaking,
becloserto the viewerthan Rembrandt
(theangleat whichher
headis turnedio acknowledge
the vieweralsoseemsunnaturaD.IsRembrandt
distancing
himself(and Saskiaevenmore)from the viewer,excludingthe viewerfrom rnvolvement
and intimacywith his new-found(and Saskia'salreadyestablished)
socialstatus,thus

Representation and interaction

150

- but the portrait is alsoa self-portrait.Rembrandt,

contradicting
the invitation?Perhaps
t h e m i l l e r ' ss o n ,n o w m a r r i e di n t o a w e a l t h ya n d r e s p e c t a bfl ae m i l ya n d l i v i n gi n g r a n d
style,alsodistances
himselffrom his newself (andto someextentfrom Saskia),as if he
As a self-portraitthe
cannotfeel fully involvedand intimatewith his new environment.
picturemay be self-congratulatory
but it also betraysa degreeof
and smug,'heartless',
a l i e n a t i o np,o s i t i o n i ntgh e r e p r e s e n t eRde m b r a n di n
t a c o m p l e xa n dc o n t r a d i c t o rsyo c i a l
classposition,betweenthe world of his origins,which is also the point of view of the
p i c t u r ea, n dt h e w o r l do f S a s k i ai n t ow h i c hh e h a sm o v e dT. h i s w
, e t h i n k ,m a k e si t a l i t t l e
lesssmug,anda little moretouchingthan Bergergaveit creditfor.
F i g u r e4 . 2 5 s h o w sa l a t e rs e l f - p o r t r a ipt ,a i n t e di n 1 6 6 1 .B y t h i st i m e ,S a s k i ah a sd i e d ,
Hendrickje,
and Rembrandt
hasgonebankrupt.He now liveswith hisformerhousekeeper,
i n a m o r ed o w n m a r k ente i g h b o u r h o oa dn ,di n m u c hr e d u c ecdi r c u m s t a n c eI ns t. h i sp o r t r a i t

fig +Zt Setf-poftrait with Sasftia(Rembrandt,1634)(Pinakotek,Dresden)

Representati on and i nteract i on

fiq+.ZS Self-portnit(Rembrandt,166I)(Kunsthist0rischesMuseum,Vienna)

he is ableto comeface-to-face
with himself,
to confronthimself(andthe viewer)squarely
i
n
t
i
m
a
t
e
l
y
and
w i t h h i m s e l f : ' H ei s a n o l d m a n n o w .A l l h a sg o n ee, x c e p at s e n s eo f t h e
, 72:112).
q u e s t i oonf e x i s t e n coef,e x i s t e n caes a q u e s t i o n( 'B e r g e rI 9
Thepictureon the coverof 'lVlyAdventure'(figure4.26),thestoryby an eight-year-old
boy whichwe havealreadyfeaturedin the previouschapter,constitutes
a'demand':the
l i t t l eb o y i s l o o k i n ga t u s ,a n ds m i l i n gH
. e s e e k so u r r e c o g n i t i o n
H.ew a n t st o b e a c k n o w l e d g e d . 0tnh eo t h e rh a n d , t h e
a n g l ei s o b l i q u ea,n dh i g h a
, n dt h e b o yi s s h o w nf r o ma g r e a t
d i s t a n c eN. o t o n l y d o e st h e w r i t e r o f t h i s s t o r ys h o wh i m s e l if n t h e r o l e o f b e i n gs h i p w r e c k e dh,e a l s os h o w sh i m s e laf s ' o t h e r '( t h eo b l i q u ea n g l e )a, s s o m e o noev e rw h o mt h e
v i e w e rh a sp o w e r( t h eh i g ha n g l e )a n da s s o c i a l l yd i s t a n ta, ' s t r a n g e r ' ( t h el o n gs h o t ) .I n

151

I52

Representation and interaction

otherwords,he usesthe interactive

resources
of the subjective
image(quiteprecociously,
w e f e e l )t o s h o wh i m s e laf s s m a l l ,i n s i g n i f i c a a
nn
t d a l i e n a t e dy ,e t d e m a n d i nrge c o g n i t i o n
from the viewer.At the sametime the act of drawinghimselflikethis affordshim.as the
producerof the image,somepoweroverthat imageof himself,an outletfor hisfeelings.
In
supportof this interpretation
it can be notedthat the boy doesnot exactlyplay a heroic
role in the story.After creatingthe raft, and just as the raft 'startedto be goodfun',
everythinggoeswrong for him: he loseshis moneyand neverfinds it again,the raft
collapses
and is lost irretrievably,
andthe herohasto walk all the way home,wet andcold.
It is an unhappyendingfor a herounableto controlthe unpleasant
eventsthat happento
him.
F i g u r e4 . 2 7 i st h ef r o n tc o v e o
r f a ' s t o r y 'o n s a i l i n gb o a t sb y a c h i l df r o mt h es a m ec l a s s
as the authorof 'My Adventure'.
Its subjectis similar:peopleon a boat.But the systems
attitude'tako
e n v e r yd i f f e r e nvt a l u e sT. h ec h a r a c t e r s
o f i m a g ea c t ' , ' s o c i adl i s t a n c e ' a n' d
do not look at us;the pictureis an 'offer'.Theangleis frontal and eyelevel,and the two
f i g u r e si n t h e b o a t a r e n e i t h e rp a r t i c u l a r l yd i s t a n t n
, o r p a r t i c u l a r l cy l o s e T
. h e r ei s n o
setting,no texture,no colour,no lightandshade.Thesailingboatis drawnwith geometrical
accuracy.But for the two figures- simplydrawn,and more or lessidentical,exceptfor
t h e i r s i z e( a f a t h e ra n d s o n ? )- t h i s c o u l db e a t e c h n i c adl r a w i n qA. s s u c hi t s u i t st h e

Fiq 4,26 Coyerillustrationof My Adventure'

Representati on and i nteracti on

($

figl.Zl

Coverillustrationol'sailing Boats'

o b j e c t i v eg,e n e r i ct i t l e ,' S a i l i n gB o a t s 'j,u s t a s t h e c o v e ri l l u s t r a t i o o

nf 'My Adventure'
suitsthat story'ssubjective,
no
specifictitle. In mostof the illustrationsinsidethe essay,
h u m a nf i g u r e sa r e s e e na/ s t h o u g ht h e c h i l da l r e a d yu n d e r s t a n dt hs a t t h e ' l e a r n i n go' f
t e c h n i c aml a t t e r ss h o u l db e o r e c e d ebdv a ' h u m a ne l e m e n t ' t oa t t r a c tn o n - i n i t i a t et os t h e
subject.
Clearly,
chlldrenactivelyexperiment
and
resources
of language
bothwith the interactive
with the interactive
resources
And
of visualcommunication.
Theyare activesign-makers.
the differentways in which thesetwo childrenrepresentboatsshowtwo very different
subjectivities
at work.

r53

5 Modality:
d e s i g n i nm
g o d e l so f r e a l i t y
M O D A L I T YA N D A S O C I A LT H E O R YO F T H E R E A L
o f t h e r e l i a b i l i t oy f m e s s a g e s .
0 n e o f t h e c r u c i a li s s u e isn c o m m u n i c a t i oi snt h e q u e s t i o n
Is what we seeor heartrue,factual,real,or is it a lie,a fiction,something
outsidereality?
To someextentthe form of the message
itself suggests
an answer.We routinelyattach
morecredibilityto somekindsof messages
than to others.Thecredibilityof newspapers,
p
f o r i n s t a n c er e, s t so n t h e ' k n o w l e d g e ' t h a th o t o g r a p hdso n o t l i e a n dt h a t ' r e p o r t s ' a r e
m o r er e l i a b l teh a n' s t o r i e s t' ,h o u g hs i n c ew e w r o t et h ef i r s t e d i t i o no f t h i sb o o kt h e r i s eo f
P h o t o s h oapn d ' s p i n ' h a vbee g u nt o u n d e r m i nbeo t ht h e s et y p e so f k n o w l e d g e .
of sight
M o r eg e n e r a l layn, dw i t h p a r t i c u l arre l e v a n ct o
e t h ev i s u a lw, e r e g a r do u rs e n s e
reliable
r
e
l
i
a
b
l
e
s
e
n
s
e
s
a
w
w
i
t
h
m
y
o
w
n
a
s
m
o
r
e
as more
t h a no u r
of hearing,'l
it
eyes'
'l
evidence
than heardit with my ownears'.
we also knowthat,whilethe cameramay not lie - or not much,at any
Unfortunately,
-those
of truth and realityremain
rate
who useit andits imagescananddo.Thequestions
even
significantly,
insecure,
subjectto doubtand uncertainty
and,
more
to contestation
and
Yet,
make
decisions
on the basisof
struggle. as members
of a society,
we haveto be ableto
produceandexchange.
to act,
the information
we receive,
And in so far as we are prepared
quite
information
we
receive,
so,
to
some
extent,
on the
we haveto trust someof the
anddo
modality
in
itself,
on
of
textual
for
what
can
be
basisof
markers the message
the basis
cues
as
what
with
circumspection.
These
modality
regarded credibleand
shouldbe treated
markershavebeenestablished
by the groupswithinwhichwe interactas relativelyreliable
guidesto the truth or factualityof messages,
and they havedeveloped
out of the central
group.
beliefs
needs
values,
andsocial
of that
I n t h i sc h a p t ew
r e w i l l d i s c u stsh e s em o d a l i t yc u e sA
. s t h r o u g h o ut th e b o o k ,w e t a k e
to
be
signs
which
have
out
of the interestof socialgroups
them
motivated
signs
arisen
power
interact
who
withinthe structuresof
that definesociallife,andalsointeractacross
produced
groups
various
within
in the
the systems
by
a society.As we havediscussed
of
signs
Introduction,
is, in
the relationbetweenthe signifiersand signifieds motivated
principle,one of transparency.
choosewhat they regardas apt, plausible
Sign-makers
We are thereforefocusingon the
meansfor expressing
the meanings
theywishto express.
rangeof signsfrom whichsuchchoicescan be made- someof themspecialized
modality
markers,otherspart of a muchwider and more generalrangeof meansof expressing
meaningsof truth and falsehood,
fact and fiction,certaintyand doubt,credibilityand
unreliability.
A s o c i asl e m i o t itch e o r yo f t r u t hc a n n oct l a i mt o e s t a b l i sthh ea b s o l u tter u t h o r u n t r u t h
representations.
It can onlyshowwhethera given'proposition'(visual,verbalor otherof
wise) is represented
as true or not. Fromthe pointof viewof socialsemiotics,
truth is a
c o n s t r u cot f s e m i o s i sa,n d a s s u c ht h e t r u t h o f a p a r t i c u l a sr o c i a g
l r o u pa r i s e sf r o m t h e

Modality

valuesand beliefsof that group.As longas the message

formsan apt expression
of these
'felicitous'fashion.Thisdoesmean,
proceeds
beliefs,communication
in an unremarkable,
however,
that our theoryof modalityhasto accountfor a complexsituation:peoplenot
only communicate
and affirm as true the valuesand beliefsof their group.They also
communicate
and accorddeqreesof truth or untruthto the valuesand beliefsof other
gr0ups.
T h et e r m ' m o d a l i t y ' c o m ef rso m l i n g u i s t i casn d r e f e r st o t h e t r u t h v a l u eo r c r e d i b i l i t y
r f m o d a l i t yf o c u s e s
o f ( l i n g u i s t i c a lrl ey a l i z e ds)t a t e m e n tasb o u tt h e w o r l d .T h eg r a m m a o
on suchmodalitymarkersas the auxiliaryverbswhichaccordspecificdegrees
of modality
to statements,verbslike may, will andmust Gf. the differencebetweenHe may comeand
He will comd andtheir relatedadjectivesG.g. possible,probable,certain) and adverbs
( s e e H a l l i d a y1, 9 8 5 : B 5 - 9 ) . B u t m o d a l i t yi s n o t o n l y c o n v e y e tdh r o u g ht h e s ef a i r l y
clear-cutlinguisticsystems(seeKressand Hodge,t978,7993'. 127). Tal<e
this example,
from )ur Societyand 0thers(}akley et a1.,1985).Clearly,it not onlycontainsstatements
s u c ha s ' A b o r i g i n apl e o p l eh a dn o r e l i g i o n ' a n d ' t hw
e h o l el a n dw a sa c a t h e d r a lb' ,u t a l s o
indications
of the truth valueof thesestatements:
G o v e r n oPr h i l l i pt,h e s e t t l e r sa,n dt h e c o n v i c tcso u l df i n d n o c h u r c h eosr c a t h e d r a l s
o r w o r k so f a r t l i k et h o s eo f B r l t a i n .P e r h a ptsh i s m a d et h e mt h i n kt h a t A b o r i g i n a l
p e o p l eh a dn o r e l i g i o nI n
. f a c t , t h eA b o r i g i n ehsa dv e r yc o m p l i c a t erde l i g i o ubse l i e f s .
Thesehad beenpasseddownfrom one generatlon
to the nextthroughDreamtime
storiesfor thousands
of years.For the Aborigines
the wholelandwas a cathedral.
Their art was joinedto their religion.Much of their art had beenkept safelyfor
thousands
of years.
(Oaklee
y t a l . , I 9 B 5 :I 4 2 )
T h es t a t e m e n t ' A b o r i g i npael o p l eh a d n o r e l i g i o n ' i sg i v e nl o w m o d a l i t yT. h ew r i t e r sd i s tancethemselves
from it by attributingit to 'GovernorPhillip,the settlers,and the convicts'by formulatingit as a subjective
idea('thismadethemthinkthat . . .') and by using
the pasttense(after all, what was true in the past neednot be true in the present).The
w r i t e r so
' w ns t a t e m e n t(se . 9 . ' t h eA b o r i g i n a lhsa dv e r yc o m p l i c a t erde l i g i o ubse l i e f s 'a) r e
not qualifiedin this way,'they are formulatedas objectivefacts ('In fact, the Aborigines
had . . .') and not attributed(it is curious,however,
that they are nol extendedto the
p r e s e ntti m e l ) .T h e s t a t e m e n ttsh a t e m b o d yt h e ' b e l i e f s ' o ft h e A b o r i g i n e sf i,n a l l ya, r e
givenlowermodality.Theyare explicitlyattributedto the Aboriginesand thereforenot
s u b s c r i b et do b y t h e w r i t e r s ,a n d t h e y a r e q u a l i f i e db y t e r m sl i k e ' s t o r y ' , ' d r e a m ' a n d
'belief' - termswhich,in Westernculture,signifylow modalityand are contrasted
with
high-modality
termssuchas 'reality','fact' and'truth'.
The exampleshowsthat modalityis 'interpersonal'ratherthan 'ideational'.It does
not expressabsolutetruths or falsehoods;
it producessharedtruths aligningreadersor
listenerswith somestatements
and distancingthem from others.It servesto createan
imaginary'we'.
It says,as it were,theseare the things'we'considerIrue,andthesearethe
t h i n g s ' w e ' d i s t a n coeu r s e l v ef rso m , f o ri n s t a n c e : ' w e ' h a vneo r e l i g i o nb,u t t h e A b o r i g i n e s

155

r56

Modality

d o ,a n da l t h o u g thh i s r e l i g i o n
i s t r u ef o r ' t h e m ' ,i t i s n o tt r u ef o r ' u s ' .N e v e r t h e l e a
s s ,( f o r
' u s ' ) a r t a n d r e l i g i o na r e n o t ' j o i n e d ' , ' w e ' c a na p p r e c i a tAe b o r i g i n arle l i g i o na s ' a r t ' , a s
b e a u t i f u'ls t o r i e s ' a n d ' d r e a m s ' ( a r ti,n W e s t e r nc u l t u r e h, a s l o w e rm o d a l i t yt h a n ,f o r
instance,
science- hencethe greaterlicencegivento artists).We call the 'we'the text
attemptsto produce'imaginary'because
manyof the childrenwho are madeto readthe
bookmay in fact 'havereligion'.However,
we realizethat the socialgroupings
discursively
institutedin thisway may be veryreal andmayhaveveryrealeffectson children'slives.
The conceptof modalityis equallyessentialin accountsof visual communication.
Visualscan representpeople/placesand thingsas thoughthey are real,as thoughthey
actuallyexist in this way,or as thoughthey do not - as thoughthey are imaginings,
fantasies,
etc.And, heretoo, modalityjudgements
caricatures,
are social,dependent
on
what is considered
real (or true,or sacred)in the socialgroupfor whichthe representation
i s p r i m a r i l yi n t e n d e d
r o md e S a u s s u r e ' s
C.o n s i d e r , fionrs t a n c e , t h e ' s p eceicr h
c u i t ' d i a g r afm
(I974ll9I6J), shownoncemorein figure5.1 (see
famousCoursein GeneralLinguistics
a l s of i g u r e2 . 1 8 ) .I t d e p i c t st w o h u m a n s , ' A ' a n d ' B ' ,a n d a p r o c e s sc/i r c u l a ra n d c o n t i n u o u sd
, escribed
a s t h e ' u n l o c k i n go f s o u n d - i m a g ei ns t h e b r a i n ' ,f o l l o w e db y t h e
'transmittingof an impulsecorresponding
to the imageto the organsusedin producing
sounds',followedby the 'travellingof the soundwavesfrom the mouth of A to the ear
o f B ' ( 1 9 7 4 t 1 9 1 6 1 : 1 1 - 1 2 ) .I n a n o t h e vr e r s i o n( f i g u r e5 . 2 ) ,d e S a u s s u rsec h e m a t i z e s
the diagramevenfurther,makingit lookalmostlikean electricalcircuit.
The photographin figure5.3 aiso represents
the speechprocess,
or ratherpart of it,
s i n c ew e s e eo n l y ' A ' s p e a k i n ga,n d o n l y ' B " u n l o c k i n gs o u n d - i m a g iensh i sb r a i n ' .I t i s a
scenefrom RobertAldridge'smovieTheBig lhife 0955), starringRodSteigerandJack
Palance.
Thethreerepresentations
of the speechprocess
differ in a numberof ways.First while
restrictsitselfto representing
the photograph
what wouldnormallybe visibleto the naked
eye,the diagramsdo not: they makevisiblewhat is normallyinvisible(mentalprocesses,
' s o u n d - i m a g ienst h e b r a i n ' )a n dt h e yd o s h o ww h a t c a n n o r m a l l yo n l yb e h e a r d( ' s o u n d
waves').To do so theytake recourse
to abstractgraphicelements(dottedand continuous
presents
lines,arrows)and to language.
Second,
whilethe photograph
us with a moment

-\-I_
- - - - - -

.-j-:
- / - - - z

A
O

pisf.f Speechcircuit(de
Saussure,t974n9l6l)

Modality

phonation

fig S.Z Schematized

speechcircuit(de Saussure,
1974[1916])

fiS S.: RodSteigerandJack patancein fhe Big Knife (Atdridqe,

1955)

f r o z e ni n t i m e ,t h ed i a g r a mdse p i c a
t p r o c e stsh a tt a k e sa c e r t a i na m o u n o
t f t i m et o u n f o l d :
o n eu t t e r a n c o
e f ' A ' a s w e r ra s o n e u t t e r a n c o
e f ' B ' , a t t h e v e r y r e a s tT. h i r d ,w h i r et h e
p h o t o g r a pdhe p i c t s ' A ' a n d ' B 'i n g r e a td e t a i ls, h o w i n g
s t r a n d so f h a i r , w r i n k r egsl i,m m e r s
of light in Steiger'sdark glasses,
the diagramsreducethe two to schematicprofiles,or
evencircles,minimalgeometricshapes,abstractelements.And, whiie
the photograph
showsdepth,modellingcausedby the prayof rightandshade,and
a setting,a background,
the diagramsomit all of these.Theyare abstractand schematic
wherethe photographis

157

158

Modality

presents
itselfas
concreteanddetailed;conventionalized
andcodedwherethe photograph
unmediated,
of reality.
a naturalistic,
uncodedrepresentation
and hencelower in
Doesthis meanthat diagramsare less'real' than photographs,
Not
modality,and that photographyis more true than diagrammaticrepresentation?
Tothe viewersfor whomde Saussure's
diagramsare intended,
theymay in fact
necessarily.
in the sensethat they reveala truth which represents
be more real than the photograph,
moreadequately
what the speechprocessis reallylike.
Realityis in the eyeof the beholder;or rather,what is regardedas real dependson
how realityis definedby a particularsocialgroup.Fromthe pointof viewof naturalism
there is betweenthe visual
reality is definedon the basisof how muchcorrespondence
of an objectandwhatwe normallyseeof that objectwith the nakedeye(or,
representation
to resolvedetailand rendertonal or
in practice,on the capacityof 35mm photography
g h o t o g r a p hcsa, n b e e x p e r i e n c eads ' h y p e r - r e a l ' ,
c o l o u rd i f f e r e n t i a t i oinm: a g e si ,n c l u d i n p
a s s h o w i n g ' t o om u c hd e t a i l '',t o o m u c hd e p t h ' , ' t o om u c hc o l o u r ' t o b e t r u e ) .S c i e n t i f i c
or
realism,on the otherhand,definesrealityon the basisof whatthingsare likegenerically
and doesnot
regularly.It regardssurfacedetailand individualdifferenceas ephemeral,
with the nakedeye.It probesbeyondthe visualappearance
of
stopat whatcanbe observed
but the eye has had a
things.In otherwords,realitymay be in the eyeof the beholder,
culturaltraining,and is locatedin a socialsettingand a history;for instance,in the
or of semioticians
in de Saussure's
day,a communitywhichsaw
communityof linguists,
s .' r e a l i s m ' i s p r o r e a l i t yi n t h a t f o r m ,i n t e r m so f a b s t r a c t i o nasn d d e e p e r e g u l a r i t i eA
ducedby a particulargroup/as an effect of the complexof practiceswhich defineand
constitutethat group.In that sense,
a particularkind of realismis itselfa motivatedsign,
in whichthe values,beliefsand interests
of that groupfind their expression.
As the examplessuggest,definitionsof reality are also boundup with technologies
The relativelyrecentchangefrom the dominance
of
of representation
and reproduction.
b l a c ka n d w h i t et o t h e d o m i n a n coef c o l o u ri n m a n yd o m a i n so f v i s u a lc o m m u n i c a t i o n
showshowquicklythesehistoriescandevelop,
and howcloselytheyare relatedto technologicalchange.For us, now as commonsenseviewers,everydaymembersof societyat
large,the definingtechnologyis perhapsstill that of 35mm colour photography,
as we
suggested
above.But the shift to digitalphotography
is alreadycreatinga newstandard
w,h i c hs t i l la i m sa t e v e rh i g h e r e s o l u t i o n ,a t u r a l i s t icco l o u rr e n d i t i o na,n d
for naturalism
as
so on,but hasin fact madea decrease
in resolutionandcontrastto becomeacceptable
t h e n o r mi n m a n yd o m a i n s .
- that is,a realismis a definitionof what countsas real
Eachrealismhasits naturalism
- a set of criteriafor the real,and it will find its expression
in the 'right', the best,the
'natural'
(most)
form of representing
that kind of reality,be it a photograph,
digitalor
o t h e r w i s eo ,r a d i a g r a mT. h i si s n o t t o s a yt h a t a l l r e a l i s mas r e e q u a lA
. l t h o u g hd i f f e r e n t
realismsexistsideby sidein our society,
the dominantstandardby whichwe judgevisual
r e a l i s ma, n dh e n c ev i s u a m
l o d a l i t yr ,e m a i n fso r t h e m o m e n tn, a t u r a l i s m
a sc o n v e n t i o n a l l y
u n d e r s t o o d , ' p h o t o r e a l i sI nmo' .t h e rw o r d s t, h e d o m i n a nct r i t e r i o nf o r w h a t i s r e a l a n d
what is not is basedon the appearance
of things,on how muchcorrespondence
there is
betweenwhat we can'normally'seeof an object in a concreteand specificsetting,and

Modality

what we canseeof it in a visualrepresentation

- again,atleastintheory,forin effectit is
b a s e do n c u r r e n t l yd o m i n a nct o n v e n t i o nasn d t e c h n o l o g i e s
o f v i s u a lr e p r e s e n t a t i oW
ne
.
judgean imagerealwhen,for instance,
its coloursare approximately
as saturatedasthose
i nt h es t a n d a r dt h, e m o s tw i d e r yu s e dp h o t o g r a p htiecc h n o r o g y .
w h e nc o r o u b
r e c o m em
sore
saturated/we judge it exaggerated,'more
than real',excessive.
When it is lesssaturatedwe judge it'less than real,,'ethereal,,
for instancgor 'ghostly,.And the samecan
be saidabout otheraspectsof representation,
the renditionof detail,the representation
of depth,and so on. Pictureswhichhavethe perspective,
the degreeof detail,the kind of
c o l o u rr e n d i t i o ne,t c .o f t h e s t a n d a r d
t e c h n o r o goyf c o l o u rp h o t o g r a p hhya v et h e h i g h e s t
modality,and are seenas 'naturalistic'.As detail,sharpness,
colour,eic. are reducedor
amplified,
as the perspective
flattensor deepens,
so modaritydecreases.
Like manyotheradvertisements,
the advertisement
in plate 2 is a compositetext. It
showsa pictureof the product(the jar of instantcoffee),
with a verbarcaption,and a
p i c t u r ew h i c hv i s u a l i z et sh e p r e a s u rteh e p r o d u c t
w i i l a f f o r d .T h i sp i c t u r es, h o w i n gt w o
loverssharingan intimatemoment,usessoft focusand soft
colours,tendingtowardsthe
samegolden-brown
hue,andso deliberately
,whatusing
loweringmodality,representing
the
productwill be like'as fantasyor promise,as'what
might
be,,
rather
than
as
reality,
as
'what is'.
The pictureof the productitself,however,
is in sharperfocusand usesmore
s a t u r a t ea
d n d d i f f e r e n t i a t ecdo l o u r st:h e p r o d u c ti s g i v e n
h i g h e rm o d a l i t yh, i g h e r e a l i t y
value,thanthe promiseof blissattachedto it, andthe advertisement
as a wholetherefore
accordsvaryingdegrees
of'credibility'tothe differentrepresentations
it contains,just
like
t h e t e x t o n ' A b o r i g i n arle l i g i o n T
' . h e l o w e rm o d a l i t yo f t h e p h o t oo f t h e r o m a n t r cs c e n e ,
however,
is not a matterof the sceneitselfbeingimprobable
or fantastic(althoughthaf
t o o ,o f t e nh a p p e nisn a d v e r t i s i npgh o t o g r a p h sP)r. o b a b laesw e l l
a si m p r o b a b e
t ev e n t m
s ay
havehighor low modality.
Just as onecan sayTherecertainlyare ghtosts
(highmodality)
and I believeghostsmay exist (ow modalitfl, so one can
also showrealisticand not-sorealisticdepictions
of ghosts.
what is the difference
between
theseusesof colour?we wouldput it thisway:the more
that is taken away,abstractedfrom the coloursof the representation,
tne more c0lour
i s r e d u c e dt h
, e l o w e rt h e m o d a l i t yT. h e r ei s a c o n t i n u u mw h i c h
r u n sf r o m f u l l c o l o u r
saturationto the absence
of colour,blackandwhite,in whichonlythe brightness
valuesof
t h e c o l o u r st ,h e i r ' d a r k n e s s ' o r ' l i g h t n e sr es m
' , a i n sT. h e r ei s a l s oa c o n t i n u u m
w h i c hr u n s
f r o m f u l l c o l o u rd i f f e r e n t i a t i ot o
n a ' r e d u c e dp a l e t t e ' a n de v e n t u a l lm
y onochromF
e .o r
e x a m p l ee,i g h t e e n t h - c e n tluarnyd s c a ppea i n t i n g( e . g C
. l a u d eL o r r a i n w
) a so f t e nr e s t r i c t e d
to variousshadesof brownfor the foregroundand to desaturated,
silverybluesfor the
d i s t a n c eT'h i si s n o tt h eo n l yw a yi n w h i c ha b s t r a c t i of nr o m' n a t u r a l i s t i c ,
c o l o u ri s p o s s i b l e .
The colour of many objectsis not even.pare skin,for instance,
may vary in redness,
may havethe blueveinsshowingthrough,and so on, and such
differences
may eitherbe
rendered
or abstracted
from.In otherwords,colourmaybe idealized
to a greateror lesser
d e g r e-e a s c a l ew h i c hr u n sf r o mn a t u r a l i s t ipch o t o g r a p hv yi a
t h ec h o i c eo f d i f f e r e nvt a l u e s
of a colourfor the representation
of lightand shade,
to the ffat,unmodulated
colourused
by childrenin their drawings,or, for example,
in the work of painterssuchas Matisse.
Matissewas not a chird,of coursgwhenhe producedthe paintings
we now admire.His

I59

160.

Modality

unmodulatedcolours expressed
a differentview of what counts as real, as do the
u n m o d u l a t ecdo l o u r si n c h i l d r e n 'dsr a w i n g-s w e w i l l c o m m e not n t h i s i n m o r ed e t a i l a t e r .
in suchimages.
The
Fromthe pointof viewof naturalism,
however,
modalityis decreased
f r o m h i g ht o l o w
continuum
f r o m m o d u l a t etdo f f a tc o l o u ri s a t t h es a m et i m ea c o n t i n u u m
modality.And in both casesthe rule applies:the greaterthe abstraction(away from
saturation,
differentiation
and modulation),
the lowerthe modality.
It shouldbe stressed
that what we are talkingabout is not abstractionfrom what we
actuallysee,from 'the real world'. The literatureof other agesand culturesatteststo
the fact that peoplehavemarvelledat the 'lifelikeness'
of workswhich,by our standards,
are f ar from 'naturalistic'.
What we are talkingaboutat this pointis abstractionrelative
to the standards
of contemporary
naturalisticrepresentation.

M O D A L I T YM A R K E R S
r f n a t u r a l i s t im
c o d a l i t vi n. t e r m so f
S of a r w e h a v ed i s c u s s et hder o l eo f c o l o u ra sa m a r k e o
t h r e es c a l e s :

(I) Coloursaturation,ascalerunningfromfullcoloursaturationtotheabsenceofcol
that is,to blackandwhite.
(2) Colourdifferentiatio4a scalerunningfrom a maximallydiversified
rangeof colours
to monochrome.
(3) Colourmodulation,ascalerunningfromfullymodulatedcolour,with,forexample
useof manydifferentshadesof red,to plain,unmodulated
colour.
A t o n e e n d o f t h e s es c a l e st h e p a r t i c u l a rd i m e n s i o on f c o l o u r i s m a x i m al yl r e d u c e d .
A t t h e o t h e re n di t i s m o s tf u l l y a r t i c u l a t e du,s e dt o i t s m a x i m u mp o t e n t i a lE. a c hp o i n to f
the scalehasa certainmodalityvaluein termsof the naturalisticstandard.However,
the
p o i n to f h i g h e sm
t o d a l i t yd o e sn o t c o i n c i d w
e i t h e i t h e re x t r e m eo f t h e s c a l e n: a t u r a l i s t i c
modalityincreases
as articulationincreases,
but at a certainpoint it reachesits highest
valueand thereafterit decreases
again.Naturalisticmodalityscalescouldthereforebe
r e p r e s e n t eads i n t h ef o l l o w i n g
example:

lvlaximumcolour saturation

E i a c ka n d w h i t e
Lowest modalrty

H i g h e s tm o d a l i t y

Low(er) modality

fig s.+ Modality scale for colour saturation

We will now discussthe other key markersof visualmodalityon which we already

t o u c h e di n o u r d i s c u s s i oonf f i q u r e5 . 3 .

Modality

(4

Contextualization,
a scalerunningfrom the absence
of background
to the mostfullv
articulatedanddetailedbackground.

W i t h i nt h e n a t u r a l i s t icco d i n go r i e n t a t i o tnh, ea b s e n coef s e t t i n gl o w e r sm o d a l i t yB. y b e i n g

'decontextualized',
s h o w ni n a v o i d ,r e p r e s e n t epda r t i c i p a n tbse c o m eg e n e r i ca, ' t y p i c a l
example',ratherthan particular,and connected
with a particularlocationand a specific
momentin time. The scaleof 'contextualization'
runs from'full contextualization,,
to
' p l a i n ,u n m o d u l a t e d
b a c k g r o u n d ' . O nset e pa w a yf r o m ' f u l l c o n t e x t u a l i z a t i owne, f i n d
settingswhichare out of focusto a greateror lesserdegree,
or which losedetailthrough
overexposure/
resultingin a kind of etherealbrightness,
or underexposure,
resultingin
muddydarkness,
or throughthe lossof visualdetailin the depiction.Furtherdecontextualizationcan be achievedthroughellipsis:a few 'props,sufficeto suggesta setting,
o r a s m a l l ,i r r e g u l a r lsyh a p e dp a t c ho f g r e e nu n d e rt h e f e e t o f a f i g u r ew i t h a f e w l i n e s
suggesting
grassindicates
the setting,whilethe rest of the paperis left blanl<.
0r perhaps
the backgroundmay merelyshowan irregularpatternof light and shade,or a field of
unmodulated
colour,or black,or white.
A g a i n ,t h e m o s tf u l l y a r t i c u l a t e db a c k g r o u nddo e sn o t h a v et h e h i g h e snt a t u r a l i s t i c
m o d a l i t yT. h e l i m i t a t i o n si m p o s e db y t h e r e s o l u t i o no f s t a n d a r d3 5 m m p h o t o g r a p h i c
emulsions
and by the depthof field of standardlenseshaveaccustomed
us to imagesin
w h i c ht h e b a c k g r o u nids l e s sa r t i c u l a t e d
t h a n t h e f o r e g r o u n dW. h e nt h e b a c k g r o u nids
sharperand motedefinedthanthis,a somewhatartificial,'more
than real,impression
will
r e s u l -t a s ,f o r i n s t a n cien o l d e rH o l l y w o om
d o v i e s h o ti n a s t u d i ow i t h b a c kp r o j e c t i o (na
c l o s e - uo
p f a n a c t o ri n a c a r ,i n f r o n t o f a r e a rw i n d o wb e h i n dw h i c hw e s e et h e r e c e d i n g
l a n d s c a pi e
n s h a r pf o c u s )o, r i n m u c hS u r r e a l i spt a i n t i n gs,u c ha s i n t h ew o r k o f S a l v a d o r
Dali.
(5) Representation,
a scalerunningfrom maximumabstraction
to maximumrepresentat i o n o f p i c t o r i adl e t a i l .
An imagemay showeverydetail of the represented
participants:
the individualstrands
of hair,the poresin the skin,the creasesin the clothes,the individualleavesof the tree,
and so on, or it may abstractfrom detailto a greateror lesserdegree.Again,there is a
p o i n tb e y o n d
w h i c ha f u r t h e ri n c r e a soef d e t a i lb e c o m e s ' h y p e r - r eaanl ,d h e n c el o w e ri n
c,
m o d a l i t yf r o m t h e p o i n t o f v i e w o f ' p h o t o g r a p h i n
a t u r a l i s mS. i m i l a r l yi,n d i s c u s s i n g
decontextualization,
above,
we havepointedout that reducedrepresentation
of detailmay
form one of the waysin whichthe modalityof backgrounds,
of what is .distant,,is lower
t h a nt h e m o d a l i t yo f t h ef o r e g r o u n(dt h e r ei s a p a r a l l ehl e r ew i t h t h e l o w e rm o d a l i t yo f t h e
p a s tt e n s ei n l a n g u a g e ) .
In photography
it is not only sharpness
of focus,but alsoexposure
which can reduce
detail.In artwork a varietyof techniques
couldbe rankedon a scale from maximumto
m i n i m u md e t a i lT' e x t u r ce a nb e c o m set y l i z e dr ,e n d e r ebdy l i n e sw h i c ht r a c et h ef o l d si n
the
clothes,for example,
and theselinesmay be manyand fine,as in detailedengravings,
or
few andcoarse/as in quickand readystylesof drawing.In medicaldrawings,
for example,

r61

r62

Modality

texturemay becomeentirelyconventional:
dotsto indicatethe textureof onelayerof skin,
short,curvedlinesto indicatethetextureof another.
Texturecanalsobeomittedaltogether
- the participantis thenrepresented
merelyby the linesthat traceits contour.Beyondthis,
the contourmay be simplifiedto differentdegrees:
a headmay becomea circle,the eyes
two dots,the moutha short,straightline.Diagramsand geometricalart take abstraction
evenfurtherand reducethe shapeof thingsto a smallvocabulary
of abstractforms,as in
s f M o n d r i a no, r i n f i g u r e5 . 2 , d e S a u s s u r es' sc h e m a t i z e' sdp e e c hc i r c u i t ,
t h e p a i n t i n go
diagram.
6)

Depth,a scalerunningfrom the absence

of depthto maximallydeepperspective.

By the criteriaof standardnaturalism,

centralperspective
hashighestmodality,followed
perspective,
by angular-isometric
perspective,
followedby frontal-isometric
followedby
depthcreatedby overlapping
only.Again,perspective
canbecome'more
thanreal,,aswhen
strongconvergence
of verticallinesis shown,or a'fish-eye'perspective
is used.
(7) Illumination,a scalerunningfrom the fullestrepresentation
of the play of light and
shadeto its absence.
participants
Naturalistic
depictions
represent
astheyareaffectedby a particularsourceof
i l l u m i n a t i o nL.e s sn a t u r a l i s t iicm a g e so,n t h e o t h e rh a n d m
, a ya b s t r a cftr o m i l l u m i n a t i o n ,
and showshadowsonly in so far as they are requiredto modelthe volume,especially
of
r o u n do b j e c t sT. h e yh a v e ' s h a d i n g ' r a t h e
t hr a n s h a d o w . 0 rt h e y u s es h a d i n gt o i n d i c a t e
recedingareasand highlightsto indicateprotrudingareas,often in wayswhich haveno
e x p l a n a t i oi n t e r m so f t h e l o g i co f i l l u m i n a t i o T
n .h i sc a nb ed o n et o d i f f e r e ndt e g r e e w
s :i t h
a fully modulateddarkeningof the areasof shadow;with just two degreesof brightness,
o n ef o r t h e ' l i t ' a r e a sa n d o n ef o r t h o s ei n s h a d o ww; i t h m o r eo r l e s sd e n s eh a t c h i n g
or
dottingof theshaded
areas;andsoon.At theextremeendof thescale,Iightandshadeareabstractedfrom altogether,
andlineratherthanshadingis usedto indicatereceding
contours.
(B) Brightness,a scale runningfrom a maximum number of different degreesof
brightness
to just two degrees:
blackandwhite,or dark greyand lightergrey,or two
brightness
valuesof the samecolour.
Brightnessvaluescan also contrastto a greateror lesserdegree:in one picturethe
difference
between
the darkestandthe lightestareamaybeverygreat(deepblacks,bright
whites),in anotherthe differencemay be minimal,so that a misty,hazyeffect is created.
Thepaintingsof Rembrandt
are interesting
from this pointof view,in part because
hisuse
o f i l l u m i n a t i oann d b r i g h t n e si s s o o f t e ni n v o k e d
asa paradigm
e x a m p l eo f n a t u r a l i s m ,
and in part becauseof the way his subtledivergences
from naturalismoften acquire
i d e a t i o n af ul n c t i o n si n, a b r o a d l ya l l e g o r i c asle n s e .
T h e a b i l i t yo f p h o t o g r a p ht yo r e n d e rb l a c ka n d w h i t e i s l i m i t e da, s i s i t s a b i l i t yt o
differentiatebrightness
values.Again,a contrastrangeand a rangeof brightness
values

Modality .163

whichexceeds
this ability may be experienced
as \morethan real,and henceas beinqof
lowermodality.
I t f o l l o w sf r o m o u r d i s c u s s i ot nh a t m o d a l i t yi s r e a l i z e db y a c o m p l e xi n t e r p l a yo f
visualcues.The sameimagemay be 'abstract'in terms of one or severalmarkersand
'naturalistic'in
termsof others.Impressionist
paintings,
for example,
oftenhavea narrow
brightness
range,andabstractfrom light and shadow,
but they havea highiynaturalistic
approachto colour.Yet,from this diversityof cuesan overaliassessment
of modalitvis
derivedby the viewer.
From all this it might seemthat the realizationof modalityin imagesis muchmore
complexandfinelygradedthanthe realizationof modalityin language.
yet language,
too,
allowscomplexcombinations
of differentmodalitycues.Take,for instance,
the sentence
I absolutelydon't think he could possiblyhave done if. Is this 'low,, 'middle, or ,high,
modality?How does one 'compute'thesevarious modalitycues into one 'degreeof
credibility'?Frequently
thereareevencontradictions:
1f isprobablydefinitelytruethat . . .
And in language,
too,the valueof modalitycuesdepends
on context.In academic
writing,
for example,
qualifications
suchas It may wellbe the case. . .or It is quite possible
that. . .
(both low modality,
strictlyspeaking)servein fact to increase
the credibilityof the text,
as indicatorsof the carewith whichthe writer,sjudgements
weremade,and henceof the
r e l i a b i l i toyf t h e s ej u d g e m e n t s .

C O D I N GO R I E N T A T I O N
So far we havedescribed
the valueof modalitymarkersin termsof the naturalistic
criteria
for'what countsas real'.We havehypothesized
that the abilityof moderncotourphotggr a p h yt o r e n d e rd e t a i l b
, r i g h t n e scso, l o u re, t c .c o n s t i t u t ef so r o u r c u l t u r et o d a ya k i n do f
s t a n d a r df o r v i s u a lm o d a l i t yw. h e n t h i s s t a n d a r di s e x c e e d e ad n, i m a g eb e c o m e s ' m o r e
than real'- an effectwhich can be achievednot only in art (and is oftenthe favoured
m o d a l i t yi n S u r r e a l i s mb) u
, t a l s ob y m e a n so f t h es p e c i at le c h n i q u em
s ,a t e r i a las n de q u i p mentof studiophotography.
A certainstandardof photographic
naturalism,
dependent
on
the stateof photographic
technology
and on currentphotographic
practices,
henceever
evoiving,has becomethe yardstickfor what is perceived
as ,real, in images,evenwhen
t h e s ei m a g e sa r e n o t p h o t o g r a p h U
s .n d e r p i n n i nt hg i s i s t h e b e l i e fi n t h e o b j e c t i v i t o
yf
p h o t o g r a p h ivci s i o n ,a b e l i e fi n p h o t o g r a p hays c a p a b l eo f c a p t u r i n gr e a l i t ya s i t i s ,
u n a d u l t e r a t ebdy h u m a ni n t e r p r e t a t i o n
B.e h i n dt h i s , i n t u r n , i s t h e p r i m a c yw h i c h i s
accordedto visualperception
in our culturegenerally.
Seeinghas,in our culture,become
s y n o n y m o uwsi t h u n d e r s t a n d i nwge. ' l o o k ' a t a p r o b l e mw. e ' s e e ,t h e p o i n t .w e a d o p ta
'viewpoint'. 'focus'
we
on an issue,we 'seethingsin perspective,.
Theworld,aswe seeit,
(ratherthan 'as we knowit', andcertainlynot ,aswe hearit,
or'as we feel it,) hasbecome
the measure
for what is 'real,and 'true,.
S ov i s u am
l o d a l i t yr e s t so n c u l t u r a l l a
y n dh i s t o r i c a l ldye t e r m i n esdt a n d a r dosf w h a t i s
real and what is not, and not on the objectivecorrespondence
of the visualimageto a
realitydefinedin somewaysindependently
of it. At the momenthologramsare probably

r64

Modality

s t i l l s e e nb y m o s t p e o p l ea s ' m o r e t h a n r e a l ' .I n t h e i m a g e sw e a r e m o s t u s e dt o , t h e
absence
of thethird dimension,
theffatness
of the picture,doesnot functionas an indicator
just asthe absence
of low modality,
of perspective
in cultureswhoseart doesnot employit
doesnot functionas an indicatorof low modalityfor membersof thosecultures.
As we havealreadydiscussed
in relationto de Saussure's'speech
circuif',however,
even
w i t h i no u r o w nc u l t u r et h e s a m es t a n d a r dfso r w h a t i s ' r e a l ' a n dw h a t i s n o t d o n o t a p p l y
in every context.In technologicalcontexts,a different conceptof reality underlies
v i s u am
l o d a l i t ya, c o n c e p t w ceo u l dc a l| ' G a l i l e a rne a l i t y 'I.n t h ee a r l ys e v e n t e e nct e
hn t u r y ,
Galileo
wrote:
I do not find myselfabsolutelycompelledto apprehendtobjectsJas necessarily
accompanied
by suchconditions
as that they mustbe white or red,bitter or sweet,
s o n o r o uos r s i l e n t s, m e l l i n gs w e e t l yo r d i s a g r e e a b l y . . . .tl h i n k t h a t t h e s et a s t e s ,
smells,colours,etc.with regardto the objectin which they appearto resideare
nothingmore than mere names.. . . I do not believethat thereexistsanythingin
externalbodiesfor excitingtastes,smells,sounds,etc.,exceptsize,shape,quantity
a n dm o t i o n .
( q u o t e di n M u m f o r dt,9 3 6 i 4 8 )
H e r e ' r e a l ' m e a n s ' w h acta n b e k n o w nb y m e a n so f t h e m e t h o d so f s c i e n c et' h
; a t i s ,b y
m e a n so f c o u n t i n gw, e i g h i n a
g n d m e a s u r i n gB.y t h i s s t a n d a r do f w h a t i s r e a l ,a t e c h n i c a l
linedrawing,withoutcolouror texture,withoutliqhtor shade,
andwithoutperspective,
can
h a v eh i g h e rm o d a l i t yt h a na p h o t o g r a p h
E.v e r y d acyo m m o ns e n s en a t u r a l i s m
a n dr e a l i s m
n o l o n g e rm e r g eh e r e .T h e r e a l i s m( a n d h e n c et h e ' n a t u r a l i s m ' o) f s c i e n t i f i c - t e c h n i c a l
i m a g e si s o f a d i f f e r e n kt i n d ,b a s e di,n t h e e n d ,o n t h e q u e s t i o n s , C awne u s ei t ? , , , C a n
we measurethe real dimensions
from it?', 'Can we find out from it how to set up the
experiment?',
and so on. Whateverdoesnot contributeto this purposemerelyaddsa
d i m e n s i oonf i l l u s i o n i s m ' ttoh e p i c t u r ea, n dd i l u t e s' G a l i l e a n
realism'with
c o m m o ns e n s e
' n a t u r a l i s mT' .h e l a t t e ro, f c o u r s e ,
i s s o m e t i m edso n ef o r t h e p u r p o s e
of communicating
s c i e n t i f i ci d e a so r t e c h n o l o g i c a
l m p l e x i t i et o
co
s a p u b l i co f n o n - i n i t i a t eIsn. h i s b o o k
writing Biology(1990) GregMyerscompares
the reportingof the'same'research
findingsin specialist
and popularjournalssuchas ScientificAmerican,andvisualrepresental ' ,h i l ei n t h e f o r m e rs p a r s e
t i o n si n t h e l a t t e rt e n dt o b e l a v i s hf ,u l l - c o l o uar n d' h y p e r - r e a w
linedrawingsare the only form of visualimage.Furthermore,
we haveto be awarethat
thereare competingtheoriesof realityin today'sscience,
despitethe fact that for many
p r a c t i c apl u r p o s eG
s a l i l e a rne a l i t yr e m a i n o
s f o v e r r i d i nigm p o r t a n c e
A.l t e r n a t i vteh e o r i e s
mightleadto differentstandards
for highand low modality.
I n o t h e rc o n t e x ttsh e ' h y p e r - r e a l ' d oneost h a v et h ed e c r e a s emdo d a l i t yi t h a si n ' p h o t o graphic'naturalism
M .a g a z i n p
e h o t o so f f o o da r e o n ee x a m p l eA. d i f f e r e npt r i n c i p l e
for
what countsas real operates
here,the converse
of Galileanreality:the morea picturecan
c r e a t ea n i l l u s i o no f t o u c ha n d t a s t ea n d s m e l l t, h e h i g h e ri t s m o d a l i t yI.n s u c hi m a g e s
e v e r y t h i nigs d o n et o a p p e atlo ' s e n s o r y ' q u a l i t i erse:a l i t yh e r ei s c o n s t i t u t epdr e c i s e lbyy
t h o s es e n s a t i o nwsh i c hG a l i l e o
brandea
d s i l l u s i o n st e: x t u r ec,o l o u r i ' f e e Il 't. i s h e r et h a tt h e

Modality

L65

affective
v a l u e so f c o l o u r sc o m ei n t ot h e i ro w n ,f o r e x a m p l eT. h ee m o t i v ev a l u eo f c o l o u r
is sometimes
seenas a generalcharacteristic
of colour.But in scientific-technological
c o n t e x t sc ,o l o u rm a yb ec o n v e n t i o n(aml o r eo r l e s sa r b i t r a r y ' c o l o ucro d e s ' t of a c i l i t a t teh e
reading
o f c o m p l ed
x i a g r a m sa) ,n di n n a t u r a l i s m
c o l o u r sa r et h e r e ' b e c a u st hee ya r et h e r e
in reality'.Fromthe pointof viewof the 'sensory'definitionof reality,on the otherhand,
coloursare thereto be experienced
sensuallyand emotively- it is for this reasonthat
peopleenjoythe highlysaturated
andunmodulated
coloursof,say,Matisse,
or that children
enjoythe highlysaturated
andunmodulated
coloursof their plastictoys.Withinnaturalism
t h e s ec o l o u r sa r e ' l e s st h a n r e a l ' ,b u t w i t h i na r e a l i s mt h a t t a k e ss u b j e c t i veem o t i o nas n d
sensations
as the criterionfor what is realandtrue,theyhavethe highestmodality.
Thereis,finally,a third area in whichthe standardof 'photographic,
naturalismdoes
not apply,the areaof'abstractreaiism'- bothin science(e.9.the'speechcircuit'diagram
i n f i 9 u r e s5 . 1 a n d5 . 2 ) a n di n a b s t r a cat r t . Hi g h e re d u c a t i oinn o u rs o c i e t iys ,t o q u i t es o m e
extent,an educationin detachment,
(and against
abstractionand decontextualization
naturalism),
andthis resultsin an attitudewhichdoesnot equatethe appearance
of things
with reality,but looksfor a deepertruth 'behindappearances'.
Justas academically
trained
personsmay accordgreatertruth to abstractexpositorywriting than to storiesabout
c o n c r e t ei ,n d i v i d u ael v e n t sa n d p e o p l es, o t h e y m a y a l s o p l a c eh i g h e rv a l u eo n v i s u a l
representations
whichreduceeventsandpeopleto the'typical,,andextractfrom themthe
' e s s e n t i aqlu a l i t i e s ' .
Whileour ideashereare drawnto a largeextentfrom the theoreticalwork of Jurgen
Habermas(especially
his Theoryof communicative
Action,1984), and to someextent
f r o m t h a t o f B o u r d i e u( 1 9 8 6 ) ,w e w i l l u s eB e r n s t e i nt' es r m ' c o d i n go r i e n t a t i o n( ,1 9 8 1 )
for thesedifferentrealityprinciples.
Codingorientations
are setsof abstractprinciples
whichinformthe way in whichtextsare codedby specificsocialgroups,or withinspecific
i n s t i t u t i o n ac lo n t e x t sW. e d i s t i n g u i st h ef o l l o w i n g :
( 1 ) Technologicalcoding orientations,which have, as their dominant principle,the
'effectiveness'
of the visual representation
as a 'blueprint'.whenever colour,for
example,is useless
for the scientificor technological
purposeof the image,it has,in
this context,low modality.
( 2 ) Sensory
codingorientations,which
are usedin contextsin whichthe pleasure
principle
is allowedto be the dominant:certain kinds of art, advertising,fashion,food
photography,
interiordecoration,
and so on. Herecolouris a sourceof pleasureand
affectivemeanings,
and consequently
it conveys
highmodality:vibrantreds,soothing
b l u e sa, n ds o o n- a w h o l ep s y c h o l o goyf c o l o u rh a se v o l v etdo s u p p o rtth i s .
( 3 ) Abstractcodingorientations,which
are usedby sociocultural
elites- in'high'art, in
academicand scientificcontexts,
and so on. In suchcontextsmodalityis higherthe
morean imagereduces
the individual
to the general,
andthe concreteto its essential
qualities.Theabilityto produceand/orreadtextsgroundedin this codingorientation
i s a m a r ko f s o c i adl i s t i n c t i o n
o ,f b e i n ga n ' e d u c a t epde r s o n ' o ar ' s e r i o u a
s rtist'.
G ) Thecommonsensenaturalisticcodingorientation,whichremains,for the time being,
t h e d o m i n a not n ei n o u r s o c i e t yI t. i s t h e o n ec o d i n go r i e n t a t i oanl l m e m b e ros f t h e

r66

M odality

c u l t u r es h a r ew h e nt h e ya r e b e i n ga d d r e s s eads ' m e m b e ros f o u r c u l t u r e 'r, e g a r d l e s s

of how much educationor scientific-technological
training they have received.
Individuaw
l si t h s p e c i ael d u c a t i oonr g r o u pa l l e g i a n cm
e a yd r a wo n n o n - n a t u r a l i s t i c
codingorientations
in certaincontexts,
but theyare likelyto revertto the naturalistic
codingorientationwhenthey are 'just beingthemselves'.
Theymay,for example,use
t h e a b s t r a c ct o d i n go r i e n t a t i ow
n h e nv i s i t i n ga g a l l e r ya, n d t h e n a t u r a l i s t icco d i n g
o r i e n t a t i ow
n h e nw a t c h i n g
t e l e v i s i oonr r e a d i n ga m a g a z i n eF.o r t h o s ew i t h o u ts u c h
education,
however,
abstractand technological
imageswill neverhavehighmodality
a n d a l w a y sr e m a i n ' u n r e a l T
' . o d a yh, o w e v e nr ,a t u r a l i s mi s c o m i n gi n t o c r i s i s a
, sa
resulo
t f n e ww a y so f t h i n k i n ga n dn e wi m a g et e c h n o l o g i eI n
s .t h i sc o n t e xtth e r o l eo f
s o m eo r a l l o f t h en o n - n a t u r a l i sct iocd i n go r i e n t a t i o ni ssl i k e l yt o b e c o m o
ef i n c r e a s i n g
importance.
T h e d i a g r a mi n f i g u r e 5 . 5 s h o w sh o w t h e s a m ec o l o u rc o n t i n u u mr,u n n i n gf r o m ' n o
abstraction'to'full abstraction'(abstraction
alwaysbeinga matterof degree)can have
differentmodalityvalues,accordingto the four codingorientations.
It is drawnherefor
coloursaturation,
but it couldalsohavebeendrawnfor anyof the othermodalitymarkers
we discussed
in the previous
section.

M O D A L I T YI N M O D E R NA R T
The issueof modalitybecomesparticularlycomplexin modernart, becauseit has,to a
largeextent,beenthe projectof modernart to redefine'reality',and to do so in contrad i s t i n c t i otno p h o t o g r a p hni ca t u r a l i s mI n. t h i ss e c t i o nw e w i l l a t t e m p t o d i s c u sas f e w o f
t h e i s s u e sb,e g i n n i nwgi t h s o m eA u s t r a l i aenx a m p l e s .
( Somewhatlessthan
f u l l c o l o u rs a t u r a t i o n )

F u lI c o l o u r
saturation
Scientific/techno
Iogical

L0wEST +-___________
MODALITY

B l a c ka n d w h i t e

> HIGHEST
MODALITY

(a0ove

Abstract
modality)

+---------+

LowEST +----_____+HIGHEST
MODALITY
IVIODALITY

Naturalistic

(below
maxtmum
+---------+
mooailty)

Sensory

H r G H E I I+ - _ _ - _ _ _ _ - _ _ _

H I G H E S T + _ _ - _ _ _ _ _ _ +L 0 w E S T
MODALITY
MODALITY

IVIODALITY

fig S.S Modalityyaluesof coloursaturationin lour codingoiientations

LowESr
MODALITY

M odality

I67

T h e p i c t u r ei n p l a t e3 s h o w sw i l l i a m D o b e l l ' sp o r t r a i to f J o s h u as m i t h ,a p a i n t i n g
w h i c hw o n t h e a n n u a lA u s t r a l i a nA r c h i b a l dP r i z ec o m p e t i t i ofno r p o r t r a i tp a i n t i n gi n
1 9 4 3 . I tw a st h ef i r s t m o d e r np a i n t i n tgo d o s o .A l l p r e v i o uws i n n e r sh a db e e nc o n v e n t i o n a l
'academic'portraitists,
s t a y i n gw e l l w i t h i nt h e b o u n d so f n a t u r a l i s t idc e p i c t i o nA. f t e r
Dobellhad beenawardedthe prize,a numberof conservative
painterstook the trustees
of the prizeto court for givingit to a paintingwhich,they argued,was not eligible,as it
was not a portrait but a caricature.The prosecutor,
GarfieldBarwick,interrogated
Dobela
l b o u te v e r yd e t a i lo f t h e p a i n t i n ga, s k i n gh i m w h e t h e rh e h a d f a i t h f u l l yr e p r e sentedthe ears,the neck,the arms,and so on. The painter,in exasperation,
answered:
'Yes,within the limits
of arf.' From a naturalisticpoint of view,the paintingdoesof
c o u r s eh a v ec o m p a r a t i v elloy w m o d a l i t yb, o t h i n t h e d i r e c t i o no f ' l e s st h a n r e a l ,a n d i n
the directionof 'more than real'. Colourdifferentiationis greatlyreduced,
to a palette
of orange,yellow and brown. The representation
of detail, on the other hand, is
a m p l i f i e de,x a g g e r a t e d , ' m ot hr e
a nr e a l ' .F r o ma n a t u r a l i s t ipco i n to f v i e wt,h e p r o s e c u t o r
was right. But he applieda criterionwhichwas no longervalid in the contextof modern
art, a way of matchingmodalityvaluesto the scalesof colourdifferentiation,
representation and so on which,in the historyof modernart, had beensuccessfully
contested
decadesearlier.In modernart, the truth of paintingno longerlies in beingfaithful to
a p p e a r a n c ebsu/ t i n b e i n gf a i t h f u lt o s o m e t h i n e
g l s e- f o r e x a m p l et ,o s o m em o d e r n
abstracttruth, in the caseof this rather 'expressionist,
painting,to .the spirit of the
man', and'the essence
of what he lookslike',as Dobellhimselfformulatedit durinqthe
t r ia l .
Attemptsto alter definitionsof reality are always likely to producescandal,and
t h e r e f o r ree s i s t e dw. h e t h e ri n t h et r i a l so f D . H .L a w r e n c e n, so v eol r D o b e l l ,psa i n t i n g , t h e
issues
arefar largerand morefar-reaching
thanaesthetics
or artisticconvention.
Changes
in the definitionof reality haveprofoundcultural and socialeffects,and this helpsto
explainextremereactiotrs
suchas the proscription
of'entarteteKunst,or the burninqof
books.
D e s p i t teh e t r i a l ,f u t u r ew i n n e r so f t h e A r c h i b a l dP r i z ew o u l d w
, i t h o u te x c e p t i o nb,e
modernartistsratherthan conventional
portraitpainters.Thepictureshownin plate4, a
portraitof the writer Patricl<
White by Louisl(ahan,is one of them.It showsa different
m o d a l i t yc o n f i g u r a t i oan ,d i f f e r e nst e t o f a b s t r a c t i o nasn d a m p l i f i c a t i o nUs n
. l i k eD o b e l l ,
l(ahandoesnot deviatefrom tlrenaturalisticrepresentation
ofcletail;ifone disregards
the
strangelyunseeing
eyes,White'sfeaturesare rendered
with naturalisticfaithfulness.
But
textureis amplified:the figureof White seemsto havebeencarvedout of somepgrous,
chalkyrock,and the renditionof its surfaceis so detailedthat one can almostfeel its
c o l d ,w e t t o u c h- a ' s e n s o r yo' r i e n t a t i o nb,u t i n t h e d i r e c t i o no f d i s p l e a s u r e
a t h e rt h a n
p l e a s u r ec .o l o u ro, n t h e o t h e rh a n d i,s g r e a t l yr e d u c e di n: w h a t i s a t o n c ea p u no n w h i t e , s
nameand a symbolicgesture,White is drainedof all colour.ThusKahandepictshim as
c o l d ,h a r d ,a l m o s tr e p u l s i vteo t h e t o u c h ,a n d t h e e x p r e s s i oonf t h i s ' t r u t h , h a s t a k e n
precedence
over the faithful renderingof outward appearances;
indeed,has become
p o s s i b loen l yb y m e a n so f t h e s e ' d e v i a t i o n s ' f r o
t hme n a t u r a l i s t isct a n d a r dT. h ev i s u aol u n
remindsus that modalityis alwaysrelatedto the values,meaningsand beliefsof a

168

M odality

particulargroup/in this casean 'ordering'of the figureof PatrickWhitewithinthe system

of Australianhighculture,andof Australiansocietygenerally.
0 u r s e c o n de x a m p l ep e r t a i n st o t h e g e o m e t r i ca b s t r a c t i o n i somf t h e 1 9 2 0 s .W h e n
Europeanpainters/after studyingvisiblerealityfor centuries,
beganto conceiveof it as
madeup of abstract,geometrical
elements(circles,cones/squares,
triangles),realitywas
just as hadalreadyhappened,
redefined
as a configuration
of basicelements,
for instance,
in physics.Within this new definitionof reality,paintersat first still soughtto produce
recognizable
representations,
as shownin figure5.6. Mondrian,who triedto painttreesin
thisway,complained
that it wasdifficultto represent
treesas arrangements
of rectangular
shapes.But soon these painterswent a step further and abandonedthe attempt to
reconcile
the visiblesurfaceappearance
of thingswith their geometricinnerstructure(see
figure5.7). Fromhereitwas onlyonestepto,forinstance,
GerritRietveld's
ColourProject
(t923-4).
for the SchroderResidence
Rietveld's
work (figure5.8) is no longera reduced,
abstractrepresentation
of reality,
but a designfor a new reality,yet to be constructed.
0f course/blueprintsand planshad
existedalongside
visualrepresentation
longbeforethe 1920s,but in separate
domains.In
the twentiethcentury,however,
they becameintertwined.Art becameintertwinedwith
design,just as sciencehad alreadybecomeintertwinedwith technology.
The boundaries
betweenrepresenting
reality and constructingreality becameblurred.And when real
thingswereproduced
from designs
suchas these,the processes
of abstraction
couldcome
' a g ea
f u l l c i r c l ea n dy i e l d' n a t u r a l i s t i ci m
s g a i n( f i g u r e5 . 9 ) .
In the work of Ryman,a contemporary
Americanartist,abstractionis perhaps
takento
its limit. Many of his paintingsare,at leastat first glance,white surfaces.Everythingis
reduced,
everything
abstracted.
Thereis no colour,no line,no background.
And, in terms
of our earlierchapters,
there is neitherrepresentation
of actionor of socialconstructs,
nor yet any indicationof textuality,of composition.This truly is the degreezero of
representation.

fi9S.O Cad-playerc(TheovanD0esburg,lg16-17)(DoesburgArchive,TheHague)

risl'z

compositiong(ahstractversionofcad-ptayers)(TheovanDoesburg,rgl6-u)(fromJaff6,1967)

rig s'a cobur Prciect fot theschrfuderResidence(Gerrit

Rietyeld,lgz3.4) (catal0gue 81, stedetijk Museum,Amsterdam)

I70

Modality

, t',1" ' '

-:l'::li:j::t:!

figS,f PhotographoftheSchrdderresidence('RietYeldHouse')(fromBrown'1958)

hidden,innertruths,the same
serveto revealotherwise
But if reductionandabstraction
o
t h e rn o n - n a t u r a l i sat irct i s t s ,
s
o
m
a
n
y
l
i
k
e
p
a
i
n
t
i
n
g
s
.
i
n
d
e
e
d
,
m i g h tb es a i da b o u tR y m a n ' s
p
a
i
n
t
i
n
g
s
'
r
e
a
l
i
s tpi ca i n t i n g st'h: e y a s p i r et o
h
i
s
h e s e e sh i s w o r k a s r e a l i s t i ca, n d c a l l s
and the processof
process
representation
of
presentthe reality and the truth of the
w
o
r
l
d
'
c
u
l
t
u
r
a
l
a
n
d
s
o
c
i
a
l
p e r c e p t i oann, dt h e r e b yp e r h a pas l s oo f t h e
muchlessabstractfeatureof Ryman'spainting:hisconcern
another,
Thereis,however,
with the materialsof representawith texture.His work showsa constantpreoccupation
Someof the paintings
processes
of
representation.
tion, and with the materialityof the
rest.Othersdisplaya
cover
the
thinly
only
patch
and
of the canvasuncovered
leavea
way in which the
the
de-emphasize
or, by contrast,completely
varietyof brushstrokes
s
u
r
f
a
c e sA. g a i n o t h e r s
p
r
e
s
u
m
a
b
l
s
y
p
r
a
y
e
d
f
f
a
t
,
p a i n t i s a p p l i e d r, e s u l t i n gi n t o t a l l y
or the flatness
walls,
to
are
attached
frames
which
lheframe,or the meansby
emphasize
its twoforegrounding
so
and
degrees,
ninety
of the painting,by rotating it through
dimensionality.
but it is a
concernin thesepaintings,
In otherwords,thereis a strongrepresentational
modality,
low
suggest
thls
Does
process
representation.
of
the
concernwith representing
the
highest
it
suggest
does
0r
naturalism?
giventhe enormousdistancefrom everyday
which
the
or
in
truth,
an
ultimate
forms
modality,in whichthe negationof representation
is?
simply
but
represent
not
does
which
to the representation
highestmodalityis accorded

Modality

As before,our answeris one which refersto the social.Whethera representation

is
judqedcredibleor not is not necessarily
amatterof absolutetruth.What onesocialgroup
considers
crediblemay not be considered
credibleby another.
Thisis why we seemodality
as interactive/
ratherthan ideational,as social,ratherthan as a matter of some indep e n d e n t lgyi v e nv a l u eM
. o d a l i t yb o t hr e a l i z easn dp r o d u c esso c i aal f f i n i t y , t h r o u gahl i g n i n g
the viewer(or reader,
or listener)with certainformsof representation,
namely
thosewith
w h i c ht h e a r t i s t ( o r s p e a k e o
r ,r w r i t e r ) a l i g n sh i m s e l fo r h e r s e l fa, n d n o t w i t h o t h e r s .
M o d a l i t yr e a l i z ews h a t ' w e 'c o n s i d et r u eo r u n t r u er,e a lo r n o t r e a l .I n t h i sl i e ss o m eo f t h e
powerof art. To the extentthat peopleare drawninto this,we,,new
values,new modes
o f t h i n k i n ga n dp e r c e i v i ncga ne s t a b l i sthh e m s e l v eAsn. dw h e ne n o u g hp e o p l e
a r ed r a w ni n ,
t h e o r g a n so f p o p u l a r f i z i n gc)u l t u r gs u c ha s a d v e r t i s i n w
g ,i l l q u i c k l ym o v ei n t o a m p l i f y
the newforms,and movethem intothe mainstream
of culture.

MODALITYCONFIGURATIONS
T h ee x a m p l eisn t h e p r e v i o usse c t i o n
s h o w t h a t t h em o d a l i t yv a l u e si n a r l c a n0 ec o m p t e x .
A paintingcan reducenaturalismin the way it treatscolour,amplifyit in the way it treats
texture,andyet represent
its subjectin a naturalisticway,as in plate4. It can be abstract
in respectof onemodalitymarker,
naturalisticin respectof anotherandsensoryIn respect
of yet another,
andthis allowsa multiplicityof possible
modalityconfigurations,
andhence
a m u l t i p l i c i t yo f w a y si n w h i c ha r t i s t sc a n r e l a t et o t h e r e a l i t yt h e y a r e d e p i c t i n ga n d
' d e f i n er' e a l i t y g e n e r a l .
in
I n m a n yo t h e rk i n d so f i m a g e st o
, o , ' m o d a l i t ym a r k e r sd, o n o t
moveen bloc in a particulardirectionacrossthe scales,say from the abstractto
the
sensory/but behavein relativelyindependent
ways.Most glossymagazinefood photog r a p h sf,o r i n s t a n c ea,r e h i g h l ys e n s o r yi n t h e i r d e p i c t i o no f
t h e f o o d .T h e c o l o u r sa r e
intense.The textureof the food is shownin sharp detail. Lightingenhances
the fresh
d r o p l e t os f w a t e ro n a b u n c ho f g r a p e so,r t h ev i s c o s i toyf a s a u c eo,r t h eg l a z i n go f t h eh a m
and the cherriesin a pie. But the surrounding
objectstend to havelowermodality.The
weaveof the tableclothon whichthe food is displayed,
for instance,
may be onlyjust be
visibleand often the settingis absentaltogether,
with the food shownagainsta black
background.
In otherwords,suchpicturesare not onlysensory,
theyare alsoabstract.The
'sensorily'depicted
food is takenout of its context,idealizedand essentialized.
And this
showsthat eachof the modalitychoicesin sucha modalityconfigurationis
expressive
of
s p e c i f im
c e a n i n gw
s ,h i c ht h e nc o m et o g e t h eirn t h ew h o l e .
F r o m o u r i n v e n t o r yo f m o d a l i t ym a r l < e r w
s e c o u l d c o n s t r u c t ' m o d a l i t yp r i n t s ,
( b o r r o w i ntgh e m e t a p h oor f ' v o i c ep r i n t , , ' D N Ap r i n t , ,
e t c . )t o c h a r a c t e r i zt h
ee m o d a l i t y
configurations,
and showwhichmodalitymarkersare reduced,
made'lessthan real,,and
w h i c ha r e a m p l i f i e dm, a d e ' m o r et h a n r e a l ' - a n dt h i s e i t h e r i nr e l a t i o nt o a n a n c h o r i n g
p o i n to f c o m m o ns e n s e
h i g hn a t u r a l i s t im
c o d a l i t y( a so n em i g h td o f o r a n a u d i e n coef ' l a y ,
p e o p l ea t a n a r t e x h i b i t i o n , t a l <ti h
ne
gr e p r e s e n t a t i o n
f uanl c t i o no f a r t a s a c o m m o ns e n s e
point of departure)or in relationto an anchoringpoint situated[n some
other realism.
Figure5.10 is an attemptto showwhat we havein mind.

t72

Modality

Colourmodulation
Colourdifferentiation
Background
representation
Detailrepresentation
Tonality
etc.

N A T UR A L I S M

(abstraction)
.r+

(exaggeration)

fiq S.fO Modalitycontiguration

Such modalityconfigurations
would describewhat, in a specificgenreor a specific
work/is regarded
as real/asadequate
to reality.And it wouldalsodemonstrate
that images
are polyphonic/
weavingtogetherchoices
from differentsignifying
systems,
differentrepresentational
modes/into onetexture.In this view,a term suchas'painting'is an artificial
constructwhich bringstogetherand treats as a homogeneous
unit what is in realitya
complexconfiguration
of differentvoices,differentrepresentational
modes.(ln the same
way it can be said that 'grammar' is an artifice of theory,describingwidelydifferent
representational
modes- phonicsubstance,
intonation,lexis,syntax,etc.) And it is of
questions
coursefrom herethat the interesting
can be asked.Are there,or couldtherebe,
socialand historicalexplanations
for thesemodalityconfigurations?
Here is an example,from a sciencetextbookfor the upperyearsof primaryschool,
p r o d u c e di n A u s t r a l i a( f i g u r e5 . 1 1 ) . T h i s i s a s c i e n t i f i c - t e c h n i pc iacl t u r ef o r c h i l d r e n .
As such it forms a compromisebetweenthe naturalisticand the technological
coding
s e c a u sae' p u r e ' t e c h n o l o g i cpailc t u r ew o u l dh a v eb e e nr e g a r d e b
o r i e n t a t i o np,e r h a p b
dy
t h e w r i t e ra s b e y o n tdh e u n d e r s t a n d i o
n fgy o u n gc h i l d r e n0.n t h e o n eh a n d i, t i s a d r a w i n g
and not a photographand it lacks a Setting;on the other hand,it usesperspective
( a n g u l a r - i s o m e t rci co)l,o u r( i d e a l i z e fdf ,a tc o l o u r )a, n di t s h o w sa t l e a s st o m e t h i nogf l i g h t
and shade(thoughin a rathersimpleand,in part, inconsistent
way),and of texture(the
grainof the wood,the textureof the headof the nail,the creasesin the pieceof cloth).The
producerof this imageperhapsoperateswith the assumption
that childrenare familiar
with the naturalisticcodingorientation(that is,'wherethey comefrom') and haveto be
i n d u c t e idn t ot h et e c h n o l o g i ccaol d i n go r i e n t a t i o(nt h a ti s ,t h ep r o g r e s s i oi nnt od i s c i p l i n a r y
k n o w l e d g eT)h. ei m a g ec a p t u r etsh i st r a n s i t i o n aplh a s e .
Diagrams,maps and charts for lay readersmay be 'naturalized'in similar ways.
(seethe Gulf
Newspaper
diagramsand maps/for instance,
may be drawnin perspective
W a r m a p i n f i g u r e4 . 1 9 ) .l v l a g a z i n e
msa ya d dc o l o u ra n d p i c t o r i a l i zpei ec h a r t sI.n c o m panybrochures
or annualreports,the barsof bar graphsmay becomethree-dimensional
a n d r i s e ,l i k ef e a t u r e l e sssk y s c r a p e r s , f r oam
c l e a nl a n d s c a poef u n d u l a t i nhgi l l si n s t r o n g ,
f f a t c o l o u rT
. h i s s h o w st h a t m o d a l i t yi s a s y s t e mo f s o c i a ld e i x i sw h i c h ' a d d r e s s e s ' a
particularkind of viewer,or a particularsocial/cultural
group,and providesthroughits
systemof modalitymarkersan imageof the cultural,conceptual
andcognitivepositionof
the addressee.
At the sametime it showsthe transitionacrossand betweensuchgroups,

Modality

C)

173

(Jennings,1986)
fig s.ff Compass

the socialaspectof modality.Most crucially,it showshow

and in doingso demonstrates
in the closematchingof modalityandthe modaladdress(location)
modalityis motivated,
(and
aspectsof the viewer'ssubjectivity.
assumed)
of specific
F i g u r e5 . 1 2 s h o w sa d r a w i n gb y N e w t o ni,l l u s t r a t i ntgh e s e t - u pf o r o n eo f h i s c o l o u r
s h o w i n gt h e s e t - u pf o r a n
e x p e r i m e n tF
s .i g u r e5 . 1 3 i s a m o d e r ns c i e n t i f i ci l l u s t r a t i o n
out in spaceas indicated
stretched
by Strattonwhichcausedhimto seehimself
experiment
yet
very far in the
advanced
not
in the drawing.To moderneyes,Newton'sdrawinghas
(
i
n
v
e
r
t
e
d
)
p
e
r
s
p
e
c
t
i
v
aen, d s h o w st h e
ml o d a l i t yh: e u s e s
d i r e c t i o no f h i g ht e c h n o l o g i c a
the forms,
and
simplifies
setting
Setting.The moderndrawing,by contrast,leavesout the
of the
representation
on
the
on the relationbetweenthem, ratherthan
concentrating
e x p e r i m e n taenr dt h e m i r r o r s .
s r i t i n gd i d n o t
A s H a l l i d a yh a ss h o w n( H a l l i d a ya n d M a r t i n ,I 9 9 3 i 5 4 - 6 $ , N e w t o n 'w
scientific
modern
of
density
lexical
yet havethe objective,impersonalstanceand the
t
h
e gramd
e
v
e
l
o
p
i
n
g
t
o
w
a
r
d
s
e oves
w r i t i n g .A t t h e s a m et i m e h e m a d es o m ed e c i s i v m
same
the
writing.
Clearly
of
scientific
that wouldbecomecharacteristic
maticalresources
great
differences
the
however
that,
And that shows
can be saidof his scientificdrawings.
betweenthe verbal and the visual grammar,they derivefrom similar concernsand
orientations.

fig l.fZ Drawingby Newton(BodteianLibrary)

fiq S.ff Drawingof Stratton'sexperiment(Gregory,1970)

6 T h e m e a n i n go f c o m p o s i t i o n

A N D T H E M U L T I M O D A LT E X T
COMPOSITION
the relationsbetween
the way imagesrepresent
In previouschapterswe haveconsidered
t h e p e o p l ep, l a c e sa n dt h i n g st h e yd e p i c ta, n dt h e c o m p l e xs e t o f r e l a t i o ntsh a t c a ne x i s t
r f s u c hr e p r e s e n s n u m b eo
i m a g e sa n dt h e i rv i e w e r sA. n y g i v e ni m a g ec o n t a i n a
between
In figure6.I,an imagefrom Bergman'sThrougha Glass
relations.
tationalandinteractive
e ental
D a r k l y( 1 9 6 1 ) ,w e s e el ( a r i n( H a r r i e tA n d e r s s o nw) ,h o s u f f e r sf r o m a n i n c u r a b lm
Fromthe pointof viewof repreand heryoungerbrotherlVinus( Lars Passgard).
disease,
t h, e s h o tc o n t a i nw
s h a tw e h a v ec a l l e da ' n o n - t r a n s a c t i rveea c t i o n('l ( a r i nl o o k s
sentation
, t s o m e t h i ntgh e v i e w e cr a n n o st e e )a n da ' t r a n s a c t i vree a c t i o n('M i n u s
o u t o f t h ef r a m ea
looksup at hissister).Thesechoicesrelateto the themesof the dramaticaction:l(arinhas
v i s i o n ss,e e st h i n g so t h e rp e o p l ec a n n o ts e e ;M i n u si s c a u g h ti n t h e h e r ea n d n o w o f h i s
p r o b l e m a t irce l a t i o n w
s i t h t h e o t h e rc h a r a c t e risn t h e f i l m . F r o mt h e p o i n to f v i e wo f
i n t e r a c t i vm
e e a n i n gt ,h e v i e w e ri s p o s i t i o n ecdl o s e rt o l ( a r i n ( ' m e d i u ms h o t ' )t h a n t o
M i n u s( ' 1 o n g
s h o t ' ) ;a n d ,w h i l eM l n u si s s e e nf r o mb e h i n dl ,( a r i nf a c e st h ev i e w efrr o n t a l l y .

,w'wk,

fhroughaGtassDarkt!(Belgman,1960)
figO.f HarrietAnderssonandLarsPassgardin

176

The meaning of composition

Clearly,
the vieweris meantto be mostcentrallyinvolved
with l(arin,andwith her mental
lurmoil.
Thesepatternsdo not exhaustthe relationssetup by the image.Thereis a third element:
the compositionof the whole,the way in which the representational
and interactive
elements
are madeto relateto eachother,the way they are integratedinto a meaningful
wholeM
. i n u sf,o r i n s t a n c ei s, p l a c e do n t h e l e f t ,a n d l ( a r i no n t h e r i g h t .I f t h i sw e r et u r n e d
around,the representational
and interactivemeaningswould not be affected.l(arin,s
reactionwould still be 'non-transactive'
and Minus, reaction'transactive,,
and l(arin
w o u l ds t i l lb e i n m e d i u ms h o t ,M i n u ss t i l l i n l o n gs h o t .B u tt h e m e a n i n o
g f t h ew h o l ew o u l d
no longerbe the same.In otherwords,the placement
of the elements(of the participants
and of the syntagms
that connectthemto eachotherandto the viewer)endowsthemwith
specificinformationvaluesrelativeto eachother.We will discussthe valueof ,left, and
' r i g h t 'i n t h e n e x ts e c t i o n .
In addition,l(arinis the mostsalient,themosteye-catching
elementin the composition,
notjust because
sheis placedin the foreground
andbecause
sheformsthe largest,simplest
elementin the picture,but also because
sheis in sharperfocusand receives
the greatest
a m o u no
t f lightT
. h r o u g h o umt u c ho f t h ef i l m l ( a r i ni s d r e s s eidn l i g h tc o l o u r sa n dm a d et o
bathein light,in an almostsupernatural
fashion,this in contrastto the othercharacters.
For thesereasons
sheis alsothe mostsalientelementin the shotswhereoneof the other
characters,
for exampfeherhusband,
is placedin the foreground.
Herwhiteclothesandthe
lighton her palefacedrawattentionto her,evenwhensheis placedin the background.
To
g e n e r a l i zpei,c t o r i ael l e m e n tcsa n r e c e i v set r o n g eor r w e a k e r ' s t r e s s ' t h aont h e re l e m e n t s
i n t h e i ri m m e d i a tvei c i n i t ya,n ds o b e c o m e
m 0 r eo r l e s si m p o r t a n t ' i t e mosf i n f o r m a t i o ni n,
t h ew h o l e .
A verticallineformedby the left edgeof the door of the shed,and continuedby the
d i v i d i n gl i n eb e t w e ean p a r t i c u l a r llyi g h ta n da d a r k e rb o a r do n t h e r o o fo f t h e s h e dr, u n s
t h r o u g ht h e m i d d l eo f t h e p i c t u r ed, i v i d i n gi t i n t o t w o s e c t i o n sl i,t e r a l l ya n d f i g u r a t i v e l y
'drawinga line'between
the spaceof l(arin,who can'look intothe beyond,,
andthe space
o f M i n u sw
, h o c a n n o tT. h ew o r l do f l ( a r i ni s t h u ss e p a r a t efdr o m t h e w o r l do f M i n u s i,n
t h i s p i c t o r i acl o m p o s i t i oans i n t h e d r a m a t i ca c t i o no f t h e f i l m a s a w h o l ew
, h e r eM i n u s ,
d e s i r ef o r c o n t a c a
t n dc o m m u n i own i t h h i ss i s t e r e m a i n su n f u l f i l l e dT.h e r ei s y e t a n o t h e r
d e m a r c a t i olni n e i n t h e p i c t u r e t: h e h o r i z o nw, h i c hd i v i d e st h e p i c t u r ei n t o t h e z o n eo f
'heaven'andthe zone 'earth'.ln
of
his discussion
of ritian's Noli Me Tangere,Arnhejm
(1982: lr2-r3) describeshow the staff of christ forms a 'visual boundary,between
c h r i s t ,w h o i s a l r e a d y ' r e m o v ef rdo m e a r t h l ye x i s t e n c ea' n
, d M a g d a l e nw, h o i s n o t ;a n d
h o w ' t h el o w e rr e g i o ni s s e p a r a t ebdyt h e h o r i z o nf r o mt h e u p p e r e g i o no f f r e es p i r i t u a l i t y ,
i n w h i c ht h e t r e ea n dt h e b u i l d i n gosn t h e h i l l r e a c hh e a v e n w a r dI n' . f i g u r e6 . 1 ,s i m i l a r l y ,
l(arin straddlesthe two zones,half still of the earth,half alreadyin the realm of 'free
s p i r i t u a l i t yw' ,h i l eM i n u si s ' h e l dd o w nb y t h e h o r i z o ni n t ot h e r e g i o no f t h e e a r t h , .M o r e
generally,
composition
also involvesframing(or its absence),
throughdeviceswhichconn e c to r d i s c o n n eec lte m e n tosf t h ec o m p o s i t i osno, p r o p o s i ntgh a t w e s e et h e ma sj o i n e do r
as separatein someway,where,without framing,we wouldseethem as continuous
and
c o m p l e m e n t a rt h
y :e r ew o u l db e n o v i s u a'ld i r e c t i v eo,f t h i sk i n d .

The meaning of composition

L77

meanings
of the image
and interactive
then,relatesthe representational
Composition,
systems:
lo eachotherthroughthreeinterrelated
(participants
( 1 ) Inforntationvalue.The
that relate
placement
andsyntagms
of elements
themto eachotherand to the viewer)endowsthem with the specificinformational
valuesattachedto the various'zones'of the image:left and right,top and bottom,
c e n t r ea n dm a r g i n .
( 2 ) Salience.The elements(participantsas well as representational
and interactive
as realized
syntagms)
are madeto attractthe viewer'sattentionto differentdegrees,
relativesize,contrasts
in the foreground
or background,
by suchfactorsas placement
i n t o n a lv a l u e( o r c o l o u r )d, i f f e r e n c ei ns s h a r p n e sest,c .
3 ) Framing.The presenceor absenceof framingdevices(realizedby elementswhich
of the
elements
or connects
createdividinglines,or by actualframelines)disconnects
i m a g es, i g n i f y i ntgh a t t h e yb e l o n go r d o n o t b e l o n gt o g e t h eirn s o m es e n s e .
ee
These
t h r e ep r i n c i p l eosf c o m p o s i t i oanp p l yn o tj u s tt o s i n g l ep i c t u r e sa,s i n t h ee x a m p lw
t e x ta n d
h a v ej u s t d i s c u s s etdh;e ya p p l ya l s ot o c o m p o s i tvei s u a l sv,i s u a l sw h i c hc o m b i n e
s ,e i t o n a p a g eo r o n a t e l e v i s i oonr c o m p u t e r
i m a g ea n d ,p e r h a p so,t h e rg r a p h i ce l e m e n t b
(andanytext whosemeanings
are
or multimodaltexts
screen.
In the analysisof composite
the questionariseswhether
realizedthroughmorethan onesemioticcodeis multimodal),
way;
or in an integrated
separately
the productsof the variousmodesshouldbe analysed
w h e t h etrh e m e a n i n gosf t h e w h o l es h o u l db e t r e a t e da s t h e s u mo f t h e m e a n i n gosf t h e
parts, or whetherthe parts shouldbe lookedupon as interactingwith and affecting
for example,
In considering,
It is the latterpathwe will pursuein this chapter.
oneanother.
'illustration'of
t h e p i c t u r eo f t h et r a i n( f i g u r e3 . 3 0 )w e d o n o s e e kt o s e et h e p i c t u r ea sa n
the verbaltext, therebytreatingthe verbaltext as prior and more important,nor treat
We seekto be ableto lookat the whole
visualandverbaltext as entlrelydiscreteetements.
and
betweenlanguage
pageas an integrated
text.0ur insistence
on drawingcomparisons
v i s u a cl o m m u n i c a t i osnt e m sf r o m t h i s o b j e c t i v eW. e s e e kt o b r e a kd o w nt h e d i s c i p l i n a r y
andwe seek/as much
andthe studyof images,
boundaries
between
the studyof language
en
, dc o m p a t i b ltee r m i n o l o gt yo s p e a la<b o u tb o t h ,
a s p o s s i b l teo, u s ec o m p a t i b llea n g u a g a
for in actual communication
the two, and indeedmany others,cometogetherto form
integratedtexts.
In our viewthe integrationof differentsemioticmodesis the work of an overarching
c o d ew h o s er u l e sa n d m e a n i n gps r o v i d et h e m u l t i m o d atle x t w i t h t h e l o g i co f i t s i n t e g r a t i o nT. h e r ea r et w o s u c hi n t e g r a t i ocno d e st :h e m o d eo f s p a t i acl o m p o s i t i ow
n i,t h w h i c h
nh
.e
w e w i l l b e c o n c e r n eidn t h i sc h a p t e ra; n d r h y t h m , t h em o d eo f t e m p o r acl o m p o s i t i oT
- for example,
former operatesin texts ln whlch all elementsare spatiallyco-present
pages.Thelatteroperates
in textswhichunfoldovertime
paintings,
magazine
streetscapes,
- f o r e x a m p l es,p e e c hm, u s i cd, a n c e( s e ev a n L e e u w e n1,9 9 9 ) .S o m et y p e so f m u l t i m o d a l
t e x t u t i l i z eb o t h ,f o r e x a m p l ef i l m a n d t e l e v i s i o na,l t h o u g hr h y t h mw i l l u s u a l l yb e t h e
i n t h e s ec a s e s .
d o m i n a nitn t e g r a t i vper i n c i p l e
I t f o l l o w st h a t t h e p r i n c i p l eosf l n f o r m a t i ovna l u es, a l i e n caen df r a m i n ga p p l yn, o t o n l y

L78

Themeaning of composition

to pictures,but also,for example,

to layouts.Plate2, an advertisement
for Bushells
instant
coffee,containstwo photographs
and a smallamountof verbaltext.The largerphotois a
pictorialrepresentation
of the 'prornise'ofthe product,and it is placedin the top section.
Thephotoof the productis smaller,
and placedbelowthe largerphotograph,
togetherwith
the text. Reversing
this wouldproducean entirelydifferenteffect,and probablyresultin a
ratheranomalouslayout.Just what informationvaluesthis arrangement
accordsto the
two sectionsof the pagewill be discussed
below.As far as salienceis concerned,
we can
notethat this pageis not dividedintotwo equalhalves.
Thetop sectionis the mostsalient,
n o t o n l yb e c a u soef i t s s i z eb u t a l s ob e c a u steh e s a l i e n coef t h e w o m a nw
, hoispositioned
o n t h e r i g h ta n dc a t c h e m
s o s to f t h eg o l d e ng l o wo f t h e l i g h t T
. h u st h e a d v e r t i s e m egnitv e s
greaterstressto the promiseof the productthan to the productitself,or the verbal
information.
Finally,a sharplinecreatesa boundarybetween
the photoandthe verbaltext,
dividingthe pageintotwo separate
sections,two
spaces/
reserved
for two differentkindsof
- onefor the promiseof the product,enhanced
meaning
intimacybetweenlovers;the other
for the productitself.Just as thereis a dividinglinebetweenheavenand earth in Titian's
Noli Me Tangere,
and in the stillfrom Bergman'sThrougha GlassDarkly,so thereis, in
this advertisement
too,a dividinglinebetween
the world of 'what mightbe',the happiness
'what
the productmight bring,and the world of
is', the productitself - and, just as
in the two earlierexamples,
this product,the jar of instantcoffee,straddlesthe two
domainsof meaning,
forminga bridgebetweenthem.The homepageof Sony'swebsite
(http://www.sony.com)
has a similarstructure.The top part showsthe pleasurederived
f r o mu s i n gt h ec o m p a n y 'psr o d u c t sa,n dw e l c o m et sh e u s e rt o t h e' w o r l do f S o n y 'w
, h i l et h e
bottompart showsa rangeof actualproductsand allowsthe userto click on the pages
from wherethe productscan be ordered.
E a r l yp r i n t e dp a g e ss t i l l t r e a t e dt e x t a s ' v i s u a lm a t e r i a l 'W
. a l t e r0 n 9 ( 1 9 8 2 : 1 1 9 f f . )
describes
how sixteenth-century
title pagesbroke up wordswithout regardfor syllable
boundaries,
and useddifferenttypesizesin a way that was not relatedto the relative
importanceof words,but servedto createpleasingvisualpatterns.However,
the printed
p a g es o o nd e v e l o p ei d
n t ot h e ' d e n s e lpyr i n t e dp a g e 'i n w h i c hr e a d i n gi s l i n e a ra n dt e x t u a l
i n t e g r a t i oanc h i e v ebdy l i n g u i s t im
c e a n s( c o n j u n c t i o ncso,h e s i vt ei e s ,e t c . ) .I n b o o k so f t h i s
kind it seemsthat the pagehasceasedto be a significant
textualunit.The pageshownin
p l a t e2 , o n t h eo t h e rh a n d i,s a s e m i o t i u
c n i t ,s t r u c t u r e dn,o t l i n g u i s t i c a lbl yu,t b y p r i n c i p l e s
o f v i s u a lc o m p o s i t i o nI n. s u c ha p a g ev e r b a lt e x t b e c o m e jsu s t o n e o f t h e e l e m e n t s
i n t e g r a t ebdy i n f o r m a t i ovna l u es, a l i e n caen df r a m i n ga, n dr e a d i n gi s n o t n e c e s s a r il li yn e a 4
w h o l l yo r i n p a r t ,b u t m a yg o f r o mc e n t r et o m a r g i no, r i n c i r c u l a fra s h i o no,r v e r t i c a l l y , e t c .
And this is the case,not only in contemporary
magazines
and websites,
but also in many
- for instance,
othercontexts
in modernschooltextbooks,
as we will showlater(e.g.figure
6.6).
It shouldbe noted,of course,that the layoutof the denselyprintedpageis still visual,
still carriesan overallculturalsignificancg
as an imageof progress.
The denselywritten
pagesof otherculturaltraditionsare laid out differently- as,for examplein the Talmud,
whichhasthe oldesttext,the M ishna,in the centre,the Gemarawrittenaroundit; and later,
m e d i e v acl o m m e n t a r i easg a i n ,a r o u n dt h e G e m a r ai,n c o n c e n t r i lca y e r sI.n s u c hc a s e s ,

The meaning of com?osition

'

u9

pagesandthe pages
however,
everypageis still readthe sameway.In the caseof magazine
pagemay havea differentreadingpath.
eachsuccessive
of moderncomputerscreens,
printedpagebeganin the late nineteenth-century
Thisdevelopment
beyondthe densely
masspress,in a contextin whichthe rulingclass,itselfstronglycommittedto the densely
p r i n t e dp a g e a, t t e m p t e tdo m a i n t a i ni t s h e g e m o nbyy t a k i n gc o n t r o lo f p o p u l a rc u l t u r e ,
it, andsoturningthe mediao/the peopleintothe mediaforthepeople(see
commercializing
e n dt h e h u m a n i t i e s
e e d l a- ' h i g h ' l i t e r a t u r a
W i l l i a m s1 9 7 7 : 2 9 5 ) .T h e i ro w nc o m p a r a b lm
t as
c f w r i t i n g .L a y o uw
g e n e r a l l-y b e c a m e v e nm o r ef i r m l yf o u n d e d
o n t h es i n g l es e m i o t i o
the powerof the denselyprintedpageas/
not encouraged
here,becauseit undermined
literally,the realizationof the most literaryand literatesemioticmode.Thegenresof the
l r m s )c o n t r o l l e d
d e n s e lpyr i n t e dp a g et,h e n ,m a n i f e stth e c u l t u r a cl a p i t a l( ' h i g h ' c u l t u r af o
b y t h e i n t e l l e c t u aaln da r t i s t i cw i n go f t h e m i d d l ec l a s st,o u s eB o u r d i e u t' se r m s( 1 9 8 6 ) .
l r o u pw h i c hh a sb e e ni n s t r u m e n t ai nl s p r e a d i ntgh e n e wv i s u a l
Y e t i t i s t h i s s a m es o c i a g
literacyto thosewho werenot,or not yet,to be initiatedinto the formsof literacywhich
c o n s t i t u t e idt s o w n m a r k o f d i s t i n c t i o n( t h e ' m a s s e so' ,r c h i l d r e n )a, n d t o e m b r a c ei t ,
of their opposas an expresslon
manifestations/
for examplein 'high'cultureavant-garde
itionalrole withinthe middleclassas a whole.As so often in the twentiethcentury,they
turnedout, in the end,to havebeensawingoff the branchon whichtheyweresitting.The
b e t w e e n ' h i g h ' a n d ' l o w ' f o r m si s n o w e v e r y w h e rien c r i s i s ,a n d n e w w a y s
distinction
, r i n s t a n cteh e d e v e l o p m eonft d i f f e r e n t
o f m a i n t a i n i ncgu l t u r a h
l e g e m o nayr e r e q u i r e df o
are no longer
and differentlyvaluedways of talking about forms which,themselves,
s 'u) t.t h em o s th i g h l y
d i f f e r e n t i a t ei ndt h e o l dw a y ( t h e ' d i s c o u r s e s 'doif f e r e n t ' a u d i e n c eB
v a l u e dw a y so f t a l k i n g( a n ds e m i o t i ci s o n eo f t h e m )r e m a i nt h e m s e l v eb so u n dt o m e t h o d s
the way in which
describethe newforms.If we are to understand
that cannotadequately
institutionslike the media,educationand children'sliteraturemake
vital text-producing
of newformsof socialstratification,
senseof the worldand participatein the development
s hich
a t h e o r yo f l a n g u a gies n o l o n g e sr u f f i c i e natn d m u s tb e c o m p l e m e n t eb dy t h e o r i e w
for instance,
can make the principlesof the new visual literacyexplicit,and describe,
that takesplaceon the pagesof the texts
of socialsemiosis
the roleof layoutin the process
- as we will try to do in this chapter.
produced
by theseinstitutions

G I V E NA N D N E W :T H E I N F O R M A T I O NV A L U E O F L E F T A N D R I G H T
we usedas oneof
spreadsin the Australianwomen'smagazines
Manyof the double-page
our datasetswhenwe wrotethe first versionof this chapterusethe layoutshownin figure
6 . 2 . T h e i r i g h tp a g e sa r ed o m i n a t ebdy l a r g ea n ds a l i e npt h o t o g r a p hf rso mw h i c ht h eg a z e
of one or more womenengagesthe gazeof the viewer(what, in chapter4, we called
'demand'pictures).Thesepagesshowwomen in specificand sometimescontradictory
a
are invitedto form a positiveidentification:
roles,with whichthe readersof the magazine
m o t h e ra; f o r m e r' s o a p i es t a r ' t u r n e dh o u s e w i faen d h a p p yi n t h a t r o l e ;w o r k i n gw o m e n
c a p a b l eo f c o p i n gw i t h ' t o u g h ' , ' m a s c u l i njeo'b s .T h e i r l e f t p a g e sc o n t a i nm o s t l yv e r b a l
on the right.Thespreadshownin figure6.2 hasa
salientphotographs
text,with graphically

The meaning of composition

{i.

l:::l

fD

fiS e.Z Gold-diggers(Iustra lian Women'sWeekry,Noyember1987)

photograph
on the left also,but this photois smallerand,in contrastto the photoon the
r i g h tp a g ei,t i s a ' f f y o n t h ew a l l ' p h o t o g r a pw
h ,h i c hd o e sn o ta c k n o w l e d gt hee p r e s e n coef
the photographer,
nor thereforethat of the viewer.It is what, in chapter4, we calledan
'offer' picture.0n suchpagesthere often is a senseof complementarity
or continuous
movement
from left to right,as in figure6.2,wherethe photograph
on the left is tiltedto
form a vectorthat leadsthe eyesto the photograph
on the right,andwherethe colourgold,
with its obviousconnections
to thethemeof the story,is usedas anotherintegrating
device:
it occursin the photograph
as the colourof the helmetsand of the liquidbeingpoured,and
is usedalsoas the background
againstwhichthe verbaltext is printed.
0n suchpagesthe rightseemsto be the sideof the keyinformation,
of what the reader
mustpay particularattentionto, of the 'message'-whetherit is the invitationto identify
w i t h a r o l e m o d e lh i g h l yv a l u e di n t h e c u l t u r eo f t h e m a g a z i n e
o r s o m e t h i negl s e ;f o r
example,
an instanceof what is to be learnedin a textbook.It followsthat the left is the
sideof the'alreadygiven',something
the readeris assumed
to knowalready,
as part of the
culture,or at leastas part of the cultureof the magazine.
In figure6.2, goldminingis
Given,and the fact that womencan engagein it, and that you,the reader,
shouldidentify
w i t h s u c h' t o u g h ' w o m e ni s/ N e wt,h e m e s s a gteh,e ' i s s u e ' .
Lookingat what is placedon the left and what is placedon the right in other kinds
of visualshasconfirmedthis generalization:
whenpicturesor layoutsmakesignificantuse

The meaning of composition

181

left,andother,differentonesright
someof theirelements
of the horizontalaxis,positioning
o f t h ec e n t r e( w h i c hd o e sn o t ,o f c o u r s eh,a p p e ni n e v e r yc o m p o s i t i o nt h) /ee l e m e n tpsl a c e d
placedon the rightas New.For something
as Given,the elements
on the left are presented
the vieweralreadyknows,as a familiar
to be Givenmeansthat it is presented
as something
pointof departurefor the message.
to be Newmeansthat
For something
andagreed-upon
it is presented
as something
which is not yet known,or perhapsnot yet agreeduponby
to which the viewermust pay specialattention.Broadly
the viewer,henceas something
information
the meaningof the Newistherefore'problematic','contestable','the
speaking,
',
is
" a t i s s u e " w h i l et h e G i v e ni s p r e s e n t eadsc o m m o n s e n s i csaell,f - e v i d e nTth. i ss t r u c t u r e
to what is the caseeither for the
ideologicalin the sensethat it may not correspond
produceror for the consumerof the imageor layout.The importantpoint is that the
as thoughit had that statusor valuefor the reader,and that
informationis presented
evenif that valuationmaythenbe rejectedby
readershaveto readit withinthat structure,
a oarticularreader.
A s i m i l a rs t r u c t u r e x i s t si n s p o k e nE n g l i s h( s e eH a l l i d a yI ,9 B 5 : 2 7 4 f f . ) .A s i n v i s u a l
p lh r a s ei s, n o ta c o n s t i t u e n t
o f a ' t o n eg r o u p 'a, n i n t o n a t i o n a
c o m m u n i c a t i ot h
ne
, structure
from
elements,
but a gradual,wave-likemovement
with strongframingbetween
structure,
left to right (or, rather,from 'before'to 'after', sincein languagewe are dealingwith
Intonationcreatestwo peaks
temporallyintegrated
texts),and it is realizedby intonation.
e i t h i ne a c ht t o n eg r o u p ' - o n ea t t h e b e g i n n i nogf t h e g r o u pa, n d a n o t h e trh, e
o f s a l i e n cw
t hienN
majorone(the'tonic',inHalliday'sterminology),astheculm
ae
t iw
o n, aotft h e e n d .
J u s ta s i n f i g u r e6 . 2 w e h a v eo n ep e a ko f s a l i e n coen t h e l e f t ,i n t h e b o l dh e a d l i naen dt h e
it from the articleitself,and anotheron the right,in the photoof
red bar whichseparates
on the syllablegold andanotheron
the two women/so we wouldhaveonepeakof salience
the syllablewo- of womenin:
c o l D - d i g g i ncga nn o wb e d o n eb y w o m e n
And just as the imageof the two womenis the New in figure 6.2, so the word women
in the clauseabove.In otherwords,thereis
wouldbethe New,the keypointof the message.
a closesimilaritybetweensequentialinformationstructurein languageand horizontal
moreabstract
of deeper,
and this atteststo the existence
structurein visualcomposition,
differentlyin differentsemioticmodes.
which find thelr expression
codingorientatlons
y h e r et h e h o r i z o n t adl i m e n s i o n
S u c hc o d i n go r i e n t a t i o nasr e c u l t u r a l l ys p e c i f i cc,e r t a i n l w
i s c o n c e r n e Idn. c u l t u r e sw h i c hw r i t ef r o m r i g h tt o l e f t ,t h e G i v e ni s o n t h e r i g h ta n dt h e
h n dt h eA r a b i cl a n g u a gvee r s i o n s
N e wo nt h e l e f t ,a ss h o w ni n f i g u r e6 . 3 , w h e r teh e E n g l i s a
a r ec o m p a r e d .
o f S o n y ' sM i d d l eE a s tw e b s i t e
So far we havetaken a compositetext as our example,but the Given-Newrelation
reliefdepictingthe
appliesalsowithin an image.Figure6.4 showsa fourteenth-century
creationof Eve.God is the Given,agreedoriginand departurepoint of all that exists.
'Woman',on the otherhand,is Newand,in the contextof the Genesis
story,problematic,
t h e t e m p t r e sw
s h o l e a d sA d a m i n t o s i n .M i c h e l a n g e loon, t h e o t h e rh a n d ,i n h i s f a m o u s
paintingThe Creationof Adam on the ceilingof the SistineChapel,placedGod on the

The meaning of composition

riq e.r Englishand Arabiclanguageyersionsot sony'swebsite(http://www.sony-middleeast.com)

Themeaning of composition

183

fig e.a TheCreationof Eve(LorenzoMaitani,lourteenth

century)(from Hughes,1969)

r i g h t ,i n l < e e p i nwgi t h t h e n e r yh u m a n i s t iscp i r i t o f t h e R e n a i s s a n ci ne .t h i s p e r i o dG o d

suddenlybecameNew,and problematic.
Generations
of philosophers
wereto attemptto
redefineHim in wayscommensurate
with the newscience,
andto try to proveHis existence
b y t h e u s eo f l o g i cI.n t h i sp i c t u r et h e m o v e m e ni st n o l o n g e frr o m G o dt o ' M a n ' , b u t f r o m
' M a n ' t o G o d . ' M a n ' r e a c h eosu t ,a s p i r i n g
t o d i v i n es t a t u sa, n da l m o s ta c h i e v i nigt - b u t
n o tq u i t e .
I n m a g a z i nlea y o u t s u c ha st h e o n es h o w ni n f i g u r e6 . 2 ,t h es p a c eo f t h e G i v e ni s f i l l e d
by verbaltext, and the spaceof the New,or at leasta largepart of it, by one or more
i m a g e sB. u t t h i s i s n o t a l w a y st h e c a s eA
. d o u b l e - p a gaed v e r t i s e m efnotr M e r c e d e s - B e n z
showed,on the Ieft, a Mercedesphotographed
objectively(rather than, for example,
f r o m t h e d r i v e r ' sp o i n to f v i e w ) a
, n dw i t h t h e w e l l - k n o w M
n e r c e d eesm b l e mi n t h e c e n t r e
o f t h e c o m p o s i t i o nT.h e r i g h t p a g ec o n t a i n e d
saying,
o n l y v e r b a lt e x t ,w i t h a h e a d l i n e
'Mercedes-Benz
agreeswith its competitors.
Youshoulddrivetheir carsbeforeyoudrivea
Mercedes-Benz.'
In otherwords,the advertisement
as an alreadytreatedthe Mercedes
k n o w n , ' G i v e n ' s y m boof ls t a t u sa, n dt h e m e s s a gteh a t ' y o u ,t o o ,m i g h to w na l v l e r c e d e s '
as the New.More generally,
if the left containsa pictureand the right is verbaltext,the
pictureis presented
pointof departurefor the text,andthe
as Given,as a well-established
text containsthe New.If the left pagehas text and the right pagea picture,the text

The meaning of composition

containsthe Given,and the picturethe New.Theexamplepointsto the socialeffectsand

usesof this structure.what is taken for grantedby one socialgroup is not taken for
grantedby another.We mightexpectto find,therefore,
systematic
differences
in the dis- for instance,
positions
of materialin layoutacrossdifferentmagazines
according
to their
readershi
o.
The conceptsof Givenand New can be appliedalso to the designof diagrams.In
Shannonand Weaver's(1949) communication
model(figure2.2) it mightseemthat the
horizontalorder of the elementsis motivatedrepresentationally:
the processof'sending
information',
for instance,
musttake placebeforethe informationcanbe received.
But the
left doesnot alwayssignify'before',nor doesthe arrowof time alwayspointto the right.A
diagramfrom a 1990 issueof Time Magazinewhichwe were not allowedto reproduce
hereshowed,
on the right,a stickfigurewhoseverylargeheadwasa piechart representing
the composition
of the workforcein the year2000 (i.e.ten yearsintothe futureat thetime
of publication).
Anotherpie chart, on the left, was superimposed
on a massiveoffice
buildingand represented
the presentcomposition
of the work force.An arrowshowedthat
the stick figurewas walkingtowardsthe door of the massiveofficebuilding,i.e.that a
changein the composition
of the work forcewas graduallycomingcloserto the present,
but it was not movingtowardsthe right, becausethe currentcompositionof the work
force had to be treatedas Givenand the future additions(morewomen,minoritiesand
i m m i g r a n t sa)s N e wa n d p r o b l e m a t i cT.h i ss h o w sh o wt h e G i v e n - N e w
s t r u c t u r ec a n b e
i d e o l o g i c aelv e ni n d i a g r a m sI .f t h e h o r i z o n t aol r d e ro f t h e c o m m u n i c a t i omno d e lw e r e
r e a r r a n g ei d
na similaw
r a y( s e ef i g u r e6 . 5 ) ,i t w o u l dn o l o n g e dr e p i c ct o m m u n i c a t i o
f rno m
t h e p o i n to f v i e wo f t h e ' s e n d e r 'w, i t h t h e ' r e c e i v e r ' a sN e w ,a n d p r o b l e m a t i(cW i l l t h e
message'hitthe target'? Wifl it havethe intendedeffect?).Instead,the readerwould
b e c o m teh eo r i g i na n dd e p a r t u r oe f t h ec o m m u n i c a t i op nr o c e s a
s /n dt h e ' s e n d e r( ' a u t h o r ' )
problematic,
as hasindeedhappened,
for instance,
in literaryreception
theory.
Given-Newstructurescan also be found in film and television.
Mediainterviews,
for
(from the viewer's
examplgoften placethe intervieweron the left of the interviewee
pointof view).Thusinterviewers
are presented
as peoplewith whoseviewsand assumptionsviewerswill identifyandare alreadyf amiliar,indeed,
asthe peoplewho askquestions
on behalfof the viewers.
The interviewees,
on the otherhand,present'New,information
and are situatedon the right (see Bell and van Leeuwen,r994i 160-4). The relation
('pans').
betweenGivenand New may be emphasized
by horizontalcameramovements
In a currentaffairs item from an ABC 7.30 Report(March I9B7), the childrenfrom a
M u s l i mC o m m u n i tSy c h o ow
l e r ei n i t i a l l ys h o w na s ' e t h n i c ' , ' d i f f e r e n t ' f r o' ums , ,v i e w e r-s

fig e.S Reversed

communication
model

Themeaning of comPosition

r85

dress,and there was Arabic music in

there was much emphaslson their non-Western
that theywere,despite
to establish
But it wasthe pointof the programme
the background.
c h i l d r e np
' , l a y f u ls, p o n t a n e o ucsr /e a t i v ee/t c .T h i sw a s
t h i s , ' j u s tl i k eo r d i n a r yA u s t r a l i a n
a ShotWhich
realized,among other things,by varioushorizontalcamerarnovements:
a youngwomanin a Western
clothesto the teacher,
pannedfrom childrenin non-Western
wall
d r e s s , t y i nag b o w i n t h e h a i r o f a l i t t l eg i r l ;a s h o t w h i c hp a n n e da l o n ga c l a s s r o o m
f r o m a n A r a b i cs i g n t o a p i c t u r eo f a c l o w n ,e t c . I n o t h e rw o r d s , ' d i f f e r e n c ee't,h n i c
as
prejudice,
wastreatedas Given;the fact that at leastthesechildrenshouldbe accepted
'like us'was treatedas NeWand formedthe message
the programmewas trying to get
acroSs.
In ongoingtexts,eachNewcan,in turn, becomeGivenfor the next New.The opening
oe
l ography
p a g e so f t h e c h a p t e r ' l ns e a r c ho f a s t r a w ' f r o mt h e D u t c hj u n i o rh i g h - s c h o g
of
column
(Bols
a
single
left,
far
on
the
have,
al.,i1986)
ef
de
Wereld
Werk
aan
textbook
f
a
v
o
u
r
i
n
ga
p
a
g
e
,
a
'
l
a
n
d
s
c
a
p
e
'
f
o
r
m
a
t
,
h
a
s
w
h
i
c
h
t
h
e
f
i
f
t
h
o
f
a
b
o
u
t
a
o
c
c
u
p
y
i
n
g
text,
p
e
o
p
l
e
T
h
i
rd
i
n
t
h
e
a
s
'
M
a
n
y
s
u
c
h
a
s
s
e
r
t
i
o
n
s
t
e
x
t
c
o
n
t
a
i
n
s
T
h
e
l
a
y
o
u
t
.
o
r
i
e
n
t
e
d
horizontally
p
e
o
p
l
e
h
a
v
e
?'
t
h
e
s
e
p
r
o
s
p
e
c
t
s
a
l
l
d
o
q
u
e
s
t
i
o
n
s
a
s
'
W
h
a
t
s
u
c
h
n
o
t
h
i
n
g
'
a
n
d
W o r l dh a v e
p
a
g
e
h
a
s
l
e
f
t
o
f
t
h
e
p
r
o
b
l
e
m
.
r
e
m
a
i
n
d
e
r
T
h
e
p
l
a
c
e
,
a
s
a
T
h
i
r
d
W
o
r
l
d
t
h
e
f
i
r
s
t
i
n
t
h
e
G i v e ni s ,
(thereis no
blanket
by
a
covered
street,
on
the
photo
sleeplng
a
man
of
large
colour
a
he'problem'
. h i sm o r ee m o t i v ew a y o f p r e s e n t i nt g
i n d i c a t i ow
n h e r et h i s p h o t oi s t a k e n ) T
photosingle
page
a
features
right
The
to
the
text.
in
relation
as
New
functions
therefore
g r a p hs, h o w i n g
a l a r g ec r o w do f p e o p l es e a r c h i nag r u b b i s hd u m p a, r m e dw i t h c a n eb a g s
s a n( a n i m a g en o wa l s o
a n d b a s k e t sI n. r e l a t i o nt o t h i s p h o t ot,h e i m a g eo f t h e h o m e l e sm
, i t h i t s ( i n N o r t h e r nE u r o p e )
s l v e nw, h i l et h e p h o t oi t s e l fw
) e c o m eG
f a m i l i a ri n E u r o p e b
presented
as New.Togetherwith the
poverty,
is
mass
shocking
of
image
lessfamiliar
as a whole.Thuseach
the
chapter
of
Given
the
constitute
images
the
two
text,
introductory
informationwhich
for
the
Given
in
turn,
becomes,
once
received,
of
information,
new item
of
characteristic
is
Given
pattern
becoming
New
of
the
This
in
figure
6.6.
shown
follows,as
both in speechand in writing.
a1so,
language

New1

G i v e nI
( l n t h e T h i r d W o r l d p e o p l eh a v en o t h i n g
What can be done?)

G i v e n3

structure
Given-New
fis O.OCumulative

( t h e h o m e l e s si n d i v i d u a l ,
instantiatingand dramatizing
the given)

186

The meaning of composition

I D E A L A N D R E A L :T H E I N F O R M A T I O NV A L U E O F T O PA N D B O T T O M
Like manyother magazineadvertisements
and marketingorientedwebsites(seeMyers,
1994: r39), the Bushellsadvertisement
(plate D andthe Sonywebsiteare structured
along the verticalaxis. In suchtexts the uppersectionvisualizesthe \prsmi5se1 th.
product',the statusof glamourit can bestowon its users,or the sensoryfulfilment
it
c a n b r i n g .T h e l o w e rs e c t i o nv i s u a l i z et sh e p r o d u c ti t s e l f p, r o v i d i n m
g o r eo r l e s sf a c t u a l
informationaboutit, andtellingthe readersor userswhereit can be obtained,
or howthey
c a n r e q u e sm
t o r ei n f o r m a t i oanb o u ti t , o r o r d e ri t . T h e r ei s u s u a l l yl e s sc o n n e c t i o nl e, s s
ongoingmovement,
between
thetwo partsof the composition
than inhorizontally
oriented
compositions.
Instead,there is a senseof contrast,of oppositionbetweenthe two. The
u p p e rs e c t i o n
t e n d st o m a k es o m ek i n do f e m o t i v ea p p e aal n dt o s h o wu s ' w h a tm i g h tb e , ;
t h e l o w e rs e c t i o tne n d st o b e m o r ei n f o r m a t i vaen dp r a c t i c a sl ,h o w i n u
g s ' w h a ti s , .A s h a r p
d i v i d i n gl i n em a ys e p a r a tteh et w o ,a l t h o u g ha,t a l e s sc o n s p i c u o ul esv e l , t h e rm
e a ya l s ob e
connective
elements.
In plate2 this is createdby the way the jar of coffeeformsabridge
between
the upperandthe lowersectionof the ad,whilein the Sonywebsiteit is createdby
t h ec o l o u rs c h e mw
e h i c hu n i t e st h e p a g ea sa w h o l e i:n b o t ht h et o p a n dt h e b o t t o mp a r t o f
t h e p a g et h e d o m i n a nct o l o u r sa r es h a d eos f b e i g ew, i t h s o m eb l u ea n db l u e - g r eeyl e m e n t s
added(thejacketof the girl,the picturesin the bottompart of the page)as welI as some
red elements(e.g.the girl's lips and the words ,what,s new, in the top half, and the
headings
of the four sectionsin the bottomhalf). Overall,however,
the opposition
between
t o p a n db o t t o mi s s t r o n g l ey m p h a s i z ewdi,t h p r o d u c tps l a c e df i r m l yi n t h er e a l mo f t h er e a l ,
as a solidfoundation
for the edificeof promise,
andwith the top sectionasthe realmof the
c o n s u m e rs' su p p o s eads p i r a t i o nasn dd e s i r e s .
In othercontexts,
the opposition
between
top and bottomtakeson somewhatdifferent
v a l u e si .n a f a i r l yc o n s e r v a t i vbeu t ( i n t h e e a r l y1 9 9 0 s )s t i l lw i d e l yu s e dD u t c hg e o g r a p h y
t e x t b o o k( D r a g te f a l . , 1 9 8 6 ) t, h e u p p e rh a l f o f t h ef i r s t p a g eo f a c h a p t eor n ,a g a i n , . T h e
T h i r dW o r l d ' ,i s f u l l y v e r b a lp, r e s e n t i ngge n e r a l i z eads s e r t i o nasn d d e f i n i t i o nssu c ha s , A
l a r g ep a r t o f t h e w o r l dh a sa l o w d e v e l o p m e n t ' a n d ' T h eusned e r d e v e l o pceodu n t r i ew
se
c a l l p o o rc o u n t r i eosr d e v e l o p i ncgo u n t r i e sT' .h i sp r o v i d eas m o r en e u t r aal n dl e s se m o t i v e
( b u t n o t l e s si d e o l o g i c a kl )i n d o f i d e a l i z a t i o n
a , r e p r e s e n t a t i oonf t h e w o r l d w h i c h i s
divestedof contradictions,
exceptions
and nuances.
The lowerhalf of the page is given
overto a map of the world which usescolour-coding
to dividethe world into regions
accordingto the averageincomeof the inhabitants,
thus providingspecificand detailed
evidence
to supportthe assertionsin the top half. Directionsfor action- for instance,
coupons
for orderinga productin advertisements,
or assignments
or questions
in textbooks
- alsotendto befoundon the lowerhalf of the page,
usually at the bottomriqht(hencealso
New).
The informationvalueof top and bottom,then,can perhapsbe summarized
alongthe
f o l l o w i n gl i n e sI.f , i n a v i s u a cl o m p o s i t i osno, m eo f t h e c o n s t i t u e netl e m e n tasr e p l a c e di n
the upperpart,and otherdifferentelementsin the lowerpart of the picturespaceor the
page,thenwhat hasbeenplacedon the top is presented
as the Ideal,and what hasbeen
placedat the bottomis put forwardas the Real.For something
to be idealmeansthat it is

The meaning of comPosition

r87

of the information,hencealso as its,

presented
essence
as the idealizedor generalized
most sallentpart. The Realis then opposedto this in that it presentsmore
ostensibly,
'down-to-earth'
as
information(e.9.photographs
specificinformation(e.g.details),more
(e'g'
practical
or mapsor charts)/or more practicalinformation
evidence,
documentary
c o n s e q u e n cdeisr e
, c t i o nfso r a c t i o n ) .
betweenIdealand
givenso far,the opposition
As is alreadyevidentfrom the examples
by the
page
is occupied
If the upperpart of a
Realcanalsostructuretext-imagerelations.
(or
the
text andthe lowerpart by oneor morepictures mapsor chartsor diagrams), text
r ot l e ( w h i c h h, o w e v eirs,
p l a y s i,d e o l o g i c a ltl h
y ,e l e a dr o l e ,a n dt h e p i c t u r e sa s u b s e r v i e n
and so on)' If
practicalconsequence/
evidence,
importantin its ownway,as specification,
so that oneor morepicturesoccupythe top section/thenthe ldeal,
the rolesare reversed/
i s u a l l ya,n dt h et e x t
t h e i d e o l o g i c a lfloyr e g r o u n d epda r t o f t h e m e s s a g ies,c o m m u n i c a t ev d
servesto elaborateon it.
A s w i t h t h e G i v e na n d N e Wt h e l d e a l - R e asl t r u c t u r ec a n b e u s e di n t h e c o m p o s i t i o n
from
textssuchas layouts.Figure6.7, reproduced
bothof singleimagesand of composite
(Bols
a
includes
e/ al.,1986),
we havediscussed
textbool<s
one of the Dutchgeography
t
h
e
l
e
f
t
o
f
t
o
t
h
e
p h o t ow h i c hm a yh a v eb e e nt a k e ni n I n d i a i t so r i g i ni s n o tm e n t i o n ebdu, t
b
s e ea m a po f i n d i a .A y o u n gm o t h e rc,a r r y i n ga b a b , o c c u p i e s /y
p i c t u r ea, s i t s G i v e n , w e
a ith
o f t h ev e r t i c a l l cy o m p o s epdh o t oa, s a ' T h i r dW o r l d 'M a d o n n w
h e r s e ltfh, et o p s e c t i o n

*"w*t
ffi*'$*i*'#it:kffi;

1
l:J.r'S,-;:.-.

ffi.
ffiffit:
@

ef ar.,1986)
figO.Z overpopulation(Bols

I88

Themeaning of composition

c h i l d .T h e b o t t o ms e c t i o ns h o w sa g r o u po f w o m e na n d c h i l d r e ns,i t t i n go n t h e g r o u n d ,
tightly packedtogether.The young mother looks at this group,a worried expression
crossingherface.In thiswaythe pictureas a wholeexpresses
a contradiction
between
the
deep-rooted
Idealof motherhood
and the Realof overpopulation.
Immediately
belowthe
p h o t ow e f i n da c o l l a g eo f n e w s p a p ehre a d l i n e(s' l n d i as t r u g g l easg a i n s t
overpopulation,,
'Unemployment
nightmarein India') as Real(the newspaper
as sourceof ,hardfacts,,of
evidence)
with respectto the moresymbolic,idealized
and emotiverepresentation
of the
p r o b l e mi n t h e p i c t u r e .
I d e a l a n d R e a lc a n a l s o p l a y a r o l e i n d i a g r a m sI.t i s s t r i k i n g f, o r i n s t a n c et h, a t
diagramsbasedon a verticaltimelinesometimes
idealizethe present,
sometimes
the past.
Thealready-mentioned
Dutchgeography
textbookWerkaande Wereld(Bolset al.,1986)
featuresa diagramwhichrepresents
the decrease
of livingspaceper headof the population,by meansof a verticalarrangement
of what look likechessboards
of differentsizes.
0n these'chessboards'
stand cartoonfigures.0n top we see a gentlemanfrom 1900,
completewith top hat,on a large('6285 m2')'chessboard'.
At the bottom,on the smallest
'chessboard',
w e s e ea ' p u n k ' c h a r a c t efrr o m 1 9 g 0 . H e r e ,a s i n m a n ya d v e r t i s e m e n t s ,
the past,the 'goodold days',is presented
as Ideal.The other Dutchgeography
textbook
we mentioned(Dragt ef al., 1986) featuresa 'geologicalcalendar,in whichthe present
('development
of vertebrates',
completewith a smalldrawingof a nakedwoman)becomes
t h e I d e a l t, h ec u l m i n a t i oonf p r o g r e sasn de v o l u t i o n .
M a n yv i s u a l sc o m b i n eh o r i z o n t aal n d v e r t i c a sl t r u c t u r i n gI n. f i g u r e6 . g ( a s i n f i g u r e
6 ' 4 ) G o di s G i v e na, n dA d a ma n d E v ea r e N e w .B u t t h e i rf a l l f r o m g r a c eh a si n t r o d u c e d
a ( New)opposition,
between
the Idealof Paradise,
of the Gardenof Eden,andthe Realof
deathand decay- and the two are visuallyseparatedby the riverwhich surrounds
the
G a r d e no f E d e n .
T h ec o m m u n i c a t i omno d e li n f i g u r e6 . 9 a l s oc o m b i n ehs o r i z o n t aal n d v e r t i c a sl t r u c t u r i n g ,i n a n i n t r i c a t ep i e c eo f v i s u atlh i n k i n ga b o u t h e i m p o s s i b i l iot yf k n o w i n gr e a l i t y ' a s
it is',objectively.
Givenis the 'event',as it exists'out there,,separate
from our perception
of it. New,andthereforeproblematic,
is our perception
of the eventand,at the lowerlevel,
t h e w a y w e c o m m u n i c a toeu r p e r c e p t i o nt hsr o u g hl a n g u a g eI d. e a li s t h e ' e m p i r i c a l , ,
the
w o r l d ' a si t i s ' ,a n do u r p e r c e p t i oonf i t , u n m e d i a t ebdy c o m m u n i c a t i ocnu,l t u r el,a n g u a g e
(whichare positioned
in the lowersection).Realare our interpretations
of thesepercept i o n s ,a s m e d i a t e d
b y c o m m u n i c a t i oCn l.e a r l yt ,h i s d i a g r a mc o u l dh a v eb e e nv e r t i c a lo, r
h o r i z o n t aB
l . u t i t i s n o t .C o m m u n i c a t i o
i snp o s i t i o n e bd e l o w t h e , e v e natn, d i t s p e r c e p t i o n .
T h e ' e m p i r i c a l ' w o r ladn d ' p u r e ' o b s e r v a t i o
an
r e I d e a l .B u t t h i s I d e a li s a l s od e p i c t e di n
isolationfrom our'statements'about i! and our perceptions
of thesestatements.
This is
whatthe lowersectionof the diagram,the Real,tellsus.Perception
is secondhand,
filtered
throughcultureand language,
which,as the double-headed
arrowsindicate,feed back
intoour perception
of nature,and henceinto natureitself.Thediagramtellsusthat reality
doesexist,but that our perception
of it can only be 'subjective,
selective,
variableand
u n p r e d i c t a b l(eM' c Q u a ial n dW i n d a h l1, 9 9 3 :2 5 ) .
F i g u r e6 ' 1 0 i s a n o t h e ro n e o f o u r o r i g i n a le x a m p l e fsr o m l a t e t 9 8 0 s A u s t r a l i a n
women'smagazines,
but it remainsa good exampleof the combinationof horizontal

The meaning of composition

'

I89

fis t.A Gott Shows Deathto Adamand Eye (French,titteenlh-century miniature from ms' ol De civitate ,eD (from Hughes'
r969)

'marriagewas madein
and verticalstructuring.Ideal is the moment,onemightsay,that
HeavenM
' . o d a l i t yi s ' d i s t a n t ' r, e p r e s e n t i nt hge ' n o t n o w ' ,t h e ' o u t o f t i m e ' .T h eb o t t o m
s e c t i o nb,y c o n t r a s tr,e p r e s e ntthsew o r l do f i s ' , ' n o w ' , ' i no u rt i m e ' .
couple,the well-established
as the quintessential
Givenis the royalcouple,presented
symbolof family values.New are Sydney'sGwenand Ray l(inkade,an instance,an
. e n c ew h a t i s G i v e ni s t h e p r e - e m i n e n c( hei s t o r i c a l lsyo, c i a l l y ,
e x a m p l eo f t h e s ev a l u e sH
. h a ti s
o f t h e m a r r i e dc o u p l eW
example
s e m i o t i c a l l yo)f t h e r o y a lc o u p l ea st h e p a r a d i g m
instances
Newis oneinstanceof the paradigm wheremanyothersof a setof acceptable
w o u l dh a v es e r v e d
equallw
y e l l .T h i sd i s t i n c t i oins p e r h a pssh a r p e si nt t h e b o t t o mp a r t .T h e
an
at the safedistanceof forty years,almostidentical,
two picturesin thetop part become,
eoualterms.
eouationbetween
it is the
the placeof the Newseemsmerelyperfunctory:
In one,and a very real sense,
classifications
of the existing
placeof the replicationof the paradigm,of the reproduction
o f t h e c u l t u r e , t h e p l a c e w h e r e t h e u n d e r l y i n g vt hael uceusl toufr e a r e r e a f f i r m e d .NTehwe

190

The meaning of composition

meansandcontrol
(or communicating)
dim!nsion

lQ

fiq O.e Gerbner,scommunicationmodel (Watsonand Hilt, t9g0: 7i)

instantiatesand 'naturalizes'these
values.But that very fact also makesthe position
problematic/
for it is at the sametime the placeof the affirmationof what is,the placeof
the reproduction
of socialmeanings,
andthe placewherethe contestation
of paradigmatic
valuescan take place,the placethereforeof the constantproductionof socialmeanings
s ork'in
(e.9o
. f n e wd e f i n i t i o nosf ' w o m e n ' w
f i g u r e6 . 2 ) , e v e n
w h e nt h a t p r o d u c t i osne e m s
to be merereproduction
and henceconservative
in its effects.Couldthere,for instance,
havebeena Vietnamese
or Lebanese
or Aboriginalcouplein this positionOn I9B7),notto
m e n t i o na g a yc o u p l e ?T h i sc o n t e s t a t i oonv e r , e s t a b l i s h e d , , , G i v a
e lnu,e sm a y h a p p e ni n
one0r two ways:a readerwho is not Anglo-Australian
will eitheridentifywith the syntagm
of Anglo-Australianness,'assimilate',
in otherwords;or will refusethe syntagmas having
no relevance
or valueto him or her.In the lattercasetherewill be pressure
on this placein
t h es y n t a g ma,n dt h i s i n t u r nw i l l r e s u l ti n p r e s s u roen t h e p a r a d i g m
a sa w h o l e .
Thereis anotheraspectto this; while the syntagmdeclaresitself as unquestionably
established,
its appearance
pointsat the sametime to a problemwith the paradigm,
to the
needpreciselyfor a testing and ke)affirmationof its legitimacy.Readfrom the right to the
left,the syntagmdeclaresthat it is the willingness
of readersto read it as a relationof
identity(withina hyponymic
structure)whichgiveslegitimacy
to the royalcouple.Royalty
is the established,
the Given.What has to be reaffirmedanew is that subjectsare still
preparedto enter into this paradigmaticrelation.A monarchytrying to establishitself,
on the other hand,might needto utilize a structurewherethe powerof the peopleis
represented
as Given,and the identityof the monarchis to be established
- that is.the
r o y a lc o u p l ew o u l da p p e a or n t h e r i g h t .

Themeaning of composition

'r/{;rgugry
, loesft&rX!,1t{ti
I Prihoe* eiirrbeit srd
sk
I Klip Marntb'ln
I (*ltd$idg4e{hand'
lsm qd.e*

${}

ttrifl.#{tys#,y'lf

HadE&

'f
h*i'r*
f relr dilr'.r$if
srrkj*. hut tl* lorg a*l rt*ngtlr
rlif,if nrfila$es
d* !l!: lafia

rJ
ir

rbt! n6lh4.hrod aMn!

@e.
r h d ) o ! . !n g l D . dr t u n r d
Pd.ce Phllo. ond Srdq'3 Gsn
d dt
6d FryrtudcSdo
,oddln! sn.lvaBry - rd lolt
iod !o(t .n 40 ts6 oltupll|*

fiq e.fO Royal couple(Australian Women,sWeekty,Nouember

1987)

Thusthis syntagmrevealsa numberof socialfacts:what is regardedas established

and Given;what the culturalclassification
systemis with respectto a certainfeature,'and
whetherthe systemis progressive
or reactionary.
It is aboveall a syntagmwhichdoesnot
permitdeviance;
or,rather,
oncean item is in the syntagm,it hasto be readas beingin the
paradigm.
Whereit doespermitdeviance
is on the part of the reader,
who can refuseto be
p a r t o f t h ec o m m u n i tdye f i n e d
b y t h i sp a r a d i g m .
In the Westernvisualsemiotic,then,the syntagmaticis the realm of the processof
semiosis,
and the top-bottomstructurethe resultand recordof semiosis,
the realm of

191

192

The meaning of composition

, e m i m e t i cr e p r e s e n t a t i o n
o r d e rt,h e p a r a d i g mt h
f c u l t u r e( H o d g ea n d l < r e s s1,9 g B ) .T o
maintainand unsettletop-bottomstructures,
onehasto work on the left-rightstructures.
That this systemgoesback a long way in westernart can be seenin genressuchas
fifteenth-century
Flemishdiptychs,
which,for instance,
may havethe Virginandthe child
as Given,and a donoror Saintas New,as in the diptychby the masterof Brugesin the
c o u r t a u l dG a l l e r ya, n d p o l y p t y c hfsr o m t h e s a m ep e r i o dw
, h i c h m a y p a r a l l e la R e a l
( e a r t h l ya) n dI d e a l( h e a v e n l yv )e r s i o o
n f t h es a m et h e m ei n t h e l o w e ra n du p p e rp a r to f t h e
panels,
as in Bosch'sLastJudgement,where
the lowerpart of oneof the left panelsshows
A d a ma n d E v eb e i n gd r i v e nf r o mt h e G a r d e no f E d e na n dt h e u p p e rp a r t t h e e x p u l s i oonf
t h e R e b eAl n g e l sf r o m h e a v e n .
As we havesaid in the Introduction,
we are largelyconcerned
with the description
of
the visualsemioticof Westerncultures.Cultureswhich have long-established
reading
directionsof a differentkind (rightto left,or top to bottom)are likelyto attachdifferent
valuesto thesepositions,
as shownin figure6.3.In otherwords,readingdirectionsmay
b e t h e m a t e r i a il n s t a n t i a t i o nosf d e e p l ye m b e d d ecdu l t u r a vl a l u es y s t e m sD. i r e c t i o n a l i t y
as such,however,
is a semioticresourcein all cultures.All cultureswork with margin
andcentre,left and right,top andbottom,evenif theydo not alI accordthe samemeanings
a n d v a l u e st o t h e s es p a t i a ld i m e n s i o nA
s .n d t h e w a y t h e y u s et h e m i n t h e i r s i g n i f y i n g
s y s t e mw
s i l l h a v er e l a t i o n so f h o m o l o g w
y i t h o t h e rc u l t u r a ls y s t e m sw, h e t h e r e l i g i o u s ,
p h i l o s o p h i coarl p r a c t i c a l .
Wewillendthissectionwithonefurtherexam
e soef s oGf i v e n a n d N e w , t h e w a y
t hpel u
in which Rembrandt
usedGivenand Newfor the expression
of affectiveaspectsof meani n g ,a n d t h i s e s p e c i a l liyn r e l a t i o nt o t h e s o u r c ea n d d i r e c t i o no f l i g h t a n d t h e e f f e c t
producedby that. In many,perhapsthe majority,of Rembrandt,s
paintings,whetherin
landscapessuch as Landscapewith a stone Bridge or in portraits such as A young
Womanin Bed or Double Portrait of the MennonitePreacherCorneliusClaeszAnslo and
his WifeAeltie GerritsdrSchouten(figure6.71),the light sourceis outsidethe left frame
o f t h e p i c t u r ea n d i l l u m i n a t em
s a i n l yt h e l e f t p a r t , l e a v i n gt h e r i g h t o f t h e p a i n t i n gi n
greateror lesserdarkness.
Iconographically
speaking
the metaphoric
rangeof lightis wide
- l i g h tc a ns i g n i f y ' t h ed i v i n e ' , ' i l l u m i n a t i o n ' , ' h o p e ' ,
e t c .I n t h e s ep a i n t i n glsi g h t w
, hatever
its meaning,is in the areaof the Given,the takenfor granted,thenow/present.,Light,
is
G i v e n , ' d a r k n e sNse' w .
Theheightof the unseenlightsourcealsovaries:in YoungWomanit is on or just below
the centre;in DoublePortrait it comesfrom a positionabovethe halfwaymark,perhaps
two-thirdsof the way up;and in Landscape
it comesfrom somewhere
highup,nearthe top
cornea
r n di n t h e n e a rm i d d l ed i s t a n c eT.h a ti s ,l i g h tm a yb e i n t h ea r e ao f t h e R e a cl o m i n g
f r o m a ' m u n d a n e ' s o u r coer i t m a y b e ' d i v i n e ,I.n o t h e rp a i n t i n gtsh e l i g h tc o m e sf r o m
withinthe painting;for instance,
in TheHolyFamilyon the Flight to Egypt,where
it forms
t h e ( d i v i n e l)i g h t ' i nt h e w o r l d ' ( t h e r ei s a l s oa s e c o n df a
, i n t l i g h tc o m i n gf r o m o u t s i d ei n
,
the sky above).In Belshazzar's
Feast,by a most unusualcontrast,thelight source(the
g l o w i n gs c r i p ta n n o u n c i nt hg e d o o mo f t h e k i n g )i s s i t u a t e di n t h et o p - r i g hqt u a d r a n-t
the
spaceof the New and the'Ideal?divine'.The variationin the sourceand directionality
of lightthus hasa complexset of meanings.
It can contrastthe secular/mundane
andthe

The meaning of composition

fis O.ff Douhle Portmit of the Mennonite Preachet Cornelius Claesz Anslo and his Wile Aeltje Gerritsdr Schouten
(Rembrandt,
1641)(StaatlicheMuseen,Preussischer
Kulturbesitz,
cat.no.828L)
Gemaldegalerie,

andall
divine/ideal;
lightas Givenandtakenfor granted,
and lightas Newandastonishing:
thesein variablecombinations.
In DoublePortrait,for example,the light comesfrom
outsidethe depictedworld,is situatedin the areaof the Given(the areawherethe scripturesare depicted),
andcomesfrom just abovethe midwaypointbetweenIdealand Real,
eg
f f e c ti s
s ot h a t i t c o u l db e i n t e r p r e t eads ' d i v i n e y' ,e tc l o s et o t h e R e a l0. n eo v e r w h e l m i n
the brightness
of the areaof the Given,andthe total darkness
of the areaof the New(the
future?)to whichthe two figureshave,in any case,turnedtheir backs.Are we entitledto
- perhapsa deep,pervasive
pessimism
readfrom this autobiographical,
affectivemeanings
about both the future,the New,and the present,the Real,whichthen contrastswith a
which
feelingof securityaboutwhat was,a faith in a divinelightfrom the pastcertainty,
entailsthat we must turn our back on the New,on the future? If so,theseaffective,
personalmeanings
and,of course/
are surelyas significant
as socialandculturalmeanings
relatedto them.

193

L94

The meaning of composition

T H E I N F O R M A T I O NV A L U E O F C E N T R EA N D M A R G I N
V i s u acl o m p o s i t i om
n a ya l s ob es t r u c t u r eadl o n gt h ed i m e n s i o nosf c e n t r ea n dm a r g i nT. h e
mosttypicalmanifestations
of this can be foundin children'sdrawingsor,for example,
in
B y z a n t i naer t . A s A r n h e i m( 1 9 8 2 : 7 3 )n o t e s ,
I n t h e B y z a n t i nceh u r c h etsh ed o m i n a nitm a g eo f t h ed i v i n er u l e rh o l d st h ec e n t r eo {
the apse.In portrait paintings,a pope or emperoris often presentedin central
p o s i t i o nM
. o r eg e n e r a l l w
y ,h e nt h e p o r t r a i to f a m a ns h o w sh i m i n t h e m i d d l eo f a
framedareatwe seehim detachedfrom the vicissitudes
of his life's history,alone
goeswith the
with his own beingand his own thoughts.A senseof permanence
centralposition.
F i g u r e6 . I 2 i s a n e x a m p l -e a B u d d h i spt a i n t i n gi n w h i c ht h e c e n t r afli g u r ei s s u r r o u n d e d
b y a c i r c l eo f s u b o r d i n a t eAsr.n h e i mi n f a c t m a k e st h e c e n t r et h e c r u c i a e
l l e m e not f h i s
theoryof composition,
conceiving
of the visualobjectsin a composition
as'somanycosmic
bodiea
s ttracting
a n dr e p e l l i nogn ea n o t h e irn s p a c eQ
' 9B2:207).
I n c o n t e m p o r a rW
y e s t e r nv i s u a l i z a t i ocne n t r a lc o m p o s i t i oins r e l a t i v e l u
yncommon,
polarizeelementsas
thoughheretoo there may be changesin train. Most compositions

fiq O.fZ Burldhistpainting (Arnheim,1982)

Themeaningof composition

. u tw h e no n eo f u sw a st e a c h i n o
g n a m e d i ad e s i g n
G i v e na n d N e wa n d / o rI d e a la n d R e a l B
playedan importantrole in the
coursein Singapore,
he foundthat centralcomposition
imaginationof youngAsian designers.
Perhapsit is the greateremphasison hierarchy,
h a r m o n ya n d c o n t i n u i t yi n C o n f u c i a nt h i n l < i n gt h a t m a k e sc e n t r i n ga f u n d a m e n t a l
o r g a n i z a t i o nparli n c i p l e
i n t h ev i s u asl e m i o t i o
c f t h e i rc u l t u r eM
. u c ho f t h ew o r kp r o d u c e d
by these studentshad strong dominantcentres,surroundedor ffankedby relatively
u n p o l a r i z emda r g i n ael l e m e n t s .
While many Anglo-Western
tabloid newspapers
tend to adhereto a basicleft-right
structurein the layoutof their front pages,othersplacethe mainstoriesand photographs
in the top section.Thefront pagesof the business
sectionsof the SydneyMorning Herald,
featuringa large photo (or,
however,
for a time invariablyusedcentral composition/
frequently,
drawing)in the centreof the page:for instance,
Asianstudentsenteringthe
n e o - G o t h iQ
c u a d r a n g loef t h e U n i v e r s i toyf S y d n e yw h e nt h e p a g ef e a t u r e da r t i c l e so n
education
as a moneyearnerfor the country'seconomy;
a cartoon-lil<e
drawingof two men
playingMonopoly(basedon Van Doesburg's
Card-players-seefigure 5.6), when corporatetakeovers
dominated
the news;and so on.Suchpicturesprovideda symbolickernel
for the issues
arrangedaroundthem- newsstories
of the day,anda centrefor the elements
evenif now
at the top and to the left,as,still,the ldeal and the Givenof the newspaper,
somewhatmarginalized;
advertisements
as the Real;anda columnof expertcommentary
as New,henceasthe elementto whichreadersshouldpayparticularattention.Figure6.13

fig e.ff Goingon holiday(from Prosser,2000:

ll7)

195

196

The meaning of composition

s h o w sa d i a g r a mf r o m a t o u r i s ms t u d i e tse x t b o o ki n w h i c h' g o i n go n h o l i d a yi's t h e c o r e

issue,
and in whicha rangeof reasons
for goingon holidayis arrangedaroundthis Centre,
withoutanysenseof polarization.
To generalize,
then,if a visualcomposition
makessignificantuseof the Centre,placing
oneelementin the middleand the otherelementsaroundit. we will referto the central
elementas Centreandto the elements
aroundit as Margins.Forsomething
to bepresented
as Centremeansthat it is presented
as the nucleusof the informationto which all the
otherelements
are in somesensesubservient.
The Marginsare theseancillary,dependent
elements.
In manycasesthe Marginsare identicalor at leastverysimilarto eachother,so
that thereis no senseof a divisionbetweenGivenand Newand/orIdealand Realelements
pageswediscussed
amongthem.In othercases- for instance,
the newspaper
above- Centre
a n d M a r g i nc o m b i n w
e i t h G i v e na n d N e wa n d / o rI d e a la n d R e a l .
l nd
N o t a l l M a r g i n sa r e e q u a l l ym a r g i n a lC
. i r c u l a rs t r u c t u r ecsa n c r e a t ea g r a d u a a
gradeddistinctionbetween
Centreand Margin,asfor instancein thecommunication
model
by Anderschet al. in figure6.I4, wherethe process
of 'structuring'is moremarginalthan
t h e p r o c e sosf ' e v a l u a t i n gI 'n. t h i s m o d e ln, a t u r e( t h e ' e n v i r o n m e n its' )C e n t r eo, r i g i na n d
p r i m em o v e o
n f n a t u r ec, o m m u n i c a r f c o m m u n i c a t i oCno. m p a r etdo t h ed o m i n a npt o s i t i o o
just as, in the medievalmapsof citieswe discussed
tion is a marginalphenomenon,
in
chapter3, the citiesthemselves
wereplacedin the centreand depictedwith topographical

-i't'i\
\**)W,'*"91
@

figO.f+ Anderscheta/.'scommunicationmodel(fronWatsonandHill,lgS0:14)

The meaning of composition

197

accuracw
, h i l et h e s u r r o u n d i ncgo u n t r y s i dwea s r e p r e s e n t eodn a s m a l l e sr c a l ea, n dw i t h
lessaccuracy.
Verbalcommentarles
do not necessarily
try to 'translate'suchmeanings.
(
1
9
8
0
:
7
6
)
,
f
o
r
W a t s o n a nH
d ill
e x a m p l e , s a y t h a t i n t h i s m ot hdee'lm e s s a g e ' i s ' i n t e r a c t the relationbetween
ing with factorsin the environment'.
Yet,the modelitselfrepresents
s /' n o n c o m m u n i c a t i oa n dt h e ' e n v i r o n m e nnt o
nu
, ta sa o n e - w apy r o c e s a
' / ta s i n t e r a c t i o b
transactivereaction',accordingto our terminologyin chapter2 (thereis an arrow only
rocesse
f r o mt h e ' e n v i r o n m e n t 'tthoec o m m u n i c a t i p
ve
t hsa t s u r r o u n idt ) .A n d' i n t e r a c t i n g '
s u g g e s tgsr e a t eer q u a l i t yb e t w e etnh e ' m e s s a g e ' a nt hde ' f a c t o r si n t h e e n v i r o n m e n t ' t h a n
d o e st h e c e n t r e d
c o m p o s i t i oonf t h e m o d e l .
A s w e h a v es e e n G
, iven-New
a n d I d e a l - R e acl a n c o m b i n ew i t h C e n t r ea n d M a r g i n .
v i s u a ls p a c ea c c o r d i n tgo t h e s ed i m e n s i o nr se s u l t si n t h e f i g u r eo f t h e C r o s sa,
Dividing
fundamentalspatialsymbolin Westernculture(seefigure6.15).Just how marginalthe
of the Centre.But
marginsare will dependon the sizeand,moregenerally,
on the salience
as the invisible(denied)
evenwhenthe Centreis empty,it continues
to existin absentia,
p i v o t a r o u n dw h i c he v e r y t h i n eg l s et u r n s ,t h e p l a c eo f t h e ' d i v i n er u l e r ' .T h e r e l a t i v e
perhaps
infrequency
of centredcompositions
in contemporaryWesternrepresentation
signifies
that,in thewordsof the poet,'thecentredoesnot hold'anylongerin manysectors
of contemporary
society.
g i v e na n dN e ww l t h C e n t r ea n dM a r g i ni s t h et r i p t y c h .
O n ec o m m o nm o d eo f c o m b i n i nG
In manymedievaltriptychsthere is no senseof Givenand New.The Centreshowsa key
r e l i g i o utsh e m es, u c ha s t h e C r u c i f i x i oonr t h e V i r g i na n d C h i l da, n dt h e s i d ep a n e l s h o w
S a i n t so r d o n o r s k, n e e l i n g
d o w n i n a d m i r a t i o nT. h e c o m p o s i t i oins s y m m e t r i c ar la t h e r
t h a n p o l a r i z e da,l t h o u g h
t h e l e f t w a sr e g a r d e ad s a s l i g h t l yl e s sh o n o r i f i cp o s i t i o nI .n t h e
the birth of
for instance,
becomemore narrative,showing,
sixteenthcenturyaltarpieces
on the centrepanel,andthe
Christor the roadto Golgothain the left panel,the Crucifixion
n l,b e i ts u b o r d i n a t e d
R e s u r r e c t i o n t h e r i g h tp a n e lT. h i sc o u l di n v o l v e
s o m ep o l a r i z a t i o a
of Adam),the
to the temporalorder,with the left as the 'bad side'(e.9.the transgression

rig O.fs The dimensions

otvisualspace

198

Themeaning of composition

r i g h ta s t h e ' g o o ds i d e '( e . 9 t. h e R e s u r r e c t i oann) dt h e m i d d l ep a n e rl e p r e s e n t i nCgh r i s t ' s

Bosch'sLastJudgement(e.ndalsohis
roleas Mediatorand Saviour(e.9.the Crucifixion).
EarthlyDelightl invertsthis,showingon the left the Gardenof Edenand on the right a
c a t a c l y s mvi ci s i o no f H e l li n w h i c ht h e r ei s n o p l a c ef o r t h e ' a s c e not f t h e b l e s s e d ' .
polarized,
with
andnewspaper
layoutsare generally
Thetriptychsin modernmagazines
a ' G i v e n 'l e f t ,a ' N e w ' r i g h t ,a n da c e n t r ew h i c hb r i d g etsh et w o a n da c t sa s ' M e d i a t o r 'I.n
August2004,thetop bannerof Nokia'swebsiteshowedon the left an imageof a fashiona b l ew o m a na n d o n t h e r i g h ta N o k i ai m a g i n gp h o n eT. h et e x t i n t h e C e n t r ec o n n e c t e d
Welcome
the two. In fact it consisted
of two alternatingtexts.First we read'lnspiringly.
to LondonFashionWeek',thenthis first text madeway for 'lnspiringlydifferent.Thenew
officialimagingphoneof LondonFashionWeek'.Theconceptof fashionis Given,andthe
p h o n ea s a f a s h i o na c c e s s o rNye w .
N o k i ai m a g i n g
T r i p t y c h cs a n a l s o b e u s e dt o s t r u c t u r ed i a g r a m sI .e d e m ae t a l . ( 1 9 9 4 : 2 1 7 ) s h o w s
h o w t h e l e f t c o l u m no f t h e o r g a n i z a t i o n cahl a r t o f a l o c a lc o u n c i l i s t st h a t c o u n c i l ' s
'CorporateServices',so making the 'administrativeand financial backboneof the
Services',
the
organization'Given,while the council's'Development
and Environmental
in the centralcolumn,as
is described
department
which'connects
all otherdepartments',
the Mediator.In a lectureon socialcognitionattendedby the authors,the lecturerused
t h e b l a c k b o a r(dc o n v e n t i o nballa c k b o a r dasl s oh a v ea t r i p t y c hs t r u c t u r e lt)o l i s t ,o n t h e
e o s to f t h o s e
l e f t p a n e l a, n u m b e ro f k e y i s s u e si n l i n g u i s t i c(st h i sw a s G i v e nb e c a u sm
p r e s e nw
t e r el i n g u i s t a
, t h e r i g h t p a n e la n u m b e r
s n ds t u d e n t o
s f a p p l i e dl i n g u i s t i c so) n
s e r em e e t i n gt o d i s c u stsh e s o c i a l
o f i s s u e si n s o c i o l o g(yt h i sw a s N e w ,a s t h e l i n g u i s tw
r e l e v a n coef d i s c o u r saen a l y s i sa) ,n do n t h e c e n t r a l ' p a n e l ' aonu t l i n eo f h i so w nt h e o r yo f
the two fields,andas an
as the necessary
link between
socialcognition,
whichhe presented
issuethat shouldbethe centralconcernof thosepresentat the meeting.
Verticaltriptychsare also commonin websites.
The triptych from the Universityof
strucas a simpleMargin-Centre-Margin
0xfordwebsite(figure6.16) canbe interpreted
t u r e ,t h o u g ht h e r ei s s o m ep o l a r i z a t i oi n t h a tt h et o p i m a g ei s a ' l o n gs h o t 'a n dt h eb o t t o m
s f h i s t o r ya n d
i m a g ea c l o s eu p . 0 v e r a l h
, h i l ei m a g e o
l , o w e v etrh, e s t u d e nits C e n t r eh e r ew
. h et r i p t y c hi n f i g u r e6 . 1 7c o m e sf r o ma G e r m a jnu n i o r
t r a d i t i o ns u r r o u nadn ds u p p o rht e rT
p lo l i t i c st e x t b o o k( N i t z s c h k e1,9 9 0 ) .A s t h e I d e a l ,w e s e e( i n c o l o u r )i m m i high-schoo
grants(Ausltinder,'foreigners')
professions.
in high-status
As the Real,we see'foreigners'
professions.
This Realis dividedinto a Givenanda New,with a colourphoto
in low-status
photoas New,as though,in the 1990s,the low statusof
as Givenand a black-and-white
i m m i g r a n tssh o u l db e l o o k e da t i n a m o r es o b e rl i g h t ,a n d n o l o n g e ra s ' G i v e n ' a si t o n c e
was.In the Centrewe see,againin blackand white,a singleimmigrantworkercleaninga
train. The accompanying
text encourages
studentsto explorewhat wouldhappenif 'one
for the
day all foreignworkershad to leave'.What, it asks,would be the consequences
theworkersthemselves,
buildingindustry,
the ownersof hostels,
the childrenof theworkers,
t h e m a n a g e rosf h o s p i t a lasn dc l e a n i nfgi r m s ?I n o t h e rw o r d s , t h itsr i p t y c h( i t s e l tf h e N e w
o n t h e d o u b l ep a g ei n w h i c hi t a p p e a r st)e l l s u s t h a t f o r e i g nw o r k e r ss h o u l dp, e r h a p s ,
to do'our'
i d e a l l yb e a b l et o m o v ei n t o h i g h - s t a t upso s i t i o n sb,u t i n r e a l i t ya r e n e e d e d
or at least mitigate,this
menialjobs. The central image is an attempt to overcome,

The meaning of composition

fig O.fO Verticaltriptych from the Universityof 0xford website(www.oxford.ac.uk)

c o n t r a d i c t i o nI t. s h o w sa w o r k e rw h o , l i k e t h e h i g h - s t a t uism m i g r a n t isn t h e l d e a l ,i s
depicted
a s a n i n d i v i d u aal ,n d a s i n v o l v e idn ' c l e a n ' w o r k b
, u t w h o ,a l s ol i k et h e w o r k e r s
job * and is shownin the sober,
modality
documentary
shownin the Real,hasa low-status
of blacl<-and-white
realism.
The structureof the triptych,then,can be eithera simp[eand symmetricalMarginC e n t r e - M a r g isnt r u c t u r eo r a p o l a r i z esdt r u c t u r ei n w h i c ht h e C e n t r ea c t sa s a M e d i a t o r
b e t w e eG
n i v e na n d N e wo r b e t w e eInd e a la n d R e a l( s e ef i g u r e6 . 1 8 ) .
In this and the precedingsectionof this chapter,we havenot drawn any parallels
w i t h l a n g u a g eT.h o u g hs p o l < eEn n g l i s hh a s i t s o w n G i v e n - N e w
s t r u c t u r et ,h i s i s n o t t h e
casewith the ldeal*Realand the Centre-Marginstructures.This is not to say that
in language,
the meanings
thesestructuresexpresscannot,in someform, be expressed
but rather that they are more readily and frequentlyexpressedvisually,and that
language
u ,n l i k ev i s u a cl o m m u n i c a t i ohna, sn o t d e v e l o p e d ' g r a m m a t i c a l ' f otromesx p r e s s
t h e m .A s w e h a v ee m p h a s i z et hdr o u g h o ut th i s b o o k s, o m e t i m el sa n g u a gaen dv i s u acl o m municationexpressthe same kind of semanticrelations,albeit in very differentways,
but there are also many types of semanticrelationwhich are more often and more
easilyexpressed
visually,just as thereare otherswhich are more often and moreeasily
e x p r e s s el idn g u i s t i c a lw
l yi,t h e p i s t e m o l o g i ccaoln s e q u e n coef st h e k i n dw e d i s c u s s ei ndt h e
i n t r o d u c t i oann dc h a p t e1r .

199

200

The meaning of composition

{ wt"ui"l" xinder ausliindircher [itern lind in eufer

Sc"r,le?
Fragt sie,seit wann ihre Elternin
Deutschland5ind,was sie arbeaten warum siehierhdrkamen,ob
sie bleibenwdhn und warum.
Ste$tim Atlas i:st woher ausl{ndischeArteiter komrnen,rioher
Fliidrdinge.taBt euch enihlert
wie es dort aussieht"Lest in E dkundebiichemoder Reiseliihrern
rach. Achtet dabei besonder auf
die Arbeits- und Lebenssit'ratisn.

If n grr*a,iut eurer Sdlule

gibt es Zahleoiiber ausliindis<ire
fthtiler
Vorschlligezum Cesprach:
. Cibt es Problememit ausliindischenSchiilerinnenund Schiilern{
o Welche Ldsungsmdglidrkeiten
seht ihr?
o W.s denkenausliindische
Schiler tiber die Eundesrepublik?
o Was denkendeutscheSchiiler
iiber Auslinder?

/l stettt euclr vor, da8 an

einsm Tag!alle ausliindisclcn
Arbeiter die Fuodesrepublikverlassenmii8ten. Eildet Gruppen
und iiberlegt was e.B.
o Bauunternehmer,
o Kinderder Arbeiter.
o caststiittenbesitzer,
o die Arbeiter selbst
o Leiter von l*artl<enhiius!m,
o Leitervon Reinigungsfrnenfiir
ProHerneUithn, trn miitlten ...
Tragt die lrgebdsse zusanrnen
und besprerl* sle.

f)

fiq e.fZ Verticaltriptychfrom a Germanschooltextbook(Nitzschke,

1990)

The meaning of compositton

Given

207

Mediatof

fiq e.fe Horizontaland verticaltriptychs

SALIENCE
is textual.Integration
Thefundamental
functionof integration
codessuchas composition
intothewhole,andto provide
codesserveto producetext,to placethe meaningful
elements
deterhow composition
coherence
and orderingamongthem.So far we havediscussed
m i n e s' w h e r et h i n g sc a n g o ' a n d h o w t h e p o s i t i o n i nogf t h e e l e m e n t isn a c o m p o s i t i o n
endowstheseelementswith differentinformationvaluesin relationto other elements.
But the composition
of a pictureor a pagealso involvesdifferentdegreesof salienceto
its elements.Regardless
of wherethey are placed,saliencecan createa hierarchyof
importanceamongthe elements,selectingsome as more important,more worthy of
or the
attentionthan others.The Givenmay be more salientthan the New,for instance,
N e wm o r es a l i e ntth a nt h e G i v e no, r b o t hm a yb e e q u a l l ys a l i e n tA. n dt h e s a m ea p p l i e tso
I d e a la n d R e a a
l n dt o C e n t r ea n d M a r g i n .
texts.Rhythmalwaysinvolves
Thesamephenomenon
occursin temporallyintegrated
of salience(stressed
sensations
cycleswhichconsistof an alternationbetweensuccessive
(
u
n
s
t
r
e
s
s
e
d
s y l l a b l e su,n a c c e n t endo t e s )
s y l l a b i e sa,c c e n t e nd o t e se, t c . )a n d n o n - s a l i e n c e
as equaleven
andthesecyclesrepeatthemselves
with the time intervalsthat are perceived
in speechas in music,
when,measured
objectively,they
of salience,
are not.Theperception
resultsfrom a complexinterplaybetween
a numberof auditoryfactors:the durationof the
s t r o n ga n dw e a ke l e m e n tosf t h ec y c l e( ' l o n g ' - ' s h o r t ' ) , t hpei t c ho f t h es t r o n ga n dt h ew e a k
h l s ot h e v o w e cl o l o u r
e l e m e n t(s' h i g h ' - ' l o w 't)h e i r l o u d n e s(s' l o u d ' - ' s o f t ' )a,n d i n s p e e c a
(vowelsmay be fully pronounced,
or pronounced
as a
for instancethe first'e' in element,
' s c h w a 'l,i k et h es e c o n d ' e ' i ne l e m e not ,r t h es e c o n d ' a ' i na l a b a s t e rI)n. d e e da n y t h i ntgh a t
can createan auditorycontrastbetweensuccessive
soundscan serveto realizesalience.
And evenwhenobjectivecluesfor salienceare absent,the first elementof eachcyclecan
b e p e r c e i v eads ' s t r o n g e rp' :e r c e p t i oi n
m p o s ersh y t h mw
, a v e so f s a l i e n caen dn o n - s a l i e n c e
t h, e r ei s n o n e .
o n s o u n d( a n do n m o v e m e net )v e nw h e n s, t r i c t l ys p e a k i n g

202

The meaning of composition

W h e nc o m p o s i t i oins t h e i n t e g r a t i om
n o d e s, a l i e n c e
i s j u d g e do n t h e b a s i so f v i s u a l
c l u e sT. h ev i e w e r o
s f s p a t i acl o m p o s i t i o nasr e i n t u i t i v e layb l et o j u d g et h e ' w e i g h to' f t h e
v a r i o u es l e m e n tosf a c o m p o s i t i oann, dt h eg r e a t etrh ew e i g h o
t f a n e l e m e n t , t hger e a t eirt s
s a l i e n c eT. h i s s a l i e n c ea,g a i n ,i s n o t o b j e c t i v e lm
y e a s u r a b l be u, t r e s u l t sf r o m c o m p l e x
interaction,
a complextrading-offrelationship
betweena numberof factors:size,sharpnessof focus,tonal contrast(areasof hightonal contrast- for instance,
bordersbetween
blackandwhite- havehighsalience),
colourcontrasts(for instance,
the contrastbetween
s t r o n g l sy a t u r a t eadn d' s o f t ' c o l o u r so/r t h e c o n t r a sbt e t w e e rne da n db l u e ) p
, l a c e m e ni nt
(
e
l
e
m
e
n
t
s
t h ev i s u afl i e l d
n o t o n l yb e c o m e ' h e a v i e r 'tahse ya r em o v e dt o w a r d st h et o p ,b u t
alsoappear'heavier'the
furthertheyare movedtowardsthe left,dueto an asymmetryin
t h ev i s u afli e l d )p, e r s p e c t i v( feo r e g r o u nodb j e c t a
s r em o r es a l i e n t t h abna c k g r o u nodb j e c t s ,
a n de l e m e n t sh a t o v e r l a p
o t h e re l e m e n tasr em o r es a l i e ntth a nt h ee l e m e n t sh e yo v e r l a p ) ,
and also quite specificculturalfactors,suchas the appearance
of a humanfigure or a
potentculturalsymbol.And,just as rhythmcreatesa hierarchyof importanceamongthe
elements
of temporallyintegrated
texts,sovisualweightcreatesa hierarchy
of importance
amongthe elements
of spatiallyintegrated
texts,causingsometo draw moreattentionto
t h e m s e l v et hsa no t h e r s .
B e i n ga b l et o j u d g et h e v i s u a w
t f t h e e l e m e n tosf a c o m p o s i t i oi n
l eigho
s b e i n ga b l et o
j u d g eh o wt h e y ' b a l a n c eT' .h ew e i g h t h e yp u t i n t ot h e s c a l e d
s e r i v efsr o m o n eo r m o r eo f
the factorsjust mentioned.
Takentogether,
the elementscreatea balancingcentre,the
p o i n t o, n em i g h ts a yf,r o mw h i c h ,i f o n ec o n c e i v eodf t h ee l e m e n tass p a r t o f a m o b i l et,h i s
m o b i l ew o u l dh a v et o b es u s p e n d eRd e. g a r d l e o
s sf w h e t h etrh i sp o i n ti s i n t h ea c t u a cl e n t r e
of the composition
or off-centre,it often becomes
the spaceof the centralmessage,
and
this atteststo the 'powerof the centre'(Arnheim,1982) to whichwe havealludedalready,
a powerwhichexertsitselfevenif the Centreis an emptyspacearoundwhichthe text is
- cf. Barthes'remarksaboutthe 'emptyheartof fokyo' (I970:44).
organized
produces
Perspective
centresof its own,and by doingso contributes
to the hierarchizat i o n o f t h ee l e m e n tisn c o m p o s i t i o nAss. a r e s u l vt i e w e r m
s a yr e l a t et o c o m p o s i t i o ni nst w o
ways:perspectivally,
in whichcasethe composition
is ostensibly
basedon the viewer'sperspective/position;
non-perspectivally,
or
in whichcasethe composition
is not basedon the
viewer'sposition/perspective.
In the formercasethe viewers,
face-to-face
with the infinite
recessof perspective,
becomethemselves
the centreof the composition,
thus takingthe
p l a c eo f ,f o r e x a m p l et h, e d e i t i e si n B y z a n t i noer B u d d h i spt a i n t i n g sI n. t h e l a t t e rc a s et h e
representation
is codedfrom an internalpointof view,as is borneout by the fact that what
is left and what is right is judgedfrom the point of viewof the represented
participants
(1975: 33-9) hasdocumented
ratherthan from the pointof viewof the viewer.Uspensky
t h i sw i t h r e s p e ct to i c o n - p a i n t i nH
ge
. c i t e st r a d i t i o n agl u i d e fso r i c o n - p a i n t ewr sh i c hs t a t e ,
f o r i n s t a n c e , ' 0 tnh e r i g h t ,o r g o o ds i d e ,i s M o u n tS i n a i ,o n t h e l e f t ,o r b a d s i d e ,M o u n t
Lebanon',
andthenshowshow,from the viewer'spointof view,MountSinaiis on the right
and Mount Lebanonon the left. He addsthat this is a generalfeatureof pre-Renaissance
art, andalsoof primitivecartographic
drawing.
In the theoryof art, compositjon
is oftentalkedabout in aestheticand formal terms
( ' b a l a n c e ' , ' h a r m o neyt'c, . ) .I n t h e p r a c t i c eo f n e w s p a p earn d m a g a z i n lea y o u ti t i s m o r e

The meaning of composition

. our view
n p r a g m a t i ct e r m s( d o e si t ' g r a b t h e r e a d e r sa/ t t e n t i o n ' ? )I n
o f t e nd i s c u s s ei d
intertwinedwith the semioticfunctionof composition.
thesetwo aspectsare inextricably
(e.g.plate 2) the top section,the
As we haveseen,in many magazineadvertisements
'promiseof the product',is the mostsalientelementdueto its size.Thissuggests,
not just
attemptto makereadersnoticethe attractivepicturefirst, so
that suchadvertisements
to
in importanceand opposed
as to 'hook'them,but alsothat Idealand Realare ranl<ed
andof feeling,
is notjust a matterof formalaesthetics
eachotherin this way.Composition
el ements
s eaningfu
o r o f p u i l i n gt h e r e a d e r (sa l t h o u g iht i st h a t a sw e l l ) ;i t a l s om a r s h a lm
i n t o c o h e r e nt te x t sa n d i t d o e st h i s i n w a y sw h i c ht h e m s e l v ef os l l o wt h e r e q u i r e m e not sf
producemeaning.
mode-specific
structuresandthemselves
Rhythmand balancealsoform the mostbodilyaspectsof texts,the interfacebetween
o u r p h y s i c aal n d s e m i o t i cs e l v e sW
. i t h o u tr h y t h ma n d b a l a n c ep,h y s i c acl o o r d i n a t i oinn
eh
. e yf o r m a n i n d i s p e n s a bml ea t r i xf o r t h e p r o d u c t i oann d
t i m e a n d s p a c ei s i m p o s s i b lT
it is to quite some
receptionof messages
and are vital in humaninteraction.Moreover,
balancethat our aesthetic
degreefrom the senseof rhythmandthe senseof compositional
pleasurein textsandour affectiverelationsto textsare derived.

FRAMING
s f r a m i n gI.n t e m p o r a l l iyn t e g r a t etde x t sf r a m i n gi s ,
T h et h i r d k e ye l e m e nitn c o m p o s i t i oi n
a g a i n ,b r o u g h ta b o u tb y r h y t h m .F r o mt i m e t o t i m e t h e o n g o i n ge q u a l - t i m ecdy c l e so f
a changeof gait,andthese
a rallentando,
interrupted
by a pause,
rhythmare momentarily
j u n c t u r e sm a r k o f f d i s t i n c tu n i t s d
o r m u s i co r m o v e m e n t
, i s c o n n e cstt r e t c h eosf s p e e c h
from each other to a greateror lesserdegree.Where such juncturesare absent,the
e l e m e n t as r e c o n n e c t eidn a c o n t i n u o ufsf o w .I n s p a t i a l l yi n t e g r a t ecdo m p o s i t i o ni ts i s
markedoff
no different.The elementsor groupsof elementsare either disconnected,
joinedtogether.And visualframing,too, is a matter of
from each other,or connected,
y r w e a k l yf r a m e d .
d e g r e ee:l e m e n tosf t h ec o m p o s i t i om
n a yb es t r o n g l o
as a separateunit of
the moreit is presented
Thestrongerthe framingof an element,
i n f o r m a t i o nC. o n t e xtth e n c o l o u r si n t h e m o r e p r e c i s en a t u r eo f t h i s ' s e p a r a t i o nT' .h e
may be shownin a groupportrait (as in groupphotos
membersof a group,for instance,
, arked
o f s c h o ocl l a s s eosr e m p l o y e eosf a c o m p a n yo) r i n a c o l l a g eo f i n d i v i d u aplh o t o sm
off by frame linesand/oremptyspacebetweenthem (as with photosof the managers
of a companyin a companybrochure).The absenceof framingstressesgroupidentity,
i t s p r e s e n csei g n i f i e si n d i v i d u a l i tayn d d i f f e r e n t i a t i o Inn. f i g u r e6 . 1 , f r a m i n ga c q u i r e s
the lightand
The left postof the doorandthe dividinglinebetween
dramaticsignificance.
separates
dark boardson the roof createa frame line which,literallyand figuratively,
a pb e t w e etnh e m . l nf i l m a n dv i d e oa
M i n u sf r o m h i ss i s t e re,x p r e s s i nt hgec o m m u n i c a t i g
ve
showingtwo or moreactorstogetherin
similareffectcanbecreatedby the choicebetween
i n d i v i d u aslh o t so f t h e a c t o r si n w h i c he a c hi s i s o l a t e fdr o m
o n es h o t ,o r e d i t i n gb e t w e e n
t h e o t h e r sb y f r a m el i n e s .
the more they are
are connected,
The more the elementsof the spatialcomposition

204 .

The meaning of composition

p r e s e n t eads b e l o n g i ntgo g e t h e ra,s a s i n g l eu n i t o f i n f o r m a t i o nI n. t h e N o k i at r i p t y c h

referredto above,for instance,
thereare no frame linesto demarcate
the elements
of the
triptychstronglyfrom eachother.Thereis a senseof continuous
ffowfrom leftto right.But
in figure6.16 the'panels'of the triptychare separateunits- thereis a sharpdemarcation
herebetweenpast and present.The sameappliesto figure6.17, wherethe emptyspace
between
the top andthe central'panel'andthe colourcontrast(thetop panelis in colour,
the middlepanelin blackand white) createa strongdivisionbetweenthe Ideal and the
realityof immigration.
Theexamplealsoillustrates
the manywaysin whichframingcanbe
- by actualframe lines,by white spacebetweenelements,
achieved
by discontinuities
of
c o l o u ra. n ds o o n .
Connectedness,
too,can be realizedin manyways.It can be emphasized
by vectors,by
(structuralelements
perspectivally
depictedelements
of buildings,
drawnroadsleadingthe
eyeto elementsin the background,
etc.) or by abstractgraphicelements,
leadingthe eye
from oneelementto another,
beginning
with the mostsalientelement,
theelementthat first
drawsthe viewer'sattention.In figure6.2 thetilting of the left-handphotoformsa vector
leadingthe eyefrom left to right,andthe repetitionof the colourgoldin all the elements
of
- visual'rhymes'of this kind,
the two pagesprividesa strongsenseof unityand cohesion
repetitionof coloursand shapesin differentelementsof the composition,
form another
keyconnection
device,often usedin advertisements
to stressthe connection
betweenthe
'promise
o f t h ep r o d u c ta' n dt h e p r o d u citt s e l f( c f .a l s ot h ec o l o u r - c o o r d i n a t i n
otnh e S o n y
homepage).
It shouldfinallybe notedthat, at a deeperlevel,thereis alsoan elementof framingin
stylesof drawingand painting.In linedrawings,
for instance,
the outlinesof objectsstrictly
demarcate
themfrom theirenvironment,
whereasin certainstylesof painting(e.9.Impress i o n i s mt)h e ya r es e ta p a r tf r o mt h e i re n v i r o n m e o
nn
t l yb y s u b t l et r a n s i t i o nosf c o l o u r .

L I N E A RA N D N O N - L I N E A RC O M P O S I T I O N S
In denselyprintedpagesof text, readingis linearand strictlycoded.Suchtexts must be
readthe way theyare designed
to be read- from left to rightandfrom top to bottom,line
by line.Any otherform of reading(skipping,lookingat the last pageto seehow the plot
w i l l b e r e s o l v eodr w h a t t h ec o n c l u s i owni l l b e )i s a f o r mo f c h e a t i n a
g n dp r o d u c eass l i g h t
s e n s eo f g u i l ti n t h e r e a d e r . O t h keirn d so f p a g e s( e . g . t r a d i t i o ncaol m i cs t r i p s )a n d i m a g e s
( e . g . t i m e l i ndei a g r a m sa)r ea l s od e s i g n etdo b e r e a di n t h i s l i n e a w
r ay.
The pageswe havedescribed
in this chapterare readdifferently- and can be readin
morethan oneway.Their readingpath is lessstrictlycoded.Readersof magazines,
for
instance,may ffick thoughthe magazine,
stoppingevery now and again to look at a
pictureor reada headline,
and perhapslaterreturningto someof the articleswhichdrew
their attention,andwebsites
are specifically
designed
to allowmultiplereadingpaths.Yet
in many pagescompositiondoesset up particularhierarchies
of the movementof the
hypothetical
readerwithin and acrosstheir differentelements.
Suchreadingpathsbegin
with the mostsalientelement,
andfrom theremoveto the nextmostsalientelement.
andso

The meaning of composition

on. Theirtrajectoriesare not necessarily

similarto that of the denselyprintedpagg leftr i g h t a n d t o p - b o t t o mb, u t m a y m o v ei n a c i r c l e a
, s i n f i g u r e6 . 2 , w h e r et h e g o l db e i n g
(somewhat
pouredisthe mostsalientelement,
because
reducedin
of its extremebrightness
photo
reproduction),
the
of the two gold-diggers
the next most salient,the headlinethe
third mostsalient,andthe text the nextmostsalient- but it may also be that the vector
formedby the tilting of the left photograph
leadsthe eyebackto the largerphoto,and so
o n ,i n c i r c u l a fra s h i o nW
. h e t h etrh e r e a d eor n l y ' r e a d s ' t hpeh o t o sa n dt h e h e a d l i n e
o ,r a l s o
part or all of the verbaltext,a complementarity,
a to and fro betweentext and image,is
guaranteed.
For anyonereaderthe photograph
or the headlinemayformthe startingpoint
of the reading.0ur assumption
is that the mostplausiblereadingpath is the onein which
readersbeginby glancingat the photos,andthenmakea newstart from leftto right,from
headlineto photo,after which,optionally,
they moveto the bodyof the verbaltext. Such
pagescan be 'scanned'or read,just as picturescan be takenin at a glanceor scrutinized
f o r t h e i re v e r yd e t a i l W
. e d e l i b e r a t em
l ya k ea m o d e sct l a i mh e r ea n d s p e a ko f t h e ' m o s t
plausible'reading
path,for thistypeof readingpathis not strictlycoded,not as mandatory,
as that of the denselyprintedpageor the conventional
comicstrip.Differentreadersmay
. i v e nt h a tw h a ti s m a d es a l i e nits c u l t u r a l l dy e t e r m i n em
f o l l o wd i f f e r e npt a t h sG
d ,e m b e rosf
differentculturalgroupings
are likelyto havedifferenthierarchies
of salience,
andperhaps
textsof this kindarethewaytheyareprecisely
to allowfor the possibility
of morethanone
readingpath,and hencefor the heterogeneity
anddiversityof their largereadership.
As non-linear
textsbecomemorecommon,evendenselyprintedpagesof text beginto
be readdifferently.
Thescientist,
readinga journalof organicchemistry,
will glanceat the
diagrammaticrepresentations
of organiccompounds
beforedecidingwhetheror not to
readthe paperorrwhen readingthat only onerat has beenusedin the experiment,
skip to
l , 9 9 4 ) .S t u d e n tps r e p a r i nfgo r t h e i re x a m sw i l l
f i n d o u t f i r s t w h y t h i sw a sd o n e( G l e d h i l 1
usethe indexof the textbookto find out and highlightthe passages
they need,ratherthan
readthetextbool<
from coverto cover.
Themorea text makesuseof subheadings,
emphatic
, o l dt y p e ,u n d e r l i n i n gn) u
d e v i c e s( i t a l i c s b
, mbered
l i n e so f t y p i c a le l e m e n t o
sr characteristicsof some phenomenon,
tables,diagramsand so on, the more likely it is to be
scanned
s k, i p - r e a d , ' u s erda't h e rt h a nr e a d :l i n e a r e a d i n gi s g r a d u a l l lyo s i n gg r o u n d .
We notedthat readingpathsmaybe circular,diagonal,
spiralling,
andso on.As soonas
this possibility
is openedup/as soonasthereis a choicebetween
differentlyshapedreading
paths,theseshapescan themselves
becomesourcesof meaning.If the readingpath is
circular,onereadsoutwards,in concentric
circles,from a centralmessage
whichformsthe
h e a r t s, o t o s p e a ko, f t h e c u l t u r a u
l niverse
I f. t h e r e a d i n gp a t hi s l i n e a ra n d h o r i z o n t ailt,
constitutes
a progression,
movinginexorably
forwardstowardsthe future (or backwards,
t o w a r d st h e ' o r i g i n ' o f a l l t h i n g s ) I. f i t i s v e r t i c a l a
, s e n s eo f h i e r a r c h yi s s i g n i f i e da,
movement
from thegeneralto the specific
to the 'footnote'.Theshape
, from the'headline'
of the readingpath itselfconveys
a significant
culturalmessage.
S i x t e e n t h - c e n t ubroy o k so f e m b l e m se x p l i c i t l yd e s c r i b etdh e m e a n i n gos f d i f f e r e n t
pathT
. h er e a d i n g
p a t ho f f i g u r e6 . 1 9 ,a n i l l u s t r a t i ofnr o ma F l e m i s h
k i n d so f r e a d i n g
book
o f e m b l e m si s, a s p i r a lw
, h i c hw a sa n e m b l e mf o r t h e i n e x o r a b p
l er o g r e sosf t i m e .I t i s a l s o
a serpent,so that the readingproceeds
from the tail, a [ow,baseelement,to the head,a

The meaning of composition

sff{F{.:: -:f

\.

t}.: t:

L'r.:"

d.d

.::::

t r 5 ; ' . , ' r | l a : : " r " r ' "' v : ! i i ' $ r

{ i r ' ; r" " ' ! . ' 1 "

C)

fis O,rs A pagefrom Alciato'sBook of Emhtems(fromBassy,1975)

s u p e r i oer l e m e nA
t .l a i n - M a r i eB a s s y( 1 9 7 5 :3 0 3 - 5 )e x p l a i ntsh es e q u e n coef t h ee m b l e m as one followsthe spiral from the centre
atically expressed
meaningsone encounters
(
'
w
o
r
k
'
)
,
(
'
i
n
t
e
l l i g e n c et'h) ,e t a i l o f t h e s e r p e n (t a ' b a s e '
the head
o u t w a r d st:h e h a n d
e l e m e n t t)h, e h a n dw h i c hh o l d sd o w na n d i m p o s eist s w i l l o n t h e t a i l .J o i n i n gt h e s em e a n i n g st o g e t h erre s u l t si n t h ev i s u a pl r o p o s i t i oanl s oe x p r e s s ei ndt h et i t l e o f t h e p i c t u r e : ' E x
I gainedimmorendeavours
literatumstudiisinmortalitemacquiri'('Throughintellectual
paths
hasbarelybegun.
tality').Today,
ihe studyof the meaningof newkindsof reading
paths
we foundthat someare easyto agreeon,others
Analysingreading
with students,
This was not, we think, becauseof a lack of analytical
harder,againothersimpossible.
but because
of the structureof the texts
abilityon our part or on the part of our students,
themselves.
Textsencodereadingpathsto differentdegrees.Some,though no longer
denselyprintedpages,still take the readersby the hand,guidingthem firmly throughthe
(wemightcallthem'semi-linear'texts)
with a few hints
at bestprovidereaders
text.Others
In againotherswe
andsuggestions,
andfor the rest leavethe readersto their owndevices.
than
can,with the bestwill in the world,not detectany readingpaththat is moreplausible
any numberof others.In figure 6.20, a comic strip from the magazineCracked,the
headlinestandsout and this, togetherwith the strongvectorformedby the waterslide

@*:*

ffi

*ffi

#ffi
ffi

@
ffiffi
ffi#
Wffi

ffi*
ffie

ffi
ffi

M
SWffi

tss
t a

1s

\s

1A
:r

I ir*

o
o

ffi

#ffi*
ffiffi
@

qf

g%

ffi
@

o
.!
i;

208 .

The meaning of composition

on the left page,predisposes

us to start our readingtop left. But wherethe eyewill move
f r o m h e r ei s d i f f i c u lt o p r e d i c tT. h e r ei s n e i t h ecr h r o n o l o g(yd e s p i tteh e r e s e m b l a n tcoea
f f o w c h a r tn) o r a c l e a rh i e r a r c hoyf s a l i e n c e .
Increasingly
many texts (newspapers,
billboards,comic strips,advertisements,
websites)areof this kind.Theyofferthe readera choiceof readingpathand,evenmoresothan
in the caseof textswherea plausible
readingcanbediscerned,
leaveit up to the readerhow
to traversethetextualspace.Theyare'interactive'-andit is perhapsno accidentthat they
h a v et h e i rc l e a r e sat n t e c e d e ni nt t h e g e n r eo f t h e ' a c t i v i t i e sb' o o k sw h i c ho f f e rc h i l d r e n
a
c h o i c eo f p u z z l e sr i,d d l e sc,o l o u r i n g - p
i ni c t u r e se,t c .f o r a r a i n yd a yd u r i n gt h eh o l i d a y sT.h i s
is not to saythat the orderof the elements
on suchpagesis random.Thecomicstrip,for
instance,
still hasits 'welcome'signat top left,and its mostgruesome
imagesin the Real,a
division
b e t w e edne p i c t i o nosf h o l i d a yf u n a n d o f s a d i s t i tco r t u r et h a t r e c a l l st h e d i v i s i o n
between
the Gardenof Edenand Deathin figure6.8.
Lineartexts,then,are like movies,wherethe viewershaveno choicebut to seethe
i m a g e si n a n o r d e rt h a t h a s b e e nd e c i d e df o r t h e m ,o r l i k e a n e x h i b i t i o ni n w h i c ht h e
p a i n t i n gasr eh u n gi n l o n gc o r r i d o r tsh r o u g hw h i c ht h ev i s i t o r sm u s tm o v ef,o l l o w i n g
signs
perhaps,
to eventually
end up at the exit.In non-lineartextsviewerscan selecttheir own
i m a g e sa n dv i e wt h e m i n a n o r d e ro f t h e i r o w nc h o o s i n gT.h e ya r e l i k ea n e x h i b i t i o inn a
large room which visitorscan traversein any way they like. But, again,the way these
e x h i b i t sa r e a r r a n g e w
d i l l n o t b e r a n d o mI.t w i l l n o t b e r a n d o mt h a t a p a r t i c u l a m
r ajor
s c u l p t u r ies p l a c e di n t h e c e n t r eo f t h e r o o m ,o r t h a t a p a r t i c u l am
r a j o rp a i n t i n gh a sb e e n
hungon the walI opposite
the entrance,
to be noticedfirst by alI visitorsenteringthe room.
Lineartextsthusimposea syntagmatics
on the reader,
describe
the sequence
of,andthe
c o n n e c t i obne t w e e nt h, e e l e m e n t sA.s a r e s u l t h e m e a n i n gosf i n d i v i d u aell e m e n tcsa n b e
l e s ss t r i c t l yc o d e da, s f o r i n s t a n c e
i n d o c u m e n t a rf yi l m s ,w h e r et h e m e a n i n g
of the individualshotscan be largelydetermined
by the editing, ratherthan by the intrinsicmeanings
of the shots.Non-linear
textsimposea paradigmatics.
Theyselectthe elements
that canbe
viewed
a n dp r e s e nt th e ma c c o r d i ntgo a c e r t a i np a r a d i g m a tlioc g i c- t h eI o g i co f C e n t r ea n d
- but leaveit to the readerto sequence
Marginor of Givenand New,for instance
andconn e c t h e m .I n t h e d e s i g no f s u c ht e x t st h e r ew i l l b e p r e s s u rteo p u t m o r eo f t h e m e a n i n ign
t h e i n d i v i d u aell e m e n tosf t h ec o m p o s i t i otno,u s em o r eh i g h l yc o d e di m a g e-s s y m b o l iacn d
conceptual
images,
tightlywritten,self-contained
itemsof information,
stereotyped
characters,drawingsor highlystructuredimagesratherthan realisticphotographs,
and so on.
Linearandnon-linear
textsthusconstitute
two modesof readingandtwo regimes
of control
overmeaning,
exactlyin the sameway as we discussed
in chapterl, in connection
with
Baby'sFirst Book(figure1.1) andthe pagefrom Dick Bruna's0n My Walk(figure1.2).

A SUMMARY
F i g u r e6 . 2 I p r o v i d eas s u m m a r yo f t h e d i s t i n c t i o nwse h a v ei n t r o d u c eidn t h i s c h a p t e r .
The double-headed
arrows(f) stand for gradedcontrasts('more or less,,ratherthan
' e i t h e r - o r 'T
) .h es u p e r s c r i p t ' l ' m e a n s ' i f ' a tnhde
s u p e r s c r i p t ' T ' m e a n s , t h Ienno, .t h e r

The meaning of composition

. 209

w o r d s \/ i f t h e r ei s n o h o r i z o n t apl o l a r i z a t i o tnh, e nt h e r em u s tb e v e r t i c a p
l olarization't h e o p p o s i tfeo l l o w sf r o mt h i s .I n t h e n e x ts e c t i o n
w e w i l l d i s c u sas n u m b e or f e x a m p l eisn
greaterdetail.

REALIZATIONS
Centred
Polarized
Triptych

Circular

Margin

Mediator

Given

New

Ideal

Real

An element(theCentre)is placedin the centre

o f t h ec o m o o s i t i o n .
Thereis no elementin the centreof the
composition.
Thenon-central
elements
in a centred
c o m p o s i t i oanr ep l a c e de i t h e ro n t h e r i g h ta n d
left or aboveand belowthe Centre.
Thenon-central
elements
in a centred
c o m p o s i t i oanr ep l a c e db o t ha b o v ea n db e l o w
andto the sidesof the Centre,andfurther
elements
may be placedin between
these
p o l a r i z e pd o s i t i o n s .
Thenon-central
elements
in a cenrreo
c o m p o s i t i oanr e i d e n t i c aol r n e a r - i d e n t i c saol ,
c r e a t i n gs y m m e t r iyn t h ec o m p o s i t i o n .
TheCentreof a polarizedcentredcomposition
formsa bridgebetweenGivenand Newand/or
I d e a la n d R e a ls, o r e c o n c i l i npgo l a r i z e d
elements
to eachotherin someway.
T h el e f te l e m e nitn a p o l a r i z ecdo m p o s i t i oonr
the left polarizedelementin a centred
c o m p o s i t i oTnh. i se l e m e nits n o t i d e n t i c aol r
n e a r - i d e n t i ctaolt h ec o r r e s p o n d i rnig h t
element.
T h er i g h te l e m e nitn a p o l a r i z ecdo m p o s i t i o n
o r t h e r i g h tp o l a r i z eedl e m e nitn a c e n t r e d
c o m p o s i t i oTnh. i se l e m e nits n o t i d e n t i c aol r
n e a r - i d e n t i ctaolt h ec o r r e s p o n d i lnegf t
element.
T h et o p e l e m e nitn a p o l a r i z ecdo m p o s i t i oonr
t h et o p p o l a r i z eedl e m e nitn a c e n t r e d
c o m p o s i t i oTn h. i se l e m e nits n o t i d e n t i c aol r
near-identical
to the corresponding
bottom
element.
Thebottomelementin a polarizeo
c o m p o s i t i oonr t h e b o t t o mp o l a r i z eedl e m e n t

2to

The meaning of composition

Salience

Disconnection

Connection

i n a c e n t r e dc o m p o s i t i oTn h. i se l e m e nits n o t
i d e n t i c aol r n e a r - i d e n t i ctaolt h e
c o r r e s p o n d itnogp e l e m e n t .
Thedegreeto whichan elementdraws
attentionto itself,dueto its size,its placein
or its overlapping
of other
the foreground
its colour,its tonalvalues,its
elements,
or definition,
and otherfeatures.
sharpness
t o w h i c ha n e l e m e nits v i s u a l l y
T h ed e g r e e
from otherelements
throughframe
separated
emptyspace
lines,pictorialframingdevices,
b e t w e eenl e m e n t d
s ,i s c o n t i n u i t ioefsc o l o u r
andshape,and otherfeatures.
t o w h i c ha n e l e m e nits v i s u a l l y
T h ed e g r e e
j o i n e dt o a n o t h eer l e m e ntth, r o u g ht h e
throughvectors
absence
of framingdevices,
a n dt h r o u g hc o n t i n u i t i eosr s i m i l a r i t i eosf
c o l o u rv i s u asl h a p ee, t c .

|
|
Centred {
I

------l

Circular

-----l

|
(

TriPtYch
Centre-Margin

f
-

M e d iaattoorr--PPooIal ar i z e d
e l e m e n t s-

P oIa r iz e d

Composition

M a x i m u ms a l i e n c e

Y
M i n i m u ms a l i e n c e
Maximum disconnection

Y
Maximum connection

fis O.Zf The meaningot composition

The meaning of composition

2r)

G I V E NA N D N E W I N C H I L D R E N ' SD R A W I N G SA N D C D - R O M s
In anysequential
structure,
that whichis aboutto be saidor shownis by definitionalways
New,not yet known.By contrast,what has (just) beenseen,heard,discovered
is, by
comparison,
nowknown,Given.In visualmedia,sequence
in a
canof coursebe represented
numberof dimensions,
right to left, bottomto top, in a spiralfrom outside,etc. (and in
medievalpaintingperspective
can indicatesequence,
with the foregroundas the present
a n d t h e b a c k g r o u nads t h e f u t u r e ) .S u c hd i m e n s i o nhsa v eb e e nu s e dt h r o u g h o uht i s t o r y ,
a n d a r e s t i l l u s e db y d i f f e r e n ct u l t u r e sa, s p r i m a r yv i s u a ls e q u e n c i nogr i e n t a t i o n T
s .h e
m e d i u mo f t h e b o o k ,b r i n g i n g
t h e p o s s i b i l i toyf t u r n i n gt h e p a g ea, d d sa f u r t h e rm e a n so f
reprintingsequence
visually,the left page/rightpagestructureand the possibilityof the
two-pagestructures(rightpageandfollowingleft page).
F i g u r e6 . 2 2 s h o w sa d o u b l ep a g ef r o m a b o o kp r o d u c e b
d y a s i x - y e a r - o lbdo yw h i l e
stayingin Parisfor half a yearwith his parents.It recordseventsand experiences
he was
i n v o l v e idn ,a n ds i g h t sa n d o b j e c t sh e e n c o u n t e r eddu r i n gh i ss t a yi n P a r i sC
. l e a r l yi n
, this
situationeverything
was Newfor the child,literally.Hewasfacedwith the questionof how
to representnew information,new ideas,new objects,without the possibiiityof relating
t h e mt o a l r e a d y - e s t a b l i s hkendo,w nd o m a i n s .
The book openswith the nameand addressof the author,on the first lefthandpage.
T h i si s t h e G i v e nf o r t h e b o o ka s a w h o l e a
, n e l e m e not f s e c u r i t ya n d f a m i l i a r i t yi n t h e
n e w e n v i r o n m e n t . 0 nt h e f i r s t r i g h t h a n dp a g et h a t n e w e n v i r o n m e ni ts r e p r e s e n t e d
visually:a pictureof the EiffelTower.
It is onlywhenthis pageis turnedthat the pictureis
named,commutedinto language.
Oncenamed,the EiffelTowerbecomes
Given,andon the
adjoiningrighthandpagethe childfacesthe nextaspectof his newenvironment.
Thusthe
bookcontinues:
the newpicture,too,is only identifiedon the nextlefthandpage- the Arc
d e T r i o m p h eT.h ec h i l do b v i o u s lrye a l i z e tdh a t t h i ss t r u c t u r e
c o u l db e m i s u n d e r s t o oadn,d
usedleft-facingarrowsto referthe readerto the pictureon the previouspage.But his
impulsewas to first visuallyrepresentParisas the New,and then to masterit, make it
k n o w na n dG i v e nb y m e a n so f l a n g u a g b
e y, m e a n so f n a m i n gi t . H i sa t t i t u d ew a se m p i r i c a l
and he used languageas an 'anchorage'in his effort to come to terms with his new
experiences.
We will end with an examplethat bringsall the elementsof this chaptertogether.
F i g u r e6 . 2 3 s h o w st h e f i r s t s c r e e no f a n ' e d u t a i n m e n t ' C D - R 0fM
o r c h i l d r e nt ,i t l e d ' 3 D
BodyAdventure'.Thetop of the screenshowsa rangeof mediaon a desktop.A slideis
p r o j e c t e od n a s c r e e nA. v i d e om o n i t o rs h o w sa n a n i m a t e d
sequencH
e .a l f - h i d d ebne h i n d
playssoft music.In otherwords,the ldealhereis whatwe might
the monitor,a loudspeaker
n e d i a 'm
c a l l ' i n f o r m a t i om
, e d i a t or e a d l, o o ka t a n d l i s t e nt o . T h eR e a lo, n t h e o t h e rh a n d ,
presents
thingsthe usercan do.It of{ersgamesto play,mediato interactively
engagewith.
' E m e r g e n c yf o
' , r i n s t a n c ei s, a g a m ew h i c hm i x e sl a s e r
s u r g e r ya n dt h e s h o o t i n g a l l e r ythe playerzapsbraincellsin a raceagainsttime ('Hurry doctor,savethe patient').And
i n ' B o d y R e c a l l ' b o d yp a r t s m u s t b e m a t c h e dw i t h t h e i r n a m e sT. h u st h e c o m p o s i t i o n
of the screenusesthe vertical dimensionto separateinformation-as-knowledge
from
information-as-action,
or information-as-knowledge
from information-as-entertainment.

2I2

The meaning of composition

.:r.z*r

\{'
r'{sr$r

l,

,iL **r,.f

"1rgc ie Tnr*flptjg

*Xt/

figO.ZZ Parisdiaryof a six-year-old

boy

on a pedestal/
it places
And,whileit continues
to put the former,literallyand figuratively,
activities.
We mightsaythat'entertaining
real learningsquarelyin the zoneof interactive
a c t i v i t i e s ' a rhee r er e p r e s e n t ea ds ' c o n s o l i d a t i n(gg'i v i n ga f i r m ' f o o t i n g ' o r ' g r o u n d i n g ' t o )
presented,'high'
knowledge.
Reversing
authoritatively
the two - puttingthe gameson top

Themeaning of composition

fig O,Zf Pagefrom 3D BodyAdventure(KnowledqeAdventures,lgg3)

andthe informationmediaat the bottom- wouldcreatea verydifferentmeaning,

perhaps
somethinglike 'knowledgeprovidesa "foundation" fer ("highly,, regarded)active
exoeriences'.
The screenalso usesthe horizontaldimension,
and this in two ways.First,the left is
t h e d o m a i no f t h e s t i l l i m a g ea, n d t h e r i g h tt h e d o m a i no f t h e a n i m a t e d ' 3 Di,m a g e so, f
the movefrom two-dimensional
representation
to 'virtual reality'.Second,
the left is the
domainof what hasalreadybeenformulatedfor the users,whilethe right is the domain
of what userscan do themselves:
they can rotatethe skeletonwith their mouseso as to
viewthe imagefrom whichever
angletheychoose,
andtheycanexitthe screenat will. Note
that the monitorstraddlesthe boundarybetweenIdeal and Real:like interactive
games,
user-activated
3D viewinghas(still) someentertainment
value,because
of its novelty;but,
like informationmedia,it alsohas instructionvalue- the animatedskeletoncan serveas
a stand-infor a real or reproductionskeletonand makea goodlearningaidfor students.In
otherwords,aswe movefrom leftto right,we movefrom the traditional2 D diagramto the
newanimated3 D diagramor drawing,andf romthetraditional'passivelearner'to the new
'interactive'modeof learning.
A n o t h ed
r i m e n s i ouns e dh e r ei s t h a t o f f o r e g r o u nadn db a c k g r o u nT
dh
. el o u d s p e a kiesr
placedbehindthe monitor,which is congruent
with the role playedby soundand imagein

213

2I4

The meaning of composition

is provided
v i s u a l l ya, n d t h e s o u n d t r a cok n l y o f f e r ss o f t
t h i s C D - R 0 M :a l l i n f o r m a t i o n
b a c k g r o u nm
du s i c .
andthisfor two
M ostsalienton the screenis the monitorimageof the movingskeleton,
the greatesttonalcontrast.Nextmostsalientare perhaps
it moves,
reasons:
and it displays
the namesof the qames.Althoughthey do not occupymuchspace,their colours- bright
red andyellow- contraststronglywith the coolwhites,bluesand greysthat dominatethe
r e s to f t h es c r e e nA.n dt h e i m a g e (st h ed o o r so f t h e E m e r g e n cWy a r da n dt h e ' B o d yR e c a l l '
keyboard)are both sharperand more saturatedin colourthan the rest of the screen.
a ,n da s a r e s u l t h e ' s l i d e ' w i t ht h e X - R a yp i c t u r e
R e l a t i vsei z ec a na l s oe s t a b l i ssha l i e n c e
t. hich
o f t h e b o d ya n dt h e t i t l e o f t h e C D - R 0 Mi s p e r h a ptsh e n e x tm o s ts a l i e net l e m e n W
l e a v etsh e l o u d s p e a kaenr dt h e ' e x i t ' s i g n .
that
Fromthe point of viewof lraming,finally,the mostsignificant'disconnection'is
gamesandthe restof the screen.
Thegames,
againsta
between
the spaceof the interactive
insertthemselves
into
on the screen,
brighter,
moregarishbluethancanbefoundelsewhere
the more traditional,naturalisticcontinuity(and naturalpalette)of the desktop.They
as a quite
couldhavebeenplacedon the desktop.But they are not.Theyare represented
l e r s p e c t i vhaol m o g e n e iot yf t h e s e m i o t i c
s e p a r a t e / ' a l i e n ' e l e m ed ni st r, u p t i ntgh e n a t u r a p
space.Withinthe pictureof the desktopon the otherhand,thereis a senseof continuity,
of the way the elements
both because
of the harmonyof the mutedcolours,and because
space.Thus the traditional
non-fragmented
homogeneous,
form part of a continuous,
to each other,but also as
as naturalisticand complementary
media are represented
e 'e d i a .
r a d i c a l ld
y i f f e r e nftr o mt h e n e w ' i n t e r a c t i vm
the component
modes
of this screenpositions
Theexampleshowsthat the composition
role,some
of the multimodaltext in relationto eachother,makingsomeplaya foreground
someascomplementary
lo eachother,othersaseachother's
role,presenting
a background
and visuallydefines
of 'edutainment',
and so on.It visuallyrealizesa discourse
opposites/
its characteristicrelationsand values,and the part playedin it by differentsemiotic
modes.

7 M a t e r i a l i t ya n d m e a n i n g

M A T E R I A LP R O D U C T I O A
NS A S E M I O T I CR E S O U R C E
Thesemioticresources
we havediscussed
in this bookabstractawayfrom the materiality
of the signifier.
Theycanbeapplied,
we haveclaimed(andtriedto demonstrate
throughour
n n d u n d e r s t a n d i nogf v i s u a l sw h i c h ,m a t e r i a l l ya,r e q u i t e
e x a m p l e st )o t h e p r o d u c t i o a
differentfrom eachother:photographs,
movies,
websites,
drawings,paintings,
and so on.
O n eo f t h e m a j o rf e a t u r e-s e x p l i c i t layn di m p l i c i t l-y o f t h ed e v e l o p m eonfto u r i d e a s i n c e
we wrotethe first editionof this bookhasbeento paymoyeattentionto the semioticrole
o f t h e m a t e r i apl r o d u c t i oonf t h e s i g n .I n m u s i ct,h e p e r f o r m a n coef a c o m p o s i t i ocno n t r i b u t e sa g r e a td e a lt o i t s m e a n i n ga,n d i n m a n yc a s e si t i s d i f f i c u l ti,f n o t i m p o s s i b lteo,
s e p a r a t ceo m p o s i t i oann d p e r f o r m a n cIen. v i s u a lc o m m u n i c a t i osni m
, e material
, i l a r l yt h
productioo
n f a d e s i g ni s n o t j u s t t h e e x e c u t i o on f s o m e t h i nagl r e a d yc o m p l e t eb,u t a
v i t a l p a r t o f m e a n i n g - m a k i nHge. r ew e w i l l f o c u so n t h a t a s p e c to f s e m i o s i isn s o m e
more detail by lookingat the materialsusedin what, in the first editionof this bool<,
w e c a l l e d ' i n s c r i p t i o an n' ,d h a v es i n c ec o m et o c a l l ' p r o d u c t i o n( 'l ( r e s sa n dv a n L e e u w e n ,
2 0 0 1) .
W h e ns, o m et i m ei n 1 9 8 8 ,w ef i r s tp r e s e n t eodu r i n i t i a li d e a so nt h ev i s u atlo t h e S y d n e y
S e m i o t i cS
s a l o n ,o n e o f o u r f r i e n d ss a i d , ' B u tw h a t a b o u tb r u s h s t r o k e sH?o w c a n y o u
d e s c r i bb
er u s h s t r o k e
a s s e m i o t i cu n i t s ? ' 0 u rf u m b l e dr e s p o n swea st o s a y , ' Y o uh a v et o
start somewhere.
We'll get to brushstrokes
later,whenwe are furtheron with our work.'
B u t t h e q u e s t i o snt a y e dw i t h u s ,a n dr e g u l a r l ky e p tc o m i n gu p i n o u r d i s c u s s i o nI tsw
. a sa
questionthat responded
to our view- then an odd one- that in a message
all aspects
matterand mean,andat the sametime showeda profoundscepticism
aboutthat assumption. However,
it is and remainsan importantquestion.Nearlytwenty years later,our
a n s w ew
r o u l di n p r i n c i p l e
b e t h e s a m e , t h o u gm
h a y b ea l i t t l e l e s sf u m b l e da n ds o m e w h a t
morethoughtthrough.In our 'grammar'of visualdesign,
we wantedto moveawayfrom a
t o t a l i z i n vg i e wo f s e m i o t i rce s o u r c eas v, i e wi n w h i c hs e m i o t i rce s o u r c easr e h o m o g e n e o u s
s y s t e misn w h i c ht h e r em a yb ed i f f e r e n c ei nst h e ' s i z e ' o u
f n i t sb, u t i n w h i c ha l l t h eu n i t sa r e
o f t h e s a m ek i n d ,a l l ' b e l o n g ' t o' t h es a m es y s t e m s' ,ot h a t a l l t e x t sa r e ,i n t h ee n d ,b u i l tu p
f r o m a s i n g l e k i n d o' m
f i n i m aul n i t ' , b e i t t h e b r u s h s t r o k e , t h e ' i c o n i c f i g u r e ' ( E c o , I 9 7 6 d ,
(
S
a
i
n
t - M a r t i n , J - 9 8o7r)t,h ep h o n e maen dm o r p h e m-e i n t h e i rr e s p e c t i v e
the'coloureme'
'tactic' arrangements.
By contrast,we wantedto maintainthat a givenform of semiosis
f o r i n s t a n c e , ' p a i n t i n gi n' -v o l v easr a n g eo f s i g n i f y i nrge s o u r c eSs o
. m eo f t h e s ea r el i k et h e
signifyingsystems
we havediscussed
in this book,resources
whichcan be used,not just in
p a i n t i n gb,u t a l s oi n p h o t o g r a p hoyr,i n d r a w i n gt ,o m e n t i o n
j u s ts o m ee x a m p l e A
s .n yg i v e n
typeof productionmediumcan,at leastin principle,realizemostof the choicesfrom the
ideational,
interpersonal
andtextualnetworkswe havepresented
in this book,thoughthere
are, in practice,historicallyand culturallyspecificrestrictionson the combinationof

216

M ater i a I ity and meani ng

f romthesesignifying
systems;
for example,
restrictions
on whatcanbe paintedand
choices
are morespecifically
tied to specificformsof material
how.But othersemioticresources
p r o d u c t i oann d c a n b e r e a l i z e df o
, r i n s t a n c eo,n l y i n t h e m e d i u mo f p a i n to r o n l y i n t h e
m e d i u mo f t h e p h o t o g r a p h .
pointof view.Materialitymatters:
In the realmof art this is a relativelyuncontentious
paintsoffer differentaffordances,
and hencedifferentpotentialsfor
oil- and water-based
with the
making meaning.The mannerof productionalso matters,as we discussed
t y m a ni n c h a p t e5r . l n t h e r e a l mo f l i n g u i s t i ci tsh a sb e e nl e s so b v i o u s .
e x a m p l eosf R o b e r R
simplequestion'What is a text?' or 'ls a writtentext the same
If we ask the seemingly
objector a differentonewhen it is writtenwith a pencilor with penand ink or is wordprocessed?/,
It is the sametext.'The
the answerof most linguistswould
be,'No question.
of thetext wouldnot beseenas a relevantissue.If we askeda
material,graphicexpression
non-linguist
the samequestion,
the answermightbe different- the teacherwho responds
(perhaps
badly
to an essaypresented
on scrappybitsof paper,
badlyhandwritten
negatively
but responds
favourably
to a'well-presented',
typedversionof the sametext,uses
spelled),
whenpresenting
a proposal
to a
a quitedifferentcriterion.So doesthe marketingexecutive
client.Theirnotionof whata text is differsfrom that of the linguist.Likeus,theywouldsee
'presentation'as
part of the makingof thetext,increasingly
oftenequalto, or
a significant
evenmore importantthan,otheraspects.For them,as for the painteror the viewerof a
painting,the mediumof inscription
changes
the text.
perhaps
that this aspectof text is rapidlygainingin importance,
It is our impression
in 'art'
of writing.Theboundaries
between
the criteriaprevailing
aidedby newtechnologies
writingare no longeras sharplydrawnas theyoncewere.
andthoseprevailingin everyday
here,and the linguistictheory
We do not want to engagein an argumentwith linguistics
oriented.But we do want
from whichwe draw muchinspirationis in anycasesemiotically
to saythat the linguisticnotionof text is an artefactof linguistictheory;as,indeed,is our
notionof text - whetherwrittenand linguisticor paintedandvisual,or both.Thequestion
of brushstrokes
comes,we think,out of a viewin whicheverything
aboutthe significance
representational
is seenas belonging
representational
to the sameunified,homogeneous
e r, p a i n t i n g )T. h eb o u n d a r i easr o u n dw h a t i s ' i n ' a n d w h a t i s ' o u t ' u s e d
s y s t e m( l a n g u a g o
was not
to be strictlypatrolled:in the linguistictrainingof oneof the authors,phonetics
part of linguistics,
knewwhat wasextra-,para-or simplynon-linguistic.
The
andeveryone
materialaspectsof handwriting
andtypography
werenot eventouchedupon.
In our approachthe materialexpression
of signs,and thereforeof the text, is always
e ra t e r i a l ' a to n e l e v e l a
, n d i t i s t h e r e f o r ea
s i g n i f i c a n itt; i s w h a t c o n s t i t u t e s ' s i g n i f im
Texts
of sign-(andthereforetext-) production.
crucialsemioticfeature.So is the process
and production
are materialobjectswhich result from a variety of representational
practicesthat makeuseof a varietyof signifierresources
organized
as signifyingsystems
(we havecalledthese'modes'),and a varielyof 'media',of 'signifiermaterials'- the
of producsurfacesof production(paper,rock,plastic,textile,wood,etc.),the substances
/ encils,
t i o n ( i n k ,g o l d ,p a i n t ,l i g h t ,e t c . )a n d t h e t o o l so f p r o d u c t i o (nc h i s e lp, e n /b r u s h p
s t y l u se, t c . ) .
Everyculturehassystemsof meanings
codedin thesematerialsand meansof produc-

Materiality and meaning . 2I7

t i o n .H e r ea, s i n a l l a r e a so f s e m i o s is i,g n si n t h e i rm a t e r i a l i tayr ef u l l y m o t i v a t e d , t h o u g h

asalwaysthe motivations
arethoseof a particularculturein a particularperiod,andthose
o f t h e m a k e ro f t h e s i g n t; h e ya r e n o tg l o b a ln, o r a r et h e ya - h i s t o r i c aPl .r e c i o um
s e t a l sa r e
precious
because
of theirscarcity,
andperhapsbecause
oftheir malleability.
But scarcityis
n o ta g l o b a l l yu n i f o r mc h a r a c t e r i s t a
i cn, dt h e p r e c i o u s n eosf so n em e t a ln e e dn o tb ee q u a l l y
m a r k e di n a n o t h ecr u l t u r eI.t w a s o n eo f t h ep a r t i c u l acr a l a m i t i eosf t h ec u l t u r e os f C e n t r a l
and South Americathat they had attacheddifferentsemioticvaluesto the material
s i g n i f i egr o l d f r o mt h o s eo f t h e i n v a d i nS
gpaniards.
We regardmaterialproductionas particularlysignificantbecauseoften it is in its
processes
that unsemioticized
materialityis drawninto semiosis.
At timesproductionis
thereforesomewhatlesssubjectto the variousformsof semioticpolicingthan are other
regiono
s f t h e s e m i o t i cl a n d s c a paen, dt h u s l e a v e sm o r er o o mf o r i n d i v i d u apl o s s i b i l i t i e s
of expression
than those regionswhich havebetter-known
cultural histories,are more
foregrounded
and havebetter-understood
To explorematerialproductionis
conventions.
thereforealsoto explorethe boundaries
and
betweenthe semioticand the non-semiotic,
betweenindividual
expression
and socialsemiosis.

P R O D U C T I OSNY S T E M SA N D T E C H N O L O G Y
L i k ea l l c u l t u r atl e c h n o l o g i ef os r, m so f p r o d u c t i oanr ee n t i r e l yr e l a t e dt o t h e o v e r a lsl t a t e
of a society's
technologies.
Indeed,dependence
may be oneof the strongest
on technology
f e a t u r e so f g r a p h i c al yl r e a l i z e ds e m i o t i c si t; d i s t i n g u i s h tehse mf r o m s e m i o t i cm o d e si n
whichsignsare articulatedby the bodywithout any technological
aids (as,for instance,
i n s p e e c hs ,i n g i n g , ' n o n - v e r cboam
l m u n i c a t i o dn a' ,n c e )M
. o d e sl i k em u s i cs t r a d d l e
t h et w o
yetthe boundaries
categories;
betweenthem are in any casealwaysfuzzy:
onecan draw
or write with one'sfingerin sand,usingonlythe bodyanda naturalsurface.But generally
the surfaces,substances
and tools of the visual semioticare made availableby technologies,as much in the caseof penciland paperas in the caseof the modernword
p r o c e s s oTr e. c h n o l o geyn t e r sf u n d a m e n t a l il n
y t ot h e s e m i o t i cp r o c e s st h: r o u g ht h e k i n d s
of meanswhichit facilitatesor favours,andthroughthe differentialaccess
to the meansof
p r o d u c t i oann dr e c e p t i ow
n h i c hi t p r o v i d e s .
W e d i s t i n g u i sthh r e em a j o rc l a s s eos f p r o d u c t i otne c h n o l o g i e(s1:) p r o d u c t i o inn t h e
narrowersense- that is,technologies
in which representations
of the hand,technologies
a r e ,i n a l l t h e i r a s p e c t sa,r t i c u l a t e b
d y t h e h u m a nh a n d a, i d e db y h a n d - h e ltdo o l ss u c ha s
c h i s e l sb,r u s h e sp,e n c i l se,t c . ;( 2 ) r e c o r d i ntge c h n o l o g i -e st h a t i s ,t e c h n o l o g i e
o sf t h e e y e
( a n de a r ) ,t e c h n o l o g i e
wsh i c ha l l o wm o r eo r l e s sa u t o m a t e ad n a l o g i c ar e
l p r e s e n t a t i o fn
photography
what they represent,
for instance,
audiotape,
and film; and (3) synthesizing
technologies
which allow the productionof digitailysynthesized
representations.
While
r e m a i n i ntgi e dt o t h e e y e( a n de a r ) ,t h e s er e i n t r o d u ct h
e e h u m a nh a n dv i a a t e c h n o l o g i c a l
' i n t e r f a c ea
' , t p r e s e nst t i l l i n t h e s h a p eo f a t o o l ( k e y b o a r dm, o u s e )t,h o u g hi n f u t u r e
p e r h a p isn c r e a s i n gtlhyr o u g hd i r e c ta r t i c u l a t i o b
spoken
n y t h e b o d y( e . g . t h r o u gihs s u i n g
c o m m a n dtso t h ec o m p u t eor ,r t h r o u g ho t h e rg e s t u r e s ) .

218

M ater i al ity and meani ng

The boundaries
betweenthesecategories
are not clear-cut;and are alwayssubjectto
furthertransformative
semioticwork.A photograph
onceit hasbeen
canbe hand-coloured
printed,for instance,
or digitallyaltered,and manyartistsexperiment
with precisely
these
m i x e dp r o d u c t i o n
s y s t e m sI .t s h o u l da l s o b e n o t e dt h a t t h e p o s s i b i l i toy f ' m e c h a n i c a l
reproduction',
to useBenjamin's
term,is not uniquelytied to any of the threecategories.
photographically,
or with a modernlaser
Printingcanbe donefrom a hand-carved
master,
printer.But we think the categories
are useful,particularlyas they can be tied to major
p e r i o d isn t h e h i s t o r yo f p r o d u c t i oann dt o t h ee p i s t e m o l o g it eh sa t w e n tw i t h t h e m .
- st e c h n o l o g i eosf t h e h a n d- h a v ec o n t i n u etdo p l a ya
W h i l ep r o d u c t i otne c h n o l o g i e
r o l e ,t h e d e v e l o p m e o
n ft r e c o r d i n tge c h n o l o g i ehsa sd o m i n a t etdh e v i s u a ls e m i o t i cf r o m
the momentRenaissance
artistsbeganto usethe cameraobscuraasan aid in painting,
and
particularlyduringthe last two centuriesor so,whena varietyof recordingtechnologies
weredeveloped,
with photography.
They,in their turn, are now beginning
to be
beginning
s u p e r s e d ebdy s y n t h e s i z i nt egc h n o l o g i eQs u
. i t ed i f f e r e not n t o l o g i c aolr i e n t a t i o ngso w i t h
Walter Benjamin(1973) commentedon the transltion
these differenttechnologies.
betweenmanualproductionand recording,stressingreproduceability
rather than the
of traditionalforms
modesof representation
themselves,
and linkingit to the dissolution
in 'masssociety'andto the disappearance
of the 'aura' of the work
of socialorganization
of art.
f r o m r e c o r d i ntgo s y n t h e s i z i nt egc h n o l o g i ei sst h e m o r ep r e s s i n g
T o d a yt h e t r a n s i t i o n
issue.The 'crisis of representation'
which has characterized
theoreticaldebateover
t h e l a s tt w o d e c a d eosr s o m a y b e a n i n d i c a t i o o
nf t h i s . ' R e c o r d i n g ' l e a d
wse, b e l i e v et o,
ontologies
of referentiality,
a view of representation
beingfoundedon direct,referential
we developed
relationsbetween
andthe world.In an earlierpublication
the representations
t egc h n o l o g i e
t h i s i d e ai n m o r ed e t a i l( v a nL e e u w eann d l ( r e s s1, 9 9 2 ) .S y n t h e s i z i n
u sn d e r mineor evenabolishsuchnotionsof referentiality,
whereasas recentlyas in the 1,970s,
' E l e c t r o n iN
c e w sG a t h e r i n g ' w ausb i q u i t o uesn o u g h
t o h a v ed e v e l o p eadn a c r o n y m , ' E N G ' ,
- wild mushgatherings
a deceptively
naivemetaphorremlniscent
of otherunproblematic
r o o m s a, p p l e st,h e c h i l d r e nT. h a t m e t a p h o irs n o w e n t i r e l yu n t e n a b l en;o t o n l y b e c a u s e
n e w sn e v ew
r a ss i m p l y ' o utth e r e ' t ob eg a t h e r e db,u t e v e nm o r es o b e c a u steh et e c h n o l o g y
nowexistsliterallyto produceit - a development
anticipatedby the criticalmediatheory
o f t h e 1 9 7 0 s . ' R e f e r e n chea' sg i v e nw a y t o ' s i g n i f i c a t i o nt h' ,e p r o d u c t i o no,u t o f e x i s t i n g
semioticresources,
of newsemioticmeans,newsigns,newtexts,new images,newvisions,
new worlds.This doesnot mean that representation
has ceased.Rather,the formerly
naturalizedrelation,the identity of representation
and reference,has broken down,
irreparablyfor the time being.A new relationis becomingestablished
instead,between
r e p r e s e n t a t i oa n d s i g n i f i c a t i o Inf . p r e s e nst o c i a la n d t e c h n o l o g i c d
snae
l v e l o p m e nct o
t i n u et,h i s r e l a t i o nw i l l , i n i t s t u r n ,f i r s t b e c o m e
t ,n dt h e nc o m e
n a t u r a l i z eadn dd o m i n a n a
into crisis.In the yearthat we revisedthis book for its secondedition,the 'production'
of photographs
of abusesof prisonersby Britishtroopsin Iraq was one (notorious)case
in point.(Leavingthe crucialmatterof veracityaside,it is interesting
to notethatthe
m u c he a r l i e r ' p r o d u c t i oonf 't h es a m e ' n e wesv e n t ' b ym e a n so f w r i t i n gp r o d u c endo o u t c r y
o f a n vk i n d . )

M ater ia I ity and meanIng

2r9

T h e s tee c h n o l o g i caanl ds e m i o t idc e v e l o p m e nptes r h a phse l pu su n d e r s t a ncdo n t e m p 0 r 'representation'

a sn o l d e rs e m i o t i cosf
t si.t h ' r e c o r d i n g ' g o e
a r y t h e o r e t i c adle v e l o p m e nW
of
and of naturalisticmodality,which itself gaverise to particularontologiesof truth,
repreTo deconstruct
around'blas').
of priordecades
debates
fact (hencethe interminable
'deconstructed',
('texts')
be
had
to
themselves
representations
as
recording,
sentation
'constructed' p r o d u c t i o n ' a ntdh e r e f o r e
d i s p l a y i ntgh e
w h i c hw a s d o n eb y e m p h a s i z i n g
n e s so/ f r e p r e s e n t a t i oinnst ,e x t s i,m a g e se,t c .T h i sc a nb e s e e na st h e p e r i o do f ' c r i t i q u e ' e sda d e s c r i p t oorf m a n yp r a c t i c e sI n. h e r e nitn t h i st h e r e
w i t ht h ea d j e c t i v e ' c r i t i c a l ' u sa
through
new
stageof semioticpractice,namelysynthesis,
of
one
theorizing
a
was already
(conout
of
new
representations
production
mal<ing
practices
and
construction
of
new
- ln the visualarts maybesomeof the practicesof'Brit Art'
structed)representations
J e f f l ( o o n sa, n d i n m u s i ct h e c u r r e n t l yu b i q u i t o upsr a c t i c eos f
a
r
t
i
s
t
or of theAmerican
m
i
x
i
n
g
.
scratching,
the present
which both heraldsand legitimizes
From sucha historyand perspective,
whichwill be to deconstruct
we couldprojecta further development,
stageof synthesis,
'underneath'productionthere is an
that
practices
production
by
showing
of
current
'programme',
a systemwhichdefinesthe limitsof proalready-produced
already-existing/
of the
a representation
duction.Thissystem,of course,is still a systemof representation,
t
h
eories
i
n
s
e
m
i
o
t
i
c
p
r
e
s
e
n
t
t
e
n
d
e
n
c
i
e
s
p
o
i
n
t
t
h
e
t
o
a
l
s
o
W
e
c
o
u
l
d
s
y
s
t
e
m
.
social/cultural
i n t ow r i t i n go, r v i c e
( i n c u l t u r asl t u d i e si n, e d u c a t i o ni n, l i t e r a ctyh e o r y t) o c o l l a p s ree a d i n g
v e r s aT
. h ed i s s o l u t i oonf t h a t d i s t i n c t i o (n' r e a d i n gi s w r i t i n g ' )w a s i n i t i a t e di n t h e o r yi n
wsh i c h
t h e 1 9 6 0 sb y R o l a n dB a r t h e st ,h o u g hi t i s n o we n a c t e di n e l e c t r o n itce c h n o l o g i e
c
h
a
n
g
e
s ta
on
i
n
(
r
e
)
w
r
i
t
l
n
g
e
l
s
e
'
s
t
e
x
t
,
w
h
e
t
h
e
r
s
o
m
e
o
n
e
r
e
a
d
i
n
g
a
n
d
o
f
a
c
t
s
combinethe
when
words,
games.
other
In
playing
computer
in
differently,
or,
email
to
the
attachment
' r e c o r d i n g ' w adso m i n a n t h
, e t e n d e n coy f c r i t i c a lt h e o r y
l ya s e dm o d eo f
t h ea n a l o g i c a lb
the'constructedness'of
emphasize
and
to
representation-as-reference/
wasto deconstruct
t
h
e
t
e
x
t
.
t h e s i g n o, r
A s t h e s y n t h e t i c a lbl ya s e dm o d eo f p r o d u c t i o ins b e c o m i ntgh e d o m i n a ntte c h n o l o g y ,
critical theory will have to turn to deconstructingrepresentation-as-programme,
possibilities
and laying
the combinatorlal
that is,deconstructing
representation-as-design;
represenon
concentrate
we
that
reason
for
this
It
is
sources.
cultural/social
baretheir
given
deconstruc(only)
the
However/
on
texts.
rather
than
book,
in
this
tational resources
t i o n o f f o r m e r l ys t a b l ef r a m e s- w h e t h esr e m i o t i oc r s o c i a lc/ u l t u r aal n de c o n o m i cf o r t h e
of socialsemioticpracticein termsof
time beingthereexistsnewlythe needfor conceiving
'rhetoric'focuses
.design',
on the socialrelatlonswhichobtain
term
the
where
rhetoricand
' d e s i g n ' f o c u s oe ns t h e a r r a n g e m e o
n ft t h e
t
e
r
m
a
n
d
t
h
e
p
r
o
c
e
s
s
c
o
m
m
u
n
i
c
a
t
i
o
n
,
o
f
inthe
message'
as
a
representation
of
the
making
in
the
resources
semiotic
available
Beforewe leavethis subject/we shouldnotetwo otheraspectsof the relationbetween
of productionmediawas basedon the way
productionand technology.
0ur classification
by more or lessautomatedrecordingor
by
hand,
produced,
whether
are
representations
andherethe
favourmodesof reception,
production
media
also
But
synthesis.
by electronic
(wal1s,
favour
screens)
clnema
surfaces
role.
Some
particularly
plays
important
a
surface
the computerscreen)
andpapergenerally,
publicreception,
andothers(pages,
for instance,

220

M ater i al ity and meani ng

favourindividualreception.
- thereis the effectof the
Also- and moredifficultto describe
physicality,
the tangibilityof the surface,the differencebetweenthe formscarvedin the
hardrockandthe ffeetingffickersof lighton the glassscreen(we returnto this in the next
section).What mattersis the site as much as the kind of surfaceon which the text is
received.
Now,unlikein previousperiods,
the surfaceof receptionis no longernecessarily
at all the sameas the oneon whichthe text was/isproduced.
Transcodings/transpositions
o f a w i d ev a r i e t ym a yt a k ep l a c eA. n i m a g em a yb ep r o d u c eidn o n em e d i u m
- a sa p a i n t i n g ,
say- and be receivedin a differentmedium- as a photograph,
for instance.
0r it maybe
producedin a recordingmedium,as a photograph,
for instance,
and received
in a svnthesis
medium,retrieved
from the imagebankof a computer.
Finally,technologyhas also developeddifferentdistributionmedia, andit is herethat
the issueof (mass)reproduceability
belongs,togetherwith that of communication
at
( l o n g )d i s t a n c eT.h el a t t e ra, l t h o u g h
o f c r u c i asl o c i a il m p o r t a n c b
ee
, a r sl e s sd i r e c l yo n o u r
subiect'Whetherimagesare distributedvia electricalwires,opticalfibresor the airwaves
is irrelevant,
- thoughnot at the levelof dissemiotically,
at the levelof representation
semination'
Thefact that the internetis crammedfull with imagesis in largepart a matter
of available
technoJogy;
and it has profound
semioticconsequences.
At anotherlevelwhat
m a t t e r sm o s ti st h e p r o d u c t i om
n e d i u mi n w h i c hi m a g e as r ep r o d u c e d
a ,n dt h ed i s t r i b u t i o n
mediumin whichtheyare received,
if the latter is differentfrom the former,be it because
(e.g.the photographic
of transcoding
reproduction
of paintings)or because
of recodingat
the otherendof the telecommunication
channel.0r, to be moreprecise,
we wouldsaythat
the modeof transmission
is relevantonly in relationto the potentialswhich it offersfor
reception
as (re-)production.

BRUSHSTROKES
If one looks at Rothko'sseagram Murals from a distanceof 4-5m, the boundaries
betweenthe largeblocksof colourseemsharpand clear-cut.Thecloseronemovesto the
paintings,however,
the more uncertainandfuzzythe boundaries
become,
the morethey
overlapand run into eachother.yet a postcardof oneof the paintings,
takenfrom much
morethan5m distance,
showsnothingof this.An aspectof meaningis lost,because
of the
distancefrom which the photographwas taken,and becauseof the transcoding,
from
paintingto photograph,
from oneproductionmediuminto another.
Thisbringsus backto
the startingpointof thischapter,
to the argumentaboutbrushstrokes
andaboutthe status
of'text'.
Paintingallowsthe viewera choicebetweendifferentwaysof relatingto the text,even
thoughthis choicemay be restrictedin practice,as whena lineon the floor in front of a
p a i n t i n gp r o h i b i t tsh e g a l l e r yv i s i t o rf r o m c o m i n gt o o c l o s et o t h e p a i n t i n g .
I m a yw i s h
to viewthe paintingas 'a representation',
concentrating
on what the painting'is about,,
or view it in terms of its varioustechniques('the effectiveuse of colour,),or effects
('depression').
In eachcaseI will standat the requisitedistance.
I maywishto engage
with
its materialityandwith the way in whichthe handof the artist ,inscribed,
the canvas- the

M ate r i a I ity and meani ng

22f

- and in that caseI wouldneedto

of the painUthe brushstrokes
matterof the application
e f v i e w i n gb, u tw i t h o u ta n y
m o v ev e r yc l o s eu p .P h o t o g r a p hayl l o w sm ec h o i c eo f d i s t a n c o
o f t h e s ee f f e c t si n m e a n i n gT.h i s ' r e c o r d i n g ' m e d i ui smr e p r e s e n t a t i o naanld c a n o n l yb e
It abstractsawayfrom the imprintof the handthat madeit, evenwhenit
representational.
art.
reproduces
na
, r t i c u l a r liyn
C e r t a i nd i s c o u r s easb o u ta r t p u t m u c he m p h a s iosn m a t e r i apl r o d u c t i o p
the violent
of Rembrandt,
relationto a handfulof great painters:the fine brushstrokes
s .h i se n c o u r a g e s
b r u s h s t r o k eosf V a n G o g h t, h e t r e a t m e not f v o l u m eb y t h e P o i n t i l l i s t T
a focuson the 'graphology'of the paintingas a symptom,a trace of the individualtemperamentof the artist.That is, it tal<es
a semioticapproachto the matterof brushstrokes.
focuson
however,
of paintlngsln art galleriesand in catalogues,
Most of the descriptions
of art historiansfor black-andThe preference
ratherthan on inscription.
representation
concern
or evenfor drawingsof paintingsalsopointsto an overriding
whitereproductions
with representation.
The questionof transcodingis closelyrelatedto this, What doesit slgnifywhen I
ma l l ?T h a tI w a n t t o s h o w
h a n go n m y l i v i n g - r o o w
p u r c h a saep r i n to f M o n e t ' sP o p p i e s t o
my appreciationand admiratlonof Monet?That t have beento a gallerywherethe
T h a t I a m f a m i l i a rw i t h
p a i n t i n gw a s e x h i b i t e dT? h a t I l i k et h e t h e m eo f t h e p a i n t i n g ?
s a p a r t ? T h e p r i n t w i l l a l l o wm e t o
t h e i n t e l l e c t u ahli s t o r i e so f w h i c hI m p r e s s i o n l sim
s i g n i f ya l l o f t h e s eb, u t i t w i l l n o t a l l o wm e t o s i g n i f ym y i n t e r e sitn t h e m a t e r i apl r o d u c t i o n
it doesnot enablemeto focuson that; it doesnot evenmakeit
of thework,simplybecause
a v a i l a b laes a q u e s t i o n .
T o t a k ea n o t h eer x a m p l ei n, a n o r i g i n apl a i n t i n gb y M o n d r i a nt h e l i n e sa r e ,i n c l o s e - u p ,
is modulatedrather
andthe colourof the variousrectangles
not straight,but overpainted
of the samepaintingmakethe lines
andotherreproductions
than plainandflat. Postcards
colour.It is these
and presentffat, unmodulated
appearstraight,removethe overpainting
, ds o
r e o r o d u c t i ow
n sh i c hp r o d u c teh e M o n d r i a no f i n n u m e r a bhl ei g h - s c h oaorl t l e s s o nas n
reinforce
a n d r e p r o d u cae p a r t i c u l a r( i n c o r r e c tv) e r s i o no f M o n d r i a na n d a p a r t i c u l a r
( i d e o l o g i c avl )e r s i o no f a b s t r a c p
t a i n t i n gI.t i s t h r o u g ht h e s ei d e o l o g i z erde a d i n gtsh a t
suchpaintingshavetheir effecton,and in,otherpractices.
(plate5). Representation
is strongly
Cossacks
considerl(andinsl<y's
As a final example,
r e d u c e di n t h i s p a i n t i n go: n e c a n ' s e e ' C o s s a c kosn h o r s e b a cal <n d d r a w na n d f f a s h i n g
or, we might say that what is represented
sabres,but figurationis not foregrounded;
. o l o u rd i f f e r e n t i a t i oonn, t h e o t h e rh a n d i, s a m p l i f i e dA. n d
i d e a t i o n a l il sy ' v i o l e nat c t i o n 'C
of whiteto represent
function(shades
colouris used,not in a representational/referential
t h e ' r e a l 's h a d e os f t h e w h i t eo f u n i f o r m sf o, r i n s t a n c eb) ,u t t o a l l o wt h e s u b s t a n coef t h e
p r o d u c t i om
n e d i u ma n dt h et r a c e so f t h e a c t o f p r o d u c t i otnh e m s e l v et os s i g n i f yC. o l o u ri s ,
jaggedblocksof white,flashesof red,curves
in this painting,the key semioticresource:
throughthe
of yellowand blue.Action,energy/movementand violenceare represented
n e d i u mh a s b e e nh a n d l e dA. n d t h i s t a k e su s b a c kt o t h e i s s u eo f
way the productiom
and perhapsalsoto the
with whichwe endedthe first sectionof this chapter,
individuality
g f p a i n ti s c l o s et o t h e i n d i v i d u a l i ot yf
d i s t i n c t i obne t w e e n ' a r t ' a n d ' d e s i gTnh' .eh a n d l i n o
t yh i c hc a m et o b e l a u d e da st h e
h a n d w r i t i na
gn
, di t w a sp r e c i s e tl yh i sm a r ko f i n d i v i d u a l iw

222 .

Materialityandmeaning

distinguishing
characteristic
of art and as its mark of difference
from 'recording,.0nthe
otherhand,this celebration
of individuality
was alsosomewhatof a last-ditch standfor
art, in its losingbattle with photography,
and the very sameart which cameto stand
for individual
expression
had its principalsocialeffectsin the form of photographic
reprod u c t i o n s- t h a t i s , w i t h o u tt h e m a r k so f i n d i v i d u a l i t- y w h i l et h e o r i g i n a l sb e c a m et h e
priceless
relicsof a pastethos.In mostotherdomainssuchmarksof individuality
became
proscribed.
Handwriting,
for instance,
has now becomeunacceptable
in all but the most
privateformsof writing,despitethe increased
emphasis
on 'presentation,
whichwe noted
earlier:this newvaluationof 'presentation'
is in no way a returnto the kind of individual
expressiveness
that hoverson the border betweenthe individualand the social,the
o s t e n s i bu
l yn s e m i o t i c i zaenddt h es e m i o t i c i z eidt ;i s t h o r o u g h lsye m i o t i a
c n ds o c i a l .
T h ei n d i v i d u a l i ot yf t h e b r u s h s t r o kneo t o n l yb e c a m a
e s y m b ool f i n d i v i d u aelx p r e s s i o n ,
of 'the essence
of thingsseenthroughan individualtemperament,,
but it alsocameto be
d r a w ni n t o t h e d o m a i no f t h e s e m i o t i ct ,h e d o m a i no f c u l t u r e I. t ' m a d e s c h o o l ,I.t w a s
reproduced,
faked,developed,
imitated,and so enteredinto the world of semiosisas a
transformative
element,in a processwhich then transformedthe brushstrokeitself.as
witnessed
by RoyLichtenstein's
popart parody(figure7 .l). Thusthe brushstroke
becomes
a paradigm
c a s eo f h o wi n s c r i p t i o(nt h e ' h o w ' o fp a i n t i n gi)s a l l o w e dt o p l a ya k e yr o l ei n

riq z.r Big painting, t965(Roy Lichtenstein)

M ater i al ity and meani ng

223

past
the domainof the greatModernistart of the immediate
somedomains- for instance,
(
t
h
e
'
w
h
a
t'of
- b u t r e q u i r e dt o p l a y a h u m b l e rr o l e i n o t h e r sw
, h e r er e p r e s e n t a t i o n
practice.
The
painting)dominates,
and where productmattersmore than processand
, h o s ew o r k w e
d i v i s i o ne x i s t sw i t h i n p a i n t i n ga l s o .A p a i n t e rs u c ha s R o b e r tR y m a nw
'how',
on the inscriptionalpracticesof painting,
in chapter5, focuseson the
discussed
focusedon the'what', on the analysisof social
whereasa paintersuchas Gainsborough
a r r a n g e m e n t s , t h e r e c o r d i n g o f s o c i a l s t aatfef as iorfsi;n s h o r t , o n t h e r e f e r e n t i a l . ( T h o u g h
lt wouldbeto saythat for Rymanthe'how' hasbecome
maybethe betterway of describing
s s k i s , ' W h a ti s p a i n t i n gf o r ? ' a n dt h a t
t h e ' w h a t ' .I n t h a t v i e wt h e q u e s t i otnh a t p a i n t e r a
ouestlonhasdifferentanswersin dlfferentperiodsand in dlfferentcultures.)
y f repreS u c hs h i f t so f e m p h a s ias r e t h e m s e l v e s ' s i g nIsn' t. e r e s ti n t h e m a t e r i a l i t o
with
practicesreflectswider socialand culturalconcerns
sentationand representational
abstract,
questions
and materialityin a world in whichthe concretebecomes
of substance
and reality'virtual'.Todaywe have,
the substantialinsubstantial
the materialimmaterial,
, h a n k e r i nfgo r t h e i n d i v i d u atlh, e s u b j e c t i vteh,e a f f e c t i v et h, e n o n - s e m i o t i c
s i d eb y s i d e a
a n d n o n - s o c i a' pl u n c t u m ' o ft h e p h o t o g r a pohr t h e ' g r a i n ' o f t h e v o i c e( B a r t h e s , 1 9 7 7 ,
semioticization
I9B4), and at the sametime (andin largepart as a result)the increasing
rather
and more.As we havenotedalready,it is as representations
of all thesephenomena
that modernart hasinformedandshapedpracticesin other
than as materialproductions
d o m a i n st h, a t p a i n t i n gssu c ha st h o s eo f M o n d r i a nc a m et o b et r a n s f o r m eidn t ob l u e p r i n t s
cities,and that paintingssuchas thoseof l(andinskyhad
objects,buildings,
for designed
suchas the Bildzeitungand The Sun,
their effect on the layoutof Europeannewspapers
but no less
of other translations/reductions,
which are further translations/reductions
potentfor that.
or of handwriting)
semioticsof materiality(of the brushstroke,
Any systematizing
t hge n o ty e ts e m i o t i c i z eRde. l u c t a n ct oe d o s ow o u l d
w o u l df o l l o wt h et r e n do f s e m i o t i c i z i n
'protect'the non-semiotic,
to protectthe
follow the opposingtrend set by Barthesto
,unsooiltnature'afterwhichthe touristhankers.
In this chapter,we(like him) are alsoat
l ya k i n ge v e r y t h i nsge m i o t i c a l l y
l e a s ta l i t t l e r e l u c t a ntto f o l l o wt h e p a t h o f r e l e n t l e s sm
In reality,we think that the choiceis not one betweenturningthe unsemiotic
accounted.
then it is
take meaningfrom somephenomenon,
into the semiotic- if I as an interpreter
in layingbare,
semioticby virtueof that action.Thechoiceratheris onebetweenassisting
c ;n d
i n m a k i n go v e r t l yv i s i b l em e a n i n gwsh i c ha r e a s y e t n o t v i s i b l ea n d m a d es y s t e m a t i a
as materialproduction
Nevertheless,
of that process.
beingclearaboutthe consequences
s o r ei m p o r t a ntto b ea b l et o t a l k a b o u ti t . S i m p l ya s s e r t i ntgh e
t b, e c o m em
i s s e m i o t i c i z ei d
aroundus it is being
at the very momentwhen everywhere
valueof the non-semiotic
s t l e a s te q u a l l yp r o b l e m a t i c .
s e m i o t i c i z em
da
, r g i n a l i z eodr r e p r e s s esde e m a

T H E M E A N I N GO F M A T E R I A L I T Y
a ,t r e n dt o
' r r e n itn M o d e r n i s m
F r o mt h e 1 9 2 0 so n w a r d st h, e r eh a sb e e na ' f u n c t i o n a l i sct u
' l e t m a t e r i a l s p e a kf o r t h e m s e l v e sw' ,h l c hi s o n l y n o w b e g i n n i ntgo c h a n g eT. h i s h a d

224

M aterial ity and meani ng

variousroots/but it culminated
in nowhappilyclich6dstylesof 'plainness,:
whetherof steel
'
b
r
u
t a l i s t ' a r c h i t e c t u rwei,t h
o r t i m b e rf u r n i t u r eo, r o f
i t s l o v eo f u n a d o r n ecdo n c r e t e .
Behindthistrendwerenotionsof 'authenticity',
themselves
explicitor implicitcritiquesof
the 'distortions/of representation,
the 'falsities'introducedby 'decorative,art and its
ideologies.
Thistrendhad its originsin art, whereit couldevenbecomethe subjectmatter
of artworks,as in Ryman'spaintings.In someperiodsor genresof art, artists haveno
choiceof materials:all paintingsare paintedon cavewallswith ochres,or on canvaswith
oil paints,all photographs
are printedon paper.In otherperiodsor genres,the material
b e c o m eas f u l l y e x p l o i t a b laen de x p l o i t e rde s o u r c eM. o d e r ns c u l p t u r ies p e r h a ptsh e b e s t
e x a m p l e . T h e s m o o t h l y t u r n e d w oBordaonfc u s i 'Hs e a d Q g I g - 2 3 ) o r t h e v e i n e d r e d n e s s o f
the alabasterof Epstein's
Jacoband theAngel 1940) (seefigure8.1) becomepart of the
meaning.
In somecasesit is preciselythe oppositionbetweenthe materialityof the material
a n d t h e m i m e t i cq u a l i t yo f t h e p r o d u c tw h i c hb e c o m etsh e i s s u eI.n R o d i n , sT h e K i s s
(1880), the figures are perfectlyworked when seen from the front.
The material
resistances
of marblehave beenentirelyand successfully
overcome.
The materialhas
just as the materialityof the canvasis invisiblein mostpaintings.
become'invisible',
If we
changeour viewingpositionby walkingaroundthe sculpture,
however,
we are permittedto
seethat this first impression
is'produced',and thereforeideological.
Thecontrastposes
the questionof the sculptor'swork, his semioticaction.It forcesus to reflecton the
borderlines
betweenthe seemingly
unsemioticized
materialityof the representation
and
t h e s e m i o t i c i z efdu,l l y c u l t u r a lw o r k o f t h e s c u l p t o ra, n d o n t h e d i a l e c t i cb e t w e e n
the
e x p r e s s i oonf i n d i v i d u a l i at yn dt h e s o c i a sl e m i o t i fcr a m e w o r ki n w h i c hi t t a k e sp l a c eT. h e
two areconnected
t h: e ' h a n d w r i t i n g ' e f f e csto, c l e a r l yv i s i b l ei n t h e l e s s , p o l i s h epda, r t s
o f R o d i n ' s c u l p t u r eb,e c o m eisn c r e a s i n gpl yo s s i b l ae s w e m o v ef r o m t h e h i g h l yp o l i c e d ,
the highlyconventionalized,
the fully semiotic,
towardsthe lesssemioticized,
thereforeless
p o l i c e da n d l e s sc o n v e n t i o n a l i z eTdh.e u n c e r t a i nl i n e so f a M o n d r i a np a i n t i n gs i g n a l
individuality,
affect and art as clearlyas the certain linesof a blueprintsignalconv e n t i o n a l i trye, f e r e n caen d d e s i g nA. p a i n t i n gb y B e n N i c h o l s o (n1 9 4 5 ) m a k e st h e s a m e
point in a differentway.It consists,
quitesimply,of two circles:a perfect,compass-aided
circleon the left,as the Given;and a hand-drawn
circleon the right,as the New.Drawing
b y h a n d ,w i t h a l l i t s s u b t l em a r k so f i n d i v i d u a l i tm
y ,a y o n c eh a v eb e e nu n p r o b l e m a t i c ,
as there was no other way of drawing,but has now becomeproblematic,
an issuefor
c0ncern.
Architectswho developblueprintsfor buildingsalso work with 'unsemioticized,
materials.
Theirintentions
are usuallylesssemioticor,to put it anotherway,it is harderfor
t h e mt o p u t t h e s e m i o t i ci n t h e f o r e g r o u n d . O t hceorn s i d e r a t i om
n sa yw e i g hm o r eh e a v i l y :
(thefact that theyare designing
functionalconsiderations
an officebuilding,
for instance),
f i n a n c i acl o n s i d e r a t i otnhse,w i s h e so f a c l i e n tA
. r t , o n t h eo t h e rh a n dh, a s s, i n c et h e 1 9 6 0 s ,
attemptedcritiquesof masssocietyand in particularof massproductionmethods,
which
ledit towardsthe verypracticesit soughtto critique.Whenthe Americanartist Jeff l(oons
commissions
sculptures
from factories,from
artisansor from otherartists,andthensigns
them with his name,he works with attenuated,abstract,surfaces,:
the whole qlobal

Materialityandmeaning ' 225

l f t h e w o r k . C h o i c es e r v e sa s t n e
n e c o m etsh e m a t e r i a o
d o m a i no f c u l t u r a lp r o d u c t i o b
culturalproductionas the materialsurface'0ur focus
productiontool, and industrialized
o n p h y s i c aml a t e r i a l i tsyh o u l dt h e r e f o r be et a k e na s a m e t a p h oIrn. t h e h i g h l ys e m i o t i c i z e d
worldwe livein,Jeff l(oons'workmaybe both morerelevantand moreusualthanthat of
G i a c o m e to
t ir M o o r eo, f R o t h k oo r R y m a nT. h en o t i o no f a r t i s t sp r o d u c i nogb j e c t s ' w i t h
t h e i ro w nh a n d s ' d a t ebsa c kt o t h e p r e - i n d u s t r ipael r i o da, n dh a da l r e a d yc o m ei n t oc r i s i s
ao
l r l di t h a sl o s t
i n t h e p e r i o do f i n d u s t r i a l i z a t i oI tn .m a y b e t h a t i n t h e p o s t - i n d u s t r iw
muchof its relevance.
The relativefreedomof the artist, perhapsgreaterin the visualthan in the verbal,
r e m a i n sf,o r t h e t i m e b e i n g i,n f o r c e ,a l b e i to n t h e m a r g i n sI.t l i e si n t h e p o s s i b i l i toyf
- either of the materialityof the meansof productionor of the object
foregrounding
p r o d u c e do ,r o f t h e s e m i o t i c i z a t i o fnt h i s m a t e r i ailn ( r e f e r e n t i aolr s i g n i f i c a t o r ay c) t so f
Wallpaperdrawsattentionawayfrom the materialityof a wall' A room
representation,
on concreteor pinewallsdrawsattentionawayfrom the facts
withouta hint of decoration
of the wall itselfandon to the materia[ityof the material'
W h a t , t h e n , i s t h e m e a n i n go f m a t e r i a l ?Q u r a s s u m p t i o rne m a i n st h a t s i g n sa r e
motivated.It is no accidentthat the statueserectedto commemorateheroicfiguresare
made of durablematerials,or that tombstonesare still carved:the durabilityof the
we intendto
feelings
of permanent
for the meanings
materialsmakesthemusablesignifiers
for
produce.
Nor is it an accidentthatcertainffowersor stonesbecomesignifiers love:their
or elsetheir colour,shapeor perfumemay mal(ethem
rarity may makethem precious/
s u i t a b l es i g n i f i e r sB. o n e - c h i nt ae a c u p sd o n o t p r o d u c et h e s a m em e a n i n gass t i n m u g s '
as if it is
Australianmoneyis printedon plastic,a shockstill to the returningexpatriate,
t o o b o l d l ya s i g n i f i eor f w h a ts h o u l dn o t b es i g n i f i e d .
of
semioticresources
the interrelated
materialproductioncomprises
To summarize,
in
their
and
Eachhasits own semioticeffects,
andtoolsof production.
substance
surface/
planes;
interactionthey producecomplexeffectsof meaning.Productionexistson many
sign,
of
the
As with Barthesinotion
that is, thereare serialrelationsbetweensurfaces.
s
erial
a
r
e
t
h
e
r
e
. nd
a s s i g n i f i e rast a h i g h e rl e v e lA
s i g n sa t o n e l e v e lb e c o m ea v a i l a b l e
relation
the
relationsof translationbetweenproductionmediaalso,as in the caseof
b e t w e epna i n t i n gasn dt h e i rp h o t o g r a p hri ec p r o d u c t i o n s .

G O L O U RA S A S E M I O T I CM O D E
andtools'We
substances
Ourfocussofar hasbeenon materialityof the sign- on surfaces,
more
now want to turn to the questionof materialityas a meansfor representation
our
but
directty,
0f coursewe havetouchedon this manytimes,indirectlyor more
centrally.
f so r
r
e
s
o
u
r
c
e
q u e s t i o n o w i s , ' H o w d o e sm a t e r i a l i t a
y c t u a l l ye n t e ri n t o a n d s h a p et h e
'grammar'
of
our
the modes?'We referredto thiswhenwe pointedoutthat
representation,
g
r
a
m
m
a
r
l
i
n
g
u
i
stic
t
h
e
o
f
t h e v i s u a cl o u l dn o t b e ,s i m p l ya, t r a n s p o s i t i oonf t h et e r m so f a
for
the
resource
of
mode,becauseit had to pay dueattentionto the materialdifferences
- notthe materialof soundas in speech,
but the materialof graphicstuff as
reoresentation

226

Materiality and meaning

in images,not the orderinglogicof time as in speech,

but thai of the space
of surfacesof
images.
0ne theoreticalissueto be 'got out of the way,,so to speak,is that of'abstraction,.If
t r a n s p o s i t i o nosf l i n g u i s t i tce r m i n o l o gtyo o t h e r m o d e sh a v eb e e nq u i t ec o m m o n p l a c e
in the past,it has beenbecause'grammars,
of languagepaid attentionto an abstract
entity,'language',
anddid not seeit or its elements
in termsof their materiality.That
is one
reason
w h y p h o n e t i c(sa sa d i s c i p l i n eh)a db e e ne x c l u d efdr o m l i n g u i s t i c(sa sa d i s c i p l i n e ) .
It had beentoo muchconcerned
with the physicalityof its domain.It is also one reason
why for much of the twentiethcenturymuch of mainstreamlinguisticswas concerned
with 'language'as such,as an abstraction,ratherthan with the distinctiveness
of the
grammaticalorganization
of speechand writing.At that levelof abstractionit seemed
possible
to movefrom onemediumto another:materialitywasnot seen,and hencedid not
figureand did not needto be accounted
for.
In our approachthe twin factorsof materialityand of culture(alwaysset in the social
organizations
in whichculturesexist)are the meanstoan explanation
of the resources
for
representation.
Materialityentersinto semiosisthroughtangiblephysicalfacts: speech
happensas sound,and soundhappensin temporalsequence.
However,
what culturedoes
with these'factsof nafure'is thenanothermatter.In anytemporalsequence,
something
is
first and thereforesomething
elseis secondand something
elselast;culturescannotget
aroundthat fact. But what meaningsmay be attachedto that orderingis quite
another
malter,a matterfor makersof signsin their cultures.'Being
first, mayhaveanynumberof
'that whichis
meanings:
mostimportantto me the speaker,,
or ,thatwhich is my starting
point,from whereI can proceed',
or'the entitywhichis responsible
for the actionwhichis
represented
a 'n, d s o o n . U s i n g' s p e e c h
s o u n d sm
, e a n su s i n gt h e p o s s i b i l i t i eosf c h a n g e s
in pressure
in the air,aswell asthe potentialof changes
in the frequency
of vibrationof the
vocalcordsand changesin the volumeand frequency
of oscillationof a columnof air in
the 'vocaltract'to fashiona semioticresource
- speechsoundsand intonation.
Again,the
sameconsiderations
apply:someculturesusethe potentials
of pitchfor syntactic/semantic
means/t0 makequestions;
othersuseit to producedifferences
in lexis- in the so-called
tone languages,
such as cantonese,Mandarin,Igbo, etc. differencein pitch produces
difference
in lexis.
What materiala culturechooses
to fashioninto a resource
for makingrepresentations.
into a mode,is a matterof the contingencies
of that cultureand of its history;thoughit is
a l s oa m a t t e r- a f u n d a m e n t ap lo i n tn e g l e c t eidn t r a d i t i o n al li n g u i s t i c- so f t h e
bodilyness
of humansas makersand as receivers/remakers
of signs.A modeis a meansfor making
r e p r e s e n t a t i otnhsr,o u g he l e m e n t(ss o u n d ss,y l l a b l e sm, o r p h e m ew
s ,o r d sc, l a u s e sa) n dt h e
possibilities
of theirarrangement
astexts/messages.
Colouris sucha material,and
herewe
willexplore
t h eq u e s t i oonf c o l o u ra s m o d ei n s o c i asl e m i o t itce r m s l,e t t i n gc o l o u rs t a n di n
for all otherinstances
of the relationsof material,cultureand modeas semioticresource.
We beginwith a bit of history,to demonstrate
the pointwe madeabove.In the Middle
Ages,pigmentshad valuein themselves.
Ultramarine,
as the name indicates,
had to be
importedfrom acrossthe seaandwasexpensive,
not onlyfor this reason,but alsobecause
it was madefrom lapislazuli.Henceit was usedfor high-value
motifs,suchas the manle

Materiality and meaning

'

227

of the Virgin Mary.Suchpigmentswerenot mixed,but usedin unmixedform,or at most

c i g m e n ths a dt o r e m a i nv i s i b l e ,
o n l ym i x e dw i t h w h i t e .T h em a t e r i aild e n t i t i eosf s p e c i f i p
and it was thesematerialqualitieswhich motivatedtheir use,their meaning'As a result
' c o l o L l r ' w a as c o l l e c t i o o
n f d i s t i n c tm a t e r i asl u b s t a n c ersa,t h e rt h a n a ' s y s t e m ' - ' l e x i s '
'grammar'.
ratherthan
A r o u n d1 6 0 0 , i n D u t c hp a i n t i n gt,h e t e c h n o l o gcyh a n g e dN. e w t e c h n i q u easl l o w e d
e a c ho a r t i c l et o b e c o a t e di n a f i l m o f o i l ,w h i c hi n s u l a t e idt a g a i n sct h e m i c arle a c t i o n
As a resultthe statusand
mixingpossible.
with otherpigmentsand mademoreextensive
from
price of specificpaintswent down and colour becameto someextentdisengaged
an
extensive
its materiality.Colourwas no longerusedand thoughtof as a collection,
and
eachwith their ownaffordances,
catalogueof distinctlydifferentindividualpigments,
(black,
white,
colours
but as a systemwith five elementary
hencesemioticpotentialities,
yellow,red andblue)from whichall othercolourscouldbe mixed'Butthis systemwas not
a s e m i o t i cs y s t e mI.t w a s a ( p r a c t i c a lp) h y s i c so f c o l o u rj,u s t a s p h o n e t i ciss a p h y s i c s
and this madeit possibleto
abstraction,
it involvedconsiderable
of speech.Nevertheless
just
as the systemof languageis appliedto the
apply the systemto differentmedia,
between
aswell asto the mediumof writing.Newtondrewa comparison
mediumof speech
(harmony)
elements
the
and
of music
the elements(tones)and rules of combination
. o l o u r sw o u l db e ' c o n ( c o l o u r sa) n d r u l e so f c o m b i n a t i o(n' c o l o u rh a r m o n y 'o) f c o l o u rC
'elements'
sonant,or'dissonant'on the basisof the sameintervalsbetweensevenordered
in the
(green,blue,incligo,
violet,red,orangeandyellow)as music(thesevendifferenttones
'
o
c
u
l
a
r
e
x
p
e
riment
o
f
a
t
y
p
e
harpsichord',
d a s t etlo b u i l ta n
o c t a v e )I.n h i st i m ei t i n s p i r e C
synaesthesia
in
in the interest
whlchhascroppedup againand againsince for instance,
psychologists
and,more recently/in the
andauditioncoloreeof earlytwentieth-century
p
o
i
n
t
i s , ' c o l o u r ' w ans0 l o n g ear
w o r ko f a b s t r a cfti l m a n dc o m p u t ear n i m a t i oanr t i s t sT. h e
r ' s. l ( a n d i n s kwyo u l dl a t e r
c o l l e c t i o on f m a t e r i asl u b s t a n c epsi /g m e n t si t; b e c a m e ' c o l o u A
,Colour
grammar
of its own,al<into the
is onlylooselyattachedto objects.. . . It hasa
say,
grammarof music.'
Of course,colourhasalwaysbeenusedas a semioticresource.
What,then,of meaning?
In the MiddleAgesthereweremanytheoreticaland practicaldebatesaboutcoloursym( g l o r yj ,o y ) ?B u tt h e r ew a s
b o l i s mS. h o u l dm o n k sw e a rb l a c k( p e n i t e n cheu, m i l i t yo) r w h i t e
s e l la S ' l i f e '
n o u n i f i e ds y s t e mG. r e e nc o u l dm e a n ' j u s t i c e ' awse l la S ' h o p er' ,e d ' c h a r i t y ' aw
(1528)
argued
andso on.Learnedtomessuchas F.P.Morato's0n the Meaningof Colours
a
r
g
uments
i
n
t
h
e
i
r
c e a n i n gosf c o l o u ra, n d
w i t h a n da g a i n set a c ho t h e ra b o u t h es y m b o l im
for
unity,
of
the meaningsof colourwere alwaysmotivated.Greencould be the colour
could
Red
Trinity'
the
of
in representations
it was usedas a background
because
instance,
t i s t h e c o l o u ro f b l o o d .S o m em o d e r na r t i s t sh a v et r i e dt o r e v i v e
m e a n ' l i f e , b e c a u si e
a w o r l d l yv i e wo f e c o n o m yr ;e d ,t h e
t h i s k i n d o f s y m b o l i s mF.o r M a l e v i c hb l a c kd e n o t e d
, o r ec o m p l e xi d e a sc o u l d
. i t h t h e s ee l e m e n t sh,e t h o u g h tm
, c t i o nW
r e v o l u t i o na;n dw h i t e a
\ c o l o u rc o d e s h
/ a v el i m i t e d
b e c o n s t r u c t e dB.u t a s i n t h e M i d d l eA g e s ,c o n t e m p o r a r y
in different
meanings
and specificcolourscan haveverydifferent
domainsof application,
attempt to
an
was
contexts.The work of Malevich,Mondrian,l(andinskyand others
o
f
c
o
l
o
u
r 'B
. u tt h e y
r , o r ew i d e l ya p p l i c a b l e ' l a n g u a g e
t h e p o s s i b i l i toyf a b r o a d e m
explore

228

M ateri al i ty and meani ng

d i d n o t m a n a g et o b r i n gs u c ha r a n g u a gien t o b e i n gA
. s G a g eh a ss a i d t, h e i re x p e r i m e n t s
'offeredthe prospect
of universality,
tbut becamelthoroughlyhermetic,i:-999:24$.
S o c o l o u rh a s ,o n t h e o n eh a n d ,d e v e l o p eidn t o a ' m o d e , ,a s y s t e m a t i c a lol yr g a n i z e d
resource.
But on the otherhand,this systemis a physical,ratherthan a semioticsystem,
a kind of 'phonetics',althoughthe basic elementsof the system,the 'primary, and
'secondary'colours, played
have
a keyrole in visualsemioticpracticesand in accountsof
the meaningof colour.semiotically,
a single,system,hasnot developed.'what
peopledo,
w i t h c o l o u rv a r i e se n o r m o u s layn, d s o c i a lg r o u p sw h i c hs h a r ec o m m o np u r p o s easr o u n d
usesof colour areoften relativelysmaliand specialized
- they do not constitutealarge
group/as is the casewith speech,
or with the systemsof visualcommunication
we have
d i s c u s s ei ndt h i sb o o k .B u t i f w es t a yw i t ht h en o t i o nt h a t ' w h a tp e o p l e
d o ,s h a p etsh et o o l s ,
andbearin mindthat verydifferentthingsaredoneby differentgroups,
we mightbeableto
m a k es o m es e n s eo f h o w c o l o u rb e c o m eas u s a b l er e s o u r cfeo r m a k i n gm e a n i n gI .f w e
relatethe meanings
of colourbothto their materialityandto what peopledo with that,we
m i g h tb e a b l et o a s kt h e r u c i a l q u e s t i o n sI s: c o l o u ra m o d eo f r e p r e s e n t a t i o
i nni t s o w n
right?Doesit offerthe full affordances
of mode?
So the task is to discoverthe regularitiesof the resourceof colouras they existfor
s p e c i f igc r o u p st ;o u n d e r s t a nt h
d e mw e l le n o u g h
t o b ea b l et o d e s c r i bw
e h a tt h e p r i n c i p l e s
for the use of the resourcein signsare (that is, to understandhow a specificgroup,s
interestin colourshapestheir signsof colour).From that we might beginto understand
g e n e r apl r i n c i p l eosf t h e s e m i o s iosf c o l o u ra n d o f s e m i o s igse n e r a l l a
y ,n d t h e s ei n t u r n
e p r i n c i p l eudn d e r s t a n d i o
m i g h tp r o v i d a
n fga l l u s e so f c o l o u ri n a l l s o c i o c u l t u rdaol m a i n s .

THE COMMUNICATIVE
F U N C T I O N SO F C O L O U R
I n H a l l i d a y 'm
s e t a f u n c t i o nsael m i o t i ct h e o r ya, c o m m u n i c a t i o nsayls t e ms i m u l t a n e o u s l y
fulfilsthreefunctions:the ideationalfunction,the
functionof constructing
representations
of the world; the interpersonal
function,the functionof enacting(or helpingto enact)
interactions
characterized
by specificsocialpurposes
andspecificsocialrelations;
andthe
textualfunction,the functionof marshalling
communicative
acts into largerwholes,into
the communicative
eventsor textsthat realizespecificsocialpractices.
We can ask the
questionsthat we haveaskedof imagesgenerallyof colour specifically.
can it create
'participants';
specificrelationsbetween
that is,betweenrepresented
people,
places,
things
and ideas?Can it represent
socialrelationsand helpenactsocialinteraction?
And can it
- for instance,
realizetextualmeanings
in a systemof reference
or in creatingcohesion
in
an text?
In the preceding
chapters
we have,we hopereasonably
plausibly,
appliedthis modelto a
numberof resources
(compositio
of visualcommunication
nt gazetangle
andsizeof frame,
a n ds oo n ) , t h e r e b(yr e ) c o n s t i t u t i n
t hge s er e s o u r c eassp a r t o f t h e ' g r a m m a t i c as ly s t e m , o f
images,in Halliday's
terms.we did not,however,
in the first versionof this book,dealwith
c o l o u ri n t h i sw a y ,b e c a u swe e f o u n di t d i f f i c u l to a s s i g nc o l o u rp l a u s i b ltyo j u s t
o n e_ a n d
o n l yo n e- o f H a l l i d a y 'tsh r e em e t a f u n c t i o nI ts i.st r u et h a tt h e r ei s a d o m i n a ndt i s c o u r s e
of

Materiality and meaning

'

229

y n do t h e r s( e . 9 .P o y n t o n ,
t o a f f e c ta, n d H a l l i d a a
c o l o u ri n w h i c hc o l o u ri s p r i m a r i l yr e l a t e d
B.u t t h e
a le t a f u n c t i o n
1 g g 5 ; M a r t i n , I g g D s e ea f f e c ta s a n a s p e c ot f t h e i n t e r p e r s o nm
colour
is
that
We
functionof colouris not restrictedto affectalone. think
communicative
andthat it is thereforea modein its ownright.
usedmetafunctionally,
Startingwith the ideationalfunction,colour clearlycan be usedto denotepeople,
placesand thingsas well as classes
of people,placesandthings,and moregeneralideas.
usespecific
increasingly
denotestates,andcorporations
Thecoloursof ffags,for instance,
c o l o u r so r c o l o u rs c h e m e tso d e n o t et h e i r u n i q u ei d e n t i t i e sC. a r m a n u f a c f u r e rfso,r
ensurethat the dark blue of a BMW is quite dlstinctfrom the dark blue of a
instance,
, n dt h e yl e g a l l yp r o t e c t ' t h e i r ' c o l o u rsso,t h a t o t h e r sw i l l n o t b ea b l et o u s e
V W o r a F o r da
t h e m . E v e nu n i v e r s i t i euss e c o l o u rt o s i g n a lt h e i r i d e n t i t i e sT. h e 0 p e n U n i v e r s i t yf o, r
stipulates:
example,
stationeryand degree
suchas high-quality
Two colours. . . for formal applications
certificates blue (referencePMS 300) for the shieldand lettering,and yellow
( P M S 1 2 3 )f o r t h ec i r c u l airn s e tS. i n g l ec o l o u rs t a t i o n e rsyh o u l db e i n b l u e( P M S
300)if possible.
( o u o t e di n G o o d m a an n dG r a d d o rl ,9 9 6 t l l 9 )
andso on.In
mountains,
0n maps,colourscanserveto identifywater,arableland,deserts,
u n i f o r m sc,o l o u rc a n s i g n a lr a n k .I n t h e s a f e t yc o d ed e s i g n ebdy U S c o l o u rc o n s u l t a n t
' r e d 'i d e n t i f i e s
F a b e rB i r r e n( L a c y , I 9 9 6 : 7 5 )',9 r e e ni' d e n t i f i efsi r s t - a i de q u i p m e nwt ,h i l e
hosesand valves(whichplay a role,of course,in fire protection).0nthe LondonUnder, n do n b o t h U n d e r g r o u n d , ' g r e e ind' e n t i f i etsh e D i s t r i c tL i n ea n d ' r e d ' t h eC e n t r aLl i n e a
groundmapsand in Underground
stationsmanypeoplelool<for thosecoloursfirst, and
s p e a ko f t h e ' g r e e nl i n e 'a n dt h e ' r e dl i n e ' .
l o: w su s t o r e a l i z e ' c o l o u r
C o l o u ri s a l s ou s e dt o c o n v e y ' i n t e r p e r s o n a l ' m e ai nt ianl g
acts' (as languagepermits'speechacts'). It can be and is usedto do thingsto or for
to warnagainstobstructions
through'power-dressing"
eachother:to impressor lntimidate
and other hazardsby paintingthem orange,or to subduepeople- apparentlythe Naval
' p i n k , p r o p e r l ya p p l i e dr, e l a x e sh o s t i l ea n d
C o r r e c t i o n aCl e n t r ei n S e a t t l ef o u n dt h a t
, 9 9 5 : 8 9 ) .A c c o r d i ntgo t h e G u a r d i a d s
l si t h l n1 5 m i n u t e s('L a c y 1
a g g r e s s i vi ned i v i d u a w
' O f f i c eH o u r ss' u p p l e m e n
( 3t S e p t e m b e2r0 0 1 :5 ) :
, C o l o u r sa r e v e r y p o w e r f ual n d c a n r e d u c eo r r a i s es t r e s sl e v e l s , ' b e l i e v e
L sl l i a n
and are good
author of ColourHealing.Bright redsare energising
Verner-Bonds,
fields.Greenis usefulif there'sdiscord
for officesin the bankingor entertainment
o r d i s h a r m o nays i t i s s o o t h i n gB. l u ei s r a t e da s t h e b e s tc o l o u rf o r p r o m o t i n cga l m
activity.
and pastelorangeis goodfor gentlyencouraging
the
can increase
in the samearticlewe learnthat addingcolourto documents
Elsewhere
the
has
reader'sattentionspanby more than eightyper cent and that'an invoicethat
p
a
i
d
t
h
a
na
on time
a m o u n to f m o n e yi n c o l o u ri s t h i r t y p e r c e n t m o r e l i k e l yt o b e

230

M ate r i al ity and meani ng

m o n o c o l 0 uorn e /I.n a l l t h e s ec a s e cs o l o u r e p r e s e n tpsr/o j e c t se,n a b l eosr c o n s t r u c tsso c i a l

relations- it is interpersonal.
It is not just the casethat colour 'expresses/
or 'means'
t h i n g ss u c ha s ' c a l m ' o r ' e n e r g y 'r; a t h e rp, e o p l ea c t u a l l yu s ec o l o u rt o t r y t o e n e r g i zoer
c a l md o w no t h e rp e o p l eP. u t t i n gi t m o r eg e n e r a l lcyo, l o u ri s u s e dt o a c t o n o t h e r st,o s e n d
managerial
messages
to workers,or parentalmessages
to children,aswe haveshownin an
a n a l y s ios f a c h i l d ' sr o o m( l ( r e s sa n dv a n L e e u w e n , 2 0 0 t ) . lits u s e db y p e o p l et o p r e s e n t
andthe valuestheystandfor,to sayin the contextof specificsocialsituations,'l
themselves
a m c a l m /o r ' l a m e n e r g e t i ca' ,n dt o p r o j e c t ' c a l m ' o r ' e n e r g y ' apso s i t i v ve a l u e sW
. ew i l l
a d d r e stsh i si n m o r ed e t a i lb e l o w , w h ewne a n a l y steh e u s eo f c o l o u ri n h o m ed e c o r a t i o n .
C o l o ua
r l s of u n c t i o n sm, a y b e v e nm o s to b v i o u s layt,t h et e x t u a l e v e lI.n m a n yb u i l d i n g s ,
the differingcoloursof doorsand other features- the colour schemesof floors - distinguishdifferentdepartments
from eachotheron the onehand,whilecreatingunityand
withinthesedepartments
coherence
on the other.Colourcanbe usedto createcoherence
in
texts.Textbooks
makewide useof this,whetherin 'readingschemes'
or in mathematics
textsto indicate'levels'of difficulty,or in sciencetextbooksto providetopical unity.In
Pasos,a Spanishlanguage
textbook(Martfn and Ellis,200I), the chapterheadings
and
pagenumbersof eachchapterhavea distinctcolour,all sectionheadingsCVocabulario
e n c a s a ' , ' G r a m d t i c ae 't,c . )a r e r e d t h r o u g h o ut th e b o o k ,a n d a l l ' a c t i v i t i e s ' ( e . 9 . ' M a k e
phraseswith es or est6')havea purpleheadingand number.In an issueof the German
(November200I), film reviewshaveorangeheadlines
editionof Cosmopolilan
and other
usesof orangein the typography,
as the background
of textboxes,
etc.Theart reviewsuse
greenin a similarway,book reviewsusered,and so on. In somecasesthis is cuedby a
s a l i e n ct o l o u ri n t h e k e y i l l u s t r a t i o o
n f t h e f i r s t p a g eo f t h e r e l e v a nrte v i e ws e c t i o nf;o r
instance,
Cate Blanchett'sorangehair in a still from the film Banditsin the film review
section.
' C o l o u r - c o o r d i n a t i or n
a 't,h e rt h a n t h e r e p e t i t i o no f a s i n g l ec o l o u rc, a n b e u s e dt o
promote
I n. t h i sc a s et h ev a r i o u cs o l o u r so f a p a g eo, r a l a r g e rs e c t i o n
t e x t u acl o h e s i o n
of a
text (or of an outfit,or a room),may haveroughlythe samedegreeof brightness
and/or
saturation.lncomputersoftwaresuchas PowerPoint,
thisfeatureis alreadybuilt in,a kind
o f a n a l o g uoef t h es p e l l - c h e c ksehro, w i njgu s tt h ed e v e l o p m einntt h ed i r e c t i o n
of a broader
u s eo f g r a m m a rC. h o o s i ntgh e i n i t i a lb a c k g r o u nadu t o m a t i c a l sl ye l e c tas r a n g eo f c o l o u r s
a s a c o l o u rs c h e m eF.o r i n s t a n c ei f,t h e i n i t i a lc o l o u ri s a p a s t e lt ,h e nt h e o t h e rc o l o u r sw i l l
a l s ob e p a s t e l sI t. i s p o s s i b lteo o v e r r i d teh i s b y s e l e c t i nagn o t h e cr o l o u rf r o m a M u n s e l l
atlastypedisplay,
but thistakesmoreeffort andskill.
Thereare two pointsto make.First, colour fulfils the three metafunctions
simult a n e o u s lT
y .h ec o l o u r so n a m a pr e t a i nt h e i ri d e a t i o n aaln dt h e i r i n t e r p e r s o nvaal l u e , t h e i r
appealingbrightness,
or stuffy dullness;on maps coloursare coordinatedto enhance
textualcohesion.
And contemporary
scientificvisualizations
are thoughtof as primarily
i d e a t i o n avl :e i n sa n d a r t e r i e sw i l l b e r e p r e s e n t euds i n gd i f f e r e nct o l o u r st o i n d i c a t et h e
a m o u n to f o x y g e n- o r t h e l e v e lo f i t s d e p l e t i o nS.e c o n dw, e a r e n o t a r g u i n gt h a t c o l o u r
a l w a y sh a s a n d a l w a y sw i l l f u l f i l a l l t h r e e o f t h e s ef u n c t i o n se q u a l l y C
. o l o u rd o e s
w h a t p e o p l ed o w i t h i t , i n m a k i n ga s i g na n d i n r e m a k i n tgh e s i g ni n i t s r e c e p t i o nW. e a r e
n o t ' d i s c o v e r i nugn' i v e r s aaln d s u p r a h i s t o r i cfaalc t sa b o u tc o l o u rh e r e W
. e a r e t r y i n gt o

M aterial ity and meani ng

23f

d o c u m e nwt h a t k i n do f c o m m u n i c a t i vweo r k c o l o u ri s m a d et o d o i n t o d a v ' si n c r e a s i n g l y

global semioticpractices,and how.The examplesprovidean indicationthat some of
t h e s eu s e so f c o l o u rh a v ef a i r l ys p e c i f i cl i,m i t e dd o m a i n sw, h e r et h e yc l e a r l yr e l a t et o t h e
) hileothers
s p e c i f i ci n t e r e s t so f s i g n - m a k e r(se . g .m a p - m a k i n gs,u b d u i n gp r i s o n e r sw
(e.g.
as a meansof
in magazines
may havewider distribution
the use of colour-coding
c o h e s i o) n
.
F i n a l l y , t hcee n t r aql u e s t i o nI f: w ea r er i g h t ,i f c o l o u rf u l f i l sa l l t h r e em e t a f u n c t i o ni ssi,t
. ut is
, u s i c ?M a y b eB
a s e m i o t i cm o d ei n i t s o w nr i g h t ,a l o n gw i t h s p e e c hi m
, a g ew, r i t i n g m
of (andhave
Language,
therenot alsoa difference?
imageand musichavebeenconceived
i n v a r i o u s ' p u r i s t ' p r a c t i c eosf t e no p e r a t e da) s r e l a t i v e l yi n d e p e n d e snet m i o t i cm o d e s .
work
Althougha novelis a materialobject,anda pagea visualartefact,its communicative
i s d o n ep r i m a r i l yt h r o u g hw r i t i n g .I n a n a r t g a l l e r yi m a g e su s u a l l yc o m ew i t h n o w o r d st h e s m a l ld e s c r i p t i vsei g no n t h e w a l l n e a r b yi s n o t a p a r t o f t h e i m a g eb u t a p a r t o f t h e
t a l l e v e r y t h i nigs c o n e n v i r o n m e no tf d i s p l a yi n t h e g a l l e r yo r m u s e u mI .n t h e c o n c e r h
o n t h e m u s i cw
, h i l ee x p r e s s i ot h
n r o u g hs e m i o t i cm o d e ss u c ha s d r e s sb, o d i l y
centrated
popularmusic
performance,
to contemporary
etc.,is heldback,certainlyby comparison
s h o w sT. h e r ei s a c h o i c ef o r t h e a u d i e n cteo f o c u so n t h e m u s i c ' a ss u c h ' o ro n t h ew h o l ea s
'performance
t h' ,e ' c o n c e r t ' .
I s t h i st h e c a s ew i t h c o l o u r ?P a i n t e rhsa v et r i e dt o m a k ep a i n t i n gtsh a t u s eo n l yc o l o u r
and nothingelse('field painting',Rothko,etc.),but this doesnot appearto haveled to
modefor the age of
a wholenew artform.Thenagain,maybecolour is a characteristic
with architecture,
typography,
multimodality.
It cancombinef reelywith manyothermodes,
p r o d u cdt e s i g nd,o c u m e ndte s i g n .
just
Let us stepbackfor a moment.As oneof uswritesthis - it is a day in mid-August,
w i n d o w so,n a t r a n q u i l
b e f o r el u n c h t i m e I a m s i t t i n gl o o k i n go u tt h r o u g ht h eo p e nF r e n c h
o ,m eC h a r o l a icsa t t l ei n
French
c o u n t r y s i dIes. e el o wh i l l st,r e e sf,o r e s ti n t h eb a c l < g r o u ns d
the pasturebeyondthe fence.Thereare a veryfew fluffy clouds,thoughaboveand beyond
g et h i n ko f a l a t e
h i l l t h e r ei s a d e n s ebr a n ko f c l o u dj,u s ta p p e a r i n gm, a l < i nm
t h ef o r e s t e d
g u t s p e c i f i ce l e m e n t s ,
a f t e r n o o nt h u n d e r s t o r mI .a m d e s c r i b i ntgh i s s c e n eb y s e l e c t i n o
n a m i n gt h e m ,p u t t i n gt h e mh e r eo n t h es c r e e no f m y l a p t o pa sw o r d si n a p a r t i c u l aor r d e r .
I h a v ea v o i d e u
d s i n ga n yc o l o u rw o r d s .
w i n d o was t a w o r l d
No w l e t m et r y t h i sa g a i nI. a m l o o k i n go u tt h r o u g ht h eo p e nF r e n c h
greensof the mostvariedhuesdominate,
thoughthereare greys
of colour.Overwhelmingly,
that I seeI seeas
of variouskindsand browns,dark purples,blues,off-white.Everything
colour.And if I representit here on my screenagain as words,it is becauseI have
translatedthe world as I seei/ into the modethat my culturehas made most readily
but as representaavailableto me. UsuallyI don't evenregardthat actionas translation,
of thistranslation
it. Theelements
tion:thatis howmy culturehastaughtmeto understand
- first on to this screenin waysI do not understand,
lateron to a
codehaveto be inscribed
p r i n t e dp a g e- t h o u g hu n t i la b o u te i g h ty e a r sa g oo r s o I w o u l dh a v et r a n s c r i b et h
d e mo nt o
directly,usinga pen.
the paper-page
I mighth
, o w e v ehr ,a v el e a r n e d( o r m i g h ts t i l l l e a r n )t o p a i n t .I w o u l db e a b l et o r e p r e wouldnot be
sentthis scene- it wouldof coursenot be a recording(evena photograph

232

M ateri al ity and meani ng

t h a t ) - i n a m o d ew h i c h i s c l o s e rt o t h e m a n n e ro f m y p e r c e p t i o nC.o l o u rw o u l db e
represented
by colour,whereasat the momentcolouris represented
by words(in syntactic
order).Thecolourswouldof coursebe organized,
as blocks,splashes,
lines,dots:the greys
- as the trunksof the
and brownswouldappearas thinner,differentlyverticalelements
treesI see,or as the ffecksof grey in the bank of cloud,the greensof varioushuesand
brightness
as leavesand bladesof grass,and the purplesas the dots of varioussizesof
p l u m sa n dr i p e n i negl d e r b e r r i eCso. l o uw
r o u l da p p e aer n t i r e l yb y i t s e l -f o n a n i n s c r i p t i o n a l
(are
surface,
of course,
no less,butalso
we rightin saying?)no moresothanthewordsI used
aboveneedthe inscriptional
surfaceof pageor, if spoken,
the inscriptional
surfaceof air
and,if heard,the receptororgansof the ear.
In my banal accountof this framedsegmentof the landscape
I usedwords as my
descriptional
resource/
havingbecomeso usedto it that it also servedas my meansof
analysisof the countryside.
Themodegavemethe termswith whichto analysethat which
I saw,and it gaveme the meansfor its description.
Themodeof colour- if we seeit as a
mode- wouldgivemedifferentterms(not of courseas herein my transcription
as words),
if I wereableto paintwhatI seehavingmixedmy owncolourson my palette,usingnowa
'terms'.
quitedifferentsetof analyticalanddescriptive
N o n eo f t h i s i s n e w ;a n d t h e I m p r e s s i o n i swt se r ej u s t o n e ' s c h o o l ' o fp a i n t e r sw h o
workedwith ideassuchasthese,evenif moresubtlythoughtandexpressed,
andfocusedon
t h e m a t e r i a l i toyf l i g h tr a t h e r t h a no f c o l o u rH
. o w e v e r , w h a t w o ubledn e w f o ru s n o wi s t o
seecolourfor what it is and what it does.Doescolourhereexiston its own?Well,yes,
of course- at leastas much as do words spokenor written.Onceposedand seenin
this contexttheanswerbecomes
somewhatoddlyself-evident.
Cancolourbe or becomea
modeonlyin a multimodalenvironment?
Well,yes,in the sameway- no moreno less,even
if differently- as everyother mode.And the experiments
of Mondrian,of Rothkoor
N i c h o l s o na,s o f o t h e r sw, o u l dn o wb e s e e nn o t s o m u c ha s e x p e r i m e nitns t u r n i n gc o l o u r
into mode,but as experiments
in abstracting
awayfrom the (attempted)realismsof blocks
(astreetrunks),slashes
(as bladesof grass),lines(asedgesof all kinds)and dots(berries
o r p l u m s ) , t u r n i nr ge a l i s mo f t h e i d e a t i o n akli n di n t oi t s a b s t r a c t i o n .

A D I S T I N C T I V EF E A T U R EA P P R O A C HT O T H E S E M I O T I C SO F C O L O U R
- and coloursare signifiers,
In l(ressand van Leeuwen(2001),we arguedthat signifiers
not signs- carry a set of affordances
from which sign-makers
and interpreters
select
accordingto their communicative
needsand interestsin a givencontext.In somecases
t h e i r c h o i c ew i l l b e h i g h l yr e g u l a t e b
d y e x p l i c i to r i m p l i c i tr u l e s o
, r b y t h e a u t h o r i t yo f
expertsand rolemodels.In othercases- for instance,
in the production
and interpretation
o f a r t - i t w i l l b e r e l a t i v e l fyr e e .I n o u r b r i e f a n a l y s i so f t h e u s e o f c o l o u ri n h o m e
decoration
below,we showhowin mostsituations
thesetwo poles,constraintandcreativity,
are both in evidence
and mixedin complexways.
Like l(andinsky,
we distinguishtwo types of affordancein colour,two sourcesfor
- the questionof
makingmeaningwith colour.First there is association,
or provenance

Materiality and meaning . 233

'wherethe colourcomesfrom','wherewe haveseenit before'.Theassociations

takenup in
many of the communicative
usesof colour,suchas in advertisingor the entertainment
symbolicvaluein
media,will usuallybewith substances,
objects,etc.that carrysignificant
given
the
sociocultural
context.Whilethe affordances
of a colourmay be wide in theory,
is taken into
in practicethey are not whenthe contextof productionand interpretation
account,as we will try to do in the analysisbelow.
The secondtype of affordanceis that of the 'distinctivefeatures'of colour.Herewe
wantto showsomeaspectsof the affordances
of the materialityof colour,andhencemake
In Jakobsonand
and phonology.
a connection,
not with the 'grammar',but with phonetics
(
1
9
5
6
)
,
t
h
e
H a l l e ' sd i s t i n c t i vfee a t u r ep h o n o l o g y
f e a t u r ens a m e dr e a lm a t e r i apl h e n o m e n a
- suchas the point of articulationof a consonanl,
or the apertureof the mouth in the
S.o
m a k i n go f a v o w e-l a n d i n d e s c r i p t i ot n
h e yw e r ed e p l o y eads o p e r a t i n ign o p p o s i t i o n
suchas /+voiced/as
oneconsonant
couldbe distinguished
from anotherby an opposition,
againstl-voiced/,an oppositionwhich would distinguish
/b/ from /p/, or /d/ from /t/, in
of the features,
English.
Wefocuslesson opposition
than on the quality,the characteristics
andtalk aboutvalueson a rangeof scales,suchas the scalethat runsfrom lightto dark,
the scalethat runsfrom saturatedto desaturated,
and so on. UnlikeJakobsonand Halle,
we seethesefeaturesnot just as servingto distinguish
differentsoundsor coloursfrom
each other,but aboveall as meaningpotentials;that is, as their potentialto become
of specificvalues
signifiers.
of colourcanbedefinedas a combination
Any specificinstance
meaningpotential,as
on eachof thesescales- andhencealsoas a complexandcomposite
w ew i l l t r y t o s h o wb e l o w .
Value
Thescaleof valueis the greyscale,the scalefrom maximallylight (white)to maximally
d a r k ( b l a c k )I.n t h e l i v e so f a l l h u m a nb e i n g sl i g h ta n dd a r ka r ef u n d a m e n t ae lx p e r i e n c e s ,
a n d t h e r ei s n o c u l t u r ew h i c hh a s n o t b u i l t a n e d i f i c eo f s y m b o l i cm e a n i n gasn d v a l u e
- eventhoughdifferentculturesmayhavedone
systems
uponthis fundamental
experience
- are often
so in differentways.Painterswho emphasize
Rembrandt
value- for instance,
d ays.
a b l et o e x p l o itth i sm e a n i n p
g o t e n t i ai ln c o m p l e xa n dp r o f o u n w
Saturation
T h i si s t h e s c a l ef r o m t h e m o s t i n t e n s e lsya t u r a t e d
o r ' p u r e ' m a n i f e s t a t i o nosf a c o l o u r
, o s t ' p a l e ' o r ' p a s t e l ' ,o r d u l l a n d d a r k m a n i f e s t a t i o nasn, d ,u l t i m a t e l y ,
t o i t s s o f t e s tm
to completedesaturation,
to black and white. Its key affordancelies in its ability to
expressemotive'temperatures',
kindsof affect.It is the scalethat runsfrom maximum
i n t e n s i t yo f f e e l i n gt o m a x i m a l l ys u b d u e dm
, a x i m a l l yt o n e d - d o w ni n, d e e dn e u t r a l i z e d
feeling.In contextthis allows many differentmore preciseand stronglyvalue-laden
adventurous,
but also vulgar or
meanings.
High saturationmay be positive,exuberant,
garish.Low saturationmay be subtleandtender,but alsocoldand repressed,
or brooding
and moodv.

234

M ate r la I ity and meani ng

Purity
' p u r i t y , t om a x i m u m
, h y b r i d i t ya, ,n di t h a sb e e n
T h i si st h es c a l et h a t r u n sf r o mm a x i m u m
at
the heart of colour theory as it developed
over the last few centuries.Many different
systemsof primaryand mixedcolourshavebeenproposed
- somephysical,
somepsychologicalandsomea mixtureof both- andthissearchfor primariesor basicshasnot resulted
in a generallyaccepted
system,but'has provedto be remarkablyinconsequential
and . . .
f r e i g h t ew
d i t ht h eh e a v yb u r d e no f i d e o l o g y( G
' a g e1, 9 9 9 :1 0 7 ) .S o m ew r i t e r sh a v es e e nt h e
i s s u ea sc l o s e l rye l a t e d
t o t h eq u e s t i oonf c o l o u rn a m e sC. o l o u r w
s i t h c o m m o n luy s e ds i n g l e
n a m e ss/u c ha sb r o w na n dg r e e nw, o u l db ec o n s i d e r epdu r e T
. h en a m e so f o t h e rc o l o u r sl ,i k e
cyantare mainlyusedby specialists,
andnon-specialists
wouldreferto themby meansof a
compositename,forinstance,
blue-green.
Suchcolourswouldthenbe perceived
as mixed.
T e r m sl i k e ' p u r i t y ' a n d ' h y b r i d i t y ' a l r e asduyg g e ssto m e t h i nogf t h e m e a n i n g
potential
o f t h i sa s p e cot f c o l o u rT. h e ' p u r e ' b r i g hrte d sb, l u e sa n dy e l l o w so f t h e ' M o n d r i a n , c o l o u r
schemehavebecomekeysignifiers
of the ideologies
of modernity,
whilea colourschemeof
pale,anaemiccyansand mauveshas becomea key signifierof the ideologies
of postmodernism
i n, w h i c ht h e i d e ao f h y b r i d i t yi s p o s i t i v e lvya l u e dT. h i si s b y n o m e a n st h eo n l y
way in whichthe affordances
of this scalehavebeentakenup,but it is a culturallysalient
o n ea
, n dh e n c eo n ew h i c hi s c u r r e n t l q
y u i t ew i d e l yu n d e r s t o o d .
Modulation
T h i si s t h e s c a l et h a t r u n sf r o m f u l l y m o d u l a t ecdo l o u r( f o r e x a m p l ef r, o m a b l u et h a t i s
richly texturedwith differenttints and shades,
as in paintingsbyClzanndto flat coJour,
(as in comicstrips,or paintingsby Matisse).It was already
recognized
as a featureof
colour in Goethe'sFarbenlehre(Theoryof Colour) (1970 tlBt0l). The affordances
o f m o d u l a t i oanr ev a r i o u as n d ,a g a i ns, t r o n g l vy a l u e - l a d eFnl.a tc o l o u rm a yb ep e r c e i v eads
s i m p l ea n d b o l di n a p o s i t i v es e n s eo, r a s o v e r l yb a s i ca n d s i m p l i f i e dM. o d u l a t e d
colour,
similarly,
may be perceived
as subtleanddoingjusticeto the rich textureof real colour.or
as overlyfussyand detailed.And,as we havediscussed
in chapter5, modulationis also
c l o s e l yr e l a t e dt o t h e i s s u eo f m o d a l i t yF. l a tc o l o u ri s g e n e r i c o l o u ri,t e x p r e s s ec so l o u ra s
qualityof things('grassis green'),whilemodulatedcolouris specificcolour
an essential
('the colourof grassdepends
on the time of dayand the weather'),
it attemptsto showthe
c o l o u ro f p e o p l ep,l a c e sa n dt h i n g sa s i t i s a c t u a l l ys e e nu, n d e rs p e c i f i lci g h t i n gc o n d i t i o n s .
Hencethe truth of ffat colour is an abstracttruth, and the truth of modulatedcoloura
naturalistic,perceptual
truth.

Differentiation
Differentiation
is the scalethat runsfrom monochrome
to the useof a maximallyvaried
palette,and its very diversityor exuberance
is one of its key semioticaffordances,
as is
the restraintinvolvedin its opposite,lack of differentiation.
In our analysisof an article
from a homedecoration
magazine
below,a couple'usesnearlythe wholespectrumin their

Materiality and meaning

. 235

h o u s e ' a n cdo m m e ntth a t ' i t ' s g r e a t h a t t h e r ea r es o m a n yb r i g h ts h a d e isn t h e h o u s eI.t ' s

It's whenyou start beingtimid that thingsgo
a shamepeoplearen't more adventurous.
wrong' (House Beautiful,SeptemberI99B: 2I). So here high differentiationmeans
' a d v e n t u r o u s n e s s ' al onwdd i f f e r e n t i a t i o n ' t i m i d i tbvu' .t i t i s c l e a rt h a t i n a n o t h ecr o n t e x t
restralntmighthavea morepositivevalue.
Hue
T h i si s t h e s c a l ef r o m b l u et o r e d .I n a d i s t i n c t i vfee a t u r et h e o r yo f c o l o u ri t b e c o m eosn l y
o n eo f t h ef a c t o r sc o n s t i t u t i ntgh e c o m p l e a
x n dc o m p o s i tm
e e a n i n gosf c o l o u ra, n dm a y b e
gf r e d - i n - g e n e r a l ,
n o t e v e nt h e m o s ti m p o r t a not n e .N e v e r t h e l e sa sl t,h o u g h ' t h em
' e a n i no
the red end of the scaleremains
of the abstractsignifier'red', cannotbe established,
andthe blueendwith cold,calm,
with warmth,energy,
foregrounding,
associated
salience,
and uses.
distance,
backgrounding.
Thecold-warmcontinuumhas manycorrespondences
airy/earthy,farlnear,
rareldense,
Itten (1970) liststransparenVopaque,
sedative/stimulant,
its warmth combineswith other
lighVheavy
and wet/dry.In an actual red, meanwhile,
features.An actual red may,for instance,
be verywarm/mediumdark,highlysaturated,
pureand modulated,
flow from
and sign interpreters
and its affordances
for sign-makers
are actually
all thesefactors.In the nextsectionwe will seehowsuchsetsof affordances
andvaluesare at work in
interests
takenup in a specificcontext,andwhat context-specific
t h i sp r o c e s s .

H 0 M E D E C 0 R A T I O NG: 0 L 0 U R C
. HARACTEA
RN D F A S H I O N
r e p e n dosn t h e s o c i o W h a t c o l o u r sa r e u s e di n h o m ed e c o r a t i oann d w h y ?T h ea n s w e d
regional
for instance,
culturalcontext.Therehavebeenmanydifferenttraditions,including,
in
differences,
suchasthe brightbluesandgreensof the doorsandwindowsof farmhouses
S t a p h o r s at , v i l l a g ei n t h e N e t h e r l a n dwsh e r et r a d i t i o n adl r e s si s s t i l l w o r n .B u t t o d a ya
new approachhasdeveloped,
in whichthe expertiseof the colourconsultantplaysa key
role.Accordingto Lacy (J996l,29),theentrancehall of a homesignalsthe identityof its
owneror owners:
A y e l l o we n t r a n c h
e a l l u s u a l l yi n d i c a t eas p e r s o nw h o h a si d e a sa n da w i d ef i e l do f
i n t e r e s t sA. h o m eb e l o n g i ntgo a n a c a d e m i w
c o u l dp r o b a b l yc o n t a i na d i s t i n c t i v e
, e a sa n da s e a r c h i n g
s h a d eo f y e l l o wa s t h i sc o l o u ri s a s s o c i a t ewdi t h t h e i n t e l l e c ti d
m i n d . . . . A g r e e ne n t r a n c e
h a l l - s a y ,a w a r m a p p l eg r e e n- i n d i c a t eas h o m ei n
w h i c hc h i l d r e nf a, m i l ya n dp e t sa r eh e l di n h i g hi m p o r t a n c e. . . A b l u ee n t r a n chea l l
i n d i c a t easp l a c ei n w h i c hp e o p l eh a v es t r o n go p i n i o n-s t h e r ec o u l db ea t e n d e n ctyo
in their ownworld.
appearaloofastheycan be absorbed
too muc.{r
character,
express
In expertdiscourse
of this kindthe coloursof a homeaboveall express
of the homeowneror
the personal
andthe valuesand interests
the identity,
characteristics,

236

M ater ia I ity and meani ng

owners/
whi lethecoIoursof workpIaces(andprisons,
schooIs,etc.) aremoreoftendiscussed
in termsof their effectson workers(prisoners,
students,
etc).
Most peoplewill encounter
this discoursein magazines
and televisionmakeoverprogrammeswhereit is mediatedby journalists,
althoughthe expertiseof colourconsultants
and interiordecoratorsis often explicitlydrawn on. In magazines
aimingat different
sectorsof the market,differenttypes
(for
of homeownersor celebrities
maybe introduced
instance,
the ownerof a Londonart galleryversusan actor in a popularsoapopera).
C o m p a r teh ef o l l o w i n g
twoquotes:
Her latesthabitat(shemovesas regularlyand happilyas a nomad)is surprisingly
spareandelegant,
asyoumightexpectfrom someone
with a senseof the aestheticin
her genes.After all, Jane'sgreatauntwas NancyLancaster,
of Colefaxand Fowler
fame,whileher brother,HenryWyndham,is chairmanof Sotheby,s.
(ldeal Homeand Lifestyle,
September
1998: 60)
Guessing
what Hamishand Vanessa
Dowsdo for a livingisn't too difficult- a pair
of feet on the housenumberplate
is a dead giveawayfor a couplewho are both
chiropodists,
but it's alsoan indicationof the fun they'vehaddecorating
their home.
( HouseBeautiful,September
1998: 20)
I n s u c ha r t i c l e sc o l o u rc h o i c ei s p r e s e n t eads a n o r i g i n aal n d u n i q u ee x p r e s s i oonf t h e
characterandvaluesofthe homeowners-asfully personal,
ratherthanmediatedby social
codes.Thetwo fun-lovingchiropodists
above,for instance,
usenearlythe wholespectrumin their house,from mustardyellowand leaf green
i n t h e s i t t i n gr o o m ,t o b r i c k r e d a n d b l u ei n t h e d i n i n gr o o m .T h e i rb e d r o o mi s a
soft butteryyellowcombinedwith orange,there'slemonand lime in the breakfast
roomandcornffower
and Wedgwood
blueson the stairs.'lt'sgreatthat thereare so
m a n yb r i g h ts h a d e si n t h e h o u s e 's, a y sH a m i s h . , I t , a
s s h a m ep e o p l ea r e n , tm o r e
adventurous.
It's whenyoustart beingtimid that thingsgo wrong.,
( HouseBeautiful,September
1998: 21)
Thisshowsthe readerhowcoloursemiosis
works,but at the sametime avoidsthe suggest i o n t h a t s u c hm o d e l sc a n b e s l a v i s h lfyo l l o w e da, n ds u g g e s t sh a t c o l o u rs e m i o s issh o u l d
naturallyffowfrom people'suniquecharacterandvalues.It is herethat the affordances
of
colourare takenup. Highcolourdifferentiation
and highsaturationbecomesignifiersof
'adventurousness',
with differentiation
standingfor the absence
of monotonyand routine,
a n ds a t u r a t i ofno r a n i n t e n s i toy f f e e l i n gf ,o r ' l i v i n gt o t h ef u l l ' a n dn o t b e i n g' t i m i d ' .
A l o o ka t t h ea c t u a cl o l o u r si nt h e i l l u s t r a t i o nosf t h ea r t i c l e( e . gp. l a t e6 ) s h o w s t h a t t h e
distinctivefeaturesare selectively
usedin the discourse.
Thereare,on thosebrightwalls,
paintedgold leavesand sunffowers
which, Hamishand Vanessa
say,'givesucha lovely
Victorianfeel'.Indeed,the photosshowa veryclutteredinterior,
with manyretro objects,
includingfringedlampshades
and statuettesof servileblackservants.But. evenwithout

Materiality and meaning

. 237

wouldbe
of the leavesandsunffowers
the provenance
the quoteandwithouttheseobjects,
relatively
dark
and
relatively
theyarealso
clear.Whilethe coloursmaybe highlysaturated,
to Modernistbrightand light interiors(and Mondrian-type
impure,certainlyby reference
p u r ec o l o u r s, )a n dt h i sa s p e cot f t h e c o l o u r st ,h e i rp r o v e n a n caes ' h i s t o r i c ' c o l o u riss,n o t
e x p l i c i t ldy i s c u s s ei ndt h e a r t i c l e .
S u c h ' h i s t o r i c ' c o l o uw
r se r ev e r ym u c hi n f a s h i o ni n t h e 1 9 9 0 s : ' T h es p e c i a l i spta i n t
firm Farrow& Ball whosecolourswere usedto recreateeighteenth-and nineteenthcentury England in televisionadaptationsof Pride and Preiudice and Middlemarch,
risenby 40o/oeachyear overthe pastten years'
reportsthat its saleshaveconsistently
( G u a r d i a nW e e k e n dM a g a z i n e1, 9 J a n u a r y2 0 0 2 : 6 7 ) . l t m a y b e t h a t H a m i s ha n d
but also follows
of their character,
Vanessa's
interioris not just an originalexpression
not
Vanessa
and
fashion,and alsotakesits cuesfrom the media.It may be that Hamish
unique
express
their
only use the affordancesof the distinctivefeaturesof colour to
, dt h e r e b y
i n t e r e s tas n d v a l u e sb, u t a l s o b a s et h e i r c h o i c eo f c o l o u ro n ' p r o v e n a n c ea' n
from.
It may
come
alsoexpress
the valuesof the placeand the time wherethesecolours
home,
symbolically
throughthe way they decoratetheir
be that Hamishand Vanessa,
s sh i c h
i d e n t i f yw i t h t h e v a l u e so f t h a t e r a ,a n dw i t h t h e n o s t a l g ifao r a ' l o s t ' E n g l i s h n ew
way
covert
in
a
had beenso salientthroughoutthe 1990s.In this article,this is expressed
o
u
n
i
q
uely
s
e
e
m
s
i n w h i c hc o l o u rp l a y sa n a b s o l u t e lcyr u c i a lr o l e .I t m a y b e t h a t w h a t
t
h
r
o
u
gh
r
e
a
l
i
z
e
d
n a n dt h r o u g ht h e m e d i ai,n d i s c o u r s e s
t h e i ro w n i s s o c i a l t cy o n s t r u c t ei d
colour.

C O L O U RS C H E M E S
I n t h i s l a s t s e c t i o nw e w i l l d i s c u s so n e f i n a l e x a m p l ei n o r d e rt o f o c u sb r i e f f yo n t h e
q u e s t i oonf t h ec o l o u rs c h e m eT.h ee x a m p l ies a p a m p h l ept r o d u c etdo d e s c r i baen de x p l a i n
o f a m a j o rp u b l i s h i nhgo u s ei n i h e U l ( ( p l a t e7 ) .
t h e c o r p o r a t ied e n t i t yc h a n g e
- colour,typeface,
iconsof severalkinds.Thepamphlet
A numberof modesare involved
'The
the functionof each.In the caseof colour,a captionstatesthat
brieffydescribes
c o l o u rp a l e t t ep r o v i d eas h a r m o n i o usse l e c t i oonf 1 6 c o l o u r sa, lI c a r e f u l l yc h o s e tno c o m p l e m e ntth e c o r p o r a t ceo l o u rP a l g r a vsei l v e ra, n dt h e ys h o u l db e u s e dw h e r e v epr o s s i b l e . '
and lntentls clear.Ratherthan the traditionallayoutof the colour
So the deliberateness
chart,herethe corporatecolouris central,to indicateits statusandrole,andthesubsidiary
c o l o u r sc l u s t ear r o u n di t i n a r e g u l a dr i s p l a yT.h i sc l u s t e r i nigs o r g a n i z e-d i n p a r t- o nt h e
p r i n c i p l eo f g r a d a t i o nisn h u e ,t h o u g hg i v e nt h e c o l o u r sc h o s e nt h i s c a n n o tb e a c h i e v e d
has been
entirely in the mannerof the traditionalchart; there are gaps.Coherence
e i t h t h e c o r p o r a t ce0 l o u r ,
d e l i b e r a t eal yi m e df o r : a l l t h e h u e sh a v et o b e a b l et o c o l l o c a t w
alreadya strongsenseof structure- both in the
in its support.Thereis, consequently,
'units'.But
explicithierarchyof coloursand in the delimitingof the rangeof permissible
Its introleft.
top
the bright-yellow
one colourchangesthe overalleffectof the scheme,
It
palette
differentin their meaning-potentials'
ductionin effectmakesall the coloursin its
w
o
r
ds
i
n
t
h
e
u
p
b
e
a
t
p u t st h i sc o l o u rp a l e t t ei n t ot h e d o m a i no f t h e s h a r pt,h e b r i g h tt,h e

238

M ater i a I ity and meani ng

'activepursuitof ideas','rapid
'a worrdof challenges
of the pamphlet,
change',
to be met,,
' a n e wc o m p a n y
a n da g l o b a fl o r c ei n p u b l i s h i n ge,t,c .
T o d a yc, o l o u r si n c r e a s i n gal yr e c o l o u r si n a ' c o l o u r s c h e m e c, ,o l o u r si n s y s t e m so f
colourwhich can be definedon the basisof specificusesof the distinctivefeatureswe
havediscussed.
We havecomeacrossseveralsuchschemes
already:the 'historic,colour
scheme,
basedon differentiation,
relativelyhighsaturationand dark value;the modernist
'Mondrian'colour
s c h e m eb,a s e do n p u r i t ya n d h i g hs a t u r a t i o nt h; e p o s t m o d e rcno l o u r
scheme,basedon hybridityand pastelvalues.All thesecolour schemeshave distinct
historicalplacements.
But they liveon beyondtheir historicalperiodas recognized
semiotic resources
whichcancontinueto be usedandcombined(for instance,
the bright-yellow
accent in the overall postmodernschemeof Palgrave)to realizedistinctlvdifferent
ideologicp
ao
l sitions.

I T h et h i r d d i m e n s i o n

'USER'
T H E T H I R DD I M E N S I 0 NFTR O Mn R E A D E RT'0
So far we have restrictedourselvesto still rather than movingimages,and to twooa
n le ss u c ha s
a t h e rt h a n t h r e e - d i m e n s i o n
d i m e n s i o n faol r m so f v i s u a lc o m m u n i c a t i or n
will
explore
we
In
chapter
this
design,
set
productdesign,architecture
or stage
sculpture,
be
applied
can
book
in
this
we
have
developed
framework
to whichdegreethe descriptive
i
m
a
g
e
.
t
h
r
e
e
d
i
m
ensional
0
u
r
m
o
v
i
n
g
t
h
e
a l s o t o t h e t h r e e - d i m e n s i o nvai sl u a la n d
T
o
y
s
a
r eo f
t
o
y
s
.
c
h
i
l
d
r
e
n
'
s
a
n
d
o
f
s
c
u
l
p
t
u
r
e
e x a m p l ews i l l m a i n l yb ed r a w nf r o mt h ef i e l d s
which are
particularinterestas they occupya spacesomewhere
in betweensculptures,
p r i m a r i l ys y m b o l i co b j e c t so, b j e c t sf o r c o n t e m p l a t i oann d v e n e r a t i o na,n d ' d e s i g n e d
objects',whichare primarilyobjectsfor use,eventhoughthey may alsoconveysymbolic
messages.
In additionwe will considereverydayobjectssuchas cupsand motorcars,and
We are not able,within the
The chapteris intendedas a first exploration.
architecture.
aspects
of three-dimensional
present
account
of
all
a systematic
spaceof this book,to
c
o
n
c e p t(ss o m eo f t h e s e
m
a
n
y
a
d
d
i
t
i
o
n
a
l
v i s u a cl o m m u n i c a t i oans,t h i sw o u l dr e q u i r et o o
we have
concepts
on
the
we
will
concentrate
Tnstead,
in van Leeuwen,2OO3).
arediscussed
play
in
three-dimensional
they
preceding
what
role
chapters,to show
describedin the
v i s u a lc o m m u n i c a t i oInn. t h i s w a y w e w i l l a t l e a s tb e a b l et o i n d i c a t ei n w h i c hw a y s
two-dimensional
t h r e e - d i m e n s i ovniaslu a cl o m m u n i c a t i oi sns i m i l a rt o a n dd i f f e r e n t f r o m
f
o
l
l
o
w
f
r
o
m
t
h
is.
i
s
s
u
e
w
s
h
i
c
h
c o m m u n i c a t i oann,dt o o u t l i n et h et h e o r e t i c a l
key categories
found
that
the
we
Starting with the issueof visual representation,
visuals,and
three-dimensional
also
to
we introducedin chapters2 and3 can be applied
for
sculptures,
Many
objects.
of
such
do indeedseemsufficientto describea wide range
E
p
s
t
e
i
n
'
J
s
a
c
ob
T
a
k
e
e x a m p l eh,a v ew h a t w e c a l l e di n c h a p t e r2 a ' n a r r a t i v e ' s t r u c t u r e .
powerful
(n946J,
form
Angel
and
the
shownin figure8.1: the armsof Jacob
and theAngel
way.TheAngel'saction
vectors,relatingthe two participantsin a complexand interesting
g
r
i
p
.
B
u
tJ a c o b ' sa c t i o nt,h o u g h
i s t r a n s a c t i o n aI tl .h a sa G o a l a
, s h e h o l d sJ a c o bi n a f i r m
h
e
d o e sn o t h o l do r g r a b
a
n
d
l
i
m
p
l
y
,
f o r e g r o u n d eids,n o n - t r a n s a c t i o n ahl i s a r m h a n g s
anything.
l(ennethArmitage'sPeoplein the Wind (1952),shownin figure8.2, also hasstrong
vectors,formedby the waythe figuresare bentforwardsastheystruggleagainstthe wind.
But here(as in sculpturesof discusthrowers,ballerinasand other activesubjects)the
a
Thevectorsdo not pointat or leadto anotherparticipant,
actionis 'non-transactional'.
something''
towards
not'strain
but
do
forwards', they
Goal.Thefigures,it seems,'strain
' R e a c t i o n s ' a ra
e l s oc o m m o na/ l t h o u g hi n s c u l p t u r teh e e y e sd o n o t u s u a l l yf o r m a s
they lack the
images,because
stronga focusof attractionas they do in two-dimensional
picturesas
pupils,
in
which,
the
eyes
and
strongtonal contrastbetweenthe whitesof the
must
dimension
of
the
third
naturalism
in nature,makeseyesso salient.It is as if the added

240 .

The third dimension

fig e.f Jacoband theAnge,f(Jacob Epstein,1940) (cranada TelevisionLtd)

be counteractedby greaterabstractionin othermeansof expression/

such as colourtto
preventsculpture
from crossing
the linebetween
art andthe uncannily
realmake-believe
of
the waxworksshow(or of certaincontemporary
formsof art). Nevertheless,
in a sculpture
l i k eR o d i n ' sT h e K i s s( l B B O ) t, h e m a na n dt h e w o m a nn o t o n l yh o l de a c ho t h e rw i t h t h e i r
arms,theyholdeachotherwith their gazeaswell,in a'transactional
reaction/,
andit is not
d i f f i c u ltto f i n de x a m p l eosf ' n o n - t r a n s a c t i o n
r eaal c t i o n s ' a l sJoa. c o b( f i g u r eg . 1 ) l o o k su p ,
in a non-transactional
reaction,
whilethe Angellooksat Jacob,in a transactional
reaction.
Again,the Angelactson Jacob,but Jacobdoesnot act on theAngel.Arnheim,s
description
o f M i c h e l a n g e l oM' so s e sp r o v i d easn o t h eer x a m p l e . ' T hdee f f e c t i oonf t h e l a w g i v e r ,hse a d
andthe fierceconcentration
of hisglanceintroducean obliquevectorthat moves0urwaros
l i k e t h eb e a mo f a l i g h t h o u sB
e .u t n o g o a lo b j e c ti s i n c l u d e d , ( A r n h e irm
9 ,B 2 : 4 ( ) .
Eventhe designof objectsand buildingscan be vectorialand hence,narrative,,as
shownin figure8.3. Thevectorformedby the tailfinsof 1950scars,for instance,
represented(in an abstractway) the idea of dynamicmotion,as if it was not enoughthat
cars are in fact dynamicmovingobjects,regardlessof whetherthey havetailfins o( not.
Thefact that carsdo not havetailfinsnowpointsto the ideological
dimension
of sculptural
representation:the
meanings
represented
werethoseofthe'jet age,.But herean important

The third dimension

fis O.z Peoptein the tyird (Kenneth Armitaqe,1952)(Tate Gallery)

242

The third dimension

VECTORIAL
DESJGN

N O NV E C T O R I A L
DE S I GN

Lightshades

E=
,"/'77

J.-.H

B ui l di n g s

\-?
(:!>

fiq e.: Vectorialand non.vectorial

lightshades,cups,carsand
buildrnqs

complication
occurs.In the caseof objects,the ideationalrelationswe havediscussed
in
chapters2 and3 canbe realizedin two ways:theycan be realizedby the designe4
asforms
to be'read'by a viewer,as whena cup has a'dynamic',vectorialhandle(and also,of
c o u r s ei /n t h ec a s eo f t h e p i c t o r i aol r d e c o r a t i vdee s i g npsr i n t e do r p a i n t e do n ,o r m o u l d e d
or carvedin,the cup);or theycan be realizedby the userof the object,as whenthe cup is
h e l do r d r u n k f r o mi,n a ' t r a n s a c t i o n a lc t i o n ' w i t hi t s u s e r - t h e v e c t o r i ahl a n d l ei s t h e na
'non-transactional
action'fromthe pointof viewof the designof the cup,and a potential
for transactional
action,a means,
from the pointof viewof its use.
Reactions
can evenoccur in objects.The toy telephoneshownin figureg.4 not only
i n c l u d etsh e d o g ' st o n g u ea, s a n o b l i q u ev e c t o rs i g n i f y i nag n o n - t r a n s a c t i o ns a
p le a k i n gi t;
a l s o h a se y e sT. h e i n c l u s i o o
n f e y e si s i n f a c t q u i t ec o m m o ni n t o y sf o r y o u n gc h i l d r e n ,
particularlyin toyswith the themesof time (clocks),communication
(toy telephones)
and
transport(toy locomotives
and cars),as if to encourage
the child to form an emouve,
personalize
bd
o n dw i t h t h e s et h r e ek e yt e c h n o l o g i a
e se a r l ya s p o s s i b l e .

The third dimension

743

riq e.n Toytelephon!

e S e t t i n gT. h e i rs e t t i n g
I n c o n t r a stto t w o - d i m e n s i o nvai sl u a l ss,c u l p t u r ersa r e l yi n c l u d a
g
a
l
a,
erya
, n i c h ei n a c h u r c ho, r a p u b l i c
i s t h e e n v i r o n m e ni nt w h i c ht h e ya r e d i s p l a y e d
canincludea Setting,as in the
sculptures
Setting.0fcourse,
square.It is not a represented
gardens,
z n d G e o r g eS e g a lf,o r i n s t a n c eo,r A s i a ns c u l p t u r e
w o r k so f E d w a r dI ( i e n h o l a
s
c
u
l
p t u rteh e
B.u t i n c o n t e m p o r a W
r ye s t e r n
s u c ha st h e T i g e rB a l mG a r d e nisn S i n g a p o r e
to
counteract
it seems,has
inclusionof a Settingis relativelyrare.Decontextualization,
often haveanother
do, however,
Sculptures
the addednaturalismof the third dimension.
pedestals
participant,the pedestalon which they stand.Such
can be (mere)framing
and so
the sculptureand its environment,
devices,
between
creatinga degreeof separation
its
from
set apart
enhancing
its statusas a representation,
an objectfor contemplation,
part
as
of the representation, in
But theycanalsoandat the sametime form
environment.
(
Canova's
PaoIi na Borghese1805), whichhasPaolinarestingon a couchwhichformsalso
a s a r c o p h a g u s - lsi kuep p o rfto r h e rr e c l i n i nbgo d yT. h ea b s e n coef s u c hf r a m i n gc a nh a v ea
corpulentman in a raincoatand
strongeffect,as in the lifesizebronzeof a middle-aged,
h a t p l a c e da s i f m i n g l i n w
g i t ht h es h o p p e ri sn t h e m i d d l eo f t h ef o o t p a t ho f a b u s ys h o p p i n g
streetin Amsterdam.
these,
Turningnowto 'conceptual'ratherthan 'narrative'structuresof representation,
(I970),
in
is
what,
in figure8.5,
reproduced
too,canbefoundin sculpture.Mir6's Woman
j
u
s
t
p
l
a
y
w
i
t
h
e o e sn o t
c h a p t e3
r , w e c a l l e da n ' a n a l y t i c a l ' r e p r e s e n t a tT
i ohne.s c u l p t u r d
the
windscreen
and a mouthof
eyesof the headlights
the formsof foundobjects,mal<ing
o f a c a r ;i t i s a l s oa n ' a n a l y s i s ' o' fW o m a n ' . ' W o m a ni n' , a l l i t s g e n e r a l i tiys,t h e ' C a r r i e r ' ,
n :h e a dw h i c hi s a l s o
v et t r i b u t e sa' ,r e ,i n M i 1 6 ' sc o n c e p t i o a
a n dt h e p a r t s t, h e ' P o s s e s s i A
the emptyshellof a car;an upperbodywhich is also a tray on which two aggressively
pointedbreastsare presented
to the viewer;anda lowerbodywhich is a barrel-shaped
l i t a n d t w o h a n d l e st o h o l d h e r b y .W h e nw e v i s i t e da n
c o n t a i n ew
r i t h a v a g i n a - l i l <s e

244

The third dimension

fig A.S Woman(JoanMi16,1970)(Parellada,Barcelona)

e x h i b i t i oonf M i r 6 ' ss c u l p t u r ewse n o t e dt h a t t h e m i s o g y n i s tqi cu a l i t yo f M i r 6 , s ' a n a l y s i s ,

w a s n o t l o s to n t h e v i e w e r sT. h e r ew a s a g u e s tb o o ki n w h i c ht h e v i s i t o r so f t h e g a l l e r y
c o u l dw r i t e d o w nt h e i r i m p r e s s i o nlsv.a n y h a d u s e dt h e o p p o r t u n i t tyo d r a w a q u i c k
c a r i c a t u ro
e f a M i 1 6w o m a nw i t h a c o n t e m p t u o u
s m m e nst u c ha s ' w o u n r u ? ? ?T
co
?h
, ese
readings
contrasted
sharplywith the artspeakin the catalogue,
whichdescribed
the formal
q u a l i t i eos f t h es c u l p t u r eosn l ya, n dd i dn o td w e l lo n M i r 6 ' sw a yo f r e p r e s e n t i n
t hgef e m a l e
gender.
Giacometti'sHour of the Traces(1930), shownin figure 8.6, is an analysisof the
g e n d e r l e shsu m a nb e i n go, f t h e h u m a nc o n d i t i o n
l . h ep a r t s :a k i n do f a n t e n n a ,
i n g e n e r aT
with an abstracteye,formsan activesensorytentacleand protrudesfrom the sculpture
at an angle,constitutingan obliquevector;a rigid rustyframe,thebody;and,within the

The third dimension . 245

1930)(TateGallery)
fis g.e Hour of the Traces(AlbertoGiacometti,

246 .

The third dimension

openframe/a plasterheart,suspended
on a thin string,andmovingslightlyto andfro. Thus
t w o a c t i o n sa r ee m b e d d ei d
n t h ea n a l y s i sb,o t hn o n - t r a n s a c t i o n a l :at hc e
t i o no f t h e s e n s o r y
apparatus,
and the movementof the heart- the humanbeingas a skeletalframethat is
a l i v ea n ds u r v e yist s e n v i r o n m e n t .
- for
We mightadd that analyticalsculptureis usednot only in art but also in science
instance,
to showthe constructionof a molecule;or as a teachingaid, for instancein
anatomy,in whichcasethe partscanoftenbe detached
from the whole.Thekineticdesign
of sculptures
and otherobjects,the way they can moveor be madeto move,takenapart
a n d p u t b a c kt o g e t h ear g a i na, n ds o o n ,i s a s u b j e ctto w h i c hw e c a n n o d
t o j u s t i c ei n t h i s
chapter,
as it wouldagaindemandthe introduction
of a newset of concepts(but seevan
Leeuwen
and Caldas-Coulthard,
2004).
Thethird dimension
createsan additionaloptionin representation,
a relationbetween
the representational
structureandthe positionof the viewer.
Seenfrom the sideEpstein's
Jacoband theAngel(figure8.1) hasa narrativestructure('transactional
action').It is in
t h e f i r s t p l a c e a b o u t w h a t J a c o b a nAd nt hgee l d o . B u t iwf e l o o k a t t h e A n g e
f rlo m b e h i n d
( f i g u r e8 . 7 ) ,w e a r ef a c e dw i t h a n ' a n a l y s i so'f t h eA n g e la, n da v e r ys t r i k i n go n e :t h et h r e e

fiS g.Z Jacoband the Angelfrom behind(JacobEpstein,1940)

(GranadaTeteyision
Ltd)

The third dimension

247

v et t r i b u t e sE' p s t e i n
, i sw i n g sp r i n c i p a' P
l o s s e s s iA
e m p h a s i z ea sr et h e A n g e l ' sl o n gh a i r h
a n dh i sb a l l s .
N o t a l l s c u l p t u r eus s et h i s p o s s i b i l i t y . 0 ncea n i m a g i n ea c o n t i n u u mr u n n i n gf r o m
imagesonly in terms of modality,to
reliefs,which perhapsdiffer from two-dimensional
'multifaceted'
fully
sculpturessuch as Jacob and the Angel. ln betweenthere are
and
o b es e e nf r o m b e h i n d
s c u l p t u r ewsh i c h , t h o u gf h
r e e - s t a n d i nagr ,ec l e a r l yn o td e s i g n et d
p
l
a
c
e
d
w
all
a
a
g
a
i
n
s
t
s e c a u steh e yw e r em e a n t o b e
l e a v et h e b a c k ' u n w o r k e dp' e
, r h a pb
J
a
c
o r i n a n i c h eA. n d e v e nw h e na s c u l p t u r ies a f u l l y m u l t i f a c e t erde p r e s e n t a t i o n , l i k e o b
can blocl<accessto alternative
and the Angel,its placementin a particularenvironment
viewingpositions,and henceto alternativereadings.This may be becausethe work is
placedwith its bacl<againstthe wall,or because
the viewerfrom access
abarrierprevents
p
o
s
i
t
i
o
n
s
.
B u t i t m a y a l s ob e d o n ei n s u b t l e r
t o o t h e rt h a n m o r eo r l e s sf r o n t a lv i e w i n g
ways.Whenwe first analysedJacoband the Angel,it was placedin the centreof the
octagonalentrancehall of the TateGallery,in sucha waythat the viewerfirst sawit from
the side,with the Angelon the left. In otherwords,its positionfavouredthe narrative
'analysis'of the Angel.
ratherthan Epstein'sstril<ing
reading,
the dramaof the sculpture,
to the othersides,asthe sculpturewas placedin the centre
But the viewerdid haveaccess
o f t h eh a l .
s .h e y
P l a y m o b i l f i g u r seus c ha st h o s es h o w ni n f i g u r eB . Ba r ea l s oa n a l y t i c aslt r u c t u r e T
'ethnic
(for
an
example)
of
showthe significantattributes,the significantcharacteristics,
r n dt h r e ec h i l d r e nE. a c hm e m b e r
f a m i l y 'T. h ef a m i l yh a sf i v em e m b e r-s a f a t h e ra, m o t h e a
o f t h ef a m i l yh a sb l a c kh a i r a n dd a r ks l < i nN. o t et h e d i f f e r e n cfer o mt h e n e u t r a l l Iya b e l l e d
' f a m i l ys e t ' :t h e c o m p o s i t i oonf t h e f a m i l yi s t h e s a m ea, n da l l t h e m e m b e ros f t h e f a m i l y
h a v ep i n k s k i n ,b u t t h e yd i f f e r i n t h e c o l o u ro f t h e i r h a i r a n d t h e r e f o r eh a v ei n d i v i d u a l
c sh,e r e a(sl e s s o n u m b eor n e )t h em e m b e r s
c h a r a c t e r i s t iacssw e l la ss o c i acl h a r a c t e r i s t i w
o f t h e ' e t h n ifca m i l y ' a r e ' o t h e r s ' w h o ' laol o
l kt h es a m e 'T. h ec h i l d r eonf t h et w o f a m i l i e s
are dressedidentically,
but the parentsare not. Lessonnumbertwo: second-generation
s a y st h a t t h e s e
y rochure
i m m i g r a n tasr e a l r e a d y ' m o r el i k eu s ' .T h e P l a y m o b icl o m p a n b

f i g e . e P l a y m o b i l ' f a m isl ye t ' a n d ' e t h n ifca m i l y '

248

f he third dimension

t o y s ' f o r ma n a i dt o t h et r a i n i n go f y o u rm i n d s ' a n dw i l l ' a c q u a i nct h i l d r e w

n i t h w h a tt h e y
w i l l m e e ti n t h e b i g r e a lw o r l d ' - b u t n o t i n a n e n t i r e l yn e u t r afla s h i o nA. s w e h a v es a i di n
chapter3, any analyticalstructureis onlyoneof the manywaysin whicha given'carrier,
can be analysed.
We haveto remember,
of course,
that Playmobilfiguresenterinto representation
in two
w a y sl,i k e o u r e a r l i e r e x a m p lceuopf s w i t h v e c t o r hi aal n d l e(sf i g u r e 8 . 4 . 0 n t h e o n e h a n d ,
theyare likesculptures,
pre-designed
representations,
to be'read'by the child;on the other
hand,they havemovablelimbsand detachable
parts and they can hold objectsin their
hands.Childrencan thereforeusethemto createa varietyof representational
structures,
narrative'scenes',
andtheycanevensubvertthe pre-designed
representations,
for instance
b y g i v i n ga n ' e t h n i c ' c h i l d r e d h a i r .T h e yc a n a l s o c r e a t et h e i r o w n c l a s s i f i c a t o royr
a n a l y t i c aal r r a n g e m e nftos r, i n s t a n cbey m a k i n ga d i s p l a yo f d i f f e r e nkt i n d so f P l a y m o b i l
children
o n t h e i rt o y s h e l fo r b y c r e a t i n ga n e wa n a l y s ios f t h e ' f a m i l y 'w
, i t h o n l ya m o t h e r i
perhaps,
(thearrangement
or with fivechildrenof different'ethnic'origins
of )ur Society
and 0thers,figure
3.29,couldbe reconstructed
with Playmobilfigures!).lnthe sameway
a c u pc a nb e u s e dn, o t o n l yi n t h e ' t r a n s a c t i o nosf' h o l d i n go r d r i n k i n gb, u t a l s o :t o c r e a t e
an analyticalstructure,
aswhenthe cup is arrangedon a sideboard
togetherwith the other
p a r t s o f t h e s e t t o w h i c h i t b e l o n g so,r o n t h e k i t c h e ns h e l f t, o b e c o m ea ' p o s s e s s i v e
Attribute' of the 'carrier' 'dishes';or to createa classificational
structure,as when a
n u m b e ro f d i f f e r e n ct u p sa r e a r r a n g e ds y m m e t r i c a l li yn a s h o pw i n d o wo r i n a d e s i g n
e x hi bi t i o n .
The vitrinen (displaycases)of the Germanavant-garde
artist JosephBeuysare an
i n t r i g u i negx a m p l o
e f t h ec l a s s i f i c a t i o nsaclu l p t u r eT.h e ya r et h e k i n do f s c u l p t u rw
e h i c hi s
u s u a l l yr e f e r r e tdo a s a n ' i n s t a l l a t i o n ' g- l a s sd i s p l a yc a s e sc o n t a i n i nagv a r i e t yo f o b j e c t s ,
someof them alteredby Beuys.vitrine 2 (1960-7o, for instance,
containsa film can
(itselfcontaining
a film whichfeaturesa performance
by Beuys),
a pair of boxinggloves,
a
sausage/
a cassetteplayer(with a tape of music performedby Beuys),a Beethoven
score
(onecoatedwith a whitesubstance,
with a smallblackboard
eraseron it, two wineglasses
t h e o t h e rl o o k i n ga s t h o u g hs o m e t h i nhga s b e e nb u r n t i n i t ) , a n d a z i n c b o x .c l e a r l ya
work like this raisesthe questionof what theseobjectshave in common(what their
'superordinate'
is, in our terms),and this is exactlythe questionBeuys,interpreters
have
('relatedness,)
asked.Accordingto Theewen(1993: r39), verwandschaft
is the key to
understanding
Beuys' vitrinen: 'By bringingrelated objectstogetheran association
betweenthem is created',and in the caseof the vitrine we havejust describedthis
association
is that all the objects'haveplayeda role in performances
of Beuys,and tthat
t h e y la l l c o n t a i ns o m e t h i n gi 9' 9 3 : 2 9 ) .
Thethird kindof conceptual
structurediscussed
in chapter3 wasthe symbolicrelation.
w e s a wh o w i n p i c t u r e sa n o v e r a lcl o l o u r- a b l u eh a z eo, r a g o l d e ng l o w- c o u l dr e a l i z e
w h a t w e c a l l e da ' s u g g e s t i v e
s y m b o l i c ' p r o c e sesn, d o w i n g
t h e d e p i c t e ds c e n ew i t h a n
o v e r a lsl i g n i f i c a n c e , ' s a y i na gs 'i,t w e r e , ' t h i ss c e n ei s c o l d a n d d e s o l a t e(,i n t h e c a s eo f
t h e b l u eh a z e )o r ' t h e s eo b j e c t a
s r ev e r yv a l u a b l e( 'i n t h e c a s eo f t h e g o l d e ng l o w ) .C o l o u r
c a n p l a yt h i s r o l ea l s oi n t h e c a s eo f t h r e e - d i m e n s i o o
nb
a jl e c t s- t h i n k o f t h e c o l o u r so f
cars,for instance,
the differencebetweena black,a bright-redand a white Mercedes,
sav.

The third dimension

249

But in additionto colourthereare otherfactors,suchasthe materialfrom whichan object

is made,the waythe surfaceof a sculptureor otherobjectis 'worked',or the overallshape
of naturalistic
of the objects,in so far as theseare not determinedby considerations
(1947)
Pointing
Man
representation,
or by the functionsservedby an object.Giacometti's
p
l
a
c
e
s
n
o
e
m
phasis
f
a
r
d
i
s
c
u
s
s
e
I
d
t
.
i s m u c hl e s s ' a n a l y t i c a l ' t htahnes c u l p t u r ewse h a v es o
hardly
parts
are
features
facial
of the humanbody.Eventhe
on the distinctness
of the
d o e so f c o u r s e
s t a t e d- s m a l l i n d e n t a t i o nt os i n d i c a t ea m o u t ha n d e y e sT. h e s c u l p t u r e
gesture.
The most
oratorical
have a clear vector,as the man is makingan expansive
surface.
It is
craggy
however,
is its rough,black,
strikingcharacteristic
of this sculpture,
whatever
put
but
surface,
difficult to
into words exactlywhat is suggestedby this
transcodingwe attempt it will haveto expresssomehowthat this figure is'weatherto - but herewe canfill in a numberof things- the elements
beaten',affectedby exposure
s u f f e r i n ga,g e i n ga n d s o o n .T h es c u l p t u r e
i s a b s t r a cet n o u g ht o a l l o wa l l t h e s er e a d i n g s
a n dm o r e .
objects.Cupscan be
The same applies,again,to other l<indsof three-dimensional
quality
perhaps,
and
of elegance
smooth,madeof delicatechina,suggesting,
an overall
suggesting,
refinement;
or they can be sturdyand solidand madeof brick-redterracotta,
perhaps,an overall quality of down-to-earthsimplicity.Cars can be elongatedand
powerand speed,or, as in the caseof the currentlyfashionable
streamlined,
suggesting
' r e t r o 'l o o k ,r o u n d e a
d n de g g l i k es,u g g e s t i nags a f ec o c o o na, l < i n do f w o m b .
cttributive'proT h es e c o n dk i n do f s y m b o l i rce l a t i o nw e d i s c u s s ewda st h e ' s y m b o l i a
participanthasno otherfunctionthanto endowanotherwith
cess,whereonerepresented
s ,h e r eb i r d s
s y m b o l isci g n i f i c a n cTeh. i so c c u r sf,o r i n s t a n c ei n, s o m eo f M i r 6 ' ss c u l p t u r ew
(
u
s
u
a
l
l
y
'
w
o
m
e n 'B
) .u t i t
q
u
a
l
i
t
i
e
s
a n d e g g sa t t r i b u t es y m b o l i c
t o t h e f i g u r e sd e p i c t e d
(
a
n
e
a
r
l
y
l
esson
o c c u r sa l s oi n t o y sf o r y o u n gc h i l d r e nw, h e r et e l e p h o n ecsa n h a v ew h e e l s
t
h
e
a b o u tt h e c o n c e pot f c o m m u n i c a t i oans ' t r a n s p o rot f i n f o r m a t i o n ' a n d ' b r i d g i n g d i s , d' i n t e r a c t i vlee a r n i n cge n t r e s ' f ocr h i l d r e n
t a n c e 'r a t h e rt h a n' s h a r i n go f i n f o r m a t i o n 'a) n
('6
colours,
agedzt/zto 5
built-infunctionswhichteachthe alphabet,numbers,shapes,
wheel
and
soundeffectsand nurseryrhymes',and all this for 829.50) havea steering
power
in
adult
toys:
0r
dashboard,
as a symbolof the
and controlaffordedby knowledge.
e ,h e r es c u l p t u r e s
e m b l e mas n do t h e rd e c o r a t i o nosn c a r s /f o r i n s t a n c e . 0irn a r c h i t e c t u rw
a n dm u r a l sc a nb e c o m se y m b o l iac t t r i b u t efso r b u i l d i n g s .
we havepresented
0n the whole,then,we feelthat the accountof visualrepresentation
i n c h a p t e r 2s a n d3 c a n b e a p p l i e dt o t h r e e - d i m e n s i o vniaslu a cl o m m u n i c a t i oYne. tt h e r e
First,three-dimensional
objectscan be placedon a conare somesignificantdifferences.
t i n u u mw h i c hr u n sf r o m o b j e c t st h a t a l l o wo n l yo n er e a d i n g( b y o f f e r i n gt h e r e a d e ro n l y
o n ea s p e c tu, s u a l l tyh ef r o n t )t o o b j e c t w
s h i c ha l l o wm o r et h a no n er e a d i n gd,e p e n d i nogn
position
the
of the viewerrelativeto the object.
S e c o n dt,h r e e - d i m e n s i o noablj e c t sc a n b e p l a c e do n a c o n t i n u u mw h i c h r u n sf r o m
onlyto be lookedat, onlyto be 'read',to objectswhich
objectswhichhavebeendesigned
enterinto representational
relationsin threeways:(1) the relationsencodedin the design
'read'
of the objectitself,to be
only by the viewer;(2) interactiverelationsbetweenthe
(e.g.
relations
objectand ils user
holdingthe cup,or drinkingfrom it); and (3) conceptual

250

The third dimension

createdby the user(e.9.creatinga classificational

syntagmwith a numberof different
CUPS).

Third,even
whenan objectdoeshavea potentialfor multifaceted
representation
and/or
f o r b e i n g ' u s e d ' a sw e l l a s ' r e a d ' ,e x t e r n acl o n d i t i o ncsa n i n h i b i tt h i s p o t e n t i a lb, l o c k
the viewer'saccessto alternativereadingpositions,
or to interactive
engagement
with the
potentialof the object.
representational

INTERAGTIVEVIEWING
We will nowturn to the interactive
relationswe discussed
in chapter4,trying to explore,
a g a i n ,h o w a p p l i c a b l e
t h e y a r e t o t h r e e - d i m e n s i o nvai sl u a lc o m m u n i c a t i o n
I n. t h a t
chapterwe distinguished
between'demand'picturesfrom whichrepresented
participants
a d d r e s tsh e v i e w e rd i r e c t l yw i i h t h e i r g a z ea n d ' w a n t s o m e t h i nfgr o m t h e v i e w e r ,a, n d
'offer' pictureswhich positionthe viewer
as an observeronly,and offer the represented
p a r t i c i p a n tass ' i n f o r m a t i o n ' t bo et a k e ni n b y t h e v i e w e r .
C l e a r l yt h, i sd i s t i n c t i ocna n b e a p p l i e da l s ot o s c u l p t u r-e b u t ,a g a i nw
, itha difference.
Henry Moore's RecumbentFigure (7938), shownin figure 8.9, addresses
the viewer

fig a.e RecumhentFigurc(HenryMoore,1938)(Tate

cailery)

The third dimension . 25I

in the surfaceof the stone,

powerfully.
Althoughthe eyesare little morethan indentations
we cantake
look.But,as viewers,
a concentrated
thewholeattitudeof the figuresuggests
(
a
s
p
h
o
t
o
g
raphe
i nrt,h e
d i dt h e
u p a p o s i t i o fnr o mw h i c ht h a t l o o kw i l l d i r e c t l ya d d r e suss
p
o
s
i
t
i
o
n
f
r
o
m
w
h
i
c
ht h e
c a s eo f f i g u r e8 . 9 ) ,s o t h a t t h e p i c t u r ef o r m sa ' d e m a n d ' ;o r a
particular,
something
in anycase,at
else,or at nothingin
figurelookspastus,at something
n o t i n c l u d e di n o u r v i e w ,a n d i n t h a t c a s et h e l o o k w i l l b e c o m ea ' n o n - t r a n s a c t i o n a l
decidewhetheror not we
mediumwe cannot,as viewers,
reaction'.In the two-dimensional
p
a
r
w i l l a l l o wo u r s e l v et os b e d i r e c t l ya d d r e s s ebdy a r e p r e s e n t e d t i c i p a ntth; e d e c i s i ohna s
b e e nm a d ef o r u s .I n t h et h r e e - d i m e n s i om
n aeld i u mw ec a n- t h a t i s ,i f t h ep l a c e m e notf t h e
sculptureallowsusto do so.In the TateGallery,Moore'ssculpturecouldhavebeenplaced
uponenteringthe
in sucha way that the figure'sgazewouldfix the viewerimmediately
t h e s c u l p t u rteh e r ea, n da s a r e s u l t h ef i g u r e
r o o m .B u tt h i sw a sn o td o n ew h e nw e v i e w e d
just
as part of a classificational
one of a numberof Moore'sworks,presented
became
s y n t a g ma,n df a v o u r i ntgh e ' o f f e r 'r a t h e rt h a nt h e ' d e m a n d ' .
T h es a m ew o u l db et r u eo f t h et o yt e l e p h o n(ef i g u r e8 . 4 ) .T h eg a z eo f t h i st e l e p h o ncea n
,f
o n l yb e c o m ae ' d e m a n db' yv i r t u eo f a n a c t i v ed e c i s i oonn t h ep a r to f i t s u s e rS. o m et o y s o
. l a y m o b icl h a r a c t e rhsa v es m a l lb l a c k
c o u r s el ,e n dt h e m s e l v emso r et o t h i st h a n o t h e r sP
dots for eyes.They are biasedmore towardsthe'offer for informatlon'thantowards
, rash-dummies,
t h e i n t e r a c t i v e ' d e m a nA
dn
' . d t h e e y e so f m a n y ' b o y s " d o l l s( B a t m e nC
glasses.
Theeyesof 'girls' '
masksor darl<
Megazords,
etc.)are oftenobscured
by helmets,
(
a
n
d
dolls
o f m a n yc u d d l ya n i m a lt o y s ) ,o n t h e o t h e rh a n d t, e n dt o b e l a r g ea n d h i g h l y
d e t a i l e dW
. h i l eb o y sa r e s t e e r e tdo w a r d sa m o r em a n i p u l a t i vree l a t i o nt o t h e i r d o l l s f, o r
g i r l st h e l o o k ,t h e i n t e r a c t i vdei m e n s i o ni s, m a d et o m a t t e rm o r e .A n d t h e s a m ei s t r u e
f o r v e r yy o u n gc h i l d r e ne:v e nt h e i rb e d c l o t h epsi,l l o w c a s ecsu,p sp/ l a t e sm a yh a v ee y e sa, n d
a r e t h u sp e r s o n a l i z eadn,i m a t e dc,a p a b l eo f e n t e r i n gi n t o a ' d i r e c t a d d r e s s ' r e l a t i owni t h
l yt,h e
the childA
. s w i t h m a n yo t h e rt h i n g ss, o m eo f t h i s m a yw e l l l i v eo n ,u n c o n s c i o u si n
a d u l tr e l a t i o nw i t h o b j e c t s .
d
T h es a m er e a s o n i ncga nb e a p p l i e dt o t h e o t h e ri n t e r a c t i vdei m e n s i o nwse d i s c u s s ei n
c h a p t e4
r . In principle
t h e v i e w e rc a n d e c i d ew h e t h e tro s e et h e o b j e c tf r o m c l o s eu p o r
with
a n g l e( h e n c e
f r o m a d i s t a n c ef r,o n t a l l y( h e n c e
w i t h ' i n v o l v e m e n to' r) f r o m a n o b l i q u e
'detachment');
(hence
from above
from a positionof poweroverthe object)or from below
( h e n c fer o ma p o s i t i o n
r v e rt h ev i e w e r )W
. es a y ' i np r i n c i p l e ' ,
i n w h i c ht h eo b j e c h
t a sp o w e o
because
here,too,theviewer'schoicemaybe restrictedby externalfactors,by barriersthat
preventviewersfrom comingup closeor seeingthe objectfrom a differentangle.And large
e .h a tt o w e r s
o b j e c t cs a nm a k et h e h i g h - a n g vl ei e w p o i natn dt h ec l o s ed i s t a n c iem p o s s i b lW
over us has,by design,powerover us, and is, by design,sociallydistant:the vertical
'highly
d i m e n s i oins t h e d i m e n s i oonf p o w e ra n d r e v e r e n t i adli s t a n c et h, e d i m e n s i oonf
s n dt h i n g sI.n t h i sc o n n e c t i oi n
p l a c e d ' p e o p lpel ,a c e a
t i s a l s os i g n i f i c a nt ht a t s c u l p t u r eass,
'
h
i
g
h
'
a
r
t
,
c a n n o tu s u a l l yb e a p p r o a c h efdr o m t h e m o s ti n t i m a t ed i s t a n c et h, e
w o r k so f
distance
t h a t m a k e st o u c h i n gp o s s i b l ea:s s o o na s t h e g a l l e r yv i s i t o rc o m e st o o c l o s ea,
g u a r dw i l l b e c o m a
elert.
W h e ns c u l p t u r easr e t a k e no u t o f t h e i r o r i g i n acl o n t e x at n d m o v e di n t oa n o t h e tr h, e i r
tal<endownfrom the
Theymay be,literally,
may changesignificantly.
interactive
meanings

252

The third dimension

pedestal- in a churchperhaps- wherethey were to be lookedat from below,with

reverence/
to be movedinto a gallery,
wheretheyare positioned
at a levelof equality,and
viewed
f r o ma m o r e ' f a m i l i a r ' d i s t a n cMei:c h e l a n g e l oD' sa v i dr,e m o v etdo t h e r o t u n d ao f a
m u s e u mn/o l o n g e cr a l l st o t h e c i t i z e n os f F l o r e n caen di s u n a w a r e
o f t h e i rc a l l i n go n h i m ,
a n d n o w ' c a nb e e x p l o r e b
d y t h e v i e w e rb,u t m a k e sn o a d v a n c et so h i m ,( A r n h e i m1, 9 8 2 :
50; also Hodgeand l(ress,1988: 2OI-3).

M O D A L I T YI N T H R E ED I M E N S I O N S
I n c h a p t e5
r w e d e s c r i b evdi s u a lm o d a l i t ya s r e s u l t i n fgr o m t h e d e g r e e
t o w h i c hc e r t a i n
l x p r e s s i o(nc o l o u rr,e p r e s e n t a t i o ndaelt a i l ,d e p t h t, o n a l s h a d e se,t c . )
m e a n so f p i c t o r i a e
are used.Eachof thesedimensions
can be seenas a scale,runningfrom the absenceof
any renditionof detailto maximalrepresentation
of detail,or from the absenceof any
rendition
o f d e p t ht o m a x i m a l l d
y e e pp e r s p e c t i vAen. d o n e a c ho f t h e s es c a l e st h e r ei s a
p o i n t h a t r e p r e s e ntthsew a yt h e g i v e np i c t o r i adl i m e n s i oins u s e di n w h a tc o u l db ec a l l e d
standardnaturalism.To the degreethat the use of a dimensionis reduced,it becomes,
at leastin onerespect,
moreabstract,'less
than real'.To the degreethat it is amplified,it
'more than real',and we associated
becomes
this with a 'sensorycodingorientation',
an
e m p h a s iosn s e n s o r yp l e a s u r (eo r d i s p l e a s u raes, i n t h e c a s eo f ' m o r et h a n r e a l ,h o r r o r
images),
and an attemptto comeas closeas possible
to a representation
that involves
all
the senses.
S o m eo f t h e s ep l a ym u c ht h es a m er o l ei n t h r e e - d i m e n s i ovniaslu a l sC. l e a r l ys,c u l p t u r e s
and toys can represent
what they representin naturalisticdetailor moreabstractly.
And
when the shapeof everydaydesignobjectsno longerbetraystheir function,when,for
instance,
refrigerators,
washingmachines
and kitchenstoragecabinetsall becomesleek
featureless
whiteboxes,
thereis alsoa strongsenseof abstraction.
Thesamecanbeargued
f o r b u i l d i n g sA. l s o ,l i k e p i c t u r e st,h r e e - d i m e n s i o nr eapl r e s e n t a t i ocnasn i n c l u d es e v e r a l
levelsof modality.The headsof HenryMoore'sKing and Queen(1952-3), for instance,
s h o w ni n f i g u r e8 . 1 0 ,a r e a b s t r a cst y m b o l sw, h i l et h e i rh a n d sa r e r e n d e r eidn n a t u r a l i s t i c
detail.Thisexpresses
the contradictory
natureof the powerful.Theirmindsmay havelost
touchwith the detailof everyday
concretereality,but lool<at their hands- they are after
all still humana
, n d t h e i r w o r k ,t h e i r d o i n g ,i s s t i l l t h e w o r k o f h u m a n sM
. a c h i na n d
S u l e i m aQ
n 0 0 4 ) h a v ep o i n t e do u tt h a t i n A m e r i c a n
c o m p u t ewr a r g a m e st h ew e a p o n r iys
represented
in realisticdetail,whilethe settingshavelowermodality,forminga generic
d e s e r t h a t c o u l db e a n y w h e r eT.h i sf o r e g r o u n dAsm e r i c a nt e c h n o l o g i csaul p r e m a cayn d
backgrounds
the specifics
of specificconflicts.In a Lebanese
computerwar gameproduced
by Hezbollah,
the landscapeis reconstructed
from photographs
of the sites of actual
conffictsand represented
in moredetail.Herethe specifics
of historicaland geographical
accuracymatter.
The representation
of detail in toys is particularlyinteresting.Barthes'still highly
r e a d a b l e s s a ya b o u tt h e s e m a n t i cosf t o y s ( r 9 7 3 : 5 3 f f . ) i s n o w p e r h a p so v e r t a k e b
ny
s e m i o t ie
c v e n t sH. ed e s c r i b eFsr e n c ht o y sa s h i g h l yd e t a i l e dh,i g h l yn a t u r a l i s t im
c iniature

The third dimension . 253

fig e.fO King and Queen(Henry

Moore,1952-3)(Tatecallery)

254

The third dimension

versions
of adultobjects- and noteshowunsatisfactory
theyare,therefore,
bothfrom the
p o i n to f v i e wo f p l e a s u r eo,f t h e s e n s o r d
y imension
a ,n d f r o m t h e p o i n to f v i e wo f t h e i r
i n t e r a c t i vpeo t e n t i a al ,s o b j e c t tso p l a yw i t h : ' t h ec h i l dc a no n l yi d e n t i f yh i m s e laf s o w n e r ,
as user/neveras creator;he doesnot inventthe world,he usesit: thereare,preparedfor
h i m ,a c t i o nw i t h o u ta d v e n t u r w
e ,i t h o u tw o n d e rw, i t h o u tj o y ' ( I 9 7 3 : 5 4 ) . T o y so f t h i s k i n d
c a nb e s e e ni n m u s e u mssu c ha s L o n d o n ' M
s useum
o f C h i l d h o o d . 0 n ef i'rss t i m p r e s s i oonn
seeingVictoriantoys is that childrenare addressed
as miniatureadults,their subjectivity
' s c a l e dd o w n ' b
, u t n o t ' r e d u c e df'r,o mt h a t o f t h e a d u l tw o r l d .T h i si s r e p r e s e n t et hdr o u g h
a l a r g en u m b e o
r f s e m i o t i cm o d e st:h e m a t e r i a l (sg l a s sc, l o t h e so f v a r i o u sk i n d s m
, etals,
etc.- all of them rarely,if at all, usedin contemporary
toys) as muchas the miniature
naturalismof the representedobjects.Today'stoys vary in their detail. Toys for
youngchildrenare abstract.Shapesand texturesare broughtdownto their essentials.
Locomotiveshave featureless
wheels,one featurelesschimney/two yellow circlesfor
windows.Theyare,from the point of view of detail of representation,
like the simplest
l i n ed r a w i n g sA. s t h e c h i l dg e t so l d e rd, e t a i li n c r e a s eTs h
. ew h e e l so f t h e l o c o m o t i v egse t
s p o k eas n dd r i v e s h a f tTsh. et e x t u r eo f t h e m a c h i n e 'bso d ya c q u i r edse t a i l H
. e a d l i g hw
t si t h
miniatureFresnellenses
areadded.But on thewholethe contempo
rary toy remainssimple,
essentialized,
as for instance(again)in the popularPlaymobilfigures,wherethe eyesare
two dots,the mouth a curvedline,the hair an almost featureless
helmet,with a few
i n d e n t a t i o ns su g g e s t i nt e
r . i r l s ' d o l l so, n t h e o t h e r
g x t u r ea n dt h e l e n g t hs i g n i f y i ngge n d e G
h a n d( a n do t h e rg i r l s ' t o y s r: e a l i s t i cw a s h i n gm a c h i n e s , ' b e a usthyo p s 'v, a c u u mc l e a n e r s ,
d o l l s ' c o t s )i ,m i t a t et h e a d u l tw o r l d ,o r a t l e a s t h a t o f t h e g l a m o u g
r i r l ,t h e h o u s e w i faen d
the mother,muchmoreso than boys'toys,whichdepicta make-believe
world of sciencefictionvehiclesand weapons,
or a world of dinosaurs
and other monsters.
The latterare
o f t e n ' m o r et h a n r e a l ' ,w i t h h i g h l yt e x t u r e dg, l i s t e n i nsgc a l e si,r r e g u l a tre e t h ,a n d m e n acing eyes,set behindwrinkled lids.They are designedto createthe kind of sensory,
visceralreactionalso sought,for instance,
in horrorfilms.Cuddlyanimaltoys alsotend
to be'more than real',with exaggeratedly
soft furs and large moist eyes,this time to
e n h a n c et h e s e n s o r yp l e a s u r eosf h o l d i n ga n d t o u c h i n g R
. e s e a r c h i nt hgi s c h a p t e rw, e
s p e n tm a n yh o u r si n t o y s h o p sa s w e l la s i n t o y m u s e u m a
s ,n dc o u l dn o t h e l pb e i n gs t r u c k
by the contrastbetweenthe 'bourgeois'naturalistictoys Barthesdescribed
and today's
makebelieveworld of brightlycolouredplasticand creaturesand objectsfrom fantasy
stories.
T h e r o l e o f c o l o u r i n t h e m o d a l i t yo f t h r e e - d i m e n s i o nv a
i sl u a lr e p r e s e n t a t i oanl s o
r e s e m b l et hs a t o f t h e t w o - d i m e n s i o nvai sl u a l sS. o m eo f M i r 6 ' ss c u l p t u r eosf ' w o m e n ,f,o r
instance,
are paintedin bright,primarycolours- yellow,blueand red.As a resulttheyare
s c h e m a t iacn d a n a l y t i c aflr o m t h e p o i n to f v i e wo f r e p r e s e n t a t i o ndaelt a i l ,a s i m p l i f i e d ,
a b s t r a cvt i e wo f ' w o m a n 'b, u t ' m o r et h a nr e a l ' , ' s e n s o r y ' f r otm
h e p o i n to f v i e wo f c o l o u r .
M i 1 6 ' sw o m e na r e n o t j u s t m a c h i n e - l i kaes s e m b l a g o
p
a
r
t
s
e fs
t, h e ya r e a l s op l e a s u r a b l y
c o l o u r f u(l ' w o m a n ' a sa m a c h i n feo r p l e a s u r e M
) . a n yt o y s ,e s p e c i a l tl o
y y sf o r y o u n gc h i l dren,haveexactlythe samekind of modalityconfiguration:
abstractand schematized,
'conceptual',
from the pointof viewof colour;'un-naturalistic'
primarycolours,coloursfor
t h e s a k eo f p l e a s u r rea t h e rt h a n n a t u r a l i s t irce p r e s e n t a t i oTnh.i s ' s e n s o r y ' a s p ei sc t h e n

The third dimension . 255

f u r t h e re n h a n c ebdy t h e w a y t h e s et o y sa p p e atl o a l l t h e s e n s e sa,n d i n c l u d et o u c ha n d

s o u n da, s s t r e s s eidn t h i sd e s c r i p t i oonf t h e ' C h i c c oa n i m atl r a i n ' : ' a l o c o m o t i vfee a t u r i n g
eight differentanimalsoundsand four differenttrain sounds.Soundsare activatedby
p r e s s i nagp p r o p r i a t esl yh a p e d
more
c o l o u rc a nb e c o m e
b u t t o n s . ' A g a i fno, r o l d e rc h i l d r e n
n a t u r a l i s t ia
c ,s i n T h o m a st h e T a n kE n g i n ew, h i c hr e p l a c ebsr i g h tr e d sy, e l l o w sb,l u e sa n d
g r e e nw
s i t h m o r em u t es t e e lb l u e g
, r e ya n d b l a c l <a,n dj u s t a t o u c ho f r e d .A n d ,w h i l et h e
c o l o u ri n a l l t h e s ee x a m p l ei s u n m o d u l a t etdh,ec o l o u ro f g i r l s ' d o l l sb e c o m em
s o r ev a r i e d ,
w i t h b l u s h eo
s n t h e c h e e k ss,h a d o w us n d e rt h e e y e sa, s h i n eo n t h e l i p s .A n d t h e s a m e
s n dg r e y sa n dp i n k so n d i n o s a u rasn dm o n s t e r s .
a p p l i etso t h e s l i m yg r e e n a
The materialsusedin three-dimensional
representation,
similarly,can be motivated
n a t u r a l i s t i c a l lays, w h e nt o y c a r s a r e m a d eo f m e t a l ,o r c u d d l ya n i m a l so f s o f t ,f u r r y
m a t e r i a l (st h o u g ho, n t h e o t h e rh a n d , e v etno y p i g sc a n b e f u r r y ! ) .B u t t h e yc a n a l s ob e
'lessthan real', abstractingfrom the varietyand specificityof the rangeof materials
available,
as in the caseof plastic.To quoteBarthesagain,
l f
M a n yt o y s a r e n o w m o u l d e df r o m c o m p l i c a t em
d i x t u r e st;h e p l a s t i cm a t e r i a o
all the
whichtheyare madehasan appearance
it destroys
at oncegrossandhygienic,
pleasure,
toysare chemicalin
the sweetness,
the humanityof touch.. . . Henceforth,
substance
of use,
and colour;their very materialintroduces
oneto coenaesthetics
n o tp l e a s u r e .
(I973:55)
A g a i nm
, a t e r i a lcsa nh a v e ' m o r e
t h a nr e a l ' m o d a l i t y , w h tehne i rc h o i c ei s m o t i v a t endo t b y
an attemptto makethe objectlook like what it represents,
but by an attemptto create
p l e a s u roer d i s p l e a s u rAen. dt h i sc a na p p l ya l s ot o e v e r y d aoyb j e c t sa,n dt o b u i l d i n g isf,w e
r e p l a c en a t u r a l i s m
b y a n a t t e m p t t or e v e atlh e m a t e r i aflr o m w h i c ht h e o b j e c ti s a c t u a l l y
madeand divergence
from naturalismby attemptsto concealthesein onedirection(say,
plain sheetsof a syntheticmaterialover bricl<s)or another(say,timber claddingover
bricl<s
or concrete).
In other respects,
however,
three-dimensional
modalitydiffersfrom two-dimensional
modality.Thereis no needfo represent
depth:the objectalreadyhasdepth,by virtue of
its three-dimensionality.
And thereis no needto represent
the play of light and shade:it
a l r e a d yo c c u r sn, a t u r a l l yI t. i s t o c o m p e n s aftoer t h e s ei n t r i n s i cn a t u r a l i s t iqcu a l i t i etsh a t
m o s tW e s t e r snc u l p t u r (e1 ) i s d e c o n t e x t u a l i zleadc,l < i nagS e t t i n g(;2 ) r e f r a i n fsr o m u s i n g
c o l o u ra s a m e a n so f r e p r e s e n t a t i oenx,c e p itn t h e s e n s eo f o v e r a l ls, y m b o l i c o l o u r a; n d
( 3 ) i n c r e a s i n gtleyn d st o w a r d sh i g h l yr e d u c e d
f o r m so f r e p r e s e n t a t i oI n .t h i s w a y ' h i g h
an ideal
art', which seel<s
to go beyondthe mere replicationof realityby representing
of beautyor an abstracttruth, distinguishes
itselffrom everydaysculpture- from the
d u m m i e si n s h o pw i n d o w sf ,r o m g i r l s ' d o l l sf,r o m m i n i a t u r ed i n o s a u r sa ,n d s o o n .T h e s e
neednot themselves
be naturalisticin everydimension
in any case,and may usereduced
naturalism
e i t h e rf o r d i d a c t i cp u r p o s e sa /s i n t o y sf o r y o u n gc h i l d r e no, r t o c r e a t ec l a s s
d i s t i n c t i o ni n
s t a s t ea, sw i t h t h e m o r es t y l i z e d u m m i e isn e x p e n s i vseh o p s .

256

The third dimension

C O M P O S I T I OINN T H R E ED I M E N S I O N S
Manysculptures
and otherthree-dimensional
objectsdo not clearlypolarizebetweenleft
and right,top and bottom,centreand margin,but whenthey do,the valuesof Givenand
New,Idealand Realand Centreand Marginapply,we think,in the sameway as theydo in
t w o - d i m e n s i o nvai sl u a cl o m m u n i c a t i oYne. tt h e t h i r d d i m e n s i odno e si n t r o d u caed d i t i o n a l
factors.
In figure 8.1 we saw Epstein'sJacoband the Angel as it would havebeenseenby
v i s i t o r se n t e r i n tgh e T a t eM i l l b a n kG a l l e r yw h e nt h e w o r kw a sp o s i t i o n eidn t h e c e n t r eo f
t h e g a l l e r y ' so c t a g o n ael n t r a n c h
e a l l .L i k e G o di n f i g u r e s6 . 5 a n d 6 . 9 , t h eA n g e la g a i n s t
whoseforceJacobis so helpless
is Given,andJacob'shelplessness
is New,the focusof the
drama.But movingto the otherside,theviewercanreverse
this,andmakeJacobGivenand
the AngelNew,providedaccess
to the othersideis not blocked.Thesameis true for other
m u l t i f a c e t esdc u l p t u r ews i t h t w o o r m o r e ' p o l a r i z e d ' p a r t i c i p a nstus c, ha s R o d i n , sl h e
l(iss,where,
depending
on your pointof view,eitherthe mancan be Givenandthe woman
New,or the womanGivenandthe man New.Thisis why photographs
cannotdo justiceto
multifaceted
sculptures a photocan alwaysgiveonly oneangle,and henceone reading.
Thesameis not true,however,
for Idealand Realand Centreand Margin.Theserelations
cannotbe invertedby changing
the anglefrom whichthe work is viewed.In otherwords,
t h e h o r i z o n t adl i m e n s i oanl l o w si n t e r a c t i v i ttyh;ev e r t i c adl i m e n s i oann dc e n t r a l i tdy o n o t .
I d e a la n d R e a la n d C e n t r ea n d M a r g i na r e o f t e nt h e m o s ts i g n i f i c a ncto m p o s i t i o n a l
d i m e n s i o ni sn t h r e e - d i m e n s i o nv a
s e r h a p st h e
i sl u a lc o m p o s i t i o nA.r c h i t e c t u r p
er o v i d e p
c l e a r e set x a m p l eL.e f ta n dr i g h ta r e n o t u s u a l l yp o l a r i z e dH. o r i z o n t a l tl yh e r ei s s y m m e t r y ,
but verticallythereis not.Theverticaldimension
is usedto polarize,toproducedifference,
w i t h t h e I d e a l ,t h e e l e m e n t ( st h
) a t g i v e ( s t)h e b u i l d i n gi t s m o r eg e n e r aal n d ' i d e a l ' s i g nificanceon top - the tower,for instance,
with its significantemblems,
the cross,or the
c l o c kt,h e g a b l es t o n eo n A m s t e r d a m
c a n a lh o u s e st h, ef r i e z eh i g hu p o n t h e G r e e kt e m p l e .
Below,on the otherhand,is the spaceof the Real- theforecourtswherewe meet,the doors
throughwhichwe enter.Moregenerally,
the fagadeof a building,its verticaldimension,
is
t h eb u i l d i n w
g e ' r e a d ' t; h e h o r i z o n t adl i m e n s i otnh,ef f o o rp l a n ,i s t h e b u i l d i n w
g e' L l s e , :
t h e c o m p o s i t i o n aslp e c t a c l ei n t h e u p r i g h td i m e n s i o ni s e s s e n t i a l lvyi s u a l .I t
restrictsthe userto observation
from a distance.. . . In the horizontalplane,the
c o r r e s p o n d idnygn a m i cisn v o l v et sh eu s e rd i r e c t l ya n di st h e r e f o r lea r g e l ys o c i a lT. h e
l e v e pl l a ni s t h e a r e n ao f h u m a na c t i o n .
( A r n h e i m1.9 8 2 : 2 I 3 )
Figure8.11,showsan exampleof centralityin an architectural
faqade:the canopywith the
M a d o n n ai n, t h ec e n t r eo f t h ef a q a d eo f t h e C h u r c ho f S a n t aM a r i ad e l l aS p i n ai n P i s a .
Multifacetedobjectsadd further dimensions
to three-dimensional
composition,
and
a l l o w( a t l e a s ti n p r i n c i p l ef)r o n t a n d b a c k ,a n d t h e l e f t a n d r i g h ts i d e( a n d ,i n t h e c a s e
of openstructures,
an interiorcentreand the exterior)to be usedfor the productionof
difference.0f
the latterwe havealreadyseenan examplein figure8.6 - the plasterheartin

The third dimension

fig e.ff Church0f SantaMariadellaSpina(Arnheim,1982)

the centreof the rigidframeof Giacometti'sHour of the Traces.

But not all multifaceted
objectsusethesedimensions.
As in nature,wheretreesor mountains
do not havea front or
b a c k( o t h e rt h a no n ew h i c hs t e m sf r o m o u r p o s i t i o ntso w a r d st h e m ,e . g .w h i c hs i d eo f t h e
mountainwe liveon),objectscan bethe same,andhavethe samemeaning,
whichever
side
w e v i e wt h e mf r o m .S c u l p t u r eosf t h e' T h r e eG r a c e sf'o, r i n s t a n c e
a ,r eu s u a l l cy o m p o s eidn
t h e r o u n do
, f f e r i n ge s s e n t i a ltl h
y e s a m ev i e wf r o m w h i c h e v esri d eo n ea p p r o a c h et hs e m .
And the samepatterncan be observed
in somechildren's
toys,for instancea kind of ball
w i t h s y m m e t r i c a ldl yi s t r i b u t epdi c t u r e isl l u s t r a t i nngu r s e r yr h y m e s( a l t h o u g h ' t hoen et h a t
e n d su p o n t o p d e t e r m i n ewsh i c hs o n gi s h e a r d ' ) . 0 ry o uf i n d i t i n b u i l d i n g s ,u c ha s t h e
V i c t o r i aB u i l d i n gi n S y d n e y , w h i cihn,i t s l a t e s tr e i n c a r n a t i oans a s h o p p i ncge n t r e ,
Queen
h a s ,o n e v e r ys i d ea, c e n t r ael n t r a n c ae n ds h o pw i n d o w so; r p e r f e c t l ryo u n db u i l d i n gssu c h
as the Templeof Vestain Rome,or the Pantheon.
front and back
In othercases,however,

257

258

The third dimension

especially
differ in meaning.It is temptinghereto take the humanbodyas a metaphor,
with the front as the publicside,the sidewherewe articulatehowwe wantto be read,and
the backas the non-social
side,the privatesidewhichis not meantto be viewedand often
has no meaningexceptperhapsfor thosewith whom we are most intimate.This is why,
paradoxically,
it can also be the most revealing,
as in the caseof Epstein'sAngel(figure
8 . 7 ) .T h i si d e ai s w o r k e do u t i n m o r ed e t a i li n v a nL e e u w e(n2 0 0 3 ) .
In chapter3 we arguedthat the structuresof diagrams(the top-downpath of the
taxonomy,the
left-rightpathof the ffowchart,thedigitalnetwork)are modelledon forms
o f s o c i a lo r g a n i z a t i o n
P.e r h a p ist c a n b e a r g u e da l o n gt h e s a m el i n e st h a t c o m p o s i t i o n ,
b o t ht w o - a n dt h r e e - d i m e n s i o ni saul ,l t i m a t e l m
y o d e l l e ed i t h e ro n t h e ' n o n - s o c i a l ' r o u n d nessof the naturalformssuchas treesand mountains,
or on the polarizedhumanbody,
with the headas the Ideal,the feet as the Real,the heart as the Centre,movementand
actionas the more interactive
and dynamichorizontaldimension,
and the front and the
public
private
backas,respectively,
social
non-social
and
side.As Lakoff
the
and
and the
a n dJ o h n s o sna i d ,
arisefrom the fact that we havebodiesof the sort we haveand
Spatialorientations
t h a t t h e yf u n c t i o na s t h e yd o i n o u r p h y s i c ael n v i r o n m e n t . 0 r i e n t a t i omneatl a p h o r s
havea basisin our physicaland culturalexperience.
Thoughthe polar opposites,
up-down,in-out,etc.,are physicalin nature,the orientational
metaphorsbasedon
t h e mc a nv a r yf r o mc u l t u r et o c u l t u r e .
( 1 9 8 0 :1 4 .o u r i t a l i c s )

T H E M O V I N GI M A G E
patternswe havediscussed
The representational,
in this
interactiveand compositional
b o o ka l s oa p p l yt o t h e m o v i n gi m a g ea, s s h o w nb y a n u m b e o
r f o u r e x a m p l eisn t h i sb o o k .
Yetthe additionof movement
doesof courseleadto differences,
and it is thesedifferences
we will discussin the final sectionof this chapter.
Startingwith the narrativeprocesses
we discussed
in chapter2, herethe principal
differenceis that the roleof the vectoris takenoverby movement.
Insteadof,for instance,
a vectorformedby an outstretched
will be realizedby
arm,as in figure2.I5,the process
the actionof raisingthe arm and pointingthe hand.Usuallytheseactionsare figurative,
, o i n t i n ga n d s o o n . B u t t h e y m a y a l s o b e
r e c o g n i z a ba
l es d r i v i n g w
, a l k i n g j, u m p i n g p
abstract as, for instance,in Walt Disney'sanimationfilm Fantasia(1941) - or in
technicalfilms,wherearrowsmay be animated,
unfoldingin front of our eyes.
B u t t h e r ei s a c o m p l i c a t i o Inn. m o v i n gi m a g e tsh e r e l a t i o nb e t w e e A
n c t o r sa n d G o a l s
may be represented
in a singleshot,showingboth Actor and Goal;or in two subsequent
shots,the first showingthe Actor,the secondthe Goal (or vice versa),as demonstrated
. u t i n t h eo n e
i n f i g u r e8 . 1 2 .I n b o t hc a s e sw e s e ea s o l d i e (r A c t o r )a n dc i v i l i a n (sG o a l ) B
casetheyare spatiallyconnected,
showntogetherin the sameshot;in the otherthey are
disconnected,
shownin separate
shots.As everyfilm andtelevision
directorknows,the two

The third dimension

( u s u a l l rye f e r r etdo a sa p a i ro f ' r e v e r s ae n g l es h o t s ' )

s h o t si n s u c ha d i s c o n n e c tseydn t a g m
'matched'
haveto be
Theyhaveto be takenfrom the
carefully,
to restorethe connection.
samesideof the imaginarylinerunningbetween
the participantsandfrom approximately
the samehorizontalangle,to mal<eit appearthat the participantsare facingeachother
by
andlookingat eachother.Thetonalityof thetwo shotshasto matchaswell,for instance
e n s u r i ntgh a tt h e ya r es h o tu n d e trh es a m el i g h t i n g
c o n d i t i o nas n
, dt h r o u g hc o l o u rg r a d i n g ,
r e m o v i nagn yd i s c r e p a n c ibeest w e etnh e c o l o u rr e n d i t i o o
n f t h es h o t s .
S h o u l dw e s e es u c ha ' d i s c o n n e c t e d ' n a r r a tpi rvoec e sasso n eu n i to f m e a n i nogr t w o ?I s
i t t h e e q u i v a l e notf a s e n t e n clei k e ' T h es o l d i e sr h o o t st h e v i l l a g e r so' ,r o f a f o r m u l a t i o n
that expresses
the soldier'sagencylessdirectly- for instance,'The soldierfires.The
villagersare shot'? But such an attempt at translatingmoving imagesinto words
c a n n o ft u l l y c a p t u r et h e d i f f e r e n c eF.i l m i c ' d i s c o n n e c t i o n ' hnaosp a r a l l eiln l a n g u a g eI t.
d o e sh a v es e m i o t i cp o t e n t i a lh, o w e v e rI t. c a n ,f o r i n s t a n c es,h o wp e o p l ea s ' i s o l a t e d '
i n d i v i d u a les v, e nw h i l et h e ya r e i n t e r a c t i nw
g i t h o t h e r sa, n d i t c a n r a d i c a l l yd i s c o n n e c t
Actorsfrom the Goalsof their actions,andfrom the effectof their actionson theseGoals,
j u s t a s h a p p e n sf o, r i n s t a n c ei n, l o n g - d i s t a n tceel e p h o ncea l l so r t h e f i r i n go f l o n g - r a n g e
missiles.

&ssl
Pr0ce$$

&*trr

fige.fz connectedanddisconnectednarratiyeprocess(GoodmanandGraddol,tgg6)

259

260

The third dimension

T h ed i s c o n n e c t i o
bn
e t w e eA
n c t o r sa n d G o a l si s a n a s p e cot f ' f i l m l a n g u a g e ' t h aotn l y
developed
twenty-fiveyearsor so after the inventionof the medium,and it has beenthe
s u b j e cot f m u c hd i s c u s s i oi n f i l m t h e o r yn, o t l e a s tb e c a u siet a l l o w s ' f a k i n g N
'. o history
o f t h e m e d i u mo m i t st h ee x p e r i m e not sf ' C o n s t r u c t i v i s t ' f i l m - m a ki netrhse S o v i e U
t n i o no f
the early1920s.In oneof theseexperiments,
film makerLev l(uleshov
cut togethershots
of two actorsmeetingand greetingeachother.Eachactor was filmed separately,
in a
differentlocation.Theyhad thereforeneveractuallyplayedout the scenetogether.0nce
the two shotsweresplicedtogether,
however,
the two appeared
to meetin oneandthe same
place,an effect which l(uleshovcalled 'creativegeography'.In 1930s Tarzanfilms,
encounters
with wild animalswere often faked in this way,by intercuttingstock shots
of wild animalswith shotsof actors acting out the appropriatereactionsand actions
i n a s t u d i os e t .T h e f a m o u sF r e n c hf i l m c r i t i c A n d r 6B a z i n( 1 9 6 7 ) ,o n t h e o t h e rh a n d ,
favouredthe'connected'
method.To seea realeventhappening
in realtimewasfor himthe
q u i n t e s s e n t ifai llm e x p e r i e n caen d t h e q u i n t e s s e n t ipaol w e ro f t h e m e d i u m H
. e praised

II
Cut to
I

e) Fiq8.12-continued

The third dimension

261

t h e s e ahl u n t i n t h e d o c u m e n t aNray n o o k otfh e N o r t h ( I 9 2 I ) , a s c e n e t a k e n a s o n e l o n g ,

u n e d i t esdh o to f a s e a lb e i n gh a r p o o n et d
h r o u g ha h o l ei n t h e i c e .
In contrastto the still image,the movingimagecan realizeeventsthat haveneitheran
A c t o r n o r a G o a l .S h o t so f s h i m m e r i nlgi g h t o n s o f t l yr i p p l i n gw a t e rc r e a t ea s e n s eo f
p u r ep r o c e s sp/u r em o v e m e n itn, w h i c hi t i s h a r d l yp o s s i b l teo d i s e n t a n g p
l er o c e s as n d
p a r t i c i p a n t s , a n d i n w h i c h p a r t i c i ptahnetysc, iaf n b e d i s c e r n e d a t a l l , a r e ' c a u g h t u p ' i n t h e
p r o c e sisn a w a yt h a t i s n e i t h e r ' a c t i v e ' n o r ' p a s s i vTeh'e. s t i l l i m a g ee q u i v a l e notf s u c ha
s h o tw o u l db e a k i n do f a b s t r a cpt a t t e r nl ,a c k i n gt h e d y n a m i cs e n s e
o f ' a c t i o n 'o r ' e v e n t ' .
T h ec h o i c eb e t w e e n ' c o n n e c t i o n ' a n d ' d i s c o n n e c t i oenx'iasltssi o
n t h ec a s eo f r e a c t i o n s .
F i l m sc a n s h o w R e a c t e ras n d P h e n o m e neai t h e ri n o n e a n d t h e s a m es h o t ,o r i n t w o
subsequent
shots.Thispattern,knownas 'the point-of-view
shot',tiestogetherthreeshots,
with the Reacterreappearing
is wedgedin
in the third shot,so that the Phenomenon
betweentwo shotsofthe Reacter.
If, for
Heretoo the shotshaveto carefully'matched'.
instance,
the Reacterlooksdown,the Phenomenon
hasto be shotfrom above,and if the
R e a c t elro o l <ast a m o v i n gP h e n o m e n o n , tahneg l eo f h i so r h e r h e a da n dt h e d i r e c t i o n
of
hisorhergazeshouldhavechangedinthethirdshot,tomatchthedistancetravelledbythe
P h e n o m e n odnu r i n gt h e s e c o n ds h o t .D i s c o n n e c t er eda c t i 0 n hs a v ea p a r t i c u l a r lsyt r o n g
' s u b j e c t i v e ' , ' f i r s t - p e r s o n ' faeset lh, e v i e w e ri s l o o k i n ga t t h e P h e n o m e n o n ' t h r o utghhe
eyesof the Reacter'.
A variantshowsthe Reacterandthe Phenomenon
in the sameimage,
'overthe shoulder'of the Reacter,
pointof view.But herewe
hencealsofrom the Reacter's
'throughthe Reacter's
do not lookat the Phenomenon
eyes',andthe effectis lessemotiona l l y i n v o l v i n ga,s w e s e et h e R e a c t efrr o m b e h i n da n d t h e r e f o r ed o n o t s e eh i s o r h e r
,
r e a c t i o ntso t h e P h e n o m e n oI n
t i.st h ea n g l eu s e di n c o n t e m p o r acr yo m p u t ewr a r g a m e st o
m a k ep l a y e r isd e n t i f yw i t ht h e ' s p e c i a l - o p s ' s o l dcihear r a c t e rt sh e yp l a y( f i g u r e8 . 1 3 ) .
F i n a l l yw, h i l es t i l l i m a g e h
s a v ed e v e l o p eddi a l o g u b
l rocesses,
e a l l o o ntso r e a l i z ev e r b a p
i n m o v i n gi m a g e ds i a l o g u ies n o tr e p r e s e n t e
r i t i n gb, u t d i r e c t l y , t h r o u g h
v ids u a l l y , t h r o uw
gh
speech.
Thesynchronization
replaces
lip movements
between
the speech
andthe Speaker's
the vector that connectsSpeakerand Speech.Without such synchronization,
moving
imagescannotsignifythat the speechwe hear is actuallyspokenby the Speakerwe see.
0 n c et h e l i n l <b e t w e etnh e S p e a k earn dt h ed i a l o g uhea sb e e ne s t a b l i s h et h
e ay
de
, d i a l o g um
become'off screen',
continuing,
for instance,
whileviewerswatchthe reactionof a listener.
Turningnowto the interactive
dimension,
in chapter4 we havealreadyseenhowcamera
positions
can createsymbolicrelationsbetween
viewersandwhat is depictedin an image.
M o v i n gi m a g e sa r e n o d i f f e r e n itn t h i s r e s p e c tw, i t h o n e p r o v i s oi:n m o v i n gi m a g e st h e
relationship
becomes
dynamic.ltcanchangein front of our eyes.Thecameracanzoomin
t o a c l o s e sr h o t o
, r z o o mo u tt o a w i d e rs h o t ;i t c a nc r a n eu pt o a h i g ha n g l eo r c r a n ed o w n
t o a l o w a n g l e ;a n d s o o n . A n d e v e nw h e nt h e c a m e r ai s n o t m o v i n gt,h e p a r t i c i p a n t s
themselves
can move/wall<awayfrom or towardsthe camera,or walk up or downa flight
o f s t a i r sw i t h t h e c a m e r at i l t i n gu p o r d o w nt o f o l l o wt h e m ,t h u sc h a n g i ntgh e a n g l ef r o m
whichviewersseethe participants.
In otherwords,the movingimagecan represent
social
relationsas dynamic,
ffexibleandchangeable.
Distance
andanglecanbe dynamicized,
and
participantsinitiatingthe change,
this in two ways;subject-initiated,with
the represented
or camera-initiated,
with the image-maker
initiatingthe change(the contrastof course

262 .

The third dimension

Cul ta

fige.ff 'overshoulder'shotinacomputerwargame

In thefirst case,thevisual
alsoappliesto syntheticimageswherea camerais not involved).
text takesa 'neutral'stance,a stanceof 'recording'what is taking place(eventhough
more overtly
the eventsmay of coursebe staged).In the secondcase,the image-maker
positions
viewerstowardswhat is beingrepresented.
In otherwords,what in the caseof
In mostfilmsdistanceand anglechangeconstantly.
cubism,the useof multipleperspectives,
still imageshasnevermovedin the mainstream,
h a sb e c o m se o c o m m o n p l a ci nem o v i etsh a t i t i s n o wh a r d l yn o t i c e dT. h eo n l yd i f f e r e n cies
oneafterthe other,ratherthan at the sametime.
that filmsshowthe differentperspectives
of distanceand anglecan be usedto signifyboththe
Figure8.14 showshowsuchchanges
between
the peoplewe seeon the screen,andthe ongoing,
relationsbetween'characters',
constantlyshiftinqrelationsbetweenthesecharactersand the viewers.It is the opening
s c e n ef r o m H o w a r dH a w k s 'T h e B i g S l e e p( 1 9 4 7 ) . ' P r i v a t eE y e ' M a r l o w e( H u m p h r e y
s a l d r o nt)o h e l ph i m
B o g a r t )h a sb e e nc a l l e dt o t h e h o u s eo f G e n e r aSl t e r n w o o(dC h a r l eW
r ,a r m e n( M a r t h aV i c k e r s ) .
d e a lw i t h a c a s eo f b l a c k m a i ln v o l v i nhgi sy o u n g e sdta u g h t eC
confronts
As Marlowewaits in the hall to be shownin by the butler,Carmenprovocatively
thenfollows.
him.Theinterviewwith Sternwood
In newsand currentaffairstelevision,
distanceand anglecreatea symbolicrelation
are shownfrontally,from
Anchorpersons
the peopleon the screenandthe viewer.
between
s l i g h t l yb e l o we y el e v e la, n di n a w i d e rs h o t h a nm o s to t h e rp a r t i c i p a n ti sn t h e p r o g r a m m e .
T h i se n h a n c et sh e i r a u t h o r i t yT. h e ya r e l i t e r a l l y ' h i g h eurp ' t h a n t h e v i e w e r sa n d s h o w n

The third dimension . 263

fiq e.f+ Dynamicinterpersonal

relationsin the openingsceneot TheBig Sleep(Hawks,1947)

264 .

The third dimension

from a respectfuldistance- initial shotsmay evenshowthem from a very longdistance,

s i t t i n gb e h i n dl a r g eg, l e a m i ndge s k sa t t h ef a r s i d eo f a n e m p t yp, a l a t i ahl a , , .
T h ed i s t i n c t i o b
n e t w e e n ' o f f e r ' a n d ' d e m a n d ' ( scehea p t e r4 ) a l s oa p p l i e st o m o v i n g
participantscan turn towardsthe
images,and it too can be dynamicized:
represented
cameraand look at the lens(and henceat the viewer),or can averttheir gaze.But the
cameracannotinitiatethis; it mustbe initiatedby the participant,whetheron their own
initiativo
e r a sa r e s u l o
t f f o l l o w i n gi n s t r u c t i o nf rso m a d i r e c t o r .
' O f f e r s ' a r es t i l l t h e r u l e i n n a t u r a l i s t idc r a m a i,n t h e t h e a t r ea s m u c ha s i n f i l m a n d
t e l e v i s i o nB. e r t o l t B r e c h tf a m o u s l ys o u g h t o r e i n t r o d u cteh e ' d e m a n d ' s t a n c ei n t h e
by meansof interpolated
theatre,especially
songs,andfilm-makerslikeJean-LucGodard
h a v ef o l l o w e d
h i mi n t h i s .I n t h e s ec o n t e x t s ' d e m a n d s ' wtehroeu g h t t oc r e a t ea n ' a l i e n a t i o n
meantto naturalizethe fictionalworld of stageand
effect',to breakwith conventions
screen/and so to makethe audiences
moreawarethat they werewatchinga fictionand
- for example,
invitethemto reffecton its content.In manyothercontexts
newstelevision
the'demand'isthe accepted
althoughnot everyone
is giventhe rightto address
convention,
the viewerdirectly.Anchorpersons
and on-camerareportersmay look at the camera,but
interviewees
maynot; in chatshowshostsmay lookat the camera,but guestsmay not,and
e h i c hm e d i ap r o f e s s i o n ahlas v er e s e r v e d
s o o n .I n o t h e rw o r d st,h e ' d e m a n di 's a p r i v i l e gw
for themselves.
T h ec o n c e pot f m o d a l i t y( s e ec h a p t e5r ) i s a l s of u l l ya p p l i c a b lteo m o v i n gi m a g e sb,u t a
furtherfactor,movement,
needsto be addedto the list of meansof expression
that cancue
modality.Likevisualdetail,background,
depth,lightandshade,
colour,etc.,movement
can
be represented
with differentdegreesof realismor abstractionand henceplay a role in
modalityjudgements.
Representations
of walking,for instance/
can rangefrom simple
a n i m a t i o ni sn w h i c hs t i c kf i g u r e sr a i s ea n d l o w e rt h e i rl e g sw i t h o u ta n ya r t i c u l a t i o on f t h e
jointsor any movement
of the rest of the body,to highlydetailedanimations
showingthe
r i p p l i n go f e v e r ym u s c l ei n v o l v e d .
M o s tf i l m si n v i t eu st o u s et h e n a t u r a l i s t iccr i t e r i o na,l t h o u g h
t h i s i s p e r h a pcsh a n g i n g
as the use of syntheticimagesand animationincreases.
In many animatedcartoons,
the background
hashigher(naturalistic)modalitythanthe foreground,
a reversalof what
n o r m a l l yh a p p e n isn ' l i v e a c t i o n ' f i l m s T
. h e r ei s , o f c o u r s ea, t e c h n i c arle a s o nf o r t h i s .
Backgrounds
do not haveto be animatedand can thereforebe paintedin detailwithout
breakingthe budget.Again,in computergamesdifferentcharacters
and actionsmay be
animatedmoreor lessintricately.
In a DeltaForcegameoneof us played,the
movement
of
e n e m i efsa l l i n gd o w na st h e yw e r ek i l l e dw a sd e c i d e d luyn c o n v i n c i a
nn
g du n n a t u r aA
l .g a i n ,
theremay be pragmaticreasons.
Detailedanimationcoststime and money,and mayslow
downthe action.But that doesnot neqatethe semioticeffectof reducinqthe naturalistic
i m p a c to f k i l l i n g .
F i n a l l yt ,h ee l e m e n tosf c o m p o s i t i odni s c u s s ei nd c h a p t e6r ( i n f o r m a t i ovna l u es, a l i e n c e
a n df r a m i n g )a p p l yt o t h e c o m p o s i t i oonf t h e s h o t si n a f i l m o r t e l e v i s i opnr o g r a m mjeu s t
as muchastheyapply to stilI imagesandothervisualcompositions,
with,again,the proviso
that the movingimagecanmakecomposition
dynamic.Something
that startsout as Given
can moveintothe Newpositionin front of our eyes.Something
that has low salience
can

The third dimension . 265

becomh
e i g h l ys a l i e nitn t h e m i d d l eo f a s h o t- f o r i n s t a n c eb ,y m o v i n go r b e i n gm o v e di n t o
the light,or by a changeof focusof the camera.In chapter6 we showedhow,in figure6.1,
the left edgeof the doorof the shedframesthe two characters
in the shot,causingthemto
i n h a b idt i f f e r e nst p a c easn ds o e m p h a s i z i nt hge l a c ko f c o m m u n i c a t i obne t w e etnh e m .B u t
in a movingimagecharacterscan moveinto eachother'sspaceand undothe framing
between
them.And all of thesewaysof dynamicizing
composition
can be subject-initiated
or camera-initiated.
T h i s b r i e f d i s c u s s i odno e sn o t e x h a u stth e ' l a n g u a g e
o f f i l m a n d t e l e v i s i o nI't. h a s
concentrated
on the spatialpatternsof individualshotsand on two specifictime-ordered
patterns,
the 'reverseangle' and'pointof view'.But film is also,and perhapsaboveall,
a temporalmode,structuredby intricatesemanticand rhythmicpatternsof editing(see
v a n L e e u w e n2, 0 0 4 , f o r a s o c i a ls e m i o t i ca p p r o a c h )a, n d i t i s a l s o c h a r a c t e r i s t i c a l l y
m u l t i m o d ailn, v o l v i nngo tj u s t t h e v i s u a lb, u t a l s os p e e c hs,o u n da n dm u s i cT. h e s ea s p e c t s
of the mediumfall outsidethe scopeof this book.But we do hopewe haveshownthat the
i d e a sp r e s e n t ei d
n t h i sb o o kc a nu s e f u l lby ea p p l i e d
t o t h es p a t i aal s p e c tosf m o v i n gi m a g e s
ortmoreprecisely,
sincemovement
is a temporalphenomenon,
to an areawherethe spatial
andthe temporalinteractand overlap.

9 G o l o u r uf l t h o u g h t s( a p o s t s c r i p t )

We have,in this book,attemptedto presenta'grammarof visualdesign',to makeexplicit

of visualgrammarform a potentialfor the representation
how the availableresources
We have
of visualelements.
throughspatialconfigurations
of meaning
andcommunication
travelleda certaindistancealongthe road,but we realizethat this work has only just
thinkingabout the limitationsof what we havedoneand the
begun.We find ourselves
amountof work and the kind of work that remains.Socialsemioticsis an attemptto
a n dc o m m u n i c a tme e a n i n ign s p e c i f isco c i a l
d e s c r i baen d u n d e r s t a nhdo wp e o p l ep r o d u c e
settings,be they settingssuch as the family or settingsin which sign-makingis well
i n s t i t u t i o n a l i z eadn d h e m m e di n b y h a b i t s c, o n v e n t i o nasn d r u l e s .B u t s i g n - m a k i nign
societyis so variedan activitythat anyattemptto captureit in a generaltheorymustlook
with the richness
of the actualsemioticworld.In anycasethe theory
crudeby comparison
w h i c hw e u s et e l l su st h a tt h ea c t i o n so f t h o s ew h o m a k es i g n si n t h ee n v i r o n m e not fst h e i r
whichthey use,as well as
the resources
culturaland socialworldsis constantlychanging
a n d d y n a m i cw o r l d i n w h i c hw e a r e ,t h i s
t h e i r p o t e n t i a l sa;n d i n t h e i n c r e a s i n gdl yi v e r s e
makesthe relationof evena generaltheoryto the specificinstancetentative.Both the
l dp r o v i s i o n a l .
a l w a y sh y p o t h e t i c a n
t h e o r ya n dt h e ' g r a m m a r ' a r e
awareof this and havetriedto write our grammarnot as
We havebecomeincreasingly
d s t a b l e ' s y s t e mn'o, r a s o n ew h i c hs e e k st o c a p t u r e
o n ew h i c hd e s c r i b ef isx e d ' r u l e s ' a n a
t h ed e t a i lo f e v e r y t h i npge o p l ec a na n dd o d o i n v i s u acl o m m u n i c a t i obnu,t a s a f f e x i b l see t
We
that peopleusein evernewand everdifferentactsof visualsign-making.
of resources
of
as we have,not in orderto achievecomprehensiveness
haveincludedas manyexamples
h e s e m i o t i cp r i n c i p l ews h i c hu n d e r l i e
b ,u t r a t h e rt o i l l u s t r a t ea n d d e m o n s t r a t e
coverage
and therebyto try and showthe flexibilityof these
and shapehumansocialsemiosis,
needsthat humans
and communicational
resourcesin relationto the representational
n ,e f e e lw e h a v eo n l ym a d ea
. u t e v e ng i v e nt h i s q u a l i f i c a t i ow
h a v ei n t h e i r s o c i a l i v e s B
and interpreted
Wewish,for instance,
that we couldhavemorefu lly documented
beginning.
or the questionof modalityconimage-making,
the socialhistoryof scientific-technical
figurationsin differentschoolsof modernart, to mentionjust some of the questions
o n w h i c hw e h a v et o u c h e da l l t o o l i g h t l yA. n d ,a l t h o u g hb y n o wq u i t ea f e w d e s c r i p t i o n s
existof how imageswork in distinctcultures,this is a field of suchvast importancein a
imperative
c u l t u r a l l-y a sw e l la se c o n o m i c a l-l yg l o b a l i z i nwgo r l dt h a t w e s e ea n a b s o l u t e
in any one of the hugelyculturally
for work in that area.As just one practicalinstance,
d i v e r s ec l a s s r o o misn c i t i e ss u c ha s L o n d o no r S y d n e yo r J o h a n n e s b u rcgh, i l d r e ns t i l l
c f t h e d o m i n a nct u l t u r ea n dy e ta r e
realized
i n t h e v i s u asl e m i o t i o
e n c o u n t ear c u r r i c u l u m
l i k e l yt o ' r e a d ' - a n dt h e r e f o r'em i s r e a d ' -t h a t r e a l i z a t i oinn t e r m so f t h e i r ' h o m ec u l t u r e ' .
t h e' t h i r dd i m e n s i o nw' ,e s a wa w h o l en e wf i e l do f p r o b l e masn d
o explore
I n b e g i n n i nt g
q u e s t i o nosp e n i n gu p ,o n ew h i c hn o d o u b tw i l l i t s e l fo p e nu p f u r t h e rq u e s t i o nws h e nw e
a b o u t h ei m p a c t
b e g i nt o d e l v ei n t oi t m o r ed e e p l yN. o r h a v ew e ,i n t h i sb o o k s, a i de n o u g h

Colourfulthoughts . 267

(We
of the digitalmediaon modesand processes
of representation
and communication.
havemadecomments
on this elsewhere;
seel(ressandvan Leeuwen
2001; l(ress2003.)
However,
it wasour aim to provideat leastinitiallya broadoverview,
to map onelarge
regionof the semioticlandscape,
andthiswe feelwe havedone,with all the faultsthat will
inevitably
clingto sucha broadenterprise.
We are verymuchsensitive
to the fact that at
t h ev e r ym o m e nw
t h e nt h e r ei s a t h e o r e t i c aml o v ei n t h e s o c i asl c i e n c et o
s w a r d sw i d e n i n g ,
in oftenquite radicalways,the framingsaroundthe domainto be studied,and henceto
expandthe scopeof thetheoreticalframeworkused- we seem,in this book,to havemoved
i n a n o p p o s i tdei r e c t i o n , f o c u s innagr r o w l yo n o n em o d e H
. o w e v e r , wdeo s e eo u r a t t e m p t s
- awayfrom the intensityof focuson
part of that broadening
hereactuallyas an essential
s p e e ca
h n dw r i t i n ga l o n et,o a r e f r a m i n g
o f t h e d o m a i no f p u b l i cf o r m so f r e p r e s e n t a t i o n
and communication.
That will, we hope,havemany advantages:
first (and foremost)to
s h o wh o wh u m a n( s o c i a ls) e m i o s iasc t u a l l yw o r k s ,i n w h a t e v esr i t e so, f s c h o ool r w o r k o r
l e i s u r ea; n d s e c o n d ltyo s h o wt h e c o n n e c t i o nwsh i c hh a v eo f t e nb e e no b s c u r e bd y m o r e
l i m i t e da n dh i g h l ys p e c i a l i z eadp p r o a c h ewsh, e t h e irn a r t - h i s t o r i c as lt u d i e o
s r in attempts
to studythe mostbanalof everyday
events.
Theseareconnections
suchasthat between
the
h i s t o r yo f a r t ( a n dv i s u a cl o m m u n i c a t i ogne n e r a l l ya)n dt h e t h e o r yo f l a n g u a g e
i n, w h i c h
the social and historicaldimensions
have,too often, been absent;or the connection
between
the 'micro'and'macro'accountsof the socialworldwhichcharacterize
different
schoolsof sociology;
or the connections
betweenthe studyof the world in 'practices'and
t h e s t u d yo f t h a t ' s a m e ' w o r l d
i n i t s r e p r e s e n t a t i oF
no
s .r u s ,t h e c h i l d ' sr e l a t i v e l y ' f r e a
e c' t
, nof sign-making
w,h i c hf o r i t s u n d e r s t a n d i nr e
g q u i r e sv e r y s p e c i f i c, ,b i o g r a p h i c acl o
t e x t u a l i z a t i o ni s ,o f a s m u c hi n t e r e sat s i s r e p r e s e n t a t i w
o ni t h i nh i g h l yi n s t i t u t i o n a l i z e d
g e n r e si n w h i c h s i g n - m a k e rm
s u s t f o l l o w w e l l - e s t a b l i s h reudl e sa n d c o n v e n t i o nasn, d
w h i c hf,o r t h e i ru n d e r s t a n d i nr e
gq
, u i r eb r o a d esr o c i aal n dh i s t o r i c aclo n t e x t u a l i z a t i o n s .
T oe m p h a s i zoen, c em o r e , t h en a t u r eo f t h e ' g r a m m aor f v i s u adl e s i g na, sa r e s o u r cfeo r
m a k i n ga n dc o m m u n i c a t i nmge a n i ntgh r o u g ht h e c o n v e r g e noc fem a n yd i f f e r e nst i g n i f y i n g
systems,
we will discussonefinal example.Thisexamplewill also helpreiterateanother
p o i n t .I n w r i t i n gs u c ha b o o ka s t h i s ,o n e s o m e t i m efse e l st h a t o n e i s a p p l y i n ga c o l d ,
clinicalapproachto semioticpracticeswhichare,in reality,alwayscolouredby affective
factors.Howto write about affect?Howto steera path betweenthe mereassertion,
or
celebration,
of the role of affect,whichwill leaveno roomfor analysisand interpretation,
andthe colddissection
whichthreatens
to destroyits objectas oneis writingaboutit? We
have,from time to time, remindedthe readerof the omnipresence
of affect which we
assertedin the Introduction:for instance,
in chapter4, wherewe discussed
the relations
between
the imageandthe viewer,relationswhichare alwaysaffectiverelations,
relations
of identification
or its opposite;in chapter5, wherewe discussed
the affectivequalityof
' m o r et h a nr e a l ' v a l u eosf m o d a l i t y
m a r k e r si ;n c h a p t e6r , w h e r ew e d i s c u s s e' bda l a n c e ' a s
a n i n t e r f a cbee t w e etnh eb i o l o g i c aaln dt h es e m i o t i ca;n di n o u rd i s c u s s i oonf ' h a n d w r i t i n g '
in chapter7. Yet we feel that affect has perhapsbeentoo thin a thread in the tapestry.
Theproductionandcommunication
of meaningcannotbe otherthan alwaysaffectiveand
(e.g.
constitutiveof subjectivities,
in the domainswe tend to regardas self-expressive
children'd
s r a w i n ga n d a r t ) a s m u c h a s i n t h e d o m a i n sw e s e e a s o b j e c t i v a t i nagn d

268 .

Colourful thoughts

( el . g s. c i e n t i f iacn dt e c h n i c adl r a w i n g s )E. v e nt h e m a x i m a l lay b s t r a cm

t odality
impersona
of diagramsis an affectivechoice,by the very fact that it attemptsto negateaffect.The
productive
theory,the
subjectis as centralhereas anywhereelse.In muchcontemporary
drawnbetween
old body-minddichotomyagainmanifestsitself in the strict boundaries
be they groundedin
and more affectiveapproaches,
rationaland cognitiveapproaches
or
discourses,
the discourses
that eulogizethe 'pleasureof the text', in psychoanalytical
otherwise.
We havetriedto avoidthis.Wefeelstronglythat the cognitiveandthe affective
alwaysco-present.
Thisis onereasonwhy we havegiven
are not antitheticalbut inevitably
hereaffecthasnot yet beencoveredover
sucha centralplaceto children'srepresentation:
for 'rationality'or 'objectivity'.
demands
by a society'sideological
will therefore
b e a n o t h ee
r x a m p l eo f c h i l d r e n 'vsi s u a sl i g n - m a k i n g ,
0 u r f i n a le x a m p l e
indicatesthat the child who made it soughtto fuse
and its title, ColourfulThoughts,
t h e c o g n i t i v e( ' t h o u g h t s 'a) n d t h e a f f e c t i v e( ' c o l o u r f u l ' )I.t i s a p a i n t i n go, n p l y w o o d ,
girl (plate B). The
and was madeby a ten-year-old
measuring120 by 90 centimetres;
artist and,in the sizeof the
motherof the girl who madethis paintingis a professional
imitated
work,the mediumand the way shehassignedthe painting,ihe girl has,perhaps,
her own styleof visualthinking.It
her mother.But the work itselfvery muchrepresents
from her mother),an academicwhose
was paintedas a presentfor herfather (separated
study,the girl felt,wastoo bareandfunctional,and lackednot onlysufficientpictureson
the wall,but also- andaboveall - colour.
we notefirst of all
Lookingat the paintingf romthe pointof viewof ideationalmeaning,
It is
the shapesin the foreground.
the mainparticipants,
that thereare no vectorsbetween
in chapter3, it
a conceptualpicture.Like many of the conceptualpicturesdiscussed
lacks a concretesetting,showsthe participantsfront-on and at eye level,includesa
The participants
linguisticelementin the picturespaceand usesabstractrepresentation.
way,and this suggests
that the
are distributedacrossthe picturespacein a symmetrical
The symmetryis in fact quite strikingand the
relationbetweenthem is classificational.
eventhough,by itself,the yellowshapetop left is heavier
objectsare delicatelybalanced:
t h a nt h e y e l l o ws h a p em i d d l er i g h t ,t h e b a l a n c ei s r e s t o r e db y t h e h i g h e rp o s i t i o no f t h e
by a horizontal,
t a l l o o n ' t o pr i g h t .T h e t i t l e , i n b l a c k ,i s b a l a n c e d
s m a l lb l u e ' t h o u g h b
s l i g h t l yc u r v e db l a c kl i n eo n t o p .A n dt h e p a r t i c i p a n tasr e p l a c e di n a k i n do f c i r c l ea r o u n d
enclosed
whichdiffersfrom the othersin that it is not completely
the centralparticipant,
by a thick blackline,but openat the bottom,sothat the redcolour,literally,dripsout of it.
Theirshapesand coloursmay be
what theseparticipantsrepresent.
Thetitle indicates
t s 'h. a t i s a t h o u g h ta, c c o r d i ntgo t h i s p a i n t i n g ?
d i f f e r e n tb, u t t h e ya l l r e p r e s e n t ' t h o u g h W
Thoughtsare (1) differentlycoloured(red,blueand
Herewe can makefour observations.
y e l l o w ) ;( 2 ) d i f f e r e n t l sy h a p e d(;3 ) m o s t l ye n c l o s e b
d y a t h i c k ,b l a c kc r u s t ( b l a c k t, h e
c o l o u ro f w o r d s ! ) ;a n d ( 4 ) s o m e t i m enso t q u i t ee n c l o s e-d i n w h i c hc a s et h e i r c o n t e n t
'bleedsout of them',soto speak.Thesestronglysaturatedcolours(thesamepaletteasthe
) re'more
n t h e p r e v i o ucsh a p t e r a
M i 1 6s c u l p t u r easn d t h e c h i l d r e n 'tso y sw e d i s c u s s ei d
than real'andhencemotivatedby affect,ratherthanon naturalistic
arounds(resemblance
t o r e a l i t y )o r c o n v e n t i o n a l(lcyo l o u r ' c o d e s ' )I t. i s d i f f i c u l t o g o i n t o d e t a i la b o u tt h e
meanings
of the differentcolours,thoughwe havemadean attemptto providemeansfor

Colourful thoughts . 269

t h i n k i n ga b o u tt h i s i n c h a p t e r7 . T h e l i t e r a t u r eo n t h e ' e m o t i v em e a n i n g so/f c o l o u ri s
i n c o n s i s t e nSt .o m ep s y c h o l o g i sht sa v e r e p o r t e dt h a t ' p e o p l e ' p r e f e rs a t u r a t e dc o l o u r
over unsaturatedcolour; others,that they prefer unsaturatedcolour over saturated
colourB
. l u e h a s b e e nd e s c r i b eads ' d e p r e s s i nagn d s a d ' a n d a s r e p r e s e n t i n g ' c a l m
p l e a s u r eG
' . o e t h ec a l l e dy e l l o w ' g a ya n d s o f t l yc h a r m i n gw
' , h i l el ( a n d i n s ksya i di t ' n e v e r
containsa profoundmeaningand is akin to utter waste'.Onething is certain,however:
c o l o u r( a n dc o l o u rc o n t r a s t i)s u s e dt o r e a l i z ea f f e c ti n t h e s e n s o r cy o d i n go r i e n t a t i o n s
that inform,for instance,certain types of art and art appreciation,
certain forms of
dressand interiordecorationand their appreciation,
and so on. In ColourfulThoughts,
thoughtsare thus represented
as affectsof differentkinds:red,blueand yellow(among
o t h e rt h i n g sf,o r t h e ya l s oh a v ea s h a p ea, f r a m e e
, t c . ) .A s f a r a s t h e e n c l o s i nbgl a c kl i n e s
a r ec o n c e r n ew
de
, m i g h tp e r h a pssa yt h a tt h ec h i l dr e p r e s e ntthso u g h tas s m o s t l y ' d a m m e d
up', repressed,
keptinside,exceptin the one,centralcase.0nthe shapeswe will comment
below.
Thebackground
maynot bea concretesetting,but it is not a neutralbackground
either:
a g r e e nf i e l d w
, i t h y e l l o wd i a g o n asl t r i p e sa n dp i n kd o t s .I t i s d i f f i c u l n
t o tt o t h i n ko f t h e s e
stripesasdownwardvectors,goingfrom top leftto bottomright,particularlysincethereis,
in the bottomright corner,a darkening
of the greenfield in the form of an arrowhead:
the
paintingas a wholethusconverges
towardsthe nameof the artist,and mightbe seenas a
, i a g r a m sh, e r a l d r y c) o l o u r s
k i n d o f m e n t a ls e l f - p o r t r a i It n
. c o n c e p t u avli s u a l s( m a p s d
often haveconventional
meanings,
and we would suggestreadingthesesomewhatless
saturated
coloursin thisway:greenasthe colourof nature,pinkasthe colourof femininity.
In otherwords,the stronglyemotive,but bottled-up,
thoughtsof the childexistagainsta
b a c k g r o u nodf n a t u r e ' d o t t ew
di t hf e m i n i n i t y , .
As far as the interpersonal
meanings
of the paintingare concerned,
the vieweris not
positioned
by any form of perspective,
althoughthereis a separation
betweenforeground
a n d b a c k g r o u n tdh, r o u g ho v e r l a p p i nagn d t h r o u g hd i f f e r e n c ei sn c o l o u rs a t u r a t i o nN. o r
d o e st h e p i c t u r ef o r m a ' d e m a n d ' .A n d a s n o h u m a ns u b j e c ti s r e p r e s e n t etdh,e i s s u e
of social distancedoes not arise. However,colourful,,
the paintingis, in many ways,
'objective'.
It existsas an objectin its ownright,regardless
of the viewer.
Something
canbe
said,however,
about its modality.The paintingabstractsfrom naturalisticdepictionin
almosteveryrespectexceptone:the vibrantandstronglysaturatedcolours.Thesecolours,
as we havenotedalready,are'more than real',and thus suggesta codingorientationin
whichthe affectiveformsthe cornerstone
of reality.
We turn, finally,to compositional,
textual meanings.
Threethingscan be noted.The
p a i n t i n gi s ,f i r s t o f a l l ,a b a l a n c ecdo m p o s i t i owni t h a s t r o n gs e n s e
o f C e n t r ea n d M a r g i n .
T h ec h i l dh a s ,l i t e r a l l yb,a l a n c eadn dc e n t r e dh e re m o t i o n isn t h i sp i c t u r eS. e c o n d , t htei t l e
is at the bottom,andsocodedas,on the onehand,moreconcreteandrealbut,on the other
h a n d ,I e s sv a l u e da n d l e s s ' i d e a l ' t h a nt h e v i s u a e
l l e m e n tA. t t h e t o p w e f i n d m e r e l yt h e
shadowof thistitle: the colour,but not the substance
of the verbal;its emotivequality,but
not its overtlyarticulatedcontent.Third,there is a differencebetweenleft and right - a
differencein the shapesof the thoughtsand a differencein colour(red is moststrongly
represented
on the right).

270

Colourfulthoughts

It is temptingto readthe shapeof the

0n the meaningof this we can only speculate.
participantson the left as phallic,and thoseon the right as/ by contrast,representing
the female.The centrality,and hencethe mediatingfunction (sinceleft and right are
p o l a r i z e d )o,f t h e c e n t r a lp a r t i c i p a ndt e r i v e sf r o m t h i s :t h e c h i l dh e r s e l if s t h e c e n t r a l
,thought'in the middle,caughtbetween
alsothat the
the fatherandthe mother.Remember
on the right,the female
colourof the centralparticipant(red) is moststronglyrepresented
side,and liesmostclearlyon the pathof the vectorswhichleadto the artist'sname.
We mentionedthat the picture was paintedas a gift to the father.In the picture the
viewer,
is
fatheris Given,the motherNew.In otherwords,whatthe father,as the addressed
thetwo is formed
the female,whilethe linkbetween
askedto payattentionto is the mother,
, e r e- u n l i k ea l l t h e o t h e rt h o u g h t-s e s c a p ef sr o m i t s
, h i c hh
b y t h e , t h o u g h to' f t h e c h i l dw
the paintingand givingit to the
itselfin the act of producing
and expresses
encapsulation
father.
T h u st h ec h i l dn o t o n l yb r i n g st o g e t h ecr h o i c efsr o ma l l t h ea v a i l a b lree s o u r c eosf v i s u a l
a n dc o m p l e vx i s u atlh i n k i n gb, u ta l s of u s e sh e rt h i n k i n g ,
s i g n - m a k i ni n
g a p i e c eo f i n t r i c a t e
her cognitivework,with her affectsin an activeprocessof workingthroughsomeof the
problemsconnected
with her identityandsubjectivity.

References

Alfen,J. (1977 )'Self-reffexivityin documen

tary,,Ci netracts2: 37 -44
Arnheim,R. (1969) VisualThinkin4Berl<eley
and Los Angeles,
U n i v e r s i toyf C a l i f o r n i aP r e s s
(I974) Art and VisualPerception,Berkeleyand Los Angeles,
Un i v e r s i t oy f C a l i f o r n i aP r e s s
(1982) ThePowerof the Centre,Berkeleyand LosAngeles,
U n i v e r s i toyf C a l i f o r n i aP r e s s
Bakhtin, M. (1981) TheDialogicImagination,Austin,Universityof
TexasPress
(1984) Problemsof Dostoevsky'sPoetics,Manchester,
ManchesterUniversityPress
Bal, M. (1985) Narratology:Introduction to the Theoryof Narrative,
Toronto,TorontoUniversityPress
(1990) Verf en Verderf:Lezenin Rembrandt,Amsterdam,
Prometheus
Barthes,R. (1967a) Elementsof Semiology,London,Cape
(1967b) Systdmede la Mode,Paris,Seuil
(1970) L'Empire des Signes,Geneva,Skira
(I973) Mythologies,
St Albans,Paladin
(7977 ) Image-M usic-Text,London,Fontana
(1984) CameraLucida,London,Fontana

272 .

References

B a s s y , A . - M (. 7 9 7 5 ) ' D u t e x t eA l ' i l l u s t r a t i o nS
' , e m i o t i c a1 1 ( 4 ) :
297-335
Bazin,A. (1967) What is Cinema?vol.1, Berkeleyand Los Angeles,
U n i v e r s i toyf C a l i f o r n i aP r e s s
Belf, P and van Leeuwen,T. (1994) The Media Interview: Confession,
Contest,Conversation,
Sydney,Universityof NewSouthWalesPress
New
Belting, H. (1990) TheImageand its Publicin the Middle,49es,
R o c h e l l eN, l A r i s t i d eD .C a r a t z a y
B e n d e i L . ( 1 9 8 8 ) F r a n c eL, o n d o nM
, acmillan
Benjamin,W. (r973) Il luminations,London,Fontana
Berger,J.(I972) Waysof Seeing,Harmondsworth,
Penguin
, o d a l i t i easn dt h e p r o c e sos f c u l t u r a l
B e r n s t e i nB
, . ( 1 9 8 1 ) ' C o d e sm
, a n g u a ga
en dS o c i e t y I 0 : 3 2 7 - 6 3
reproductioa
n :m o d e l ' L
Bindon,H.and Williams,H. (1988) GeographyResearchProjects:A
EdwardArnold
SeniorStudent'sHandbook,Melbourne,
Bogatyrev, P (1971) TheFunctionof Folk Costumein Moravian
Slovakia,The Hague,Mouton
Bols,P, Houppermans,M., Krijger, C.,Lentjes,W., Savelkouls,T.,
Terlingen,M. and Teune,P (1986) Werkaan de Wereld,Den Bosch,
Malmberg
Booth,W.C. (1961) TheRhetoricof Fiction,Chicago,
Chicago
UniversityPress
Bourdieu, P (1986) Distinction:A Social Critiqueof the Judgementof
Iasfe,London,Routledge
B r e c h t ,B . 0 9 6 7 ) ' U b e r d i e M a l e r e d
i e r C h i n e s e n ' , iG
nesammelte
Werke,vol. 18, Frankfurt-on-Main,
Suhrkamp

References.2T3

Brown,C.,Kefch,J.andThiel, PV. (1991) Rembrandt:TheMasterand

his Workshop,New Haven,Yale UniversityPress
Brown,T.M. (1958) The Work of G. Rietveld,Architect,Utrecht,
A . W .B r u n a
Bruna,D. (1988) 0n My Walk,London,MethuenChildren'sBooks
Brunsden,C.and Morley,D. (I978) EverydayTelevision:
Nationwide,
L o n d o nB, F I
Bryson,N. (ed.) (1990) VisualTheory,Cambridge,
Polity Press
Burgin,V. (ed.) (1982) ThinkingPhotography,
London,Macmillan
Ghatman,S.(1978) Storyand Discourse,lthaca,
Nt CornellUniversity
Press
Clarke,G. (ed.) (1992) ThePortrait in Photography,London,
Reaktion
Books
C o m o l l i , J . L .( 1 9 7 1 )' T e c h n i q ueet i d 6 o l o g i ec:a m e r ap, e r s p e c t i v e ,
profondeurdu champ',Cahiersdu Cin1ma229: 4-23
Coupe,S.andAndrews,M. (1984) TheirGhostsMay be Heard:
Australiato 1900, Melbourne,
Cheshire
DanceE
, E . X . ( 1 9 6 7 ) ' A h e l i c am
l o d e lo f c o m m u n i c a t i o ni n' , F . E . X .
Dance(ed.) HumanCommunication
Theory,NewYork,Holt, Rinehart&
Winston
Defeuze,G. (1992) Cinema,vol.I,TheMovementImage,London,
A t h l o n eP r e s s
Dondis,D.A. (1973) A Primer of VisualLiteracy,Cambridge,
MA, MIT
Press
Dragt, H., Hofland,W.A.andTamsma,R. (r986) De GeoGeordend,
A m s t e r d a mM, e u l e n h o f f

274

References

DSP (1991) Arts Literacy,Sydney,Disadvantaged

SchoolsProgramme
Report
Dyer,G. (1982) Advertisingas Communication,London,Methuen
E c o ,U . ( r 9 7 6 a ) ' A r t i c ul a t i o n so f t h ec i n e m a t i c o d e ' i, n B . Ni c h o l s( e d ).
Moviesand Methods,vol 1, Berkeleyand LosAngeles,Universityof
C a l i f o r n i aP r e s s
(1976b) A Theoryof Semiotics,Bloomington,
IndianaUniversity
Press
(7979) TheRoleof the Reader,Bloomington,
IndianaUniversity
Press
Faircfough, N. (1992) Discourse
Polity
and SocialChange,
Cambridge,
Press
( 7 9 9 3 ) ' C r i t i c ad
l i s c o u r saen a l y s i as n dt h e m a r k e t i z a t i oonf
publicdiscourse:
Discourse
the universities',
and Society4(2):133-69
Finch,C. ( 1968) Pop Art: )bject and Image,London,StudioVista/
Dutton
Finnegan,R. (2002) Communication,
London,Routledge
Fiske,J. (1982) Introduction to CommunicationStudies,London,
Me t h u e n
Fiske,J.and Hartley,J. (l.979) ReadingTelevision,
London,Methuen
F r e s n a u f t - D e r u e l lPe ,( 1 9 7 5 ) ' D u l i n 6 a i r ea u t a b u l a i r e ' ,
Comm uni cati ons 24 : 7-23
Gage,J. (!993) Colourand Culture- Practiceand Meaning from
Antiquityto Abstractio4London,Thamesand Hudson
(1999) Colourand Meaning- Art, Scienceand Symbolism,
L o n d o nT, h a m e sa n d H u d s o n

References

Gauthier,G. (1973) InitiationA la sdmiologiede l'image,Paris,Cahiers

de I'Audiovisuel
Genette,G. (I972) NarrativeDiscourse,
Oxford,Blackwell
G h a o u iC
, . ,G e o r g eS
, . M . ,R a d a ,R . ,B e e r ,M . a n d G e r t a J, . ( 1 9 9 2) ' T e x t
t o h y p e r t e xat n db a c ka g a i n ' i;n P .0 ' B r i a n H o l t a n d N . W i l l i a m s( e d s )
Computersand Writing: State of the Art, Dordrecht,l(luwer
G l e d h i l lC
, . ( 1 9 9 4 ) ' T o w a r d as g e n r eo f a b s t r a c t i n gp' ,a p e rp r e s e n t eadt
t h e 2 1 s t I n t e r n a t i o n aSl y s t e m i cF u n c t i o n aCl o n g r e s G
s ,e n t , 1 - 5A u g u s t
r994
Goethe,J.W von (1970l1B10l) Theoryof Colours,Cambridge,
MA,
MIT Press
Goffman,E. (1976) GenderAdvertisements,
London,Macmillan
GombrichE
, . ( 1 9 6 0 ) A r t a n d I l l u s i o n L, o n d o nP
, haidon
Goodman,N. (1969) Languages
of Art, London,0xford University
Press
G o o d m a nS
, .a n d G r a d d o lD
, . ( 1 9 9 6 ) RedesigningEnglish: New Texts,
New ldentitiegLondon,Routledge
Gregory,R.L. (1970) TheIntelligentE y g N e wY o r k ,M c G r a w - H i l l
Habermas,J.(1984) The Theoryof Communicative
Action,vol.I,
C a m b r i d g eP,o l i t yP r e s s
Hafl, E. (1964) 'S ilentassumptions
n', Disorders
in socialcommunicatio
of Commun i cation 42: 41-55
(1966) TheHiddenDimension,
NewYork,Doubleday
H a l f, S . 1 9 8 2 ) ' T h e d e t e r m i n a t i oonf n e w sp h o t o g r a p h si n' , S . C o h e n
andJ. Young(eds) Ihe Manufactureof News,London,Constable

zIi

276

References

Halliday,M.A.K. ( 1978) Languageas SocialSemiotic,London,Edward

Arnold
(1985) An Introduction to FunctionalGramma4 London,
E d w a r dA r n o l d
(1993) Languagein a ChangingWorld,Canberra,ALAA
)ccasional Paper13
Halliday,M.A.K.and Martin,J.R. (1993) Writing Science:Literacy
and DiscursivePower,London,FalmerPress
Hartley,J. (7982) Understanding
News,London,Methuen
Hauser,A.(1962) TheSocialHistoryof Art,vol.2,london, Routledge
Hermeren, G. (1969) Representation
and Meaningin the VisualArts,
L u n d ,S c a n d i n a v i aUnn i v e r s i tB
y ooks
Hifl, J. (1980) IntroductoryPhysics,London,Macmillan
Hjelmslev,L. (1961) Prolegomena
Madison,
to a Theoryof Language,
U n i v e r s i toyf W i s c o n s i P
n ress
Hodge,R. and Kress,G. (1988) SocialSemiotics,
Polity
Cambridge,
Press
(1993) Languageas ldeology(2nd revisededition),London,
Routledge
Hogg,J. (ed.) (1969) Psychology
and the VisualArts, Harmondsworth,
P e n g iun
H o n z l , J .( 7 9 7 6 ) ' D y n a m i cosf t h e s i g ni n t h et h e a t r e 'i,n L . M a t e j k aa n d
I.R. Titunik Gds) Semioticsof Art: PragueSchool Contributions,
C a m b r i d g eM, A , M I T P r e s s
Hughes,R. (1969) Heavenand Hell in WesternArf, London,Weidenfel.d
& Ni c o l s o n

References . 277

H u m p h r e yS
, . ( 1 9 9 2 ) ' E x p l o r i n gt h e l a n g u a goef s c h o ogl e o g r a p h y / /
u n p u b l i s h er d
e s e a r c rhe p o r t ,S y d n e yD, i s a d v a n t a g e
Sdc h o o l s
Programme
I e d e m a ,R . e t a l . ( 7 9 9 3 ) ' M e d i aL i t e r a c yR e p o r t 'u, n p u b l i s h er d
esearch
r e p o r t ,S y d n e yD, i s a d v a n t a g e
Sdc h o o l sP r o g r a m m e
( I 9 9 4 ) ' T h e l a n g u a goef a d m i n i s t r a t i o nu'n, p u b l i s h er d
esearch
r e p o r t ,S y d n e yD, i s a d v a n t a g e
Sdc h o o l sP r o g r a m m e
Iser,W. ( 1978) TheAct of Reading,Baltimore,M D,JohnsHopkins
U n i v e r s i tP
y ress
Itten,J. o970) TheElementsof Colour,NewYorl<,Van Nostrand
Reinhold
Jaff6, H.L.C. (1967) De Stijl: i9i7-i93i,
Phaidon

Visionsof utopia,Oxford,

Jakobson,R. (1971) Studiesin VerbalArt, AnnArbor,Michigan

UniversityPress
Jakobson,R. and Halle,M. ( r956) Fundamentals
of Language,
M o u t o nT
, h eH a g u e
J e n k i n s ,C . ( 1 9 9 0 ) S c i e n c e
S c e n eB, o o l <1 , M e l b o u r n eE, d w a r dA r n o l d
Jennings,T. (1986) The YoungGeographerInvestigatesMountains,
0xford,0xford UniversityPress
Joos, M. (1967) TheFive Clocksof Language,NewYork,Harcourt,
Brace& World
Kandinksy, W. 0977 L19741)Concerningthe Spiritual in Art, New
Y o r k ,D o v e rP u b l i c a t i o n s
Kress,G. 1977)'Tense as modality',UEA Papersin Linguistics5:
40-50

278

References

( 1 9 8 5 )' I d e o l o g i c aSl t r u c t u r e isn D i s c o u r s ei 'n, T . A .v a n D i j k

Gd.) Handbookof DiscourseAnalvsis,New York,AcademicPress
(1987a) Communicationand Culture:An Introduction,Sydney,
U n i v e r s i toyf N e wS o u t hW a l e sP r e s s
( 1 9 8 7 b ) ' E d u c a t i nrge a d e r sl:a n g u a g ien a d v e r t i s i n gi 'n, J .
Hawthorn kd) Propaganda,Persuasionand Polemic,London,Edward
Arnold
(1989) Linguistic Processesin Socio-culturalPractice,London,
0 x f o r d U n i v e r s i tP
y ress
( L 9 9 2 ) ' E x p l a n a t i o inn v i s u a lc o m m u n i c a t i o nr e' ,p o r tt o t h e
E SP R I T I I B a s i cR e s e a r cP
t n E x p l a n a t i o nW, o r k i n gG r o u p2 3 6
hrojeco
nf t h es i g n
l r o d u c t i oo
0 9 9 3 i l ' A g a i n s t a r b i t r a r i n e stsh:e s o c i a p
and
as a foundationalissuein critical discourseanalysis',Discourse
Society4(2): 169-93
( 1 9 9 3 b )' M e d i al i t e r a c ya s c u l t u r a tl e c h n o l o giyn t h e a g eo f
E. BrevortandJ. Savino(eds)lVeraz
transculturalmedia',in C. Bazalgette,
Directions:Media Education Worldwide,Londonand Paris,BFI/
CLEMI
( 1 9 9 5 ) ' R e p r e s e n t a t i o nr e
a sl o u r c easn ds u b j e c t i v i t yi 'n, C .
Analysis,London,Routledge
CoulthardGd.) Critical Discourse
(1996) Before Writing: RethinkingPaths into Literacy,London,
Routledge
( 2 0 0 0 ) ' D e s i g na n dt r a n s f o r m a t i o ni n' , B . C o p ea n d M . l ( a l a n t z i s
Gds) M ulti I iteracies,London,Routledge
(2003) Literacyin the New MediaAge,London,Routledge
K r e s s ,G .a n d T r e w ,T . ( 1 9 7 8 )' l d e o l o g i c at rl a n s f o r m a t i o nosf d i s c o u r s e :
or, how the SundayTimesgot its messageacross',SociologicalReview
261755-76

References . 279

Kress,G.andvan Leeuwen,T.(1990) ReadingImages,Geelong,

D e a k i nUn i v e r s i t yP r e s s
(7992)'Structuresof VisualRepresentation',
Journalof
L iterarySemantics11(2 ) : 91-117
(2001) Multimodal Discourse- The Modesand Media of
Contemp orary Commun i cati on, London,Edward Arnold
Lacy, M.L. (1996) The Power of Colourto Heal the Environment,
L o n d o nR
, a i n b o wB r i d g eP u b l i c a t i o n s
Lakoff,G, (1987) Women,Fire and DangerousThings:What
Categories
RevealAboutthe Mind,Chicago,
Universityof ChicagoPress
Laf<off,G.and Johnson,M. (1980) MetaphorsWe Live By, Chicago,
U n i v e r s i toyf C h i c a g oP r e s s
Leibovitz,A. (1990) Photographs
i970-J990, NewYork,
HaroerCollins
Lindeftens,R. ( 1971) El1mentspour unesdmiotiquede la photographie,
P a r i sD
, idier
Lupton,E. o989) 'Readingisotype'inM. Victor Gd.) DesignDiscourseHistory/Theory/Cr i t i cism, Chicago,University of Chicago Press
McKenzie,A.E.E. (1938) Magnetismand Electricity, Cambridge,
CambridgU
e n i v e r s i tP
y ress
McQuail,D. and Windahl,S. (1993) Communication
Models,London,
LOngman
M a c h i n ,D . a n d S u l e i m a nU
, . ( 2 0 0 4 )' T w oC o m p u t eW
r a r G a m e sS e t i n
L e b a n o nT: h ei n f l u e n coef a g l o b a tl e c h n o l o goyn d i s c o u r s eu' ,n p u b l i s h e d
paper,Cardiff University
M a r t i n ,J . R . ( 1 9 8 5 ) F a c t u a lW r i t i n g , G e e l o nD
ge
, a k i nUn i v e r s i t yP r e s s

References

(1992) English Text- Systemand Structure,Amsterdam,

Benjamins
M a r t i n , J . R . , W i g n e l lP, , E g g i n s ,S . a n d R o t h e r y , J .( 1 9 8 8 ) ' S e c r e t
E n g l i s hd: i s c o u r stee c h n o l o giyn a j u n i o rs e c o n d a rsyc h o o l 'i,n L . G e r o t ,
and T.van LeeuwenGds) Languageand Socialisation:
J. Oldenburgh
Homeand School,Sydney,MacquarieUniversity
Paris,Editionsdu
Martin, M. (1968) Le LangageCindmatographique,
Cerf
M a r t i n , R . M . a n dE l l i s , M .( 2 0 0 1 )P a s o sB, o o k1 , L o n d o nH, o d d e r a n d
Stoughton
s ,e wY o r k ,M c G r a w - H i l l
M e r r i t t , P G . ( e d . )( 1 9 8 4 ) B o o ko f F l o w s h e e fN
NewYork,0xford UniversityPress
Metz, C. (1974il Film Language,
(I974b) Languageand Cinema,The Hague,Mouton
Modley, R. and Lowenstein, D. (1952) Pictographsand Graphs,New
Y o r k ,H a r p e r
Morris, D. (7977) Manwatching,
London,Cape
of the
MukarovskyrJ. (r97 6) 'Art as semioticfact' and 'The essence
visualarts', in L. Matejkaand I. R. Titunik Gds) Semioticsof Art:
MA, MIT Press
PragueSchoolContributions,
Cambridge,
Mumford, L. (1936) Technics
and Civilizatio4NewYork,Harcourt,
B r a c e& W o r l d
Myers,G. (1990) Writing Biology,Madison,Universityof Wisconsin
Press
(1994) Wordsin,4ds,London,EdwardArnold
Nash,J.M. (1974) Cubism,Futurism and Constructivism,London,
T h a m e sa n d H u d s o n

References

Nattiez,J.-J. ,1976) Fonddmentsd'unesdmiologiemusicale,Paris,Uge

Nitzschke,V.(1990) Politik: Lernenund Handelnftir Heuteund
M orgen,Frankfurt,Diesterweg
The Hague,
Novitz, D. (1971) Picturesand their Usein Communication,
Ni j h o f f
Oakley,M. et al. (1985) )ur Societyand )thers,Sydney,McGraw-Hill
, p p s a l aE, s s e l t e
0 h m a n ,C . ( 1 9 8 9 ) H i s t o r i aU
Studium
of the Word,
0ng,W.J. (1982) 0rality and Literacy: The Technologising
L o n d o nM
, ethuen
Ostwald,W.(1935 t19167) ColourScience,London,Windsorand
Newton
O'Suflivan,T., Hartley,J., Saunders,D. and Fiske,J. (1983) lGy
Conceptsin Communication,London,Methuen
0'Toofe, M. (1994) The Languageof DisplayedArt,Leicester,Leicester
UniversityPress
Panofsky, E. (1953) Early NetherlandishPainting,New York, Harper
andRow
(1970) Meaningin the VisualArts, Harmondsworth,
Penguin
Pask,R.and Bryant, M. (1982) Peoplein Australia:A Social
Geography,
Melbourne,EdwardArnold
P i e r c e J, . R . ( r 9 7 2 ) ' C o m m u n i c a t i o nS' c, i e n t i fci A m er i c a n2 2 73 ) :
30-42
Poynton, C. (1985) Languageand Gender:Making the Difference,
G e e l o n gD, e a k i nU n i v e r s i tP
y ress
Prosser,R. (2000) Leisure,Recreationand Tourism,London,Collins
Educational

28r

282

References

R i l e y , J . W . a n dR i l e y ,M . W . ( 1 9 5 9 )' M a s sc o m m u n i c a t i oann dt h e s o c i a l
system'/in R.l(. Merton,L. Broomand S. Cottrell(eds)SociologyToday,
N e wY o r k ,B a s i cB o o k s
Rimmon-Kenan,S. (1983) Narrative Fiction: ContemporaryPoetics,
L o n d o nM
, ethuen
Ringbom,S. (1965) Icon to Narrative:The Riseof the DramaticClose
Up in Fifteenth CenturyPainting,Abo, Abo Akademie
R o w l e y - J o l i v eE
t ,. ( 2 0 0 4 ) ' D i f f e r e nvt i s i o n sd,i f f e r e nvt i s u a l sa: s o c i a l
s e m i o t i ca n a l y s i o
s f f i e l d - s p e c i fvi ci s u a cl o m p o s i t i oinn s c i e n t i f i c
presentations'
conference
3Q): 145-77
, VisualCommunication
Sacks,H. (1992) Lectureson Conversation,Oxford,
Blackwell
Saint-Martin,E (1987) Semioticsof VisualLanguage,
Bloomington,
I n d i a n aU n i v e r s i tP
y ress
Safe,C.,Friedman,B. and Wilson,G. (1980) )ur ChangingWorld,
B o o kI , M e l b o u r n eL,o n g m a nC h e s h i r e
Saussure,Ede(1974 t19161) Coursein GeneralLinguistics,London,
PeterOwen
S c a n n e l lP
, ( 1 9 9 4 ) ' C o m m u n i c a t i vi net e n t i o n a l i ti yn b r o a d c a s t i n g , ,
u n p u b l i s h epda p e r
Schapiro,M. (1973) Wordsand Pictures,The Hague,Mouton
SchrammW
, . ( 1 9 5 4 )' H o w c o m m u n i c a t i owno r k s , i,n W . S c h r a m m
(ed.) fhe Processand Effectsof Mass Communication,l)rbana,
U n i v e r s i toyf I l l i n o i sP r e s s
Scollon,R. and Scollon,S, (2003) Discourses
in Place:Languagein
the M ateri a I Wor Id, London,Routledge
S e f a n d e rS, . ( 1 9 9 4 ) ' P e d a g o g i s kt eax t e ro c h r e t o r i k ' i,n S . S e l a n d e r
and B. Englund(eds)Konstenatt informeraoch overtyga,Stockholm,
HL S F 0 r l a g

References

Shannon,G.and Weaver,W. (1949) TheMathematicalTheoryof

Urbana,Universityof Illinois Press
Communication,
S h a r p l e sM
, . a n d P e m b e r t o nL, . ( 1 9 9 2 )' R e p r e s e n t i nwgr i t i n g :e x t e r n a l
r e p r e s e n t a t i oannsdt h e w r i t i n gp r o c e s si'n, P . 0 ' B r i a nH o l t a n d N .
Wil liams Gds) Computersand Writing: State of the Art, Dordrechf,
l(luwer
S h e p h e r d , J(.1 9 7 7 ) ' T h em u s i c acl o d i n go f i d e o l o g i e isn' ,J . S h e p h e r d ,
P.Virden,G. Vulliamyand T. Wishart(eds) WhoseMusic?A Sociology
of MusicalLanguages,
London,TransactionBooks
Penguin
Sontag,S. (1977) 0n Photography,
Harmondsworth,
l en t e r s u c h u n guebne r
S t e f a n e s c u - G o a n gFa:,( 1 9 1 2 ) ' E x p e r i m e n t e lU
d i e G e f i j hl s b e t o n u ndge r F a r b h eIilg k e i t e nu n d i h r e rC o m b i n a t i o n e n ' ,
Philosophische
Studien10: 601-17
Walter l(onig
Theewen,G. (1993) JosephBeuys:Die Vitrinen,Cologne,
University
Thibauft,P (1991) SocialSemioticsas Praxis,Minneapolis,
of Minnesota
Press
Thompson,Pand Davenport,P (1982) TheDictionaryof Visual
Language,Harmondsworth,
Penguin
T h r e a d g o l dT, . ( 1 9 8 8 )' s t o r i e so f r a c ea n dg e n d e ra: n u n b o u n d e d
in D. Birchand M.0'Toole Gds) Functionsof Style,London,
discourse',
Pi nter
T r e w ,T . ( 1 9 7 9 ) ' T h e o r ya n d i d e o l o gay t w o r l <a' n d' L i n g u i s t i cv a r i a t i o n
, . l ( r e s sa n dT . T r e w
a n d i d e o l o g i c adli f f e r e n c ei'n, R . F o w l e rR, . H o d g eG
Gds) Languageand Control,London,Routledge
Tufte, E.R. (1983) The VisualDisplayof QuantitativeInformation,
s ress
C h e s h i r eC,T ,G r a p h i c P
Uspensky,B. (I973) A poeticsof compositio4Berkeleyand Los
A n g e l e sU, n i v e r s i toyf C a l i f o r n i aP r e s s

284

References

( 1 9 75 ) ' L e f t a n d r i g h ti n i c o np a i n t i n g, 'S e m i o t i c a
13(I): 33-4I
v a n L e e u w e nT, , ( 1 9 8 5 )' R h y t h m i cs t r u c t u r eos f t h e f i l m t e x t , ,i n T . A .
van Dijk Gd.) Discourseand Communication:NewApproachesto the
Analysisof MassMedia Discourseand Communication,Berlin,Walter de
Gruyter
(1986) 'Proxemicsof the televisioninterview,.
Australian
J ournal of ScreenTheorv17/IB: I25-4I
( 1 9 8 8 ) ' M u s i ca n d i d e o l o g yt o
: w a r d sa s o c i o s e m i o t iocfsm a s s
mediamusic',SlSSC WorkingPapersIOQ): t 99-220
( 1 9 8 9 )' C h a n g etdi m e s c, h a n g e tdu n e s m
: u s i ca n dt h e i d e o l o goy f
the news',in J. Tullochand G. TurnerGd) AustralianTelevision:
Programs,Pleasures,Politics,Sydney,Allen and Unwin
( 1 9 9 1 a )' C o n j u n c t i vset r u c t u r ei n d o c u m e n t a rfyi l m a n d
t e l e v i s i o nC
' , o n tni u u m5 ( 1) : 7 6 - 1 1 5
i f S S r O l' T h es o c i o s e m i o t iocfse a s yl i s t e n i n gm u s i c ' ,S o c i a l
S e m i o t i c1
s(1):67-80
( 1 9 9 2 ) ' T h es c h o o l b o oaks m u l t i m o d atle x t , .I n t e r n a t i o n a l e
SchuIbuchforsch ung 14 (I) : 35-8
( 1 9 9 3 ) ' G e n r ea n df i e l di n c r i t i c a ld i s c o u r saen a l y s i sa: s y n o p s i s , ,
Discourseand Society4(2) : 193-223
(1999) Speech,Music,Sound,London,Macmillan
( 2 0 0 3 ) ' A m u i t i m o d apl e r s p e c t i voen c o m p o s i t i o ni ,n, T . E n s i n k
and C. Sauer Gds) Framing and Perspectivisingin Discourse,
Amsterdam,Benjamins
(2004) I ntroduci ng Socia I Semi ofi cs, London,Routledge
y a n L e e u w e nT, .a n d C a l d a s - G o u t t h a rC
d ,. R .e o o q . T h e s e m i o t i cos f
kinetic design',in D. Banks Gd.) Textand Texture- Systemic-

References

Functional Viewpointson the Nature and Structure of Text, Paris,

L'Harmattan

( 1. 9 9 6 ) ' 0 nl e a r n i nt og l o o k
v a nL e e u w e n , T . aH
nd
umphrey,S
(eds)Literacy
geographer's
through
a
eyes',
in R.Hasan
andG.Williams
in Society,
London,
Longman
v a n L e e u w e nT, . ,a n d K r e s s ,G . ( 1 9 9 2 )' T r a m p l i n g
a l l o v e ro u r u n s p o i l t
s p o t :B a r t h e s"' p u n c t u m "a n dt h e p o l i t i c so f t h e e x t r a - s e m i o t i c ' ,
SouthernReview25(7): 27 -38
( 1 9 9 5 ) ' C r i t i c a ll a y o u ta n a l y s i sI' n
, ternationale
ch
u
I
buchfo
rsc
h
u
ng
17
S
G) : 25-43
v a n L e e u w e n , T . a n dS e l a n d e r , S(.1 9 9 5 ) ' P i c t u r i n g" o u r " h e r i t a g ei n
the pedagogicfext', Journal of CurriculumStudies27 6): 50I-23
van Sommers,P (1984) Drawingand Cognition,
Cambridge
Cambridge,
UniversityPress
V e e l ,R , ( 1 9 9 2 ) ' T h el a n g u a goef s c h o osl c i e n c eu' ,n p u b l i s h er d
esearch
r e p o r t ,S y d n e yD, i s a d v a n t a g e
d
S c h o o l sP r o g r a m m e
Voloshinov,V.N.(1973) Marxismand the Philosophyof Language,
L o n d o nS, e m i n a rP r e s s
Watson,J.and Hill, A. (1980) A Dictionaryof Communication
and
MediaStudieELondon,EdwardArnold
Whorf, B.L. (1956) Language,Thoughtand Reality,Cambridge,
MA,
MIT Press
W i e r z b i c k a , A .( 1 9 9 0 ) ' T h em e a n i n g
o f c o l o u rt e r m s :s e m a n t i c sc ,o l o u r
and cognition',CognitiveLinguistics1: 99-150
Williams,N. and Holt, P (eds)(1989) Computers
and Writing,
N o r w o o dN, J ,A b l e x
Wiffiams,R. |1977) Cultureand Society7780-1950, Harmondsworth,
Penguin

285

286 .

References

Wifliamson,J, (1978) DecodingAdvertisemenfE

London,Marion
Boyars
Wofkers,J. (1965) Een Roosvan Vlees,Amsterdam,Meulenhoff
W o l l e n ,P ( 1 9 8 2 )' G o d a r da n dc o u n t e r - c i n e mvae:n td ' e s t ' ,i n B . Ni c h o l s
Gd.) Moviesand Methods,vol.2, Berkeleyand LosAngeles,University
o f C a l i f o r n i aP r e s s

Index

abstractcodingorientation
165,252,268
abstraction29, 55, 56, 60,
90-7, r04,268
(as
accompaniment
atvfttmctanaat

I )

t^

itiated
camera-in
d y n a m i c i z a t i o2 n6 1
C a n o v a4,. 2 4 3
, I-I03,
c a r r i e r5 0 , B B - 9 0 9
1 0 6 - 9 ,r r r , 2 4 3 , 2 4 8
c D - R O M3 0 , B 0 , 2 r r - 1 3
cenlre194-20I, 209,256,
269
B a l ,M . 4 8
chartsseediagrams
balance
202-3,268
Barthes,R. 6, 17-I8, 24-6, C h a t m a nS, . 1 1 5
c h i l d r e n 'bso o k s2 3 - B
4 7, 6 8 , r r L , 2 0 2 , 2 I 9 ,
2 2 3 , 2 2 5 , 2 5 2 - 3 , 2 5 5 c h il d r e n ' sd r a w i n gas n d
w r i t i n g 9s - 7 2 , 1 6 , 2 0 ,
B a s s yA, . - M . 2 0 6
36-8,r71-r3,751-3,
Bazin,4.260
processes
2\r,268-70
76,
behavioural
circles54-5, 56-7
7B
c i r c u l a cr o m p o s i t i o2n0 9
B e l l ,P .1 8 4
c i r c u m s t a n c7e2s, 7 4 - 5
B e l t i n gH,. 1 1 8
c lassifi
cationalprocesses
76
beneficiary

a t t i t u d eI 4 9 , I 5 2
a t t r i b u t e5 0 ,7 5 , 9 I - 7 0 3 ,
r07-9, l-l-o,712-L3,
243,248
a t t r i b u t i v sey m b o l i c
processes
105-7

actionprocesses
63-6,
7 4-5
actor 50, 59, 63-6, 7 4, 7 6,
78,108,111,258-61
a d v e r t i s i ni gm a g e 6
s 7 ,B I ,
r 2 2 , 7 4 2 ,1 4 3 - 4 , 7 5 9 ,
17B, ).83, 186
affect267-B
affordance232-3, 236
A l l e nJ, . 1 2 0
a m p l i f i e vde c t o r s7 l
analyticalprocesses
50, 51,
83,87-r05,247
B e n j a m i nW, . 2 1 8
a n c h o r a g1e8 , 1 1 1
B e r g eJr ., I 4 9 , 1 5 0 ,1 5 1
a r b i t r a r ys i g n8 , 1 2
architecture
240,256-8
B e r g m a nI ., 1 7 5 ,1 7 B
Bernstein
B,. 1 6 5
A r m i t a g el ,< . 2 4 0
B e u y sJ,. 2 4 8
A r n h e i mR,. 4 7 , 4 9 , 7 3 0 ,
66,
L 7 6 , 1 9 4 , 2 0 2 , 2 5 2 , 2 5 6bidirectionalprocesses
7
4
,
7
8
A r p ,J . 5 5
art 53-7, 60,90-2, I 05-6, Bogatyrev,P.6
, . C .1 1 5
1 l B , r 2 2 , ! 2 5 , 1 2 9 - 3 0 , B o o t hW
B o u r d i e uP, .1 6 5 , 1 7 9
l3r-2, r59, t67-7r,
B r a n c u sCi ,. 2 2 4
176,I8l-3,188,189,
1 9 2 , 1 9 4 ,r 9 7 - 8 , 2 0 2 ,
Brecht,B. I3I,264
b r i g h t n e s( sa sm o d a l i t y
205-6,22r-5,227-8,
2 3 9 - 4 r , 2 4 3 - 7|
cue)162
248-57, 252-4, 256-8
vectors7l
C a l d a s - C o ual trhd ,C . R .2 4 6
attenuated

4 7, 7 9 - 8 7, 1 0 7, 2 6 8
codingorientation163'6,
r66
c o l o u r1 5 9 - 6 0 , 2 2 5 - 3 8
c o l o u rs c h e m e2s3 7 - B
C o m o l lJi ,. L . 1 3 0
c o m p o s i t i o1n76 - 2 1 4 ,
256-8,264,269
c o m p o u n d i 5n 2g , 9 7 - B
computergames252,26I,
264
processes
59,
conceptual
79-Lr3,268,269
c o n j o i n i n5g2 , 9 7 - B
c o n n e c t i o2n0 4 ,2 I 0 , 2 6 1
(as
contextualization

288

m o d a l i t yc u e )l - 6 1 /s e e
a/sosetting
process
conversion
68-7 0,
75,78
coverttaxonomy79| 87t
107
c r i t i c a ld i s c o u r saen a l y s i s
2,14
D a n c eF, . E . X 7. 1
D a v e n p o rPt , 5 4 , 5 6
d e m a n dI I B , 1 2 0 - 2 , I 2 3 ,
1 2 5 ,7 2 7, I 4 B , I 4 g ,
250-r,264
d e p t h( a sm o d a l i t yc u e )
162,252
detachment
136-40,
145-9, l5r,25l
diagrams48,50-3, 55, 60,
6I-2,65-6,68-70,78,
82-7,95-703, r2t,
1 2 6 ,t 4 3 - 7 , 1 5 6 - 9 ,
172,rg4, IBB, 196-7,
258
d i a l o g ub
e a l l o o n6s 8 , 7 5
a sc o l o u r
d i f f e r e n t i a t i o( n
feature1
) , 6 07, 6 7 ,2 3 4 ,
236,238
d i r e c ta d d r e s1s 1 7
d i s c o n n e c t i o2 n0 4 ,2 I O ,
214,258,261
Disney,
W. 258
distinctivefeatures233-5
distributionmedia220
D o b e l lW
, . 167
D o n d i sD, . A .5 4 , 5 6
Dyer,G. 6, B
d y n a m i c i z a t i oi nnf i l m
2 6 t- 5

EcoU
, .47,1).5,215
G a b oN
, . 53
e d u c a t i o1n6 , 1 7 , 3 4 ;C D
G a g eJ,. 2 2 8
R 0M S 3 0 ,3 3 ; c h i l d r e n ' s Genette,
G. 116
geometrica
w r i t i n g3 9 , I I O - I 3 ,
s yl m b o l i s m
1 , 5 2 - 32t 1 1 ;c o n c e p t
70-2,94
maps39-40; textbooks G h a o u iC, . 8 6
3 0 , 3 3 , 4 5 - 7, 9 0 , 9 4 - 5 , G i a c o m e t tAi ,. 2 2 5 ,2 4 4 ,
II9-20, r2I, 126,
249,257
g i v e n5 7 , 1 7 9 - 8 5 , 2 0 9 ,
I37-8, r42,155-6,
172-3,185,186,
2 r 1 , 2 2 4 , 2 5 6 , 2 6 4 ,0
27
r87-9,230
G l e d h i lCl ,. 2 0 5
E i s e n s t e iS
n ., 2 9
goal50, 63-6, 7 4, 77, 1.09,
e r o n g a u o5n/
258-61
e m b e d d i n5g0 , I 0 7 - 9
G o d a r dJ ,. - L . 2 6 4
E p s t e i nJ,. 2 2 4 ,2 3 9 - 4 0 ,
G o e t h eJ,. v o n 2 6 9
2 4 6 - 7, 2 5 6 , 2 5 9
Goffman,E. 67-8
e q u a l i t yI 4 0 , l 4 B - 9
G o o d m a nN,. 4 7
evenl64, 7 6, 7 I
Goodman
5 ,. 2 2 9
exhaustive
analytical
G r a d d oDl ,. 2 2 9
p r o c e s s e9s5 - 8 , 1 0 4 ,
grammarI-5, 228,266-7;
110
seealsolanguage
processes
existential
109-10
e y el e v e Il 4 0 , 7 5 2 , 2 6 8
e y e i l nvee c t o r1 1 /
fashionphotography
87-B
f i l m1 8 , 2 9 , I 2 0 - I , I 2 4 ,
I27-8,157-9,175-6,
203,258-65
F i n n e g a nR,. 3 6
h t s K e /J . b / /

flowcharts84-5
f r a m i n g1 7 6 , 2 0 3 - 4 , 2 I 4
Frank,R. 64
Fresnault-Deruel
le, P.6
frontal angle144-5, I52,
268
TUSIO5
NJ

H a b e r m aJs., 1 6 5
H a l l ,E . 1 2 4 - 5 , I 4 9
H a l lS
, .68
H a l l eM
, .233
H a l l i d a yM, . A . l < I. , 2 , 1 5 ,
2 0 t4 9 t 6 8 , 7 3 , 7 6 - 7 ,
1 0 9 - 1 0 ,r r 7 , r r g , r 2 2 ,
r 5 5 , 1 7 3 ,I B r , 2 2 9 - 9
H a r t l e yJ,. 6 , 7
H a u s eA
r ,. 1 3 0
H a w k sH, . 2 6 2
h e l i c avl e c t o r s7 1
Hermeren,
G. 47, 105
Hodge,R. 102/ L55,l'92
homedecoration
235-7
Honzl,J. 6
horizontalangle133-40

Index

hue234
i d e a1
l 86-93,2A3,209,
256,269
i d e a t i o n af ul n c t i o n1 5 ,4 2 ,
155,228,268
i d e o l o g1y 4
i ll u m i n a t i o(na sm o d a l i t y
cue)162
illustration1B
i m a g ea c t I I 7 - 2 O . I 2 3 ,
r52
image-text
relation18,
26-7,177-8,L87
impersond
a il s t a n c 1
e25,
).48-9,151
i n c l u s i vaen a l y t i c a l
processes
95-8
i n d i r e cat d d r e s1s 1 9
informationvalue777,
179-20I
i n t e r a c t i vm
e ul t i m e d i a3 0 ,
33,34,80,95-6,128,
t 32-3 , r9r-2 , 186,
t98*9,2rt-13,252/
26r,264
participants
interactive
48,
774-t 6
interactive
viewing(3d i m e n s i o noabl j e c t s )
250-2
i n t e r a c t o r6s6 , 7 4 , 7 6
i n t e r e s8t, 1 2 , 1 3
interordinate
80, 86
i n t e r p e r s o nf au ln c t i o n1 5 ,
42,155,228
involvement
136-40,
r4B-9,25I
Iser,W. 48

M a r t i nJ, . 6 2 , 7 0 2 , I 7 3 ,
229
M a r t i nM
, . 140
m a t e r i ai tl y 2 1 5 - 2 5 ,2 2 6
76, 78
materialprocesses
(as
Kahan,L. 167
means circumstance)
l ( a n d i n s l <Wy.,2 2 I , 2 2 3 ,
7 2 , 75
m e d i a t oIr9 B - 9 , 2 0 9
2 2 7, 2 3 2 , 2 6 9
mentalprocess68,75,
l ( i e n h o l zE,. 2 4 3
77-8
K o o n sJ,. 2 1 9 , 2 2 4 , 2 2 5
m e t a p h oBr ,2 5 8
l ( r e s sG
, . 5 , 3 4 , 5 9 ,I 0 2 ,
M e t zC
, .6 , 2 4 , 4 - 7
r 5 5 ,r 9 2 , 2 r 5 , 2 r 8 ,
M i c h e l a n g e2l o5 2
230,232,267
M i r 6 ,J . 2 4 3 - 4 ,2 4 9 ,2 5 4 ,
Kuleshov,
L. 260
268
m o d a l i t y9 I , 1 5 4 - 7 4 ,
L a c yM
, .L.229,235
L a k o f fG
, .258
252-5,264;
17I-2;
configurations
language
2-3,l-8-20,
76-R 1n9-10. 122-t
markers160-3,77U
129,139-40,155,181/
s c a l e s1 6 0 /1 6 6 , 2 5 2
modesof reception219-20
199,233,259
m o d u l a t i o(na sc o l o u r
L i c h t e n s t eR
i n.,2 2 2
f e a t u r eL) 6 0 , 2 3 4
L i n d e k e nR
s ,. 6
l i n g u i s t i csse el a n g u a g e
M o n d r i a nP, .5 3 , 5 4 ,5 5 ,5 6 ,
2 2 1 .2, 2 3 ,2 2 4 ,2 2 7, 2 3 2
L i s s i t z k yE,. 2 9 , 5 5 , 5 6 ,
M o n e tC, . 2 2 1 '
60
mood122-3
literacy2, 22-3
L u p t o nE, . 1 0 1
M o o r eH
, , 5 5 ,2 2 5 ,2 5 0 - 1 ,
Lyon,D. 140-1
252
M o r r i sD
, .I2I
motivatedsignB, 12
M a c h i nD, . 2 5 2
Mukarovsky,
J. 6
m a g a z i n e2s9 , 3 0 ,3 I , L 4 3|
y7
1 7 9 - B t , r B 3 - 4 , 1 8 8 - 9 , m ul t i - l e v el el dt a x o n o m 8
m u l t i m o d a l i t1y7 7 - 8 ,
235-7
M a l e v i c hl <
, .2 9 , 6 0 ,2 2 7
r B 7 - B /2 0 r , 2 6 5
M u m f o r dL, 1 6 4
m a p s5 9 - 6 0 , 6 5 , 8 8 , 9 2 ,
M y e r sG
, .164
9 7, 9 9, 1 4 5 - 6 ,1 72
m a r g i n1 94 - 2 0 I , 2 0 9 ,2 5 6 ,
59-72
narrativeprocesses
269

J a f f 6H
, .53,54,55
J a k o b s oR
n ,. 6 , 2 3 3
J o h n s o nM, . 2 5 8
J o o sM
, . 129

'

vt

+v

Lvt

LLL

290

Index

Nash,J.M.54
N a t t i e zJ, . - J .6
n a t u r a l i s t icco d i n g
o r i e n t a t i o1n6 5
n a t u r a l i s t im
c o d a l i t y2 8
3 0 , g I , 7 5 g - . g ,1 6 4 - 5 ,
219,255,264
networks84-6
new57, \7 9-85, 209, 2II,
224,256,264,270
newmediaseeinteractive
multimedia
Ni c h o l s o n
B,. 2 2 4 , 2 3 2
N o l d eE
, . 106
non-linearity27-8, 84-6,
204-B
processes
non-projective
73
non-transactional
actions
6 3 - 4 , 74 , 76 , 7 8 ,r r 3 ,
239
non-transactional
reactions
68,74,76,78,240
objectiveimages143-B
objects240-2,249-50
o f f e r1 1 9 - 2 0 ,) . 2 2 ,! 2 7 ,
148-9,250,264
0 n g ,W . 1 7 8
0'Sullivan,T. 6, 9
0 ' T o o l eM
, .6
overttaxonomy87
Panofsky,
E. 47,718
P a r i ss c h o o6l
participants
47-59, 7 4-5,
79-80, Br, lr4,
1 1 6 - 2 0 , 2 3 92t 6 8
P e i r c eC, . S .B

p e r s o n adli s t a n c 1
e24,
148-9
perspective
729-33,
7 3 4 - 7, I 4 4 , 1 4 6 - 9 ,
1 6 2 ,1 7 2 , 2 6 2
p h e n o m e n o6n7 , 7 5 , 7 7 ,
26r
photography
63-4, I4O-I
p o i n to f v i e ws h o t s2 6 1
p o l a r i z a t i o2n0 9 , 2 5 6 ,
270
possessive
attributes50,
9 L - 1 0 3 , 1 0 7 - 9 ,I 1 0 ,
ll2-3,243,248
potentialseemeaning
potential
power140-3, 148-9, I5I,
251
P o y n t o nC, . 1 4 0 - 2 , 2 2 9
P r a g u es c h o o6l
printedpage178-9
processes
49,59-78,
79-1 I 0
p r o d u c t i otne c h n o l o g y
2I7-I8
projectiveprocesses
73, 7 7
publicdistancesee
impersonal
distance
p u r i t y( a sc o l o u rf e a t u r e )
234,237,238
quotedspeech77-8
r e a c t e r6s 6 , 7 5 , 2 6 7
reactionalprocesses
67-8,
239,242,26r
readingpaths204-B
r e a l1 8 6 - 9 3 , 2 0 3 , 2 0 9 ,
256,269

r e c o r d i ntge c h n o l o g y
217-I9
rectangles
53-4,56-7
relationalprocesses
I 09-10
r e l a yI B , 6 8
Rembrandt
I49-5It
192-3,22r
reportedspeech77-8
(asmodality
representation
c u e )1 6 1 ,) . 6 7 , 2 5 2
participants
represented
48,174-16
reverseangleshots259
r h y t h m2 0 1 , 2 0 3
R i e t v e l dG, . 1 6 8
R i m m o n - l ( e n aSn.,4 8 , 1 1 5
R i n g b o mH,. 1 2 5
R o d c h e n kAo., 2 9
R o d i nA, . 2 2 4 , 2 4 0 , 2 5 6
R o t h k oM, . 2 2 0 , 2 2 5 , 2 3 1
R o w l e y - J o l i vE
e t. ,9 0
R y m a nM, . 1 6 8 , 2 1 6 , 2 2 3 ,
225
S a c k sH
, . 121
S a i n t - M a r t i nF,.d e 2 1 5
s a l i e n c1e76 , 1 7 7 ,2 0 I - 3 ,
2r0,214
saturation760,233, 236,
238
F.de 12, 156
Saussure,
s a y e6r 8 , 7 5 , 7 7 , 2 6 I
S c a n n e lP
l ,.1 1 5
S c h e f eJr ., - L . 6
S c o l l o nR, . 4
S c o l l o nS, . 4
Segal,G. 243
s e m i o s isse es i g n - m a k i n g

Index

s e n s e6r 8 / 7 5
s e n s o rcyo d i n go r i e n t a t i o n
1 6 4 - 5 t1 6 7, 2 5 2 ,
254-5
setting72, 74, It], 7I2,
r 5 2 ,1 7 2 , 2 4 3 , 2 5 5 ,
269
s e q u e n t i ab li d ri e c t i o n ai tly
66
S h e p h e rJd.,1 3 0
s i g nB ,1 2
s i g n ie
f ir m a t e r i a l 2s 1 6 - 1 7
s i g n - m a k i n7 g- I 2 , 2 7 0
s i m ul t a n e o u s
b i d i r e c t i o n a l i6t y6
s i n g l e - l e v e l tl e
ad
xonomB
y7
sizeof frame124-9
s o c i acl o n t e x1t 3
s o c i adl i s t a n c 1
e 2 4 - 9 ,1 4 8 ,
I49,I52,25I
spatio-temporal
analytical
s t r u c t u r e1s0 1 - 3
speakerseesayer
<noaanaaf<

t))-<

speechprocess
seeverbal
pr0cess
cr rhionf-inifiefarl

d y n a m i c i z a t i o2 n6 1
subjectiveimage129*42,
1,43
subordinate
79-80,I07
s u g g e s t i vsey m b o l i c
processes
105-7
Suleiman
U,. 2 5 2
superordinate
79-80
s y m b o l iac t t r ib u t e s1 0 5
symbolicprocesses
91/
105*8,248

s y n t h e s i z i tnegc h n oolg y
217-r9
taxonomy79-87
t e c h n o l o g i ccaol d i n g
oflentauo1
nb5
t e c h n o l o g2yI 7 - 2 0
t e l e v i s i o1n2 6 , I 2 O - I ,
r27-8, r32,784-5,
262-4
t e m p o r aal n a l y t i c a l
p r o c e s s e9s4 - 5 , I 0 4
textbooksseeeducation
t e x t u afl u n c t i o n1 5 ,4 3 ,2 2 8
T h e e w eG
n ,. 2 4 8
T h i b a u l tP, 6
T h o m p s o nP,. 5 4 , 5 6
t h o u g hb
t a l l o o n6s 8 ,7 5
Threadgold,
T. 6
t i m e l i n e9s4 - 5
T i t i a n1 7 6
top-downangle144-5
t o p o g r a p h i cparl o c e s s e s
98-101,104
t o p o l o g i c aplr o c e s s e s
9B-100,104
t o y s2 3 9, 2 4 2, 2 4 7- 8 , 2 4 9,
25r,252*4,255
transaction50
actions64-6,
transactional
74,r08,239
t r a n s a c t i o n rael a c t l o n6s8 ,
74 , 2 4 0
t r a n s d u c t i o3n9
42,82-4
t r e es t r u c t u r e
frew,T.2
t r i a n g l e5
s5*7,60
t r i p t y c hI 9 B - 2 0 I , 2 0 9

u n i drie c t i o n at lr a n s a c t i o n
a c t i o n s7 5 , 7 8
u n s t r u c t u r eadn a l y t i c a l
nrncFqqFq 9?-4

' t '1-0 4 . r

7r3
U s p e n s k8y.,2 0 2
utterance75
value(ascolourfeatLtre)
2 3 3 , 2 3 ,72 3 8
v a nD o e s b u/rfg. 9 0 - L ,
I68_9
v a nE y c k , J 1. 1 8
V a nG o g hV
, .221
f ., 6 , 3 4 , 5 9 ,
v a nL e e u w e n
184,2r5,2r8,230,
232,239,246,258,
265,267
v a nS o m m e r sP, .9 2
v e c t o r4 2 , 4 6 , 5 9 , 6 0 - 2 ,
7 O-2, 7 4-5, 91, r77 ,
r80,239,240-2,244,
258,268,269
verbalprocesses
6B/75l
77-8,261
v e r t i c aal n g l e1 4 0 - 3
v o l u m e4s 9
v o nN e u r a t h , 01.0 1
websitesI2B,132-3,
r3l-2,186, r9B-9
W h o r f ,B . L . 1 - 2
W i l l i a m sR, . 1 7 9
W i l l i a m s oJn.,4 7
W o l l e nP
, .1 2 0
w r i t i n g4 , 1 7 , 2 I - 3 , 3 9 ,
1 1 0 - 1 3 ,1 1 5 - 1 6 , I 4 2 ,
152-3, r9r-2,2r1

297

Relatedtitles from Routledge

Discoursesin Place
Language in the Materialworld
Ron Scollon and Suzie Wong

Scollon

'Written
with directness and charm, and an abundance of persuasive
e x a m p l e s ,t h i s b o o k l o c a t e s m e a n i n g n o t j u s t i n l a n g u a g e b u t i n t h e
r i c h n e s s a n d c o m p l e x i t yo f t h e l i v e d w o r l d . . . i t s i n s i g h t s w i l l s t a r t a
g e n e r a t i o n o f n e w t h i n k i n g , a n d r e s e a r c h .l t m a r k s a t u r n i n g p o i n t i n
linguistics and semiotics alike.' - Gunther Kress,lnstitute of Education,
Univenity of London, IJK
Discoursesin Place develops the first systematic analysis of the ways we
i n t e r p r e tl a n g u a g ea s i t i s m a t e r i a l l yp l a c e di n t h e w o r l d .
It argues that we can only interpret the meaning of public texts like road
signs, notices and brand logos by considering the social and physicalworld
t h a t s u r r o u n d st h e m . D r a w i n go n a w i d e r a n g eo f r e a le x a m p l e sf,r o m s i g n s
i n t h e C h i n e s e m o u n t a i n s t o u r b a n c e n t r e s i n A u s t r i a . F r a n c e ,N o r t h
America and Hong Kong, this textbook equips students with the
methodology and models they need to undertake their own research in
'geosemiotics',
and is essentialreading for anyone with an interest in
l a n g u a g ea n d t h e w a y s i n w h i c h w e c o m m u n i c a t e .
ISBNl0: 0-415-29048-1 (hbk)
ISBNI0: 0-41 5-290a9-X (pbk)
ISBN10: 0-203-42272=+ (ebk)
ISBNI3: 9-78-0-415-29048-7 (hbk)
ISBNI3: 9-78-0-a15-290a9-a fubk)
ISBN13: 9-78-0-203a227 z-+ (ebk)

Available at all good bookshops

For ordering and further information please visit:

www.routledge.com

Related titles from Routledge

An Introduction to DiscourseAnalysis
Theorv and MetJrod
Second Edition
James Gee
D i s c o u r s e a n a l y s i sc o n s i d e r s h o w l a n g u a g e ,b o t h s p o k e n a n d w r i t t e n ,
enacts social and cultural perspectivesand identities. ln this book, James
P a u l C e e i n t r o d u c e st h e f i e l d a n d p r e s e n t sh i s u n i q u e i n t e g r a t e da p p r o a c h
to it.
A s s u m i n g n o p r i o r k n o w l e d g eo f l i n g u i s t i c s ,t h e a u t h o r p r e s e n t sb o t h a
. l e a r l ys t r u c t u r e da n d
t h e o r y o f l a n g u a g e - i n - u saen d a m e t h o d o f r e s e a r c hC
written in a highly accessible style, Ari lntroduclion to DiscourseAnalysis
i n c o r p o r a t e sp e r s p e c t i v e sf r o m a v a r i e t y o f a p p r o a c h e sa n d d i s c i p l i n e s ,
i n c l u d i n g a p p l i e d l i n g u i s t i c s , e d u c a t i o n , p s y c h o l o g y ,a n t h r o p o l o g y a n d
c o m m u n i c a t i o nt o h e l p s t u d e n t sa n d s c h o l a r sf r o m a r a n g eo f b a c k g r o u n d s
to formulate their own views on discourse and engage in their own
d i s c o u r s ea n a l y s i s .
T h e s e c o n de d i t i o n h a s b e e n c o m p l e t e l yr e v i s e da n d u p d a t e da n d c o n t a i n s
substantial new material and examples of oral and written language,
ranging from group discussions with children, adults, students and
teachers to conversations, interviews, academic texts and policy
documents.
ISBN10: 0-41 5-32860-8 fhbk)
I S B N 1 0 :0 - 4 1 5 - 3 2 8 6 1 - 5 ( p b k )
I S B N 1 0 :0 - 2 0 3 - 0 0 s 5 7 - 8 ( e b k )
I S B N 1 3 :9 - 7 8 - 0 - 4 1 5 - 3 2 8 6 0 - 9 f h b k )
I S B N l3 : 9 - 7 8 - 0 - a 1 5 - 3 2 8 6 1 - 6 ( p b k )
ISBNl3: 9-78-0-203-00567-5 (ebk)

Available at all good bookshops

For ordering and further information please visit:

www.routledge.com

Relatedtitles from Routledge

Analyzing Multimodal Interaction
A Methodological Framework

SigridNorris
'This is

a n o u t s t a n d i n gb o o k , s t r i k i n g l yw e l l f o c u s e dt o t h e p o i n t o f v i e w o f
r e a d e r sn e w t o i t s c o n c e p t s .l t w i l l b e e s s e n t i a l r, e q u i r e dr e a d i n go n m a n y
c o u r s e sa n d w i l l b e v e r y u s e f u l i n o p e n i n g u p a m a j o r c r o s s - d i s c i p l i n a r y
literatureand approach.'- RonScollon,CeorgetownUniversity,IJSA
O u r p e r c e p t i o no f o u r e v e r y d a yi n t e r a c t i o n si s s h a p e db y m o r e t h a n w h a t i s
said. From coffee with friends to conversationswith strangers,we draw on
b o t h v e r b a l a n d n o n - v e r b a lb e h a v i o u r t o , j u d g e a n d c o n s i d e r o u r
experiences.
AnalysingMultimodal lnteractionis a practical guide to understanding and
i n v e s t i g a t i n gt h e m u l t i p l e m o d e s o f c o m m u n i c a t i o n , a n d p r o v i d e s a n
e s s e n t i a lg u i d e f o r t h o s e u n d e r t a k i n gf i e l d w o r k i n a r a n g e o f d i s c i p l i n e s ,
i n c l u d i n g l i n g u i s t i c s ,s o c i o l o g y ,e d u c a t i o n ,a n t h r o p o l o g ya n d p s y c h o l o g y .
The book offers a clear methodology to help the reader carry out their own
integrativeanalysis,and considers a range of real examples,such as trafiic
police officers at work, doctor-patient meetings, and teachers with
s t u d e n t s ,t o p r e s e n tl i v e l yd e m o n s t r a t i o n so f m u l t i m o d a ld i s c o u r s ea t w o r k .
l l l u s t r a t e dt h r o u g h o u t a n d f e a t u r i n g a d v i c e o n p r a c t i c a l i s s u e s s u c h a s
m a k i n g t r a n s c r i p t i o n sa n d v i d e o a n d a u d i o r e c o r d i n g st,h i s p r a c t i c a lg u i d e
i s a n e s s e n t i a lr e s o u r c ef o r a n y o n e i n t e r e s t e di n t h e m u l t i p l e m o d e s o f
human interaction.
I S B N I0 : 0 - 4 1 5 - 3 2 8 5 5 - 1 ( h b k )
I S B N 1 0 :0 - 4 1 5 - 3 2 8 S 5 - X ( p b k )
ISBNl0: 0-203-37 9+9-7 (ebk)
ISBNl3: 9-78-0-41s-328ss-s
ISBN13: 9-78-0-al5-32856-2
ISBN13: 9-78-0-203-379a9-3

(hbk)
(pbk)
(ebk)

Available at all good bookshops

For orderlng

and further

information

}v'w\M.routledge.com

please visit:

Relatedtitles from Routledge

Introducing Social Semiotics
Theo van Leeuwen
tntroducingSocial Semioticsis a lively look at the ways in which different
'semiotic
a s p e c t so f m o d e r n s o c i e t yc o m b i n e t o c r e a t e m e a n i n g . T h e s e
r e s o u r c e s ' s u r r o u n d i n gu s i n c l u d eo b v i o u s m o d e s o f c o m m u n i c a t i o ns u c h
a s l a n g u a g eg, e s t u r e ,i m a g e sa n d m u s i c , b u t a l s o l e s so b v i o u so n e s s u c h a s
f o o d , d r e s s a n d e v e r y d a yo b j e c t s , a l l o f w h i c h c a r r y c u l t u r a l v a l u e a n d
significance.
T h e o v a n L e e u w e nu s e s a w i d e v a r i e t y o f t e x t s i n c l u d i n g p h o t o g r a p h s ,
a d v e r t s ,m a g a z i n ep a g e sa n d f i l m s t i l l s t o e x p l a i nh o w m e a n i n g i s c r e a t e d
. r a c t i c ael x e r c i s e sa n d e x a m p l e sa s
t h r o u g h c o m p l e xs e m i o t i c i n t e r a c t i o n sP
w i d e - r a n g i n ga s f u r n i t u r ea r r a n g e m e n t si n p u b l i c p l a c e s ,a d v e r t i s i n g . i i n g l e s ,
photo.iournalismand the rhythm of a rapper's speech provide readerswith
t h e k n o w l e d g ea n d s k i l l s t h e y n e e d t o b e a b l e t o a n a l y s ea n d p r o d u c e
s u c c e s s f um
l u l t i m o d a lt e x t s a n d d e s i g n s .T h e b o o k t r a c e st h e d e v e l o p m e n t
of semiotic resourcesthrough particular channels such as the history ofthe
P r e s sa n d a d v e r t i s i n g a
; n d e x p l o r e sh o w a n d w h y t h e s e r e s o u r c e sc h a n g e
o v e r t i m e , f o r r e a s o n ss u c h a s a d v a n c i n gt e c h n o l o g y .
F e a t u r i n g a f u l l g l o s s a r y o f t e r m s , e x e r c i s e s ,d i s c u s s i o n p o i n t s a n d
suggestionsfor further reading, lntroducingSocialSemioticsmakes concrete
t h e c o m p l e x i t i e so f m e a n i n g m a k i n g a n d i s e s s e n t i a lr e a d i n gf o r a n y o n e
i n t e r e s t e di n h o w c o m m u n i c a t i o nw o r k s .
ISBN1O:0-415-24943-0 (hbk)
ISBN10: 0-41 5-z+9+a-9 @bk)
ISBNl0: 0-203-6a702-5 (ebk)
ISBN13: 9-78-0-415-24943-0
ISBN13: 9-78-0-a15-za9aa-7
ISBN13: 9-78-0-203-5a702-8

(hbk)
(pbk)
(ebk)

Available at all good bookshops

For ordering

and further

information

wlvw.routledge.com

please visit:

A libraryat your fingertips!

eBooksareelectronic
versions
of printedbooks.Youcan
storethem on your PCllaptopor browsethem online.
Theyhaveadvantages
for anyoneneedingrapidaccess
to a wide varietyof published,
copyrightinformation.
eBookscan helpyour research
by enablingyou to
bookmarkchapters,
annotatetext and useinstantsearches
to find specificwordsor phrases.
Several
eBookfileswould
fit on evena smalllaptopor PDA.
NEW:Savemoneyby eSubscribing:
cheap,onlineaccess
to any eBookfor as long asyou needit.

Annual subscriptionpackages
We now offerspeciallow-costbulksubscriptions
to
packages
of eBooksin certainsubjectareas.Theseare
available
to libraries
or to individuals.
Formoreinformationpleasecontact
webmaster.ebooks@ta
ndf.co.uk
We'recontinually
developing
the eBookconcept,so
keepup to date by visitingthe website.

www eBookstore.tandf.co.uk