Workshop at Animatronica/Microwave New Media Art Festival, (www.microwavefest.net), Hong Kong, 4-15 Nov.

2006

Time Delayed Cinema

by Alvaro Cassinelli, Assistant Professor Ishikawa-Komuro-Namiki Laboratory, Meta Perception Group (www.k2.t.u-tokyo.ac.jp/), University of Tokyo

This talk:
Experimental techniques to interactively scramble Space & Time in the moving image, thanks to the

metaphor of time as a tangible substance

...and resulting interesting visual & narrative implications.

Plan
I) Early devices to reproduce the illusion of motion
Victorian optical gadgets The rise of the Cinematograph Breaking the rules

II) Expanded Cinema (device and codes)

Deconstruction of the medium & experimental cinema

III) Space-Time Experiments:

Early analog techniques Digitally Era & the Video Cube paradigm

IV) Khronos Projector experiments

Analog techniques Digitally Era & the Gods Viewpoint paradigm

V) Considerations and Conclusion

I. Early devices recreating the illusion of motion

Early motion-illusion devices

Shadow puppets
javanese

chinese

Magic Lantern (probably 2d century, China): precursor of slide projector

Victorian Era (second half of 19th century). Systematic study of basis of the moving picture

afterimage and stroboscopic effects

thaumatrope

zoetrope

phenakistiscope

praxinoscope

kineograph kinetoscope

And next?

 thaumatropes (only two images / motion, metamorphose) [W. Herschel, 1824]

Phenakistoscope (or phantascope, or stroboscope) [J. Plateau, 1832]

zoetrope (or daedalum) [G. Horner, 1834 or Ting Huan (), 180 A.D.] praxinoscope (uses mirrors) [Ch. E. Reynaud, 1877] kineograph (just a flip-book!) [J. B. Linnet, 1886] Kinetoscope (a cinema for one person) [Th. A. Edison, 1888]

Really? Purkinje (physiologist, persistence of vision, 1818):

predicted that one day it was going to be possible to tell whole stories with moving images

The rise of the cinematograph

Photography well developed by the end of 19th
Earliest use: lantern slides (real places and people) Eventually it became possible to take and display pictures fast enough to reproduce the illusion of motion...
Cinematograph, Lon Bouly/ Freres Lumiere (~1892).

The introduction of the Kodak camera in 1888 was a dramatic event.

Cinematograph (Lumire, 1892)

Lots of limitations! (no stereo vision, no panoramic, linear narrative)
but powerful enough as to create a new form of art!

Seventh Art of filmmaking

narrative films documentary film animation film

The Arrival of a Train at the station (1895),

Freres Lumiere

[ An early divide ]
Double legacy (photography / magic lantern) of the cinematographic hardware creates a permanent tension:
Faithful depiction of reality (scientific purposes &

documentaries). From modernist realism to neo-realism. Myth of total cinema revived today with VR immersive tools?

Vittorio De Sica's Ladri di biciclette (1948). Non-professional actors, hardships of working-class

Powerful tool for (fictional) storytelling. Artistic / political uses. Persuasive (and pervasive in preTV times).
Battleship Potyomkin (1925), Sergei
Eisenstein: power of editing and montage.

Whatever way, Hollywood entertainment code dominant in the 1920s. Continuity of editing (invisible camera and sound editing)

Breaking the rules

Beyond/besides Hollywood: pre-WWII avant-garde / postWWII Experimental Cinema, Video Art and Digital Era.

Pre-WWII modernist (European) Avant-Garde:

Influenced by visual-arts movements of the time (Surrealism,
Dadaism , Cubism, Futurism, Suprematism, Constructivism) Expanded cinematographic code (through montage)
Un chien andalou , 1929 (Luis
Buuel and Salvador Dal)

Post-war avant-garde (50s, 60s):

Experimental Cinema emerges as independent genre, (New American Cinema) Explosion of the independent filmmaking Expanded Cinema experiments: extended cinematographic code (Structuralists, Fluxus, minimalist, conceptual, etc) but also extended medium. Precursor to todays new media arts

Maya Deren, Meshes of the Afternoon (1943)

David Lynch, (1997)

Lost Highway

Video revolution & Video Art (70s):

Electromagnetic support and sophistication of special effects Influences: post-structuralism & deconstruction, minimalism, pop art, Fluxus experiments (film & musical score, repetitions, etc.) Emergence of Video Art Experimental Cinema. Interactiveness, loose or no narrative. Closed-circuit and interactive installations break the traditional spectator-spectacle relationship.

Magnet TV, 1965.

Nam June Paik,

Digital revolution & digitally expanded cinema (80s~):

cheap and popular technology virtual reality links with computer game industry Computer graphics industry Virtual Cinema Digital Video Art (algorithmic image, Programming languages for the visual arts such as DBN Design by Numbers and Processing)
MACHINIMA (machine-animation): 3D games and first-person shooters as a platform for production and film genre itself. John Maeda Nature, 2005

II. Expanded Cinema Revolution & the Media Arts

End of cinematograph reign (filmic code and apparatus):

Expansion of cinematographic code (invisible edition)

Medium expansion (techniques, material detournements)

Expanded Cinema Experiments

Material experiments Retro-toys Multi-modal imagery Narrative experiments EXPERIMENTS Sound experiments Found sound and images (form of found art) Space experiments

Time experiments
Space-Time Experiments

60s experiments resumed in todays digital Media Arts

Material experiments
Challenging the convention of the filmic machine

Scratched, punched, painted the celluloid Replacement of the film by a thread or the light beam by a rope!
P. Weibel, Lichtseil, 1973

Rohfilm, B. & W.

Hein, 1968

Turning the whole mechanism upside-down New recording mediums (film, electromagnetic)
J. Maire, Demi-pas, 2004

In search of the origins (mediaarcheology idiom)

Retro-toys and the archeological idiom

Exploring the medium: back to origins & re-appropriation

Julien Maires Demi-pas (2004) or the moving magic lantern revisited Philip Worthington, Shadow Monsters, 2005

Toshio Iwai (3D zoetrope)

Toshio Iwai, Morphovision distorted house, (2005)

Space experiments
Challenging the painting paradigm: image is liberated from its frame/screen
panoramic shooting and projection ( immersiveness) support of the projection (screen: flat, curved, multiple, mobile, people, water, smoke, wood) projection environment (beyond the dark room, multitheatres, building, road from cars)
Dan Graham, Cinema, 1981 (model)

Light Attack, 2004

Daniel Sauter

Cinelabyrinth, Radz

inera, (1990)

Josef Svoboda, Polyekran, 1967

Robert Whitmans Prune Flat, 1965 (projection on people)

Time experiments (we are getting closer )

Challenging the notion of classical narrative time & montage

Real time in the movies. Experimental cinema, but prefigured by neo-realists & french nouvelle vague. Manipulated time in the movies. delayed or compressed time Psychological time / psychedelic connotations. Film shortening and lengthening, repetition.

Douglas Gordon, 24 Hour Psycho (1993)

Warhol, Empire State Building (1963), 8 hours film

Time-Lapse / Speed-Up photography

Scientific use:
Edgerton rapatronic cameras could take a million frames per second ( 1945, Los Alamos first Atomic Bomb test)

slow phenomena (ex: biology, plants, cells) fast phenomena (biomechanics, explosions)

Andrew Dunn E.M. Kinsman

GraniteBay Software

Bill Viola

The quintet of the astonished (2000)

Extreme form of slow-motion & fast-forward:

see the unseen / enhanced perception, fleeting emotions time distortion psychedelic experiences with drugs

[ Hyper-Slow motion: the Bullet-Time special effect ]

Bullet-Time effect achieved by setting dozens of still cameras surrounding the subject. These are usually triggered at once or sequentially (very similar to Muybridges 1878 setup)

Virtual Bullet-Time is much easier to do in computer graphics (virtual camera).

III. Space-Time experiments

a) Analog techniques for mixing Space & Time b) The digital Era & the Gods Viewpoint

[]the distinction between past, present, and future is only a stubbornly persistent illusion
Albert Einstein

a) Analog techniques for mixing Space & Time

Preserving the integrity of the image on the screen:
Non-linear editing can scramble the chronology, while preserving the integrity of the images on the screen

Scrambling space and time on the same screen:

Discrete: polyptych cinema Continuous!

Long exposure & Mareys chronophotography

Slit-Scan photography

P. Greenaway, The Pillow book, 1996 multi-screened scene

Example of a long exposure shot with moving subject

Chronophotography
Jules Marey Chronophotography: pictures on the same plate using a chronophotographic gun
Flying pelican captured by Marey (around 1882) Zoopraxiscope disk

Eadweard Muybridges batteries of camera setup is the precursor of bullet-time special effect (virtual camera) - the difference is that in bullet-time, all pictures are taken at the same time (a sort of ultra-slow chrono-photography)
Muybridge's The Horse in Motion (1878)

Fine art and Medical applications (study of gait, biomechanics) Imagery reminiscent of Futurism (representation of speed, motion)

Slit-Scan Photography
A moveable slide with a slit replaces the shutter
(Effect known as focal plane shutter distortion in a normal camera)
Hammer thrower, George Silk 1960

Can dramatically enhance perception of action Imagery reminiscent of early Analytic Cubism (1912): different (time) views of the same object are merged on the canvas

Bob Mumford

Streak Photography

Henri Lartigue photography of a race car: use of focal place distortion while panning Robert Doisneau, controlled slit-scan (couple spinning on a turntable )

Slit-Scan film (analog!)

Slit-Scan during shooting? (Similar to slit-scan photography, but with continuous film?)
I dont know any example of this (would need optical buffers!)

Slit Scan Movie, 35mm film (2000)

Christian Hossner

Post-produced Slit-Scan (record everything on film, delay sections of the image while printing):
time-delaying mask most likely fixed. extreme case of screen Expanded Cinema (screen smoothly broken into time-clusters)
Zbig Rybczynski, The Fourth Dimension (1988), 27-minute, 35mm color film

...edition becomes invisible again: it appears as if objects themselves are warped (special effect?)

(already explored in the pre-war avant-garde!)

Subject broken up, analyzed, and re-assembled in an abstracted form.
Subject depicted simultaneously from a multitude of viewpoints.

Analytic Cubism

Cubist/Futurist Synthetic Cubism

(i.e. papier coll)

The first Synthetic Cubist work, Picasso's Still life with chair caning (1911-12)

Georges Braque,

Woman with a guitar, painted 1913

Nu descendant un Escalier (1912)

Marcel Duchamp,

Putting a little order

shooting
Single Place (SP)
SP MP
Pictorial motif (repetition, a la Warhol)

Multiple Places (MP)

SP
SPATIAL CUBISM (Brake, Picasso) Slides, and if same subject: bullet-time special effect

reproduction

MP
Collection of stills, and if same subject: Panorama

Single Time (ST)

ST

Still, portrait

ST

MT

MT

SP

MP

SP
SPACE-TIME CUBISM (maximum mess or Synthetic Cubism)
Traveling, pan, tilt, dollying, craning etc in classical CINEMA

MP
Muybridges CHRONOPHOTOGRAPHY (Muybridges horses)

Multiple Times (MT)

ST

Mareys Mareys Chronophotogra CHRONOphy & Time- PHOTOGRAPHY based CUBISM (jumps, (Bacon, Duchamp machines) & Futurists)

ST

MT

Early CINEMA (fixed camera), TIME-LAPSE photography

MT

POLYSCREEN CINEMA (Radok, Svoboda)

Not easily categorized as a cinematographic technique or a visual genre

Working definition: time-based Cubism

Simultaneous viewpoints are points in TIME instead of views in SPACE Mareys Chronophotography (~1880): time-evolution in a single photographic plate Aesthetics related to Futurism movement (representation of speed)

Umberto Boccioni, The Charge of the Lancers, 1915

"I see every image all the time in a shifting way and almost in shifting sequences."
Francis Bacon, Three Studies for Self-Portrait, 1974 Marcel Duchamp. Nu descendant un Escalier (1912)

In analog times, painting / collage were the only ways to arbitrarily blend multiple point of views on the canvas

b) The Digital Era & the Video Cube paradigm

Total recording on cheap random access memory enables:
Frame by frame non-linear edition Total recording & total control of reproduction ( any case of table can be filled a posteriori) Gods viewpoint: total cinema, immersive cinema, virtual reality
A. Garcia adaptation (2006) of Bioy Casares Morels invention (1940)

Real-time interaction (Live Cinema: at the image level pshychadelic VJeeing, or at the narrative level)

But mass of information needs new interfaces:

Multi-track non-linear editing software Video as a volume that can be sculpted

Video Cube (from E.Elliott Video Streamer)

mass of information saturates cognition (infinite possible storylines invisible)

Still, we can appreciate this bunch of data as an object in space

Graphical-User-Interface (GUI) based software

Post-production tools Art? Special-effect tools?
Video Streamer (1992)
Eddie Elliott

Michael Cohen et al.

Stylized Video Cubes (2002)

Sidney Fels, Kenji Mase & Eric Lee Video Cubism (1999)

Martin Reinhart & Virgil Wildrich tx-transform (1998)

Explicitly treated and displayed as an object (not for editing purposes)

Tams Waliczky & Anna Szepesi, Time Crystals, (1997) Sidney Fels, Kenji Mase & Eric Lee Video Cubism (1999)

Alvaro Cassinelli, Khronos Projector (2005) Deformable screen and video volume sculpture

Joachim Sauter & Dirk Lsebrink

The Invisible Shape of Things Past (1995)

Video volume mapped in real space!

[ Toshio Iwais tour de force ]

The video-cube is not only mapped in real space:

the 3D buffer is the real object!

(this is a kind of 3D zoetrope)

Toshio Iwai with NHK Science & Technical Research and Labs. Morphovision: Distorted House (2005)

Live Slit-Scan (software invisible treats video as an object)

live video-feed

no physical interaction

Metaphor of the spatio-temporal WARPING MIRROR

T. Iwai Another Time, Another Space (1993)

Steina Vasulka, Bent Scans (2002) Jussi ngeslev & Ross Cooper Last Clock (2001)

The Time Stretcher (1988-1994)

Daniel Sauter & Osman Khan We interrupt your regularly scheduled program (2003)

Bill Spinhoven

Christian Kessler, Transverser (1998)

Interactive space-time sculpting (non GUI-based)

Metaphor of TIME AS A SUBSTANCE (oil, water-like)

Khronos Projector (2004) , A. Cassinelli

Camille Utterback, Liquid Time (2000)

Tangible interface enhances the physicality of the act:

Painting with Time

TIMESCAPE (2006), A. Cassinelli & H. Naito (Tangible + LIVE input)

IV. Khronos Projector experiments

(details of the project at the artist talk, 18:00-18:30)

Possible Khronos Projector contributions to cinematic expansion

Support of projection
tangible, deformable images on real space sensual interface ( cold digital technology)

Video as a 3D object

Found media processor

Recycling video as rough material for sculpting space-time Beyond the DVD player.

Interactive Cubism:
Smooth blending of thousands of images

Time as a physical substance

Deformable time-mirror

Interactive Paintings

Some examples (click on images to launch video)

<www.k2.t.u-tokyo.ac.jp/members/alvaro/Khronos/>

Interactive, instant Cubism

Francis Bacon, Three Studies for Self-Portrait, 1974

Three (Khronos) interactive snapshots of M. Bressaglia, 2005

Combinations: Chromatic-Time Arrow + LIVE video:

Chromatic-Time Arrow:
mixing Time and Color spaces

.. A live Francis Bacon-like portrait!

Further experiments with the Khronos Projector

Large screen & story that implies physical effort and displacement of the spectator Producing 3D prints of the sculpted video cube Mixing dimensions: color-space, gesture-space (face expressions could freeze time around the actor), etc. Full-body interaction screen (dancers inside the box).

Inversion the interaction principle: gestures in real space modify the shape of the projection screen
The Volume Slicing Display: virtual volumes intersect real space Spatialized sound (more on this at artist talk)

IV) Considerations & Conclusion

Important Questions:
Despite interactivity, most of these time-warping installations look like contemporary versions of Victorian optical gadgets playing tricks on peoples perception of reality

Can these interfaces lead to a form of performative cinema based on real-time space-time manipulation?
And in general, how much todays Media Art installations are no more (and no less) than 21st century tricks? Can we really do more than tricks? (i.e., generate a full-fledged interactive / cinematic code).

And do we need to be concerned by that???

Remember: the cinematograph was perceived as an invention without future by the Lumire Brothers. We may be just in the age of the experiments (perhaps the most fun, lets enjoy)

Some References
Expanded Cinema:
Expanded Cinema by Gene Youngblood, Publisher E.P. Dutton (1970) Online: <www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html> Future cinema: the cinematic imaginary after film, Jeffrey Shaw and Peter Weibel [Eds.], Book: The MIT Press, Cambridge, MA, 2002. Online: <http://www.zkm.de/futurecinema/index_e.html>

Slit-Scan photography and video (and more):

Levin, Golan. An Informal Catalogue of Slit-Scan Video Artworks, 2005 <http://www.flong.com/writings/lists/list_slit_scan.html>

Time Lapse photography:

Software for Canon cameras: <www.granitebaysoftware.com/Product_gbt.aspx> Plants in Motion: <plantsinmotion.bio.indiana.edu/plantmotion/flowers/flower.html>

and also Khronos Projector website:

Online: <www.k2.t.u-tokyo.ac.jp/members/alvaro/Khronos/>

Khronos Projector in cellular video mode