Interactive Content Development

Interactive Content
Development Guide
A guide to understanding and producing for
Group Interactive Immersive Environments
Rodney Hoinkes,
Rob Krieger, Editor
Immersion Studios Inc.
Interactive Content Development Guide
Assistance for this project provided by the Minister of Canadian Heritage and the
CANARIE Applied Research and Interactive Media program
Interactive Content Development Guide © 2004 Immersion Studios Inc.

All images copyright Immersion Studios Inc. unless otherwise noted.
All rights reserved.
This document, without extraction or modification is freely distributable as long as
no fee is charged without the prior permission in writing from Immersion Studios
Inc.
For the latest updates and revisions to this document, please visit http://icdg.imm-
studios.com/

285 Manitoba Drive
Toronto, Ontario M6K 3C3
Canada
info@imm-studios.com
www.imm-studios.com
All brand names and product names mentioned in this book are trademarks or
service marks of their respective companies. Any omission or misuse (of any kind)
of service mark or trademarks should not be regarded as intent to infringe on the
property of others. The publisher recognizes and respects all marks used by
companies, manufacturers, and developers as a means to distinguish their products.
Immersion Interactive Cinema and Poetic Dimensions are trademarks of Immersion Studios.
Director, Flash, Lingo, ActionScript and ShockWave 3D are trademarks of
Macromedia. PhotoShop is a trademark of Adobe. Gameboy and Gamecube are
trademarks of Nintendo. Playstation2 is a trademark of Sony. Internet Explorer,
PowerPoint, Xbox and DirectX (DirectSound) are trademarks of Microsoft. IMAX is
a trademark of IMAX Corporation. Maya is a trademark of Alias|Wavefront. 3D
Studio MAX is a trademark of Discreet. Digital Fusion is a trademark of eyeon
Software Inc. Creator is a trademark of Paradigm-Multigen. Vizx3D is a trademark
of Virtock. Designer is a trademark of Cult3D. Web 3D Repository is a trademark of
the Web3D Consortium. Modelbank is a trademark of Viewpoint. AutoCAD is a
trademark of Autodesk. GeForce2 is a trademark of Nvidia. RADEON 9800 Pro is a
trademark of ATI. Onyx4 UltimateVision is a trademark of SGI. PhotoModeler is a
trademark of Eos Systems. Polytrans is a trademark of Okino. PolyTRIM is owned
by Centre for Landscape Research, University of Toronto. ArcInfo is a trademark of
ESRI.
© 2004 Immersion Studios Inc i

Contents
Acknowledgements ...........................................................ix
Chapter 1 Introduction
Introduction ............................................ 1-1
Content Domains....................................... 1-4
Background ............................................. 1-4
Who should read this guide? ......................... 1-6
How should this guide be read?...................... 1-7
Chapter 2 What is a Group Interactive

Immersive Environment?
What is a Group Interactive
Immersive Environment?.............................. 2-1
Understanding immersion ......................................2-2
What does Immersion mean?.............................................. 2-2
What is sensory immersion? ............................................... 2-4
What is data immersion? .................................................. 2-5
Understanding group interactivity............................2-7
What is group interactivity?............................................... 2-7
Collaborative group interactivity ........................................ 2-8
Competitive group interactivity.........................................2-10
Types of Group Interactive Immersive Environments ... 2-11
Cinematic ...................................................................2-12
Real-time ...................................................................2-16
Intranet versus broadband Group Interactive Immersive
Environments .................................................. 2-19
Face-to-face communications ...........................................2-20
Interactive transmission ..................................................2-20
Immersive transmission...................................................2-21
© 2004 Immersion Studios Inc ii

Chapter 3 Perception and Cognition

Issues of perception in GIIE development.......... 3-1
Perception issues in the education sector ......... 3-1
Research to date ................................................3-3
Summary of perceptual impact factors......................3-5
Future research..................................................3-7
Perception and cognition issues in the professional
sector .................................................... 3-8
Research to date .............................................. 3-10
Future research................................................ 3-14
Chapter 4 Technology
Introduction ...................................... 4-1
Software Systems ...................................... 4-4
Coordination System .................................. 4-4
Presentation Systems ................................. 4-4
Interactivity Systems .................................. 4-6
Data Management System ............................ 4-7
Facility Management System ......................... 4-7
Media Control Components........................... 4-8
Core Components ...................................... 4-9
Computer Systems ............................. 4-10

Immersive servers ....................................4-10
Coordination servers .................................4-11
Interactivity consoles ................................4-11
© 2004 Immersion Studios Inc iii

Cinematic Group Interactive Immersive

Environments ................................... 4-12
Real-time Group Interactive Immersive
Environments ................................... 4-14
Local PD/RT server model...........................4-15
Centralized Big Iron PD/RT server model .........4-15
Grid computing server model .......................4-16
Chapter 5 Production Team Composition

Production team composition........................ 5-1
Team composition...............................................5-1
Team organization ..............................................5-1
Management .....................................................5-2
Design .............................................................5-2
Artwork ...........................................................5-2
Software ..........................................................5-2
Audio ..............................................................5-2
Quality assurance ...............................................5-2
Support............................................................5-3
Management-related roles ........................... 5-4
Producer ..........................................................5-4
Project Manager .................................................5-4
Project Coordinator or Production Assistant................5-4
Technical Lead...................................................5-4
Visualization Professional ......................................5-5
Design-related roles ................................... 5-6
Director or Creative Director..................................5-6
Writer .............................................................5-7
Content Manager ................................................5-7
© 2004 Immersion Studios Inc iv

Artwork-related roles ................................. 5-8

Art Director ......................................................5-8
Storyboarder .....................................................5-8
Concept Artist ...................................................5-9
Interface Designer...............................................5-9
2D Designer.......................................................5-9
3D Modeler .......................................................5-9
Texture Artist .................................................. 5-10
Animator ........................................................ 5-10
Compositor ..................................................... 5-10
Video Editor .................................................... 5-10
Software-related roles ...............................5-12
Interactivity developer ....................................... 5-12
3D Programmer ................................................ 5-12
Tools Programmer............................................. 5-12
Audio-related roles ...................................5-13
Audio engineer................................................. 5-13
Composer ....................................................... 5-13
Mixing engineer................................................ 5-13
Quality Assurance-related roles ....................5-14
QA Lead ......................................................... 5-14
Integration Specialist ......................................... 5-14
Usability Specialist ............................................ 5-14
Support-related roles ................................5-15
Technical Support Specialist ................................ 5-15
Network Support Specialist .................................. 5-15
Chapter 6 Production Process (IIC)

Introduction ............................................ 6-1
Key concepts ........................................... 6-2
© 2004 Immersion Studios Inc v

Show flow.........................................................6-2
Audio/Video segments..........................................6-3
Conditionals ......................................................6-4
Group objects....................................................6-5
Device control ...................................................6-6
Unique production components ..................... 6-7
Show Flow ........................................................6-7
Creating and managing the show flow .................................. 6-9
Console Networking ..................................6-13

Branching conditional group objects ....................... 6-13
Shared multi-user group objects............................ 6-14
High-performance game interactivity...................... 6-14
Scoring methodology.......................................... 6-15
Time synchronization events ................................ 6-15
Immersive Screen Sprites (for HDVS) ..............6-16
Immersive Screen Playback QA .....................6-18
Show flow and Group Interactivity QA.............6-19
Conventional production components .............6-20
Console Content ............................................... 6-20
Console Interactivity.......................................... 6-21
Console Interactivity Testing and QA ...................... 6-22
Immersive Audio/Video Content ............................ 6-23
Immersive Audio/Video Content (for DMS)................ 6-24
Immersive Video Content (for HDVS)....................... 6-26
35mm film ..................................................................6-26
High Definition (HD) live video shoots..................................6-26
Computer Generated (CG) video production..........................6-28
Interactive multi-path branching .......................................6-28
Multi-screen camera alignment .........................................6-29
Converting a 3-screen show to a single-screen show (or converting
4:3 to 16:9 aspect ratio) .................................................6-31
© 2004 Immersion Studios Inc vi

Naming Convention for Video (“Image”) Files............ 6-32

Image File Format Specification ........................................6-32
Immersive Audio Content (for HDVS)....................... 6-33
Steps for converting audio content .....................................6-33
Naming convention for audio files ......................................6-34
Immersive Video Content (for DMS) ........................ 6-35
Naming convention for DMS video files ................................6-35
Chapter 7 Production Process (ISL)

Introduction ............................................ 7-1
Key concepts .....................................................7-2
Object naming .............................................................. 7-2
Object attributes ........................................................... 7-3
Flat scenes versus scene graphs.......................................... 7-3
ISL production flow .................................... 7-5

Conventional components ............................ 7-5
Immersive visual content ......................................7-4
Immersive model considerations ......................................... 7-6
Model sources for Real-time .............................................. 7-6
Automated routines ........................................................ 7-6
Optimizing for Real-time .................................................. 7-8
Model assembly ............................................................7-13
Immersive audio content..................................... 7-15
Immersive device control .................................... 7-16
Immersive navigation Quality Assurance .................. 7-16
GIIE unique components .............................7-16
Basic interactivity Quality Assurance ...................... 7-16
Interactivity camera and model options................... 7-17
Interactivity content linking................................. 7-17
Full experience Quality Assurance ......................... 7-18
Design and planning ..................................7-19
Source Data – available databases.......................... 7-19
© 2004 Immersion Studios Inc vii

Model complexity and levels of abstraction .............. 7-20

Modeling context (versus an object) ....................... 7-21
Focus on assembly ............................................ 7-21
Construction – base model ................................... 7-23
Textures, Transparency, Shading, and Lighting .......... 7-28
Interactive moves ............................................. 7-29
Design and Planning Reference Links ..............7-31
Exhibits and Games...................................7-32
Terrain – height maps......................................... 7-32
Interiors – BSP and portals ................................... 7-32
Character animation – bones vs physique/biped ......... 7-33
Chapter 8 Conclusions
Conclusions ............................................. 8-1
Findings ................................................. 8-2
Future Research........................................ 8-5
Benefits of this guide ................................. 8-7
Appendix A IIC Evaluation

Appendix B ISL Evaluation
Appendix C IIC 3-Screen Immersive CG
Production
Appendix D IIC Object, Variable and Data
Referencing Standards for Console Content
Appendix E CANARIE Partners and Profiles
© 2004 Immersion Studios Inc viii

Acknowledgements
It is critical to extend my warmest thanks and appreciation to all those
who contributed to the contents and editing that have made this guide
possible. Of particular note is my editing partner on this project Rob
Krieger, who kept the ball rolling, stick poking, and editing moving
during the trying moments.
The larger group is recognized more fully in Appendix E, but I would
like to single out Shiraz Baboo, Arkyin Eng, Chris Quine, and
Matthew Goldblatt, of Immersion Studios and Tim Archer of Masters
Workshop, who took it upon themselves to travel the arduous path of
editing and contributing critical technical components of this guide.
I would like to thank the full Immersion Studios team who have over
the years achieved wondrous works of creativity and technical
brilliance that have ‘made it all come together’ and inspire others with
interest in this New Media approach.
Finally, I would like to thank my partner in founding Immersion
Studios and its CEO, Stacey Spiegel, who has been a constant source
of inspiration, challenge, and creativity in the evolution of these ideas
from day one. None of this would be possible without you!
Rodney Hoinkes, CTO
© 2004 Immersion Studios Inc ix

Chapter 1
Introduction
Interactive Content Development Guide Introduction
Introduction
We see the evidence of advances in content development using digital
tools every day. Fantastic worlds, battles, and special effects are now
commonplace in movies. Game developers routinely create richly
detailed and broad networked virtual worlds. Rich information spaces
multiply on the web daily. This content has led to and now serves
well-established distribution channels and formats like movies,
television, the World-Wide Web, traditional PCs and game systems.
Many books, courses, and online resources are also available to help
people express themselves in these digital media environments.
This guide focuses on an expanded set of ‘New Media’ or ‘eContent’
technologies that are starting to have a significant impact on the ways
we work, learn and play together. The expanded set of technologies
includes immersive environments, group interactivity, and rich media.
To understand why we would focus upon these capabilities, let us look
at each independently for its potential value:
Immersive environments. Being immersed in a setting, topic,
or experience has been clearly noted in research as improving
the sense of ‘presence’ which in turn has shown value in
focusing attention, amplifying perception, and expanding the
context of inquiry.
Group interactivity. Most complex situations in the world
today are structured, designed or decided upon by groups or
teams. Humans are largely social beings who gain comfort,
support and challenge from others around them.
Rich media. The growing capabilities of digital
representations have helped broaden our awareness of concepts
such as learning styles. This has demonstrated the value of
using different representational forms that allow for inclusive
situations versus language-dependant exclusion (e.g. legalese,
computer jargon).
For the past eight years, Immersion Studios has been a pioneer in the
field of immersive and group-interactive eContent. It has produced
over 14 large-scale, immersive group-interactive content productions
and developed a high-end intranet “theater” product to run and create
advanced eContent. It has also installed over a dozen group interactive
immersive environments (theatres, classrooms, etc.) worldwide in the
past four years. The popularity of its content and approach to advanced
eContent has led to a number of post-secondary curriculum courses,
the technology being used as a foundation for university research
© 2004 Immersion Studios Inc. 1- 1

projects and several 3rd-party production companies pursuing the

market.
When Immersion Studios started its research, it set out to develop the
“right” characteristics of a system for advanced eContent rather than
building based on what “can readily be done” with current network
limits. It has set its sights on finding the right capabilities for
interactive experiences and then finding ways of supporting and
achieving these capabilities through technology. This has resulted in
its intranet theater technology, streaming more than 1Gbps of data.
The value of these experiences for education and entertainment has
been the focus of studies in media psychology over several years by
Prof. Dr. Peter Vorder and his Masters students at the University of
Hannover (and now at the Annenberg school at USC in California).
This guide is focused upon explaining the integration of these
concepts. It will do this within the construct of what we call a Group
Interactive Immersive Environments (“GIIE”). The guide is designed
to assist those interested in designing GIIE experiences, producing
content for them or evolving the state of the art from a research
perspective. The guide will not only focus upon today’s capabilities,
but will present research and challenges for the broader application of
these concepts to next-generation broadband networks (as exemplified
in Canada by CA*Net 4).
With the rapidly increasing capabilities of broadband networks, there
are significant market potentials for adapting what has been developed
in the high-end intranet environment. What must be understood are the
challenges to overcome in adapting this technology.
For the purposes of this guide, we define the existing baseline as a
350Mbps to 1Gbps networked environment (intranet), and the
broadband environment under examination to be between 10 and
100Mbps. To address the underlying characteristics that are likely to
change, this guide focuses upon three primary objectives:
1 Perceptual and cognitive foundations.
Description of and differences in end-user perception and
understanding of immersive interactive eContent delivered to
different audiences.
2 Technological foundations.
Description of and broadband adaptation issues involved in the
technology for delivery of immersive interactive eContent.
3 Production foundations.
Description of and broadband adaptation issues involved in the
production of immersive interactive eContent.

Content Domains
This guide focuses on two content domains of emerging value to new
media content production. The first, is the educational context, in
particular the domains of “experiential learning”, “informal learning”,
and “simulation learning.” In this context, the underlying systems are
typically formalized and scripted with clearly targeted learning
objectives. The technology is often based upon pre-rendered computer
graphics for accuracy, perceptual quality (important for user
engagement), and commercial viability (in comparison to other
products/experiences).
The second content domain is focused upon professional applications.
Professionals are using immersive, group, and rich media
environments primarily to support decision-making. The
representation and engagement of political, financial, cultural issues
and multiple points-of-view are critical to this group. The underlying
systems must be more dynamic, highly iterative and extremely
interactive. The technology is typically based upon real-time computer
graphics for malleability.
Each of these domains focuses upon particular technology types,
interaction domains and user expectations. We focus upon these, as
they have been the early adopters of GIIE-type experiences. These
types of user contexts contain many of the same qualities that are
present and needed in more generalized content domains such as
entertainment, training, and simulation, clear future market domains
for these concepts.
Background
There are no development guides of this type to help developers
seeking the limits of high-end richly integrated and interactive
production. This guide focuses on the ideals of production that will
reach a significant, compelling, and truly useful level, rather than
simply what can readily be achieved using conventional digital media
approaches today. It establishes a bar for future development in
specialized venues and the challenges posed to wider distribution
through broadband networks.
Currently, GIIEs are limited to specialized venues. These venues are
usually found in museums, science centres and schools whose
primarily objective is educationally-oriented. Within the professional
and research realms, GIIEs have primarily been located within design,
planning and environmental-studies related universities and
organizations.

The participants in the development of this guide cover a broad

spectrum from research, to production businesses, to teaching. Many
participants are currently using the GIIE concepts and technology
developed by Immersion Studios Inc. over its past eight years. This
guide is a vehicle to expand the awareness of students and new media
producers, foster future content production for GIIEs and highlight
where the potential exists for broadband adoption of GIIE
technologies and content. Where significant challenges exist, this
guide seeks to help define directions for future research in advanced
eContent.
The experiences, technologies and content productions developed by
the participants and captured in this guide define the current state-of-
the-art in network-based group interactive immersive experiences.
This experience has been taken as the baseline for this guide. Research
into related works in contributing areas are presented to support and
challenge the reader’s understanding and expand the natural biases
presented by the guides’ producers.
Finally, a set of research experiments have been undertaken by these
participants to understand the importance of different factors in a GIIE
when converting from a specialized venue (an intranet) to a broadband
environment. These experimental observations are noted throughout
this guide to assist readers as to the key technical and production
factors that must change for broadband adaptation of GIIE content.

Who should read this guide?

A wide range of readers should benefit from this guide, in different
ways.
Audience Benefit
Educators Provides a model for future new media
applications to apply to education that
have proven learning value.
Supports ongoing research into rich
enabling technology.
Students Provides directions in new media that
may transform future professional’s
approaches, tools and capabilities.
New Media Demonstrates new approaches that have
Producers current market value and future
networked distribution opportunities.
Provides useful notes on new media
adaptation to broadband internet
distribution.
Technology Provides approaches to technology
Developers development, challenges, future
opportunities and areas of needed
research.
Suggests changes to development
methods.
Researchers Provides focus for future content and
broadband technology research.
Identifies key areas requiring future
research in perception and cognition.
Business Suggests opportunities and challenges
Development for future markets and distribution
Managers channels.
Suggests levels of technology and
production to support markets.
Provides information on end-user
expectations.

How should this guide be read?

For those readers who are interested in the conceptual value,
approaches and challenges to wider adoption of GIIEs, the first part of
this guide, chapters 2 and 3 should provide a useful resource. Chapter
2 explains the key concepts of a GIIE, the two most common types of
GIIEs and the key issues for adapting these concepts to broadband
environments. Chapter 3 discusses a range of perceptual and cognitive
issues and their potential impacts upon a participant’s experience and
the related influence upon technical and production factors.
For those interested in the specifics of technology and production of a
GIIE, the second part of this guide will be most useful. In part two,
Chapters 4 and 5 are each divided into two sections, focusing first on
the Cinematic and second on Real-time environments. Due to the size
and specificity, Chapters 6 and 7 are each dedicated to only one of
these environment types but in the greatest depth.
Where concepts get highly specific, description lengthy, or details are
considered optional, this material has been placed in appendices to
keep the core guide focused and accessible.
While this guide emphasizes the differences between the Cinematic
and Real-time models of technology and content, this should be
understood by the reader as a current conceptual and technical
differentiation only. Cinematic environments are defined by the nature
of their quality, usually in terms of visuals. Real-time graphics are
often focused upon for their malleability. The future of technology
seems clearly poised for significant change and the boundaries
between what have often been considered lower-quality (but more
interactive) simulations and games will blend with the high fidelity
imagery presented through cinematic representations.
However, the current definitions are not about to change completely or
rapidly. As long as live actors or locales are valuable, desirable or
necessary and cinematic real-time effects limited in scope or by
expensive bleeding-edge equipment, then the current separation will
continue. It will continue to be important to understand the practical
limitations on content production and distribution.
We hope that this guide proves useful to you, can inspire and promote
the next generation of eContent production and evolve the technology
platforms and opportunities for engaging, rich and supportive work,
learning and play.

Chapter 2
What is a Group
Interactive
Immersive
Environment?
Interactive Content Development Guide What is a GIIE?
What is a Group Interactive Immersive

Environment?
“Tell me and I’ll forget, show me and I may remember,
involve me and I’ll understand”
Old Native American proverb

Hi-tech media pervade our lives and are increasingly important in our
work, study and play. Web-based content and e-learning have
significantly changed our understanding of linked media experiences.
With greater interactivity in educational multimedia and games and
development of more stimulating programs the way we can learn is
rapidly changing. Immersive Virtual Reality environments are
becoming more pervasive where working with material in context is
critical such as in design, planning, and management. As much as each
of these capabilities is advancing, they are often seen as independent
and contrary, competitive, or exclusive.
What is perhaps more troubling is the pervasive focus upon single-user
experiences, single-user interfaces, and single-user engagements. In
our lives, we perform many actions as an individual; many more are
influenced, shared, or learnt in a social setting. It is time we put as
much effort into the developing contexts that promote the social
construction of knowledge as those developed for the individual.
Group Interactive Immersive Environments (GIIEs) are an inclusive
concept of social media interaction. GIIEs are built upon an active or
“hands-on” approach to the application of knowledge within social
contexts. Unlike many media pursuits, this is not a reductionist view,
rather a more holistic view of new media integration and interaction. A
GIIE combines concepts of immersive environments to establish
context, rich media integration to provide critical knowledge, and
interactivity to support investigation, play, collaboration, and
competition. A GIIE provides a range of media, representations, and
modes of interaction that support diverse learning, working, and
playing styles, providing a level playing field for dialogue and debate.
To introduce the concept of a full Group Interactive Immersive
Environment, it is useful to understand the two key, and typically
discrete components, that of ‘immersion’ and ‘group interactivity’. An
understanding of these two terms and the challenges of their interplay
are the critical framework for this guide.
© 2004 Immersion Studios Inc. 2-1

Example of an Immersion Interactive Cinema (IIC)

The picture above shows an example of an Immersion Interactive
Cinema (IIC). An IIC offers a large-format, panoramic triple-screen
theatre in high-definition (3072 x 768 pixels) with Dolby Digital
surround sound. The audience, referred to as users, interacts with the
content via touch screen LCD consoles to control an experience. This
is performed in a myriad of ways such as by entering choices via using
voting, playing games or performing personal explorations. The
content is seamlessly integrated and synchronized between the big
screen and the consoles.
Understanding immersion
What does Immersion mean?
“If you have ever settled into a book for the evening,
leaving your body to occupy that of the protagonist, you
know that immersion is powerful.
If you have ever played make-believe with a child,
delving into the drama of the quest, you know that
immersion is fun.
If you have ever entered “The Zone” while crafting a
piece of prose, poetry or programming, you know that
immersion is productive.”
Linda Jacobson, Immersion Quest1
1 http://www.abbedon.com/electricminds/html/edg_vr_1668.html

Immersion is simply referred to here as a state of being encompassed

in a context strong enough to allow for the suspension of disbelief.
This state of presence allows for a greater focus on activities at hand.
Where immersion encompasses enough breadth, it can provide a
stronger awareness of the context in which an activity occurs.
Many common uses of the term immersion focus upon the sensory
aspects, typically the visual, and often the auditory senses. An IMAX
movie or VR game are examples of this type of immersion.
In a GIIE, immersion is not limited to a sensory definition, but also
includes what we call data immersion. Being fully engrossed in a
documentary, researching a topic of interest, or studying a foreign
language are examples of data immersion.
Sensory Data
Immersion Immersion
Hollywood
Documentary
Blockbuster
Video Games Most Websites
Virtual Reality Non-fiction

Sensory Immersion versus Data Immersion
The concept of sensory and data immersion are not mutually
exclusive. In GIIEs focused on work or learning, the ideal includes
both sensory and data immersion.

What is sensory immersion?

In the Nineteenth Century, panoramas, dioramas, and cycloramas
became an accessible, understandable and immersive visual sensory
experience available to the public2. The scenes presented places that
were otherwise inaccessible due to distance, often foreign cities,
distant areas of nature (i.e. the New World, the Arctic), or battlefields.
One of the first cycloramas was opened in Edinburgh in 1787 by the
Irish painter Robert Barker, who, when climbing the hill in the city of
Edinburgh challenged himself to find a way to capture the expansive
view.
The Nineteenth Century saw a proliferation of these concepts and the
range of their use. In Paris in 1822, a French painter and inventor
named Louis Daguerre created a three stage dioramas, each with a 72’
x 48’ canvas showing scenes such as an Alpine village before and after
an avalanche. Some of dioramas even included special lighting and
smells. A key challenge of creating these immersive presentations was
the need for large or custom-built structures and the time needed for
crafting the single frame.
Cyclorama of Early Melbourne, Australia
Detail of a Cyclorama of Early Melbourne, Australia

In Canada in the 1960’s and 1970’s new technologies were pushing
the idea of immersion to new levels. IMAX launched at Expo ’67 in an
early form supporting motion video at a scale rarely seen in the past.
However, space requirements, cost of installation and costs of
production still to this day make this type of experience highly
specialized and limits its use. In the 1970’s at the University of
2 Oetterman S (1997). The Panorama: History of a Mass Medium. Zone Books MIT Press,
Boston

Toronto, Jerry Angler developed a system for full 360-degree

panorama using 35mm slide projectors. While not as advanced as
IMAX, the technique was a cost-effective way to display full surround
imagery. It became practical for anyone, student designer or public
artist to use in perceptual and spatial analysis of design, something
rarely pursued in the past.
Despite greater access of these immersive experiences and significant
improvements in production methods and costs3, even today, most of
our experiences are still highly framed, picture-window experiences –
computer screens, televisions, photographs, and prints.4 One might
argue that things are improving with the advent of home theater
surround sound, HDTV and multi-screen capable video cards for
computers. The reality is that these are baby steps in sensory
immersion, and while useful, are still the lowest common denominator,
not that which reaches a critical threshold of user experience.
We are left with two options: on the one hand, high-end experiences
for which installation, technology, and most critically production
needs make them largely inaccessible (IMAX for example) and on the
other, low-end standards of technology and installation that everyone
can use but rarely reaches a high level of sensory immersion without
huge production costs (for example, Hollywood blockbusters
transferred to DVD or multi-million dollar video games).
Currently, most users in Group Interactive Immersive Experience
research and development are working towards what might be called
the sweet spot in the middle of these upper and lower thresholds.
While cost of installation is still a factor, the steady changes in
computing power are removing this obstacle rapidly5. The biggest
challenges are still in the area of content production. This guide
focuses extensively on content production models for immersive
sensory environments.
What is data immersion?

Looking at any scene in the real world is not a pure act of sensory
reception and geometric processing. Interpretation is the term we use
to describe the understanding derived from the sensory input of the
3 Digital video, stills, CG production, etc.

4 Gibson JJ (1979). The Ecological Approach to Visual Perception. Houghton Mifflin,
Boston
5 What used to require high-end SGI computers at .5 to multiple millions of dollars in
hardware can now be achieved (to a large degree) with machines costing only $3-5,000,
even high-end laptops!

environment when processed through the associative knowledge a

person possesses. The range of knowledge that influences our
interpretation is as broad as the cultural, historic, and disciplinary
backgrounds of the person making the interpretation. As we gain more
knowledge, interpretations are often given to change. We can readily
see these influences in the changing styles of our designed landscapes
throughout history. The control of the French classical gardens to the
horticultural English garden and current trends in natural and
uncontrived landscapes clearly show influences beyond a common
visual interpretation.
It can also be argued that one of the most important roles of visual
representations is that of amplifying the mind, as a tool for dialogue
(Robbins, 1994)6. Robbins discusses the cultural and societal values
that are expressed or purposely rejected through the production of
drawings. In these aspects a drawing is primarily a vehicle for
discussion or debate of ideas, not simply a representation of a physical
environment. This is an implicit added value of what may appear
simply as a sensory immersion experience.
The other side of data immersion is more explicit in its expression.
Spreadsheets, text, web data, scientific visualizations, reports, and
database content are all representational forms that are heavily data-
oriented. Support for a broad range of data sources is an essential
element of an inclusive immersive environment. In design disciplines
for example, using only visual artifacts such as maps, plans, paintings,
photographs, and videos can exclude from debate, those who are not
spatially literate. Likewise, using only data artifacts such as
spreadsheets, text, web data, scientific visualizations, reports, and
database content can exclude those who are not data literate (simply
think about most legal documents you have read).
The GIIE concept tries to bring the right information, in the right
representations to bear on our professional and educational processes.
This is often discussed as a learning or participation style. Grasha
(1996)7 has defined learning styles as, “personal qualities that
influence a student's ability to acquire information, to interact with
peers and the teacher, and otherwise participate in learning
experiences” (p. 41). The same qualities can impact a professional in a
work environment or a participant in a play experience.
6 Robbins, Edward. April 1994. Why Architects Draw. MIT Press (Cambridge, MA) ISBN
0-262-18157-6
7 Grasha, A. F. (1996). Teaching with style. Pittsburgh, PA: Alliance.

Understanding group interactivity

What is group interactivity?
In his 1994 compilation Sociomedia, Edward Barrett8 coined the term
“sociomedia” to express the social purpose behind hypermedia (a
machine conception). He defines this term to imply the use of
computer media to: “objectify, exchange and collaborate, invoke,
comment upon, modify, and remember thoughts and ideas.”
In an educational context, the social construction of knowledge plays
many significant roles. Ritterfeld et al.9 note that interactive play in
educational settings helps foster teamwork. Furthermore, collaboration
among students themselves and with their teachers requires interaction
including conversation and argumentation, which in turn is linked to
enhanced cognition, cooperation and shared satisfaction10. You can see
this clearly for example, when pairing novices and skilled students in
interactive activities. There is also some belief11 that under these
circumstances novices perform at levels slightly above their normal
levels of competence.
Ritterfeld et al. note the critical concept that by introducing play in the
classroom you quickly remove hierarchies, apprehension of failure,
and stress. This has also been noted in the professional applications of
GIIEs that were studied in this guide, where the normal barriers of
professional language and formality quickly broke down through
heightened group interactivity.
Group interactivity is a more accessible term for some concepts in
sociomedia. On one level, sociomedia implies actions between people
using media, while it also implies personal social interaction with
media (objectify, remember, reflect). In this guide, priority is placed
upon interactions between people using computational media, while
focusing upon the shared network effect of interactivity.
8 Barrett, E, (1994). Sociomedia Multimedia, Hypermedia, and the Social Construction of

Knowledge. The MIT Press. ISBN 0-262-52193-8
9 Ritterfeld, U., Weber, R., Fernandes, S., Vorderer, P., (2004).Think science!
Entertainment Education in interactive theaters, ,Computers in Entertainment (CIE),
Volume 2, Issue 1 (March 2004). Association of Computing Machinery. ISSN:1544-3574
10 Frederickson, E. (1999). Playing through: Increasing literacy through interaction. J.
Adolescent and Adult Literacy 43, 2 (1999), 116-124.
11 Vygotsky, L. S. (1997). Educational Psychology. St. Lucie Press, Boca Raton, FL
(originally published in 1926).

When personal social interaction with media is discussed, the term

personal interactivity will be used. This guide is intended to be more
inclusive and therefore will occasionally refer to personal interactivity,
as it is a significant production component. More in-depth information
about personal interactivity can be found in references cited by the
authors of this guide in Appendix B –IIC Evaluation. In this guide,
detailed evaluations will largely be restricted to group interactivity.
On a technical level, the difference between personal interactivity and
group interactivity also serves as an important point of discussion in
this guide. Interactivity that expects a single input (personal
interactivity) need only have one axis of output influence on the
experience. With group interactivity, you not only have multiple inputs
to consider and how each affects the experience, but also how the
multiple inputs may affect each other. As the inputs increases, the
computational challenge increases exponentially due to potential
cross-interactions.
When considering user perceptions, a critical assumption of group
interactivity in this guide is that groups interacting in all current – and
studied – types of GIIEs are located in the same place at the same
time. Future studies can and should focus on the distribution of the
group and its effect on group interactivity; however this aspect was
beyond the scope of this guide.
There are however, indications of the challenges this will pose from
other studies, particularly on distance education. Diaz and Cartnal
(1999)12 have noted that on-campus students showed far more
collaborative and competitive group tendencies due to perceptions
about being good class citizens. In online situations, these types of
group interaction were not obvious or natural and often occurred only
when forced or strongly guided by instructors. The sense of reward
from being a good class citizen has not shown itself to be self-evident
in online environments, which may hinder or certainly change how we
engage group interactivity through distributed networks.
The spectrum of group interactivity can be broken into two sub-
classifications: collaborative interactivity and competitive interactivity.
Collaborative group interactivity

Collaboration is often described as a sharing or joint work on tasks
between two or more individuals to achieve an objective. Another type
12 Diaz, D.P. & Cartnal, R.B. (1999). Students’ learning styles in two classes: Online
distance learning and equivalent on-campus, College Teaching 47(4), 130-135.

of collaboration involves the division of tasks between two or more

individuals, which when combined can achieve an objective. These
forms of teamwork are commonly used in work, play, and increasingly
in learning environments.
Butler and Coleman13 suggest a scope for collaboration that includes
five fundamental models.
Model Description
Library A few people place material in a repository,
many draw on it.
Solicitation A few people place requests, many respond, for
example, a Request for Proposal system.
Team A small group working together on a project.
Community A Community of Practice or Interest, where
users seek out opportunities to share and help
each other in pursuits.
Process Complex systems that require combined skills
Support to handle exceptions.
In many over-simplified group interactivity immersive environments,

the only form of collaboration supported involves voting. This form
simply takes the multiple inputs, picks the highest selection level and
converts it to a single output. While voting allows for rapid mapping to
traditional, single-user software input models, it does little to support
the range of collaboration defined by Butler and Coleman.
13 Butler ,Timothy and Coleman ,David (2003). Suggest five fundamental models of
collaboration (figure reproduced from Collaborative Strategies newsletter).

Collaborative results compared against personal choices in an ecological discovery

experience (Immersion Studios “Storm Over Stellwagon” IIC Show).
Competitive group interactivity

Competitive group interactivity is defined in this guide as the
challenge involved in multiple users striving for the same resources or
goals. Competition is however a concept of significant concern and
debate.
Competition between businesses is generally recognized as providing
societal benefit. Competing firms force innovation and cost-
effectiveness. Efficiency is rewarded and inefficiency punished. While
collaborative work is generally considered more productive for small
teams, competition also at higher levels in companies is also
recognized as valuable. In large companies, M-Form concepts force
divisions to compete with one another for resources, while on other
levels they must cooperate. Some research has shown that in large
companies at least, M-Form is one of the few business concepts that
are successful over the long run14.
14 Ouchi, William G. (1984). Teamwork and competition: how the Japanese computer
industry has developed. . Creative Computing Vol. 10, No. 8 / August 1984 / Page 145.
http://www.atarimagazines.com/creative/v10n8/145_Teamwork_and_competition.php

Example of a user being the first to discover something in this medical simulation
environment, gaining extra points and status. Discoveries in this environment are
shared with the team to encourage further inquiry and to reward this user (Immersion
Studios “Vital Space” IIC Show).
Within learning experiences, there is a greater divide as to the value
and role of competition, in particular with youth. Where winning is the
key goal of competition, only a few people can succeed while the
majority feels beaten and defeated. Where the focus can be upon the
means rather than the end (winning), many benefits can be observed:
Increased learning, performance or retention (simple drills or
speed-related tasks for personal improvement).
Measure oneself providing goals for improvement.
Sense of accomplishment through personal progress
measurement.
Monitoring method assisting in the identification of problems.
Types of Group Interactive Immersive

Environments
For the purposes of this guide, Group Interactive Immersive
Environments are classified into two functional groups: Cinematic and
Real-time, based upon distinct technical and production differences.
Cinematic systems are correctly best used in entertainment-oriented
and public environments. In these contexts the quality of the

immersive environment has been emphasized due to the routine

comparisons against competitors such as 35mm film or IMAX (70mm
film). Real-time systems are best used in dynamic discussion and
decision-making environments where rapid change and testing in
simulated settings take priority.
While many components of these systems can be mixed for specific
purposes, most applications are commonly broken into these two. It
should also be noted that real-time systems are increasingly moving
towards cinematic quality, driven by improved technological
components, such as graphics card and increased computer processing
power. This will take several years to become viable but should be
monitored as a future trend.

Cinematic
The extreme of the cinematic.

Within the public sphere of paid experiences, for example movies,
IMAX, and motion rides, audience feedback and expectation clearly
push towards the extremes of the cinematic. In this context, cinematic
refers to the audio and visual fidelity typically associated with cinema
shows. This implies (but does not force) characteristics such as high-
resolution imagery, advanced lighting, depth-of-field, special effects,
environmental effects (snow, rain, fog), real-world reflections, motion,
and physics.
A cinematic approach to an interactive immersive environment has
two critical components:
Use of video (pre-generated or captured) with fixed frames.
Production of cinematic effects.
Using video has one key benefit and one key limitation. The key
benefit of video is that it provides a high quality of visual
representation to the users. As a limitation, video is pre-produced and
linear in production. This means that all interactive options must be
pre-conceived, and pre-produced, with each option another complete
video segment. Of constant concern with video is the users’ belief in
whether they, as individuals, are truly affecting the show. If user
actions are not rapidly or realistically portrayed, the sense of
interactivity can be very quickly shattered.
This introduces significant combinatorial challenges where a show
design may wish interactive choices after interactive choices. In a

simple, 3x2 interactivity flow, where there may be three options that
each have two further options, the production effort can multiply
significantly.
Intro Intro
Challenge Challenge Challenge

Challenges
A B C
Solution Solution Solution Solution Solution Solution

Solutions
A-1 A-2 B-1 B-2 C-1 C-2
Conclusions Conclusion
Example of a show flow diagram.

In the above show flow, if each box represents a 5-minute video
segment, and a linear show might be 20 minutes in runtime (left), an
interactive version can quickly require two, three, or more times the
production effort, in this case a 55 minutes show total (right).
A further challenge to this combinatorial problem is the desire to
support high-resolution video. Typical 35mm films use 2k+ (2000+
horizontal pixel resolution) imagery in their digital processing. This
puts production into a specialized realm of High-Definition (HD)
video or higher. The hardware and software to manipulate this type of
content are still not in general use and so costs significantly more to
produce.
We can summarize the critical impacts of using video as:
A significant multiplier on production show lengths.
A requirement for pre-definition (and therefore limitation) on
interactive choice.
A high hardware and software cost for production of this
resolution and volume of material.
A high quality of visual representation to the users.
The second component of the cinematic approach is production effort.
While 35mm film and HD video using live environments or actors are
commonly produced, the costs and limits of these productions often
means that live alone is often not sufficient or desirable. Film and
video shoots are quite expensive and can only readily create certain
effects without extreme cost.

Many cinematic GIIE productions use computer graphics (CG)

extensively or blend it with HD video. CG is much easier to edit and
can show the unseen or alternatives that may not even exist to shoot
live. An additional benefit for immersive environments is that CG can
achieve higher-resolutions than standard HDTV or even 35mm film
since it is largely resolution-independent. Another challenge of using
live material arises due to the flat frames in typical live recordings
whereas immersive environments can be curved, domed or whatever
shapes provides the best peripheral effect for users.
CG production has three key benefits over live shooting in GIIEs:
CG can handle the higher resolutions of many GIIEs.
CG can handle the variety of display formats that are non-flat
framed.
CG can more readily be transformed to create interactive
‘alternative’ sequences.
If one accepts a larger role for CG in these environments, the impact
on production effort and time must be carefully considered. Most
cinematic CG effects are highly computational tasks. A single
Hollywood-scale film like Toy Story had single frames that took
anywhere from 45 minutes to 20 hours to process – each! While
computational power continues to grow rapidly, so too do our
expectations of CG.
The current state of the art is far from what can be imagined in the
future. As techniques and computation capability improve, it may be
possible to apply dynamic changes to video or generate some
interactive alternatives on the fly. One method discussed in this guide
involves using dynamic overlay sprites15. Graphics hardware designers
are also looking to a future where real-time computer graphics may
compete with high-end computational rendering. It is not yet clear
when this might happen or what the perception threshold might be, but
it will be no easy task to reduce what now takes 20 hours to process
down to something computable within 33 milliseconds (1/30 of a
second).
15 Bitmaps and graphics that can be dynamically manipulated on top of running video.

Real-time
Example of real-time graphics (Immersion Simulation Lab, courtesy University of

Toronto, Centre for Landscape Research).
Real-time graphics are defined here as displayed imagery that can be
dynamically adjusted at a rate of 30 or 60 discrete images per second.
This is typically the type of representation in video games, scientific
visualization or developmental design and engineering. In other
definitions, real-time means the ability to change within a timeframe
that matches the users’ ability to comprehend and decide upon change
which can be as low as 2 frames per second. A measure of 30+ frames
per second (fps) is generally accepted in the industry.
The use of real-time immersive representations has a number of
benefits. The most important benefit is the vastly improved ability of
users to interact with the content thus allowing greater collaboration,
dialogue, and debate without cinematic limitations on choice. On a
technical production level, there are also a number of benefits:
No extensive processing time (only what can be computed on-
the-fly will work).
Relatively easy to support dynamic changes to reflect extensive
interactivity without pre-definition.
Easy support for high resolution and multiple screen
immersion.

Linking data to representations is more easily accomplished, as

all objects are clearly referenceable and therefore linkable.
Real-time environments in a GIIE have minimal time (33ms) to
perform all group interaction dynamic changes and render all sensory
information (primarily audio and video). This means that most work
goes into constructing or producing content that will perform within
these limitations. The key benefit is that content and interactivity can
be tested immediately without the days or weeks that often go into
each CG test in a cinematic environment.
As discussed in the cinematic section, any CG worlds, whether pre-
rendered or real-time, has a clear advantage in that it supports multiple
cameras, screens, and projection format that are not as readily
supported in live video. Typically each screen or angular division (we
typically use 45 degrees per screen) of a projection surface can have its
own virtual camera and viewport. Finally all viewports are
synchronized such that the overall imagery moves and changes
simultaneously.
Most immersive visualization systems are purely concerned with
visual representation and simulation. The GIIE adds the ability to link
in a wide variety of rich media that add value to decision-making and
comprehension. This rich media can be shared with a group (on the
big-screens), or explored personally on interactive consoles.
Commonly used rich media in the Immersion environments include:
digital video, imagery, Flash interactives, websites, and PowerPoint.
Within the real-time GIIE this linking of sensory and data immersion
is accomplished easily as a clear reference point exists in the naming
of each object in the real-time world. Cinematic environments flatten
this object relationship, making additional information linking a
tedious or often-ignored activity due to complexity and production
cost.
Rich Media Linking, the connection of hypermedia to interactive 3d models.

Along with the benefits of real-time immersion, there are many

challenges including:
Quality of graphics is below many publicly-accessible
representational forms including video and film.
Playback requires high-end graphics capabilities.
Difficulties handling live material and real actors.
Modeling and animating live creatures and environmental
dynamics is difficult with acceptable quality.
False stage set design philosophy of cinematics is quickly
exposed and unacceptable.
Production requires skill to manage appropriate abstraction and
representation levels.
Real-time audio is a less understood design environment than
pre-produced linear forms.
Despite improvements in real-time graphics over the past decade, the
quality is still not perceived as close to that of cinematic video and
film projects using live footage or computationally intensive CG. This
will likely remain the case for many years to come. In the future this is
likely to change such that CG pre-rendered material and real-time
graphics will be acceptable to the general public. CG will continue to
require the highest level of computer and graphics card capabilities on
the market and so limit its general accessibility.
Real-time environments also have a media format problem with live
footage. While a number of techniques (video billboarding to name a
popular one) exist to incorporate live video content into real-time
environments, this blending is usually unsatisfactory and obvious to
the user. Improvements in compositing of real-world video and
managing camera matching between virtual and real (such as that
found today in virtual sets) will help improve blending, but the
introduction and management of extra tools in the 33ms timeframe of
a real-time environment remain research and development challenges.
Of equal challenge to research, development, and integration is the
incorporation of more realistic virtual creature (including human) and
environment behaviour. This might include physics governing the
interactions of objects, character motions and lip synced audio, wind
effects on leaves in trees, or realistic clouds. Many of these techniques
are being worked on as individual actions that can be made to work in
real-time. As these types of effects are combined however, they can
quickly begin to overwhelm any current hardware capabilities.

Within an immersive environment that gives users greater freedom,

comes greater production challenges. As people look around more,
you can’t do what moviemakers do, using false stage sets. This places
a much higher demand on the quality of the foreground throughout the
entire environment. Managing these representations within the
computational budget of a real-time environment requires well
designed tools and most importantly, well honed modeling skills along
with an understanding of how representation and technical limitations
intersect.
Real-time environmental audio is very different in production
methodology from cinematic linear audio. Audio clip design,
placement, and spatial parameters are all distinctly different in both
language and style and require a different experience than is
traditionally taught. This extra effort can often reduce or eliminate the
use of spatial audio where budgets are limited.
Intranet versus broadband Group Interactive

Immersive Environments
The GIIE starts with what is necessary to achieve the right capabilities
for a group to interact with the right information to reach their
objective, whether in work, learning, or play. GIIE design has focused
on finding ways of supporting and achieving these capabilities through
technology. Research, particularly on learning, has demonstrated that
this model of an intranet GIIE design can be extremely effective (see
Chapter 3 – Perception and Cognition, and Appendix B- IIC
Evaluation.
The argument for broadband over intranet GIIEs includes two key
factors. The logistics of bringing teams together for complex
collaborative work, or simply the costs of travel and time involved are
one. The ability to access new types of content for learning and play
without the need for specialized facilities is another. For the content
producer, a move to broadband could open up new distribution routes
and access to users that could ultimately lead to a sustainable and
profitable industry.

As desirable as a move to broadband, may be there are differences that

should be evaluated to ensure that broadband GIIEs are as viable and
productive an experience as intranet GIIEs. Some of these differences
include:
Face-to-face communication and presence.
Transmission issues involved in the interactive environment.
Transmission issues involved in the immersive environment.
Many technologies have been developed over the years to pursue
broadband group interactivity and deal with these issues. Often the
overriding factor has been the need to work within the technical limits
of broadband or narrowband pipes. This is a lowest-common
denominator approach to design. This is contrary to the fundamental
design of the GIIE to date and must be challenged to ensure that
compromises do not undermine the successful translation from intranet
to broadband either from a technical or perceptual perspective.
Face-to-face communications
Face-to-face meetings and interactivity have a great value as noted in
research cited in this guide. There is also evidence that the translation
to online environments is not without difficulty. This guide does not
directly answer the challenges to this aspect of the GIIE but does
identify a number of concerns for future evaluation.
Interactive transmission
Transmission changes within the interactive environment are largely a
pure technical challenge. Four key challenges exist in the broadband
transmission:
Firewall/NAT handling for security.
Bandwidth for transmission of interactivity data.
Latency in transmission of interactivity data.
Guaranteed concurrency of interactivity data/state.
Firewall/NAT (Network Address Translation), which hides a users real
address from the Internet, also means that establishing connections TO
a user is very tricky, and designs should be focused on a user
connecting TO the server (which is more standard in any case but this
is an issue). Also the use of not standardly used ports (80, 23 are
standard for web and mail) can lead to firewall issues and users
opening dedicated ports. Which ports are used for interactivity should
therefore be designed in a flexible manner and well documented for

users. This is also a problem for users who do not really know this
technical information, such that when the interaction does not work,
and they have to configure their own device (not easy or consistent
from one firewall/router to another), this may hinder adoption.
Interactivity itself does not need to be high bandwidth. Examples from
the ISI Sharks GIIE show clients requiring an average of 10Kbps for
both send and receive data in a typical 50-player 3d world. This does
not send all data about what someone is seeing, it only sends messages
that contain ‘changes’ (move item X 5m to the left) instead of sending
the whole item again in the new location (maybe thousands of
polygons).
Latency is the biggest issue facing broadband transmission of
interactivity data. If user selections and choices are not received in a
timely fashion, they may not be included where critically needed (may
come after a decision time) or users may be acting on an old state of
information (like the location of a target in a game). The sensitivity to
latency is usually a function of the type of content you are interacting
with. Reaction-based games are the highest demand on accurate and
rapid transmission of interactive data. For this type of content, many
systems include a path-prediction method allowing them to estimate
where things will be in the future even if they have not received
timeline updates. Latency measurements in the range of 100ms are
commonly desired for online gameplay. Decision-making
environments are usually less time-sensitive, where user reaction,
consideration, and feedback require more cognitive processing. In
these contexts, we have found that response (full round-trip) rates of <
500ms are typically sufficient.
Guaranteed concurrency is a final key concern with interactive data.
When the full final representation (e.g. a video) is transmitted you can
be more certain (depending on compression method) that the user will
be seeing something current. When interactivity data is transmitted
there are usually fewer feedback mechanisms to make you aware of
the current state (in the video example above you might see blocks
updating them becoming solid to know it is finished). This issue is
critical to assessment strategies of user learning success, and in
particular professional applications to know all users are ‘on the same
page’ in a discussion/revision.
Immersive transmission
Transmission changes within the immersive environment would
appear to pose one of the biggest potential problems for GIIE adoption
over broadband. Current three-screen immersive environments

(3072x768 pixel resolution) consume a bandwidth of over 1.1Gbps

when uncompressed 16bit YUV encoding is used. The natural next
step in resolution (already approaching use) is up to 1.9Gpbs. Even
where next generation broadband is currently in use or proposed, it is
unlikely that any one content source is likely to be allowed a stream of
this size on demand in the near future. It is even more unlikely that this
will reach broad distribution (i.e. schools ‘maybe’, homes less likely).
This means that compression (in resolution and/or image quality) is
likely necessary. The question is at what cost to the quality of
experience. This report identifies many of the factors involved here
and reports on findings to date on this standard compression approach.
Where real-time environments are involved each image is
programmatically derived from the source content. In this scenario
replication can often be seen as a simpler solution, where the source
content is initially transferred to all sites (in the professional
design/planning projects under study here, a typical content size might
be 1-300MB uncompressed and half of that size using lossless
compression – standard ZIP compression for example).
On a 100Mbps broadband connection, this might only take seconds to
perform such an initial synchronization. After this, only interactivity
data need be updated (i.e. model parametric changes or camera
movements).
The added benefit is that the final representation is full-quality (i.e. no
lossy compression or reduced resolution). The potential problem is that
a suitable real-time capability (CPU and graphics card and more
RAM) is needed than for simple video decoding/playback.

Chapter 3
Perception and
Cognition
Interactive Content Development Guide Perception and Cognition
Issues of perception in GIIE

development
To this point we have investigated the concepts leading to the
development of group interactive immersive environments (GIIEs)
from a conceptual level. We must also understand that in complex
environments such as these, individual concepts are both amplified
and simplified by the perception (sensory) and cognition (mental
aspect) of users. The amount of attention a participant can pay to any
particular aspect of an experience is limited by the number of inputs,
both sensory and cognitively they must process. This also can mean
that certain anomalies can be spotted more readily as there are
multiple ways and vehicles to compare, synthesize and evaluate the
experience – a double-edged sword.
The understanding and appreciation for participant perception and
cognition can have significant impacts upon technology development
and content production, both positively and negatively. This chapter
will seek to discuss a range of perceptual and cognitive issues and
their potential impacts upon a participant’s experience, and the related
influence upon technical and production factors.
Of particular focus are those issues that may impact the support of
both Immersive Cinema and broadband engagement in GIIE content.
We will look at distinct differences between the educational and
professional contexts under presentation in this guide.
This portion of the guide is an area of active but early research and
investigation. The approach and findings range from entirely
preliminary to those gaining clearly measured results. Where clear or
complete knowledge cannot be presented, our approach is to
illuminate a range of issues and factors that we believe to be critical to
ongoing research efforts in the industry.
Perception issues in the education sector

Within the educational sector, participant experiences are usually
evaluated with respect to desired learning outcomes. Often these are
well structured and very detailed with hard measurable knowledge
assessments. Within most GIIE settings, the experiences that are
educational in nature would fall into what is typically classed as
© 2004 Immersion Studios Inc 3-1

Entertainment-Education1. This paradigm focuses on “out of

classroom” learning where school and leisure time contexts are
combined. It also promotes educational goals through impactful
cognitive and emotionally involving entertaining experiences.
Entertainment-Education can be seen to be based on a number of
motivational concepts surrounding enjoyment and information
processing. Ritterfeld and Vorderer2 summarize these concepts as
follows:
1 Entertaining experiences allocate attention that can be used for
information processing. The entertainment experience serves
as a “door opener” to process information.
2 Entertainment experiences can lead to a strong feeling of
presence/immersion (sense of being there). Presence has the
potential of linking input information with a thorough
processing of this input. Distracting influences are neglected in
a state of presence.
3 The educational goal is transmitted implicitly and can therefore
prevent counter arguing and reactant attitudes or behaviors.
4 The learning possibilities are not prescribed, but come as a
surplus of a joyful experience that people voluntarily seek out.
5 People process educational information embedded within the
entertaining experience that they would not voluntarily seek
out.
6 Event related pleasure reinforces learning opportunities. In the
long run this reinforcement enhances people’s motivation to
seek further information on a given subject through
Entertainment-Education experiences.
1 Slater, M. D. (2002). Entertainment Education and the persuasive impact of narratives. In

M. C. Green & J. J. Strange & T. C. Brock (Eds.), Narrative impact. Social and cognitive
foundations. (pp. 157-182). Mahwah, NJ: Lawrence Erlbaum Associates.
2 Ritterfeld, U., Weber, R., Fernandes, S., Vorderer, P. (2003). Think Science!
Entertainment Education in interactive theatres. Computers in Entertainment (CIE),
Volume 2, Issue 1 (March 2004). Association of Computing Machinery. ISSN:1544-
3574

The following list summarizes the requirements for a show to

successfully fulfill the goals of Entertainment-Education.
Requirements for entertainment
High usability
Compelling narrative (involvement)
Aesthetic quality
Requirements for valuable education
Accurate information
Elaboration of a concept
Defining of a problem
Providing means to solve the problem
Show embedded in a broader learning context
Requirements for blending entertainment with educational goal
Trigger intrinsic motivation (fulfilling the goal)
Use game playing instead of learning frame
Explicit achievement goal
Intelligence of the system (feed-back)
Adequate level of challenge (neither too challenging
nor boring)
Social experience
Research to date
Previous research by Ritterfeld and Vorderer3 has directly focused
upon the Immersion Studios’ Cinema-based GIIE. That research has
indicated that enjoyment and presence, media competence and prior
interest, and perceived challenge and usability account for over one-
third of the educational impact of the immersion experience. Research
constraints did not allow for investigation of factors that might account
for the other two-thirds and that might include such things as prior
learning experiences, attention span, memory capacities, prior
knowledge, prior interest in science, among others.
3574.

Student enjoyment has been found to be most predictive criteria of

educational impact. The more a student enjoyed the show, the higher
s/he rates all criteria for the educational impact. Besides enjoyment,
presence (a sense of being there), prior interest in the topic, and
usability play a major role in explaining aspects of educational impact
in GIIE studies done to date. Just as significant are those factors that
have proven to be unimportant like “achievement self concept”,
“sensation seeking”, or even “media competence.” The unimportance
of those factors demonstrates the broad applicability of the concepts of
studied GIIE shows: students’ performance in school, frequency of
computer game playing at home and attraction to complex multimedia
presentations does not significantly affect the appreciation of the
show, nor its educational impact.
Media determines
Usability
Competence
inhibits
Academic
Achievement
Educational
Challenge Presence Entertainment
Impact
Age
Prior interest
in Topic
Most powerful determinants of the Entertainment Education link

Ritterfeld and Vorderer4 summarize their findings through the above
relationship diagram. The red arrows indicate inhibiting processes
(challenges) such as the influences of media competence, academic
achievement, and the students’ age. Media competence has a positive
impact on usability leading to a higher degree of presence. Where
challenge is too high, this has a significant negative impact on
presence.
The successful establishment of presence is the leading indicator of
entertainment. Entertainment in turn is the main predictor for the
educational impact of the show5.
3574.
5 Students’ prior interest in the topic does directly influences the educational impact, but
this has proven to be minimal in comparison to the role of entertainment.

Several factors, while relatively minor compared to the positives,

nonetheless play an important role in understanding participant
perceptual impacts. The primary of these include impacts of limited
time, challenging interface devices, and personal engagement/interest.
A significant portion of students complained about not having enough
time for playing the games within the context of a 25minute show.
This show length has been determined over numerous years to balance
the throughput of audiences and ticket price against the infrastructure
costs of an Immersion Cinema. Research has also demonstrated a
small but significant decrease in the success of the immersion
experience for students using a laptop and mouse as opposed to an
interactive touch screen. Finally, this research has noted that some
students, notably the highly interested, learned regardless of the
technology or design of the show.
Summary of perceptual impact factors

The research to date was based upon testing within relatively carefully
designed technical environments (ideal technical conditions) of
Immersion Cinemas. The transition from an Immersion Cinema to a
broadband-based experience is likely to affect certain environmental
factors that contribute to the educational impact of the show more than
others.
Environmental Comment Effect on user

impact experience
Lighting, sound There is no guaranteed Reduced aesthetics,
quality and base context in a stronger potential
acoustics, seating broadband environment. for distraction.
Choice of A user may have a Some tools,
technology laptop with small techniques, and
keyboard, use a mouse media types may be
vs. a touchscreen, or inaccessible or
have a very slow difficult to use,
computer system. limiting choice or
impacting the
aesthetics.
Individual In most broadband Groups generally
activity vs. group contexts a user is alone offer support and
setting at their personal encouragement,
machine while the while individual can
cinema is always a reduce the feeling
group setting. of pressure.


impact experience
Familiar vs. A visit to a cinema has An existing location
unique/new different starting may be “boring” or
location/space expectations from “comfortable”
logging in at your home while the cinema
computer desk. “new and exciting”
or “foreign and
uncomfortable.”
Some potential impacts that the technical constraints of broadband

networks could have on a GIIE include:
Network Comment Effect on user

constraint experience
Bandwidth Restricted bandwidth Image freezes; slow
primarily affects the loading of server-
quality of video delivered video.
transmitted from the May require lower
server to the user, fps and/or
especially when a resolution.
connection is shared
among a number of users
and bandwidth cannot be
guaranteed to one
application.
Latency System latency may Sound and image
affect synchronization of are out of sync;
sound and video and delay between user
delay system response to input and system
user feedback, response.
particularly when
generating a single
response to multiple
inputs.
Dropped Lost packets may have Image stutters or

Packets to be resent. freezes; sound cuts
out, complete
segment or content
loss if recovery is
not possible.

Further studies related to these topics and focused upon broadband

network constraints were undertaken within the context of the
development of this guide. This research is presented in the full report
in Appendix A IIC Evaluation. They were focused upon the same
Entertainment-Educational constructs as the prior research, but added
the dimension of testing differing levels of network constraints for
their impacts. The findings directly reinforced the prior study while
adding additional content types to the study mix. The core perceptual
finding from this new study can be summarized as: the reduction in
quality tested (from 360Mbps, to 10Mbps, to 1Mbps) does have a
measurable impact on user enjoyment and presence, but the overall
experience is rich enough in social, content, and technical approach
that the impacts do not take a well designed piece of content to a sub-
par level for achieving its educational and entertainment goals.
Future research
To date research has found that the combination of effects in studied
GIIE educational content is a successful educational construct.
Furthermore, research performed in the development of this guide has
shown that broadband adaptation of this type of high-complexity
content need not critically diminish its successful perceptual goals for
most common network reasons. Future research into environmental
impacts is needed however to ascertain whether the social and
environment presence changes of a distributed group setting are
material to the broadest applicability of these concepts. This type of
research is both the most difficult but also the most insightful into the
fullest accessibility to this type of eContent.

Perception and cognition issues in the

professional sector
In the professional sector, the evaluation of experiences are often
based upon the achievement of specific goals, often also judged by
achievement within a minimal timeframe. In virtual environment
research, the notion of presence, much as that described in the
educational context above has also gained some favour as an
evaluation method. The research undertaken to date in professional
GIIE environments has focused upon design and planning contexts
and has not seen formal evaluation methods employed due to a range
of complicating factors:
1 Problems (to be resolved through the use of the system) are
defined differently by each participant.
2 Participants are widely varied in skills, goals, and many are
transitory to the particular professional activity.
Oversimplification of the context and structure of professional GIIE
usage may yield some useful results but the combination of factors has
often been seen to defy simple relationships. Many business meeting
contexts can be seen as good examples of these challenges. Wolf6
outlines the key criteria for a good meeting:
A strategically designed agenda with commonly understood
goals and objectives.
A clear, agreed upon process for reaching those goals and
running the meeting.
An awareness that people come with their personal
preoccupations and feelings, as well as an interest in the
subject at hand.
A sense of involvement and empowerment; people feeling that
the decisions are their decisions; that they are able to do what
needs doing.
A skilled facilitator.
Wolf also outlines a range of reasons why so many meetings are
unsuccessful; too long, unfocused, petty, unproductive, led by
domineering or disagreeable individuals, meandering, time wasting, or
simply boring to name a few. A group interactive immersive
6 Wolf, Kevin. (1997). The Makings of a Good Meeting. http://members.dcn.org/kjwolf/

environment can be seen to have similar characteristics and

challenges.
Attaining goals and objectives are central to the evaluation of the
success of a meeting. The following verbs7 can be used to appreciate
the diverse types of dialogue and negotiation activities used to attain
objectives:
analyze announce appropriate
approve assign brainstorm
categorize clarify decide
delegate determine develop
evaluate explore identify
improve learn plan
prioritize reevaluate refine
request review summarize
Performing these actions can be impacted negatively by a wide range

of issues, both personal and technical. If we focus here upon a
technologically-mediated environment such as a GIIE, it is easy to see
how a lack of appropriate tools can immediately restrict the range of
actions that can be undertaken and thereby significantly hinder the
possible success of a meeting.
Personal input and perceptions play a large role in the sense of
achievement and empowerment. Tools and representations that isolate,
restrict, devalue, or confuse can rapidly diminish the sense of
involvement critical to success. Similarly, the environmental setting
that an experience occurs in can play a strong role in success.
Factors that impact the ability to perform these types of actions have a
negative impact upon the perception of a successful GIIE. These
impacts can be studied under the categories of:
1 Impacts caused by tools.
2 Impacts caused by environment.
3 Impacts caused by representations.
4 Impacts caused by group interactivity.
5 Impacts caused by user ability.
7 Wolf, Kevin. (1997). The Makings of a Good Meeting. http://members.dcn.org/kjwolf/

The issues discussed in this guide, while general in nature where

possible, are clearly biased towards professional application in design
and planning. The reader should take careful note of this in any
thoughtful application to other professional disciplines.
Research to date
Within the realm of perception and cognition, most of the above
impacts have a role to play. As with the prior section on educational
GIIEs, we are focusing upon those aspects of user perception and
cognition in relation to content that may have the strongest roles to
play in the transition between Intranet and Broadband development.
Prior research has identified tools and representational issues, at least
for design and planning8. They have identified an essential base set of
tools for group interactive design inquiry:
Terrain tools (navigation over terrain, objects interpolating
over terrain).
Database access for non-visual attribute representation and
tracking.
Rapid 3D generation and ongoing manipulation of form from
2D data and attributes (parametric modeling using attributes
and dynamic number systems like a spreadsheet).
3D object geometric parameter manipulation (position, scale,
rotation, texture, colour).
Rapid kit of parts library of objects/forms to draw upon.
A similar set of tools can be defined for most disciplines. The
difference in a GIIE is the broad range of specific representations and
tools needed for a diverse team of participants. This interconnected set
of tools is not simply visual and thus points out potential problems for
certain network representational approaches that assume only visual
representations. An example of this type of problem can be seen in the
use of straight video compression (and sometimes scaling) to transmit
an experience from one location to another. These methods can have
significant impacts upon textual representations (documents,
spreadsheets, etc.) making them unreadable, and menus difficult to
use. For these situations, many video conferencing solutions offer a
high-resolution but low frame-rate alternative. Frame-rate is a critical
8 Danahy, J. and Hoinkes, R. (1995). PolyTRIM: Collaborative Setting for Environmental

Design, CAAD Futures ’95.
http://www.clr.utoronto.ca/PAPERS/CAAD95/caadf.jd8.html

aspect of discovery and negotiation in a group setting. If the rate of

response to a question is too low, the process becomes tedious and the
sense of involvement and empowerment is greatly diminished.
Tool and Comment Effect on user

representational experience
impact
Representational Detailed 3D models, Low resolution will
detail required by numerical data, and limit the range of
tools text information content that can be
require a great deal expressed with usable
of resolution without parameters, hindering
loss for clear the usefulness of a broad
representation. representational system.
Response-rate of Frame rate needs to Where response rates
tools match user drop below iteration
exploration and needs, the system loses
cognitive iteration its value for interactivity
rates. and direct group
engagement.
The range of possible settings or ‘environments’ in which these

experiences occur are quite wide, each with significant perceptual and
cogitative implications. A range of typical environments include:
User(s) at a single desktop computer and monitor.
User(s) with a projection display.
User(s) with a VR headset.
User(s) within a partially-immersive (panoramic) environment.
User(s) within a fully-immersive (CAVE or other)
environment.
The question within this realm of environments is how to understand
impacts of environment upon the professional group activities. The
baseline condition that has been under study to date is that of a
partially-immersive (3-screen, 135 degree) environment. This setting
has been found to be a fair balance between cost, representational
capability, and size to facilitate group engagement. While this may be
a desirable environment, it requires a moderate amount of dedicated
space, numerous computers or an expensive single computer. To
broaden the capabilities and distribution of professional content
experiences over Broadband, other settings must be considered as the
dedicated environments are not generally feasible. The most

commonly available environments include the single desktop

computer and monitor and the projection display, common in board
rooms and council chambers. The standard range of impacts from
environment can be summarized in the following table:

and Group impact experience
Individual activity In most broadband Groups generally
vs. group setting contexts a user is offer support and
alone at their encouragement,
personal machine while individual can
while the immersive reduce the feeling of
simulation lab is pressure.
always a group
setting.
Immersive quality Awareness of Decisions may be
context or detail can made upon
positively or incomplete or
negatively impact contextually-lacking
performance. information. Fraser
and Glover9 have
indicated that field-
of-view reduction
does have a negative
impact upon
performance.
Familiar vs. A visit to an ISL has An existing location
unique/new different starting may be “boring” or
location/space expectations from “comfortable” while
logging in at your the ISL “new and
home computer exciting”, “foreign
desk. and uncomfortable”,
or possibly “neutral
and unbiased.”
9 Fraser, M. & Glover, T. (1998). Representation and control in collaboration virtual

environments. In Proceedings of the 1998 United Kingdom Virtual Reality Special
Interest Group Conference, Exeter, UK. Online at
http://www.dcs.ex.ac.uk/ukvrsig98/pap3_03.htm.

During the process of developing this guide, there has been an

opportunity to further the understanding of the impacts and relative
value of these factors. As the content, problems, and contexts of
professional cases, especially in design and planning are so varied, it
has not been possible to establish a quantitative measure of these
factors ourselves. Further study in relation to real-world situations (as
complex as they are) has identified some key findings that can aid in
understanding the key factors upon user perception and cognition in
this type of professional design and planning application.
Evaluations to assess the perceived advantages and disadvantages of a
single-screen PC-based version compared with the existing immersive,
three-screen version were conducted at the University of Toronto and
the University of Waterloo respectively.
The intent of developing a single-screen PC version was to make it
available over a Broadband Internet connection. The evaluations used
a focus group methodology involving diverse participants including
designers, municipal planners, and community groups. The groups
were shown both versions of the environment and asked to comment
on the perceived differences, the advantages, and the relative merits of
each in assisting users to make informed decisions in a planning
exercise that they had or were engaged in as a real-world situation.
Additionally, the groups were asked to assess the importance of group
dynamics in the immersive version and how a distributed format
would affect the decision-making process. Key concepts that have
emerged include:
The immersive, interactive, three-screen version of the
software has the biggest “wow” factor, but the single-screen
version has the benefit of widespread dissemination and
accessibility (fewer time and space constraints).
Participants valued highly the group dynamics promoted by the
three-screen immersive environment, but overall believed that
the same level of decision-making could be achieved with the
single-screen version, although not necessarily in the same
timeframe.
Some participants could not distinguish between the
information provided by the single-screen and three-screen
versions of the software. They believed that that they were
seeing the same information compressed to fit in a single
screen. Further research would be required to explore these
perceptual issues.

A majority of participants welcomed the potential for the

single-screen software runable on standard PCs to democratize
planning by providing an avenue for community exploration
and input into planning proposals.
The software in all versions is a compelling and useful tool
both for expert users (planners, architects, developers, etc.) and
for those who lack the ability to think in abstract three-
dimensional form.
A majority of participants expressed the opinion that the most
powerful aspect of the software in any format was the ability to
change/manipulate/substitute elements of the environment, in
order to better understand the relationship between elements
and their context. The ability to make changes on the spot was
seen as enabling deeper, more critical discussion of
alternatives, since iterations on paper documents or models
were difficult and time-consuming.
Participants, particularly planners, described the requirement
for progressively sophisticated versions of the environments
ranging from simple massing models for early decision-making
or visioning through to versions rich in detail and texture for
public presentation.
There was consensus that being together physically for
discussions, especially when those discussions are
confrontational is the most desirable scenario, but again
accessibility was a major concern, particularly where the need
for public consultation and input is imperative. While not ideal,
a distributed Broadband version could aid many stages of the
professional process.
Many participants expressed the opinion that this type of
software would increase public confidence in the planning
process.
Future research
Current research has demonstrated a mixed view of professional
application of GIIE techniques from a group Intranet to a distributed
Broadband setting. A simple assessment of research to date might say
that the single-screen environment, which would be the most common
for users over Broadband, may be perceptually suitable. However
direct group interactivity that gets watered down in Broadband is
critical to many aspects of negotiation in these professional domains

and therefore argues against a Broadband future for this type of

content.
A more critical evaluation points out another set of perspectives on
group interactivity. Logistical and accessibility constraints, especially
in those processes involving gatherings with large community
participation, promote the support of networked solutions. Of
secondary but often ignored importance is the transitory nature of live
meetings where the investigation and representations are gone as soon
as the session is over. A networked environment can provide a
persistence to these sessions, allowing for greater reflection and study
in an asynchronous fashion. These issues point out that a broadband
approach may not be necessary or desirable for all professional
activities, but several specific cases could certainly benefit from
greater research and development.
In terms of environment, the acceptability of the single-screen version
is based upon untested user responses and is contrary to some findings
in the literature10. It is clear however that an interactive
representational environment, even if only single-screen, is of
significant value and may be completely suitable to a range of
professional activities. Of particular note is the ability of design and
planning professionals (as opposed to a more lay public) to handle the
loss of context associated with only a single-screen view with little
perceived impact. However, the groups noted to likely gain the
greatest benefit from a networked solution (community groups), are
the ones that appear to gain the greatest benefit from the immersive
environment, but are the least likely to have access.
The key concept to take from this study is that one size does not fit all
situations. The Intranet immersive environment is clearly the best for
all users, although not perceived as critical to the most experienced
professionals. Broadband applications are desired and feasible but
would appear to only be suitable for specific purposes. Even within the
realm of experienced design and planning professionals, the factors of
immersion and group interactivity are not normally well understood
and opinions of participants are varied and often contradictory. Only
through the direct use of tools in real-world situations can more
concrete results be observed, and these may not be entirely
generalizable. A simple example of this is that we have observed
significant municipal differences of opinion between large and
10 Fraser, M. & Glover, T. (1998). Representation and control in collaboration virtual

environments. In Proceedings of the 1998 United Kingdom Virtual Reality Special
Interest Group Conference, Exeter, UK. Online at
http://www.dcs.ex.ac.uk/ukvrsig98/pap3_03.htm.

medium sized cities. In many traditional cases Broadband has been

seen to help level the field for small or remote users who do not have
the resources of the large city. Within the study undertaken here, the
reverse has been seen. Broadband community support has been seen as
more critical to the large city we studied where the number and
diversity of users, combined with the quantity of professional cases
(more large-scale development causing impacts) causes greater issues
than in small areas.

Chapter 4
Technology
Interactive Content Development Guide Technology
Introduction
I think that we're also almost always dealing with
people for whom the computer is a bit of a mysterious
player in the decision-making process ... or the working
process. So they either have a sense that it's far too
complex and far too technical for them to really get
anything out of it. Or they have a feeling that somehow
it's incredibly automatic and to design something using
a computer, or to manipulate some big piece of
territory with a computer, is actually this kind of
automatic thing where you just press a button.
Mark Sterling, ICDG Interview

This chapter is not intended for an end user. It is a technical
description of the components of a Group Interactive Immersive
Environment (GIIE). It illustrates those components that are simple to
understand and automatic as well as those that are complex and
difficult to understand and use effectively.
General components are discussed first in detail, then how these are
combined to form the core Cinematic and Real-time system models.
The components of a GIIE can be broken into seven groups of
functions:
1 Coordination Systems.
2 Presentation Systems.
3 Interactivity Systems.
4 Data Management Systems.
5 Facility Management Systems.
6 Media Control Components.
7 Core Components.

The following high-level diagram illustrates the components of a GIIE

and their connection to one another.
Coordination System
PD/AVS PD/GS PD/RS
(Audio-Video (Game (Registry
Sequencer) Server) Server)
ISSimEngine ISMultiSim
Facility
Management Core Components
System
ISNet
IsNetControl
ISSync
ISDevControl
Presentation System Interactivity

System
PD/HDRender (High-Definition)
PMN (Personal
Media Navigator)
PD/MPEGRender
PD/Controller
Data PD/DGRender (Dynamic Graphics)
Management
System
ISCMS (Content
PD/DMS (Dynamic Media Server)
Management
System
PD/RT (Real-time) Media Control
Components
ISTransport
PD/DDAS (Dolby Digital Audio Server) ISLiveVideo
PD/SAS (Spatial Audio Server) ISEncodeVideo
PD/Multi-Flash ISWebControl
High-level diagram illustrates the components of a GIIE and their connection to one
another.
An eighth component, production support tools are described
separately in Chapter 6. These components are specific to the
Immersion Studios’ approach to GIIEs. Others may use a different
approach but we have found this organization useful for development
and ongoing research and production flexibility.

All of these systems form the complete Immersion Studio Inc. (ISI)
Poetic Dimensions (PD) software suite. PD's core technology supports
a broad range of capabilities, which in other commercial and research
systems are only available individually and are not as closely
integrated.
Many commercial systems often lack one or more of the following
important features:
Multi-user triggering of events.
Triggering of events on multiple computers simultaneously.
Triggering of device events (e.g. lighting, doors).
Look-ahead buffering to ensure seamless transitions in audio
and video playback, or real-time animations.
Linking of multiple installation sites and clustering of servers
to provide scalable group interactivity.

Software Systems
Coordination System
Some GIIE environments are completely free-flowing and some are
require time-based control over actions and interactions. Where time-
based coordination of interactivity with immersion is required, Poetic
Dimensions/Audio-Video Sequencer (PD/AVS) is typically employed.
PD/AVS provides the core control over scripted group interactive
environments. A coordination system controls the flow of interactivity,
timing of events including device changes (lighting, video source
switching) and synchronization with immersive presentation systems.
PD/AVS is an entirely network-distributed system built on the core
technology of ISSimEngine. ISSimEngine encompass a high-level
object collaboration system designed for handling complex group
interactivity. ISSimEngine supports group objects that do not require
extremely low latency interaction. To support group objects that
require low latency interaction, the PD/GameServer support
application is available. PD/GameServer provides support for high-
performance multi-player games.
ISSimEngine and PD/GameServer allow applications to operate on
shared objects without any special knowledge of, or technical concern
with network transmission and synchronization over hundreds of
individual users or group decisions. PD/AVS can also handle group
interactivity across multiple installations or large groups (greater than
100 users) using ISMultiSim. ISMultiSim acts as a synthesizer of
inputs and relay server amongst multiple remote servers.
PD/RS is a small support server which acts to connect different
interaction and presentation systems together as may be needed for
broad multi-site network environments where collaborations may not
be pre-defined. An application announces itself to the registry server
and requests a list of possible appropriate remote servers or clients.
PD/RS then supports the brokering of authority and permissions to
allow for a direct connection to be made between these systems.
Presentation Systems
In this guide, presentation systems are categorized as any systems that
render content for the participants, that is, primarily concerned with
representations used for the immersive aspects of the GIIE. While
these applications are specific to Immersion Studios Inc., the

descriptions of each may suggest others that have similar

characteristics.
System Used for

PD/HD(High- Pre-generated video rendering consisting of
Definition)Render a high-definition computer display system
supporting interactive frame-accurate video
jumping, dynamic overlay sprites, multi-
screen synchronization, and uncompressed
video quality.
PD/MPEGRender A compressed MPEG2 version of
PD/HDRender, however it does not support
the range of overlay sprites that
PD/HDRender does.
PD/DGRender Real-time graphics special effects,
dynamically-changing content, with multi-
screen synchronization for highly
immersive environments.
PD/DMS (Dynamic Rich-media server with support for
Media Server) playback of many digital audio and video
formats, PowerPoint, Web pages, bitmaps,
and Flash files. This server is a single-
screen only system at this time.
PD/RT Dynamic design and simulation
applications. This 3D graphics program
supports multi-screen synchronization and
high-performance graphics subsystems
including SGI workstations.
PD/DDAS (Dolby Audio to match with the high-definition
Digital Audio Server) video, Dolby Digital 5.1 audio is provided.
This server can also manage standard
uncompressed stereo audio.
PD/SAS (Spatial Audio 3D positioning of audio samples and their
Server) synchronized surround playback to
multiple speakers in a 3D real-time video
system (such as PD/RT).
PD/Multi-Flash Synchronized multi-screen Flash player
allowing for very dynamic 2D graphics
with extensive scripting support.

Interactivity Systems
Generally, the PD suite is not the final software tool for controlling
interactivity. It is more a framework over which interactive interfaces
can be built. This framework controls core timing mechanisms, group
interactivity mechanisms, scoring systems, and shared object control.
This is accomplished with a set of group object technologies discussed
previously under the coordination system and transmitted through the
core networking components discussed later. To support a range of
richer media capabilities without extensive technology development, a
series of media control components are also available.
The general philosophy has been to permit customized interactive
development purpose-driven to pre-scripted content and user groups.
This used to be accomplished with an in-house tool called Personal
Media Navigator (PMN). PMN required custom training and extensive
development to support the range of interactions desired in GIIE
content. This production process was significantly improved by adding
a simplified framework that sits underneath and supports more
conventional software tools such as Director , Flash or Internet
Explorer programming. By using standard, component-based
approaches to the underlying technology (COM and Xtras) and a well-
defined event process to integrate the remote communications needed
for group interactive experiences the production process improved
significantly.
While Director and Internet Explorer provide mechanisms (Xtras and
ActiveX respectively) to act as containers, they can also communicate
directly with the entire PD suite. If you use Flash however, it must
communicate through Director or Internet Explorer at this time.
A number of projects are also underway that require higher-levels of
interactivity than can be supported through these interpreted
programming tools. In these cases, direct C++ coding is used on top of
the same core framework. This is particularly true where high-
performance, real-time 3D simulations and games are required which
are not well supported in Director or the other interpreted tools. The
availability of the core framework allows for simple use of the
appropriate 3D or other representational engines that might be desired.
The Immersion Simulation Lab (ISL) GIIE presented a different model
of an interactivity communications framework. In this model, users do
not have a pre-scripted set of interactive moves they can make and
thus a controlled interactivity environment would be too restrictive. To
accommodate this, the PD/Controller (PDC) software was created. It
sits on top of the same code framework but does not structure

interactivity in a timed or formal manner. This is the most comparable

system to a traditional software 'tool' that can be used for many
purposes.
Some pre-scripted, interactive content outside of the ISLs is also a
little more flexible than a timed show-based experience. In these cases,
a variety of interactivity tools and options are available but are often
controlled by a host, not the end-user. Any blend of these concepts are
readily supported in the interactivity system as designed but can
significantly effect the complexity, usability, and restriction or
generality of the interactive experience.
Data Management System

The IS Content Management System (ISCMS) provides a production-
time and post-production updating system for dynamic data. The
ISCMS provides formal mechanisms for storing, sorting, updating, and
linking multimedia data into immersive and interactive exhibits. Edits
can be made, tested, go through formal approval workflow processes,
and allow ongoing flexible updating to stay current, in an entirely web-
driven system. Automated deployments, including multi-site support
(deployment and intelligent data merging) are integral components of
the ISTransport component of the Data Management System.
Facility Management System

GIIE technology is used to operate from tens to hundreds of computers
at a single time. The operation and management of these computers
must be as transparent as possible to the end-user and the system
operators.
In the ISI design, all computers run an ISNetControl program, that
controls settings, applications, and system resources for any computer
in the GIIE network. For non-computer devices, ISDevControl
provides a wide range of utilities to control, query, and test electronic
devices. ISDevControl supports devices like audio decoders, mixers,
video matrix switching, scan converters, projector control, lighting
control, fire alarm sensing, lamp hour monitoring, and even low-level
power circuit control.
In the commercial marketplace, there are many device control systems.
Some of the most common include AMX and Crestron.

Media Control Components

The PD suite includes many media control components (MCCs) that
add specific media functionality to interactive applications designed
for a GIIE. A few MCC components currently used include:
ISLiveVideo
This module allows the use of external live video sources such
as video cameras, remote video feeds (through video decoder
boxes, video conferencing units, etc.), etc.
ISEncodeVideo
This module allows the use of video capture devices for such
tasks as a digital speaker’s corner.
ISWebControl
This module wraps Internet Explorer providing domain and
URL filtering (restrict/allow lists), popup control, download
control, and some mime-type control (restrict certain media
types).
All of these MCCs can be supported through direct, low-level OS APIs
but they are often quite complex and include many features not critical
to a GIIE. Some of these features can add interface complexity under
normal conditions or when combined with other components
complicating the user experience. MCCs, on the other hand, simplify
the interface aspects of each module such that they can be used
together in unified and clean/simplified interfaces.

Core Components
The PD software suite uses several core components shared across
most applications. ISSync coordinates interactive show playback and
changes to audio, video, devices, etc. with time-based effects. ISNet is
a foundation network communication system operating upon TCP/IP,
therefore suitable for Intra-, and Internet applications. It handles all
low-level messaging between applications within the PD suite.

Computer Systems
Most GIIEs are networked, multi-computer systems. The setup and
organization of computers for a GIIE require careful consideration due
to practicalities and technical limitations:
Immersive servers use CPU and graphics, sound, and disk
resources at maximum levels. Visual servers in particular
usually run in full-screen mode.
Coordination servers often use network bandwidth extensively
and advanced game servers in particular can generate intense
CPU activity.
Interactive consoles are usually interfaces used directly by
participants and require an individual computer for each
participant. Some designs allow multiple (2 to 3) participants to
share a console or pass one around a group (in a wireless
installation).
Immersive servers
Due to the high-performance demands of immersive servers, most
require a fully dedicated computer for each screen of display. This is
why multi-screen synchronization is so critical when working with
immersive servers. Using high-end, SGI super graphics workstations
that can effectively display across multiple screens on one computer is
one notable exception to this guideline.
Running multiple screens on one computer divides the overall
performance of the system so that a machine, which might be 10 times
faster than another machine, is only a little more than 3 times faster
when it must render 3 screens of graphics. This performance issue
accounts for the emerging popularity of PC computer graphics
clusters.
The common servers run on individual computers include
PD/HDRender, PD/DMS, and PD/RT and PD/SAS.

Coordination servers
Coordination servers are a variable demand resource depending upon
the number of simultaneous users. Resources limitations must be
monitored primarily for networking bandwidth and CPU
maximization.
Depending upon the number of users, the following servers can be run
individually or multiple on a single computer PD/AVS, PD/GS
(PD/GameServer and variants), and PD/RS (a simple registry server).
Interactivity consoles
The greatest variable in number of computers is those that serve as
interactivity consoles. To allow multiple users to simultaneously
interact with each other and their immersive environment, every so
many users need their own computer. In fixed environments, these are
often racked computers linked to ruggedized touchscreen LCD
consoles. In more flexible environments, laptops or tablet PCs are
often used, usually in a wireless configuration.
The common interactive console applications include PMN and
PD/Controller.

Cinematic Group Interactive Immersive

Environments
Immersion Studios has created several variants of Cinematic GIIEs.
The most common is the Immersion Interactive Cinema (IIC). These
environments are typically custom-built cinema rooms with one or
three screen immersive environments with 20 to 80 interactive
consoles.
PD/HDRender
(Master)
PD/HDRender PD/HDRender
(Slave) (Slave)
ISMultiSim
PD/AVS PD/GS
PMN PMN PMN PMN
PD/HDRender
(Master)
PMN PMN PMN PMN
PD/AVS PD/GS
PMN PMN PMN PMN
Diagram of two Immersion Interactive Cinemas (a 3-screen and a 1-screen)

connected together

The immersive environment is operated through the PD/HRender

software running on individual High-Definition Video Server (HDVS)
computers. One or two servers running the PD/AVS, PD/DDAS and
PD/GameServer software systems coordinate overall show flow. The
appropriate numbers of interactive consoles are supplied running the
PMN software. Where multiple sites are connected, the ISMultiSim
software is deployed at one site set as the controlling site.
A variant of the IIC is the Immersion Classroom (IC). The IC is
typically a single-screen immersive environment (16:9 widescreen
aspect ratio) using the standard-definition PD/DMS software for audio
and video. Coordination and Game Servers are the same as those used
in the full IIC. Interactive consoles also run the PMN software but
typically on wireless laptops to allow for flexible space use. Most ICs
are installed in existing venues such as multi-purpose auditoriums or
computer labs.
PD/DMS
PD/AVS PD/GS
PMN PMN PMN PMN
PMN PMN PMN PMN
Diagram of typical Immersion Classroom configuration

Real-time Group Interactive Immersive

Environments
Immersion Studios has developed a combination of technologies
designed for real-time GIIEs. This combination is called the
Immersion Media Lab (IML) or Immersion Simulation Lab (ISL). For
clarity, the term ISL will be used in this guide.
Most ISLs are composed of three-screen immersive display
environment run on three, lower-powered computers (PCs or SGI O2s
typically) or a single, high-end SGI Onyx2/3/4 level workstation with
multiple screen display capability. A single Spatial Audio Server can
also be connected to automatically synchronize surround audio
playback. Any number of interactive consoles running the
PD/Controller software can also be connected.
PD/RT PD/RT
(Master) (Master)
PD/RT PD/RT
(Slave) (Slave)
PD/Controller
PD/RT
PD/Registry Server (Master)
PD/Controller
Diagram of a three-screen ISL connected to a one-screen ISL and a single display

screen for presentation only.

The role of the coordinating PD Registry Server (PD/RT) is simply to

connect disparate ISL sessions together. Once a match has been made,
direct communications are handed off to the ISLs themselves.
For the ISL, the most common deployment model uses a full, real-time
server. Two other real-time server models are currently under research
testing and worth discussing for their differences and future potentials.
Before describing these, a few critical parameters of the default PD/RT
setup should be understood.
Local PD/RT server model

The most common model for ISLs uses a local PD/RT server. In this
model, the computers that run the immersive environment are located
at the environment. Each installation must have sufficiently powerful
computers, even if only high-end PCs. ISLs were originally designed
using this model and it generally follows computing standards to
which IT groups and users are accustomed. However, this does not
always mean it is the best approach.
The local model has the key benefit that all core computation and
media bandwidth is local to the machine it is running on. This means
that when multiple local servers are connected and synchronized, they
do not need to send media data (unless new media data is loaded and
then only once at that time) across the network. They only need to
send the parametric interactivity messaging and each local server
reconstitutes the changes to their local media representations. This
model is very network-friendly and can readily be run across most
broadband connections. The lower bandwidth demands also tend to
result in less latency conflicts. It also runs in a peer-to-peer model,
which allow for scaling of the number of simultaneous sites connected
without a big pipe approach at any one location.
The main drawback of this approach is that every user must have
sufficient computing power at their site. Where very complex data is
being explored, this need either limits the work possible (to what is
computable), limits the interaction usefulness (if too slow), or requires
spending more money at each location (to ensure it is computable).
Centralized Big Iron PD/RT server model

Another emerging approach is streaming where each location needs
access to high-performance immersive rendering. Streaming video on
the Internet has been a standard fare but this approach takes it to a new
level. A high-performance server renders the real-time graphics

allowing highly interactive changes. It then takes the resulting frame

and treats it as though it was a video frame and streams it out over a
networked environment. The client computers then only need to
decompress video (a CPU-only task, no heavy graphics burden)
instead of high-end, real-time 3D graphics.
The challenges for this model involve available network bandwidth
and latencies. This is not a model suitable for broadband since the
videos are usually very high resolution like a computer monitor
(1024x768 or higher, not Internet streaming video standards such as
320x240). Furthermore, unless all portions of the network readily
support multicasting, the bandwidth required multiplies at the server
by the number of simultaneous users. These challenges often result in
lower frame-rates which make real-time difficult to interact with, low
resolution, making detailed data and text difficult to work with.
The CODECs used for this type of work are rapidly improving and
should help improve the quality of experience, much as the newer
MPEG-4 based CODEC for video conferencing provides the same or
better quality in less than ½ the bandwidth.
Grid computing server model

Another approach being proposed in industry and research is that of
grid computing. This will allow for a real-time renderer distributed
over a high-speed network across potentially hundreds of PCs doing a
portion of the same job. The benefits possible via grid computing are
immense. Grid computing may let us compute a whole city with
context and deliver it to a desktop in a more efficient way. Polygon
and texture memory limits can be ignored to a large degree. Grid
computing may provide some tools and processes that are non-real-
time on a single PC to suddenly be available to users for interactive
use.
Grid computing faces many of the same challenges as that of big iron
on the network. Latencies become more challenging as many computer
results must be synthesized, each with different feedback timeframes.
Update of data to each grid computer, and return of results must be
very rapid and synchronized, falling under a 33ms timeframe,
including transfer and latency, if the simulation is to retain even a
basic 30fps resolution.
This is an area of intense research but great challenge and questionable
general utility for real-time interactive work due to the timeframes
demanded. Grid computing will certainly play an important role in
bringing new non-real-time tools to bear upon professional activities

but it is likely that dedicated local or centralized big iron approaches

will be the most common for the foreseeable future.

Chapter 5
Production Team
Composition
Interactive Content Development Guide Production team composition
Production team composition

A production team consists of individuals from different disciplines,
backgrounds and areas of expertise. Together their goal is to create a
Group Interactive Immersive Experience (GIIE) that meets the needs
of the users.
Team composition
In producing a GIIE, the complete production team is organized
around the goal of delivering the goods to the users. What makes
producing a GIIE unique are the different disciplines involved, as
follows:
Software Development Production.
Game Development Production.
Video/Film Production.
Animation Production.
As well, common across all of these disciplines is the role to manage
the production.
The guide focuses on the integration required in producing a GIIE
rather than the intricacies of the production process itself. In general,
the production process is described in other text and documents —film
and video production, animation production, software development,
game development and project management. What is really of interest
here is how the different disciplines interact and integrate with each
other —this being where the guide does focus.
Team organization
A complete production team consists of the following organization:
Management.
Design.
Artwork.
Software.
Audio.
Quality Assurance.
Support.

Management
The management team consists of a diverse group of individuals with
many different responsibilities. During production of a GIIE, the
management team:
Ensures the GIIE meets target-marketing requirements.
Develops a plan to meet these requirements.
Executes the plan on time and on budget.
Design
The design team forms the creative heart of a GIIE. This team
develops, nurtures and cultivates a vision for the user experience. In a
GIIE, the design team also contributes to content research,
development and management.
Artwork
The artwork team wears a host of hats, creating the 2D and 3D
graphics and special effects for a GIIE.
Software
The software team creates the code and realizes the vision of a GIIE.
Audio
In general, the audio team designs, develops and manages three basic
components:
Sound effects.
Music.
Voice-over.
Audio can be developed either by a single audio house or by
specialists in each area.
Quality assurance
Although the industry has no unified, best practice methodology to
assure a GIIE meets an acceptable level of quality, there are de facto
approaches that can be followed (for example, module, system, alpha
and beta testing). The quality assurance team ensures that these
approaches are applied throughout the production lifecycle.

Support
The support team provides and manages a well-defined, stable digital
environment within which the production team can communicate and
share the work.

Management-related roles
Management-related roles within a management team consist of the
following:
Producer.
Project Manager.
Project Coordinator / Production Assistant.
Technical Lead.
Visualization Professional (ISL initiative).
Producer
The Producer manages the production at the highest level from a
business, strategic, creative and technology perspective. From a
business and strategic perspective, the Producer negotiates contracts
and financing and also communicates with external investors and
funding bodies. From a creative perspective, the Producer ensures that
the team maintains a consistent project vision. To that end, they work
closely with the Project Manager and the technical and design teams.
From a technology perspective, the Producer ensures that the
technology decisions are sound.
Project Manager
The Project Manager acts as the “communication hub of the
production team”. The Project Manager is the main point of contact
for feedback on project progress and any other project-related
information. The Project Manager manages the production team
ensuring that the project is completed on scope, on time and on
budget. The Project Manager maintains a clear vision of the project
and keeps the production focused in that direction.
Project Coordinator or Production Assistant

The Project Coordinator or Production Assistant assists the Project
Manager with all aspects of the project, including but not restricted to
the schedule, the budget, the media assets and the content components.
Technical Lead
The Technical Lead manages all technology aspects of production.
They define, design, construct and evaluate technical production

requirements. Central to their role is the design, development and

maintenance of software design, and programming practice and
process standards. They also provide technological consulting to
clients and staff.
Visualization Professional
The Visualization Professional works with real-time, real-world
simulation exercises in areas such as Urban/Regional Design and
Planning, Oil Exploration and Training initiatives. The Visualization
Professional is the content expert to both the client and the production
team, and requires an understanding of the real purpose behind the
modeling/simulation, as well as the specific techniques to properly
construct, render and display models to assist clients in this decision-
making endeavour.

Design-related roles
Design-related roles within a design team consist of the following:
Director or Creative Director.
Writer.
Content Manager.
Director or Creative Director

Based on the strategic requirements, the Director defines the creative
concept of the GIIE. The Director works with the production team to
develop the production schedule, assign deadlines, and allocate staff.
The Director runs brainstorming or creative meetings and locates
scientific consultants to work on concept development, console
content or teacher's guides.
The Director approves designs, treatments and everything produced by
the design and artwork teams. The Director works with the scriptwriter
on script development and also works with the audio team on all audio
for the GIIE, from initial concept meetings to final mix.
The Director works closely with the Art Director to
Maintain a strong sense of visual style.
Generate ideas and solutions.
Develop the main storylines.
Create inspirational designs.
Oversee storyboard development.
Oversee the concept and theme development.
Ensure the integrity of the design process.
Understand the technical constraints and the software tools
needed to build the vision.
The Director assists in developing the outline or flow of the GIIE,
including initial navigation and user interface design. As well, the
Director assists the Art Director in writing the creative guidelines for
the GIIE.

Writer
The Writer writes a compelling narrative with a high degree of
interactivity required of a GIIE. Working with the Director, the writer
defines the story arc, adds dialogue and action ensuring the writing is
consistent with the characterizations, and the interactivity fits within
the story.
Content Manager
The Content Manager defines, acquires and tracks all content media.
Sometimes working with a Content or Media Asset Researcher, the
Content Manager ensures that all content is scientifically and factually
accurate.

Artwork-related roles
Artwork-related roles within an artwork team consist of the following:
Art Director.
Storyboarder.
Concept Artist.
Interface Designer.
2D Designer.
3D Modeler.
Texture Artist.
Animator.
Compositor.
Video Editor.
Art Director
As the conduit between concept and design, the Art Director maintains
the design integrity and visual style within a GIIE. Working with the
rest of the design team, the Art Director is a design advocate and
driving force of innovation and quality craftsmanship. The Art
Director mentors and sets creative agendas for the design team,
pushing boundaries, exploring new technologies and ensuring the
creative concept is realized from a visual perspective.
As well, the Art Director develops appropriate design solutions. The
Art Director is generally skilled in graphic design, illustration,
multimedia design and both interface and interactivity design.
As the manager of the artwork team, the Art Director must ultimately
translate the Director’s vision into visuals.
Storyboarder
Working closely with the Director and Art Director, the Storyboarder
designs and articulates the scenes in a sequence for either or both the
Big Screen and Consoles. Once the storyboard is complete, it is
submitted for internal and external approval. At this stage, the Concept
Artist begins sketching the designs.

Concept Artist
The Concept Artist must accurately convey the design of a media asset
before it is created. Working from the storyboard, the concept artist
creates black-and-white sketches, which are submitted for both
internal and external approval. Once approved, the concept artist
applies colour to these sketches and submits them again for internal
and external approval. Only at this stage does production actually
begin.
Interface Designer
The Interface Designer, generally an expert in graphic design and
interactivity, designs and implements the graphical elements of the
user experience. Of all the members on the production team, the
Interface Designer needs to play the role of user advocate, ensuring
that each team is focusing on the user experience.
The Interface Designer must also maintain consistent user interface
quality across the entire GIIE.
2D Designer
The 2D Designer, generally an expert in elements of typography,
illustration, photography and printing, implements the look of a GIIE
under the direction of the Art Director. The 2D Designer develops the
overall graphic layout and creates graphic elements such as templates,
menus, buttons, typography and color schemes. The 2D Designer also
performs photographic and image manipulation.
As well, the 2D Designer produces high-resolution artwork for
marketing materials such as brochures, advertising, signage, posters,
slide shows and other forms of printed or graphic communications.
The 2D Designer is a “visual problem solver” within the artwork team.
3D Modeler
The 3D Modeler creates models of 3D elements, for either pre-
rendered (high definition with potential for photo-realistic rendering)
or real-time rendered (game play). These models range from
characters to props to environments. The 3D Modeler also creates
rendering effects to depict 3D scenes either photo-realistically or
within a prescribed artistic design.
Sometimes a Character Modeler is required as part of the CGI
component of the artwork team especially when either photo-realistic

humans are part of the GIIE, or in real-time rendering situations. For

real-time rendering on the Consoles low-polygon counts are required.
The 3D Modeler must have a special understanding of how to make
the most economical use of the polygon count.
On real-time, real-world simulation exercises as is performed in
professional GIIE initiatives, a different/additional set of skills and
capabilities for 3D Modeling are required. The three basic data types
that the 3D Modeler requires expertise in within this context are
terrain, build form and trees (and sometimes vegetation). These skills
are often most understandable to those with experience in CAD, GIS,
and landform tools. As with real-time rendering on the Consoles
where low-polygon counts are required, the 3D Modeler needs to work
closely with the Visualization Professional to obtain the optimum
visual quality to meet the goals of the initiative.
Texture Artist
The Texture Artist textures 3D environments, props and characters
within a GIIE. The Texture Artist creates and applies believable
photo-realistic textures to 3D models and environments.
Texturing involves applying images onto 3D models constructed from
polygons. The Texture Artist is virtually a 2D Designer who “skins” a
3D object creating compelling images that are painted onto a 3D
model.
Animator
The Animator creates motion for 3D elements including animating
characters, props, cameras, lights and environments.
Compositor
The Compositor combines all the elements of a scene, layering them
together for final output.
Video Editor
The Video Editor assembles footage and audio to convey the vision of
the Director – piecing shots together into a complete production. The
Video Editor creates Lyca reels. A Lyca reel is a preliminary, timed
movie using scanned storyboard images. As they become available,
storyboard images are replaced by various slug-shots and then by low-
resolution complete shots. Eventually, the Lyca reel becomes a
complete, low-resolution version of the GIIE.

Once the show is composited, compiled and approved, the Video

Editor creates the EDL (Edit Display List) which lists the shots and
locations on the server drives, so that the programming team can
properly integrate the Big Screen visuals with the show's interactive
elements.

Software-related roles
Software-related roles within a software team consist of the following:
Interactivity developer.
3D Programmer.
Tools Programmer.
Interactivity developer
The Interactivity Developer plans, designs, programs and integrates
aspects of the interactive multimedia components of the GIIE.
3D Programmer
The 3D Programmer plans, designs, programs and integrates aspects
of the interactive 3D real-time rendering components for the GIIE.
Tools Programmer
The Tools Programmer plans, designs, programs and integrates aspects
of any software tools required for the GIIE.

Audio-related roles
Audio-related roles within an audio team consist of the following:
Audio engineer.
Composer.
Mixing engineer.
Audio engineer
The Audio Engineer plans, designs and records all audio components
(except music) of the GIIE including dialogue and sound effects. As
well, the Audio Engineer edits all audio components including music.
Composer
The Composer plans, designs, creates and records all the music of the
GIIE.
Mixing engineer
The Mixing Engineer combines all audio components of the GIIE.

Quality Assurance-related roles

Quality assurance-related roles within a quality assurance team consist
of the following:
QA Lead.
Integration Specialist.
Usability Specialist.
QA Lead
The QA Lead defines and manages the testing and acceptance process
of the GIIE. As well, the QA Lead creates the specification for, and
manages the testing laboratory. The QA Lead coordinates the testing
of the GIIE with the other members of the quality assurance team.
The QA Lead proofreads all reported defects and discards duplicate
and erroneous reports, often rejecting reports back to the reporting
tester, requesting clarification and/or testing.
Integration Specialist
The Integration Specialist ensures that the show flow between the Big
Screen and the Consoles is correct during the GIIE.
Usability Specialist
The Usability Specialist ensures that the user experience is sound,
measuring the ease of use of interactive systems based on the five
well-known criteria: user-satisfaction, effectiveness, efficiency,
learnability, memorability. Throughout the lifecycle of the GIIE, the
Usability Specialist performs usability tests including focus groups,
interviews, surveys and paper prototypes. Note that for educational
contexts it is imperative that usability testing with the target market is
performed and completed before a product is released.

Support-related roles
Support-related roles within a support team consist of the following:
Technical Support Specialist.
Network Support Specialist.
Technical Support Specialist

The Technical Support Specialist ensures that all equipment for the
production team works properly and that software is installed
correctly.
Network Support Specialist

The Network Support Specialist ensures that all communication
channels are working optimally, including e-mail, FTP sites, project
management websites, internal network and the rendering farm. As
well, the Network support specialist sets up the naming conventions
for the files and filing system and ensures that it is understood and
adhered to by the production team.

Chapter 6
Production Process
(IIC)
Interactive Content Development Guide Production Process (IIC)
Introduction
The following section focuses on the types of technology used to
produce the Immersion Studios’ Immersion Interactive Cinema (IIC)
product. Where this technology is similar to other industry approaches
and products, we try to present the key differences. The material
focuses specifically on the Immersion platform, however much of the
information can benefit those pursuing other technology solutions.

Key concepts
Before discussing the IIC production process in detail, it is important
to explain some key concepts. These are core components of a
Cinematic GIIE show that relate specifically to organizing the
interactivity of the show. These components are very similar to those
of most programming languages, and those familiar with any type of
programming should be able to follow these interactivity concepts.
Show flow
Cinema shows with a fixed time and story design usually require a
structured representation of the interactive show flow. This is also
required to allow shows to run unassisted (no live host required) but
with some control and coordination.
A structured representation is typically one of the following:
Software program designed to run and control a particular
experience.
This is a custom program with some of the highest flexibility
but typically requiring the greatest technical skill to construct.
A description language designed to layout a show.
Typically, the languages are tuned to minimize complexity to
the types of activities needed for a timed media experience.
The language uses the professional terminology of a content
producer as opposed to generic terminology in a software
programming language.
The new media industry is largely divided in its approach. Many
interactive experiences are developed in programming languages, even
though they may be fairly high level such as Director Lingo and Flash
ActionScript. One of the most common description languages for rich,
synchronized media is SMIL (Structured Multimedia Integration
Language). Even basic HTML now supports enough features for
describing a content flow by linking timed “pages.”
Each of these languages supports the following show flow concepts:
Time-based triggering of events.
User-based triggering of events.
Playback of rich media content.
The IIC uses the Poetic Dimensions/Audio-Visual Sequencer
(PD/AVS) description language for its show flow, as this is typically

easier to teach and learn than a full programming language. All of the
other common approaches listed above lack support for one or more of
the following:
Multi-user triggering of events.
Triggering of events on multiple computers.
Triggering of device events (for example lighting, doors).
Look-ahead buffering to ensure seamless transitions in audio
and video playback.
A more detailed discussion of these critical functions is provided in
Chapter 4 dealing with technical factors.
The PD/AVS description language supports the following key
concepts that are the most critical to understand:
Audio/video segments.
Conditionals.
Group objects.
Device control.
Audio/Video segments
In an interactive show, there is no single linear playback concept. The
show may branch and diverge or shift direction and timing based on
interactive events. The PD/AVS language lets you control by breaking
up aspects of each show into segments and only providing linear
timing within each.
Segments are typically defined by a sequence of time until the next
decision-point that will change the course of the overall show.
To those with programming backgrounds, a show flow can be seen as
a flowchart with each segment being a subroutine for the purpose of
logically organizing events in a show. Each segment ends with either a
link to a single other segment, or with a set of conditionals that allow
show flow to branch based on interactive inputs.
Within each segment, timed events can be inserted to control:
Devices (for example lighting, audio levels, electronic doors,
fog machines).
Special interactive events, if appropriate (for example special
sounds, unlocking features based on users achieving particular
goals).

Synchronized events (for example having all consoles show a

special message).
Changing global objects (for example reset counters, change
scoring).
Typically audio and video are synchronized in IIC shows. Within the
high-definition IIC system, PD/AVS can support independent audio
and video flows and interactivity.
Production Note: As there may be many paths through a show, it is
difficult to simply provide a single total show length.
For interactive shows, we define lengths in two ways:
runtime as a measure of the length of the user
experience, and total show time as a measure of the
combined length of all immersive audio/visual
content. If branching segments are constructed to be
roughly (or precisely) the same length, a general
runtime of a show can be established. Adding up the
linear time of all unique segments provides the total
show time measurement.
Conditionals
As discussed under segments, events and changes in show flow can
occur through the use of conditionals.
A conditional is defined in PD/AVS as an event that occurs only if a
particular condition is met. In its current form, most conditionals are
simple numerical comparisons.
A simple example for an interactive decision on a right or left
direction might be:
if chooseLeft > chooseRight then goto segment “DriveLeft”
if chooseRight > chooseLeft then goto segment “DriveRight”
if chooseLeft = chooseRight then goto segment “DriveStraight”
Production Note: In a single-user application environment, the user
could simply be presented with two arrows, a left and
right, and therefore only those two conditions could
occur. In a multi-user environment, there is no control
that says the totals must be one way or another and
thus the equal conditions must also be considered and
handled.

This type of situation introduces additional media requirements into

the production process. In the example above, a normal single-user
interactive production would have to display for two cases (double the
quantity of a produced linear show) while a multi-user scenario would
have to produce triple the quantity. If the distinction is not critical,
many productions simply default to one of the options in the case of a
tie. Note however that depending on what the users see, this may be
obvious to them and seem incorrect.
A more sophisticated conditional may be a small program or
subroutine that can perform more complex comparisons, evaluations
and decision-making.
Group objects
Interaction requires decisions to change actions. Within the IIC
system, decisions are made through conditionals. Conditionals require
an evaluation of data in the form of numeric variables. To support
multi-user group interactivity, these variables need to be shared or
synthesized across the network. This is accomplished through the use
of group objects in the IIC terminology.
These objects are globally defined, updated and synchronized. It is the
values in these objects that are tested within PD/AVS conditionals.
Each value that is needed globally or to be compared and updated
remotely is established as a group object.
Production Note: Group objects are only controlled in an absolute
manner through a group object server, not through a
client. While a particular client may set a group
object’s value, it could just as easily be set or
modified by other clients. Typically group object
values are managed in one of two ways – value
modifiers or shared value averaging. Value
modification is typically done through an increment
(which can be negative). For a straight vote, each
client might simply increment the selected object by
one (+1). If a dynamic vote were required, allowing a
user to “change” their vote until a time limit is met,
all subsequent votes would decrement (-1) the
previous vote and increment the new voted on object.
Another common group object management
technique is value averaging. This allows individual
clients to set an object value they might desire (for
example selecting how fast you would like the ship to
go.) The group object can automatically average these

values as they are submitted providing the common

value in response to any subsequent requests. Other
methods are available but these are the most
commonly used.
Device control
Within the IIC, additional dimensions of an experience can be
manipulated through device control. This can control electronic
devices such as audio mixers, lighting systems, dry ice generators, or
water sprayers. This can also control networked computer systems,
stop or launch applications or set machine parameters (audio levels,
display brightness).
Device control can follow as a time-based event (turn off lights 30sec
into the show), or as a conditional event (flash the lights if the
submarine is hit and damaged more than 50%).

Unique production components

The diagram below highlights the production aspects unique to the
Immersion Interactive Cinema. These areas will be the primary focus
of our descriptions. Those components that are noted as
“conventional” will be discussed to outline any specific formats or
requirements for their eventual use in an IIC.
Unique Group Interactive Cinema Conventional Production Unique Group Interactive Cinema
Production Modules Modules Production Modules
Immersiv e Immersive
Immers ive Immersive
Screen Screen
Video Audio
Interactiv e Playback
Content Content
Sprites QA
Show Flow
Show and G roup
Flow Interactivity
Creatives QA
Console
Console Console Console
Interactiv ity
Networking Content Interactivity
QA
Production aspects unique to the Immersion Interactive Cinema.
Show Flow
Establishing the core show flow is one of the first key production
activities needed for an IIC show. A show flow establishes the key
segments of the production based on the interactive options desired by
the show’s creative staff.1
Segments in an IIC should first be broken up based on show
interactivity. Wherever you wish to offer choices to the audience is
considered an interactive branching point. For example, allow the
audience to turn left or right, pick an ending, or choose a hypothesis to
explore. At these points, the previous segment should be ended and
two (or more as needed) segments should be added to handle each
optional media segment.
1 A segment name is analogous to a scene name in a film production.

Intro Intro
Challenge Challenge Challenge

Challenges
A B C
Solution Solution Solution Solution Solution Solution

Solutions
A-1 A-2 B-1 B-2 C-1 C-2
Conclusions Conclusion
An example of a show flow diagram.

Production Note: Be wary of the combinatorial factor that can increase
your media production significantly. If your show
branches with three options and each of those then
have two more, you will have six parallel segments to
develop. It is useful from a practical production
model to design shows that do not continually
diverge, but rather re-converge after a range of
options to minimize the future combinatorial impacts.
Once you have organized a show flow based on the immersive media,
review it from the perspective of console interactivity. If there are
discrete console interactivity components within long immersive
media chunks, it is helpful to break the long segments into multiple
smaller ones. The main benefit of this is that highly interactive
sections of a show (for example a console game) will often undergo
modification in terms of time (shorten or lengthen the time available to
play the game) once user testing and feedback is observed.
Finally, if you are putting any special sections into your show, such as
a tutorial, or trailers, etc., it is highly advisable to make these separate
segments in the show flow as they are also often subject to change.
Timed events within a segment are relative to the start of that segment.
This makes it much easier to modify timing within smaller segments
when things change versus large time blocks where all subsequent
events of conditionals would need timing modification.
From this show flow breakdown, the PD/AVS description language
files can be created. These include a .VS file (for immersive video and
console interactivity) and a .AS file (for immersive audio). For
standard definition (DMS) shows, as well as broadband shows, the
.AS file is not used as combined digital audio-video files are used and
are referenced only from the .VS file.

Creating and managing the show flow

The following illustration presents the steps to create and manage the
show flow. For each step there is one or more deliverables required.
STEP 1: Establish Rough STEP 2: Establish Final STEP 3: Establish Final Show STEP 4: Establish Final Show
Show Flow: Show Video: Flow (picture lock): Audio:
Rough Segments Shots Segments Match Show Flow Segments
Shot Name: Shot_Logo

Duration: 150 frames
Shot Name: Shot_01

Duration: 1,630 frames
Segment Name: Intro Shot Name: Shot_02

Duration: approx 3 minutes Duration: 2,000 frames Segment Name: Intro Audio Name: Intro
[5400 frames (@30fps)] Duration: 5,630 frames Duration: 5,630 frames
(@30fps) (@30fps)
Shot Name: Shot_03

Shot Name: Shot_04

Shot Name: Shot_FadeOut

Deliverables Deliverables Deliverables Deliverables

1 Show Flow Diagram. 1 Shot List EDL. 1 Revised Show Flow 1 Audio Specifications: 6
2 Show Segment Name/ Diagram. Channel (if AC3) or 2
Length List. 2 Revised Show Segment Channels of Mono audio
Name/Length List. matching picture cut exactly
on segment boundary.
2 AC3 file: to specifications if
suitable encoder is
available.
Steps to create and manage the show flow.

The following illustration presents the steps to create and manage the
show flow. For each step there is one or more deliverables required.
Step 1 Establish Rough Show Flow: Rough Segments
Deliverables
1 Show Flow Diagram: establishes the number of segments in
the show and where branching occurs, as previously shown.
2 Show Segment Name / Length List: establishes the number of
segments in the show, the length of each segment and what is
shown on both the Consoles and the Big Screen during each
segment. Note that at this point in the project, the timings are
only approximate and as such are shown in seconds and
minutes. The following table provides an example of this list.
Segment Name Length Console Activity Big Screen Activity

S-LogoScreen_1 20 sec Logo screen Logo Screen
IntroScene_1 2 min Logo screen Play video of intro
IntroScene_2 30 sec Keyboard entry of Play video of intro (continues)
Participant NAME
IntroScene_3 30 sec Graphical selection of Play video of intro (continues)
Participant AVATAR
… … … …
OutroScene_1 2 min Logo screen Play video: 1 (you are right) or 2
(look more closely at …) depending
on results of choice
OutroScene_2 30 sec Status screen of Participants Play video of outro
results (avatar, name, points,
stamps, badges)
E-LogoScreen_1 20 sec Logo Screen Logo Screen

Step 2 Establish Final Show Video: Shots

Deliverables
1 Shot List EDL (Edit Display List): presents the segments in the
show as previously listed in STEP 1. Included within each
segment are the specific shots on the Big Screen and the length
of each shot (in frames, @ 30 FPS), as the following example
demonstrates.
Segment Name Shot Name Length Big Screen Activity

S-LogoScreen_1 S-Logo_1_1 400 frames S-LogoScreen_1 / S-Logo_1_1 shot description
IntroScene_1 Intro_1_1 900 frames IntroScene_1 / Intro_1_1 shot description
IntroScene_1 Intro_1_2 1200 frames IntroScene_1 / Intro_1_2 shot description
IntroScene_1 Intro_1_3 1500 frames IntroScene_1 / Intro_1_3 description
… … … …
OutroScene_1 Outro_1_1 900 frames OutroScene_1/ Outro_1_1 shot description
E-LogoScreen_1 E-Logo_1_1 400 frames E-LogoScreen_1 / E-Logo_1_1 shot description
Step 3 Establish Final Show Flow (picture lock): Segments

Deliverables
1 Revised Show Flow Diagram: as the show is developed the
show flow diagram as previously shown is updated as required
revising the number of segments in the show and where the
branching occurs.
2 Revised Show Segment Name / Length List: as the show
develops the show segment name / length list as previously
shown is updated as required revising the number of segments
in the show, the length of each segment and what is shown on
both the Consoles and the Big Screen during each segment.
When sequences of images are rendered and prepared for loading to a
HDVS, an EDL file loading format specification as illustrated below is
needed. A HDVS system uses an optimized Redundant Array of

Independent/Inexpensive Disk (RAID) that commonly consists of 8

high performances SCSI hard drives over two channels with a RAID-5
specification. Basically, it is a very stable and highly redundant disk
storage system configured to playback high definition video at 30 fps
with no compression.
Segment Name Shot Name Network Path Start End Length

Frame Frame
S-LogoScreen_1 S-Logo_1_1 \\NetworkStorage\ShowName\Screen-1\S-Logo\Logo_1_1.%4d.bmp 1 400 400 frames
IntroScene_1 Intro_1_1 \\NetworkStorage\ShowName\Screen-1\Intro\Intro_1_1.%4d.bmp 40 940 900 frames
… … … … … …
OutroScene_1 Outro_1_1 \\NetworkStorage\ShowName\Screen-1\Intro\Outro_1_1.%4d.bmp 1 900 900 frames
E-LogoScreen_1 E-Logo_1_1 \\NetworkStorage\ShowName\Screen-1\E-Logo\E-Logo_1_1.%4d.bmp 1 400 400 frames
EDL File Format Description

Segment Name: Specifies a unique name of a segment within the
show and may be numeric.
Shot Name: Specifies a unique name of a shot within the
show and may be numeric.
Network Path: Specifies the location and filename of the first
frame of the segment sequence. Note, the “%4d”
postfix replace/mask the 4 digit image sequence
number. The path should include the name of the
first file.
Start Frame: Specifies the first frame in the specified segment
to load and must be greater than or equal to 1.
End Frame: Specifies the end or last frame of the segment to
load.
Length: Specifies the number of frames of the segment
(note that the show runs at 30 FPS).

Step 4 Establish Final Show Audio: Match Show Flow

Segments
Deliverables
1 Audio Specifications: 6 channel (if AC3) or 2 Channels of
Mono audio matching picture cut exactly on segment
boundary. See the Immersive Audio Content section later in
this document.
2 AC3 file: to specifications, if suitable encoder is available. See
the Immersive Audio Content section later in this chapter.
Console Networking
Console networking involves defining five shared show
communications types:
1 Branching conditional group objects.
2 Shared multi-user group objects.
3 High-performance game interactivity.
4 Scoring methodology.
5 Time synchronization events.
Branching conditional group objects

The most significant elements of interactivity occur on the console
computers. This interactivity typically drives the interactive branching
of the immersive environment. Before pursuing active development of
the console interactivity, you must define the group objects that will
drive immersive changes. Usually each option that might be pursued
will have a global object created for it and the key branching
conditionals for the show flow specified. This also includes a
definition of the method of group aggregation to be used for the group
objects and an expectation of the numeric value range(s) that are
expected.
Shared multi-user group objects

The second type of group objects that require definition are for shared
communication. Shared communication includes:
Data that other users may need to see or avoid (the information
on clues that have been found or not).

Status of common vehicle elements (the whole audience is in a

submarine and someone turned on a certain sensor or changed
the engine power).
Number of users that have succeeded at a particular task (a
training skill, reached a new “level” in a game).
These objects should be named, documented, and described as to their
purpose including the initial value (set in the PD/AVS script before the
object is to be used), numeric value range, and group aggregation
method.
High-performance game interactivity

There are types of games or simulations that cannot be managed by
group objects alone. This is typically due to complex object
aggregation logic, the need for AI bots, speed of update feedback, or
the quantity of objects per client computer that would be impractical to
enumerate as group objects. In these situations, a dedicated Game
Server may be required to support high-performance game
interactivity. In these cases, one of the existing IIC Game Servers may
be suitable or a new one may need to be created. Game Servers
provide a common pass-through for group objects, meaning only
unique objects and logic must be created. In addition, all core
networking code is supplied through the ISNet libraries that already
handle both server and common client code, further simplifying the
task of programming a new server.
Scoring methodology
If a Game Server is used, it routinely handles and tracks scoring. In
cases where a special server is not used and a shared scoring system is
desired (where high scores need to be displayed), there are two
methods of tracking individual scoring:
1 Built-in server-side group object scoring.
2 Custom client-side group object recording of scores.
The group object server provides some common methods of assigning
and tracking scoring based on the following criteria:
Interacting with a group object.
Spending time interacting with a group object.
A score can be assigned to a group object, and any time a client
interacts with that object (changing its value – not simply reading
values), that client will receive a set score. This can be set to only give

a score the first time it is used or each time. The “first time” counter
can be reset within the show script. Secondly, a score can be obtained
by spending a sufficient time interacting with a group object. This is
used to represent cases where a user should get a score for sufficiently
researching a topic (spending time reading something).
If these automated methods are not sufficient, client-side (but shared)
scoring can be used by creating a discrete score group object for each
client. From that point the client can increment its score as is
appropriate to its local scoring algorithms.
Time synchronization events

The final critical client networking aspect to be designed and
documented prior to interactivity development is time synchronization.
During a show, an interactive console (or DMS, if not a high-
definition show), may change its state in relation to the immersive
media or time in the overall show (for example changing from
showing a logo to playing game one, back to a logo, then playing
game two, and then high scores).
PD/AVS automatically broadcasts to all clients the name of each
segment as it is being started. If no further detailed timing information
is needed, nothing further need be defined. However, if more detailed,
sub-segment information is desired, there are two methods of handling
this communication:
1 Named trigger events.
2 Group object numeric progress events.
If only a few sub-segment-timing events are required, named triggers
can be sent at particular times within segments. If you wish a client to
be able to change the flow of the show, or to support remote network
starting of a show (in a multi-site condition), you must define a single
group object that can have its value changed from the show flow at
key timing events. Whenever group object values change, it is
automatically sent to all clients. You can also use this method for
shows that must run across multiple installations simultaneously as
only group object updates are shared across multiple locations.
Production Note: Currently, all group objects are kept synchronized
across all servers and clients in the IIC. If there are
too many group objects created that are specific to
each client, you may exceed practical performance
limits.

Immersive Screen Sprites (for HDVS)

It is often desirable to represent group interactive states on the public
immersive screen. Showing the dynamic state of group voting
decisions, notes on special events (if a user found a clue for example),
or group distribution in a simulation (who is doing what) are useful to
show to the whole audience.
Presenting dynamic data on top of the immersive video is called
immersive screen sprites. A sprite is defined as a small bitmap image,
most often found in animated games. In the IIC, a sprite can be any
graphical element (text, circle, and line) to be animated (or simply
displayed) on top of a high-definition (HDVS) video stream. This
method does not apply, or work in the same way for standard-
definition (DMS) video streams that cannot overlay sprites due to
standard video playback limitations.
Due to the high-performance implementation of the HDVS video
playback, which utilizes overlay graphics already, sprites cannot use
this technique (only one overlay is typically permitted). This means
that sprites must use a more computationally intensive method of
directly merging graphics pixels with the streaming video each frame
(at 30fps). The challenge with this approach is that frames must be
updated every 33ms for 30fps video. Within this time the data must be
loaded from disk, modified with the sprites, and transferred to the
graphics display memory at the right time (to coincide with vertical
refresh timing). With the current limits on PC architectures 33ms is a
very limited amount of time. Depending on the PC configuration that
has typically been available, this leaves anything from 5-16ms of time
available for sprite processing.
The number of sprites that can be processed in this time depends upon
three key factors:
1 Quantity of sprites.
2 Size of sprites.
3 Complexity of sprites (to sprites include transparency).
Quantity and size can be combined and calculated as total number of
pixels to be processed per frame. Complexity adds an additional check
per pixel, which increases this processing by up to 100% or more
(depending if levels of transparency are supported vs. simple on/off
compositing). In practical sample scenarios, on older (less powerful)
systems, a total of 80,000 pixels/frame appears to be a safe level of
processing permitted. This is the equivalent to 10 rectangles, 200 by
40 pixels each in size.

Only through early sample testing on the target systems (or at least the
lowest performance ones) can sprite complexity be safely validated at
this time.
Updating sprites must be done through a game server, as these are
typically high-performance update items. Some game servers translate
between group objects and sprites in some cases without additional
coding, however, it is just as common to have to create custom code.

Immersive Screen Playback QA

Testing with an IIC production is far more essential than in a
traditional linear environment. Quality assurance of immersive screen
playback is necessary to:
Test the media playback of each possible segment (not all
segments may be seen in any given show!).
Ensure audio and video playback synchronize including:
Ensuring all segment lengths match between audio and
video.
Ensuring lip sync is as expected.
Ensuring audio and video are playing back (and were
produced properly) for 30fps (HDVS IIC only).
Branching and buffering from one segment to another is
seamless (HDVS IIC only).
Test tools are available to playback specific segments on an HDVS or
DMS system to perform these types of testing. If errors occur in
HDVS content, surrogate pure 30fps clips should be made using
standard-definition digital video as a test. Numerical calculations of
data sizes can also be made to ensure files are of the correct
specification. Production or QA should recheck these numbers and
calculations if errors are found.
Production Note: One of the simplest but most common production
problems is that the number of frames in a video
segment changed after “picture-lock” which an audio
production team worked on but nobody mentioned
this (sometimes because it is only one frame in a
segment). Even very small changes that appear minor
(3-5 frames over a 30min show) are sufficient to be
noticeable from an audio lip sync of special effects
timing perspective. Segment lengths must match
exactly and all staff must be aware of how critical this
is.
As the only method of final synchronized and combined playback is
within an IIC (as no standard or cost-effective HDVS systems are on
the market), the numerical accuracy and validation of data is
particularly critical.

Show flow and Group Interactivity QA

When all components of a show are together, these five quality
assurance steps must be followed:
1 Ensure show flows are properly constructed (segments
playback in the expected order).
2 Ensure conditionals are evaluated correctly to branching when
values are as expected.
3 Ensure group object updates are being shared across all clients
as expected.
4 Ensure conditionals and timed events occur as expected.
5 Ensure immersive screen sprites update in a timely fashion
based on user interactivity.
Steps one and two are linked and form the basis for the immersive
branching and playback being correctly coded. If branching is not
occurring as expected, use a group object monitoring tool to validate
the values under conditional testing. Manually double-check these
conditionals using the values you are receiving to ensure they are
properly coded.
If clients are not showing the results of remote group object changes,
check that network communication is generally occurring using a
group object-monitoring tool. If it does not reveal the changing values,
double-check the source script or client code is working (you can also
manually increment group objects using some monitoring tools). If the
value changes in the monitoring tool but not in the client, ensure its
network settings are correct (pointing at the correct PD/AVS server
OR Game Server, if one is used) and turn on its logging to see what
messages it returns.
Step four can be monitored and tested in the same manner as three.
Any faults in step five are often the cause of code faults in your Game
Server or a timing error in your PD/AVS script. If you are trying to do
something new or complex, try to use a known sample set of sprites to
verify basic loading and verify that communication is occurring
correctly.

Conventional production components

This section is primarily focused upon identifying key standards and
formats that should be adhered to within the realm of conventional
digital media production tools.
Console Content
When developing content-oriented interactive experiences (often the
most educational or accurate simulation-based), or any broad audience
experiences, interactivity and content will change often. It is best to
accept this from the outset and plan intelligently for it.
Content used in interactivity, for example, videos that can be looked
up, HTML information in a virtual guide, a picture of a creature, help
instructions or captions can take many media forms. A common
approach to organizing data and managing changes to it becomes
critical if, for example, user testing shows base assumptions to be
incorrect or science advisors inform you that your data is outdated.
To help manage this process, the following three standards are
routinely used in IIC production:
1 Variable data naming.
2 Linking variables to their data.
3 Locating external variable data.
In IIC production, variable data is applied to as much media as can
reasonably be accommodated in a project, not only reference material
(pictures or text) but also buttons, text labels, help files, captions, etc.
Also, all key timing elements in a show that are managed by the local
client (instead of the PD/AVS show flow) should be made as
variables. You may be surprised at what you may need to tweak when
you see hundreds of real users experiencing your show.
Each piece of variable data is given a name or “variable”, and it has a
value that may be a simple number or string, or be an external file.
Appendix D –IIC Object, Variable and Data Referencing Standards for
Console Content describes the method employed in most IIC
productions. If you already employ coding standards, you may wish to
use your own, as these are simply provided as a guide.

Console Interactivity
IIC console interactivity is typically produced using a variety of
production methods including:
Director.
Web Browser.
Flash (within Director or a Web browser).
Direct C++ coding.
Most development work to date has occurred within Director for
several reasons:
Extensive support for plug-ins through its “Xtra” concept.
Extensive media type support (digital video of many formats,
web pages, etc.).
Reasonably powerful coding language (Lingo).
Direct 3D support (Shockwave 3D).
In some cases, Flash has proven to be a better tool for content control
or animation. However, Flash does not have the same Xtra support as
Director. It must be controlled through a container application that
hosts the networking and utility tools of the IIC. Director is the richest
container application that can serve this purpose. Internet Explorer
using ActiveX and JavaScript is also a reasonable capable container
application.
If you want high-performance 3D, Director’s Shockwave 3D can be
very limiting or under-performing. In these cases, development has
gone directly into C++ code and has used IIC support tools through
COM or DLLs. For 3D, many commercial and OpenSource engines
are available.
When designing interactive consoles, a key consideration is how they
handle group objects. Group objects can be copied into a local variable
when changed, thus allowing the rest of the code to behave normally
or, they can be read and modified directly ensuring an accurate current
state at all times.
A new capability of the IIC worth considering is using multi-site
group objects. Multi-site group objects are objects that have a local
group context (for example, current to all users in the cinema, or on
this server) and a universal group context (for example, across all
servers or all connected cinemas). In this case, you must manage and
decide between the two states of each group object, modifying and
displaying the correct one at the correct time.

An example of this might be to see how people in an urban setting

react to an interaction on pollution vs. those who are based in rural
environment (on another server or in another cinema). This can work
across countries to illustrate differences or to have local games interact
with a shared overall game (the success or failure of individual sites
contributes to a whole).
As noted under Console Networking, time synchronization must also
be considered. Typically in an IIC, the mode or state of interaction
(which game, intro, tutorial, etc.) is controlled by the server. This
means that your console should be able to accept messages over the
network that changes your state, and your console does not have that
ability unless it is of a greater free flow concept than most.
Do not design your console to assume that time synchronization events
will occur in one particular manner. As the show flow might be
modified, assumptions on event order can lead to problems in your
console code. Where possible, do not assume what prior events have
occurred in your console. Design your console to handle steps out of
order and you will find it much easier to debug and handle show
modifications that often occur during user testing.
Console Interactivity Testing and QA

In the group interactive IIC environment, a user does not have full
control over their interface and therefore local testing alone cannot
reveal all critical problems. To get around these challenges, a group
object modification tool was created. This tool provides a means of
simulating messages from a PD/AVS server to control timing events
and group object values that are likely to change over the course of a
show.
It is useful to be able to test this networking functionality without the
full server setups that are usually associated with an IIC and this tool
provides all of these necessary functions. It is also capable of
monitoring the group object modifications your console interactivity
code performs to validate it quickly.
In testing it is also very useful to simulate harsh user input. Interfaces
for IIC content are not elaborate tools such as word processors or
spreadsheets, they are akin to games. Users will do the most
unexpected things, clicking almost anywhere at any time to gain a
perceived advantage or select something they think is a button. To
simulate this fairly random activity, a mouse cursor-based random
clicking utility is often useful. When this can be run across a network

of interactive consoles, it can also simulate very high-speed user input

that will aid in stress-testing servers and network load.
Immersive Audio/Video Content

Immersive audio/video content can be produced for one of two
primary immersive server types. Which server type you use can
significantly influence the quality of experience, range of suitably
equipped cinemas, cost, and complexity of production.
1 DMS – Dynamic Media Server.
This is a standard-definition (NTSC) server that uses industry-
standard digital audio-video files (or other rich media) as its
primary display content. It can be modified to perform any
standard dynamic graphics display using Macromedia Director
(including 3D) when not playing a media file. Its primary
benefits are flexible in rich media playback as well as system
cost (almost all IICs include a DMS at this time). Its primary
drawbacks are single-screen display, limited resolution, lack of
seamless audio/video playback, and less precise
synchronization. These limitations are due to its use of
standard operating system media playback systems that are
very limited in clean interactivity support of the type desired
for a cinema.
Another benefit of the DMS is that it can be readily adapted to
broadband internet streaming.
2 HDVS/DDAS – High-Definition Video Server/Dolby Digital
Audio Server (hereafter referred to by HDVS only).
The IIC high-definition server uses proprietary but high-
performance systems and content media design. Its primary
benefits are high-resolution, uncompressed video, surround
sound, seamless audio/video switching and playback, and
precise multi-screen synchronization. It supports direct (but
limited) sprite overlay for dynamic graphics on top of full-
motion video. The primary drawbacks include custom high-end
computer system hardware and special production
considerations. A 3-screen 1024 x 768 and a 1-screen 16:9 are
the most common theater environments.
The content produced for an HDVS presentation follows a
simple file format specification. The standard image resolution
is 1024 x 768 (4:3 aspect ratio) at 24-bit colour, 30 frames per
second and the image sequences are Bitmap (.BMP) file

format. The current system also supports content created in

16:9 aspect ratio and any resolution lower then 1024 x 768.
Immersion Studios has also developed a number of other immersive
servers for specific projects built upon the same networking core as
the DMS and HDVS. These servers demonstrate a broader range of
environments that can be imagined for immersive cinema use. Some
of these servers are:
DFS – Dynamic Flash Server.
This is a variant of the DMS exclusively using Flash playback
but supporting multi-screen-synchronized playback. This
server has not seen wider use because Flash cannot handle high
frame-rates at high-resolution (1024x768) even on fairly high-
end systems.
DGVS – Dynamic Graphics Video Server.
DGVS is a custom production server similar to DFS but with
less functionality than the full Flash system. It does however
use accelerated graphics to provide a seamless and high-quality
experience with very dynamic graphics. It is not designed for
video playback but rather dynamic 2D graphics animation and
effects.
HDVS/MPEG – High-Definition Video Server/MPEG
Format.
It is possible to create an HDVS server that uses more standard
production content such as HD-MPEG2. This requires
specialized hardware decoding for full quality in most cases.
While such a server has been created for MPEG2, many
restrictions apply. The challenge with most of these servers are
similar to those of standard media players, including an
inability to adequately overlay dynamic sprites, and a focus on
linear playback introducing significant artifacts when jumping
from segment to segment due to interactivity.
Production notes for the core two server types are discussed separately
below due to their significantly different processes. The additional
special servers are discussed for interest only and have been used in
limited special-circumstances installations.
Immersive Audio/Video Content (for DMS)

Immersive standard-definition content (for DMS) is produced using
the same standards as for any digital audio/video production. MPEG2
(Moving Picture Expert Group) files with a constant bit-rate (CBR) of

8-10Mbps and stereo audio are the norm. Frame rates of 29.97fps for
content shot with video cameras and CG content at 29.97fps or 30fps
are acceptable.
There are several differences and challenges to production that must
be noted for the DMS as opposed to HDVS including:
DMS does not support seamless audio/video junctions.
This means that each segment jump will be noticeable to the
users. Minimize the number of show segments or use fades in
and out wherever possible so the “jumps” between clips are not
as noticeable.
DMS does not support direct sprite overlays.
Sprites must either not be shown during digital video
sequences or be shown adjacent to the video. Video resolution
must be further reduced and the video must playback at less
than full-screen.
As well, note that there are issues when converting from a 4:3 aspect
ratio to a 16:9 aspect ratio. Please see the section entitled Converting
4:3 to 16:9 aspect ratio later in this chapter.
Production Note: To take advantage of the DMS ability to show more
dynamic graphics (including 3D) through Director, a
customize-coded DMS server must currently be built
using the core DMS as its foundation. A future
version allowing simple cast plug-ins is a likely
improvement to this development process.
Customization of the DMS code is also required for
sprite construction at this time although a future
version will allow external configuration identical to
that of the HDVS system.

Immersive Video Content (for HDVS)

Immersive video content is typically produced using a variety of
production methods including:
35mm film.
High Definition (HD) live video shoots.
Computer Generated (CG) video production.
Composited/animated effects.
When dealing with the high-definition immersive environment
standard in the IIC, imagery currently (higher-resolutions have already
been tested) totals 3072x768 pixels per frame, with a pure progressive
30fps rate. This is divided up into three individual screens of
1024x768 each, in the 135 degree standard IIC. Even an individual
screen is over two times the data load of standard-definition video, and
with three screens is over six times.
The high resolution of the IIC introduces one critical production
implication in that the tools used to develop content must support
resolution independent production (not be limited to standard-
definition). Some of the tools used routinely for this type of production
include Maya, 3D Studio MAX, Digital Fusion, and PhotoShop. Tools
that support work for 35mm film development often support these
types of high variable resolutions (but be careful, as this 3K resolution
is higher than most 35mm film CG work). Be sure to double check
that the tools support a true progressive 30fps, not 24 (film standard)
or 29.97 (NTSC standard) only.
Plan for the disk space needs early! With only one layer, a typical
production of 20-30 minutes total show time requires 250-400GB for
three screens.
35mm film
Generally, the time, cost, and quality issues using 35mm (analog) film
for this type of production does not recommend this content capture
method. HD Video has proven to be a much less costly and cleaner
choice where the option exists.
High Definition (HD) live video shoots

For HD video and CG work, there are specific conditions that should
be carefully noted in IIC production versus standard methods.

Standard HD video has a resolution of 1920x1080 and may have

various frame rates (24,29.97,30) either progressive or interlaced. It is
best to minimize conversion and use a 1080p30 (1920x1080,
progressive, 30fps) capable camera. 24p recording is also readily
convertible to 30p using a pulldown method.
The second issue with HD video is that of resolution. When converting
HD video to single screen IIC, standard methods of cropping or
scaling can readily be employed.
Standard methods of cropping or scaling.

However, when one looks to convert HD video to a three (or more)
screen IIC, it is not high enough resolution to fill all three screens and
therefore one of a number of methods to adapt the content must be
used.
Some of the most common methods include:
Fit the HD video content on only one or two of the screens,
thereby very closely matching resolution (or only imposing a
cropping procedure). Use the third screen for dynamic show
information, sensor displays, CG surroundings (the HD video
can be the view through a window of a plane/ship/vehicle or
building), or other “data”.
Scale the HD video to the full width of the three-screen
environment. This is far less desirable as scaling introduces a
softening of the image and a significant decrease in vertical
resolution.
Composite the HD video into a CG environment, therefore
only using a maximum of one or two screens of material with
HD at any given time. This requires a blue or green screen
shooting process.

Triple Screen
3x4:3 aspect ratio
(3072x768 total)
HDTV against IIC

3-screen
Composite the HD video into a CG environment.

Production Note: One of the challenges with integrating HD video into
an immersive multi-screen environment is the
relationship of the flat HD video content to the
curved, surround, or angled multi-screen displays.
Properly designed content should be shot using
multiple cameras at angles matching the angles and
position of the display screens. This is easy to
account for in a virtual CG world (as discussed
below) but how you manage for HD may not be
intuitive. Surprisingly, you can display flat content
like HD video in the angled immersive environment
(three-screen at least) without it being a noticeable
problem for the users.
Computer Generated (CG) video production

Most rules that apply for CG production for 35mm film apply in the
IIC context. There are numerous guides on the market to discuss these
issues in great detail. These issues include management of complexity,
disk space, and selection of tools for high-resolution work. This also
includes considerations about texture resolution, geometry detail, and
special effects standards. There are however two key differences:
1 Interactive multi-path branching.
2 Multi-screen camera alignment.
Interactive multi-path branching

For multi-path branching, where you wish a seamless transition of
direction that is not a cut (for example an audience decision on going
down a left or right tunnel without stopping first), cameras must be

aligned carefully between the initial path and the alternative branching
paths.
Two approaches used to date include:
1 Method one involves creating a series of splines and aligning
the connection points to create a smooth tangent (in and out
tangents are the same) between the connecting paths.
2 Method two is more suitable for quick multi-path segments
where the action takes place in a specific area (for example
multiple attacks on a monster). Free (not connected to
geometry) cameras are used and animated through time in
Free-Form without paths. Align the Cameras to each other (the
point where they diverge.). With Animate on, the Cameras are
moved with respect to their time and position along the
sequence. To connect a camera to the next connection point, go
to the frame of convergence and Align the camera to the other
camera.
Whichever method is chosen, the following steps are then necessary
for testing the end result and tweaking for proper alignment.
1 In both instance, check for the proper connections by sliding
the time-slider (with everything hidden except for the cameras
and paths). This should give a quick check to see if cameras
are linked to each other and the individual cameras are not
animated by accident.
2 Render each camera at divergent/convergent points to compare
each camera in an image-processing program (such as
PhotoShop).
3 Render the center cameras in shaded mode to see if they are
connecting properly (60 frames before and 60 frames after
merge) to see if the connection and motions are smooth.
Multi-screen camera alignment

Where production is being developed for multi-screen IIC
environments, multiple cameras are often required in the CG rendering
environment. A standard, three-screen IIC uses screens set at a 45-
degree angle to one another. As one server (and one projector) are not
capable of the producing the full resolution in this environment, each
screen is driven by a separate computer. This can be thought of as
separate camera viewpoints in the CG world.
The process of handling this in CG involves the following steps:

1 Create the primary camera, usually the center screen of a

multi-screen environment (the screen either at the direct front
of the cinema or simply chosen as the focus camera in a full
surround or other configuration).
2 For each additional screen, you align another camera in your
CG environment and render another set of frames. For a 135-
degree, three-screen environment you need two additional
cameras linked (same shared camera position) to the primary
one, but with offsets of plus and minus 45 degrees.
3 Each camera renders to a separate set of frames and loads on
separate HDVS playback system.
4 When played back using synchronized servers, it appears as a
seamless composition.
Example of a 3-screen setup.

A more detailed explanation of this process can be found in Appendix
C–IIC 3-Screen Immersive CG Production for those involved in this
type of production.

Converting 4:3 to 16:9 aspect ratio

There are two approaches to convert a digital image from a 4:3 aspect
ratio to a 16:9 aspect ratio. For HDVS display resolution is 1024 x 768
and the source content generally can also be 1024 x 768, or even larger
(HD video 1920 x 1080).
1 Cropping.
The first is to crop the screen so that a 1024 x 768 would become a
1024 x 576 show losing 96 pixels from both the top and bottom. Note
that the video is still ultimately projected at 4:3 with black bars at the
top and the bottom.
4x3 16 x 9
This results in a loss of picture, of course, but does not modify any
data in the picture (no distortion to scaling/stretching).
2 Compression.
The second approach is to compress the image.
4x3 16 x 9
With this technique the entire picture is preserved. That is, every pixel
is still displayed, but is compressed into the smaller area. The result is
that an image appears shorter and fatter, although the resolution is still
1024 x 768.
Immersion uses the compression technique when changing an aspect
ratio in its productions, valuing the ability to see the full image over
the resulting distortion of the image.

Naming Convention for Video (“Image”) Files

There are three parameters for any set of video frames for storage. The
generic naming conventions are outlined below:
1 There is always the screen number in which the video image
file is to be displayed (via Left/Screen-0 (“0-”), Center/Screen-
1 (“1-”), or Right/Screen-2 (“2-”)). Each video folder and file
should always have this prefix.
2 The folder name should correspond to the same name as the
video image, as for instance the segment name corresponding
to the project, scene, or shot. The name should be concise, and
straightforward. Do not name the folder with too much
description of the scene. There should be a “readme” file for
reference.
3 In each set of video frames, there is the frame count (postfix
with an underscore “_” followed by 4-6 digits) at the end of the
file name. The first frame number should always begin with
“1” (unless it was edited otherwise) and padded with zeros “0”
if less than 4-6 digits.
4 Each video image is required to output in the following
specification.
Image File Format Specification

Windows Bitmap Format (BMP extension)
Uncompressed image only (if not in BMP, for example, the TGA file
format is also supported)
1024 x 768 pixels (or smaller, preferably 4:3 ratio to avoid distortion)
24-bits colour depth
For example, a scene is targeted for the center projector, with the
project-scene name as “Scene_01”. This is a scene where a dogfish-
cod gets eaten by a shark. Assume that the scene is rendered to a
network storage device name MEGARAID-FISH\SCREEN-1.
So, the layout of the path of the first rendered image should be as
follows:
\\MEGARAID-FISH\SCREEN-1\1-SCENE_01\1-
SCENE_01_0001.bmp
Production Note: \(E.g. Readme.txt)
SCENE_01: This is a scene where a dogfish-cod gets eaten by a
shark.

Immersive Audio Content (for HDVS)

In developing audio content for the immersive environment, the
biggest design issue is handling audio (especially loops) across
segment boundaries.
Where branching occurs in video segments, audio continuity must be
maintained from the initiating segment into each segment exactly. This
is increasingly difficult if you use looping audio and optional branches
are not exactly the same length (when they join back to a main show
segment, the audio loop can lack synchronization).
A simple way of dealing with this is to ensure that audio/music drops
off at segment junctions where there is branching.
Traditional audio production is largely matched against a video source.
As most audio environments do not have the exact 30fps IIC playback
capabilities, a process for converting between IIC and more widely
available standards must be used.
Steps for converting audio content

The following conversion process has been used successfully:
1 Picture-locked video is typically prepared with 29.97 NDF
(Non-Drop Frame) timecode.
2 Use a standard BEEP in the video, for example a single frame
with the word “2 BEEP” exactly 2 seconds before (and black
filler until) the first real frame of video. A TAIL “BEEP” 1
frame after the last frame of picture for each segment is also
critical.
3 Audio is typically produced at 44.1kHz sample rate, with 6
channels (standard 5.1 configuration). For the 29.97 NDF
production, this actually means an actual audio sample rate of
44,056Hz.
4 Prior to cutting audio into the necessary named segments (each
video segment would have 6 discrete audio files matching 6
channels for Dolby Digital 5.1 Surround Sound), audio is
converted to 30fps rate by resampling to precisely 44.1kHz.
5 Once cut, each potential segment junction should ideally be put
back together in the audio environment and tested when butted
together to ensure no “bumps” occur that might need editing.
6 The final 6 files are combined into a single AC3 (Dolby Digital
5.1) using commercially available encoding tools.

Production Note: AC3 files are exactly the same file size as an
uncompressed stereo audio file. Once you produce an
AC3 file, you can confirm that it is the correct length
to match the 30fps segment using the following
calculation:
#of video frames (30fps) = (AC3 File size in Bytes - 44)/5880
To confirm the audio files prior to AC3 encoding simply divide by
2940 (instead of 5880) as the individual channel files are mono.
(These sizes are based on 44.1kHz sample rates).
For consistency in production, use standard file naming for the 6
channels of audio. One standard file naming convention you might
want to consider is to use filenames composed of the segment name
and the channel abbreviation, for example intro-FL.wav (for intro
segment, front left channel). The channel abbreviations typically used
are FL, C, FR, RL, RR, and SUB (or LFE – Low Frequency Effects).
Production Note: Be very careful with checking these numbers and
following a rigorous procedure. Many listeners can
readily detect faults of only three frames as a lip sync
error.
Naming convention for audio files

Audio is played back in the IIC as Dolby Digital (AC3) 5.1 Surround
Sound. Audio material should be supplied as 6 separate channels, each
sampled at 16 bit, 44.1kHz. We do the 5.1 encoding at Immersion.
For consistency in production, use standard file naming for the 6
channels of audio. One standard file naming convention you might
want to consider is to use filenames composed of the segment name
and the channel abbreviation noted on the postfix, for example intro-
FL.wav (for intro segment, front left channel). The channel
abbreviations typically used are FL, C, FR, RL, RR, and SUB (or LFE
– Low Frequency Effects).
The six (6) files should be mono WAV files (FL, C, FR, RL, RR,
SUB) for each separate piece of the show (intro, livefeed, outro) etc.
So we would receive (on a data CD-ROM) something like:
intro-FL.wav - Front Left (FL) Channel
intro-C.wav - Center (C) Channel
intro-FR.wav - Front Right (FR) Channel
intro-RL.wav - Rear Left (RL) Channel
intro-RR.wav - Rear Right (RR) Channel
intro-SUB.wav - Sub Woofer (SUB) Channel

etc. for each of the key 'chunk'. The wav files MUST be the exact
same length as the video to be captured (or the closes number of
samples just over the video time).
We run video ultimately at a pure 30fps progressive (not 29.97 NTSC)
so no drop frame (NON-DROP FRAME) as well. To
mathematically double-check things, 2940 bytes of MONO 44.1kHz
audio = 1 video frame, so a 1sec segment (30fps video) should be
88,200 bytes in size (+ a wav header which is usually 44 bytes,
meaning a pure filesize per channel of 88,244 IF the audio was for a
1sec video).
filesizeOfAudio = time(in sec)*30*2940 + 44
Always provide a test set of files to ensure proper timing/sizing prior
to full production.
Immersive Video Content (for DMS)

As with HDVS, for a DMS environment there are the same two
approaches to convert a digital image from a 4:3 aspect ratio to a 16:9
aspect ratio: cropping and compression. Note that while display
resolution is 1024 x 768 for DMS environments, source content from
MPEG2 can be as great as 720 x 480.
Naming convention for DMS video files

File naming for each segment is based on the show flow
documentation. In general, for each segment the filename should
follow <SegmentName.mpg>.

Chapter 7
Production Process
(ISL)
Interactive Content Development Guide Production Process (ISL)
Introduction
Real-time Group Interactive Immersive Experiences (GIIEs), such as
the Immersion Simulation Lab (ISL) are primarily used for high-value
professional design, simulation, or training purposes in such
disciplines as (but not limited to):
Design and planning.
Oil exploration.
Automotive design.
Pharmaceutical and chemical processes.
Training and simulation.
Military.
Aircraft.
Ship.
Nuclear plant operations.
These environments are increasing used for less professional and more
educational and entertainment purposes, such as:
Highly dynamic exhibits.
Immersive video games.
This chapter deals with the above real-time GIIEs in a general way,
indicating many of the characteristics shared amongst these
environments. Bias may be shown towards design and planning
applications as these are the most common currently under study.
After the generic detail is provided in this chapter, some specific detail
for design and planning, and exhibits and games is also given.

Key concepts
The Immersion Simulation Lab (ISL) shares many characteristics with
other real-time GIIEs. There are also, a number of differences
particularly where the ISL focuses upon the needs of design and
planning industries. These differences will be discussed in detail after
the shared characteristics are described.
The following concepts are critical to real-time model data linking and
sharing:
Object naming.
Object attributes.
Flat scenes versus scene graphs.
Object naming
Naming components is a key activity in all CG modeling
environments. It can be treated either in a very casual or very strict
manner, depending on the organization or on the number of CG
specialists working on a project. Names of data objects are the primary
method to access the data and control it interactively.
Naming occurs at three levels depending upon how the applications’
data is organized:
Files.
Layers.
Objects.
Object naming using common language rather than technical is
critically important. End users of a GIIE will use these names to select
objects or data to interact with or create links to. Non-intuitive (i.e.
FM-01-LXD-0443) or generic (i.e. BOX01) naming can hamper or
even eliminate interactions altogether. Ideally, naming has been found
to focus on the common name, domain-specific type of data, its
position or scale, and perhaps a version or option identifier (i.e.
HOUSES-SE-2story-optionA, or POSTOFFICE-45m-v1).
Another important use of naming is in controlling complexity. To
simplify models, change options or show detail when focused on one
area, a user can manipulate data by its name organization or through
direct geometric manipulation. Naming provides a mechanism to
group data together to quickly be manipulated.

Object attributes
Some data creation applications can associate specialized attributes to
geometry. This can be used to specify information not typically
inherent in the graphical representation (i.e. material composition,
property value). Many immersive environments can only manage
geometric data and strips out attribute data in conversion.
Attributes can be used for many valuable purposes including:
Identification/searching for particular data that basic naming or
geometry may not reveal.
Automated geometric generation or transformation based on
attributes (i.e. extrude building footprints by the value of their
height attribute).
Direct database linking or relational links to broad databases
not contained within the geometric descriptions themselves.
Flat scenes versus scene graphs

Naming objects in real-time systems can create large lists that are
difficult to manage in a linear fashion. Some software systems enforce
a flat linear model for data naming, either by filename or layers. These
systems alone often result in unwieldy models to use interactively (you
must scroll through all data equally with no prioritization).
Another naming method comes largely from the notion of a scene
graph, named due to the (typically) hierarchical graph nature of all
objects in the system. This method allows smaller objects to be
uniquely identified when needed but within a hierarchy that allows
higher-level manipulations most of the time. An example of this would
be a human model (highest level) with a head, torso, arms, and legs.
Below each leg would be a knee and ankle below which is a foot and
toes. A user can simply control the whole ‘body’ which in turn
changes everything below it, or can expand the body and go right
down to manipulate a toe. In a scene-graph approach with 100
characters, you could operate most of the time with only 100 objects in
your named list, and when you needed, dive deeper into one or two.
With a flat approach you would have to either eliminate the ability for
sub-manipulation (usually undesirable) or have many hundreds of
names to sort through any time you wished to manipulate a single
component.
Even many flat systems allow the separation of subcomponents into
separate files, sometimes called Blocks or XRefs (external references).
This allows for a modicum of scene organization with objects uniquely

identified within any particular file or layer. Furthermore, when

brought into a system supporting scene-graphs, flat systems can often
be re-assembled into a graph form to allow for easier manipulation.

ISL production flow

The diagram below highlights the production aspects unique to the
Immersion Simulation Lab. These areas will be the primary focus of
our descriptions. Those components that are noted as ‘conventional’
are discussed to outline any specific formats or requirements for their
eventual use in an ISL.
Conventional Production Unique Group Interactive Real-time

Modules Production Modules
Immersive Immersive Immersive

Visual Audio Navigation
Content Content QA
Full
Experience
Project Goal QA
Definition
Interactivity Interactivity
Simulation Interactivity
Device Content
Coding Options
Coding Linking
Unique aspects of the Immersion Simulation Lab.
Conventional components
Immersive visual content
Working on real-time projects involves a number of tasks specific to
its visual immersive content:
Immersive model considerations.
Model sources.
Optimizing for real-time.
Model assembly.
Immersive model considerations

Immersive models require a more elaborate construction process than
those typically used in single-screen environments. Several
considerations must be respected:
Greater foreground detail.

Larger context needed.

More volume, less stage set construction.
The modeler must consider that a user typically sees more of a model
in the foreground and therefore often desires or requires a greater
degree of detail. The model is also seen from different or much wider
angles at any one time, requiring a larger context. The users are not
simply looking ahead, they are also seeing to the side, behind, above,
and below depending upon the degree of immersion. As more of the
environment is visible at one time, it is also more critical to fill in
models with real volume and avoid a stage-set mentality that can work
in more confined viewing environments.
Model sources for Real-time

Some immersive environments are complete and custom-designed for
a single professional activity, in which case data is handled
consistently and automatically. Other immersive environments are
more general-purpose such as the ISL GIIE. Preparing data for these
types of environments can often be complex and come from many
different sources including:
Base data sources.
Data digitizing and manual crafting.
Data object libraries.
Automated routines
General-purpose, immersive environments are common in professions
where work is undertaken between different teams and there are few
standards for immersive collaboration.
Most professional disciplines involving work of any contextual
complexity rely upon a range of base data sources. For large-scale
contexts, GIS data is a common geometric input, as is remote sensing
data (aerial photography, satellite imagery) common for imagery, and
forms the base for many automated routines.
Where sufficient base data may not exist, it is often necessary to use
digitizing and manual crafting. As many uses of a GIIE involve the
creation and investigation of future environments (buildings, cars, oil
wells, etc.), crafting is also a necessity. Many tools can create new
geometric models, including 3D Studio MAX, Maya, and AutoCAD.
The most critical aspect with many of these tools is that they are
designed for computational rendering or construction drawing. This
means that they often focus on complexity and quality of rendering or

effects to the exclusion of real-time 3D. Careful attention to model

complexity and naming to allow future culling of non-critical data (i.e.
inside walls when only the outside of a building will be seen) is
imperative with such software.
Many of the 3D modeling packages are now starting to support
optimizations (geometry simplification and texture memory
optimization) for real-time 3D as these tools are adopted for creating
content for video games. Dedicated real-time modeling systems also
exist such as Paradigm-Multigen Creator1, Virtock Vizx3D2, and
Cult3D3 Designer. Dedicated software helps to monitor and optimize
model generation for real-time but often lacks very advanced
construction tools, optimizing for lower-level polygonal tools.
Many parts of a visual model may exist that are common or could be
treated as common. A city model may include lightposts and cars that
are generic, or an ocean simulation might use a range of whales and
dolphins that are the most common even while building styles or the
reefs may be specific. Generic model objects or libraries are fairly
common and are available for purchase online. Often there are
versions made for computational rendering with greatly detailed
geometry and textures, and other versions optimized for real-time
environments. Free models can be found at sites such as the Web3D
Repository4, and commercial sources include Replica Technology5,
Viewpoint Modelbank6 or a wide array listed in 3DLinks.com7.
Finally, 3D real-time models may be generated from other non-3D
sources using automated routines. 2D GIS data with the right
attributes8, aerial photogrammetry, satellite imagery, and multi-
viewpoint images9 are just a few techniques to rapidly generate 3D
geometric data from image and other geometric 2D sources.
1 http://www.multigen.com/products/index.shtml
2 http://www.vizx3d.com/
3 http://www.cult3d.com/
4 http://www.web3d.org/vrml/oblib.htm
5 http://www.replica3d.com/
6 http://modelbank.viewpoint.com/
7 http://www.3dlinks.com/links.cfm?categoryid=9&subcategoryid=90
8 http://www.clr.utoronto.ca/LINKS/GISW/origarticle.html, GIS World, July 1995
entitled: 3D for Free - Toolkit Expands Visual Dimensions in GIS
9 Such as PhotoModeler, http://www.photomodeler.com/

Optimizing for Real-time

Real-time graphics are not automatic or without significant limitations.
Whereas a cinematic computational model of graphics can spend
minutes to hours on a single image, real-time only permits what can be
displayed in approximately 33ms (1/30th of a second). There are two
categories of basic real-time graphics limits, each with two
optimization techniques:
Geometry limits.
Polygonal reduction.
Level-of-detail.
Texture limits.
Texture size.
Texture compression.
Any graphics hardware system has thresholds on the number of
polygons that can effectively be processed into 3D, colours, lit, etc. per
frame. A standard measure is polygons per second (PPS), although
most measurements are often simply based on triangles. Many factors
impact PPS including screen resolution (how big polygons are), colour
depth (how many colours can be referenced), how many lights are
modifying the materials of each polygon. Due to the number of
variables, there are few easy ways to truly compare numbers given in
marketing documents (what we call ‘marketing polygons’). The table
below provides some simple numbers10 to give a rough comparison
from game systems to the top PC graphics and top super workstations:
10 These numbers are derived from a number of sources and are not all directly comparable
(as the methods of determining these numbers varies greatly). They do give a good
indication of the magnitude of change in capabilities over time however.

Game Systems Polygons Per Polygons Per

Second (no Frame (30fps, no
texture) textures)
Sony Playstation2 20 to 75M 600K to 2.5M
(2000)
Microsoft Xbox 60 to 125M 2M to 4M
(2001)
PC Graphics
Hardware
Nvidia GeForce2 25M 800K
(2000)
ATI RADEON 9800 380M 12.6M
Pro (2003)
Super Graphics
Workstation
SGI Onyx4 300M to 4800M 10M to 160M
UltimateVision
(2003)
In general, PPS numbers provided from marketing data are very

different from real-world polygonal numbers, often by an order of
magnitude or more. These differences can also be amplified by good
or poor software and data optimization internal to the software. Only
real-world benchmarks with the intended software and data can reveal
anything approximating good data on which to base decisions upon.
The numbers in the above table do however indicate numbers that
provide a rough relational measure of different technology systems.
It is quite often the case that original models or those acquired from 3rd
parties are too complex in terms of the count of polygons for real-time
use. To make these models effective, the number of polygons used to
craft a form must be reduced.

Using reduced number of polygons.

Polygon reduction is undertaken by removing unnecessary data
(interior walls of a building only seen from the outside), or abstracting
a model geometry (a 6-sided post instead of an 80-sided perfectly
smooth surface).
If models are named and organized well, unnecessary data can quickly
be turned off or deleted. Where models are poorly structured, the user
may spend significant time and effort struggling to simplify the model.
Abstraction is an area of intense computational research. There are
many tools available that provide effective polygon reduction.
Automated reduction works best on highly curved objects and may not
work as effectively on rectilinear objects.
Example of level of detail (LOD) modeling.

Where the quality of a model cannot be reduced and maintain

sufficient detail for its intended purpose, more intelligent approaches
must be employed. The most common of these is level-of-detail
(LOD) modeling. This approach automatically (usually) substitutes
higher or lower quality versions of a model and is dependent upon the
viewer's distance from it. To accomplish this LOD change, either
multiple models (at differing levels of complexity) must be provided
along with information about what distance to switch at, or automated
reduction tools can be used ‘on-the-fly’ (if the rendering software
supports this). Again, this approach usually only works well for curved
surface forms.
At this time in the evolution of real-time graphics, coloured and lit
polygons are often not enough. Highly detailed textures are usually
applied to polygonal surfaces to create a more detailed and realistic
environments.
Example of drawing textures over polygons.

Drawing textures over polygons has two challenges: an increasing load
of activities for the graphics hardware to perform and a much heavier
memory load than most geometry. In the late ‘90s, use of texture
increased graphics load significantly. Today however, graphic load is
fairly minimal as long as all textures routinely used fit within the
texture memory of the graphics hardware itself. If the texture memory
is larger than what is available, the graphics card cannot maintain full
speed rendering because valuable (recall this is all happening in under
33ms) time is lost transferring in and out from regular RAM to the
graphics card RAM.
Most graphics cards for PCs today have between 64MB and 256MB of
video RAM while high-end machines like the SGI have perhaps

several GBs available. This total memory is not usually available for
textures alone. The same video RAM also holds the screen data and
depth data for 3D (and sometimes other data). To provide seamless
animation, most software also uses double (or more) buffers (memory
that is screen-sized but non-visible) to draw off-screen and then just
display the final image. This multiplies the basic screen RAM used (by
close to two, three or more times).
While the number of graphics options used in rendering significantly
impacts these numbers (and hardware optimizations can minimize this
too), a simple set of math might indicate how much of the video RAM
is used just for basic rendering and display before textures are loaded:
Video RAM for Rendering = screen pixels * (pixel colour
depth*2+16)/8
Pixel colour depth is usually 24- or 32-bit (8 bits of Red, Green, and
Blue). This number is multiplied by 2 for double buffering. The extra
16 is a basic assumption of the number of bits per pixel used for depth
(Z) information. To convert from bits to bytes divided by 8.
Therefore, a standard 1024x768 screen in 32-bit colour might
consume:
1024*768*(32*2+16)/8 = 7,864,320 bytes (approx. 7MB) of
video RAM
And this memory consumption increases to 13MB for a 1280x1024-
resolution screen and 19MB for 1600x1200. If you only have a 64MB
graphics card, screen resolution consumes a significant portion of the
available memory.
The amount of memory used for a texture can simply be calculated as:
width*height*(pixel colour depth)/8
Typically, graphics systems are optimized for textures with a width
and height that are powers of two (2,4,8,16,32,64,128…1024 pixels)
per side (width and height can be different, just that each should be a
power of two, e.g. 32x128). If you always use large 1024x1024x32-bit
textures, then each texture would consume 4MB of video RAM.
As with polygons, if you reduce the pixel count or their colour depth,
you can fit your models entirely within the graphics card memory and
render at maximum speed. There are also methods of using
compression textures in video memory (much like JPEG images) on
some systems, but these must be pre-generated and are often used only
for video games.

Model assembly
The above section describes a number of ways of obtaining or creating
models for a real-time immersive environment. The model assembly
often uses data from a number of sources that must be assembled into
a single environment.
Model assembly can be the weak link in the visual data production
phase for the following reasons:
Model conversion problems.
Model coordinate problems.
Image rectification.
Model meshing problems.
Model complexity problems.
While model data sources can often be planned out and well
structured, the model assembly process can make or break an entire
production process. To ensure success, it is important to establish file
format standards that all data contributors must meet before beginning
a project. Next, it is wise to test all formats and software systems early
to catch problems before they disrupt production deadlines.
Converting a model from one file format to another can present
another large black box of problems. Proprietary formats are the most
difficult to manage while more generic ones often result in the loss of
some specific data. Ideally, you want to work with a format that at
least supports:
Hierarchical scene descriptions.
A variety of polygonal forms (strips, quads, etc. not just
triangles).
Texture coordinates.
Normal coordinates.
Material properties.
Animation key-frames.
Camera export for static views and animations paths.
There are few general data translator tools on the market. Most
software packages come with a number of built-in data translators or
sometimes have the option to buy or create plugins to natively

unsupported formats. One common converter on the market worth

examining is Okino Polytrans11.
Most software packages work in a 2D XY, or 3D XYZ coordinate
system. Some impose or structure a model with a given unit size (1
unit = 1 meter, or 1 unit = 1 inch) to match the real world. When
converting models from one system to another, the unit size does not
always translate. For example something that might have been one
inch in one package might become one meter in another. You should
validate this before and after conversion, applying the correct scale to
the model if needed, within the final assembly software package.
Different 3D packages may actually map X, Y and Z differently. In the
computer graphics mathematical worlds, most systems assume that X
is horizontal to your screen, Y is vertical, and you are looking down
the positive Z-axis. With software designed for physical mathematical
worlds, it is often assumed that X is horizontal to the screen, Z is
vertical, and you are looking down the positive Y-axis. If you have an
object rotating incorrectly on its axis (on its side, for example), it may
be due to this rotational difference in axes between different modeling
programs.
Large landform software systems (GIS or satellite imagery) may not
even use a simple Cartesian XYZ coordinate system. Small objects
and spaces are for simplicity sake assumed to reside in a flat
coordinate space. Large environments on the earth’s surface however
are curved and flat approximations only hold for small distances. This
is where coordinate systems such as latitude and longitude (Lat/Long),
and Universal Trans-mercator (UTM), amongst others come into play.
These systems require a more sophisticated mathematical conversion
to or from Cartesian coordinates. If flattened, the user must respect the
limits of the assumption of perceived heights of objects in the distance
(due to curvature of the earth, as much as a floor of a building can
disappear over the curve within several kilometers of distance).
Numerical accuracy presents another problem in data conversion.
Some software packages, in fact most 3D rendering libraries
(OpenGL, DirectX, etc.), only store coordinate data in single-precision
format (32-bit values). Single precision numbers actually give you a
range of 1e-30 to 1e30 with a precision of about 6 decimals. That
means millimeter accuracy with a range of about 1 kilometer, or 1
meter accuracy with a range of 1000 kilometers. If data originally
comes from systems that use large coordinate spaces such as UTM (for
11 http://www.okino.com/

a city) and at the same time other data with high precision on a smaller
scale (a circuit board, for example), you find data gets rounded off and
you lose precision. A common way to overcome this problem is to
move a model that is in large UTM (or similar) coordinates closer to
the (0,0,0) axis center. This technique is called making a local origin
point. Such conversions must be done with software that provides
double-precision (64bit) support otherwise; the conversion can be
corrupted in the translation process itself.
Once most geometric data is organized and placed, often you must
perform a number of image rectification steps. Surface textures
gathered from imprecise or non-orthogonal sources (camera photos,
satellite images or aerial photographs) often need correction to match
the Cartesian coordinate space (or UTM) that the whole model is now
in. This correction is sometimes done in the model creation phase but
may need re-adjustment or initial adjustment after model assembly.
At this point, all key model components should be in the same
coordinate space with matching scales. You should see roads in the
right place relative to buildings or trees in the correct locations. Often
however, the separate pieces of a model do not quite align correctly. A
terrain model sampled at a 30-meter grid often will not match a local
riverbed model crafted at a 1-meter accuracy. Furthermore, many
models are originally built or designed for a flat model and do not
properly sit on newly introduced surfaces under them. A series of
meshing activities is often needed including:
Interpolation – making objects sit in the Z-axis on top of one-
another properly.
Meshing – connecting overlapping surfaces or gaps in meshes.
Alignment – adjusting the position of objects that may have
been estimated from less precise base data now that detailed
model components can be referenced.
Immersive audio content

Immersive audio content is largely produced using game-oriented,
positional audio technologies such as DirectX (DirectSound). The
audio content is connected to the visual model through direct
coordinate placement, usually in a configuration file or through a
named attachment (connects FireEngine.wav to FireEngine.3Ds). A
named attachment is often used where sound should follow the
animation of an object automatically.
Unlike cinematic audio where effects or background soundtracks are
pre-mixed, real-time audio is created as discrete stand-alone sound

files. These files are mixed in software or hardware as the user

navigates through the 3D space, calculating the position and effects
(reverb or blockage by walls) constantly and outputting the results to a
series of speakers positioned to match positions in the virtual model.
Immersive device control

Many real-time immersive environments include specialized devices,
either to facilitate more natural interaction modes (gesture interfaces,
pointer wands or head trackers) or to create authentic interfaces
matching the real-world (airplane cockpits controls or a steering wheel
in a car). Most of these devices provide special input controls but
many must also provide feedback and output that cannot be displayed
on the immersive visual display system.
Immersive navigation Quality Assurance

Once an immersive model has been created, collected and assembled,
it should be validated for clean navigation. This means two things:
Check that all data loads properly – all geometry, textures,
colours, and materials appear correctly.
Validate model assembly – look for any problem joining data
models, textures, or data positions.
To perform these checks, you should carefully navigate around the
model from key eye-level views, normal movement paths and oblique
contextual angles. Any problem areas should be corrected or noted for
explanation to the interacting participants later.
GIIE unique components

Basic interactivity Quality Assurance
The simplest interactivity that many systems support is manipulating
model and camera options:
Test camera navigation control within the immersive
environment (verify movement speeds and comfort with
navigation).
Test display control over all model components:
Ensure naming matches and is understandable.
Ensure parametric controls work as expected.

Production Note: Many parametric controls of model elements work

based on the point of insertion of the model. If the
model geometry was created originally at real-world
coordinates, some controls may not work as expected.
For example, scale multiplies coordinates so a point at
a coordinate such as 50,000 in X and 50,000 in Y,
scaled by 2 will suddenly appear at 100,000 by
100,000, not simply get twice as big in its current
position. If the same model was made originally at 0,0
and inserted at 50,000 by 50,000, many systems will
allow it to scale around the insert point as though it
were 0,0 (which stays 0,0 regardless of what you
multiple its scale by).
Interactivity camera and model options

As the immersive environment is completed, interactivity can be
layered in. Some interactivity requires specific configuration of
options and pre-sets to allow for easy control and navigation in real-
time. Some commonly configurable elements in GIIEs include:
Create and control switching of model alternatives or states
(for simulations).
Create and control fixed viewing positions for cameras.
Create and control key camera or model animations.
Each of these elements typically requires setting a particular view or
model state and then saving the configuration with a particular name
that can be quickly recalled within an interactive session.
Interactivity content linking

Within our definition of a GIIE, data immersion is equally as
important as sensory immersion. In most real-time GIIEs however, this
balance is typically not even considered while content is often
incorporated as an afterthought.
Some GIIEs can and do directly incorporate attribute data but few
support external rich media. The key to external content linking is
object name (or relational attribute name) mapping.
Two levels of content linking are possible:
1 Uni-directional linking.
Accessing rich media from the immersive environment is
possible but results do not link back to the immersive model.

2 Bi-directional linking.
Links and searches from the sensory to data immersion
environments work both ways, allowing data inquiries to
feedback and change the sensory models. This model typically
requires a built-in or external database capability. Other types
of data links are possible and can be of great value for
example, Geographic Information System (GIS) or links to
spreadsheets.
During the production phase of a GIIE, geometric data gets linked to
external applications. Uni-directional linking can be done on a one-to-
one basis, matching any object in the GIIE by name (or attribute ID) to
an application and its startup parameters:
HOUSES-SE-2story-optionA=iexplorer.exe
http:///www.asite.com/search_buildingsDB?keys=housing&height=2
For bi-directional linking, a more formal link must be established
between attributes in the real-time model and attributes in the remote
application. In addition, a formal communications protocol must be
understood and supported, protocols like ODBC, DDE or SOAP.
Full experience Quality Assurance

The critical aspects that must be tested in a real-time GIIE content
experience include:
Technical functioning of all data loading.
Technical functioning of all desired interaction modes.
Maintenance of real-time across all critical elements:
Graphics performance.
Audio response.
Content link responses, especially in a bi-directional model.
Device control interaction meets usability requirements.
Quality assurance in a real-time environment is much more difficult
than for a cinematic, scripted experience. There are no formal
interactivity moves and actions that can fully be tested under all
conditions, as pre-conditions are unknown. The core software moves
are however, more limited and controlled at a software QA level.
Where quality assurance must focus is upon unique programming such
as specialized input devices (wands, gesture interfaces or head
tracking), performance characteristics (real-time optimizations) and

data validity checks (coordinates, loading or manipulation insertion

points).
Design and planning

One of the areas with the greatest current ISL use is that of design and
planning, including the following professional disciplines:
Urban Design.
Architecture.
Landscape Architecture.
Urban Planning.
Regional Planning.
Landscape Planning.
Forestry Management.
Within this disciplinary context, a number of specific GIIE production
issues must be considered.
Source Data – available databases

The following table lists some common database modeling sources.
Data source Capabilities Consequences
GIS Combines spatial or To improve a traditional
(Geographical geographic data with mapping by relating attributes
Information database information for the to spatial features, which
Systems) purpose of storing, enables users to query and
retrieving, managing and analyze data spatially.
analyzing information.
Allows users to select or query
The organization of data a specific geographic location
within a GIS system or feature.
provides for very quick
spatial analysis via a SQL
front-end query capability.
CAD (Computer- One of the most powerful Excellent element for
Aided Drafting) tools to represent accurate production and output in
and detailed visual designs. architectural engineering and
planning projects.
CAD production is
relatively easy to change, to
measure, to manipulate and
to exchange.
Databases When the information is The main applications are:

Data source Capabilities Consequences

easily accessible, then the Material lists and suppliers.
designs, the images, the
Parts lists.
plans, and the ideas have a
better opportunity for Facility management details.
analysis and choice.
Recording of customers.
Contains and can be
organized into any type of
information.
Imagery (Satellite This information provides a Help to understand and
image - Aerial real visualization of “what recognize the environmental
Photography - the context is.” features like colors, material,
Textures) shapes, atmospheric
phenomena, and more.
Terrain Elevation The best virtual For example:
(Grids, Lattices, visualization of physical
Calculate the course water
Contours, context where construction
will take after rainfall on the
Surfaces) of a speculative
land.
environmental scene is
required. Calculation of a path and the
digging of land in the
construction of roads.
GPS (Global Capability to know Useful in collecting onsite
Positioning geographic location on positional information without
Systems) (x,y,z) without requiring time-consuming survey
mathematical calculations. methods. Of particular value in
large or remote environments.
Within each of these data types, there are a myriad of file formats and
standards, some well supported (DXF, DWG, E00, etc.) but many
more that have only partial (only old versions or partial data type
support) capabilities in any particular package. This makes data type
linking one of the most time consuming and constantly-changing
aspects of working digitally on multi-source design and planning
projects.
Model complexity and levels of abstraction

In working with clients there are several factors that impact their
comprehension of what they are seeing and this further impacts their
capability to make decisions. There is an art to finding the right level
of abstraction in modeling visualization to allow you to focus your
own attention, and your potential audience's attention, on the real
issues at hand.

Clients or other members of the public are somewhat uncomfortable

with larger scale modeling and the high levels of abstraction that come
with it. They cannot always grasp the myriad of missing
representational issues and sometimes they feel that they just cannot
deal with it all.
Note that when models are created to provide the best-looking static
image, they are often not capable of performing within a real-time
environment.
In viewing a ‘cityscape’ remember that when you introduce new
models into this environment that have been developed by an architect
for instance, make sure to remove all the interior partitions and other
interior information as it will only slow down the rendering process
and not add anything to how the exterior design of the building fits its
environment. This is especially true if you are viewing the model from
half a kilometer away.
Modeling context (versus an object)

A 3D model should never be constructed without its context; it should
never be constructed separately from the terrain in which it is situated,
or ignore the surrounding environment (buildings, landscape, etc.).
However, adding terrain will add about ten times the time needed to
build a context model. Without the terrain though, the model will be
missing one of the most important decision factors within the
environment.
One of the main barriers to developing professional expertise in the
area of terrain modeling is the lack of tools and clear literature on the
techniques available. Professionals need to continually be training ‘on
the job’ to keep up-to-date and on top of their profession.
Surrounding structures, vegetations, and landmarks (key buildings,
sculptures, etc.) should be provided in any well-constructed immersive
environment. Levels of abstraction can be increased with distance
from the key areas in most cases, however landmarks often require
more detail to aid in navigation and orientation in the virtual
environment.
Focus on assembly
In many instances, software tools for real-time simulations are still just
in their infancy, though there are tools that can be learned in very short
order. However, the greatest challenge at the moment is the data that is
available to work with. With this, though there are a multitude of tools

available there is not the capability to integrate all the required data at
this point. As such we need a broad spectrum of capabilities to support
all that is required, and these people are not readily available since
there are too many capabilities for one person to master.
The data sets can be seen as the raw materials to put into the modeling
structures. However, unless the raw materials are there and in the right
form, an exorbitant amount of time is required hunting and gathering
the data – from cities and GIS and then trying to get the architect to
provide a file that actually works. There are no particular standards on
which to construct data sets and therefore models come in any and all
types of forms.
With so many architectural model or geometric modeling packages
without any standards to follow, there is much work to assemble data
sets. The architectural model and geometric modeling packages range
from Maya at one extreme to AutoCAD and the other base
design/modeling packages at another. Generally DXF is used as the
standard file format to share the data sets. However, exporting from
one package to another creates spurious results, depending upon the
coordinate systems being employed as well as the scale of the model
being constructed. Here as well, surface normal issues can easily arise
depending on how a model is constructed, and some systems or even
the way operators work insert multiple polygons that are not visible or
known at the time (for example, the system makes six copies of the
same surface, three of which point one way, and the other three are
pointing another way).
For a moderate sized urban context (several square blocks), it can take
anywhere from five days to two weeks to get the raw materials in a
form that can be properly integrated. In many instances when data is
received it is incorrect in any or all of the following ways:
Normals are all wrong (backwards).
Model is the wrong scale.
Incomplete data sets.
Another factor impacting the assembly of data sets for modeling is
there is no single package available that is designed to provide this
capability – a hybrid meta-package that has elements of all of the
technologies for the purpose of assembling data sets. One of the most
useful package that we have found today is PolyTRIM12. Without this
12 PolyTRIM is a research tool developed by the Centre for Landscape Research at the
University of Toronto. See http://www.clr.utoronto.ca/POLYTRIM/polytrim.html

type of integrated assembly tool, most users have to use a number of

different packages. From 3D Studio Max to assemble textures or bring
them together, to Archicad to make architectural models, to Land
Developer AutoCAD to create the terrain, where spot elevations and
contours are used to create the terrain.
In bringing all these elements together to seamlessly fit within the
model being constructed you have to understand the language of urban
design, architectural design, vegetated form and computer graphics.
For instance, you have to have the terrain cut, not just to create a
surface, but also to put the elements on the surface (i.e. curves for
roads, etc.) and then create an architectural massing model. You need
to know how to build vegetated form, or at least reference a library of
forms and locate them in a model on the terrain. As well as knowing
how to associate texture information with the geometry.
Construction – base model

The following table lists the activities for assembly.
Assembly activity Requirement Challenges
Import file, data and To create a primitive A lot of information is created
existing 3D Model space with original data. from many sources (software).
To properly integrate them into
a single environment.
Scanning and To create an existing To make an efficient model that
digitizing contour landform model on which represents the key
lines for a terrain to place all geometry characteristics of the landform
model accurately. without excessive 3d geometry.
Register in the same To align all models in a To provide a consistent data
coordinate system - consistent coordinate model across the same
Geo registration space and data precision. geographic coordinate system.
Editing, cleaning, To prepare data and to During the transfer corrections
aligning data (2D ensure correct data for any unforeseen errors so as
footprint) and model matching. to be visualized correctly.
Link data to a single To create a logical Not all environments support
entity through an ID connection between GIS data attribute linking in a
number world and 3D space. consistent manner.
Build a massing To automatically create Correct source data attributes
model through data an interactive model via that can be used to automate 3d
(extrusions, tin the attributes linked to a generation.
terrain model) model and thereby save
time and money since it is
not necessary to have a
fixed single instance of a
model.

Assembly activity Requirement Challenges

Drape function for To implement a landscape Most assembly programs treat
aerial photography with a recognizable level imagery as generic ‘textures’
of critical surface detail losing and geographical
(sidewalks, roads, paved alignment data.
surfaces, lawn, etc.).
Building façade via To provide a Accurate calibration and
texture-mapping representation that is rectification of each façade is a
indicative of the material, time-consuming activity.
detail, and usage of
buildings.
Create random To rapidly create volumes Few tools exists which can
objects (for massing of objects, typically export this type of data or
environments) vegetation (forests, etc.). assemble it with all other
required data types.

The following illustrations present the steps to design and develop

real-time environments for many design and planning projects.
Developing a terrain model.

Developing a 2D model.

Developing a 3D model.

The draping process described.
Textures, Transparency, Shading, and Lighting

Although computationally expensive there are a few very important
capabilities that will greatly enhance the real-time visualization
experience for the participant. These include:
Realistic texture mapping.
Adding transparency to models.
Shading.
Lighting.
Textures have become a common element of design and planning
model-building, from aerial or satellite imagery to building facades, to
vegetation billboards.
Sometimes textures are also the only computationally-efficient way to
represent certain geometry that is very complex (intricate curved
shapes, complex material patterns, trees, etc.) In most systems,
textures are assumed to be rectangular forms, even if they are used to
represent non-rectangular forms such as vegetation. In these cases,

transparency is used to ‘mask’ out (eliminate the drawing of) the non-
rectangular portions of the shape. Modelers must be aware however, of
the different (and sometimes incompatible) methods of specifying
transparencies in a texture. Some systems (many 3d modelers like 3d
Studio MAX) require one texture for the colour data and another
texture to specify the transparency (usually a grey-scale image) where
black is entirely see-through, and white is solid (no transparency).
Other systems use a single, usually 32bit (8 bits of red, green, blue,
and alpha) texture to achieve the same effect. This can require an extra
data conversion step for textures.
Shadows are another critical representational capability for most
architectural 3d systems. Shadows assist in defining edges, light access
and patterns. Most systems now support volume shadows which
accurately project geometry volumes through space. This
unfortunately does not work for real-time systems where geometry is
only defined in a texture (a tree or roof gable), where the attached
rectangular polygon would be used for shadow calculation. In these
cases alternative methods such as projective shadows must be
employed. Most current systems do not support such variable hybrid
methods within a single real-time environment and therefore shadows
remain a challenging and limited capability for highly integrated
design and planning work.
Most systems supporting architectural work also have some levels of
lighting support. Most of these systems operate in a non-real-time
rendering mode as lighting calculations, bounces, and reflections of
the complexity in most buildings is beyond real-time graphics
capabilities (which are often limited to 8 or fewer dynamic lights).
Non real-time systems (such as many radiosity systems) can often be
used to render light maps into or overtop of surface textures. These
resultant textures can be viewed but typically not changed in real-time
environments. Changes in pixel shader technologies of leading-edge
graphics cards are starting to make some levels of dynamic complex
lighting possible.
Interactive moves
Prior research has identified tools and representational issues, at least
for design and planning13. They have identified an essential base set of
tools for group interactive design inquiry:
13 Danahy, J. and Hoinkes, R. 1995. PolyTRIM: Collaborative Setting for Environmental

Design, CAAD Futures ’95.
http://www.clr.utoronto.ca/PAPERS/CAAD95/caadf.jd8.html

Terrain tools (navigation over terrain, objects interpolating

over terrain).
Database access for non-visual attribute representation and
tracking.
Rapid 3D generation and ongoing manipulation of form from
2D data and attributes (parametric modeling using attributes
and dynamic number systems like a spreadsheet).
3D object geometric parameter manipulation (position, scale,
rotation, texture, colour).
Rapid kit of parts library of objects/forms to draw upon.
This interconnected set of tools is not simply visual and thus points out
potential problems for certain network representational approaches
that assume only visual representations. An example of this type of
problem can be seen in the use of straight video compression (and
sometimes scaling) to transmit an experience from one location to
another. These methods can have significant impacts upon textual
representations (documents, spreadsheets, etc.) making them
unreadable, and menus difficult to use. For these situations, many
video conferencing solutions offer a high-resolution but low frame-rate
alternative. Frame-rate is a critical aspect of discovery and negotiation
in a group setting. If the rate of response to a question is too low, the
process becomes tedious and the sense of involvement and
empowerment is greatly diminished.

Design and Planning Reference Links

The following links outline useful software to aid in producing real-
time content as well as some links showing concepts and examples of
content assembly that may be inspirational:
www.viewtec.ch
www.manifold.net
www.urbansimulation.com
www.okino.com
www.earthviewer.com
www.itspatial.com
www.multigen-paradigm.com
www.gregdowning.com
www.photomodeler.com
www.realviz.com
www.sanborn.com
www.terrasim.com
www.u-data.com (www.metroblocks.com)
www.cybercity.tv

Exhibits and Games

Typically geometry in the game does not change (unlike design and
planning where exploring change options is the priority) and thus
many optimizations on geometry for an environment can be invoked.
This has resulted in a number of common data creation methods that
should be understood.
Terrain – height maps

Unless a game occurs completely indoors, terrain modeling will be
required. Creating ‘fantasy worlds’ (vs. the real world environments
required for design/planning) is made fundamentally easier by using
height maps or height grids as the data source for the terrain.
A height map is a collection of height values set at fixed intervals in a
horizontal plane. Typically the height map is read from a file such as a
grayscale bitmap. The shades of gray represent the height of mesh
vertices (triangle corners). When creating a height map consideration
must be taken for the eventual size and proportion of the final terrain
area. For instance, how many columns and rows of triangles are
required in the map? Including more rows and columns in the map will
make the terrain less blocky but increase the number of triangles
within the terrain. 14
Interiors – BSP and portals

Within a real-time rendering environment the speed at which drawing
objects occurs can be a major constraint to the playability of a game.
BSPs are employed to assist in accelerating drawing geometric
surfaces and to assist in deciding which objects are visible.
One of the simplest solution for rendering objects employs the
“Painter’s Algorithm.” Here all visible objects are sorted into back-to-
front order, and then they are rendered in that order. However, it is
relatively too easy to find special cases that cannot be sorted, and
therefore rendered correctly.
A more sophisticated method to achieve this objective, though still
having certain constraints is by constructing a Binary Space Partition
(BSP) tree. A BSP tree is a data structure that divides the world space
into two components with a plane (called a BSP node), and then
14 For more information on height maps and terrain modeling see the following:
• http://www.qeradiant.com/manual/Terrain_Manual/pages/height_maps.html
• http://home.planet.nl/~monstrous/terrain.html

repeats that operation recursively on the two components, until it is

divided to a predetermined satisfaction level. There is also the
assumption that in most cases geometric surfaces are almost
universally flat, and as such one can make such surfaces part of BSP
nodes. Each BSP node divides the space into near and far components
as was the case with the Painter’s Algorithm and then sorting the BSP
trees from back-to-front. To effectively handle cases where set of
surfaces appear to be ostensibly incapable of being sorted, these
surfaces are split between partitions.
Although BSP trees are a widely-used (e.g. Doom and Quake), as
mentioned there are some shortcomings with using this mechanism.
The first drawback is that overlapping objects (i.e. any polygons that
intersect) yield spurious results (since near-far sorting cannot be easily
determined). To manage these situations multiple BSP-tree sub-maps
have been employed. Here, some of the surfaces are set to being
portals to other sub-maps. However, these are not the only
shortcomings with BSP trees. Other issues include being very time-
consuming to build and being inflexible once built.
Another approach that requires description is portal geometry. With
this technique the accessible area is divided up into rooms (polygons),
where room is a convex polyhedron. This makes depth sorting a trivial
task, since there is only one visible surface in any one direction, and a
room's objects are always in front of it. The one drawback with this
technique is that in general, most interesting environments end up with
connected rooms and therefore the environment contains a lot of
concavity. 15
Character animation – bones vs physique/biped

Modeling and animating characters is a mainstay of gameplay.
However, creating character animation for real-time environments is
still very much of a challenge as is shown is the table below.
Animation Type Bones Physique/Biped
Skeletal System Easy to export. Hard to export keyframe data
correctly.
Easy to import into real-
time engines. Best solution within the
animation application, because
Interpolation curves are
of ‘hidden gems’ like auto-
limited (standard set).
correcting posture. But this
Simple and results are information does not export
15 For more information on BSP Trees and Portals see the following:
http://www.webcom.com/~petrich/games/duality/dualism_articles/portbsp.html
ftp://ftp.sgi.com/other/bspfaq/faq/bspfaq.html

Animation Type Bones Physique/Biped

predictable and accurate. well.
To remain accurate with
original then must resort to a
frame-by-frame export it.
Naïve exports yield:
Large file sizes.
Animation anomalies
(broken hierarchies seen as
skating or foot in mouth).
Meshes Not integrated with Mesh deformations (e.g. bulge,
respect to character smooth, collapse, weld) are
animation: built into the application and
cannot readily be exported due
N-link blend.
to the procedural nature of the
Multi-texture. surface characteristics.
Workarounds include
Frame-by-frame export.
Don’t use deformation
techniques (i.e. bulges,
etc.).
System Bones are unconstrained Somewhat constrained.
and in turn exporters are
There are no 100% WYSIWYG
able to capture all data to
exporters.
a high degree.
The Physique/Biped technique for character animation is superb when

the output is meant for frame-by-frame rendering. However, in the
world of real-time rendering, custom-designed exporters must be
employed to maintain all the character animation information both
intact and as desired.

Chapter 8
Conclusions
Interactive Content Development Guide Conclusions
Conclusions
As outlined in the introduction, this guide has three main purposes:
1 Expand awareness, interest and capabilities for production of
group interactive and immersive environments.
2 Elaborate the issues and challenges surrounding these
environments.
3 Describe key areas for future research and development.
Each chapter of this guide has played a role in presenting and
elaborating the concepts that have shown themselves to be the most
important towards these three goals.
We started this guide with a discussion of what makes a group
interactive immersive experience valuable and a key indicator of the
future evolution of new media in professional, educational, and
entertainment experiences. We spent time examining the key findings
from research into the key user perceptions and cognition when
engaged in these environments. This examination revealed the most
important challenges and roles for a Group Interactive Immersive
Environment (GIIE) as a new media form.
The second part of this guide delved deeply into the technical structure
of a GIIE and the production steps necessary to fully exploit them. The
guide has reviewed the technology and technological concepts that
have been created to allow for this holistic integration of components.
Finally, the guide has reviewed in some detail, the process, knowledge,
and skills necessary to produce content of this form. Within each of
these steps the guide has examined in more detail the changes
necessary for adaptation to broadband distribution that can open the
markets and user accessibility to this rich new media approach.
With the explorations and knowledge presented through this guide, we
have arrived at a number of conclusions about GIIEs. In particular, we
can look at the opportunities surrounding their release from dedicated
venues to broader use over the Internet.
© 2004, Immersion Studios Inc. 8-1

Findings
While there has been a strong focus upon the full sensory quality of a
GIIE experience, it has become evident that the overall experience is
far greater than the sum of its parts:
Immersive visuals.
Immersive audio.
Rich media content.
Personal interactivity.
Group interactivity.
Social interplay of participants (non computer-mediated).
In most media environments only one or two of these are engaged at
any one time, while in a GIIE, a wide range are available and exposed
over the course of an experience. Many viewers of this new media
have expressed that it is “too much” for a person to take in. In fact,
early user testing had revealed similar concerns. What has been found
over the years however is that the concept itself is fine but the specific
instances of experiences can try to do too much, too quickly and in too
complex a fashion. While there are huge challenges in designing
“good” usable systems for a single user using a small number of types
of media, the challenge is only amplified in a GIIE. Careful attention
to design and most importantly, usability testing has proven to be key,
even for experienced designers.
Of a second note on the concept of ‘too much’, is the changing nature
of media engagement and access in the broader world around us. This
may not be perceived by all, and we have certainly found that age
plays a role as the nature of media and interaction has changed
radically over the past 15 years. It is not uncommon today to see:
Rich media engagement reaching new levels through the
expansion of the Internet. Recent studies have now placed up
to 75% of Americans online, while passive TV consumption is
dropping yearly.
People watching TV while exploring related content on the
Web. There are many examples of enhanced TV that pushes
this approach.
Most video games now having a multi-player online mode with
game-play interaction and chat and even voice interaction.

Multi-screen environments to capture a greater sense of

immersion or show more content. Most new PC graphics cards
now have two video outputs and some with three have already
emerged.
New game systems are appearing linking multiple displays
(Nintendo Gameboy with a Gamecube), or even with dual-
screens built-in (Nintendo DS) to afford personal views and
shared or strategic views simultaneously.
Children playing a game online with friends with MP3’s
streaming in the background and a movie on the TV.
Individually, these may seem to be small occurrences but when taken
as an emerging pattern, we can see that the types of interaction and
media synchronization afforded by a GIIE support these emerging
trends.
The challenge to date has largely been one of the traditional “chicken
and egg.” While there is desire for new content expressions that match
these trends, there are few standards for technology integration that
support a rich enough or high-enough bandwidth experience to be
meaningful. Without standards, the content industry stays away as the
return on investing in a new format is quite high. The challenge with
even a proven structure such as the GIIE is broad adoption. This is
typically hampered by two factors:
1 Lack of awareness and skilled individuals to take advantage of
a new media format.
2 Lack of a broad (and lucrative) enough distribution channels to
warrant business and content development efforts.
A great deal of knowledge has been gained about GIIEs and producing
content for them over the past ten years. We understand how to
leverage existing digital content tools and skills and where specific
new capabilities are needed. We have found that the greatest
challenges exist in understanding and designing successfully for group
interactivity. The common model of group interaction has been one of
simple voting. This has proven to be a useful but far from satisfying
for most people. Allowing for some levels of personal interaction
ensures people can reach their own goals or find aspects of content that
have a personal relevance. Collaborative and competitive interactions
enrich a sense of inclusion, teamwork, and dependencies amongst
users that are so common and essential in the real world. This guide
has hopefully illustrated further the importance of these ideas and
methods to achieve them.

Where distribution is concerned, specialized venues clearly provide

the best experience but the cost and accessibility of these facilities is a
limiting factor. This guide points out the opportunities of emerging
broadband networks for opening up new distribution options. What
was largely unknown however was the importance of the immersive
quality and direct group engagement on the success of the GIIE
content. The research undertaken for this guide has shown three key
things:
1 It is technically feasible to convert content for broadband
distribution without substantive additional cost or complexity.
We would estimate the typical costs as anywhere from 2-10%
of the original production cost.
2 Degradation in sensory quality (even as low as 320x240
MPEG-1) is a measurable but not a key determining factor in
the overall success of a GIIE experience. The multiple
elements of engagement combine to provide a strong overall
effect. This rule holds true where the original immersive
content is highly rated. We can extrapolate from the amount of
experiential “loss” that occurs in a high-quality content piece
that a borderline piece of content will likely fall below the line
when converted to broadband.
3 The importance of in-person (group together in one location)
aspects of a venue-based GIIE should not be underestimated.
While the research for this guide could not systematically
explore this aspect, user feedback revealed that in-person
contact is very important in a number of contexts. In education,
in-person contact supports concepts such as mentorship. In
professional applications, the feedback loop and trust factors in
negotiation, as one example, pose potential weak points for
broadband use.
While points one and two are very positive and show great promise the
third, the social factor could completely eliminate the overall
broadband opportunity if viewed on its own. The social factor is
primarily concerned with “same-place” versus “different-place”
challenges. Broadband has traditionally been seen as a single-user on a
personal computer, with the broadest distribution being right to the
home. In this mode, users are more likely to be in different places and
as relates to the social factor, this is the most problematic.
There is however a middle ground that does not afford the full
distribution benefit of home access (largest number of potential users),
but is not as limited as the special venues of current GIIEs. We
describe this as the “lab” or “conference room” model. Many

educational environments have shared computing environments where

multiple users can be together but still interact with remote content. In
a professional setting, this is often occurs in corporate boardrooms,
often with laptops and large plasma or projection screens. While these
environments are not available everywhere, they are relatively
common and support the notion of “same-place” known to work for
GIIEs. With this in mind, it is clear that GIIE content can be
successfully (from an effective user perspective) translated and
delivered over a sufficient broadband pipe. How such content
distribution and access is charged for is left as a business case
discussion outside of the scope of this guide.
Future Research
Unfortunately, one guide such as this cannot answer all questions,
even when restricted to the GIIE context. It has however been the goal
of this guide to identify key questions, even where answers are not yet
available. These questions can then be used to direct future research
efforts more effectively. We have classified the research questions into
three categories, which we have organized this guide around:
perception and cognition, technology, and production.
Some of the biggest challenges and opportunities lie within the realm
of perception and cognition. As we identified in the findings, a clear
understanding of how “same-place” versus “different-place” groups
impact the utility and acceptance of GIIE content can have a large
effect upon the scope of utility, accessibility of the concepts and
market potential for this technology and content production. We
believe there are two key aspects that should be carefully considered in
establishing any such research programs:
1 An assessment of the role played (the magnitude of impact) in
the overall value of the experience by “same-place” versus
“different-place” user distribution. This parallels the
assessment of the ‘resolution/bandwidth’ study reported in this
guide.
2 A more detailed assessment of what professional conditions
and activities (in particular) are appropriate for broadband use.
This will require a running application environment and real
tests under real-world conditions to ensure the goals and
needs/desires of the users are honestly reflected. This will
necessarily be domain-specific.

Within the overall discussion of technology, we have presented a small

but critical number of problems for proper deployment of GIIE content
over broadband networks:
Seamless video jumping in interactive streaming video. Most
video systems that work with pre-recorded video for streaming
do not readily support highly interactive (and seamless)
switching of video source files that are needed for the sense of
immersion in many types of content. Buffering times and file
pre-opening are not often high on the priority for support of
generic media presentation.
Most streaming video formats do not support dynamic overlay
of sprites. This prevents the rapid display of changing overall
statistics, group choices or actions that is critical to group
interaction feedback. Those that do support technologies like
SMIL are not high quality or dynamic enough.
Streaming media server with group interactivity
synchronization. Most streaming media servers support a form
of client synchronization (updating web pages timed to a
video). They do not currently support concurrent multi-user
feedback and video interaction methods needed for a smooth
translation from specialized venue to broadband.
Concurrency of interactive states. In professional applications
in particular, clear methods of determining that all users are
viewing an equal immersive environment is essential to
negotiation and decision-making. This is most likely a user
interface and feedback mechanism that requires invention.
Digital production techniques for cinematic environments are richly
supported and have extensive product and research support in industry.
Some professional domains, such as design and planning that we
focused on in this guide do not have solid industry support and
innovation. Immersive environments are too rare and expensive to
have extensive mass-market tools. Where these disciplines are large
enough to support common tools, they often only support the lowest
common denominator of capabilities, not those for high-end
simulation. Where tools do exist, they are typically more generic,
require extensive training and often carry large price tags beyond the
means of design offices with limited technology budgets.
While some of this may be marketing and business factors, they have
resulted in one very large and critical gap in the model assembly phase
of a GIIE. As discussed in the professional content development in
Chapter 7, assembly remains an area of extensive complexity with few

supporting tools. Most users today employ a hybrid of custom and

strung-together tools that work in limited cases. This results in wasted
time and money for new projects with different data input sources,
which is quite common. Differences in scale, coordinate spaces, 3D
model resolution and meshing all contribute to the complexity of this
task.
Another base technology component that continues to be a stumbling
block in professional real-time model building is that of texture
limitations. Current graphics cards in an affordable end-user price
range today support in the range of 128-256MB of memory. This is
still not enough for most models with a comprehensive context for
immersive use. This highlights the need for additional research in
texture compression.
While brute force improvements in texture memory show an almost
yearly doubling, the current standards can easily be exceeded for some
years to come as specific buildings are textured individually and
accurately in high resolution and detailed aerial photography becomes
common for draping over terrain. Techniques for texture compression
will continue to be important along with new pixel shading
technologies that now offer much greater capabilities for this task. For
example, many smaller textures can be combined (bricks tiled with
windows overlaid) to create large building faces with minimal texture
techniques. Brick itself could be defined procedurally and not require
bitmap pixel memory. These are just a few methods that could be used
over time to improve the quality of representations while minimizing
video RAM usage.
There are many more research questions posed by GIIE environments,
within specific sub-components and particularly, within the realm of
component integration or “systems-level research.” We have noted
above some of the most challenging or important factors at this time.
Benefits of this guide

The value of this guide can be measured in a number of different ways,
all of significant value to the eContent and broadband research
industries:
A document outlining the production process for eContent and
its dependencies upon base technology and bandwidth
capabilities.
An understanding of critical bandwidth thresholds for
delivering valuable immersive interactive eContent, based
upon real user perception.

A clear set of research questions and issues facing future

broadband eContent efforts, derived from educational,
professional, technical, and business perspectives.
Economic benefits can be seen in both the short and long-term. Short-
term benefits include clear documentation to the educational, research,
and production institutions and corporations on methods of production
suitable for different bandwidth levels of eContent. Long-term benefits
include clear benchmarks for future technology and perceptual
research for eContent users and producers. A clear appreciation of the
effectiveness and challenges of eContent production and delivery over
different high-bandwidth scenarios makes clear the likely timelines of
business ROI versus infrastructure deployment for the whole eContent
broadband marketplace.
Social benefits are no less significant. Combination immersive and
interactive environments are at the leading edge of educational and
new media exploration and theory. Immersion Studios and its partners
in this guide are defining the next generation of eContent and
strengthening the role of a broad cross-section of industries all with an
international impact but with clear Canadian foundations.
These new eContent experiences are proving very successful in a
range of professional and education contexts but currently require very
high-end intranet environments that limit their reach and accessibility
to broader industries, schools, and individuals. Progress in adapting
this work for delivery to a wider range of broadband-connected sites
now and in the future will add significant value to education and
professional work across the country.

Appendix A
IIC Evaluation
Interactive Content Development Guide Appendix A
Effects of Resolution, Grade, Gender, and Script on

Student and Teacher Appraisal of an Immersive
Interactive Entertainment Education Experience:
Towards an Interactive Content Development Guide
(ICDG)
Annabel J. Cohen1, Sandy MacAulay2,

Betty Bailey1, Jennifer Ghiz 1,2, and Kelti MacMillan1
Department of Psychology, Faculty of Arts1 &

Faculty of Education2
University of Prince Edward Island
Charlottetown, PE C1A 4P3
March 15, 2004
Address for Correspondence

Annabel J. Cohen
Department of Psychology
Charlottetown, PE C1A 4P3
(902) 628-4325 telephone

(902) 628-4359 fax
© 2004 Immersion Studios Inc A-1

Summary
The present study investigated factors influencing the value of an
interactive, immersive Entertainment Education experience about sea
lions or dinosaurs provided to over 450 male and female students in
grades 5 and 9 in Charlottetown, Prince Edward Island. Students
completed questionnaires immediately after the show and after a delay
of one week. Their high ratings of enjoyment, presence, and acquired
knowledge generally confirmed results of a prior study of the sea lion
show (Ritterfeld, Weber, Fernandes, & Vorderer, 2003). Teachers,
who had accompanied their students, also rated the experience
positively both immediately and after the one week delay. They
believed that students benefited most from the combined large screen
and interactive component, that all students enjoyed the experience,
and that even students with special needs would benefit from it.
Enjoyment, interest in learning more, and presence were highly
correlated for both shows, and in general the two different shows led
to the same pattern of appraisal. Girls responded significantly more
positively to the presentation than did boys, and grade 5 students
responded significantly more positively than did grade 9 students, but
these differences were small from a practical standpoint. Although
student-self-assessed and teacher-assessed learning was high, learning
of specific facts was not well-reflected in the results of a 32-item yes-
no questionnaire when compared with a control group of over 200
students who had not experienced the presentation. Similar findings
were also observed by Ritterfeld et al. (2003) suggesting that the
educational value of the presentation lies not in acquisition of new
facts but rather in the establishment of new perspectives. One primary
variable of interest was degree of resolution or visual clarity of the
immersive presentation. For the dinosaurs show, one third of the
students viewed the show under low, medium, or high visual
resolution. Mean student ratings on several scales (e.g., “the show was
fun”) varied with increasing degree of resolution; however, these
differences were negligible practically speaking. Small and moderate
degradations in quality of a large-screen display of an interactive
Entertainment Education presentation had little impact on enjoyment,
learning, and sense of presence. High appraisal of the immersive
interactive educational experience by students and teachers regardless
of the level of resolution could warrant more widespread availability
through broadband distribution at lower resolution than is technically
possible under more costly presentation conditions.

Effects of Resolution, Grade, Gender,

and Narrative on Student and Teacher
Appraisal of an Immersive Interactive
Entertainment Education Experience
The purpose of the Interactive Content Development Guide (ICDG)
Project is to develop guidelines for the effective transfer of state-of-
the-art, interactive, multimedia, immersive experiences to the next-
generation broadband environment as exemplified by CA*Net 3 and 4.
Currently no such guidelines exist. In the K-12 sector, the basic issues
are to identify:
1 Factors that are critical for a successful immersion experience.
2 The threshold at which they come into play.
3 The types of students they affect.
The present project conducted in the province of Prince Edward Island
aimed to address these issues by exploring the effectiveness of
Entertainment Education shows presented to groups of students
representing a wide age and grade range. Of particular interest was the
impact of the technical constraints of broadband networks on the
effectiveness of a state-of-the-art interactive, multimedia, immersive
Education-Entertainment experience. To this end, two different shows
were presented at three levels of resolution (image and sound clarity)
in a special theatre with a large immersive screen and individual
interactive terminals.
Background
The expansion of multimedia technology has resulted in the
development of innovative Entertainment Education which blends
entertaining experiences with educational goals. In recent research
(Ritterfeld, Weber, Fernandes, & Vorderer, 2003), 656 children
participated in a science based immersion show in an Immersion
Theatre environment which combined a movie-like cinematic
presentation on a large screen with a computer-game-playing
environment on a personal screen. Through interplay between
information presented on the global screen and individual game play,
students participated in solving a real scientific problem about sea
lions. Results indicated that the Entertainment Education immersion
experience had positive educational effects, for example on enjoyment
and sense of presence. However constraints of the research did not

allow for investigation of all of the factors which may have

contributed to the educational impact. Ritterfeld et al. (2003) suggest
that variables such as prior learning experiences, attention span,
memory capacity, prior knowledge and prior interest in science may
have influenced the educational value of the show. Additionally it was
suggested that highly motivated students would assimilate information
regardless of the technology or artistry supporting its presentation. It
was concluded that the benefits of the Entertainment Education
immersion experience were greatest for students who were less
motivated.
The costs of providing the highest quality immersion experience deter
its widespread distribution. Much of this cost is tied to the large, high
bandwidth screen. The extent to which this high resolution is the basis
for the success of the Entertainment Education product remains
unclear. More affordable broadband transmission may still allow the
educational benefits of the other features of the experience in spite of
additional reductions in sound quality, acoustic and seating of the
immersive theatre environment.
The current project focuses on the potential impact that certain
technical constraints of broadband networks could have on the
educational value of the immersion experience. It also aims to
examine the same show as was examined by Ritterfeld et al. (2003) to
determine the generality of the findings with a Canadian sample. As
well, to further determine the generality of the findings, an additional
show about Dinosaurs was included. Whereas Ritterfeld et al. (2003)
presented the Sea Lions show to over 650 students across 3 different
settings, they obtained follow-up data on retention or learning from
less than 20% of the original sample. The present study aimed to
obtain such data from a majority of those who saw the shows.
The study employed two interactive science-based shows: Jason XIV
(about Sea Lions) and Dinosaurs: Beyond Extinction. All showings
took place in the Immersion Theatre at the Atlantic Technology Centre
in Charlottetown, PEI. Each show was to be delivered at three levels
of degradation:
1 A full, HD unaltered version of Intranet quality (~360Mbit).
2 A version altered to simulate slight degrees of network
constraints – MPEG2 at 10Mbit but low-res bitmaps (tutorial,
maps).
3 A version altered to simulate medium degrees of network
constraints – MPEG1 at 1Mbit (3 shows) and low-res bitmaps.

Each resolution level of each show was to be viewed by two grade 5

elementary school and two grade 9 junior high classes (average class
size 22 students) creating 24 class viewings in all. To avoid
confounding the effects of the condition of interest with unwanted
effects of a particular school, the two different schools were to be
tested at each level of each condition. Eight additional control classes
and six pilot groups also participated.
Method
Procedure
Acquisition of the participation of 38 classes in the study entailed
contacts with the school boards, principals, and teachers; creation of
schedules and clearance of transportation services (school buses).
More specifically, the Director of Program Services of the Eastern
School District was contacted by telephone and by letter to request
permission to contact the elementary and junior high school principals
in the PEI Eastern School District to participate in the study. Upon
receiving permission to contact the principals, an information packet
was sent to the school board and principals at all applicable schools.
The packet contained: (1) a letter describing the research and
requesting permission to recruit students within the school, (2) the
student consent form, (3) parent consent form, (4) the teacher short
term impact questionnaire, (5) the teacher long term impact
questionnaire, (6) the student short term impact questionnaire and (7)
the student long term impact questionnaire. Once permissions had
been obtained, classes were assigned to control and experimental
conditions. When actual testing took place, two experimenters and
typically one teacher were on hand to organize the testing of each
single class of students in the immersion theatre. Teachers oversaw
any questionnaire completion conducted in the classroom.
Participating schools received a small monetary contribution to their
library as a token of appreciation for their cooperation.
Approximately one to five days prior to the immersion experience, the
students filled out a preliminary questionnaire in their class to develop
a personal code number, and to collect demographic information of
age, and hobbies and interests (Appendix A). On the day of the
immersion experience, students and their teacher arrived by bus at the
Atlantic Technology Centre (ATC) and were escorted to the
Immersion Theatre.
The Theatre is a purpose-built facility, sloping to a two-story height of
the wall supporting the screen. It is sound-treated and carpeted. There

is one computer terminal for every pair of seats spaced in several

rows, with 25 terminals in all, 22 of which were functional for the
study.
The students participated in the interactive show while their teachers
observed them interacting at their terminals or watching the large
screen. Following the show, the students were asked to complete a
questionnaire about their experience of the show as well as questions
about their general school experience (see Appendix B). At this time,
teachers were also asked to complete a short-term evaluation of the
show related to their observations of the students during the show (see
Appendix C). Additionally, a week after the show, students were
asked to complete a second questionnaire related to the long term
impact of the show (see Appendix D) and the teachers also completed
a second more reflective evaluation of their attitudes about the show
(see Appendix E).
Design of Experiment
A primary variable of interest was the level of resolution of the show.
The high resolution version was HD Intranet quality and the same
quality that had been experienced by participants in the study of
Ritterfeld et al. (2003). Two additional levels were compared. The first
designated “medium resolution” was perceptibly reduced in resolution
when compared directly with the high resolution version, however,
without the comparison, it might be hard to imagine a superior quality.
The third level, designated “low resolution”, was perceptibly of
reduced resolution without the need of comparison to a HD standard,
although discernability of information was not impaired. The
technology underlying the creation of the levels of resolution of the
shows was developed by Immersion Studios where additional
information can be obtained.
Testing of pilot classes of grade 6 students, one for each of two shows
for each level of resolution (approximately 150 students) enabled the
checking of the procedures for running the experiment.
The testing session in the Immersion Theatre lasted about 45 minutes,
whereas the presentations were approximately 22 and 25 minutes in
duration. Upon arrival and settling down of the class in the theatre, the
first 22 students were assigned to a separate terminal, with the
remainder (if any) required to double up. A technologist familiar with
the Immersion Studios technology initiated the first run for the day,
and, subsequently, the experimenters ran a second, third or fourth
show depending on the day’s schedule. Immediately upon completion
of the presentation, the students were asked to complete a

questionnaire. Teachers were asked to fill out another questionnaire. A

week later, students completed another questionnaire (30 minutes) in
their classrooms, and the teacher also completed a follow-up
questionnaire.
The design of the experiment is shown below. Table 1a shows the
different schools tested at each level of resolution, for each grade, for
each of the shows. Where possible, classes from the same school were
distributed across the three levels of resolution to rule out the
extraneous variable of school (e.g., some schools may normally
provide more access to computers, or more trips). To further avoid an
effect of a particular school, there were two schools for each level of
resolution for each show for each grade. Table 1b shows the number
of students who were tested in each of the three sessions where 1
refers to the pre-test in the home classroom, 2 refers to the test at the
Immersion studio immediately following the immersion theatre
presentation, and 3 refers to the post-test carried out about a week later
in the home classroom. A total of 567 students viewed the Immersion
show and completed the immediate impact questionnaire and 531
completed the delayed impact questionnaire (93.6% retention).

Table 1a:
Experiment Design
Sea Lions Dinosaurs
low res medium res high res low res medium res high res
Grade Glen Stewart Glen Stewart Glen Stewart Spring Park Spring Park Spring Park
5 Sherwood Sherwood Sherwood St. Jeans West Royalty West Kent
Grade Birchwood Birchwood Birchwood East Wiltshire East Wiltshire East Wiltshire
9 East Wiltshire East Wiltshire East Wiltshire Stone Park Queen Charlotte Stone Park
Table 1b:
Experiment Design: number of participants in each of three successive questionnaires
Sea Lions Dinosaurs
low res medium res high res low res med res high res
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
Grade 26 26 25 27 27 26 24 24 24 24 23 23 23 22 21 19 16 23
5 24 24 23 20 20 20 20 19 19 23 23 20 26 26 21 17 15 14
50 50 48 47 47 46 44 43 43 47 46 43 49 48 42 36 31 37
Grade 24 24 24 26 26 26 23 25 21 20 29 20 26 21 25 32 31 18
9 23 22 17 29 24 24 22 22 22 23 19 18 31 31 29 28 28 28
47 46 41 55 50 50 45 47 43 43 48 38 57 52 54 60 59 46
Total 97 96 89 102 97 96 89 90 86 90 94 81 106 100 96 96 90 83

Control conditions
There were also 201 students who completed the first and final
questionnaires in order to determine a baseline for the amount known
about the topics of Sea Lions and Dinosaurs and to obtain
demographic data. These students also viewed one of the shows, but
only after the questionnaires were completed. There were
approximately 50 students in each Grade (5 and 9) x Show (Sea Lions
/ Dinosaur) combination.
Pilot testing
Prior to testing of the Grade 5 and 9 classes (approximately 150
students) in the main experimental design, six Grade 6 classes were
presented with one of the two shows in one of the three resolution
conditions in order to test the basic methodology and instrumentation.
Grade 6 was chosen for the following practical reasons. Falling
between the two grade levels in the main experiment, it would
sufficiently represent each. Second, same schools could provide the
grade 6 classes that were providing some of the other classes in the
main experiment thus avoiding the approach to additional schools and
principals for Grade 5 control classes. Finally, the procedure of
transporting students from several of the schools involved in the main
experiment would receive a “dry run.”
Results
Each student who participated in the main experiment produced three
sets of data: (1) in their school classroom, preliminary demographic
data, 1 to 5 days prior to the Immersion presentation (2) in the
Immersion Theatre, immediately after the Immersion presentation, and
(3) in their school classroom, exactly 1 week after the show. As well,
the teachers of students in the main experiment completed two
questionnaires to give their impressions of the experience of the show
for their students, (1) immediately after the Immersion presentation
and (2) approximately a week after the Immersion presentation.
The study by Ritterfeld et al. (2003) carried out the student and teacher
assessments immediately after the Immersive experience, and a post-
test for a small proportion (< 20%) of both students and teachers. We
will begin with the student and teacher questionnaires following the
immersive experience (referred to as Short term impact and Study 1
for students and Study 2 for teachers by Ritterfeld et al. (2003).

1. Short term impact of the show from the student’s perspective.

The students had completed a questionnaire of 29 items (see Appendix
B) providing information about their self-assessed: academic
achievement, pre-show interest in scientific issues related to the topic
of the show, interest in computer related activities, sensation seeking,
and post-show evaluation in terms of enjoyment, degree of absorption
in the show, amount learned, confusion, and desire for a repeat
performance.
The data were analysed as a whole, and in two subgroups that arose
because in many showings, there were too many students to provide
each with his or her own terminal. Over the 24 classes tested,
approximately 100 students shared a terminal. With this number who
shared terminals, it is impossible to determine whether sharing
interacted with any of the independent measures or affected directly
the dependent measures examined. Taking a conservative approach,
only the data from participants who had his or her own terminal during
the presentation are presented here. This left 162 participants with his
or her own terminal in the lowest level of resolution, 155 in the middle
level, and 147 in the highest level of resolution, for a total of 464
students at individual terminals: 216 males, 248 females, 216 saw Sea
Lions, 248 saw Dinosaurs, 219 were in Grade 5, and 245 were in
Grade 9.
The mean rating for each grade, and for each level of immersion is
shown in Table 2a for Dinosaurs and in Table 2b for Sea Lions. These
data include only those who had an individual terminal. It can be seen
that mean ratings are uniformly high (near 4-5) or low (near 1-2).
Students were, in general, very enthusiastic about the experience, as
indicated by near ceiling responses that the show was fun, that they
would have loved to play longer, that they wanted to play the show
again, and that they wished the teacher would provide similar
opportunities again. There was a tendency for this enthusiasm to be
greater for the younger age groups.
To focus on the role of degree of resolution, for every question, for
every grade, where mean ratings change in the expected direction with
changes in resolution (e.g., mean rating of “the show was fun”
increased with resolution), the background for the three cells was
coloured green. Where the pattern was opposite to that expected, the
background is coloured red. For variables describing characteristics of
the participant (e.g., “my school grades are excellent”) change
systematically with resolution, the background is yellow. Ideally, the
personal characteristics would be the same across conditions. If they
are not due to unfortunate sampling, a confounding variable is

introduced that either explains systematic increases or decreases with

resolution. In the present study there were several cases where, for
example, measures of intelligence, or experience in surfing the
Internet did increase for higher levels of resolution for the Grade 9
students who saw Sea Lions. This would act against the possibility of
an effect of resolution having an influence, if for example, resolution
affected the less intelligent or the less computer-skilled students, as
Ritterfeld et al. (2003) suggested.
Of greater importance is the fact that for the Dinosaur Show there are
far more green cells than red (18 vs. 2), that is, more patterns in the
dependent measures that are consistent with the hypothesis that
resolution will increase the effectiveness of the show. Nevertheless,
these differences in values as a function of lowest and highest
resolution are in general very small, and typically not greater than .50
of a scale value on the 5-point scale and often much less. It is also the
case that the two grades do show the same sensitivity to resolution for
a dimension. For Sea Lions, however, this pattern is not evidence, with
an equal number of red and green cells.
An unfortunate experimental error, discovered after all testing and
initial data analysis, made it possible to evaluate the resolution
variable completely only for Dinosaurs. Logs of the host computer at
Immersion Studios revealed that for Sea Lions, high resolution
presentations replaced one of the four intended low resolution
conditions and three of the four intended medium resolution
conditions. Thus although overall transitivity of the resolution variable
was preserved for Sea Lions, the factor was not linearly distributed as
it was for Dinosaurs. The origin of this error is unclear, however, it
was specific to the Sea Lions show and absent from all Dinosaur
presentations. The validity of the resolution variable primarily for the
Dinosaur condition is consistent with the transitive pattern of means,
indicated by the larger number of green over red cells for Dinosaurs
versus the equal numbers of red and green cells for Sea Lions.

Table 2(a)
Mean rating for each question immediately following the presentation of Dinosaurs
Each cell shows the mean rating, the standard deviation, and the number of respondents.
consistent pattern inconsistent pattern personal trait potential confound
Grade 5 - Dinosaurs Grade 9 - Dinosaurs
Low Mid High All Low Mid High All
The show was fun. 4.81 4.69 4.97 4.82 4 .26 4.39 4.52 4.39
.58
.70 .62 .18 .74 .79 .83 .79
110
43 36 31 35 38 33 106
I learned a lot during the show. 4.36 4.11 4.35 4.28 3.75 3.55 3.63 3.64
.69 .89 .91 .83 .97 1.10 .86 .98
42 36 31 109 36 38 33 107
There were too many pictures, information and 1.78 1.78 1.32 1.64 1.64 1.63 1.52 1.60
games in the show.
1.08 1.27 .83 1.1 .99 .97 1.06 1.00
41 36 31 108 36 38 33 107
I figured out how to play the games. 4.33 4.33 4.70 4.44 4. 3.97 3.91 3.96
.93
1.12 .93 .53 1.29 .85 1.33 1.16
109
42 36 31 36 38 33 107
At home I love to listen to loud music. 4.29 3.60 3.87 3.94 4.5 4.34 4.30 4.38
1.31 1.65 1.25 1.43 .65 .97 1.02 .89
42 35 30 107 36 38 33 107
I often surf the internet. 3.30 3.56 3.43 3.43 4.31 4.32 4.24 4.29
1.62 1.44 1.53 1.53 .86 .96 1.06 .95
42 34 28 104 36 38 33 107
I got upset during the show. 1.29 1.06 1.03 1.14 1.39 1.35 1.30 1.35
.81 .34 .18 .56 .90 .75 .88 .84
42 34 30 106 36 37 33 106
I wanted to learn why the dinosaurs became extinct. 4.21 3.79 3.37 3.84 2.89 3.22 3.79 3.28
1.18 1.59 1.54 1.46 1.16 1.32 1.08 1.24
42 34 30 106 36 37 33 106
My grades in school could be much better. 3.50 2.80 3.25 3.1 3.75 3.11 3.24 3.36
1.48 1.49 1.38 1.47 1.16 1.41 1.44 1.36
38 35 28 101 36 38 33 107

Today I would have loved to play longer. 4.36 4.66 4.87 4.60 4.47 4.58 4.55 4.53
1.26 .73 .51 .95 .70 .79 .87 .78
42 35 30 107 36 38 33 107
After playing, I want to learn more about dinosaurs. 4.17 3.34 3.97 3.84 3.08 3.21 3.13 3.14
1.18 1.43 1.05 1.28 1.18 1.40 1.21 1.26
41 35 29 105 36 38 32 106
I often did not know if I would have to look at the big 2.9 2.15 2.34 2.50 2.33 2.21 2.53 2.35
screen or the small screen.
1.55 1.48 1.37 1.50 1.31 1.30 1.50 1.36
41 33 29 103 36 38 32 106
The show was boring. 1.17 1.52 1.31 1.32 2.08 1.54 1.76 1.79
.58 1.06 .93 .86 1.16 .93 1.15 1.09
42 33 26 101 36 37 33 106
When I came in, the topic of the show did not interest 1.65 2.09 1.63 1.79 2.81 2.59 2.25 2.56
me.
1.21 1.42 1.19 1.28 1.37 1.24 1.41 1.34
40 35 30 105 36 37 32 105
The teacher should take us to more shows like this. 4.66 4.62 4.77 4.68 4.58 4.74 4.76 4.69
1.02 .70 .90 .88 .80 .72 .75 .76
42 34 30 105 36 38 33 107
After playing, I want to learn more about the oceans, 4.33 3.61 4.10 4.04 2.94 3.32 3.33 3.20
animals and weather. .
1.20 1.30 1.03 1.22 1.07 1.25 1.14 1.16
42 34 30 106 36 38 33 107
I am an excellent student. 4.14 3.91 4.07 4.05 3.53 3.89 3.91 3.78
.93 1.09 1.05 1.01 1.06 1.06 1.23 1.12
42 35 30 107 36 38 33 107
I want to play the show again. 4.52 4.43 4.93 4.61 3.92 4.23 4.36 4.19
.99 .98 .26 .87 1.27 1.04 1.14 1.16
42 35 30 107 36 38 33 107
At first I did not really care why dinosaurs 2.76 2.77 2.65 2.73 3.37 2.92 1.97 3.08
disappeared from the earth.
1.59 1.54 1.50 1.53 1.24 1.24 1.47 4.32
42 35 31 108 35 38 33 106
During the show I sometimes got angry. 1.62 1.14 1.13 1.33 1.44 1.45 1.25 1.39
1.31 .60 .73 .99 .88 1.06 .80 .92
42 35 30 107 36 38 32 106

I often play computer games. 4.05 3.53 3.55 3.74 3.66 3.95 3.97 3.86
1.43 1.65 1.55 1.54 1.26 1.06 1.33 1.21
42 36 29 107 35 38 33 106
I am often bothered by loud noise. 2.29 1.97 1.84 2.06 1.47 2.08 1.64 1.74
1.60 1.46 1.16 1.44 .70 1.30 1.25 1.13
42 36 31 109 36 37 33 106
In the show it was hard to know what to do. 2.62 2.14 1.97 2.28 2.39 2.76 2.36 2.51
1.56 1.38 1.27 1.44 1.29 1.24 1.39 1.31
42 36 30 108 36 38 33 107
I got distracted by stuff on the big and small screen 2.56 1.59 1.58 1.96 2.09 2.34 2.15 2.20
to look at.
1.57 1.02 .96 1.32 1.09 1.21 1.48 1.26
41 34 31 106 35 38 33 106
I wanted to investigate why the dinosaurs became 4.26 3.46 3.80 3.87 2.72 2.74 3.30 2.91
extinct.
1.17 1.42 1.54 1.39 1.16 1.46 1.31 1.34
42 35 30 107 36 38 33 107
During the show, I forgot about everything else. 3.93 3.75 3.97 3.88 3.08 3.24 3.59 3.29
1.40 1.44 1.33 1.39 1.30 1.32 1.41 1.35
42 36 31 109 36 38 32 106
Today I learned important things about dinosaurs. 4.57 4.31 4.65 4.50 3.36 3.58 3.91 3.61
.77 .95 .87 .87 1.22 4.27 1.18 1.23
42 36 31 109 36 38 33 107
I felt that I was part of the show. 4.20 3.72 3.81 3.93 3.28 3.66 4.06 3.65
1.15 1.56 1.35 1.36 1.32 1.15 1.09 1.22
42 36 31 109 36 38 33 107
I would tell my friends to play. 4.10 4.00. 4.55 4.19 3.56 3.79 4.18 3.83
1.45 1.31 .99 1.29 1.36 1.26 1.21 1.29
42 36 31 109 36 38 33 107

Table 2(b)
Mean rating for each question immediately following the presentation of Sea Lions
Each cell shows the mean rating, the standard deviation, and the number of respondents.
(Resolution variable invalid).
Grade 5 - Sea Lions Grade 9 - Sea Lions
Low Mid High All Low Mid High All
The show was fun. 4.87 4.74 4.67 4.76 4.54 4.54 4.4 4.50
.41 .61 .86 .65 .72 .75 .74 .74
38 35 36 109 46 46 47 139
I learned a lot during the show. 4.11 4.34 4.47 4.30 4.38 4.28 4.38 4.35
1.13 .97 .84 1.00 .65 .72 .77 .71
38 35 36 109 45 46 47 138
There were too many pictures, information and 1.79 2.11 1.72 1.87 1.41 1.80 1.85 1.69
games in the show.
1.26 1.23 1.34 1.28 .78 1.07 1.12 1.01
38 35 36 109 46 46 47 139
I figured out how to play the games. 4.42 4.03 4.28 4.25 4.46 4.29 3.91 4.32
.95 1.32 1.21 1.16 .72 .83 1.33 .83
38 35 36 109 46 46 33 139
At home I love to listen to loud music. 3.29 3.57 3.89 3.57 4.24 3.89 4.23 4.12
1.52 1.63 1.39 1.52 1.14 1.39 1.15 1.23
38 35 35 108 46 45 47 138
I often surf the internet. 3.82 3.36 3.89 3.70 4.04 4.11 4.17 4.11
1.39 1.71 1.39 1.50 1.25 1.02 1.22 1.16
38 33 35 106 46 46 47 139
I got upset during the show. 1.32 2.26 1.41 1.66 1.80 1.84 1.72 1.79
.82 1.64 1.04 1.27 1.17 1.15 1.17 1.16
37 34 32 103 46 45 47 138
I wanted to learn why the sea lion population went 4.11 3.91 4.0 4.01 4.02 3.78 4.17 3.28
down.
1.07 1.40 1.26 1.24 .97 1.05 .96 1.24
37 35 35 107 45 46 47 106
My grades in school could be much better. 3.18 3.0 2.45 2.89 3.93 3.80 3.64 3.79
1.47 1.41 1.48 1.48 1.12 1.31 1.36 1.27
38 33 33 104 46 46 47 139

Today I would have loved to play longer. 4.55 4.66 4.24 4.49 4.51 4.74 4.38 4.54
1.01 .54 1.44 1.06 .79 .53 .95 .78
38 35 34 107 45 46 47 138
After playing, I want to learn more about science. 3.37 3.35 3.53 3.42 3.50 3.24 3.36 3.37
1.32 1.45 1.31 1.35 1.15 1.23 1.26 1.21
38 34 34 106 46 46 47 139
I often did not know if I would have to look at the 2.38 2.45 2.77 2.53 2.22 2.35 2.35 2.31
big screen or the small screen.
1.50 1.37 1.50 1.45 1.20 1.27 1.40 1.29
37 33 34 104 45 46 46 137
The show was boring. 1.39 1.27 1.26 1.31 1.61 1.65 1.73 1.66
.84 .67 .90 .80 .95 1.04 1.05 1.01
36 33 34 103 46 46 45 137
When I came in, the topic of the show did not interest 2.20 1.80 2.43 2.14 2.69 2.85 2.53 2.69
me.
1.47 1.30 1.63 1.48 1.36 1.37 1.27 1.33
35 35 35 105 45 46 47 138
The teacher should take us to more shows like this. 4.63 4.94 4.91 4.82 4.62 4.82 4.40 4.61
.79 .34 .38 .56 .78 .65 1.08 .87
38 35 34 107 45 45 47 137
After playing, I want to learn more about the 3.92 4.23 4.46 4.19 3.64 3.67 3.74 3.69
oceans, animals, and weather.
1.02 1.00 .85 .98 1.14 1.08 1.06 1.09
38 35 35 108 44 46 46 136
I am an excellent student. 3.92 3.74 4.18 3.94 3.48 3.65 3.98 3.71
1.12 1.16 1.0 1.10 1.03 1.30 .99 1.13
37 34 34 105 46 46 47 139
I want to play the show again. 4.57 4.53 4.80 4.63 4.37 4.63 4.13 4.37
.99 .93 .76 .90 1.00 .88 1.24 1.07
37 34 35 106 46 46 47 139
At first I did not really care why the sea lion pups 2.53 1.59 1.80 2.00 2.28 2.35 1.94 2.19
died.
1.62 1.10 1.35 1.43 1.15 1.30 1.19 1.22
38 32 35 105 46 46 47 139
During the show I sometimes got angry. 1.81 1.88 1.45 1.72 1.67 1.76 1.83 1.75
1.37 1.39 1.09 1.29 1.22 1.13 1.17 1.17
37 32 33 102 45 45 47 137

I often play computer games. 3.84 3.76 4.03 3.88 3.59 3.59 4.02 3.73
1.36 1.60 1.36 1.43 1.26 1.31 1.13 1.24
37 33 35 105 46 46 47 139
I am often bothered by loud noise. 2.49 2.39 2.09 2.33 2.11 2.30 2.02 2.14
1.39 1.60 1.46 1.48 1.02 1.39 1.31 1.24
37 33 34 104 46 44 46 136
In the show it was hard to know what to do. 2.24 2.31 2.29 2.28 2.02 1.80 2.23 2.02
1.30 1.47 1.30 1.34 1.11 .94 1.24 1.11
38 32 35 105 46 45 47 138
I got distracted by stuff on the big and small 2.22 2.0 1.94 2.06 1.85 1.84 1.80 1.83
screen to look at.
1.44 1.37 1.25 1.35 .94 1.09 1.11 1.04
36 33 34 103 46 45 46 137
I wanted to investigate why the sea lion pups 3.76 4.33 4.37 4.14 3.79 3.80 3.93 3.84
died.
1.48 1.02 1.09 1.25 1.25 1.16 1.31 1.23
38 33 35 106 46 45 46 137
During the show, I forgot about everything else. 3.50 3.45 3.49 3.48 3.50 3.78 3.50 3.59
1.59 1.42 1.58 1.52 1.41 1.37 1.22 1.33
38 33 35 106 46 46 46 138
Today I learned important things about the ocean. 4.45 4.61 4.73 4.59 4.28 4.29 4.24 4.27
1.01 .90 .80 .90 .89 .82 .90 .86
38 33 33 104 46 45 46 137
I felt that I was part of the show. 3.62 3.67 4.29 3.86 3.76 3.85 3.85 3.82
1.46 1.55 1.23 1.44 1.32 1.17 1.19 1.22
37 33 35 105 46 46 46 138
I would tell my friends to play. 3.97 4.36 4.34 4.22 4.15 4.02 3.78 3.99
1.35 .99 1.21 1.20 1.03 1.32 1.28 1.22
38 33 35 106 46 46 46 138
To determine whether there were effects of age or gender, and whether

Sea Lions and Dinosaurs were equally interesting to the students,
statistical analyses were conducted on each of the variables. The
original Resolution variable was maintained, however, it was only
completely valid for the Dinosaur show, as some of the intended low
and medium resolution conditions were actually high resolution, as has
been previously pointed out. The rating (on the 5-point scale) on each
item for each subject was entered into an analysis of variance having
one within subjects factor of Resolution (3 levels: low, middle, high)

and three between groups factors of grade (5 vs. 9), show (Sea Lions
vs. Dinosaurs) and gender (male vs. female) each having two levels.
Regarding the Resolution variable, if low resolution led to lower levels
of enjoyment or learning then obviously there was an educational cost
to employing lower level equipment in an educational setting. If,
however, this degradation variable was statistically negligible, then
less costly, more widely distributed Immersive programming for
education could be warranted. The data are available from the UPEI
researchers but are not included here as a simpler approach to the data
analysis, following Ritterfeld et al. (2003) was taken which reduced
the number of variables for consideration from 29 to 15. Separate
analyses for Dinosaurs alone were also conducted.
Ritterfeld et al. (2003) did not report their data as a function of the
independent variables of gender and grade, nor did they provide the
data for each of the test items as in Table 2b (Table 2a refers to the
data from Dinosaurs, which was only investigated in the present study
and not by Ritterfeld et al.).
Ritterfeld et al. (2003), however, pointed out that for all but one of the
29 questions, each question was a member of a pair of questions
examining the same concept, for example “I am excellent student” and
“My grades in school could be better” test the self-assessment of
scholastic performance, or, as another example: “I wanted to
investigate why the sea lion pups died” and “I wanted to learn why the
sea lion population went up”. For purposes of comparison with
Ritterfeld et al. (2003), the 28 questions were recoded into pairs that
tested the 14 concepts specified by Ritterfeld et al. (2003) as shown in
Table 3.
The overall means for the two shows appear in Table 4.

Table 3.
Recategorized rating dimensions in accordance with Ritterfeld and Vorderer (2003). The term “recoded” refers
to a reversal of sign to give both scales the same high and low values on the dimension
Achievement 9. My grades in school could be much better. (Recoded)
17. I am an excellent student.
Sensation seeking 5. At home I love to listen to loud music.

22. I am often bothered by loud noise. (Recoded)
Media competence 21. I often play computer games.
6. I often surf the Internet.
Enjoyment 1. The show was fun.

10. Today I would have loved to play longer.
Presence 26. During the show I forgot about everything else.

28. I felt that I was part of the show.
Challenge 13. The show was boring. (Recoded)

3. There were too many pictures, information and games in the
show. (Recoded)
Interest in topic prior to attending the show. 14. When I came in the topic of the show did not interest me.
19. At first I did not really care why the dinosaurs disappeared
from the earth.
Negative emotions 7. I got upset during the show.

20. During the show I sometimes got angry.
Usability 4. I figured out how to play the games.
23. In the show it was often hard to know what to do.
(Recoded)
Interplay 12. I often did not know if I would have to look at the big screen
or the small screen.
24. I got distracted by stuff on the big and small screen to look
at.
Motivation 25. I wanted to investigate why the dinosaurs became extinct.
8. I wanted to learn why the dinosaurs became extinct.
Interest increased during the show. 11. After playing I want to learn more about science.
16. After playing I want to learn more about evolution, survival
and dinosaurs.

Learn 2. I learned a lot during the show.

27. Today I learned important things about dinosaurs.
I want to play the show again. 18. I want to play the show again (appreciation)
Recommend 29. I would tell my friends to play.

15. The teacher should take us to more shows like this.
Table 4
Students: Questionnaire Immediately Following Immersive Presentation
Descriptive Statistics
Sea Lions Dinosaurs
Variable Mean N Mean N
SENSS 3.19 240 3.32 208
ACHIEV 3.84 239 4.15 212
MEDCOMP 3.88 241 3.83 207
ENJOY 4.57 245 4.59 213
PRESEN 3.7 243 3.69 205
CHALL 4.36 240 4.40 217
TOPIC 2.29 239 2.53 207
NIGEMO 1.74 232 1.30 214
USABIL 4.07 243 3.91 216
INTERPL 2.13 233 2.24 205
MOTIVAT 3.98 240 3.48 210
INTERES 3.66 242 3.55 210
LEARN 4.36 240 4.01 215
PLAY AGAIN 4.39 240 4.36 211
RECOMM 4.49 245 4.4 214

Summary of the Analyses of Variance of the

15 Rating Dimensions used by Ritterfeld et
al. (2003).
For each of the 15 variables assessed by Ritterfeld et al. (2003), an
analysis of variance was conducted having one within factor of
Resolution with 3 levels, and three between groups factors of gender,
grade, and show, each having two levels. The results are provided for
each of the dimensions analysed. Responses to the show are presented
first, followed by self-assessments of the characteristics of the
participants. Results specifically related to the Resolution variable are
typed in bold.
Enjoyment
The Enjoyment variable aggregated the judgments that the show was
fun and that the participant would have loved to have played the game
longer. Grade 5 children (Mean = 4.67) significantly enjoyed the
experience more than the Grade 9 students (4.50), although Enjoyment
level was exceedingly high and almost at ceiling (4.58, SD. 0.66) on a
5-point scale, F (1, 23) = 7.3, p < .01. Of considerable interest is the
fact that for the Dinosaur show, Enjoyment increased with
increasing Resolution (4.47, 4.60, 4.73 respectively), but the same
pattern was not observed for the Sea Lion show (4.61, 4.67, 4.43).
The interaction of Resolution and Show was significant, F (2, 23) =
4.30, p < .05 and is attributable to the flatter function for the Sea
Lion show as compared to the steadily increasing function for
Dinosaurs.For Dinosaurs the effect of Resolution is consistent with
expectation, that the highest resolution would be associated with the
highest enjoyment level. However, it is clear that even the poorest
level of resolution here leads to high ratings of enjoyment, and the
effect of resolution, although significant for one of the shows has an
effect of just over one-quarter of a scale value on the 5-point rating
scale.
Presence
Presence is one of the key effects of the Immersion experience, and is
in a sense a measure of Immersion. It represented the aggregate of the
degree to which all else was forgotten during the show and the feeling
that the participant was part of the show. The overall rating was high
(3.69) and almost identical for both shows. While it might have been
expected that the Resolution variable would have influenced the sense

of Presence, this was not the case. There were no significant effects of
Resolution. The only significant effect was that of Gender, with
females (3.80) expressing higher levels of Presence than males (3.57),
F (1, 23) = 5.85, p < .05. The effect was stronger in the Dinosaur
show, and the interaction between Gender and Show approached
significance (p < .054). Females may be willing to report higher levels
of Presence or they may be more willing to “let themselves go”.
Regardless, the effect is again on the order of .25 of a scale value.
Challenge
The Challenge factor aggregated the degree to which the show was
boring, and the excess of information in the show. The mean value
overall was high (4.29) indicating that students were neither bored by
nor overloaded with information from the show. There were no
statistically significant effects of this variable. Clearly, low resolution
did not lead to greater levels of boredom or information overload.
Interest in the Topic Prior to Attending the Show

This variable of Prior Interest in the topic aggregated reports of prior
interest in the topic and specific interest in the fate of sea lions or
dinosaurs, respectively. The rating was relatively low (2.4) indicating
that the students disagreed with the statements about prior interest,
which were presented in the negative. Nevertheless, translated into the
positive, the values would still have been low (2.6), but the values
given by the students, may have resulted from poor encoding of the
questions put into the negative. There were several significant effects.
Grade 5 students expressed lower values than Grade 9 students (2.17
vs. 2.60) indicating their greater prior interest, F (1, 23) = 21.03, p <
.001. There was greater prior interest in Sea Lions (2.29) than
Dinosaurs (2.53), F (1, 23) = 5.84, p < .05, which is not consistent
with the popularity of dinosaurs over sea lions (consult the toy
department and bookstore and much more can be found on dinosaurs
than sea lions). The difference between the shows was particular to
females who showed much less prior interest in the Dinosaur show
than the Sea Lion show (2.65 vs. 2.23), leading to a significant Gender
x Show effect, F (1, 23) = 6.59, p < .05. Prior interest was also
unevenly distributed across the combinations of Resolution, Grade and
Show, F (2, 23) = 3.97, p < .05, which must be attributed to
unfortunate sampling. For example, the Low Resolution Grade 9
group had a mean Prior Interest of 3.1 for Dinosaurs, whereas the
Medium Resolution Grade 5 group had a mean Prior Interest of 1.73
for Sea Lions.

Motivation
The Motivation variable measured the desire during the show to
investigate the problem of the sea lions or the dinosaurs, respectively
(e.g., why the dinosaurs became extinct). Grade 5 children (3.96) were
more motivated than were Grade 9 students (3.56), F = 22.02, p <
.001. Motivation was greater for the Sea Lion than the Dinosaur show
(3.98 vs. 3.48), F = 18.9, p < .001. The effect of show however was
strongest for Grade 9 for whom Sea Lions led to a much higher
Motivation (3.91) than did Dinosaurs (3.11). Comparable values for
Grade 5 were 4.07 and 3.86 respectively, F (1, 23) = 7.3, p < .01.
There was a complex interaction with Resolution, F (2, 23) = 3.10, p <
.05, with Grade 9 students increasing motivation with Resolution on
the Dinosaur show, due largely to a low value on the low resolution
condition (2.8). A more modest incline was shown for Grade 5
students on the Sea Lion show. Since the effect is not found for the
other two grade x show conditions, there is only very modest cause to
suggest that higher resolution increases interest in the scientific issues
of the immersion show. Finally, Gender also played a role in
Motivation in interaction with the show. Females were relatively more
motivated for the Sea Lion show than were males, though equally
motivated for the Dinosaurs as were the males, F (1, 23) = 4.48, p <
.05.
Negative Emotions
Negative Emotions measured anger and upset during the show. It
might have been expected that if the Resolution variable were to have
an impact, it would be here. One has only to recall interruptions and
poor transmission on television or broadcasts in an airplane to
appreciate this possibility. There were in fact several significant
effects observed. Sea Lions led to more negative emotions than
Dinosaurs (although whether this had to do with empathy for sea lions
as compared to problems with the transmission is not clear). The mean
negative emotion score for Sea Lions was 1.74 and for Dinosaurs was
1.30, F (1, 23) = 24.05, p < .001.For the Dinosaur show, the degree
of Negative Emotions decreased with degree of increasing
Resolution (1.45, 1.23, and 1.16 for increasing Resolution). This is
as would be predicted. The same pattern was not obtained for Sea
Lions (1.58, 1.92, and 1.61), consistent with the reduced validity of the
resolution variable here. The interaction between Show and
Resolution was significant, F (2, 23) = 3.79, p < .05. There was also
an interaction with Gender and Show. Females became relatively more
upset with the Sea Lion show (1.23, vs. 1.89), possibly indicative of
the emotional content of the narrative (compare males 1.37 vs. 1.56), F

(1,23) = 7.47, p < .01. Grade, Gender, and Show was also significant,
with the effect for females being most pronounced in Grade 9 (1.21 vs.
1.95) as compared to males (1.49 vs. 1.47) who show no difference at
this grade, F (1, 23) = 4.00, p < .05.
Usability
Usability referred to the ease of figuring out how to play the game and
in knowing what to do during the show. The overall Usability was
high (3.97). Surprisingly, the older and younger students fared equally
well, however, the Grade 5 students found the Dinosaur show
relatively more easy than the Sea Lion (3.92 vs. 4.06) show and the
Grade 9 students found the opposite (4.14 vs. 3.72 for Grade 9 Sea
Lions vs. Dinosaurs), F (1, 23) = 12.14, p < .001.
Interplay
Related to Usability was the Interplay factor which measured the
confusion regarding when to look at the big and small screen, and the
degree of distraction created by the two screens. Males reported less
confusion (1.98) than females (2.36), F (1, 23) = 15.58, p < .001. This
is consistent with the literature on gender and spatial ability and it
would also be consistent with the female willingness to admit
difficulties in spatial tasks. Whether there is a real gender difference
here bears further investigation.
Learn
The Learn factor aggregated the self-assessed amount of learning
during the show including the learning of specific important
information about sea lions or dinosaurs respectively. The overall
mean amount of Learning was high at 4.20. Not surprisingly, Grade 5
students reported higher Learning than Grade 9 (4.42 vs. 4.20), F (1,
23) = 32.31, p < .001. Females reported more learning than males
(4.27 vs. 4.11), F (3.85, p < .05. There was more Learning reported for
the Sea Lion show than the Dinosaur show (4.36 vs. 4.01), F = 20. 31,
p < .001, although this was mostly attributed to the Grade 9 students
who felt they learned much less in the Dinosaur show (3.62 vs. 4.31
for Grade 9 students, and 4.40 and 4.43 for Grade 5 students), F (1,
23) = 13.42, p < .001). Finally, females overall felt that they learned
more in the Sea Lion show (4.00 vs. 4.5 for females and 4.02 vs. 4.20
for males), F (1, 23) = 6.00, p < .05. Resolution however played no
role in the self assessed amount learned in the show. It might have
been the case that the highest resolution would have led to the highest
judged assessed amount of learning.

Recommend
The Recommend variable reflected the interest of the participant in
recommending the Immersion experience to a friend and the wish of
the participant to be provided similar learning experiences by the
teacher. The overall rating was high (4.38). The mean values
increased with the degree of Resolution (4.25, 4.42, and 4.49). In
other words, there was a greater tendency to recommend the
experience with increasing quality of the presentation. However
the effect was not large, but it was significant, F (2, 23) = 3.13, p <
.05. Moreover, this effect was confined to the Dinosaur show (4.22,
4.32, and 4.62 for Dinosaurs vs. 4.29, 4.52, and 4.36 for Sea Lions).
The interaction of Resolution and Show only approached significance,
F (2, 23) = 2.83, p < .06 and statistically does not rule out attention to
the effect of Resolution on the Recommend reaction.
The younger Grade had a significantly higher value (4.49) than the
older Grade (4.28), F (1, 23) = 8.19, p < .004. Finally, the females had
higher Recommend responses for the Sea Lion show (4.50 vs. 4.33) as
compared to males (4.24 vs. 4.40), F = 5.75, p < .05, though it is
difficult to interpret this partiality.
Wish to play the show again

This variable represented this single question, and of the 15 dependent
measures was based on one question only. The wish to play the show
again was greater for Grade 5 (4.62) than for Grade 9 (4.30), F (1, 23)
= 13.74, p < .001. For this Recommend variable two interactions with
Resolution approached the conventional level of statistical
significance and therefore are mentioned here, bearing in mind the
validity of Resolution only for the Dinosaur show. The Grade 5
students wanted to play the show more when they had seen the
highest resolution version, and for both grades, the lower
resolution led to the least desire to play the show again, F (2, 23) =
2.89; p < .056. These two variables further interacted with gender,
such that females in Grade 5 show the linear effect of resolution (4.39,
4.72, 4.98), but the pattern is not found in the other three groups, so
again it is difficult to make a strong case for the effect of Resolution
here, F (2, 23) = 2.97, p < .052.
Increased Interest during the Show

Of related importance to that of Prior Interest is the Increase in Interest
during the Show. This variable aggregated the responses concerning
the wish to learn more about science and to learn more specifically

about the topic of the show (be it survival of sea lions or dinosaurs,
respectively).
The mean Increased Interest was relatively high (3.61). Not
surprisingly, the Increased Interest was higher for Grade 5 (3.88) than
for Grade 9 (3.37), since the Grade 5 students have more to learn than
Grade 9 students, F (1, 23) = 29.86, p < .001. The effect was stronger
for Dinosaurs (3.94 vs. 3.16) than for Sea Lions (3.81 vs. 3.54)
suggesting that the Dinosaur show may lead to lower self-assessed
learning for the older than younger students more so than the Sea Lion
show. The Resolution variable also interacted with the grade and
gender, F (2, 23) = 3.84; but no particular pattern is evident other than
for Grade 5, there is a small systematic increase with Resolution.
Sensation Seeking
This variable represented an interest in loud music and noise. Grade 5
children sought this sensation significantly less than Grade 9 students
(3.75 vs. 4.11) F (1,23) = 13.26, p < .001, and those in the Dinosaur
show had a significantly higher value (4.13) than those who saw Sea
Lion show (3.78), F (1, 23) = 13.62, p < .001. In view of the fact that
the difference due to show can result only from unfortunate sampling
and is of the same order of magnitude as that for the difference
between grades, it is unclear what to make of the difference between
grades. The direction of this difference is consistent with that
associated with the recent literature on adolescence which
characterizes adolescents as sensation seeking. The variable did not
interact with any others in the analysis.
Achievement
This variable was based on two self-assessments of excellence as a
student and grades. Ritterfeld et al. (2003) had suggested that poorer
students benefitted most from the immersive educational experience,
again based on self-assessed achievement. It was therefore possible
that better students would be less affected by the resolution variable
than the poorer students would. If the best students by chance ended
up in the high resolution condition, the chance of revealing effects of
high resolution would be reduced (i.e., good students could learn by
and handle any presentation condition). It did so happen that those
students with highest self-assessed achievement were significantly
more prevalent in the two highest categories of Resolution. Mean
Achievement for Low Resolution was 3.067 (.08 SE), for Medium
Resolution was 3.31 (.08), and for High Resolution was 3.40 (.09).
The Resolution factor was statistically significant, F (2, 23) = 4.41,

p < .05, and was attributable to the difference between the low and
high resolution groups (Tukey HSD = .33, p < .01).
The effect interacted with Show because for Dinosaurs, the group that
received the Medium resolution rated themselves higher than any of
the remaining 5 groups. Nevertheless, for both shows, both groups
receiving the Low Resolution rated themselves lower than both groups
receiving the High Resolution, F (2, 23) = 3.07, p < .05.
This distribution of achievement decreases the chances of observing
an effect of immersion because those most likely to benefit from high
resolution (the lower achievers) were not well represented in the high
resolution condition and were disproportionately distributed in the low
resolution condition. Thus, if effects of resolution do emerge, they are
likely to be strong effects that counteract this achievement variable.
The argument is based on the validity of Ritterfeld et al.’s conclusion
that the Immersion experience benefits the poorer students.
Mean Achievement was statistically higher for the Grade 5 students
than for the Grade 9 (3.41 vs. 3.11), most likely due to a greater
concern for grades with increasing grade level, F (1,23) = 10.32, p <
.001.
Media Competence
Media Competence referred to frequency of playing computer games
and surfing the Internet. The overall level was relatively high at 3.85.
Here it would be expected that older students and males would give
higher values than younger students and females and other effects
would be the result of unfortunate sampling. Indeed, Media
Competence of Grade 9 students exceeded that of Grade 5 (3.99 vs.
3.69), F (1, 23) = 7.07, p < .01, and males (4.0) exceeded females
(3.73), F = 6.17, p < .05. The gender variable also interacted with
show, with only a negligible difference between males and females in
the Sea Lion show as compared to the Dinosaurs show, F = 4.04, p <
.05.
Correlations among the 15 dependent measures

The above analyses examine each dependent measure separately. It is
also of interest to know the extent to which the various dependent
measures are correlated. To this end a correlation matrix was produced
separately for the two shows. High correlations (Pearson 2-tailed)
were obtained between Presence and Interest in Learning More for
both of the shows (.556 for Sea Lions and .504 for Dinosaurs). Since a
characteristic of the Immersion experience is Presence (where

Presence is to some extent synonymous with Immersion), this

connection with interest in learning speaks highly for the value of the
Immersion Experience for learning. Multiple regressions were also
carried out as had been conducted by Ritterfeld et al. (2003) and the
results were similar. Complete details of these results are available
from the authors.
The correlation between the 15 dependent measures for the two shows
was .98 and is plotted in the next figure. It is clear that the two shows,
though different in content, lead to the same pattern of appraisal and
impression. This is one of the most important outcomes of the study.


Short term impact of the show from the

teacher’s perspective
Each class was typically accompanied by its teacher. The teacher was
asked to complete a questionnaire immediately after the show. The
questionnaire contained some items similar to those to be completed
by the students. For example they were asked whether they agreed or
disagreed that most of the students had enjoyed the show, or had had
problems understanding the games. Several teachers did not complete
the questionnaire because they did not stay in the room during the
presentation. For three classes, the Principal of the high school
attended while the teacher stayed back at the school. In this case, the
Principal completed only one questionnaire. This left 27 teachers who
completed the questionnaire. The extra numbers arose from the control
groups who also had the opportunity to experience the Immersion
studio, although only after they had completed questionnaires 1
(demographic) and 3 (yes-no content) in their classroom.
Value of the show

The questionnaire administered to the teachers was identical to that
used by Ritterfeld et al. (2003) for Sea Lions. The means for both
groups are shown in Table 5. It can be seen that the means are quite
compatible. For example, teachers from the present study did vote for
the interplay between the big screen story with the interactive console
activities as the most valuable contribution of the show (M = 4.21) as
compared to the narrative aspect (M = 3.16) and the games (M =
3.08). These results are comparable to that of the previous study (4.07,
3.58, and 3.29 respectively). A Pearson correlation carried out on the
two sets of data was significant (.74, p < .0001) and warrants the
generalization of the interpretation of the teacher’s data from the past
study to the present one.
Assessing the entertaining impact of the show

The teachers reported that the students enjoyed the show very much
(M = 4.56, as compared to 4.52 in the previous study), and reported
the student’s involvement in the whole game playing (M = 4.17,
compare 4.2).
Identifying which children are most and least

successful.
Previous results with teachers suggested that “the entertainment
experience can overcome motivational deficits and focus attention, but

does not compensate for cognitive limitations.” Similar results were

obtained here.
Table 5
Teacher’s Assessment (N = 25) Charlottetown and Comparison Data From USA Study
Charlottetown Teachers USA Teachers
N Mean S. D. N Mean S.D.
Most of the students really enjoyed 25 4.56 0.65 31 4.52 0.89

the show.
I observed that many of the
students seemed to have problems 24 2.96 1.08 31 1.94 1.093
understanding the games.
I think that this kind of
entertainment has little educational 25 4.04 1.06 31 4.55 0.961
value.
Most of the students were confused
about when to look at the big 25 3.96 0.98 31 4.1 0.908
screen and when to interact with the
consoles.
I think the story presented on the
big screen was the most valuable 25 3.16 0.94 31 3.58 0.848
part of the show.
This show has a high educational 25 3.88 1.01 31 4.32 1.013

value.
This show is especially good for 25 3.68 0.99 31 3.68 1.137
students with little motivation.
I think the games presented on the

consoles were the most valuable 25 3.08 0.91 31 3.29 1.071
part of the show.

I imagine that the show will motivate

students to undertake scientific 25 3.36 0.99 31 3.74 0.999
projects.
The interactive component of the

show made the kids become more 25 4.44 0.71 31 4.55 0.961
involved.
To be honest, reading a book about
the topic would have been of equal 25 3.64 1.19 31 3.71 1.131
educational value to the students.
I believe the show to be very 25 4.08 1.12 30 4.23 1.104

unique.
I do not believe that entertaining

media in general can contribute in 25 4.56 0.71 30 4.53 0.9
any way to education.
The show was too challenging for 25 3.56 1.12 31 4.26 0.729
most of the kids.
I would like to see more shows of

this type integrated in the 25 3.72 1.1 31 4.39 0.989
curriculum.
I am afraid only very smart kids

could follow the game. 25 3.40 1.12 31 4.13 0.846
This show doesn’t fit into the

curriculum at all. 25 3.84 1.25 31 4.58 0.886
I discovered that most of the kids

seemed to have fun during the 24 4.33 0.82 31 4.48 0.926
show.

I am not convinced that a student

will be any more interested in 25 3.08 1 31 3.77 1.117
science after the show.
I observed that the kids were
confused about what they were 25 3.20 1 31 4.1 0.831
expected to do.
The interactive component of the

show allowed students to be 25 4.4 0.87 31 3.87 1.176
attentive for the duration of the
show.
I believe that the social interaction

required during the show made for 25 3.48 1.23 31 3.9 1.076
a unique and valuable educational
tool.
Considering the amount of time and
money spent to produce this kind of 25 3.64 1.08 31 4.26 1.064
show, I prefer to stick to traditional
teaching methods.
I did not see anything really new or

original about the show. 25 4.28 0.89 31 4.39 1.022
The students seemed to be 25 4.04 0.73 30 4.1 0.662

captivated by the story.
The educational value of this show 25 3.92 0.91 30 4.57 0.728
is negligible.
I had the impression the kids were

really involved in the whole game. 24 4.17 0.82 30 4.2 1.064

I got the impression that the

combining of a big screen story with 24
4.21 0.78 29 4.07 1.067
interactive console activities was
the most valuable characteristic of
the show.
The social character of the show
(being part of a research team,
interacting with others, and working
on a mutual goal) increased its 24 3.67 1.05 30 4.3 1.088
educational value.
I am afraid that slower students will

not be able to complete the 24 2.54 1.06 30 2.3 1.022
activities in the games.

The long term impact of the show from the

student's perspective
A week after the show, the students completed a 40-item
questionnaire of which 8 of the items assessed their general
impressions of the Immersion Educational experience, 16 of the items
tested knowledge presented during the show, and 16 items tested
knowledge about the topic that was not presented during the show. A
control group also received the 32 items specific to the topic of the
show, however, for them this was entirely a test of general knowledge.
There were separate 32-item tests for the Sea Lion and Dinosaur
shows. The test for Sea Lions was the same as had been used by
Ritterfeld et al. (2003).
The long-term impact questionnaire was administered one week after
the original immersion experience in the present study, but in the
Ritterfeld et al. (2003) study, the delay was one month. From the
Table, it is clear that the Immersion experience had a very positive
impact on the Canadian students from both a general as well as
scientific attitude perspective.

Table 6
Follow up Appreciation of the Students for Sea Lions
Comparison data from Ritterfeld et al. (2003) Included
Sea Lions Charlottetown Sea Lions USA
Delay of 1 week Delay of 1 month
(Ritterfeld et al., 2003)
Valid N % Agree Valid N % Agree
I would love to play the game again 270 96.7 126 85
I learned quite a bit during the show 267 86.9 121 77
The show helped me to understand how 270 80.4 125 70
scientists work
I talked to friends or relatives about the 270 79.7 126 66
show
The show encouraged me to become 268 67.2 124 61
more interested in science
I still have vivid memories of the show. 263 81.7 126 59
I want to learn more about El Nino 266 60.5 126 52
The show helped me become more 267 55.8 124 42
interested in biology
The show encouraged me to become 267 48.7 125 35
more interested in geography

Mean performance on the factual items for Sea Lions for both
experimental and control groups was low at 58%. Chance performance
is 50%. Had many facts been conveyed during the show, the score of
the experimental group would have been much higher. Mean
performance for both of the groups tested on Dinosaur information
was higher at 68% and 65% for experimental and control group
respectively, but this increase over the Sea Lion study reflects general
knowledge or ease of the test. The difference between experimental
and control groups is negligible and the difference between means was
not significant in a t-test. The r2 on the 32 items for both Sea Lions
and Dinosaurs was moderately high again suggesting that participants’
were relying on general knowledge rather than facts acquired during
the test.
More detailed analysis of the questionnaire in terms of those items for
which knowledge had been presented during the test may provide
useful data, however, the overview here suggests that the educational
benefit of the show is not in the communication of facts that can be
tested through yes-no questions. The more detailed analysis reveals
that for several questions, exposure to the show led to much higher
scores, but this was not always the case, and indeed some scores were
lower with exposure to the show, though not to the same extent.
Ritterfeld et al. (2003) also reported low scores on these questionnaire
items. They found that control questions led to higher responses and
argued that the students had been so inspired by the Sea Lion show
that they had actually sought out additional information in the
intervening month between the show and the questionnaire. This
explanation seems less likely given the high correlation between the
control group and the test group in the present study. The breakdown
for facts contained in the show and those not contained in the show
further supports this claim. At first glance the low scores on the test
for those who have seen the shows challenges the view that much was
learned through the show. There are many kinds of learning and the
kind tapped by the yes-no questionnaire is not that which is acquired
through the immersion experience. Learning how to do something,
broadening a perspective and changing an attitude cannot be easily
tested this way.
Performance for the experimental and control participants for both
shows is shown in the figures.


Dinosaurs

Long Term Impact from the Teacher’s

Perspective
There were18 teachers who completed the second questionnaire as
compared to only 7 in the study by Ritterfeld et al. (2003). The
teachers typically focused on the technology of the presentation rather
than details of content. They had quite systematic judgments about
who would benefit most from the Immersive experience. Their
judgments were spread across those with an interest in science as well
as those less motivated. This pattern is consistent with the teachers of
the USA study.

Table 7a
Teacher’s Report One Week After Sea Lions (N = 8)
Discussion Topic
Class Discussion Topic After the Show Per Cent Responded Yes
Technology of the show 75
Aesthetics of the show 75
Causes for the sea lion population decline 25
How to solve scientific problems 12.5
Global warming 37.5
Pollution to the oceans 25
Commercial fishing 12.5
Food chain 0
Table 7b
Teacher’s Report One Week After Dinosaurs (N = 10)
Discussion Topic
Class Discussion Topic After the Show Per Cent Responded Yes
Technology of the show 70
Aesthetics of the show 40
Extinction 30
Animal Adaptation 10
Dinosaurs 40
Pangea 10
Earths’ Periods 0
k/t event 30
Meteor Impact 10
Nothing related to the show 9.1

Table 8
Teacher’s Judgment About Suitability of Immersion Shows (Sea Lions and Dinosaurs) for
Particular Student Groups One Week Later. Question stated: For which students do you
believe the show was especially valuable?
Sea Lions N = 8 Dinosaurs N = 10

Gifted students 75 40
Students who have difficulty concentrating 50 80
Students interested in science 62.5 70
Students who are difficult to motivate 62.5 70
Students with learning disabilities 0 40
Hyperactive students 12.5 30
No specific group of students 0 20

Conclusion
The effect of resolution, though modestly apparent from a statistical
standpoint particularly for the Dinosaur show, was for all practical
purposes negligible. To review Enjoyment increased modestly with
Resolution for Dinosaurs (from 4.47 to 4.73) but showed the opposite
pattern (but more weakly, 4.61 to 4.43) for Sea Lions leading to a
significant interaction of Show with Resolution. Similarly, Negative
Emotions decreased with increasing Resolution for Dinosaurs (from
1.45 to 1.16) leading to a significant interaction with Show and
Resolution. This effect was retained in the analysis of variance for
Dinosaurs alone. There was one significant main effect of Resolution
and that was for the willingness to Recommend the show (increase
from 4.25 to 4.49 with increasing Resolution) but again the effect
interacted with Show and was most prominent for Dinosaurs (range of
.40 as compared to range of .13 for Sea Lions). The Grade 5 students
wanted to play the show more for the highest resolution, and for both
Grades, the lowest resolution led to the least desire to play the show
again. As a potential confounding variable, it so happened that the
highest self-assessed achievement was disproportionately found in the
students who received the highest Resolution condition. If high
achievers are least likely to benefit from high Resolution, this could
account for weaker effects of Resolution than would otherwise have
appeared. Nevertheless, additional analysis revealed that achievement
was not significantly correlated with any other of the 14 dependent
measures, so its confounding effect, if any, is negligible.
That resolution of the presentation a negligible effect on the many
variables examined in the present study may well be consistent with
several theories of multimedia perception and learning. As described
by Ritterfeld et al. (2003), several theorists have also pointed to the
importance of visual imagery in the absorption of an audience into a
narrative (e.g., Green & Brock, 2003). Immersion into the narrative is
referred to as transportation, and absorption of the narrative is referred
to as transformation. Together transportation and transformation lead
to learning. Again, the extent to which visual resolution influences
absorption into a narrative or absorption of the narrative may have less
impact than many other variables that comprise the multimedia
material presented. From another perspective, the Congruence-
Associationist theory proposes that multimedia perception elicits the
creation of a working visual narrative (Cohen, 2001, in press) in an
audience member who aims to create a story to explain the sensory
data received. The establishment of such a narrative depends on
external cues from the media presentation that prompt the cognitive

hypotheses about their meaning. The best match between the

hypotheses and the received media create the working narrative. In
such a theory, the effectiveness of visual cues to the generation of the
working visual narrative could be quite independent of their degree of
resolution, assuming a basic level of resolution has been achieved.
Information obtained from the present study in regard to the resolution
variable may serve then to advance the accessibility of immersive
environments in schools able to exploit the broadband network for
Education Entertainment. In other words, it appears from the present
work, that moderate levels of resolution for a large screen presentation
remain effective in conveying the benefits of Education Entertainment.
Thus, delivery from a central site, such as that of Immersion Studios,
to many large screens, not simply to purpose-built Immersion Studios,
could offer the advantages of Education Entertainment to an infinitely
greater audience, worldwide. Continuing studies of the effects of
resolution remain important, as the limits of the effects of this variable
need to be determined. In future work, it is necessary to provide
independent measures of visual clarity by the participants. The impact
of audio resolution and immersive effects (surround panoramic versus
stereo or mono) need also to be determined.
In regard to other results of the study, effects of gender, grade and
patterns among the many dependent measures such as enjoyment were
highly similar for the two different shows despite differences in
content. Students’ high ratings of enjoyment, presence, and acquired
knowledge generally confirmed results of a prior study of the sea lion
show (Ritterfeld, Weber, Fernandes, Vorderer, 2003). Enjoyment,
interest in learning more, and presence were highly correlated for both
shows, and in general the two different shows led to the same pattern
of appraisal. Girls responded significantly more positively to the
presentation than did boys, and grade 5 students responded
significantly more positively than did grade 9 students, but these
differences were small from a practical standpoint. Teachers who had
accompanied their students also rated the experience positively both
immediately and after the one week delay. They believed that students
benefited most from the combined large screen and interactive
component, that all students enjoyed the experience, and that even
students with special needs would benefit from it.
Although self-assessed and teacher-assessed student-learning was
high, learning of specific facts was not well-reflected in the results of
the yes-no questionnaire when compared with control groups who had
not experienced the presentation. Similar findings were also observed
by Ritterfeld et al. (2003) suggesting that the educational value of the
presentation lies not in acquisition of new facts but rather in more

global aspects such as positive attitudes toward educational

experiences.
The overall validity of the findings is supported by the high
correlations between the results for Sea Lions and Dinosaurs in the
present study and the correlations between the Canadian and US study.
Since the individual results provide evidence for the achievement of
the goals of the Entertainment Education experience that uniquely
employed a large interactive screen and interactive games at individual
consoles, it can be concluded that the principles used by the
Immersion Developers for creation of e-learning content may well
provide an excellent beginning for an Interactive Content Developers’
Guide.
Acknowledgment
The project was funded under a contract from Immersion Studios Inc
and CANARIE Assistance. Cooperation of Rob Krieger and Dr.
Rodney Hoinkes of Immersion Studios Inc is gratefully appreciated.
As well, at the Atlantic Technology Center, the assistance and
cooperation of John Hughes and Christina Fontaine are acknowledged.
In regard to the contributions of the individual contributors to the
present document: Sandy MacAulay was the primary contract
negotiator during the formative stages of the project. Annabel Cohen
supervised all aspects of the research and is the primary author of the
report and included full consultation with research assistants on the
project Betty Bailey, Kelti MacMillan and Jennifer Ghiz who each
read and commented on an earlier draft. All three worked in
remarkable coherent collaboration so as to complete the project within
a reduced time line that arose for reasons beyond the control of the
team. Kelti MacMillan and Jennifer Ghiz took the primary
responsibility in testing the participants, with Kelti MacMillan
assuming the role of project manager for this aspect. Betty Bailey took
the primary responsibility in writing the ethics reviews and handling
their revision. She also assisted with testing and played a major role in
data entry and analysis. Jennifer Ghiz also assisted with data entry and
analysis. Kelti Macmillan managed the complex organization of the
data collection from the schools. The cooperation of the
administrators, principals, teachers, and students of the Eastern School
District of Prince Edward Island, and the teachers from the Glen
Steward, Spring Park, Sherwood, St. Jeans, West Royalty, West Kent,
Birchwood, East Wiltshire, Stone Park, and Queen Charlotte Schools
is gratefully acknowledged.

The team will submit an abbreviated paper to the New Media

Research Networks Conference which will take place March 27, 2004
and for which short proceedings papers will be published.
References
Cohen, A.J. (2001). Music as a source of emotion in film. In P. Juslin
& J. Sloboda (Eds). Music and emotion (pp. 249 - 272). Oxford:
Oxford University Press.
Green, M. C. & Brock, T. C. (2003). In the mind’s eye:
Transportation-imagery model of narrative persuasion. In M. C. Green
et al., (Eds). Narrative impact: Social and cognitive foundations (pp.
315-342). Hillsdale, NJ: Erlbaum.
Ritterfeld, U., Weber, R., Fernandes, S., Vorderer, P. (2003). Think
Science! Entertainment Education in interactive theatres. Unpublished
paper. University of Southern California.
Appendices (not included here)

A. Preliminary questionnaire completed by experimental and control
participants.
B. Student’s Short term impact questionnaire: (a) Dinosaurs (b) Sea
Lions
C. Teacher’s Short term impact questionnaire:
D. Student’s Long term impact questionnaire: (a) Dinosaurs (b) Sea
Lions
E. Teacher’s Long term impact questionnaire
F. Control participants’ questionnaire: (a) Sea Lions and (b)
Dinosaurs

Appendix B
ISL Evaluation
Interactive Content Development Guide Appendix B
Executive Summary
Evaluations to assess the perceived advantages and disadvantages of a
single-screen version of Poetic Dimensions software compared with
the existing immersive, three-screen version were conducted at the
University of Toronto and the University of Waterloo respectively.
The intent of developing a single-screen version is to make it available
over a high-speed Internet connection. Since the single-screen version
of Poetic Dimensions is in its formative stages and the demo was a
simulation running on a stand-alone PC, the evaluations used a focus
group methodology. The groups were shown both versions of the
software and asked to comment on the perceived differences, the
advantages, and the relative merits of each in assisting users to make
informed decisions in a planning exercise.
Additionally, the groups were asked to assess the importance of group
dynamics in the immersive version and how a distributed format
would affect the decision-making process.
The initial intent of the evaluators was to have at least twenty
participants in each group, but it was extremely difficult to find expert
subjects who had sufficient experience with the software and could be
available for 2 ½ hours. Most of the participants had extremely
demanding schedules, and we would like to acknowledge their
generosity in making time available for this project. The evaluations
were conducted on October 30 and November 26 respectively with
seventeen participants. The first group had considerable experience
with the immersive software, having used it in the spring of 2003 to
develop a position on the impact of proposed development on the
Lakeshore on Fort York, in Toronto. The second group of participants
was primarily new to both versions.
The evaluations went well with excellent participation and valuable
input from both groups. It should be noted, however, that the groups
were quite different, with the Waterloo group being more focused on
how the software could assist them in their daily planning activities.
As such, they were more forthcoming on questions pertaining to their
perceptions of the existing software than on discussing the potential
advantages and/or drawbacks of a broadband application.
Notwithstanding, the following major conclusions were reached:
The immersive, interactive, three-screen version of the
software has the biggest “wow” factor, but the single-screen
version has the benefit of widespread dissemination and
accessibility (fewer time and space constraints).
© 2004 Immersion Studios Inc B-1

The software in all versions is a compelling and useful tool

both for expert users (planners, architects, developers, etc.) and
for those who lack the ability to think in abstract three-
dimensional form.
Participants valued highly the group dynamics promoted by the
three-screen immersive Poetic Dimensions, and to a lesser
extent, a projected single-screen version, but overall believed
that the same level of decision-making could be achieved with
the single-user, single-screen version, although not necessarily
in the same timeframe.
A majority of participants welcomed the potential for the
broadband, single-screen software to democratize planning by
providing an avenue for community exploration and input into
planning proposals.
Several participants referred to the advantages of enhancing
information available through the environment by means of an
underlying database. In this way, users could access additional
information to help them make informed decisions. Planning
proposals, earlier rulings, site maps, graphics and statistical
information are examples cited.
A majority of participants expressed the opinion that the most
powerful aspect of the software in any format was the ability to
change/manipulate/substitute elements of the environment, to
better understand the relationship between elements and their
context. The ability to make changes on the spot was seen as
enabling deeper, more critical discussion of alternatives, since
iterations on paper documents or models were difficult and
time-consuming. Much of the discussion, particularly with the
second group, was dominated by specifics of usability, display
and desired features. This information should guide further
development.
Participants, particularly planners, described the potential for
progressively sophisticated versions of the environments
ranging from simple massing models for early decision-making
or visioning through to versions rich in detail and texture for
public presentation.
Some participants could not distinguish between the
information provided by the single-screen and three-screen
versions of the software. They believed that that they were
seeing the same information compressed to fit in a single

screen. Further research would be required to explore these

perceptual issues.
A majority of participants expressed the view, some very
fervently, that there is a strong need for software such as this to
involve the community in planning discussions and
recommended further research and development.

Purpose
The purpose of the evaluation was to explore the perceptual issues
around the delivery of high-end immersive and group-interactive
experiences over next-generation broadband networks (as exemplified
by CA*Net 3 and 4). To undertake the evaluation, Immersion Studios
developed a single-screen version of their Poetic Dimensions software
to simulate broadband delivery. The evaluation was designed to
encourage participants to articulate their perceptions of how their
ability to identify issues and to make and share decisions was affected
by the transition from a three-screen 135-degree immersive
environment to a single-screen version of the same environment. It
was hoped that feedback from the evaluation would determine the
feasibility of and critical next steps in the development of a
commercially available broadband application.

Methodology
Two groups were invited to participate in a 2 ½ hour session that
provided the opportunity to compare a 3-screen immersive version of
the software in a group setting with a single-screen version. Every
attempt was made to ensure that the functionality of the software was
similar in each version, certainly to the extent that the group could
explore the environment from multiple perspectives, instruct the
operator to move about the site from both aerial and eye-level
perspectives and add and/or remove elements of the environment in
order to facilitate discussion and ideation. The evaluator and the
operator had pre-determined the “path” that would be taken through
the environment, so that it could be replicated in each version. Each
demonstration began with an overview of the project, a brief
explanation of the purpose of the evaluation, and the guidelines for
responding to questions and probes. The duration of the focus group
for each version was approximately 40 minutes, followed by a
brainstorming session of twenty to thirty minutes designed to explore
potential uses for the broadband delivered version of the software.
Questions were prepared in advance of the sessions. While set
questions and probes had been identified, the intent was to allow the
group as much freedom as possible within the time constraints to
express their opinions in a free-flowing, spontaneous fashion. The
evaluator asked and was granted permission to tape the sessions.
It should be noted that the participants were not able to run the
software themselves or to communicate remotely through some form
of communication tool.
Group 1
The first focus group session was conducted at the Centre for
Landscape Research at The Faculty of Architecture, Landscape and
Design at the University of Toronto. The invitees were part of a group
that had worked with Professor John Danahy at the Centre in the
spring of 2003. The group was composed of city employees from the
planning, urban design and mapping departments (4) and community
activists from the Friends of Fort York group (5), who had come
together to explore the impact of proposed high-rise waterfront
development on Fort York. As such, the group was quite familiar with
the three-screen immersive version of the software. The lab at the
Centre for Landscape Research is not very large, but all nine
participants were seated and able to see the screens clearly, although
they were not ideally seated to interact with one another.

Description of Session
The session began at 6:40 PM with an introduction from Rodney
Hoinkes, Chief Technology Officer at Immersion Studios, who
explained that initially the computing power required to create the
environments was so great that expensive, dedicated systems were
needed. With recent improvements in the power of desk top and lap
top systems and the potential for ubiquitous high speed access, the
ultimate goal for Immersion Studios is to have a version of their
software that can run on a home computer, with users sharing files
over the Internet.
As John began the walkthrough, reminding the participants of the
issues they had met to discuss in the spring, the evaluator initiated a
short discussion on how the issues would have been approached prior
to the availability of the immersive environment. Respondents
indicated that paper documents and foam core models would have
provided the main points of reference for discussion. When prompted
to describe how the immersive software differed from these traditional
references, the group offered the following:
A foam model is tougher. You can’t look inside it. There is no context.
Size is an issue. The model shows a view seen in no other view – it
isn’t real.
It’s difficult to lug a model around.
A drawing captures a moment in time and place, as interpreted by the
artist. The point of view is dictated by the elevation of the drawing.
There is no sense of how the model/drawing relates to anything else in
the environment.
I have no confidence in a paper model.
It is very difficult to make a case with a model.
Specific comments on the advantages of the immersive software over
traditional methods were:
You could see things from eye level, as though you were actually in the
environment.
You are aware of surrounding elements. You can ask to have a closer
look or walk around a building and see it from all angles.
John could add/remove storeys, move buildings closer or farther
away, so that you could see the impact immediately. You didn’t need to
wait for someone to re-render the drawings.
You could brainstorm solutions and see them right away.

You can respond to any question immediately.

We knew that we were likely to be in contentious negotiations. We had
a substantially better strategy to go into those negotiations, by being
able to try out various approaches and practice countering opposition.
We weren’t happy to make decisions without getting our strategy in
place.
Some disadvantages were also noted:
Not having the shadows showing.
It’s a massing model. It’s not realistic enough.
Massing envelopes look scarier than they were.
There’s a sense of madness from the density of the lines.
More realistic models are available now, with more detail.
This led us into a discussion of the need for photo-realism in terms of
the ability to make decisions. Is detail an essential to informed
decision-making or a distracter? The group had mixed opinions.
It depends on the audience whether you need realism or not. Without
detail, there is a limited ability to deliver accurate information.
Decision-making will be better if there is not too much detail. Perhaps
you need to add detail in stages.
It’s very important to have smooth movement through the
environment. If it’s choppy, it doesn’t feel real.
The general community might be skeptical of an environment that does
not contain a recognizable level of detail – trees, bricks, grass, etc.
They’re not trained to understand massing models.
If you can’t think in abstract shapes, it might be more difficult. The
trained mind can make accommodations; people with an architectural
background would be comfortable with this kind of thinking.
Maybe the way to go is to begin with a version that uses wire-frame
and massing models just to get the general context and make the big
decisions, get general agreement, then add detail as you get to more
specifics.
Participants seemed to be in agreement that there was room for a
progression in the level of detail required in an environment. They felt
that a fairly stripped down version would be effective at the consensus
building stage of a discussion. Once general agreement had been
reached, they felt it was appropriate to add more detail to the
environment, such as textures and architectural detail to make better

decisions on more specific issues. They noted that they had seen other
environments at the lab which had considerably more detail. John and
Rodney pointed out that the software has the capacity to handle much
more detail.
Participants were asked to articulate what advantages/disadvantages
they perceived in having the 135-degree immersive environment.
Since they had not yet seen the single- screen version, the purpose was
to have them concentrate on and evaluate the experience of being in
the immersive environment, rather than focusing on a specific task.
I’m sitting dead centre of the screen, and I don’t see any advantage in
having the side screens. If anything my focus is drawn away from the
main information.
This comment elicited many counter arguments.
It’s not the same information. One screen would be very flat, like
looking at a TV screen.
You wouldn’t get the same sense of being in the environment and
having the context of the things you can see in your peripheral
vision…
The respondent asked John to illustrate this point with a particular
view of Fort York that showed the Gardiner Expressway on one of the
side screens.
…Now imagine what that would be like without the side screen. It’s
not the same at all. You wouldn’t have all the information. You
wouldn’t see what you would see or be aware of if you were actually
there.
It’s important to have the peripheral stuff, but it is more important to
be able to explore the environment and to be able to change the point
of view and buildings.
It’s so important to be able to change things quickly without having to
send people out with the equipment.
Participants were asked their opinions on the perceived
advantages/disadvantages of being in a group setting and having the
ability to see, hear and respond to each other in real time.
It’s very important. We were able to talk back and forth, share
reactions.
We could get input from different directions, try out scenarios really
quickly.

There were many different arguments over the meetings, but

eventually we were able to come to a common position, and we all
knew what it was.
Very important to be together – creative ideas, vision, quick reaction
to ideas.
We could discuss an idea quickly. There was no delay.
Confrontation planning works better in a face-to-face setting. If you
are going to speak from a common stance you have to understand it.
What would it be like if you were using software to share your
reactions? You could see the same things on the computer, but you
would communicate through some form of groupware.
Well it would be useful. But it would be rather like having a dinner
party with all the guests in different rooms. Conversation would suffer.
If I were watching it alone, I would worry that I was still seeing what
the “sponsor” wanted me to see. This system (immersive, real-time)
builds real confidence, because you have a measure of control over
what is displayed and you all see it happening.
Well, my computer has just crashed; hers has a fatal error (laughter).
The group was asked if a distributed asynchronous approach would
obviate the difficulties of getting a group together in one room at the
same time.
It would be easier, but I don’t think you would get the same focus or
commitment as you do like this.
We recognized very quickly that this tool had the potential to really
democratize the system. It could level the playing field with the
developers and help them to look ahead. But the question is “How do
you convince people that these tools are neutral?”
The majority of the group clearly had a high measure of confidence
and comfort with the immersive 3-screen experience; however, several
of the city representatives were seeing it for the first time. It was noted
that these participants’ reactions were different from those in the group
who had used the software for an actual task. In particular, those who
had not used the software for specific decision-making were more
concerned with the look and feel of the software (usability). It should
be also noted they were technically highly skilled and quite
sophisticated in their knowledge of graphics.
For those who had logged many hours on the Fort York project, there
was a high level of congruity in their belief that the group setting was
a very valuable component of the experience. Interestingly, no-one

specifically referred to body language, although there was a reference

to the effect of a screen as a barrier to communication.
At this point the evaluator asked John to switch to the single-screen
version of the software. Rodney and John had decided to show the
software first on the laptop without projection, to be as close
replicating the experience of a single user as possible. If time
permitted, they planned to project the single-screen image at the end of
the session. The group repositioned themselves, so that everyone could
see the computer screen. Ideally each participant would have been
seated at a computer, but the level of expertise required to manipulate
the program in its present experimental state was deemed too high to
make this feasible.
John began by explaining that with the advances in computing power,
the PC version was actually running 5 times faster on a $3500 laptop
than the version on the SGI machines. The participants were asked to
consider two models:
1 A member of the general public could view a particular model
from a computer in a public facility or from home by
connecting to the city website and downloading the model.
2 A specialized version could be created to support urban design
enquiries.
Initial reaction to the single-screen version showed wide variances in
perception.
I feel it’s a better experience. It’s sharper, more dynamic, more
coherent.
You feel as though you have blinkers on.
It’s like watching a TV screen; it takes away from the realism, from
the sense of being there.
It’s rather like the disappointment you feel when you see a
photograph, after you have viewed something through a 35mm lens.
It takes away from the realism.
Within the first few moments, members of the group reacted
negatively to a “shimmer” in the PC version, which they found
distracting and annoying. Rodney and John explained that it was
caused by the pixel to pixel shifts due to the speed of the processing.
Other comments referred to distortion. A participant asked how this
view differed from an IMAX image. The group was clearly trying to
understand how what they were seeing differed from the first
demonstration.

I am seeing the same thing, but now it’s been squashed into a single
screen. Right?
I don’t think so. It’s different.
This was most interesting. John and Rodney explained how trying to
stitch the three screens into one would cause huge distortion and that
what they were seeing now was equivalent to the centre screen of the
earlier demo. Two thirds of the information was no longer available to
them.
Let me look around. I want to turn my head. Show me what happens
when I turn my head.
I think there’s a risk that with that narrow aperture, you will lose your
sense of the whole.
It might be OK for home use, once you’ve seen the large screen.
It could be scary for those who are not used to thinking in abstract
ways.
I see this as a quite different use of the technology. If you accept the
limitations, it can be quite useful, because you could make it more
available, more democratic.
At this point there was considerable discussion about how the product
might work in a distributed version. Several scenarios were
considered.
1 Interested parties could go to a website and download the
model to their computer. They could explore the environment
and then either e-mail or phone in comments and suggestions.
2 A more sophisticated version would allow users to request
changes to the environment, such as changing the height or
density of structures. All users would see these updates and be
able to comment.
3 An even more advanced version would allow
developers/planners to prepare and “drop in” their plans to the
existing environment. Rodney explained that all of these
scenarios were possible. There were considerations, such as the
current lack of standards and file compatibility that would have
to be addressed:
Who would be responsible for the integrity of the model? It would
need to be one person.
I can imagine there will be all sorts of problems with the accuracy and
currency of the environment.

This could be made available from the city website. Everyone could
see the same thing and make comments.
Having the power to change the model is the most important feature.
It’s not enough to just walk around and look at things.
Who is in control with a distributed model?
Hijack the bus!
A developer should be able to drop a piece into the environment. Look
at how much time we would save.
The group was asked if they felt that a distributed version would be
useful in terms of enabling larger numbers to have input into planning
decisions. Had it been difficult for their group to meet, given the busy
schedules of all participants?
It would have been very difficult if John hadn’t agreed to be available
to us twenty-four hours a day.
I can see somebody not being able to make a meeting because of kids
or whatever, going home and looking at this and then being able to
make comments and contribute.
I think you could use it as preparation for a group meeting.
What about the level of participation? You can’t know whether
someone is serious or just on the periphery.
Is it a problem for the city to get community input because of
scheduling issues?
There are always people who are upset that they can’t make a
meeting. Anything that we can do to make information available to the
community is good. It’s important to connect the community to
ongoing information.
The discussion again turned to level of detail and the possibility that at
both ends of the spectrum – the developers and the general public –
credibility could suffer if an appropriate level of detail in textures,
shadows, etc. was not made available.
A suggestion was made:
Perhaps what we need is an analogy to the building model; first you
need a box which shows the zoning envelope, then you show cast
shadows; then you add detail.
If this was to be made available in a distributed mode, would the
general public have enough knowledge to know what they are looking
at?

There would need to be more information. Could it be linked to a

database?
There are various levels of 3-dimensional skills in a community –
different learning curves depending on background.
That’s where the group experience is important. For instance, in our
group, we had those who were comfortable with abstract
representations, very familiar with interpreting architectural drawings
and others who needed help. I am sure it was much more difficult for
(Name) than it was for the rest of us, but he could ask questions and
we could explain.
John explained that Rodney has already developed a model in which
the user could access additional information about any building in an
environment by clicking or touching the screen. It was felt this would
be an important element in a distributed model.

We probed for how important the group dynamic was versus the
distributed model where communication could be synchronous or
asynchronous, but not face-to-face.
Ideally, bring people together, but failing that, it’s better than no
community involvement.
It’s sort of like a speaker phone. You can’t see the person’s real
reaction. They could be agreeing with you, but making faces at the
same time.
The benefit is in the numbers you could reach. I mean, how many
installations are there like this?
I think what you have to be aware of is that many decisions are made
without information. There are so many other things going on. I might
be there to discuss the design intent, but there are all kinds of hidden
agendas; it could be money or politics.
Do you think that you could make the same decisions using distributed
software as you did with the immersive 3-screen version?
Yes, I believe that we would have moved in the same direction. I can’t
assume that we would have come to the same conclusions.
It was useful to sit in front of the screen together.
Individual decisions and discussions would probably have taken
longer. You lose the ability to have that back and forth dialogue.
Do you think that there is a commercial application for this software?
Yes, I think it is viable. It could be used by city planners, architects,
etc.
I think it would be very useful when you had multiple projects on the
go.
It needs to be tied to an underlying database. It needs to interface with
related information.
Its portability is very attractive. You could take it to reluctant
observers.
It needs to be widely available to the community. We need something
like this to democratize the decision-making process.
I think we all agree that the best is the immersive, 3-screen version in
a group setting, but because it reaches so few, a distributed version
would be useful.
The session wrapped up at 8:40 PM with thanks to all participants.

Group 2
The group consisted of planners from the City of Kitchener and the
region of Waterloo. In all, there were eight participants. Professor
Thomas Seebohm led the discussion with Kevin Moule, the systems
manager and applications programmer running the software. Rodney
Hoinkes from Immersion Studios provided the introduction and
context for the evaluation. Two graduate students were present as
observers and to answer questions on the proposed buildings they had
prepared for insertion into the environment. This was a very different
group from the one at U of T, being quite homogeneous both in
occupation and age. All are directly involved in planning activities on
a day-to-day basis. From their questions and comments, it was clear
there was a high level of knowledge of and comfort with technology.
The planners at the City of Kitchener regularly work with software
that allows them to create a simple massing model of a development
proposal and to insert that into a model of the existing context where
the latter is also a simple massing model.
The planners use the aforementioned software to evaluate the visual
impact of the height and massing of a proposal. Throughout the
session, it was noted that some of the planners were comparing their
existing software with the Poetic Dimensions software. Their focus
was narrower than the U of T group and discussion about a broadband
version of the software was harder to elicit.
Description of Session
The session was conducted in a lab at the School of Architecture,
Faculty of Environmental Studies at the University of Waterloo. A
three-screen set-up, similar to the one at U of T, had been set up with
an actual viewing surface of approximately 135 degrees. The screens
were custom built in the engineering department at the university and
have been coated with an anti-reflective paint. The coating was
employed to reduce “bounce” in the stereoscopic projection. With the
front-projected three-screen version, it had the effect of making the
two side panels appear darker than the centre panel. The most
beneficial viewing point was directly in front of the middle panel.
Accordingly, throughout the second part of the session when we
investigated the three-screen projection, we encouraged participants to
rotate, until each had the opportunity to view the environment from the
most advantageous position. Without further testing, it is difficult to
know how much this phenomenon may have affected their perceptions
of the three-screen set-up.

Rodney explained the background and purpose of the evaluation. He

explained that the conversion from the immersive environment to the
single-screen version had been made and they were to imagine a
scenario where many users could be connected to and could
manipulate elements of the same model. In addition, discussion could
take place simultaneously, either through conferencing software or
telephony. Participants were asked to think about how they currently
explore issues and to keep these traditional methods in mind as they
worked through the various scenarios. They were to consider two
themes in particular:
1 How they might use such software in the execution of their
daily tasks.
2 The advantages/disadvantages of the software.
Since this group was not familiar with the software, we had decided to
let the group work through the first demo with as little interruption as
possible, so that they could actually use it to complete a task, rather
than being distracted by questions. However, right from the start, the
group demonstrated the ability to address the planning issues, at the
same time as asking questions about the software and offering insights
into their perception of the experience.
The session began with a demo of the single-screen version of the
environment. The single image was projected on to the centre screen
of the three-panel screen. The task was to explore Centre block in
downtown Kitchener, as it is currently and then to see and discuss four
options for a proposed development in the downtown core. At the
heart of the proposed development is a new library, with mixed
residential and commercial buildings making up the remainder.
As we began the walkthrough, participants were asked how they
currently deal with a planning issue. Normally, they would have
access to aerial photographs of the existing site, plans and elevations,
computer graphics, and site visits. The group was asked to compare
those traditional methods with what they were seeing.
How high up are we – what elevation?
It provides a very realistic image as you move through.
Very powerful colour. It grabs me.
Why does the ground look the way it does?
This question was in reference to the fact that the ground was not
realistically rendered. Rodney and Thomas provided a brief

explanation; however, this clearly bothered several participants as they

kept returning to the point.
I guess that I understand the limitations, but where could this go next?
You can keep re-examining points, whereas in a site visit, you tend to
look at something once from eye level and then move on.
You get a much better understanding of the relationship between
things. For instance, if we were looking at a problem of a retaining
wall, it’s really difficult to understand the relationship between the
wall and other elements from an aerial photograph.
We rarely have any 3-dimensional information.
It would be very good for testing concepts. I mean you can intuitively
believe in something, but this allows you to test the concepts.
It allows you to test concepts of massing and setbacks.
Kevin responded to requests from the group to “walk through” the site
from a number of different vantage points.
You really need to see the block from many different vantage points.
This allows you to do that quickly and easily.
It’s critical to be able to see all sides of the block.
At this point, the proposed library was inserted into the block.
Wow. It’s big. How large is that?
That’s quick. Wait we didn’t have a public meeting. (Laughter)
It gives you instant feedback. You can really see how things fit
together. It’s great to be able to see it from above and see how the site
interacts with other elements.
The evaluator asked if the software changed the way they thought
about the issue and, if so, how.
People tend to get stuck in a single view. Here you can see different
vantage points.
It’s as though someone had taken the imagination out of trying to
understand the context – in a good way. It isn’t so open for
interpretation.
Generally, we have to come up with the criteria for a site. If there
were different variations you could try them out quickly and easily.
It makes me want to change things.
Can you manipulate the buildings?

Unlike the U of T test software, there were no manipulable elements in

the Centre block environment. Rodney demonstrated how elements
could be switched off and on and indicated that it was relatively easy
to program variations. For instance, storeys could be added, the floor
space ratio could be changed. The group clearly felt it was essential
for the software to accommodate quick and easy changes to the design.
When asked why this was critical:
If you’re working off paper-based plans, there’s a limit to how many
changes you can make. Two people get in there with a felt marker and
it’s all over. Here, if the changes were quick and easy, you could
really explore alternatives.
It’s like those music programs on the computer. You play the flute or
whatever and the program writes the music.
80,000 sq. ft might be a given, but we can re-arrange it on the site in
many different ways.
At this point there was quite a lengthy discussion about the ability to
manipulate the environment, including such things as adding detail to
the roads and sidewalks, landscaping, casting shadows, etc. While
these are noted as being common concerns of both groups and will aid
further software development, they are not the main thrust of this
report. They are more likely to have an impact on the writing of the
development guide for designing effective interactive immersive
experiences. As in the U of T discussion, there were those who felt
that massing models would be most appropriate for early discussions
of broad concepts. Detail would be most appropriate when general
agreement had been reached and more specific discussions began.
Others felt the detail should be there from the start to add credibility
and realism to the environment.
Rodney indicated that everything they were asking for had been done
or was possible and that software refinements were currently in the
works.
When probed about how detail would affect the nature of discussion,
the group responded:
Detail in the model will change the nature of the questions from the
general to the specific. The nature of the questions will drive the
discussion.
The pedestrian view is most important to me. I need to be able to see
what the average person will see.

We want to start out from the bird’s eye view, because we are
planners, we need to see the big picture, but to feel it on the ground
level, you need the detail.
You need more (detail) than in the existing ground level. It becomes
important when you have to consider things like parallel parking and
landscaping.
If you were trying to develop a vision, rather than a specific plan, do
you need the level of detail you have been discussing?
Can you add dynamic elements, for instance, could you superimpose
items like an LRT on the existing environment, so people could see the
train running through it? That would be amazing. (Rodney answered
in the affirmative.)
It depends on the audience. Politicians, bankers and Joe Public would
need the level of detail to relate. They would find it very compelling
and infinitely more valuable than anything we can currently do.
You could show them the existing environment and then say “Let’s
work with you” and begin the dialogue.
At this point, we took a short break and moved to the three-screen
immersive software.
The evaluator asked the group for their reactions to the three screen
environment.
I didn’t notice there were three screens. I am still focused on the
centre one.
Does it have to be three screens? Two would be good. The third
doesn’t add anything.
Could the single screen just be panoramic?
Do you believe that you are seeing the same information?
It’s more realistic. It fills your peripheral vision. I’m not sure if the
information is the same. I think so.
I think you are seeing more. Now I see it. I think there’s more
information. It makes you feel more as though you are actually there.
This would be great for showing store owners what their storefronts
look like. Most of them have no idea. It makes it so realistic.
It makes you move your head more, unless you are right in the middle.
The public needs this. The public needs to feel they are there. We
(planners) can fill in the gaps. This would make the public feel more
relaxed. It would give them a sense of comfort.

The one screen makes you feel that you are watching a movie; this is
more real.
It makes your eye denser somehow.
Are you moving faster through the environment? I felt the other was
too slow, but it could be just that there is more to look at here and it
keeps the eye involved.
Yes, it’s more comfortable than the single screen, because the eye has
so much to look forward to. There’s more in the space.
It feels very real. I became more conscious of the things I would do
normally (on a site visit), like looking to the left or right as I went
around a corner. I feel I want to do those intuitive things now.
At this point Rodney switched off the two side screens. The evaluator
asked how the group was affected.
I didn’t notice going back to the single screen.
His imagination filled in the two screens.
I don’t think it is important how many screens there are; it’s how you
discuss the issues.
It’s not critical to have three screens with developers, etc. Once the
“Wow” factor is over, the real power is in the tool having
manipulability.
That’s the most important thing, to be able to change elements of the
environment as we are looking at them and discussing them.
But it would be good to have the “wow” factor at final presentation.
The group responded to a question about how important it would be to
be remotely connected. Did they have difficulty bringing a group
together?
It’s not so much an issue of travel time; it’s more a function of being
too busy.
I don’t think it would be important for this group to have an Internet
connection to a version you could manipulate remotely.
It could be used as a method of doing your homework for a planning
session. There might be a better sense of coordinated control. On the
Internet, you’d be jumping back and forward. It might be too crazy.
The group saw the main advantages of the software to be:
The ability to do infinite numbers of changes and adjustments quickly
and easily. A paper plan takes too long. People are more likely to

settle because of the time constraints. This gives the freedom to

suggest changes. The number of screens, the level of detail, these are
not as important as the ability to change things in real time.
When questioned about the importance of being together in the same
space and time, the group responded:
Our work involves negotiation and compromise – coming to consensus
– a very human activity. Being in the same space is important because
you can see how someone reacts and work out how best to approach
the situation.
What about the ability to have more public interaction and
involvement?
A website with info on a potential plan might be good. People might
go there. They might be interested rather than confrontational.
With three screens only the three people in the middle are getting
beneficial views. Would this be a problem for public view?
The sense is that it paints a rosier view than the reality. Look at those
buildings. Anyone who knows the Centre Block knows that there are a
lot of social issues. You can’t really see that.
It’s still more real than conventional models, plans, elevations.
I still think the public would need a lot more information in the ground
plane, such as parking and landscaping.
How does this version change the nature of your discussion?
It changes the content of the discussion because, unlike with a model,
interpretation is not necessary. You are relying less on the skill of the
artist; you are seeing good and bad. There is much more room for
criticism, because there is more information. They (artists/developers)
can’t manipulate to the same extent.
The evaluator asked the group if they felt it would be important to
“drive” the software themselves.
No. It doesn’t really matter as long as s/he follows instructions, as
long as you have a cooperative driver.
How might this be used outside of your day-to-day planning activities?
Very useful in negotiation. Often we are in the middle between the
developers and the public trying to facilitate agreement. Having the
ability to quickly display options could help to bring mediation faster.

The community needs representation. Often with site plan discussions,

there is no community rep. There could be an opportunity to improve
community representation.
Visit to Stereoscopic Lab

At this point, respondents were invited to view a stereoscopic version
of the software, located in the Engineering wing of the university. At
the lab, small groups were able to don glasses for a brief exploration of
the same Centre Block environment. Participants were clearly
impressed with the set-up, but given the late hour, did not have time to
respond to questions. Accordingly, this component has not been
included in the report.
Findings
It is important to acknowledge some contributory factors in the
evaluations, before attempting to outline the major findings.
The two groups were quite different, both in demographics and in their
familiarity with the software. For many of the Waterloo participants,
the demo session was their introduction to the software, whereas the U
of T group had worked together with the immersive software over a
period of weeks.
There was a difference in the way that the single-screen software was
presented to the two groups. At U of T, the participants saw the
software demonstrated on a laptop computer, to more nearly simulate
the experience of a single user; at Waterloo, the image was projected
onto the centre screen of the three-screen set-up, because there were
too many participants to gather around a single computer. The
differences may have affected responses. It was not possible at the
time of testing to fully simulate the experience of being a single user
connected through broadband technology to other networked users. As
such, we asked the respondents for comments on the usefulness and
feasibility of a concept. The information gathered through the sessions
has confirmed the perceived value of a single-screen version of the
software utilizing broadband networks, and more clearly defined the
next stages of its development, as well as identifying several
interesting areas for further research.
Persistence and involvement

Both groups showed considerable persistence and involvement in the
evaluation. Despite the length of the sessions (3+ hours), body
language indicated interest and attention, all participants sat forward in

their seats, eyes on the screen and there was little unrelated discussion.
Although the sessions ran slightly longer than planned, none of the
participants left before the end.
Immersive three-screen version
Advantages
Both groups alluded to a “Wow!” factor with the three-screen version.
They felt that being together in the same physical space, with the
opportunity to manipulate the information and discuss changes to the
environment in real time was the preferred option. Participants
expressed the opinion that the immersive version provided the most
realistic experience, the closest to actually being there, while still
offering the benefits of manipulability. Several participants who are
not trained planners or architects explained that the software promoted
confidence to examine a proposed development because they could
walk through the environment, view it from multiple perspectives and
understand the relationships amongst the elements.
Many participants from both groups expressed the opinion that this
type of software would increase public confidence in the planning
process. It was felt that models, plans and artist’s renderings do not
provide enough context for the average viewer who cannot visualize a
finished building from a two-dimensional plan.
Several participants felt that this software would be very useful for
final presentations to small groups of proposed developments.
Disadvantages
Participants in both groups acknowledged the constraints of space,
time and money in setting up such a system and its limited
accessibility. While there was nobody who indicated the immersive
version was not useful, some participants were unable to distinguish
between the three-screen and the projected single-screen versions.
There was discussion amongst both groups about the level of realism
required to make the software credible, particularly with the public.
Both groups suggested a graduated approach, where wire frame and/or
massing models could be used in early planning with experienced
personnel. Graphic sophistication could be added for presentation to
those less able to think in abstract three-dimensional forms. These
discussions were prompted by both the three-screen and single-screen
versions.

Conclusions
Overall it was perceived that the three-screen immersive environment
made for the most realistic and interactive experience, because of the
engagement of the peripheral vision that made it most analogous to the
real experience of walking through an environment, but clearly the
groups recognized its limitations in terms of cost and accessibility.
Single-screen version
Advantages
The perceived advantage of a distributed version for both groups was
for the ability to provide a publicly accessible website, perhaps as an
extension to the municipal planning authority, where plans under
consideration could be downloaded, examined and commented upon.
The U of T group described differentiated levels of manipulability,
depending on the expertise of the contributor. At the highest level,
developers and planners could amend the plan or even insert their own
concepts into the existing environment. It was felt by the planners that
this would encourage more creative discussion in the time saved on
paperwork and administration. However, there was concern about
standards and the manageability of multiple versions of a plan.
The ability to change elements in the plan instantly to test hypotheses
rather than suffer the current delays in creating iterations was seen as a
very powerful advantage for both versions of the software. This
advantage was the one referred to most often by both groups. The
second group kept coming back to it, perhaps because their demo
environment only allowed the user to remove and substitute elements,
not change the nature of the elements themselves.
Several references were made to using the single-screen version of the
software as a tool that would help users to prepare for planning
meetings and discussions. This preparation was seen as particularly
valuable when confrontational negotiations were anticipated.
Disadvantages
Initially, members of the Waterloo group indicated that they did not
see value in a broadband application for the software; however on
further prompting they conceded that it might be a useful vehicle to
promote community involvement in planning decisions. There was
also consensus that being together physically for discussions,
especially when those discussions are confrontational is the most

desirable scenario, but again accessibility was a concern, particularly

where the need for public consultation and input is imperative.
The group appeared focused on the software’s potential in their
workplace. As such, they stressed the need for development of the
software’s capacity to manipulate the environment in real time. Two
participants indicated they would not be interested in using any
version of the software in their workplaces if it did not have this
feature.
There was considerable discussion with both groups about the level of
realism as regards texture, richness of detail and background
information required to ensure credibility, particularly with the public.
The rendering of the ground plane was seen as particularly deficient in
the Waterloo demo.
It should be noted that these comments were recorded for both
versions of the software. Those participants who had the most
experience with abstract conceptualization suggested a graduated
approach to the sophistication of the information. In early planning
discussions, massing models and wire frame renderings would be
adequate to achieve consensus on broad-based concepts and visioning.
More detail could be added to facilitate planning discussions on more
specific issues and the highest level could be added for public
consultation and/or presentation.
Both groups quickly reacted to a perceived “shimmer” in the single-
screen version.

With the University of Toronto group, there was a strong feeling that
this software in its distributed format would democratize planning and
make public consultation more accessible and feasible, although
concerns were expressed about how to manage the process to maintain
credibility.
The Waterloo participants were more reluctant to embrace a
broadband version of the software. While they could clearly see the
application of the software in their jobs, they initially rejected the
concept of a broadband application.
Of interest and a possible avenue for further research was the
confusion expressed by some of the participants as to the differences
between the three-screen and the single-screen version. Some
respondents were unable to discern that the information they were
seeing was significantly different. Although they were seeing one third
of the information available to them on the three-screen immersive
projection, they believed it was the same, just compressed to fit on the
smaller screen. Others immediately recognized the differences. With
the first group, the confusion seemed to lie in a clear division between
those who worked with abstract visualization as a matter of course and
those who did not. However, the Kitchener/Waterloo participants
expressed the same confusion and they are all planners. It would be
interesting to pursue research to ascertain whether they actually did
not perceive the difference in the amount of information, or whether,
as one respondent explained, “Imagination fills in the information
from a memory of what was/should be there.”
Finally, respondents indicated that they believe there is a future for a
distributed version of the software. They would definitely like to see
developments in the software that would make it a simple, user-
friendly tool to improve community access and input in planning
decisions.
Although the tests conducted by Egenuity and Immersion Studios
were simulations designed to replicate the performance of broadband
performance, The University of Waterloo has already installed
software at the municipal offices to facilitate collaboration between
their organizations. The installation and established relationship
between the university and the municipality could provide an ideal test
bed for further development and for ongoing research to assess how
well the software meets its perceived capabilities to improve
community access to and input on public planning.

Acknowledgements
Assistance for this project was provided by the Minister of Canadian
Heritage and the CANARIE Applied Research and Interactive Media
program. CANARIE implements and manages the national research
and education broadband network CA*Net 4. In addition, it
administers and supports research and development initiatives in e-
Content, e-learning, e-health and e-business.
Special thanks go to Professor John Danahy of the University of
Toronto’s Centre for Landscape Research and Professor Thomas
Seebohm of the University of Waterloo’s Faculty of Environmental
Studies who generously provided of their time and facilities to make
the evaluation a positive experience for all.

Appendices
Questionnaire 1
Comparison of Immersive and PC Environments: Focus
Group Questions for Immersion Studios October 30,
2003

Opening
Thank you so much for agreeing to attend this session and help us with
the evaluation. As you know, Immersion Studios has been a leader in
the field of developing immersive technologies for a variety of
purposes. Since Rodney is with us, I will ask him to give us a brief
history of the development of the software you will use tonight and tell
you a bit about the CANARIE contribution, their requirements and the
purpose of tonight’s evaluation.
Introduce Rodney – Chief Technology Officer of ISI and former
professor at U of T.
Thanks, Rodney. We have one last little piece of business. I would like
to record the session tonight for the sole purpose of being able to refer
to the tape to ensure accuracy in my report for ISI. No names will be
used in the report, only a general description of the number of
attendees and the session. Once the report is written, a copy will go to
ISI for their records. No other copy will be kept.
Comparison of Immersive and PC Environments: Focus Group
Questions
Note: As we do not want to lead the subjects, the probes in red will
only be used if the subjects lead the discussion into the specific topics
covered by the probes.
In contrast, the probes in blue will only be used if the subjects do not
mention a particular topic in the context of their responses.
Introduction
Goal of session (i.e., to compare single-screen version to
immersive environment) Not usability of the software.
Who we are
Names of participants, roles
Issues explored by Friends of Fort York
Can you tell me a bit about your group, Friends of Fort York?
What kinds of issues has your group looked into using the
immersive environment?
How did you explore these issues in the past prior to the
availability of the immersive software?

Overview of use of immersive technology by Friends of Fort York

Describe how your group used the immersive environment a
few months ago?
Probes if necessary (i.e., if subjects don’t discuss):
• What was involved? (e.g., number of people, tasks)
• What information were you looking for?
• How did you use the information?
How successful was your use of the immersive visualization
technology for your purposes?
Probes if necessary:
• What were the advantages/disadvantages over your
traditional exploration of these issues?
Possible probes depending on their answers:
• Was there anything you wanted to see but couldn’t?
• How well did the technology work in a group
environment?
Exploration of Single-screen Version
Here is a single-screen version of what you saw in the immersive
environment. Currently it is a stand alone version running on 2 PC's,
one for visualization and one for control, but the ultimate goal is to
have a version that can run on a home computer. Users would be able
to share the application over the Internet.
Let’s take a look at it now. John will operate it for the group.
(Direct John to what you want to see.)
Think about the tasks and decisions you were making using the
immersive environment – and let’s explore the single-screen
version to look at the same things.
How is it different from what you remember of the immersive
environment?
• How does the single screen affect your ability to do the
task?
• In what ways does the single screen limit your ability to
make decisions?
• Probes regarding scale, immediacy, involvement, etc.
How might you use this at home on your own?
• Would you find it useful to have this kind of access? Why?
How might you use this if you were sharing it with one or two
others over the Internet?

How might you use this at one of your meetings, if one of the
group members brought it in on a laptop?
• Would it have been useful at your meetings a few months
ago when preparing your case?
• Would it be useful to bring the laptop version to any other
meetings?
Comparison of single-screen Version and Immersive Environment
(before seeing immersive environment)
What benefits, if any, are there using the single-screen version
vs. the immersive environment for you or your group (based on
their memory of the immersive experience)?
Possible Probes, depending on their answers:
• Would you benefit in using the single-screen version in any
way from a singular/personal exploration of the space?
• Are there any benefits to being able to control the
exploration yourselves?
• Would your group benefit from being able to return to the
environment multiple times?
What drawbacks, if any, are there using the single-screen
version vs. the immersive environment for you or your group
(based on their memory of the immersive experience)?
• Does it give you identical views?
• Could you use it to make decisions?
Exploration of Immersive Environment and Comparison
• Let’s go to the immersive environment now and explore it.
John will operate it for the group. (Direct John to what you
want to see so that you can make a comparison to the single-
screen version.)
• How would you compare it to the single-screen version now
that you’ve had a chance to see them side-by-side?
• Do they both give the same information?
• Are there things in the immersive environment that you did
not see in the single-screen version?
• Do they both allow you to make decisions or draw
conclusions? Do you draw the same conclusions?
• Does this environment change the dynamics of the group at
all?

Use of Both Environments

How might you use these tools if you or your group had access
to both?
Possible Probes depending on their answers:
• Would you explore the personal space on your own to
prepare yourself for the group discussion with the

Questionnaire 2
Comparison of Immersive and PC Environments: Focus
Group Questions for Immersion Studios, November 24,
2003

Opening
Thank you so much for agreeing to attend this session and help us with
the evaluation. As you know, Immersion Studios has been a leader in
the field of developing immersive technologies for a variety of
purposes. Since Rodney is with us, I will ask him to give us a brief
history of the development of the software you will use today and tell
you a bit about the CANARIE contribution, their requirements and the
purpose of today’s evaluation.
Introduce Rodney – Chief Technology Officer of ISI
Thanks, Rodney. We have one last little piece of business. I would like
to record today’s session for the sole purpose of being able to refer to
the tape to ensure accuracy in my report for ISI. No names will be
used in the report, only a general description of the number of
attendees and the session. Once the report is written, a copy will go to
ISI for their records.
Comparison of Immersive and Single-screen Environments: Focus
Group Questions
Note: As we do not want to lead the subjects, the probes in red will
only be used if the subjects lead the discussion into the specific topics
covered by the probes.
In contrast, the probes in blue will only be used if the subjects do not
mention a particular topic in the context of their responses.
Introduction
• Goal of session (i.e. to compare single-screen version to
immersive environment). Not usability of the software.
• Who we are
• Names of participants, roles
Focus group participants’ introduction
• Introductions
• Can you tell me a bit about your group?
• What kinds of issues does your group consider?
Probes if necessary (i.e. if subjects don’t discuss):
• What is involved? (e.g., number of people, tasks)
• What information are you looking for?
• How do you use the information?
• How do you explore these issues currently
Probes if necessary
• What types of media/materials/documentation do you use?
• How many people are consulted/involved?

Today we are going to work through a planning exercise using the

immersive software. You will be concentrating on discussing
issues, suggesting solutions and trying to come to consensus.
Throughout the exercise I will be asking you about how the
immersive environment contributes to your understanding of the
issues and if/how it aids in the decision-making process. However
the bulk of the discussion of the technology will be at the end of
the exercise. As you work through the session, try to be aware of
how the immersive software is different from traditional methods
of exploring planning issues and the advantages/disadvantages of
having such a system.
Questions as they explore the environment?
• In what ways does this visualization tool aid your
understanding of the issues?
• Is it more or less useful than the traditional methods? How?
Why?
• What can you do or see or feel that you cannot in current
explorations?
• How useful are these attributes?
• Do you feel it is an accurate representation of the physical
space?
• Level of detail
• Texture
• Field of view
• Colour
Questions for post-session discussion?
• How successful was your use of the immersive visualization
technology for your purposes?
• What were the advantages/disadvantages over your
traditional exploration of these issues?
• Was there anything you wanted to see but couldn’t?
• How did the environment contribute to your decision-
making?
• How important was it to be together in the environment?

Exploration of single-screen Version

Here is a single-screen version of what you saw in the immersive
environment. Currently it is a stand alone version running on 2 PC's,
one for visualization and one for control, but the ultimate goal is to
have a version that can run on a home computer. Users would be able
to share the application over the Internet.
• Let’s take a look at it now. Tom will operate it for the group.
• Think about the tasks and decisions you were making using the
immersive environment – and let’s explore the single-screen
version to look at the same things.
• How is it different from what you remember of the immersive
environment?
• How does the single screen affect your ability to do the
task?
• In what ways does the single screen limit your ability to
make decisions?
• Probes regarding scale, immediacy, involvement, etc.
• How might you use this at home on your own?
• How might you use this if you were sharing it with one or two
others over the Internet?
• How might you use this at one of your meetings, if one of the
group members brought it in on a laptop?
• Would it have been useful at your meetings a few months
ago when preparing your case?
• Would it be useful to bring the laptop version to any other
meetings?
Comparison of single-screen Version and Immersive Environment
• What benefits, if any, are there using the single-screen version
vs. the immersive environment for you or your group
Possible Probes, depending on their answers:
• Would you benefit in using the single-screen in any way
from a singular/personal exploration of the space?
• Are there any benefits to being able to control the
exploration yourselves?
• Would your group benefit from being able to return to the
environment multiple times?

• What drawbacks, if any, are there using the single-screen

version vs. the immersive environment for you or your group?
• Does it give you identical views?
• Could you use it to make decisions?
• Do they both give the same information?
• Are there things in the immersive environment that you did
not see in the single-screen version?
• Do they both allow you to make decisions or draw
conclusions? Do you draw the same conclusions?
• Does this environment change the dynamics of the group at
all?
Use of Both Environments
• How might you use these tools if you or your group had access
to both?
Possible Probes depending on their answers:
• Would you explore the personal space on your own to
prepare yourself for the group discussion with the
Do you see a potential commercial application for this software?
• Describe it.
At this point, if there is time, the group may go to the engineering lab
to see the stereoscopic view of the same environment.

Questions for viewing the stereoscopic version of the software.

How does the stereoscopic version of the software differ from the
immersive version? The single-screen version?
Are there things you see in this view that you did not see in either the
immersive or single-screen versions?
Does the stereoscopic view improve your ability to make decisions?
Are there disadvantages to the stereoscopic version vs the others?
Can you describe a situation where you would prefer to use the
stereoscopic view and why?

Appendix C
IIC 3-Screen
Immersive CG
Production
Interactive Content Development Guide Appendix C
Introduction
Example of an Immersion Interactive Cinema.

The picture above shows an example of an Immersion Interactive
Cinema (IIC). An IIC offers a large-format, panoramic triple-screen
theatre in high-definition (3072 x 768 pixels) with Dolby Digital
surround sound. The audience, referred to as users, interacts with the
content via high-definition touch screen consoles (1024 x 768 pixels)
to control decision points of an experience. This is performed in a
myriad of ways such as by entering choices via using voting, playing
games or performing personal explorations. The content is seamlessly
integrated and synchronized between the big screen and the consoles.
The virtual 3-Camera setup

Note that to model, animate and render computer graphics
environments, characters and props Immersion Studios uses 3D Studio
Max extensively. As such, all references to the techniques detailed in
this document are with respect to this particular CG package.
When animating for an IIC, you must first create a compatible virtual
studio within a 3D Studio Max scene. Creating a virtual studio
includes all objects, environmental effects, lighting and, unique to an
IIC, the 3-camera setup.
© 2004 Immersion Studios Inc C-1

Create center camera

First, you create your center camera. The center camera is your main
camera. As such if a camera move or effect is called for then this is the
camera that you animate with and create keyframes for.
Production Note: You do not move or animate the side screen
cameras. Once they are created and linked to the
center camera, be very careful not to touch them,
for fear of offsetting them.
The center camera is created normally, ensuring that the field-of-view
is 45 degrees (camera focal length of 43.456mm) You must not
deviate from these numbers, as IIC projection screens are always
offset by 45 degrees. The other camera parameters, such as target
versus free, environmental ranges or clipping ranges, may be adjusted
according to the needs of the scene. Please note that if you change the
environmental settings for one camera, you must remember to also
change them for the other two cameras.
Name centre camera

Once you have created the center camera, then name that camera
‘cam_01’ or ‘c_01’ or ‘raptor_shot5_01.’ Be sure to use a name that is
appropriate to a scene, along with the number ‘01’ to indicate that it is
the center or main camera.
Create, name and link side cameras

To make the side cameras, clone the center camera, and locally offset
each camera plus 45 degrees and minus 45 degrees, respectively. Then
name the new side screen cameras accordingly, with the left-screen
camera having the number ‘00’ and the right-screen camera having the
number ‘02.’ Once the side cameras are created and named, link the
side cameras to the center camera. Now the 3-camera setup can start
being used.
Configure viewports for three views

To visualize what is being rendered with the three cameras, the
viewport configuration is used. Choose the configuration with three
views on top and one big one on bottom. Make the three views on top
display the three cameras and also display safe frame for these three
views.

Production Note: Ensure that in the rendering dialogue box that it is

set to render a 4:3 aspect ratio, such as 640:480 or
1024:768.
This will make the safe frames in the camera viewports accurately
reflect the image borders. Fortunately, since Max 4.0, you can drag
and change the shape of viewports, so you can fine tune the viewport
to match your rendering composition. You can change your viewport
configuration to fit the current needs of the animator and this
configuration is handy for visualizing three cameras simultaneously.
Production Note: If you have already created and animated a camera
without side cameras, then when cloning and
linking side cameras to the pre-animated camera, be
sure to delete keyframes on the side cameras right
after cloning. You want to link them to the
animated camera but not allow them keyframes
themselves.
A simple 3-camera setup.

Understanding the 3-Screen IIC theatre

Why a three screen projection system at all? Why three cameras? Why
not just one rounded screen or even one super wide 4:1 flat screen?
Why not just one simple, 16:9 flat screen like commercial movie
theatres?
The ideal viewing location of the participant.

By using three screens, you are creating a unique and immersive
experience. In an IIC, the participant is treated to a full 135 degrees of

imagery. It is important, however, not to confuse this visual effect with

others more easily achieved.
The 3-screen theatre uses three separate centers of perspective
rendered with three separate cameras. As previously mentioned,
cameras are offset 45 degrees each matching an IIC screen and
associated projector. In the diagram above, although different camera
angles cause major perspective changes, this is only because you are
viewing three different angles on a flat plane, on paper or on a screen.
In an IIC however, the screens are angled appropriately, as is shown
above.
The diagram above illustrates the ideal viewing location of the
participant. Although all theatre consoles aren’t located in the focal
center of the 3 projectors, the effect is similar. Lines straighten out
between screens.
Problems and common misconceptions

It is important to correct some common misconceptions. First, one
wide-angle lens cropped into a 4:1 image and divided between three
screens is not the same as using three cameras. One wide-angle lens
only allows for one center of perspective, radiating from the center
screen, like in the diagram below.
Incorrect
One wide-angle lens only allows for one center of perspective, radiating from the
center screen.
Although the complete image is in three separate 4:3 images, there is
only one center of perspective in the center screen. This causes
unreasonable stretching in the side screen images (e.g. wide-angle
distortion) and simply does not look correct in the 3-screen theatre.
In the diagram below is the proper rendering, using three separate
cameras with therefore three centers of perspective.

Correct
Using three separate cameras with therefore three centers of perspective.

The mismatching lines of the cylinder occurring between screens are
acceptable based on the assumption that these images will be viewed
on angled screens in an IIC theatre.
One alternate approach may be to use one camera with a normal or
even telephoto lens far enough back from the action to capture
everything. This provides one image, not 135 degrees but with a much
narrower field of view that can be cut up into 3 images. Although this
nearly eliminates the stretching of image along the outer screens, it
does not total 135 degrees, nor is it three separate cameras. It simply
defeats the purpose of having three angled screens in the theatre and
once these images are projected on three angled screens, the image in
fact looks awkward.
One instance where the one camera technique has been employed with
some measure of success was for a live shoot where a human character
had to be composited into a virtual environment. After the character
was composited into the CG background, the whole scene with the
character added was sliced up into three separate images for each of
the three screens. The result was acceptable; however the 3-screen
effect was lost. The fundamental issue here is that you cannot readily
duplicate a virtual 3-camera setup in the real world. A workaround
solution is to keep the composited element inside one screen and
thereby never encounter the inevitable shift in perspective that occurs
when a character crosses into another camera view.
The illustration below shows a character moving in a straight line,
across all three screens. Notice the change in perspective. On the right
screen the character is moving toward us. By center screen the
character is walking straight across our view and by left screen, the
character is moving somewhat away from us.

A character moving in a straight line, across all three screens.

This cannot be simulated in a studio setting, unless a real 3-camera
system exists, which to our knowledge it does not. Therefore, as
outlined above, you must restrain the human actor to one screen.

Appendix D
IIC Object, Variable
and Data
Referencing
Standards for
Console Content
Interactive Content Development Guide Appendix D
Introduction
When working in any production development environment it is
paramount to follow standards and none is more important to be
following than well-defined naming conventions.
Object naming
Naming components is a key activity in all CG modeling
environments. It can be treated either in a very casual or very strict
manner, depending on the organization or on the number of CG
specialists working on a project. Names of data objects are the primary
method to access the data and control it interactively.
Naming occurs at three levels depending upon how the applications’
data is organized:
1 Files.
2 Layers.
3 Objects.
Object naming using common language rather than technical is
critically important. End users of a GIIE may end up using these
names to select objects or data to interact with or create links to. Non-
intuitive (i.e. FM-01-LXD-0443) or generic (i.e. BOX01) naming can
hamper or even eliminate interactions altogether. Ideally, naming has
been found to focus on the common name, domain-specific type of
data, its position or scale, and perhaps a version or option identifier
(i.e. HOUSES-SE-2story-optionA, or POSTOFFICE-45m-v1).
Another important use of naming is in controlling complexity. To
simplify models, change options or show detail when focused on one
area, a user can manipulate data by its name organization or through
direct geometric manipulation. Naming provides a mechanism to
group data together to quickly be manipulated.
Variable Naming Standard

Variables (field names, tags, etc.) used in data files should follow the
format:
wordWordWord
Where the first word is lowercase and subsequent ones start with an
uppercase letter. The exception is acronyms, such as userID, where
the entire “word” is uppercase. If the acronym starts the name, it
should be entirely lowercase, such as idCode.
© 2004 Immersion Studios Inc D-1

Interactive Content Development Guide Appendix D
Variable Value Standard

Values for variables should be noted in standard INI1 file format,
ending with a carriage return:
wordWordWord=Value
Where Value is a string, the following rules must apply:
For carriage returns within the string, use the ‘C’ language
convention of ‘\n’.
The following special codes should also be followed:
Tab \t
CR \n
\ \\
If you need to specify an actual ‘\’, use a double ‘\\’.
Variable Data Storage Standard

Use the following project directory naming convention:
####-Brand-Name
Where #### (a 4-digit code) is the docket number for the project
which will be unique by content piece and major version number.
The Brand is the organization (i.e. BNFL, Leafs).
The Name is the project name. Brand and Name will be centrally
assigned when a docket number is assigned.
For IIC network control, the FULL directory name (including
c:\isshow) should be less than 30 characters long.
For example, the Leafs Fanzone Interactive would be:
C:\isshow\1727-leaf-fanzone
Data should be organized into logical subdivision, then media type, for
example:
1727-leaf-fanzone\team\images
1727-leaf-fanzone\team\text
1727-leaf-fanzone\history\images
If there are variations within a media type, subdivided further, for
example:
1727-leaf-fanzone\team\images\sml
1727-leaf-fanzone\team\images\med
1 INI files are a standard notion for most developers under Microsoft Windows. While
Macromedia Flash can have a script written to parse INI format files, it is much easier to
use Flash INF files which are slightly different in formatting, using a ‘&’ between
variable/data pairs instead of carriage returns.
© 2004 Immersion Studios Inc D-2

Appendix E
CANARIE Partners
and Profiles
Interactive Content Development Guide Appendix E
Immersion Studios Inc

Dr. Rodney Hoinkes and Mr. Rob Krieger
Immersion Studios has been pioneering in this field of immersive and
group-interactive eContent development for the past 8 years. In this
time, it has produced over 14 large-scale immersive group-interactive
content productions, and developed a high-end intranet ‘theater’
product in which to run and develop advanced eContent (it has
installed 12 theatres worldwide in the past 3 years). The popularity of
its content and approach to advanced eContent has led to a number of
post-secondary curriculum courses being offered, the technology being
adopted as a foundation for a range of University research projects,
and several 3rd-party production companies pursuing the market.
Immersion Studios will be the project lead, provide key software
technology and programming skills, as well as content to this project.
University of Toronto
Mr. John Danahy
The University of Toronto, Faculty of Architecture, Landscape and
Design, Centre for Landscape Research is a world leader in real time
visualization as a design and planning tool. The CLR currently
operates an Immersion Simulation Lab, and continues to develop and
experiment with a variety of Urban Planning, Architecture and
Landscape Architecture projects. The City of Toronto, City of Ottawa,
and National Capital Commission are frequent clients utilizing the
immersive and interactive capabilities of the CLR. The CLR will be
providing content and content editing for urban models being used in
professional decision-making as well as consulting on evaluation goals
and methodologies for professional design and planning applications.
UofT has provided regional planning content, content editing and test
groups, as well as its Immersion Simulation Lab for testing to this
project.
University of Waterloo
Professor Thomas Seebohm
University of Waterloo, Integrated Centre for Visualization, Design
and Manufacturing, is conducted a variety of large scale planning and
design exercises for the Region of Kitchener/Waterloo, using its
© 2004 Immersion Studios Inc E-1

Immersion Simulation Lab as the visualization and interactivity

medium. Development, experimentation and modification of content
continues and Waterloo, shaping both the program, and the
effectiveness of the immersive lab as a teaching and research aid.
UofW has provided regional planning content, content editing and test
groups, as well as its Immersion Simulation Lab for testing to this
project.

Dr. Annabel Cohen and Mr. Sandy MacAuley
As part of the Institute for the Study of Culture, Multimedia,
Technology and Cognition (CMTC), a joint project with the Université
de Moncton and the University of New Brunswick, the University of
PEI conducted the education evaluation portion for this study. The
CMTC focus is to determine the best use of multimedia in education in
a cultural context.
At UPEI, investigators in the faculties of Arts, Science and Education
conduct behavioral studies of learning, memory and cognition, musical
and multimedia competence, cognitive function in dementia, and the
effects of drugs and enrichment on developing and aging nervous
systems.
Envision Sustainability Tools Inc.

Mr. Mike Walsh and Mr. David Biggs
Envision Sustainability Tools Inc. (ESTI) develops regional
sustainability visioning tools designed to engage planners, policy-
makers and regional stakeholders in a discussion about desirable and
plausible regional futures. Envision’s tools are used by regional
planning agencies to develop publicly supported regional visions that
translate into healthier, more livable regions. ESTI has provided
guidance to professional evaluation needs of the study.
Egenuity Inc
Ms. Marilyn Welsh
Egenuity Inc. is a full-service e-learning firm offering consulting
services to education and training organizations in the areas of
strategic planning, project management, instructional design, training,

and evaluation. Its principal officer, Marilyn Welsh, has over twelve
years experience in multimedia and e-learning development and is a
published author and frequent speaker on the use of technology in
education. She has advised educational institutions and private
organizations in Canada, the USA, Mexico, South America, the UK,
Europe, Australia and New Zealand. Egenuity conducted the
professional evaluation portion for this study.
Sheridan College
Mr. Avrim Katzman
Sheridan College, SCAIT, working with its Immersion Simulation
Lab, has been conducting research in a variety of new media areas;
including interface design and functionality, group interactive content
development, and animation techniques.
Centennial College
Mr. Nate Horowitz
Centennial College’s Centre for Creative Communications offers
comprehensive teaching in Interface Design, Video Production and
post, 3D animation, Online Writing and Editing; Instructional Design
and Information Architecture. The faculty and students, including
adjunct professors have all worked as developers and designers in
broad bandwidth applications within the private sector for at least 5
years.

Interactive Content Development

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Interactive Content Development

Uploaded by

Copyright:

Available Formats

Interactive Content

Interactive Content Development Guide © 2004 Immersion Studios Inc.

Immersion Studios Inc.

© 2004 Immersion Studios Inc i

Chapter 2 What is a Group Interactive

© 2004 Immersion Studios Inc ii

Chapter 3 Perception and Cognition

Computer Systems ............................. 4-10

© 2004 Immersion Studios Inc iii

Cinematic Group Interactive Immersive

Chapter 5 Production Team Composition

© 2004 Immersion Studios Inc iv

Artwork-related roles ................................. 5-8

Chapter 6 Production Process (IIC)

© 2004 Immersion Studios Inc v

Console Networking ..................................6-13

© 2004 Immersion Studios Inc vi

Naming Convention for Video (“Image”) Files............ 6-32

Chapter 7 Production Process (ISL)

ISL production flow .................................... 7-5

© 2004 Immersion Studios Inc vii

Model complexity and levels of abstraction .............. 7-20

Appendix A IIC Evaluation

© 2004 Immersion Studios Inc viii

© 2004 Immersion Studios Inc ix

© 2004 Immersion Studios Inc. 1- 1

projects and several 3rd-party production companies pursuing the

© 2004 Immersion Studios Inc. 1- 2

© 2004 Immersion Studios Inc. 1- 3

The participants in the development of this guide cover a broad

© 2004 Immersion Studios Inc. 1- 4

Who should read this guide?

© 2004 Immersion Studios Inc. 1- 5

How should this guide be read?

© 2004 Immersion Studios Inc. 1- 6

What is a Group Interactive Immersive

Old Native American proverb

© 2004 Immersion Studios Inc. 2-1

Example of an Immersion Interactive Cinema (IIC)

© 2004 Immersion Studios Inc. 2-2

Immersion is simply referred to here as a state of being encompassed

Video Games Most Websites

Virtual Reality Non-fiction

© 2004 Immersion Studios Inc. 2-3

What is sensory immersion?

Cyclorama of Early Melbourne, Australia

Detail of a Cyclorama of Early Melbourne, Australia

© 2004 Immersion Studios Inc. 2-4

Toronto, Jerry Angler developed a system for full 360-degree

What is data immersion?

3 Digital video, stills, CG production, etc.

© 2004 Immersion Studios Inc. 2-5

environment when processed through the associative knowledge a

© 2004 Immersion Studios Inc. 2-6

Understanding group interactivity

8 Barrett, E, (1994). Sociomedia Multimedia, Hypermedia, and the Social Construction of

© 2004 Immersion Studios Inc. 2-7

When personal social interaction with media is discussed, the term

Collaborative group interactivity

© 2004 Immersion Studios Inc. 2-8

of collaboration involves the division of tasks between two or more

In many over-simplified group interactivity immersive environments,

© 2004 Immersion Studios Inc. 2-9

Collaborative results compared against personal choices in an ecological discovery

Linking data to representations is more easily accomplished, as