View Management for Virtual and Augmented

Reality
Blaine Bell

Steven Feiner
Presented By
Team Avatar
Damien Rompapas

Tobias Hllerer

Overview
Background
Summary
View Management Explained
Object Properties and Constraints
View Management Approach

My opinion
Q&A

Background on the problem

Whats wrong with this interface?
Visibility => Bad
Clarity => Total Mess,
Very unclear
Content occludes
environment

Aim of the paper

Augmented/Virtual Reality systems with labels can be

messy.
How to place labels/Annotations in 3D environments?
Key constraints:
Visibility
Clarity

Implementation of View Management System

Multi-User environment

Overview
Background
Summary
View Management Explained
Object Properties and Constraints
View Management Approach

My opinion
Q&A

View Management?
By View Management we mean the following:
Adjusting objects in our field of view to get a
better understanding of our intended target or
information in relation to the current scene

View Management?
We often make view management tasks daily, usually
Subconsciously

View Management
System aims to automatically perform such tasks to
maintain visual constraints on the projections of labels
and objects on the viewing plane so that they
- are near relative objects
- dont occlude each other.
- satisfy applied constraints
(described later)

Viewing plane?

OpenGL Theory Tutorial - http://www.falloutsoftware.com/tutorials/gl/gl0.htm

Overview
Background
Summary
View Management Explained
Object Properties and Constraints
View Management Approach

My opinion
Q&A

Object Properties and Constraints

Display objects have properties tagged as
Controllable
Constrained
By the
User
View-Management Component
Other (tracker, simulation ect..)

Object Properties and Constraints

Constraint Types
Visibility
Position
Size
Transparency
Priority

Visibility
Whether an object can/cannot occlude other
defined objects on the viewing plane

Position
Min/Max distance from
Other Objects
Point/Area/Volume
Screen-Stabilized
Users Pose
World-Stabilized
World Pose

Size/Transparency
Range of possible sizes
- Preference towards the high end of range
Range of object transparency values
- Can be modified to minimize

Priority
Determine which order objects are occluded.
=> Useful for satisfying other constraints

Overview
Background
Summary
View Management Explained
Object Properties and Constraints
View Management Approach

My opinion
Q&A

View Management Approach

Ensure constraints in dynamic environment.
Adjust position &
size of objects at
every frame

View-Plane Representation
/Visible Surface Determination
Each 3D Object: Upright rectangular extent of its projection on the
view plane. (largest empty-space rectangles),
Sorted using BSP Tree for visibility representation.

Largest empty-space rectangles

Given an incrementally specified input set of possibly overlapping, axis-aligned, rectangles. It
automatically maintains an efficient representation of the dual of this set of rectangles: the area
that has not been rendered.
This is noted as the Largest empty space rectangle
Any object rendered in this region will not occlude other objects
Each largest rectangle lies along the outer edge of any input rectangle.

Suppose that:
axis-aligned rectangular extents of the
projections of all scene objects
have been added to the representation,
and that we are given the extent A
of some new objects projection.

Largest empty-space rectangles

Multiple empty-space rectangles
can be merged to create a larger
Empty-space rectangle

BSP Tree
Best known in the video game Doom

Standard Binary Tree, Sort/Search for polytypes n n-dimensional space

Tree as whole represents entire state

Each node stores a Hyperplane, dividing the space into two halves

Internal & External Labelling

Area Feature Label/Annotations

Temporal Continuity
Each frame is computed independently.
Frames computed independently result in
discontinuous changes in:
- Size
- Position
This can result in jumpy and
confusing transitions
between states.

Temporal Continuity
Steps to solve using previous frame
- State Hysteresis
- Positional Stability
- Interpolation

State Hysteresis
There are several situations in which objects may change state,
resulting in a discrete visual change,
such as from an internal label to external one,
or from being displayed to not being displayed.

They borrow the approach modify the definitions of:

Absolute minimum Size (absMin)
Sustained minimum Size (min)

Positional Stability
To keep the labels clear,
They are placed without opportunity of jitter.
Two situations:
Best Possible Layout independent of
the previous layout.
Closest t Possible layout to previous
layout.

For Label L attached to Object A

=> L Uses largest available space
within A
L Uses largest available space
outside A

Interpolation
To minimize the effect of discontinuous jumps
Interpolate the values of all constraints between
frames

Implementation
Hardware
CPU:
1.4 GHz Pentium 4
RAM:
512 MB Ram
Graphics: SONICBlue FireGL 2
Software
OS: Windows 2K
View Management: Java
Rendering: Java 3D
View Management/Rendering
Seperated.

Overview
Background
Summary
View Management Explained
Object Properties and Constraints
View Management Approach

My opinion
Q&A

Fantastic
Very General Problem
The paper goes in-depth on
expressing this with previous work.
Very applicable
(Surprised current systems dont
implement)
Can improve on bad UI
Representation: Second Slide.

Great
Uses the world in miniature (WIM)
to assist in orientation and
labelling environment
Labels Jump between real world and
world in miniature

Great
Provides relation to research and human
habits to generate overview, easily able to
understand context of the problem.
My favorite paragraph in the paper.
In the real world, we make simple view
management decisions routinely, and often
subconsciously. For example, we push aside a
centerpiece to enable eye contact with others
at dinner, or we hold a city guidebook so that
we can read a relevant section, while
simultaneously viewing an historic building
that it describes as we pass by on a bus

Great
Detailed Description of the method
involved, extends on research from
2D environments
Open to re-implementation
(Source code not available, explained
later)

Good
Paper does show one comparison
between their labelling and other
typical labelling methods.
(Left to right: Nave,Suppression, View Management)

(Why is this the only one though?)

Good
Wide Variety of references, Some of
them are good papers!
B. Bell and S. Feiner. Dynamic space management for
user interfaces. In Proc. ACM UIST 2000

Many parts of this paper are included and

extended in this research.

References include 2D environment, 3D,

Human Behaviour and Analysis, Even a book!
A famous book: Computer Graphics: Principles
and Practice

Strange
Lacks teaser image description,
Instead referred in Introduction
through repeat of images.
Most papers in the same year journal
(I checked about 30) didnt have a
teaser image, very strange

Doubtful
Lack of Usability Studies
They talk about this in future work

Our goal is to compare performance

with and without different versions of
the view-management component on
modeling tasks in cluttered
environments that would appear to
be good candidates for its use.
They already have a working system

Doubtful
Some parts of the paper were hard
to understand
Far too wordy, slightly overexplained.
Requires background knowledge
Had to read some of the referenced
papers before beginning to understand

Diagrams can be better explained

Doubtful
The video shown was recorded
using a HMD on a tripod.
Prevent rapid shaking
How Stable the positional stability is

Bad
They very briefly talked about, but didnt
show the GUI used for adding/creating
annotations.
(As far as the paper is aware)
We dont know how difficult it is to add
these annotations, it could all be textbased?
If its complex, it could make the system
useless for most use cases.
William Chiong and Yoav Hirsch developed the
software infrastructure that we used to create a GUI for
specifying user parameters.

On the previous note

This video below only briefly
shows the interface, which looks
complicated. (Live Demonstration)

Sad
This system is so good!
Why arent we using it?

Possibilities
Computation Power

Very Low cost, running on low powered (Ancient)

machine

Annotation Creation GUI

We dont know how complex/time consuming for

the creation of annotations.

Live annotation creation

Didnt demonstrate On The Fly annotations

Paper Mentions Funded By ONR Contracts

American Naval Research Company

Might be top-secret (In which case, why a paper?)

Related Observation
The Labelling system generated using
this view-management system tends
to favor the outside edges of the
screen, something that many user
interfaces already do.

Why is it related?
If you added many more external
labels to the Journalism tag,
Wouldnt we end up with
a cluttered and confusing labelled
environment, akin to my second slide?
The problem may be unavoidable.

Another personal opinion

This paper really isnt a Research
Paper
Doesnt present Hypothesis/User
Study etc.
Rather it presents a Novell
implementation/Solution to a
problem.
It doesnt stress a significant
improvement over other methods
However it is clearly visible.

Final Opinion
What makes a system good and easy to use can
depend entirely on how information is
represented on the screen as well as the quality,
too much can be intimidating and confusing, too
little is a waste of space.
This paper opens up a new automated method
to represent such information .

Discussion Points (Q&A)

Fantastic
Very general problem

Great
Detailed Description of the method involved

Good
Wide Variety of references

Strange
Lacks teaser image description

Doubtful
Lack of Usability Studies
Paper hard to understand

Bad
GUI not explained in detail

Sad
We dont use the system.