A design advice tool presenting usability guidance for

virtual environments
Kulwinder Kaur Deol, Alistair Sutcliffe and Neil Maiden
Centre for HCI Design, City University, London
Abstract
There is a lack of design advice on how to construct usable virtual environment
(VE) systems, even though major usability problems have been reported. This
paper details the development and evaluation of a design advice tool for
presenting usability guidance to VE designers. The guidance was derived from
theoretical work on human-computer interaction in VEs, which detailed stages of
interactive behaviour and design requirements to support the stages. The design
requirements were translated into concrete design guidelines and a hypertext tool
developed to present the guidelines. The design of the tool incorporated
knowledge of design practice from studies of VE designers. The usability and
utility of the tool was evaluated. Industrial designers found the tool and
guidelines to be useful in highlighting usability considerations and in providing a
checklist for different design elements. Areas of improvement and extensions to
the tool were suggested.
Keywords: Virtual Environments, Usability, Guidelines, Design Assistance
1 Introduction
Virtual environments provide a computer-based interface representing a three-
dimensional real-life or abstract space. VEs offer new possibilities and challenges to
human-computer interface design, but have been noted for being significantly more
difficult to design and use than 2D interfaces (Herndon et al., 1994). In our previous
work, we carried out studies of the design of VEs (Kaur et al., 1996) which showed that
designers lacked a coherent approach to interaction design. They appeared to be pre-
occupied with difficult technical issues and thought little about supporting user
interaction. However, major interaction problems have been found with current VEs,
such as disorientation, perceptual misjudgements and difficulty finding available
interactions (COVEN, 1997; Miller, 1994). These problems result in user frustration and
a low usability and acceptability for the VE (Kaur et al., 1996; Miller, 1994).
VE designers need guidance which highlights relevant usability issues and advises on
how to address them. Conventional guidance is only partially applicable to VEs and does
not cover the range of issues and phenomena that arise in VE interaction. For example,
VE users face further problems in perception, orientation and object manipulation
because of the added spatial dimension. Movement through the VE and the locating of
objects is a major area of potential difficulty, and the novel and more complex interface
technology results in additional difficulties. There is little usability guidance specifically
for VEs and only fragmentary knowledge of user issues. Most research so far has been
directed to requirements for supporting perception and wayfinding, and the design of
viewpoint controls and user representations (e.g. Benford et al., 1995; Darken and Sibert,
1996; Drucker and Zeltzer, 1994; Rushton and Wann, 1993), as well as understanding
motion sickness in immersive VEs (e.g. Oman, 1993). Although, this research may lead
to detailed advice for specific parts of the VE design problem, advice on general usability
requirements is needed (Herndon et al., 1994) to cover the range of interaction tasks a
user may be engaged in, from goal-formation to perception and action. This paper
reports work towards a design adviser for VE usability. Usability guidelines were
developed from theoretical work on interaction in VEs. A hypertext tool was developed
to present the guidelines and tested with industrial VE designers.
2 Theory
The research focused on the more basic type of VE, i.e. single-user systems modelled on
real-world phenomena, since there was little previous work in this area. The theoretical
work and the related evaluation studies are briefly outlined here, but full details can be
found in Kaur (1998). Models of interaction for VEs were developed and from these
models abstract usability requirements were proposed. Normans (1988) basic plan-
based theory of action was elaborated to include required behaviours in VEs, in particular
exploratory and reactive behaviours. Three inter-connected models described the major
modes of interactive behaviour:
Task action model - describing behaviour in planning and carrying out specific actions
as part of user's task or current goal/intention. Requirements for 3D actions included
the need to approach and orient as required to action objects, and the possibility of
object inspection rather than manipulation actions.
Explore navigate model - describing opportunistic and less goal-directed behaviour
when the user explores and navigates through the environment. The user may have a
target in mind or observed features may arouse interest.
System initiative model - describing reactive behaviour to system prompts and events,
and to the system taking interaction control from the user (e.g. taking the user on a
pre-set tour of the environment).
The models were evaluated through user studies, comparing observed physical and
mental behaviour (from verbal protocols) with that predicted in the models. Major
predicted patterns of activity were supported and the models were refined in light of the
results (Kaur et al., 1999).
The interaction models provided a clear breakdown of interactive behaviour. The models
were used to systematically reason about what generic properties are required in a VE
design to support the user during each identified stage in interaction. Previous research
on human-computer interaction was used to inform required design properties, where
relevant. Forty-six design properties were identified in categories covering requirements
for: the user task, spatial layout, viewpoint and user representation, objects, system-
initiated behaviour, actions and action feedback. For example, the property identifiable
object states that an object should be easy to identify or recognise; clear navigation
pathways states that areas that the user can move through should be clear; and declared
system control commencement/ termination states that the beginning (and end) of a
session of system control (e.g. a guided tour) should be clearly cued. Use of the models
of interaction meant that the identified set of properties could cover a broad range of
interaction issues for VEs, including planning and task execution; exploration, navigation
and wayfinding; action identification and execution; object approach and inspection; and
event interpretation and response. The design properties were evaluated in a controlled
study where interaction success with two versions of a test application, with and without
implementation of the properties, was compared. Results were very encouraging and
showed a 66% reduction in usability problems and significantly improved task
performance, with the design properties present. Analysis of detailed results provided for
individual assessments of properties.
3 Design Advice
Having shown that the theory could be applied to improve interaction, the next step was
to provide the abstract theory advice in an appropriate form for designers. Guidelines can
be useful prompts for designers, by highlighting the variety of issues that need to be
considered. However, they need to be given with extra information, such as scoping
rules and caveats, to avoid the guidance being too vague or conflicting (Reisner, 1987).
Each abstract design property was translated into a concrete design guideline, which
included the design advice itself, motivation for the advice (based on supporting
interactive behaviour in the models), a context-of-use giving information about the
applicability of the advice for different applications, and two practical examples of
implementation of the guideline. A tool was designed to present the guidelines in a
suitable form for designers. Previous studies of VE development (Kaur et al., 1996) had
indicated that designers had varied backgrounds, but a generally poor understanding of
usability. They followed informal, iterative design approaches, with projects typically
having short development times (i.e. a number of weeks). Therefore, a presentation style
was required which would be simple to understand and allow quick, easy and flexible
access to the guidance.
As a result, a hypertext tool was designed to present the guidelines in a structured and
accessible form. The hypertext tool structured guidelines according to the environment
elements involved (e.g. events, objects and actions) and general development stages when
the guidelines would apply. A manageable amount of guidance (avg. 5 guidelines) was
given for each element type. Development stages were defined using knowledge of the
VE design process, from the designer study, and the common stages of activity found in
system development methods. The stages outlined were: define requirements, specify
components in VE, specify interactions, design components, design interactions, build
environment and evaluate environment. A demonstration version of the tool was
implemented in HyperText Modelling Language. It included 12 guidelines chosen to
cover a range of major issues for three element types: objects, actions and system control.
The corresponding design properties had been tested in the controlled study outlined
above and found to significantly reduce related problems. For each element type, a bullet
point list of guidelines was given, which was linked to full descriptive texts and screen-
shots illustrating examples. A navigation panel enabled rapid access to guidance for
different stages, although only the introduction and two design stages were available in
the demonstration version. For example, figure 1 shows the bullet point list for the object
guidelines and figure 2 shows the guideline for property identifiable object.
Figure 1: The bullet point list for the object guidelines in the tool.
Figure 2: Description of the guideline for the property identifiable object.
4 Evaluation Study
The tool and design guidelines were tested using expert evaluation and critiquing to gain
direct qualitative feedback about their usability and utility. Four industrial VE designers
with an average (median) of 3 years experience took part in a one-day study. The
designers were given nine design scenarios and asked to produce paper storyboards of
design solutions. They worked in groups and had access to the demonstration version of
the tool throughout. The scenarios included five for designing original elements (objects,
actions and system controls) from given requirements, two for re-designing elements
chosen by the designers from their previous experiences, and two for re-designing
elements from given descriptions of a current design (including the usability problems
with the design). For example, designers were asked to design a corridor object with
accessible and inaccessible doors and re-design the action of drawing a blind, which users
currently had difficulty working out how to do. Scenarios chosen by designers included
the action of adding a new card to a personal computer. After all scenarios had been
completed, designers filled in retrospective questionnaires eliciting their views about the
guidance and tool, and took part in a discussion session.
A simple scoring scheme was used to score the completed storyboards of design
solutions, to give a general indication of usability. The solutions were assessed for how
well each relevant guideline in the tool had been incorporated in the design solution.
Individual scores for guidelines were either 0 for no consideration of the associated
usability issues, 1 - for a partial or inadequate consideration, or 2 for a full
consideration of all associated usability issues. The retrospective questionnaires covered
important aspects of the usability and utility of HCI guidance, described in detail in Lim
and Long (1994). Learnability could not be covered because longitudinal data was not
available in this one-day study.
5 Results
Usability scores given ranged from 50 to 100%, median 75%. Most individual scores
were either 1 or 2 (53 of the 57), indicating that usually the guidelines were addressed to
some extent. Points were generally lost where designers had considered some but not all
associated usability issues with a guideline, or where there was missing information in
the storyboards.
Table 1 gives the average response to each aspect of usability and utility in the
retrospective questionnaires. Responses were generally positive, with averages for
individual areas ranging from 4.75 to 6.00 (on a scale of 1 to 7, where 7 represented the
most positive response). The average response for the usability questions was 5.3, and
for the questions on utility the average was 5.25. The most positive response was for the
question on how acceptable the tool would be in practice.
Question on avg. (1-7)
Usability accessibility 5.75
understandability 5.00
applicability 4.75
specificity - pitched at appropriate level 5.00
acceptability - compatible with design practice 6.00
Utility uncovered usability issues 5.75
helped address issues 5.25
helped validate usability of designs 5.25
improved usability of design 4.75
Table 1: Average responses to questions on the usability and utility of the guidance tool.
In the questionnaires and during the discussion, designers commented that the tool was
useful as a reminder of all the usability issues that needed to be considered for a design
element. Designers used the tool particularly for assessing design solutions with respect
to usability, although there was often direct (copycat) reuse of techniques from the
examples given. Some changes to the design of the hypertext in the tool were suggested
as well as useful extensions to the tool. Designers wanted more example techniques for
incorporating the guidelines, to enable re-use by copying. They also felt the guidance
could be extended to help address the issue of how faithfully to model reality, and the
possible trade-offs between adding support information and maintaining the realism and
naturalness of the VE.
6 Discussion
In this paper, we have outlined the development of usability guidelines for VEs, from
theoretical work on interaction behaviour. A prototype tool was designed to present the
guidelines to designers. The tool was evaluated, using design scenarios together with
storyboarding. The results were encouraging and provided useful qualitative feedback
about the guidance and tool. The design solutions produced generally had good usability
scores, indicating that the designers were able to incorporate interaction support in their
designs, through the use of the tool. Designers perceptions about the guidance tool were
very positive. They felt the design advice was usable and that it helped to analyse and
address the usability of VEs. They also felt it could have an important impact on the
design process and could fit in with current design practices. Therefore, the tool appears
to represent an effective way of delivering the theory-based knowledge to industrial
designers. The evaluation has also highlighted some possible improvements and
extensions.
Previous work had provided fragmentary knowledge about usability requirements in VEs
and focused on only certain parts of the VE design problem. There were tools for
constructing the graphics for VEs, but no tools presenting design advice directly to
designers. The design properties in this research provide a more broad and
comprehensive set of requirements that are based on a theoretical model of interaction.
The prototype tool represents early work towards an effective design advisor for VE
usability.
Future work includes further development of the guidance tool, by researching and
including a range of example techniques for implementing guidelines, for varying levels
of realism constraints. For example, various techniques have been used for providing
appropriate user representations, such as human bodies, block figures and detailed faces
(Benford et al., 1995). Different styles could be applied to incorporate properties, such as
manipulating levels of detail and realism, using highlighting techniques for attentional
design or active support techniques. Further evaluation work is also planned to test
complete versions of the guidance tool with more designers and in full development
projects. Our overall aim is to address problems in interaction design for VEs, through
guidance and tools based on human-computer interaction theory.
References
Benford S., Greenhalgh C., Bowers J., Snowdon D. and Fahlen L.E. (1995). User embodiment in
collaborative virtual environments. In: Human Factors in Computing Systems: CHI 95
conference, Denver 1995. Proceedings. New York: ACM, pp. 242-249.
COVEN (1997). D3.3: Usage evaluation of the initial applications. Public Deliverable report of
COVEN (Collaborative Virtual Environments) ACTS Project N.AC040.
Darken R.P. and Sibert J.L. (1996). Wayfinding strategies and behaviours in large virtual worlds.
In: Human Factors in Computing Systems: CHI 96 conference, Vancouver 1996.
Proceedings. New York: ACM, pp. 142-149.
Drucker S.M. and Zeltzer D. (1994). Intelligent camera control in a virtual environment. In:
Graphics Interface 94 conference, Banff, Alberta, 1994. Proceedings. Toronto: Canadian
Information Processing Society, pp. 190-199.
Herndon K.P., Van Dam A. and Gleicher M. (1994). The challenges of 3D interaction: a CHI'94
workshop. SIGCHI Bulletin 26(4): 36-43.
Kaur K. (1998). Designing virtual environments for usability. PhD thesis. Centre for HCI Design,
City University, London. June 1998.
Kaur K., Maiden N. and Sutcliffe A. (1999). Interacting with virtual environments: an evaluation
of a model of interaction. Interacting with Computers 11:403-426.
Kaur K., Maiden N. and Sutcliffe A. (1996). Design practice and usability problems with virtual
environments. In: Virtual Reality World 96 conference, Stuttgart, Germany, 1996.
Proceedings. Munich, Germany: IDG Conferences & Seminars.
Lim K.Y. and Long J. (1994). The MUSE method for usability engineering. (Cambridge, UK:
Cambridge University Press).
Miller L.D. (1994). In the context of the European project 'Metrics for Usability Standards in
Computing' (MUSiC): A usability evaluation of the Rolls-Royce virtual reality for aero
engine maintenance system. Masters thesis: University College London.
Norman D.A. (1988). The psychology of everyday things. (New York: Basic Books).
Oman C.M. (1993). Sensory conflict in motion sickness: an Observer Theory approach. In:
Pictorial Communication in virtual and real environments, ed. by S.R. Ellis, M. Kaiser and
A.J. Grunwald. (London: Taylor and Francis).
Reisner P. (1987). Discussion: HCI, what is it and what research is needed? In: Interfacing
thought: cognitive aspects of human-computer interaction, ed. by J.M. Carroll. (Cambridge,
Massachusetts: Massachusetts Institute of Technology Press).
Rushton S. and Wann J. (1993). Problems in perception and action in virtual worlds. In:
Virtual Reality International 93: third annual conference, London 1993. Proceedings.
London: Meckler, pp. 43-55.