GOLDEN ANNIVERSARY SPECIAL ISSUE

Cognitive Architectures: Choreographing the Dance of

Mental Operations With the Task Environment

Wayne D. Gray, Rensselaer Polytechnic Institute, Troy, New York

Objective: In this article, I present the ideas and trends that have given rise to the use of
cognitive architectures in human factors and provide a cognitive engineeringoriented
taxonomy of these architectures and a snapshot of their use for cognitive engineering.
Background: Architectures of cognition have had a long history in human factors
but a brief past. The long history entails a 50-year preamble, whereas the explosion
of work in the current decade reflects the brief past. Understanding this history is key
to understanding the current and future prospects for applying cognitive science the-
ory to human factors practice. Method: The review defines three formative eras in
cognitive engineering research: the 1950s, 1980s, and now. Results: In the first era,
the fledging fields of cognitive science and human factors emphasized characteristics
of the dancer, the limited capacity or bounded rationality view of the mind, and the
ballroom, the task environment. The second era emphasized the dance (i.e., the dynamic
interaction between mental operations and task environment). The third era has seen
the rise of cognitive architectures as tools for choreographing the dance of mental
operations within the complex environments posed by human factors practice.
Conclusions: Hybrid architectures present the best vector for introducing cognitive
science theories into a renewed engineering-based human factors. Application: The
taxonomy provided in this article may provide guidance on when and whether to
apply a cognitive science or a hybrid architecture to a human factors issue.

INTRODUCTION mechanisms, and a much larger capacity in the

Cognitive modeling began in the mid-1950s
as a tool for studying the relationship between the
structure of the environment and behavior. The
work of Herbert Simon and Paul Fitts exempli-
fies the shared view of limited human capacities
and the importance of the task environment
that characterized cognitive modeling in the
emerging cognitive science and human factors
communities.
In his influential Psychological Review pa-
per, Simon (1956) championed the emerging
information-processing approach that would later
become cognitive psychology and contrasted it
with the models postulated by economists, saying
that in almost all respects the latter [economists]
postulate a much greater complexity in the choice
organism for obtaining information and perform-
ing computations, than do the former [psycholo-
gists] (p. 129). The engineering psychologists
|of the day are well represented by Paul Fitts,
who, busily developing information-processing
approaches to modeling motor movement, con-
cluded that the fixed information-handling ca-
pacity of the motor system probably reflects a
fixed capacity of central mechanisms (Fitts, 1954,
p. 391).
Simons (1956) main point in his article, that
adaptive behavior depends upon the structure of
the environment (p. 130), was also one shared by
Fitts (1958), who defined the aims of engineering
psychology as providing quantitative data on
human performance characteristics . . . as a func-
tion of task variables (p. 269).

Address correspondence to Wayne D. Gray, Rensselaer Polytechnic Institute, Carnegie Building, 110 8th St., Troy, NY 12180;
grayw@rpi.edu. HUMAN FACTORS, Vol. 50, No. 3, June 2008, pp. 497505. DOI 10.1518/001872008X312224. Copyright
2008, Human Factors and Ergonomics Society.
498
Although Simon and Fitts recognized two of
the three elements of the current approach to cog-
nitive science and cognitive engineering, it is
clear that they and their immediate intellectual
descendants emphasized the dancer (bounded
cognition) and the ballroom (the task environ-
ment) but largely ignored the dance that is, the
interplay of the diverse set of mental resources
with a dynamic task environment.
A sympathetic reading of history is that it
would have made little sense to worry about the
dance before we had a better understanding of the
dancers that is, our limited-capacity resources.
Indeed, this divide-and-conquer approach to hu-
man cognition did lead to advances in cognitive
theory and human factors practice (e.g., Fisk &
Schneider, 1981; Schneider, 1985; Shepard, 1963;
Sperling, 1963). However, as Newell (1973) point-
edly put it, You cant play 20 questions with
nature and win, an approach that focuses on the
dancers and ballroom but not the dance cannot
take us to our goal of understanding the human
mind at work in complex task environments.
Between the 1950s and the 1980s, human fac-
tors researchers began to believe that progress
could be made on persistent problems such as cog-
nitive workload by designing the task environ-
ment to facilitate the dance of multiple resources
(Klapp & Netick, 1988; Logan, 1988; Polson &
Friedman, 1988; Wickens & Liu, 1988; Wickens,
Mountford, & Schreiner, 1981; Wickens, Sandry,
& Vidulich, 1983). Likewise, the solid and reliable
human factors workhorse of task analysis (Kirwan
& Ainsworth, 1992; Shepherd, 1998, 2001) was
extended downward into the realm of immediate
interactive behavior, where cognitive, perceptual,
and motor processes interacted directly with each
other as well as with the task environment (Card,
Moran, & Newell, 1983).
Between the 1980s and now, cognitive archi-
tectures have emerged from the cognitive science
community (Anderson, 1983, 2007; Anderson &
Lebiere, 2003; Elman et al.,1996; Kieras & Meyer,
1997; McClelland & Rumelhart, 1986; Newell,
1990; OReilly & Munakata, 2000; Rumelhart
& McClelland, 1986; Schneider & Chein, 2003)
as a tool for basic research. Cognitive architec-
tures enable a type of cognitive modeling that
emphasizes the interplay and control of multiple
resources. Different architectures emphasize dif-
ferent sets of resources. Indeed, modeling the con-
trol of the integrated cognitive system is an active
June 2008 Human Factors
area of contemporary research and theory (Gray,
2007b; Monsell & Driver, 2000), with new archi-
tectures emerging and older architectures chang-
ing almost monthly. (An interesting discussion
of the limits and progress of architectures of cog-
nition is provided by the Newell Test, a list of 12
criteria for architectures proposed by Anderson
and Lebiere [2003], which was inspired by various
writings on this subject by Allen Newell.)
In the next section, we skip from Simon (1956)
and Fitts (1954) to the early 1980s. Here we discuss
the first attempts to choreograph the dance that
is, to put information gleaned about cognitive, per-
ceptual, and motor processes into a form that could
be used by human factors professionals. We then
jump ahead another 20 years to the new century to
discuss types of architectures and how they have
been used in cognitive engineering.
DEFINING THE DANCE STEPS OF A
COUPLED SYSTEM: TASK ANALYSIS
FOR COGNITIVE ENGINEERING
Once the strategy was selected, the course of the
search depended only on the structure of the
problem, not on any characteristics of the prob-
lem solver. (Simon, 1969, pp. 3031)
Simons (1969) statement places into words
an open secret long known by the human factors
community namely, that almost any human be-
havior that extends in time longer than a few min-
utes can be conceived of as a hierarchical series
of tasks and subtasks. The structure of this hier-
archy is, for the most part, determined by the
nature of the task and task environment and less
so by the human operator. Rather than having to
deal with hours of behavior as one unit, the human
factors analyst can break the behavior down to
the level required by the goals of the analysis.
However, before cognitive engineering could
take advantage of this open secret, cognitive sci-
ence had to define the right level of functional
analysis. During the 1960s, 1970s, and 1980s,
cognitive models tended to focus on operations
such as the problem-solving steps involved in
cryptarithmetic problems (Newell & Simon,
1972) or writing a statement in a programming
language (Anderson, Conrad, & Corbett, 1989).
Although such steps were structured by the tasks
being performed, they were at too high a level of
analysis to reveal the coupling of brain-based and
COGNITIVE ARCHITECTURES 499

world-based systems (Clark, 2007). For cognitive
engineering, this disembodied approach to cog-
nition changed radically with the publication of
The Psychology of Human-Computer Interaction
(Card et al., 1983).
Card et al. (1983) defined a new construct, the
unit task, that is fundamentally a control con-
struct, not a task construct (p. 386); that is, the
unit task partitions the behavior stream (p. 385)
beneath the level at which the task hierarchy is
defined by the task itself and at the level at which
task structure is defined by the control problems
faced by the user. Unit tasks range in duration
from 3 to 30 s, with an internal structure com-
posed of steps that range in duration from 1/3 to
3 s. Although many of the steps within a unit task
retain a purely in-the-head flavor, many steps
implicitly combine cognitive and perceptual or
cognitive and motor resources in interactions
with the task environment (for examples of re-
cent analyses of complex tasks at the unit task
level, see Sohn, Douglass, Chen, & Anderson,
2005; Taatgen & Lee, 2003).
In a subsequent development, the steps of a
unit task were fragmented into networks of cog-
nitive, perceptual, and motor operations, with the
operations of different steps interleaved in an
activity network formalism (Gray & Boehm-
Davis, 2000; Gray, John, & Atwood, 1993; John,
1990; Schweickert, Fisher, & Proctor, 2003). As
Figure 1 suggests, this formalism provided an
important step toward choreographing the dance
among cognitive, perceptual, motor, and envi-
ronmental resources.

Figure 1. A cognitive model of the first four steps of a unit task for a toll and assistance operator handling a phone
call: LISTEN-FOR-BEEP, READ-SCREEN(1), READ-SCREEN(2), and GREET-CUSTOMER. Each step is broken
down into the cognitive, perceptual, and motor operations required to perform that step in the phone operators task
environment (workstation and customer). The middle row shows cognitive operators with a default execution time
of 50 ms each. Above that line are the perceptual operators (separate lines for visual and aural operators), and below
it are the motor operators (separate lines for eye movement and speech; note that no hand movements are required
in this part of the unit task). The flow of operators is from left to right with connecting lines indicating dependen-
cies. Within an operator type, the dependencies are sequential. However, between operator types, the dependencies
may be parallel. The numbers above each operator indicate the time, in milliseconds, for that operator to execute.
Time is accumulated from left to right along the critical path. Differently textured backgrounds are used to group
the operators for each of the four steps. This grouping shows that the three operators of the LISTEN-FOR-BEEP
step are interleaved with the four operators of the READ-SCREEN(1) step.
500
It may seem curious that this major beachhead
for applying cognitive science theories to human
factors practice was in human-computer inter-
action (HCI) rather than mainstream human factors
(Card, Moran, & Newell, 1980a, 1980b; Card et
al., 1983; Newell & Card, 1985; Norman, 1981,
1982, 1986). In a sense, this beachhead represents
a simple case of science and engineering practice
following technological developments. The
1980s saw a shift of work of a large segment of
the white-collar workforce to computers. The use
of this technology allowed detailed, keystroke-
level logs to be collected and analyzed with the
same care and precision previously available only
in the experimental psychology laboratory. The
applications of these techniques to the larger
human factors community lagged their applica-
tion to HCI and had to await the digitization or
digital simulation of large-scale systems.
CHOREOGRAPHING THE DANCE:
COGNITIVE ARCHITECTURES FOR
COGNITIVE ENGINEERING
The impetus for developing and using archi-
tectures within the human factors community has
been our need to bring the engineering-style ap-
proach of the human factors community (Byrne &
Gray, 2003) into increasingly complex task en-
vironments. The types of cognitive architectures
deployed by the human factors community reflect
the ambitions of the researchers as well as the
scope of the problems they are tackling.
New architectures are born almost monthly,
but old ones never die. They just fade away. In
one of the first taxonomies of architectures, Pew
and Mavor (1998) identified and discussed 12.
Five years later, Ritter et al. (2003) added 7 to this
list, whereas Morrisons (2003) review added 7
more. A few years later, Langley, Laird, and
Rogerss (2006) review listed 15 architectures
that were not mentioned by the prior reviews.
Finally, the Wikipedia entry for cognitive archi-
tectures retrieved the week before this article was
submitted to the journal (February 22, 2008; http://
en.wikipedia.org/wiki/Cognitive_architecture)
adds 9 architectures that are not discussed by
these other four sources. Taken as a group, these
five sources review 50 architectures, and although
this set of 50 may be exhausting, it is certainly
not exhaustive. As I view an exhaustive listing to
be futile, my more modest goal is to provide the
June 2008 Human Factors
reader with a taxonomy (see Figure 2) that might
help sort through the various architectures while
providing a guide to the cognitive engineering con-
tributions that might be expected from different
categories of this taxonomy.
Architectures for Developing Cognitive
Science Theory (Figure 2, Box 1)
The vast majority of architectures are devel-
oped by one or two researchers as research tools
for exploring and developing cognitive science
theory. Of these, most are used only by the devel-
opers and their students and are used only for
basic research. This category, Architectures Only
Used for Basic Research (Figure 2, Box 1.A), is
outside the scope of this review and will not be
discussed further.
Of more interest to the readers of this journal
is the category of Architectures Occasionally
Used for Cognitive Engineering (Figure 2, Box
1.B). This category entails the application of
basic research systems such as ACT-R, EPIC,
Soar, the Queuing Network Model Human Pro-
cessor (QN-MNP) (Liu, Feyen, & Tsimhoni, 2006),
or connectionist architectures (e.g., Schneider
& Chein, 2003) out of the box, with no signif-
icant changes to the underlying architecture.
In terms of application domains, successes of
this approach have included the modeling of pilot
performance in uninhabited air vehicles (Gluck,
Ball, & Krusmark, 2007), taxiing on runways by
commercial airline pilots (Byrne & Kirlik, 2005),
flight deck displays and radar systems (Fu et al.,
2006; Gray, Schoelles, & Myers, 2002; Schoelles
& Gray, 2001; Wickens, Sebok, Bagnall, & Ka-
mienski, 2007), improving the design of handheld
devices (St. Amant, Horton, & Ritter, 2007), the
classic human factors topic of graph reading
(Peebles & Cheng, 2003), and the closely related
topic of diagrammatic reasoning (Ritter & Bibby,
in press).
This approach has also been applied to model
various effects of interest to the human factors
community. Such effects include multitasking
(Kieras, Meyer, Ballas, & Lauber, 2000; Liu et
al., 2006), interruptions (Altmann & Trafton,
2002), task switching (Altmann & Gray, 2008, in
press), mental workload (Wu & Liu, 2006a), human
error (Byrne & Bovair, 1997; Byrne & Davis,
2006; Gray, 2000), menu search (Byrne, 2001;
Hornof, 2001), and dual-task performance
(Schneider & Detweiler, 1988). To a large degree,
COGNITIVE ARCHITECTURES 501

Figure 2. Taxonomy of architectures for cognitive engineering. Most of the 50 architectures found in the survey of
reviews would be considered Architectures Only Used for Basic Research (Box 1.A). This branch is not elaborated
in this figure, as by definition its architectures have not been applied to cognitive engineering purposes.

success in these types of efforts reflects the suc-
cessful advance in the ability of cognitive theory
to account for increasingly complex behavior in
increasingly complex task environments. The
limitations of this approach help emphasize the
limits of theory-based architectures in account-
ing for the whole of human cognition (see also
the comment by Gray, Schoelles, & Myers, 2003,
on the Anderson & Lebiere, 2003, Newell Test).
Hybrid Architectures for Cognitive
Engineering (Figure 2, Box 2)
Hybrid architectures freely mix components
derived from machine learning, control theory,
artificial intelligence, mathematical modeling, or
standard programming techniques with compo-
nents that are firmly grounded in cognitive theory.
Gray and colleagues have referred to such hybrids
as simBorgs (Gray, Schoelles, & Veksler, 2004;
Myers, Neth, Schoelles, & Gray, 2004). As cogni-
tive architectures were created as a tool with which
to study the control of the integrated cognitive
system (Gray, 2007a; Kieras, 2007), a major dis-
tinction among hybrid architectures is whether
control and interleaving of the components of the
architecture are based on cognitive theory or
whether this control is one of the noncognitive,
hybrid, components of the architecture.
Hybrids emphasizing cognitive theories of the
control of cognitive systems (Figure 2, Box 2.A).
Those systems that largely conform to the control
imposed by cognitive theory include the ACT-R-
based and QN-MNP-based models of driver
performance (Salvucci, 2006; Wu & Liu, 2006b;
Wu, Tsimhoni, & Liu, 2007). In contrast, the
SNIF-ACT models of information search replace
the ACT-R 5.0 (Anderson & Lebiere, 1998)
model of control with a theory-based alternative
(Fu & Pirolli, 2007; Pirolli, 2007).
Notable in this category is the Soar family of
hybrid architectures (Wray & Jones, 2006) that
maintain the control structure of Soar while devel-
oping models that generate humanlike behavior
for large-scale military simulations. These models
include TACAIR-Soar (Jones et al., 1999; Tambe
et al., 1995) and MOUTBots (Wray, Laird, Nuxoll,
Stokes, & Kerfoot, 2005).
The subcategory of hybrids that emphasize
cognitive control in dynamic task environments
502
is Interactive Cognitive Agents (Figure 2, Box
2.A.1). This interaction requires the careful
choreographing of multiple cognitive, perceptual,
and motor resources. Interestingly, all of the sys-
tems discussed in this section (i.e., Figure 2, Box
2.A) belong to this taxonomic subcategory. Al-
though I was unable to find nonembodied exem-
plars of this category, they seem feasible (and are
represented in Figure 2, Box 2.A.2). For example,
one could imagine modeling the interplay be-
tween bounded memory and bounded attention
during a complex decision-making task. If such
decision making did not rely on external resources,
then the model would be a member of this category
of hybrids (Box 2.A) but would not be considered
an interactive cognitive agent (Box 2.A.1).
Hybrids incorporating noncognitive control
of some cognitive components (Figure 2, Box
2.B). More common in cognitive engineering
practice are hybrid architectures that avoid the
troublesome issues of modeling the human con-
trol system and rely on standard computer science
techniques for controlling a complex software
system. The scope of such systems varies widely
along a number of dimensions.
A notable example (Figure 2, Box 2.B.1) of a
very specialized hybrid is VDM2000 (Witus &
Ellis, 2003). This may be considered an architec-
ture in that it consists of subcomponents that
model human early vision, object recognition,
and basic psychophysics to provide a criterion-
independent measure of target conspicuity (Witus
& Ellis, 2003, p. 47). VDM2000 is very successful
at what it does namely, accurately predict the
detection of hidden military targets, in daylight,
by the unaided human eye.
A class of hybrids with a long lineage within
the cognitive engineering community includes
COGNET/iGEN (Zachary et al., 2005; Zachary,
Ryder, & Hicinbothom, 1998) and the SAINT fam-
ily (SAINT, MicroSAINT, IMPRINT, etc.). (For an
excellent and recent discussion of these systems,
see Pew, 2007.) The control structure for these
architectures is explicitly not modeled on human
control but takes the form of an activity network
(Schweickert et al., 2003) (see Figure 2, Box 2.B.2).
These architectures were developed to simulate
the human element in large man-machine systems.
As such, details of individual human cognition were
not viewed as particularly important. Rather, the
human element could be incorporated by a broad-
brush approach that captured human work times
June 2008 Human Factors
and reliability as one system among many others.
Likewise, these systems tended to be descriptive,
not generative. If even small changes were made
in the task environment, the model would have to
be amended before it could perform the task.
(Note that although not developed to simulate the
human element in large man-machine systems,
the CPM-GOMS models that were discussed
earlier [Gray & Boehm-Davis, 2000; Gray et al.,
1993] also have an activity network structure and
also fall prey to this criticism.) However, it is very
much the case that this state of affairs is chang-
ing, and the components of these architectures
are increasingly based on cognitive theory. For
an update on the state of affairs in this line of
research, I refer the reader to the chapter by
Zachary et al. (2005) as well as to recent reviews
by Laughery and Lebiere (Laughery, Lebiere,
& Archer, 2006; Lebiere, Archer, Warwick, &
Schunk, 2005). However, it is not clear whether
this laudable drive to insert cognitive theory into
these architectures could be extended to include
a cognitive model of control without destroying
the power of this class of hybrids.
Proprietary or otherwise restricted (POOR) hy-
brids (Figure 2, Box 2.C). The last set of hybrids
is of a class that causes much trouble for the author.
These are hybrid architectures that are mentioned
in some of the reviews (for a listing and brief
description of many of these architectures, see
Morrison, 2003; Pew & Mavor, 1998; Ritter et al.,
2003) but appear to be proprietary architectures
that are seldom used outside of the group that
developed them. Published descriptions seem
limited to conference proceedings or technical
reports, with very little or no presence in peer-
reviewed publications such as Human Factors.
Tutorials are nonexistent, and access to source
code or runnable versions of the system is un-
heard of. From the outside, it is not clear how
much of the humanlike behavior is based on cog-
nitive theory and how much is based on artificial
intelligence, machine learning, and mathematical
algorithms plus the intuitions of the programmers.
Hence, although such architectures are filling a
gap not covered by better known systems, their in-
fluence in the development of new hybrid and cog-
nitive sciencebased architectures seems to be
limited. (I thank Kevin Gluck [personal commu-
nication, April 28, 2008] for suggesting the label
and acronym for this category, which has the dual
advantage of being both descriptive and emotive.)
COGNITIVE ARCHITECTURES
SUMMARY AND CONCLUSIONS
The application of cognitive architectures to
human factors issues has had a long history but a
brief past. As traced here, the long history extends
back to Simon (1956) and Fitts (1954, 1958),
whereas the brief past is reflected by the explo-
sion of work in the current decade. In contrast
to cognitive modeling, cognitive architectures
enable us to model the role of multiple mental
resources (whether multiple cognitive resources
or a combination of cognitive, perceptual, and
motor resources) in determining the adaptation
of human performance in complex, reactive task
environments.
In this article, I distinguished between two
types of cognitive architectures: cognitive science
architectures based on cognitive theories (Figure
2, Box 1) and hybrid architectures that include
significant noncognitive components (Figure 2,
Box 2). The cognitive science architectures are
limited by the state of the art in cognitive theory.
Hence, the areas in which cognitive science
lacks strong theories, such as vision (though see
recent work by OReilly & Munakata, 2000),
language (though see Lewis, Vasishth, & Van
Dyke, 2006), stress and fatigue (though see
Gunzelmann, Gluck, Kershner, Van Dongen, &
Dinges, 2007; Gunzelmann, Gluck, Price, Van
Dongen, & Dinges, 2007; Ritter, Reifers, Klein, &
Schoelles, 2007), and more (Anderson & Lebiere,
2003; Gray et al., 2003), are areas in which cogni-
tive architectures are severely limited. Attempts to
bypass these limits for cognitive engineering pur-
poses have given rise to a variety of hybrid archi-
tectures. These hybrids overcome the limits in
cognitive science theory by incorporating a vari-
ety of computer-based control ideas, machine
learning algorithms, or artificial intelligence
techniques. In the near term, it is the hybrids that
would present the best vector for introducing cog-
nitive science theories into a renewed engineering-
based (Byrne & Gray, 2003) human factors.
ACKNOWLEDGMENTS
Thanks to Hansjrg Neth for his close reading
and thoughtful comments on an earlier version of
this article. The work was supported, in part, by
Grant N000140710033 to Wayne Gray from the
Office of Naval Research, Dr. Ray Perez, project
officer.
503
REFERENCES
Altmann, E. M., & Gray, W. D. (2008, in press). An integrated model
of cognitive control in task switching. Psychological Review, 115.
Altmann, E. M., & Trafton, J. G. (2002). Memory for goals: An
activation-based model. Cognitive Science, 26, 3983.
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA:
Harvard University Press.
Anderson, J. R. (2007). How can the human mind occur in the physical
universe? New York: Oxford University Press.
Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition
and the LISP Tutor. Cognitive Science, 13, 467505.
Anderson, J. R., & Lebiere, C. (Eds.). (1998). Atomic components of
thought. Hillsdale, NJ: Erlbaum.
Anderson, J. R., & Lebiere, C. (2003). The Newell Test for a theory of
cognition. Behavioral & Brain Sciences, 26, 587640.
Byrne, M. D. (2001). ACT-R/PM and menu selection: Applying a
cognitive architecture to HCI. International Journal of Human-
Computer Studies, 55, 4184.
Byrne, M. D., & Bovair, S. (1997). A working memory model of a
common procedural error. Cognitive Science, 21, 3161.
Byrne, M. D., & Davis, E. M. (2006). Task structure and postcompletion
error in the execution of a routine procedure. Human Factors, 48,
627638.
Byrne, M. D., & Gray, W. D. (2003). Returning human factors to an
engineering discipline: Expanding the science base through a new
generation of quantitative methods Preface to the special section.
Human Factors, 45, 14.
Byrne, M. D., & Kirlik, A. (2005). Using computational cognitive
modeling to diagnose possible sources of aviation error. International
Journal of Aviation Psychology, 15, 135155.
Card, S. K., Moran, T. P., & Newell, A. (1980a). Computer text
editing: An information processing analysis of a routine cognitive
skill. Cognitive Psychology, 12, 3274.
Card, S. K., Moran, T. P., & Newell, A. (1980b). The keystroke-level
model for user performance time with interactive systems.
Communications of the ACM, 23, 396410.
Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of
human-computer interaction. Hillsdale, NJ: Erlbaum.
Clark, A. (2007). Curing cognitive hiccups: A defense of the extended
mind. Journal of Philosophy, 104, 163192.
Elman, J. L., Bates, E.A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D.,
& Plunkett, K. (1996). Rethinking innateness: A connectionist
perspective on development. Cambridge, MA: MIT Press.
Fisk, A. D., & Schneider, W. (1981). Control and automatic processing
during tasks requiring sustained attention: A new approach to
vigilance. Human Factors, 23, 737750.
Fitts, P. M. (1954). The information capacity of the human motor
system in controlling the amplitude of movement. Journal of
Experimental Psychology, 47, 381391.
Fitts, P. M. (1958). Engineering psychology. Annual Review of
Psychology, 9, 267294.
Fu, W.-T., Bothell, D., Douglass, S., Haimson, C., Sohn, M. H., &
Anderson, J. (2006). Toward a real-time model-based training
system. Interacting with Computers, 18, 12151241.
Fu, W.-T., & Pirolli, P. (2007). SNIF-ACT: A cognitive model of user
navigation on the World Wide Web. Human-Computer Interaction,
22, 355412.
Gluck, K. A., Ball, J. T., & Krusmark, M. A. (2007). Cognitive control
in a computational model of the predator pilot. In W. D. Gray
(Ed.), Integrated models of cognitive systems (pp. 1328). New
York: Oxford University Press.
Gray, W. D. (2000). The nature and processing of errors in interactive
behavior. Cognitive Science, 24, 205248.
Gray, W. D. (2007a). Composition and control of integrated cognitive
systems. In W. D. Gray (Ed.), Integrated models of cognitive
systems (pp. 312). New York: Oxford University Press.
Gray, W. D. (Ed.). (2007b). Integrated models of cognitive systems.
New York: Oxford University Press.
Gray, W. D., & Boehm-Davis, D. A. (2000). Milliseconds matter: An
introduction to microstrategies and to their use in describing and
predicting interactive behavior. Journal of Experimental Psychology:
Applied, 6, 322335.
Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine:
Validating a GOMS analysis for predicting and explaining
real-world performance. Human-Computer Interaction, 8, 237309.
504
Gray, W. D., Schoelles, M. J., & Myers, C. W. (2002). Computational
cognitive models ISO ecologically optimal strategies. In Pro-
ceedings of the Human Factors and Ergonomics Society 46th
Annual Meeting (pp. 492496). Santa Monica, CA: Human Factors
and Ergonomics Society.
Gray, W. D., Schoelles, M. J., & Myers, C. W. (2003). Meeting Newells
other challenge: Cognitive architectures as the basis for cognitive
engineering. Behavioral & Brain Sciences, 26, 609610.
Gray, W. D., Schoelles, M. J., & Veksler, V. D. (2004). Simborgs: Towards
the building of simulated human users for interactive systems
design. In Proceedings of the Human Factors and Ergonomics
Society 48th Annual Meeting (pp. 362366). Santa Monica, CA:
Human Factors and Ergonomics Society.
Gunzelmann, G., Gluck, K. A., Kershner, J., Van Dongen, H. P. A., &
Dinges, D. F. (2007). Understanding decrements in knowledge
access resulting from increased fatigue. In D. S. McNamara &
J. G. Trafton (Eds.), Proceedings of the 29th Annual Meeting of the
Cognitive Science Society (pp. 329334). Austin, TX: Cognitive
Science Society.
Gunzelmann, G., Gluck, K. A., Price, S., Van Dongen, H. P. A., &
Dinges, D. F. (2007). Decreased arousal as a result of sleep
deprivation. In W. D. Gray (Ed.), Integrated models of cognitive
systems (pp. 243253). New York: Oxford University Press.
Hornof, A. J. (2001). Visual search and mouse-pointing in labeled
versus unlabeled two-dimensional visual hierarchies. ACM
Transactions on Computer-Human Interaction, 8, 171197.
John, B. E. (1990). Extensions of GOMS analyses to expert perfor-
mance requiring perception of dynamic visual and auditory
information. In Proceedings of the ACM CHI 90 Conference on
Human Factors in Computing Systems (pp. 107115). New York:
ACM Press.
Jones, R. M., Laird, J. E., Nielsen, P. E., Coulter, K. J., Kenny, P., &
Koss, F. V. (1999). Automated intelligent pilots for combat might
simulation. AI Magazine, 20, 2741.
Kieras, D. E. (2007). Control of cognition. In W. D. Gray (Ed.), Inte-
grated models of cognitive systems (pp. 327355). New York:
Oxford University Press.
Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC
architecture for cognition and performance with application to human-
computer interaction. Human-Computer Interaction, 12, 391438.
Kieras, D. E., Meyer, D. E., Ballas, J. A., & Lauber, E. J. (2000). Modern
computational perspectives on executive mental processes and
cognitive control: Where to from here? In S. Monsell & J. Driver
(Eds.), Control of cognitive processes: Attention and performance
XVIII (pp. 681712). Cambridge, MA: MIT Press.
Kirwan, B., & Ainsworth, L. K. (Eds.). (1992). A guide to task analy-
sis. Washington, DC: Taylor & Francis.
Klapp, S. T., & Netick, A. (1988). Multiple resources for processing and
storage in short-term memory. Human Factors, 30, 617632.
Langley, P., Laird, J. E., & Rogers, S. (2006). Cognitive architectures:
Research issues and challenges. Retrieved April 28, 2008, from
http://cll.stanford.edu/~langley/papers/final.arch.pdf
Laughery, K. R., Lebiere, C., & Archer, S. (2006). Modeling human
performance in complex systems. In G. Salvendy (Ed.), Handbook
of human factors (pp. 967996). New York: Wiley.
Lebiere, C., Archer, R., Warwick, W., & Schunk, D. (2005). Integrating
modeling and simulation into a general-purpose tool. In Pro-
ceedings of the 11th International Conference on Human-Computer
Interaction [CD-ROM]. Mahwah, NJ: Erlbaum.
Lewis, R. L., Vasishth, S., & Van Dyke, J. A. (2006). Computational
principles of working memory in sentence comprehension. Trends
in Cognitive Sciences, 10, 447454.
Liu, Y., Feyen, R., & Tsimhoni, O. (2006). Queueing network-model
human processor (QN-MNP): A computational architecture for
multitask performance in human-machine systems. ACM Trans-
actions on Computer-Human Interaction, 13, 3770.
Logan, G. D. (1988). Automaticity, resources, and memory: Theoretical
controversies and practical implications. Human Factors, 30,
583598.
McClelland, J. L., & Rumelhart, D. E. (Eds.). (1986). Parallel distributed
processing: Explorations in the microstructure of cognition (Vol.
2). New York: MIT Press/Bradford.
Monsell, S., & Driver, J. (Eds.). (2000). Control of cognitive processes:
Attention and performance XVIII. Cambridge, MA: MIT Press.
Morrison, J. E. (2003). A review of computer-based human behavior
representations and their relation to military simulations (P-3845).
Alexandria, VA: Institute for Defense Analysis.
June 2008 Human Factors
Myers, C. W., Neth, H., Schoelles, M. J., & Gray, W. D. (2004). The
simBorg approach to modeling a dynamic decision-making task.
In Proceedings of the 6th International Conference on Cognitive
Modeling (pp. 372373). Pittsburgh, PA: Erlbaum.
Newell, A. (1973). You cant play 20 questions with nature and win:
Projective comments on the papers of this symposium. In
W. G. Chase (Ed.), Visual information processing (pp. 283308).
New York: Academic Press.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA:
Harvard University Press.
Newell, A., & Card, S. K. (1985). The prospects for psychological
science in human-computer interaction. Human-Computer Inter-
action, 1, 209242.
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood
Cliffs, NJ: Prentice Hall.
Norman, D. A. (1981). Comments on cognitive engineering, the need
for clear system images, and the safety of nuclear power plants.
In Conference Record for 1981 IEEE Standards Workshop on
Human Factors and Nuclear Safety (pp. 9192, 211212). New
York: IEEE.
Norman, D. A. (1982). Steps toward a cognitive engineering: Design
rules based on analyses of human error. In Proceedings of the 1982
Conference on Human Factors in Computing Systems (pp.
378382). Gaithersburg, MD: ACM Press.
Norman, D. A. (1986). Cognitive engineering. In D. A. Norman &
S. W. Draper (Eds.), User centered system design: New perspectives
on human-computer interaction (pp. 3161). Hillsdale, NJ: Erlbaum.
OReilly, R. C., & Munakata, Y. (2000). Computational explorations
in cognitive neuroscience: Understanding the mind by simulating
the brain. Cambridge, MA: MIT Press.
Peebles, D., & Cheng, P. C. H. (2003). Modeling the effect of task and
graphical representation on response latency in a graph reading
task. Human Factors, 45, 2846.
Pew, R. W. (2007). Some history of human performance modeling. In
W. D. Gray (Ed.), Integrated models of cognitive systems (pp.
2944). New York: Oxford University Press.
Pew, R. W., & Mavor, A. S. (Eds.). (1998). Modeling human and
organizational behavior: Application to military simulations.
Washington, DC: National Academy Press.
Pirolli, P. (2007). Information foraging theory: Adaptive interaction with
information. New York: Oxford University Press.
Polson, M. C., & Friedman, A. (1988). Task-sharing within and between
hemispheres: A multiple-resources approach. Human Factors, 30,
633643.
Ritter, F. E., & Bibby, P. (in press). Modeling how, when, and what
learning happens in a diagrammatic reasoning task. Cognitive
Science.
Ritter, F.E., Reifers, A.L., Klein, L.C., & Schoelles, M.J. (2007). Lessons
from defining theories of stress for cognitive architectures. In
W. D. Gray (Ed.), Integrated models of cognitive systems
(pp. 254262). New York: Oxford University Press.
Ritter, F. E., Shadbolt, N. R., Elliman, D., Young, R., Gobet, F., &
Baxter, G. D. (2003). Techniques for modeling human and organi-
zational behaviour in synthetic environments: A supplementary
review. Wright-Patterson Air Force Base, OH: Human Systems
Information Analysis Center.
Rumelhart, D.E., & McClelland, J.L. (Eds.). (1986). Parallel distributed
processing: Explorations in the microstructure of cognition
(Vol. 1). New York: MIT Press/Bradford.
Salvucci, D. D. (2006). Modeling driver behavior in a cognitive
architecture. Human Factors, 48, 362380.
Schneider, W. (1985). Training high-performance skills: Fallacies and
guidelines. Human Factors, 27, 285300.
Schneider, W., & Chein, J. M. (2003). Controlled & automatic
processing: Behavior, theory, and biological mechanisms. Cognitive
Science, 27, 525559.
Schneider, W., & Detweiler, M. (1988). The role of practice in dual-task
performance: Toward workload modeling in a connectionist/
control architecture. Human Factors, 30, 539566.
Schoelles, M. J., & Gray, W. D. (2001). Decomposing interactive behav-
ior. In Proceedings of the Twenty-Third Annual Conference of the
Cognitive Science Society (pp. 898903). Mahwah, NJ: Erlbaum.
Schweickert, R., Fisher, D. L., & Proctor, R. W. (2003). Steps toward
building mathematical and computer models from cognitive task
analyses. Human Factors, 45, 77103.
Shepard, R. N. (1963). Analysis of proximities as a technique for the
study of information-processing in man. Human Factors, 5, 3348.
COGNITIVE ARCHITECTURES
Shepherd, A. (1998). HTA as a framework for task analysis. Ergonom-
ics, 41, 15371552.
Shepherd, A. (2001). Hierarchical task analysis. New York: Taylor
& Francis.
Simon, H.A. (1956). Rational choice and the structure of the environment.
Psychological Review, 63, 129138.
Simon, H. A. (1969). The sciences of the artificial. Cambridge, MA:
MIT Press.
Sohn, M.H., Douglass, S.A., Chen, M.C., & Anderson, J.R. (2005). Char-
acteristics of fluent skills in a complex, dynamic problem-solving
task. Human Factors, 47, 742752.
Sperling, G. (1963). A model for visual memory tasks. Human Factors,
5, 1931.
St. Amant, R., Horton, T. E., & Ritter, F. E. (2007). Model-based eval-
uation of expert cell phone menu interaction. ACM Transactions
on Computer-Human Interaction, 14. Retrieved from http://doi.
acm.org/10.1145/1229855.1229856
Taatgen, N. A., & Lee, F. J. (2003). Production composition: A simple
mechanism to model complex skill acquisition. Human Factors,
45, 6176.
Tambe, M., Johnson, W. L., Jones, R. M., Koss, F., Laird, J. E., Rosen-
bloom, P. S., et al. (1995). Intelligent agents for interactive simulation
environments. AI Magazine, 16, 1539.
Wickens, C. D., & Liu, Y. L. (1988). Codes and modalities in multiple
resources: A success and a qualification. Human Factors, 30,
599616.
Wickens, C. D., Mountford, S. J., & Schreiner, W. (1981). Multiple
resources, task-hemispheric integrity, and individual differences in
time-sharing. Human Factors, 23, 211229.
Wickens, C. D., Sandry, D. L., & Vidulich, M. (1983). Compatibility
and resource competition between modalities of input, central pro-
cessing, and output. Human Factors, 25, 227248.
Wickens, C.D., Sebok, A., Bagnall, T., & Kamienski, J. (2007). Modeling
situation awareness supported by advanced flight deck displays. In
Proceedings of the Human Factors and Ergonomics Society 51st
Annual Meeting (pp. 794798). Santa Monica, CA: Human Factors
and Ergonomics Society.
Witus, G., & Ellis, R. D. (2003). Computational modeling of foveal
target detection. Human Factors, 45, 4760.
Wray, R. E., & Jones, R. M. (2006). Considering Soar as an agent
architecture. In R. Sun (Ed.), Cognition and multi-agent interac-
tion: From cognitive modeling to social simulation (pp. 5378). New
York: Cambridge University Press.
505
Wray, R. E., Laird, J. E., Nuxoll, A., Stokes, D., & Kerfoot, A. (2005).
Synthetic adversaries for urban combat training. AI Magazine,
26, 8292.
Wu, C. S., & Liu, Y. (2006a). Queueing network-model of a real-time
psychophysiological index of mental workload: P300 amplitude in
event-related potential (ERP). In Proceedings of the Human Factors
and Ergonomics Society 50th Annual Meeting (pp. 11171121).
Santa Monica, CA: Human Factors and Ergonomics Society.
Wu, C. S., & Liu, Y. (2006b). Queueing network modeling of driver
workload and performance. In Proceedings of the Human Factors
and Ergonomics Society 50th Annual Meeting (pp. 23682372).
Santa Monica, CA: Human Factors and Ergonomics Society.
Wu, C. S., Tsimhoni, O., & Liu, Y. (2007). Development of an adaptive
workload management system using the queueing network-model
human processor. In Proceedings of the Human Factors and
Ergonomics Society 51st Annual Meeting (pp. 15401544). Santa
Monica, CA: Human Factors and Ergonomics Society.
Zachary, W., Ryder, J., Stokes, J., Glenn, F., Mentec, J.-C. L., &
Santarelli, T. (2005). A COGNET/iGEN cognitive model that
mimics human performance and learning in a simulated work envi-
ronment. In K. A. Gluck & R. W. Pew (Eds.), Modeling human
behavior with integrated cognitive architectures: Comparison,
evaluation, and validation (pp. 113175). Mahwah, NJ: Erlbaum.
Zachary, W., Ryder, J. M., & Hicinbothom, J. H. (1998). Cognitive task
analysis and modeling of decision making in complex environ-
ments. In J. A. Cannon-Bowers & E. Salas (Eds.), Making decisions
under stress: Implications for individual and team training (pp.
315344). Washington, DC: American Psychological Association.
Wayne D. Gray is a professor in the Cognitive Science
Department at Rensselaer Polytechnic Institute and was
awarded the 2008 Franklin V. Taylor Award for Out-
standing Contributions in the Field of Applied Experi-
mental & Engineering Psychology by the American
Psychological Association. He received his Ph.D. in psy-
chology from the University of California, Berkeley,
in 1979.
Date received: October 22, 2007
Date accepted: April 28, 2008