Argumentation Based Design Rationale

International Journal of Human-Computer Studies, 1994, 40 (4), 603-652
Argumentation-Based Design Rationale: What Use at What Cost?

Simon Buckingham Shum and Nick Hammond Human-Computer Interaction Group Department of Psychology University of York York, YO1 5DD UK
Now
at:
Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K. Phone: +44 1908-655723 Fax: +44 1908-653169
sbs@acm.org
Abstract
A design rationale (DR) is a representation of the reasoning behind the design of an artifact. In recent years, the use of semiformal notations for structuring arguments about design decisions has attracted much interest within the human-computer interaction and software engineering communities, leading to a number of DR notations and support environments. This paper examines two foundational claims made by argumentation-based DR approaches: that expressing DR as argumentation is useful, and that designers can use such notations. The conceptual and empirical basis for these claims is examined, firstly by surveying relevant literature on the use of argumentation in non-design contexts (from which current DR efforts draw much inspiration), and secondly, by surveying DR work. Evidence is classified according to the research contribution it makes, the kind of data on which claims are based (anecdotal or experimental), the extent to which the claims made are substantiated, and whether or not the users of the approach were also the researchers. In the survey, a trend towards tightly integrating DR with other design representations is noted, but it is argued that taken too far, this may result in the loss of the original vision of argumentative design. In examining the evidence for each claim, it is demonstrated firstly, that research into semiformal argumentation outside the design context has failed to substantiate convincingly either of the two claims implicitly attributed to it in current DR research, and secondly, that there are also significant gaps in the DR literature. There are emerging indications, however, that argumentationbased DR can assist certain kinds of design reasoning by turning the representational effort to the designers advantage, and that such DRs can be useful at a later date. This analysis of argumentation research sets an agenda for future work driven by a concern to support the designer in the whole process of externalising and structuring DR, from initially ill-formed ideas to more rigorous, coherent argumentation. The paper concludes by clarifying implications for the design of DR training, notations, and tools. Keywords: design rationale, argumentation, knowledge structuring, claims analysis, empirical evaluation
ARGUMENTATION-BASED DESIGN R ATIONALE
1. Introduction
It is curious to observe how the authors in this field [programming logic], who in the formal aspects of their work require painstaking demonstration and proof, in the informal aspects are satisfied with subjective claims that have not the slightest support, neither in argument nor in verifiable evidence. Surely common sense will indicate that such a manner is scientifially unacceptable. The deplorable situation of programming logic outlined here is part of a much more widespread pattern of attitudes and manners prevailing in academic computing and mathematics, that tend to accept sales talk in the place of scientifically sound reasoning. Naur (1989reprinted in Naur, 1992: p. 477)
In the above extract, Naur expresses a critical view of much of the research in his field, deploring the unsubstantiated, handwaving nature of some of the claims made for programming languages and computing environments. In the context of the above extract, Naur was referring to many of the informal claims made for formal notations, which often appealed to the naturalness, clarity of thought, or flexible reasoning which, it was argued, such representations engendered. In contrast to much research in computer science, there is little doubt that human-computer interaction (HCI) as a field recognises to a far greater extent the importance of making computational representations usable. However, our contention is that even within a field of enquiry specifically concerned with the development of high-level languages for human reasoning (namely computer-supported argumentation), as opposed to the formal programming languages which concerned Naur, an inordinate amount of activity is devoted to building new representations and tools to support them, compared to the effort given to evaluating those tools, and that claims are made for these notations and environments which are subtantiated far less vigorously. In this paper, we take the case of argumentation-based design rationale, and examine the robustness of the conceptual foundations on which so much work has taken place in the last decade. A design rationale (DR) expresses elements of the reasoning which has been invested behind the design of an artifact. A DR answers Why...? questions of different sorts, depending on the class of DR represented. Recently, interest in DR has been growing amongst the development and research communities in both software engineering (Potts and Bruns, 1988; Conklin, 1989; Arango, Bruneau, Cloarec and Feroldi, 1991; Jarczyk, Loffler and Shipman, 1992; Johnson, 1992; Ostwald, Burns and Morch, 1992), and human-computer interaction (Conklin and Begeman, 1988; MacLean, Young and Moran, 1989; Lee, 1990; Carroll and Rosson, 1991; Fischer, Lemke, McCall and Morch, 1991; Moran and Carroll, 1994) . It has been argued for some time (e.g. Conklin, 1989; Floyd, 1987) that at present the software design process is largely artifact oriented, that is, the emphasis is on generating and tracking the intermediate design artifacts requirements, specifications, prototypes, user documentation culminating in the final system itself. However, the process by which these artifacts assume their final form remains largely implicit (e.g. hidden in minutes of meetings, design notebooks, email archives, or designers memories), and is consequently difficult to recover, and even harder to reuse. The challenge for DR research is to find the most helpful, accessible representations for design reasoning for both initial developers and subsequent designers, which minimise the non-productive effort required to create it (capturing useful DR is bound to require some effort). We shall shortly describe how the expression of design reasoning as arguments about issues has been attracting research interest, and shall refer to these approaches as argumentation-based DR. It is the justification for pursuing these approaches which this paper examines. 1
B UCKINGHAM SHUM & HAMMOND
Representations, and methods or tools for using them, have certain defining characteristics which can be regarded as hypotheses about the way in which they relate to the world. The world, in the case of a DR approach, comprises DR users (designers), a design context (when, by whom, and for what will DR be used?), and the DR domain (potentially useful design reasoning which should be captured). The idea of DR approaches making hypotheses draws on the perspective developed by Carroll et al., (e.g. Carroll and Kellogg, 1989), who see artifacts (such as tools, notations, methods) as embodied theories, making testable claims about how they can be used. The aim of this paper is to examine two key claims which underly argumentation-based approaches to DR: in essence, that it is useful, and usable (these terms are expanded shortly). We do this by examining evidence pertaining to these claims from two research areas: research into argumentation and knowledge structuring in non-design domains research into argumentation schemes for design rationale Since argumentation-based DR approaches are based both historically and conceptually on work in the first two areas, it is instructive to ask if the evidence within the roots to DR is strong enough to support the claims currently made for it. This is an important question, since notations and methods which have been proven effective in one context (e.g. writing) may not transfer to the software design context in which DR operates. In the next section we briefly identify three strong influences, which can be seen as historical, and conceptual roots to the current growth in DR research effort. We then introduce examples of argumentation-based DR notations to acquaint the reader with the representations of concern, and describe the scheme used to classify claims about argumentation schemes in use. Attention then turns to different research literatures, examining the basis on which conclusions have been drawn. The main requirement for inclusion in this analysis was that claims made for notations and tools were presented in publically available reports of their use. The results of this survey hold implications for the future development of training, notations, and tools for argumentation-based DR.
1.1. HISTORICAL AND CONCEPTUAL ROOTS TO ARGUMENTATION-BASED DR 1.2. THE VISION, AND A REPRESENTATION
Possibly the earliest mission statement which argumentation research today continues to pursue was set out by Englebart (1963), in his seminal paper, A Conceptual Framework for the Augmentation of Mans Intellect. This foresaw the day when computers, far more powerful than those available at the time of writing, would enable people to overcome some of the limitations of their cognitive faculties by manipulating externalised concept structures:
A concept structure (...) is something that can be designed or modified, and a basic hypothesis of our study is that better concept structures can be developedstructures that when mapped into a humans mental structure will significantly improve his capability to comprehend and to find solutions within his complex-problem solving situations. Englebart (1963 reprinted in Greif, 1988: p. 54)
This vision of designing better computer-supported notational structures with which to think marks the beginning of the trail which much of the work surveyed in this paper
has continued to explore. We shall not discuss this work in any more detail, except to note that Englebarts framework1 embraced not only cognitive tools, but notational and training issues, facets of DR research which we focus on in more detail towards the end of this paper. The second root concerns the representational form for arguments. The Uses of Argument by Stephen Toulmin (1958) was originally written as a challenge to the dominance in philosophy of formal, Aristotelian logic. Toulmins aim was to develop a view of logic which was grounded in the study of reasoning practice. Taking argumentation as the most common form of practical everyday reasoning, he posed the question, what, then, is involved in establishing conclusions by the production of arguments? His analysis of the logical structure of arguments led to a graphical format for laying out the structure of arguments, a representational approach reflected in much subsequent argumentation work. The notation consists of five components and four relationships (Figure 1). According to the analysis, whether or not it is made explicit, all arguments logically comprise a fact or observation (a Datum), which via a logical step (a Warrant), allows one to make a consequent assertion (a Claim). The Warrant can be supported by a Backing if necessary (why the assumed Warrant is valid), and the Claim qualified with a Rebuttal (specifying exceptions to the rule). Datum
Harry was born in Bermuda so Certainly
Claim
Harry is a British subject
since
unless
Warrant
A man born in Bemuda will generally be a British subject
Rebuttal
Both his parents were aliens/he has become a naturalised American
on account of
Backing
See legal statutes...
FIGURE 1: An example of Toulmins graphical argument structure (from Toulmin, 1958) The purpose of this review is to set the conceptual background for the use of argumentation-based DR, so the considerable amount of debate which Toulmins model stimulated is not of immediate relevance. Further discussion can be found in Hair and Lewis (1990), and Newman and Marshall (1991) who set the model in context within the wider philosophical literature on argumentation. It is sufficient to note that
Englebart developed what he called the H-LAM/T framework (Humans using Language, Artifacts, and Methodology, in which they are Trained).
Toulmins work, through its use of a semiformal graphical representation, was the first step towards the argumentation schemes being used at present.2
1.3. DEVELOPING THE VISION
Englebarts call helped set in process research efforts in numerous locations. One of the most influential of these was based at the Xerox Palo Alto Research Center, where research into computational support for idea structuring tasks like writing and argumentation was just beginning in the early 1980s. In a forward looking paper, Brown (1983) focussed on the potential value of representing the process by which ideas and artifacts develop, as well as the final product:
Process versus product By focussing in the design of empowering environments on the product of a creative effort, we are missing the real source of power for computer-based tools: the computer can record and represent the process underlying the created product. By making explicitly available to the user the series of steps and missteps that leads to the creation of a particular object or result, we create a basis on which to build extraordinarily powerful editing, merging, undoing, and transforming tools. Tools designed to manipulate this historical information, or audit trail, can be used to carry out intellectual and creative tasks of great complexity. (p.182, original emphasis) (...) By maintaining an explicit audit trail of the steps that led to the created object, not only can more powerful tools be constructed but, perhaps more importantly, the ability for one to understand what another has done is greatly enhanced. (p.183)
Complementary to this challenge, and perhaps as one possible solution, was a subsequent interest in the potential of tools for manipulating and filtering arguments (Brown, 1986):
Current communications tools and methods force the crafting of complex arguments into linear form for presentation, so that the web-like connections among ideas is hidden from view, making it difficult to see alternate interpretations and points of view. (...) As a result many of the underlying ideas, arguments and assumptions either remain implicit or are lost altogether. But consider the possibility of crafting new information tools to capture not just conclusions and the view of matters that supports them, but to allow the explicit representation of underlying assumptions and argument structures. (p. 484)
It was noted in particular that work was needed on developing notations with an appropriate vocabulary for the task domain:
To accomplish these goals, we need a taxonomy of epistemological links for relating ideas, as well as link-related filters. That is, we must now think about giving users access to and utilisation of not just undifferentiated links, but links with appropriate kinds of labels. (p. 485)
Representation of these two key elements deliberation process and arguments with appropriate kinds of labels are precisely the concerns dominating DR research. In a subsequent project on computational support for meetings, Stefik et al., ( 1 9 8 7 ) explicitly set out to substantiate the first claim examined in this paper with a tool called Argnoter:
Lee and Lai (1991a) have, however, noted several reasons why Toulmin form is not suitable as a notation for expressing design argumentation of the sort required for DR.
ARGUMENTATION-BASED DESIGN R ATIONALE [Design] is essentially a dialectic between goals and possibilities... in collaborative design tasks, this interaction and tension between goals and alternatives must play itself out in the communications among collaborators. ...A major theme of Argnoters design is that alternatives be made explicit: Proposals themselves are explicit, as are assumptions and evaluation criteria. (p. 38) A major working hypothesis behind the design of Argnoter is that making the structure of arguments explicit facilitates consensus by reducing uncommunicated differences. (p. 40)
The motivation behind Argnoter was clearly that of a group DR tool a way to represent design arguments explicitly, but with the group process adding another dimension. It was hypothesised that the process of striving to agree on rankings and assumptions in Argnoter would help designers recognise where their differences lay (cf. Claim 1 in this paper). Although Argnoter was never built (Bobrow, Stefik, Foster, Halasz, Lanning and Tatar, 1990), we note this work in passing since it was one of the earliest specifications for a DR tool using modern technology, and together with Englebart, Toulmin, and Brown, contributes to the conceptual and historical context in which to understand current work on argumentation in design. 2. Argumentation-based DR notations The idea that DR should be represented as semiformal arguments can be traced back to wider research into the development of computational support for reasoning. Given that the formulation and critique of arguments is a central activity in many key areas (research, design, writing, management, government), a challenge with many potential benefits has been to develop representational schemes which enable computers to assist in authoring, retrieving, evaluating, and modifying arguments. The main hypothesis behind research into supporting argumentation was that by making the structure of arguments explicit, they could be more rigorously constructed and communicated (Brown, 1986; Smolensky et al., 1988), and these are still key goals in DR research. Differences between notations lie in the constructs chosen to capture arguments, which, in turn, determine the kinds of argument which can be captured, and the level of detail which can be expressed (fewer, more general types, or a larger number of more specific types). Proponents of DR approaches such as this argue that they have the potential to play several roles in design: structuring design problems maintaining consistency in decision-making keeping track of decisions communicating design reasoning to others as a chronological record of the design process assisting the integration of theory into design practice (as principled decisionmaking) supporting the building of cumulative design knowledge, through reusing DR. Several approaches to argumentative DR are now outlined to provide a flavour of this approach to DR. For fuller descriptions, see Carroll and Moran (1991) and Moran and Carroll (1994). Design Space Analysis (DSA) using the QOC notation (Questions, Options, and Criteria) has been proposed by MacLean, Young, Bellotti, and Moran (1991). 5
Assessments are the relationships between Options and Criteria (supports or objects-to), and Arguments are used to conduct debate about the status of the above entities and relationships. These elements are shown in their graphical form in Figures 2 and 3 (a matrix of Options against Criteria can also be used). The DSA approach to making a design more transparent is to create an explicit representation of a structured space of design alternatives and the considerations for choosing among them. Thus a particular artifact is understood in terms of its relationship to plausible alternative artifacts (MacLean, et al., 1991, p.203). Thus, DSA is a process of identifying key problems (Questions), and raising and justifying (via Criteria) design alternatives (Options). A DSA-oriented DR emphasises the cumulation and reuse of knowledge through the articulation of coherent design spaces within which different solutions can be located (e.g. as implemented in other systems). Generic QOC vocabulary:
Argument Argument Argument
objects to
Option
Criterion
Question
Option
supports
Criterion
Option
Criterion
Consequent Question Question
Question
FIGURE 2: The generic QOC notation as used in Design Space Analysis. Note the use of link-thickness to indicate relative weights of Assessment.
QOC from a Smalltalk design session:

preload data patterns one instrument instances for every event limited no. of inst. inst. and reallocate events limited no. of inst. inst. and several data patterns in real time low memory requirements low data transfer rate required
allocation of instruments
low processor calculation required
download data patterns into new allocation how to reallocate inst. inst. to events?
low data throughput requirement during play
prestore all data patterns
low memory requirement
User interface QOC:

O: pop-up Q: what kind of menus? O: pull-down C: user orientation to position of menu options C: speed of access C: context sensitivity of menu options
O: top Q: where should cursor be on pop-up menu? O: middle O: last used option
C: consistent C: minimise average cursor distance to selection C: adapt to user
FIGURE 3: Two examples of QOC representing DR for very different levels and domains of design. The IBIS notation (Rittel, 1972; Rittel and Webber, 1973), extended for the graphical IBIS tool gIBIS (Conklin and Begeman, 1989; Conklin and Burgess Yakemovic, 1991) is similar superficially; the main difference lies in the IBIS emphasis on capturing the design process for a single design, as opposed to retrospectively rationalising the DR to clarify the dimensions defining the space in which the design sits in relation to other possible designs. The work on gIBIS has led to the only commercial argumentationbased DR tool currently available (CMSI, 1992), as far as these authors know. Lee and Lai (1991b) have proposed a notation which extends QOC and gIBIS. Decision Representation Language (DRL) was developed to explore the potential for computational support within the semiformal paradigm (Figure 3a). SIBYL (Lee, 7
1990), the tool supporting DRL, provides a number of computational services, including dependency management (monitoring decisions which depend on each other), precedence management (other decisions sharing same Goals), viewpoint management (arguments sharing common assumptions), and plausibility management (the strength of supporting argumentation for an alternative).
Decision Problem Goal
is a subgoal of
Goal
is a subgoal of is an alternative for
Question Goal
achieves denies supports achieves queries
Alternative
claim
denies
claim
Alternative
is a sub-decision of
influences presupposes supports
claim
claim Question
claim
is an answering procedure for
Decision Problem
are possible answers to
procedure
is a result of is a subprocedure of
Group
answers is a member of
procedure
procedure
claim
claim
claim
FIGURE 3a: The DRL notation
FIGURE 3b: Two views of a DRL network (an Argument browser and a Decision matrix) using SIBYL, its support tool (from Lee, 1990). Attention turns now to examine the evidence that approaches such as the above are useful and usable strategies for representing DR. 3. Claim 1: Argumentation-based DR is useful Design rationale approaches need to demonstrate their utility, both for those who invest the effort in generating and maintaining it (a requirement of collaborative systems in general Grudin, 1988), and for those who come subsequently to use it. In the first part of this literature survey, we investigate the first major claim made by, and for, argumentation-based DR as an approach that it is indeed a useful process in which to engage. Three classes of evidence informing this claim have been gathered, and classified according to a scheme described next.
3.1. A SCHEME FOR CLASSIFYING CLAIM 1 EVIDENCE
In order to systematise our examination of evidence for Claim 1, a scheme was developed which identifies three classes of evidence based on the research contribution which a study makes, that is, what question is it answering?. These are as follows: 9
E1:
Evidence that using an argumentative notation augments the reasoning of those who use it. This kind of evidence is particularly valuable for strengthening the argument that DR need not only benefit other people, but can itself assist design deliberation. Augments will be taken to mean both augmented product compared to working without the argumentation scheme (that is, better reasoning), and augmented process (making work faster or easier) even if there is no evidence that the final result was improved. Evidence that an existing record of earlier argumentation, either ones own or someone elses, is useful. This focuses on the product of using the notation, as opposed to the process by which it was createdis it useful to have such representations? One of primary arguments made for DR is that it provides a useful resource for subsequent work. Evidence that using an argumentative notation impedes reasoning. This sets a boundary condition on the utility of the argumentative approach. E3 evidence is not simply failure to demonstrate an E1 effect; it demonstrates that an argumentation scheme can actually obstruct the task at hand.
E2:
E3:
The reference to earlier argumentation in E2 refers to records which need to be explicitly retrieved in some sense. In one respect, the E1-E2 boundary is a grey one: when does DR become old DR? However, it is intended to capture the difference between using a notation interactively as a (possibly shared) focus of attention and working representation for developing ideas (E1), versus retrieving notational records of past discussions for reference (E2). Tang, (1991) originally highlighted this distinction, between the role of representations created during a meeting as ideas develop, versus their role as a memory aid in subsequent meetings. Having identified the research question which a particular study claims to address, we then classify E1-3 according to whether it is anecdotal or based on a systematic experimental study. Experimental type studies often (but not always) constrain designers in some way (e.g. task; notational size; system functionality) in order to conduct more rigorous qualitative or quantitative analyses which may not be possible in field settings. We use the label experimental not to refer in the narrow sense to classical hypothesis testing experiments which contrast two conditions in order to isolate a variable of interest. Rather, it is used to reflect the depth of analysis, which is usually only possible when large quantities of data have been systematically gathered under at least partially controlled conditions. An example would be studying a specific design task in a real office, but wth audio and video recording equipment. In contrast, anecdotal evidence invariably highlights interesting incidents but ignores less interesting ones. Experimental studies might provide a firmer basis for generalising from such incidents, by recording the frequency of such incidents. Although the label anecdotal may have derogatory connotations, anecdotal evidence is often embedded in realistic work settings, in which it may be impossible, for instance, to collect the data needed for experimental level analyses. Particularly in a field of study like DR, the incidents of interest may arise opportunistically in many different contexts, making it impossible to capture each as it actually happens. Whilst anecdotal evidence can be useful, we are concerned to distinguish between empty and substantive claims, and also between self-reporting and reporting of others using DR. Consequently we further classified anecdotal reports as detailed or general, depending on the detail in which incidents are documented, analysed and presented. Detailed anecdotal evidence is often found in carefully conducted case studies. Experimental studies are by definition quite detailed, but have been further classed as observational or self-reported depending on whether a second party or the researchers 10
themselves used the DR. This yields four classes which will be regularly referred to in discussing DRs utility (e.g. general anecdotal E1 evidence; self-reported experimental E2). Each is defined in Figure 4 and illustrated with respect to a fictitious example. It should be noted that the classes reflect only what has been presented in a report, not necessarily the validity of the results observational experimental evidence might turn out to be seriously flawed as a basis for the claims made.
General anecdotal: key incidents from use of an argumentation scheme are reported, from any kind of user, with no analysis. Claims made are not substantiated by accounts of the relevant design processes. E.g. Use of the notation helped to clarify issues.
Detailed anecdotal: key incidents from use of an argumentation scheme, by any kind of user , are analysed and presented. Claims made are substantiated by accounts and analysis of the relevant processes and possible causes. E.g. Use of the notation helped to clarify issues in several cases . In one incident, we (or three ) designers documented an issue, but found that in listing alternative solutions, we (or they) were tackling two closely related problems. Once these were made explicit, each was addressed separately, and the relationships between them clarified. (possible supplementary data: e.g. transcript extract and figures showing evolution of the DR)
Self-reported experimental: detailed data from use of an argumentation scheme were gathered and analysed, but the users were the researchers themselves. Claims made are substantiated by accounts of the relevant design processes. Indication E.g. Use of the notation helped to clarify issues in five cases. In Incident 3 for example, we documented an issue, but found that in listing alternative solutions, we were tackling two closely related problems. Once these were made explicit, each was addressed separately, and the relationships between them clarified. (possible supplementary data: e.g. transcript extract and figures showing evolution of the DR)
Observational experimental: detailed data from use of an argumentation scheme were gathered and analysed, and the users were designers, whose professional background and experience with DR is documented. Claims made are substantiated by accounts of the relevant design processes. E.g. as above, except users are not the researchers .
FIGURE 4: Classification scheme for Experimental and Anecdotal evidence. Underlined text in the examples highlights the defining features of a class of evidence. Whilst field evaluations sometimes have to sacrifice the systematicity which is possible in more controlled settings for genuine pragmatic reasons , it is often possible to conduct studies in real or highly naturalistic work settings (e.g. Harrison, Minneman and Stults, 1988; Minneman, 1991; Tang, 1991; Olson, Olson, Carter and Storrsten, 1992). It need hardly be emphasised that detailed anecdotal and observational experimental DR research is to be encouraged, although we recognise the role of selfreported experimental studies as a powerful strategy for initial evaluations of new approaches. General anecdotal claims serve at best to point towards genuine underlying relationships, but all too often disguise the fact that no attempt has been made to critically evaluate the work.
11
A categorical analysis such as this faces the challenge of studies which straddle the boundary between two classes. This cannot be avoided, but we do endeavour to explain what was done in all studies which we review, before classifying it; this will, we trust, clarify how the scheme is being used, and justify the assigned classification. This scheme is also open to the criticism that it fails to make explicit various important differences between studies. More specialised classes could of course be devised and the literature reclassified, but (as is the case in developing argumentation notations) pragmatic concerns dictate when to stop. The four classes defined above proved useful when conducting this analysis, and provide, we hope, a helpful overview of the state of current research and practice (summarised in Figure 8). We would encourage others to conduct their own analyses to highlight patterns which this paper does not address.
3.2. CLAIM 1 (UTILITY)NON-DESIGN EVIDENCE
In some studies which we now review, argumentation representations were supported by tools, but in others, they were assessed independent of any implementation. However, since usability problems with computer supported argumentation will depend not only on the user interface and functionality, but significantly on the adequacy of the argument notation, analyses of notations without technological support serve as valuable baseline indicators. There are now a number of research efforts which have developed tool support for argumentation schemes. All of the following have made use of Toulmin form (introduced earlier): SYNVIEW, for structuring scientific discussion (Lowe, 1985), the Safety Argument Manager ( SAM) for certifying safety-critical systems (McDermid, 1991), the Authors Argumentation Assistant (AAA Schuler and Smith, 1990), which is part of the SEPIA writing environment (Streitz, et al., 1989; Streitz, Haake, Hannemann, Lemke, Schuler, Schuett and Thring, 1992). All of these sources present the rationale for and functionality of a system. However, whilst in some cases claims are made about utility and usability (e.g. SEPIA is motivated by certain cognitive models of writing) no empirical evidence could be found for any of these systems which either supports or rebutts Claim 1. Only one properly controlled study could be found which evaluated the effect of using an argument notation on the quality of argumentation (process or product). LEGALESE is a tool for legal argumentation (Hair, 1991). The LEGALESE argumentation model comprised a hierarchy of issues, supported by arguments, in turn supported by facts and laws. Hair (1990) describes a study in which lawyers and law students formulated arguments to support their analysis of a hypothetical legal case, using either pen and paper or LEGALESE . All subjects then wrote out their arguments drawing on their analyses of the case; these were independently ranked for quality by professional lawyers. A number of rankings and correlations are reported; the main statistical result was that all subjects judged to have used the LEGALESE approach correctly (eight) were ranked in the top ten of the overall rankings. Closer analysis by Hair suggested that understanding the LEGALESE argumentation model was critical, since performance actually got worse if this method was not grasped. Hair points out a number of weaknesses in the experiment, but concludes that the results strongly suggest that the argumentative structure assisted analysis of the case and improved legal argumentation. This study provides observational experimental evidence of improved reasoning (E1), and of impeded reasoning when the method was used without sufficient training (E3). Apart from this study, the most systematic comparison of notations which came to light was a report by Hair and Lewis (1990) of their own experiences. This was conducted as part of the EUCLID project (Smolensky, Bell, Fox, King and Lewis, 1987; Smolensky, et al., 1988; Bernstein, 1992) which aims to provide computational support for theoretical 12
reasoning. Hair and Lewis selected two scientific articles, and represented the arguments in each using three formalisms: Toulmin form, which was described above; EUCLID s Argument Representation Language, also represented as a semiformal, graphical notation; and Prolog statements (the only non-graphical notation) which required full formalisation of the arguments logic, and thus permitted some degree of formal checking for correctness. The aim was to find out if the use of different schemes led to different insights into the arguments, or might help others grasp them. Overall, Hair and Lewis felt that they could not directly attribute the insights they gained into the articles to any of the argumentation schemes, although the analysis required to use the representations helped clarify issues and relationships. They note that any extra effort might, of course, have yielded the same benefits, and were also doubtful as to whether presenting the arguments in any of the three formalisms would help somebody else grasp them better, since they were quite cryptic (Prolog especially), and there was a significant degree of subjectivity involved in constructing them.3 This self-reported experimental study led Hair and Lewis to the conclusion that much work still needs to be done to make argumentation structures useful as a medium of communication (E2 evidence), and to demonstrate that their use benefits the author (E1 evidence). Newman and Marshall (1991) discuss Toulmins formalism in some detail based on their experiences using it to represent legal arguments. This analysis was based on use and analysis of Toulmin form over several years, as part of a project examining hypertext representation issues. As such, this constitutes a combination of longitudinal, detailed anecdotal and self-reported experimental evidence, and is possibly the most extended use of a specific argumentation notation to date. Although some work was carried out in representing Toulmin form with NoteCards (Marshall, 1987; 1989), Newman and Marshalls analysis focusses on the notation rather than computational support. They document a number of notational weaknesses relating to Toulmins notation, which relate broadly to evidential classes E2 and E3 as follows: problems with the notational coverage of the domain in which they were working (legal argumentation), and with encoding arguments (E3), its comprehensibility as a medium for presenting arguments (E2), and its perspicuity (highlighting important features of argument structure) (E2). Significantly, during this extensive exploration of Toulmin form, no incidents are reported in which insights into arguments (E1) could be attributed to the final representation or process thereof. Storrs (1989) describes the use of Toulmin argumentation in a tool designed to support government policy formulation. An initial evaluation exercise is reported, in which a policy maker, shadowed by members of the development team, used a prototype to provide initial feedback on the adequacy of the tool. He was asked to evaluate it in terms of the accuracy of the policy formulation model it embodied, the quality of the support it provided, and its usability. It is reported that feedback was very positive generally, which constitutes objective, experimental evidence of adequacy, but leaves unspecified the extent to which it actually assisted his work (E1 criterion). No other studies in the general argumentation literature could be found which analysed the use of a predefined notation like Toulmin or IBIS (DR evidence is described shortly). However, the emergence of semiformal idea processing or knowledge
3
Comprehension problems related to the cryptic nature of semiformally structured DR have also been reported (Burgess Yakemovic and Conklin, 1990; Shum, 1993). The absence of important contextual information in most notations means that the reader needs to be familiar with the design to understand the DR on its own; alternatively, conventional design representations can be used to provide context in which to understand rationale, as described later.
13
structuring tools provides another perspective. These tools provide support for spatially laying out and linking ideas as graphical nodes. Examples are addressed in more detail later in the discussion of usability, but one account merits mention here. VanLehn (1985) reports that use of the NoteCards hypertext system (Halasz, Moran and Trigg, 1987) exposed two major flaws in his arguments. The representation scheme being used was not a particular notation, but the generic NoteCards facilities for constructing and viewing hierarchies, networks, and tables of nodes and links which have been given types. In one incident, by viewing a graph as a matrix, blank cells highlighted gaps in reasoning. In the second, a graph was computed over a set of ideas stored as notecards; the resultant structure showed disconnected subtrees (sets of issues) which should have been connected (as one theory). This led to an analysis of different inter-issue relationships, which in turn led to further insights. VanLehn concluded that NoteCards main advantage was its facility to fool around with scratch organisations in a way which paper-based media preclude. He also makes the important point that a NoteCards graph browser is driven by the contents of its constituent cards (the links embedded in the text of each card). To change the structure, the content of the nodes needs to be changed, so that it is impossible to compute a structure independent of the content of ones ideas. Significantly, what had hidden the flaw in the argument up to that point was a decoupling between content and structurethe neat hierarchical structure to the document he was creating was an independently imposed structure which finessed the conceptual weaknesses. NoteCards forced the true structure of his ideas into the open. In VanLehns view, a NoteCards database could be a tangible representation of a theory, which could be explored by others to understand how it came to be the way it is. Although VanLehn does not make explicit reference to Brown, it can be seen that NoteCards would in this role facilitate the recovery of conceptualaudit trails of the sort which Brown envisioned; close parallels with design rationale are also apparent here. This account provides detailed anecdotal evidence of the productive role which computational structure can play in reasoning, although in the absence of a notation such as IBIS or Toulmin form, strictly speaking, Claim 1 is not directly addressed.
3.3. CLAIM 1 (UTILITY)DESIGN RATIONALE EVIDENCE
We turn now to argumentation-based DR approaches, to examine evidence relating to Claim 1. Three strands of early work were acknowledged above as landmark points in this field. We introduce a fourth at this point, because its claims can be directly assessed, and overlap significantly with Claim 1. The decision to adopt semiformal argumentation as the representational basis for DR has been strongly influenced by the seminal work of Horst Rittel (Kunz and Rittel, 1970; Rittel and Webber, 1973)4. Several DR research efforts (e.g. Conklin and Burgess Yakemovic, 1991; Fischer, et al., 1991; McCall, 1991) have described in some detail how their approaches derive from Rittels concept of argumentative design, which was motivated, at least in the first instance, by different concerns from those of Englebart and Toulmin. Rittel et al., made two testable claims of direct relevance to this review: first, that many design problems are wicked, in contrast to tame or benign problems which can be modelled computationally, and secondly, that an argumentative process was the most effective way to tackle such problems. Let us consider the evidence supporting each of these in turn.
Other reviews of Rittels philosophy and of IBIS can be found in Conklin and Begeman (1989) and Fischer, et al. (1991).
14
Wicked and incorrigible [problems]...defy efforts to delineate their boundaries and to identify their causes, and thus to expose their problematic nature (Rittel and Webber, 1973). Such problems lack a single, agreed-upon formulation or well-developed plans of action, are unique, and have no well-defined stopping rule, because there are only better or worse (rather than right or wrong) solutions. Closure is often forced by pragmatic constraints (e.g. managerial or political) rather than rational scientific principles. How do these claims stand up to subsequent work? The features originally proposed by Rittel as characteristic of wicked problems are now well substantiated, particularly within cognitive science design research. The class of ill-structured problems is widely acknowledged, beginning with Reitman (1964) who characterised ill-defined problems, and continued by Newell (1969) and Simon (1973) who defined ill-structured problems within their Information Processing Theory (IPT). More recent work within IPT attempts to delineate ill-structured problems from design problems (Goel and Pirolli, 1989; 1991). This literature cannot be properly surveyed here, but there are strong grounds to conclude that this research has not only identified features congruent with those identified by Rittel, but refines the original conception of wicked problems with respect to design. The second claim made by Rittel et al., derived from their response to the challenge of wicked problems. Such problems could not be solved by formal models or methodologies, classed by Rittel as the first-generation design methodologies. Instead, an argumentative approach to such problems was proposed (a second-generation design method). The essence of this perspective was that an open-ended, dialectic process of collaboratively defining and debating issues is a powerful way of discovering the structure of wicked problems:
First generation methods seem to start once all the truly difficult questions have been dealt with already (...) The second generation deals with difficulties underlying what was taken as input for the methods of the first generation. [Second generation] methods are characterised by a number of traits, one of them being that the design process is not considered to be a sequence of activities that are pretty well defined and that are carried through one after the other, like understand the problem, collect information, analyse information, synthesise, decide, and so on... My recommendation [for the future of design methodologies] would be to emphasise investigations into the understanding of designing as an argumentative process ... how to understand designing as a counterplay of raising issues and dealing with them, which in turn raises new issues, and so on... [Argumentative design] means that the statements are systematically challenged in order to expose them to the viewpoints of the different sides, and the structure of the process becomes one of alternating steps on the micro-level; that means the generation of solution specifications towards end statements, and subjecting them to discussion of their pros and cons. (from an interview with Rittel, 1972)
This perspective motivated the development of Issue Based Information Systems (IBIS) as a medium to encourage the open deliberation of issues. The three key IBIS entities were Issues, Positions and Arguments, which could be linked by relationships such as supports, objects-to, replaces, temporal-successor-of, more-general-than, and their converses. Visualised as a graph, an IBIS grows into a network as more Issues are posted and debated (Figure 5).
15
P P
A A
Issue responds to
Position Position
+ supports + objects to
P I P P
Argument
A A A
Argument A A A
P P
FIGURE 5: The basic IBIS structural unit of Issues, Positions and Arguments, developed to support Rittels argumentative approach to wicked problem solving. To summarise, Rittels work established a bridge between design and argumentation. The argumentative approach to design elevated the importance of the process of understanding a problem from its minor status as a preliminary step to using firstgeneration design methods, to the central activity in tackling wicked design problems. IBIS was the first explicit representation for argumentation in a design context. Rittel et al., hypothesised that a particularly powerful way to tackle such problems is by an argumentative approach. There is a clear convergence here with Claim 1 that using a structured argumentation scheme augments design reasoning. Let us now consider the evidence to support this hypothesis provided by argumentation-based DR work to date. A study evaluating IBIS-based DR is reported by Burgess Yakemovic and Conklin (1990) and Conklin and Burgess Yakemovic (1991). A positive impact on project team communication is reported from use of IBIS 5, through two processes in particular. Firstly, it is proposed that a knowledge of the underlying form of the dialogue [i.e. rhetorical moves in IBIS] allowed the team to determine when the discussion had wandered from the intended topic (Conklin & Burgess Yakemovic: p.375). Although discussion was not structured explicitly around the IBIS model, it is argued that knowledge of the IBIS model (recorded privately by a DR scribe), helped maintain a shared awareness of the meetings process. Designers did not generally make explicit reference to IBIS constructs like Issues and Positions when speaking, but could for instance decide that something should be recorded (by the scribe) for later attention, or query something which might be answered by reference to the DR. It is not clear whether part of the DR scribes role was to interject when discussion veered off-course thus bringing IBIS to the explicit attention of the group or whether the scribe played a purely passive role leaving the other members to direct discussion. If, during a meeting, the IBIS exists primarily as an implicit, private record, with only occasional recognition by the main group of designers, it is difficult to conclude from the data reported that its rhetorical structure can shape deliberation more beneficially than, for instance, the personal notes which are often made during meetings. To demonstrate that simply being aware of a rhetorical model can improve group process would require analysis of the dynamics of design meetings in greater detail than that
5
A purely textual indented-text IBIS (itIBIS) was used, with the Issues, Positions and Arguments indented in that order, like hierarchical levels of headings in an outlining tool.
16
presented. Without supporting evidence, Burgess Yakemovic and Conklins conclusion can only be regarded as general anecdotal E1 evidence. A second role played by IBIS was in setting meeting agendas. It is reported that the IBIS record helped a meeting coordinator to identify seven interrelated Issues which needed to be resolved together, summaries of which were then circulated to members with all the Positions and Arguments which had been discussed. This clear identification of the decisions needing action and the record of related arguments, enabled the different stakeholders to prepare fully for the meeting, and led subsequently to an elegant solution. In this incident, it is as a concrete record of past discussions that IBIS seems to have been most valuable (E2 evidence) both to the meeting planner in its raw IBIS form, and to the other members in the form of the agenda. This incident is reported in more detail than the first, and is best classified as a mix of general and detailed anecdotal evidence. Details are provided about the context in which the Issues were identified: during conversion from indented-text IBIS to gIBIS, the meeting coordinator thought there were two Issues to resolve at a meeting, but the itIBIS record showed him that there were five others which needed solving simultaneously. Unfortunately, it is not then made clear how the itIBIS highlighted this. For example, did the designer accidentally stumble across these five other Issues when browsing the IBIS, or were they explicitly linked in some way to the first two Issues which the designer initially thought were relevant? Perhaps the designer was stimulated by the DR to think creatively which led him to the five other Issues which were distributed in the itIBIS record. Without this level of analysis, it is difficult to fully exploit the successes of this study what representational properties should be highlighted or given more support? In summary, the context of the incident is described, but the claim is simply asserted, leaving the underlying process unspecified. Consequently, this incident has to be classed as supported by only general anecdotal evidence. Perhaps the most widely reported result of this case study was the discovery of 11 problems during the conversion from itIBIS to gIBIS (Conklin and Burgess Yakemovic, 1991). This result constitutes the only example of E2 evidence which has been costed in any way (the time savings gained being estimated at between three and six times greater than the time cost of converting from itIBIS to gIBIS, and then using the gIBIS tool to review the DR). In reflecting on this result, Conklin and Burgess Yakemovic argue that reviewing an issue base provides a different view of a design, and thus highlights different problems from more conventional design reviews. They speculate more generally that comparing and reviewing DR might actually reduce the cost of the design process by making it more rigorous and error free (p.375). A criticism which has been levelled at this claim, although it is qualified as a preliminary result, is that the effort of converting from itIBIS to gIBIS was atypical of normal design activities, and even of DR approaches more generally. Moreover, it is asked, might not any equivalent effort have detected the problems? We venture to suggest that answers to these questions might in fact be that whilst the itIBIS to gIBIS conversion process was doubtless an artifact of that particular case study, analysis of the QOC authoring process has demonstrated quite clearly that revision is a natural process when developing DR. Furthermore, maintaining DR as a living representation during a project, and beyond, involves similar browsing and retrieval tasks to those which uncovered the 11 problems. It may also be true that any extra effort could have detected the problems, but the crucial question is surely, would it have happened? Explicit argumentation-based DR encourages the designer to make space to reflect, which they might not otherwise do. We speculate, however, that the process of problem detection is not a function of effort in the abstract, but a function of tool-mediated effort (i.e. via the argumentative structural representation). Conklin and Burgess Yakemovic do hint at 17
the process which they believed operated when the gIBIS DR was scanned: The process of browsing the DR and looking for a specific entry seems to serendipitously trigger detection of problems or inconsistencies in the data (p.375)however, they recognise that this general level of explanation (which we would class as a general anecdotal E2 claim) requires more rigorous investigation. Several efforts have been made to study the QOC Design Space Analysis approach to design rationale in use. McKerlie and MacLean (1993; in press) describe the use of QOC in a small hypermedia design team, of which they were members. They report several examples of QOC playing E1-3 roles, which we shall examine in turn. Firstly, they report that using and creating QOC records (even roughly) during design discussions could help bring out assumptions which might otherwise have remained implicit. Whilst this is clearly an E1 type claim, no examples or analysis are presented which explain how often, or by what process this occurred. It is therefore best classed as general anecdotal evidence. Secondly, QOC was used for integrating two strands of work on a problem into a common design space representation. Some indication of the dynamics of this process is provided in this example, insofar as it is reported that QOC helped to highlight the strengths of each strand, and their interrelationships; however, more detail would have been of interest, describing for instance the process by which common Questions and Criteria were agreed, or how the fitting of the ideas into a common QOC space highlighted tradeoffs and interrelationships between the two solutions. Turning to E2 evidence (DR proving useful at a later date), it is reported that QOC was used to communicate with an absent member of the team to convey how the others understood a scenario, which helped the originator of the ideas to understand how they had been interpreted by the rest of the team and therefore helped her contribute to the evolving design space. This claim that QOC was useful to another party (E2) is not elaborated (i.e. general anecdotal evidence), although one might argue that commonsense would indicate that seeing a QOC provides insight into someones view of the problem (at least when the problem is familiar to both parties). More detailed analysis is required into the utility of argumentation-based DRs for communicating understandings and analyses (cf. Hair and Lewis, 1990) reported above, who concluded that the argumentation schemes they tried did not assist in this). McKerlie and MacLean also reported that QOC records have been a valuable resource when it was necessary to revisit previous decisions either to alter them or to recollect the considerations which contributed to them. A quite detailed account is then presented, with accompanying QOC, of a decision reversal which led to other changes, whilst preserving the best points of the existing design. Specifically, articulating a new Question is shown to clarify the relationship between two existing Questions. It is concluded that, QOC records most important role for supporting the revisiting of decisions is to provide a broader context around the decisions of concern... which includes the relationships among Questions, the alternative Options considered, and the Criteria used to assess the Options. This detailed anecdotal report is better substantiated than the others, and seems to illustrate well the added leverage which the notation can provide in understanding the structure of a problem space. As is typical of anecdotally based reports, no indication is given of the frequency of such incidents was this the only one all year, or one of a number? The case for DR needs to be strengthened by demonstrating that such benefits are not so rare as to invalidate the effort. If we turn finally to E3 evidence in this study, McKerlie and MacLean identify an interesting phenomenon: as QOC began to be used, there was an early tendency towards compulsive rationalising detailed QOC analysis of local spaces which were not of great interest, such as non-controversial, or out of date issues. This is an instance of DR 18
impeding design in that it did not lead to insights, it distracted from more productive work , and was never utilised by the project. The provision of examples of situations in which deliberate QOC analysis proved of little use, draws this report closer to that of detailed anecdotal E3 evidence. To summarise, McKerlie and MacLeans report represents one of only a few studies of DR in practical use, and as such is extremely useful as a reality check on the relevance and practicality of recent DR research; more such studies are needed. Detailed context and analysis are presented for some of the claims, but much more could be provided for others. A final point to note is that most of the QOC construction and maintenance effort was expended by one of the researchers, although the design team commented and fed into QOC representations during meetings. In another authoring study, a designer working in Smalltalk-80 was trained to use QOC, and was then studied over three monthly sessions. He worked in a mock-up office which he constructed for the purposes of the study to provide all the resources which he would normally have used (e.g. architectural diagrams; design notebook). Some sessions from this case study which have been reported elsewhere (Buckingham Shum, 1994), and described how QOC proved to be an obstacle to the work in hand (see review of E3 evidence below). However, in another session, the process of recording and restructuring QOCs Options and Criteria assisted the designer in clarifying and developing several issues which had up until then remained clouded in design notes and in his mind. The main focus of this session was on the gradual refinement of Options. He re-drew his QOC structure in order to make this explicit and went on to develop the hierarchy further, as shown schematically in Figure 6. In the process of redrawing, additional insights were gained: another level of Option decomposition was added, by refining prestore data elsewhere (Option 1.2.2 1.2.2.1 and 1.2.2.2) it was realised that the third Option to the first Question, was really another refinement of the second Option (i.e. Option 1.3 became 1.2.3). This Option was therefore moved down a level in the Option hierarchy.
19
Option 1.1
one instrument instance for every event
preload data patterns low memory requirements low data transfer rate required
Option 1.2 limited no. of inst.

allocation of instruments inst. and reallocate events
Option of inst. inst. limited no. 1.3

and several data patterns in real time
low processor calculation required
Option 1.2.1
download data patterns into new allocation how to reallocate inst. inst. to events? low data throughput requirement during play
Option 1.2.2
prestore all data patterns low memory requirement
if Option 1 when to download data to new instrument allocation? decide whether there is time to download compute time to load low (high level) computation time
Option 1.2.1 Option 1.1

1. one inst. inst. for each event Question allocate instruments download data during play to each new allocation in advance low memory requirement
Option 1.2.2.1 Option 1.2

2. limited no. inst. inst. and reallocation events (?) store data only + reload data into existing UG
* low data transferplay rate req. during
Option 1.2.2
prestore data elsewhere
Option 1.2.2.2
store data in second UG and reconnect
* low processor calculation req.

during play low amount of decision making
Option 1.2.3
download data bit by bit in real time
*
low (high-lvel) software calculation req. during play
FIGURE 6: Overview of QOC analysis to show how it was restructured in order to refine the Options (Option numbers added to show the transition). Note that the process of making the Option hierarchy explicit prompted the designer to change Option 1.3 to Option 1.2.3, and to decompose Option 1.2.2 one level further. (As the designer was using pen and paper and renamed and restructured extensively, the original QOC was littered with changes which have been omitted for clarity). (From Buckingham Shum, 1994) In a second incident, identification of a parent-child relationship between Criteria led to them being merged. Figure 7 shows two Options and three Criteria. The designer decided that (the top Criterion) low processor calculation reqt. during play was a child of the more general Criterion low amount of decision making, and grouped them together. However, he then realised that in fact they were independent, but recognised another relation insteadthat low amount of decision making served (the bottom Criterion) low (high-level) software calculation reqt. during play. He unboxed the first pair, and boxed the second pair (see Figure 7). This was an interesting conclusion to have reached, given that the Assessments were not identical for each of these 20
Criteria, as one might have expected if one was a direct consequence of the other. Nonetheless, the designer was clearly following this logic, as he proceeded to delete duplicate Assessment links to the constituent Criteria in the new composite.
(Options) (Criteria)
low processor calculation req. during play
download data during play to each new allocation in advance download data bit by bit in real time
low amount of decision making low (high-level) software calculation req. during play low processor calculation req. during play low amount of decision making
download data during play to each new allocation in advance download data bit by bit in real time
low (high-level) software calculation req. during play
FIGURE 7: Options and Criteria from part of a QOC before and after regrouping Criteria. Note that duplicate Assessment links to the members of the new composite Criterion have been deleted. Although it is possible that there was no causal relationship between using QOC and the insights gained (ideas do arise serendipitously out of the blue), the video data and verbal transcript of the designers use of the representations do appear to show that the requirement to articulate Options and Criteria forced him to clarify relationships and distinctions which had up until that point remained unaddressed. The designer declared that these were new insights, rather than old ones simply recorded as QOC. This may be taken as observational experimental E1 evidence. Argumentation-based DR was used in a slightly different role by Bellotti (1993), who used QOC as a common framework for integrating argumentation about a user interface design from multiple parties. Insights into the problem were contributed by designers, cognitive modellers, and system modellers, and re-expressed in terms of Questions, Options and Criteria. This study constitutes self-reported experimental E1 evidence (the researcher was also the QOC author), and makes the claim that QOC can be useful by clarifying the nature of, and relationships between, different contributions from theoretical HCI modelling. The effectiveness of using QOC in this way for communicating modelling to designers (E2 evidence) has yet to be investigated.
21
If we turn to E2 evidence regarding the longer term utility of DR for outside parties, there are no realistic studies to be found as yet. All incidents of E2 evidence reported so far have been isolated incidents over the short term. However, one objective, experimental study has been conducted into the intelligibilty of QOC to parties not involved in the original deliberation context (Shum, 1993). In this video-based laboratory study, seven HCI researchers were asked to search QOC representations in order to retrieve DR about two hypertext systems.The study investigated the cognitive tasks in retrieving relevant QOC in response to a query, and studied how users managed multiple representations of a QOCwhen answering queries. Subjects had available to them for each system, in both textual and graph forms, and at different levels of detail representations of the QOC, plus a hierarchical Criterion tree showing interCriterion relationships (all on large sheets of paper, not online). The main aims of this work were to document the success with which subjects were able to retrieve DR in response to a set of queries, and the demands of trying to formulate appropriate structural requests in QOC. It was shown that information which matched the QOCs Question structure was easily retrieved, but that certain types of query requiring more complex search patterns caused difficulties; in these cases different levels of detail and perspectives on the QOCs were necessary. A cognitive task model of the QOC retrieval process was developed as an analytic framework, and different classes of errors in specifying QOC browsers were traced to different points in the task model. Requirements for computational support for QOC retrieval were also made. The general conclusion was that the coherence of a QOCs structure is critical to its reusability, but that this requirement is in tension with its semiformality; this is particularly the case when the QOCs structure is shaped by the deliberation process, rather than its conceptual content. Moving from the laboratory to the field, both E1 and E2-related issues were addressed when QOC was used for about 9 months to document the evolution of design concepts over the life of a three year collaborative academic-industrial project (Shum, MacLean, Forder and Hammond, 1993). The goal of this exercise was the production of a DR document which retrospectively structured the projects fragmented memory into a more accessible form, which could also support the incremental addition of concurrent rationale as the project progressed. QOC was used to organise what were termed Sources (design discussions from many contexts and in different media), through a process of filtering (i.e. reviewing relevance), integrating (pooling otherwise fragmented Sources), and indexing (providing links into the body of Sources for reference). All of the QOC work was carried out by a DR researcher, although QOC constructs came to be used by one designer as outline headings in project discussion documents. With respect to E1 evidence, Questions in particular proved to be a useful construct in the process of collating and rationalising the Source materials. As the disparate sources slowly revealed clusters of related discussion, initially, Questions provided general conceptual pigeonholes into which related material could be posted; once sufficient Sources had been collated, the Question-based approach forced the DR researcher to tease out the key issues, and make the areas of common ground explicit. Sometimes it was possible to expand Questions into local design spaces in which Options could be evaluated against Criteria. In the report, examples of such Questions are presented with a level of detail characteristic of what we have been classifying as detailed anecdotal (E1) evidence. Whilst QOC served as a useful framework in this respect (E1), a major goal was to see if a DR created by a relative outsider would prove useful to the design team whose ideas it represented (E2). This is an interesting twist to the more usual E2 scenario (designers creating DR as a for others). At a meeting to evaluate the results of the DR work 22
(approximately 50 pages indexing the design as QOC), the answer to this was a clear No. Shum et al., identified in this evaluation several obstacles to DR documents acceptability, including the notational form (QOC graphs and tables versus lighterweight structures), the content of the DR (the importance of capturing dominant themes and principles), ease of DR navigation and retrieval (such a large resource needs hypertext support), and management issues (coming late into the project, it was difficult to get commitment to use DR from the design team). This and the previous study represent initial investigations into what makes DR intelligible and useful as a resource. In-depth experimental-based analyses should of course be complemented by broader field studies of DR re-use. There are many interesting issues to explore regarding the extent to which a DR constructed by one community in a specific context (developers), proves either intelligible or useful to another community (e.g. maintenance; management; upgraders) with their own set of concerns and background. The richness and complexity of real work environments suggests that ethnographic studies could prove fruitful in understanding of how organisational knowledge and memory are passed on (cf. Orr, 1991). Let us now consider E3 empirical evidence instances where using an argumentation scheme seemed to impede rather than facilitate work. McCall (1986; 1991) extended the IBIS notation by adding a quasi-hierarchical Issue-serves-Issue structure (Procedural Hierarchy of IssuesPHI). The prime representational benefit is that it helps to focus deliberation on Issues which serve the stated aims of the design (i.e. the top Issue)if an issue cannot be shown to serve part of the hierarchy, then it is likely to be irrelevant. McCall argues that PHI improves IBIS as a stimulus to reflective reasoning, but as presented this seems to be an intuition rather than an empirical conclusion. Of interest, therefore, are the contradictory conclusions of studies into PHIs use for architectural design (as reported by Fisher et al., 1991). They report that using PHI on its own prolonged the reflective process and encouraged a lot of analysis of high level or peripheral issues, distracting designers from getting down to implementational issues. These studies were conducted using design students over several years, with analyses of PHIs use varying from assessment of student assignments, to detailed video protocol analysis of individual students. As such, these data are classed as observational experimental E3 evidence of an argumentation scheme obstructing the design process. As another example of E3 evidence, Buckingham Shum (1994) describes a situation in which the argumentative mode of reasoning proved incompatible with the construction task at hand. A designer who was familiar with QOC (and who had found it useful in a previous design session) encountered severe difficulties in breaking another problem into discrete Questions, Options and Criteria. Analysis of this session led to the conclusion that he was engaged in depth-first design, in effect refining a single, complex Option (a Smalltalk-80 data structure), as opposed to the breadth-first reasoning required in an earlier session when QOC had been useful (clarifying the relationships between several alternative implementations). The trend towards construction-driven argumentation DR research is reflecting a growing recognition that argumentation about a design must be driven by implementational concerns at some level, or DR can remain too abstract or address problems of little consequence. As Fischer et al., (1991) put it, In a good design project, construction generates and regulates argumentation. Argumentation arises out of construction and is often tested by construction. (p.403). The artifact being constructed may be any product of the design process, from initial requirements to code. The point is that being engaged in developing the artifact itself is different from reflecting on it. 23
This perspective is reflected in ongoing efforts to build software environments which integrate DR much more tightly with the artifact under development. Reeves and Shipman (1992b) tie rationale created by system maintainers to network diagrams; Ryan (1992) has enriched the Smalltalk environment with QOC for class hierarchies; Carey and Crentsil (1992) embed QOC in user interface toolkit widgets, and Potts (1994) has considered different ways in which IBIS can be more closely linked to software engineering representations. Fischer et al., (1991) and McCall, Bennet, DOronzio, Ostwald, Shipman and Wallace (1990) have focused explicitly on the separation between Issue based deliberation and construction of the artifact, to the extent of modularising, but closely linking, these two components in their JANUS , and more recently, PHIDIAS design environments. Knowledge-based critics monitor a CAD design, and when a guideline is violated, alert the designer, who can then display the PHI argumentation relevant to the problem, in order to assist reflective decisionmaking. Concepts developed from Schns (1983) analysis of expert design, particularly reflection-in-action, have been a source of inspiration and rationalisation in this work. Schns model was based on observational studies of experts working, or tutoring junior designers, and as such, is relatively well-grounded. However, although JANUS and PHIDIAS are presented as work environments strongly based on Schns model of reflective practice, there is no evidence to confirm that reifying this model as either a software architecture, or as the users conceptual model, actually supports design activity in the intended way. Questions which require empirical answers include: how much do designers consult the PHI argumentation? do they modify it? does the issue-base work as a stimulus to reflection? is the PHI argumentation judged to be relevant or at an appropriate level of detail for the current problem being considered? have the efforts to ease switching between the two modes of working changed designers work patterns? Nonetheless, the reflection-construction perspective continues to have increasing influence (see the work cited above). Reports of these tools in use are needed to demonstrate whether this is an effective strategy to pursue. An important question arises at this point: where does this leave the essence of the argumentative approach collaborative, issue-based deliberation as a way of designing? In identifying artifact construction as the primary activity of design, have efforts to bring DR closer to the artifact in fact left the argumentative process by the wayside? Is argumentative deliberation no longer regarded as a central activity, but rather, one into which one switches only to reflect on a breakdown in construction? We suggest that it should be possible to benefit from the insights into reflective construction, as well draw on the power of collaborative, explicit argumentation. Notations such as IBIS, DRL, and QOC should be used to gain breadth-first analytic understanding of issues, setting the context for concrete refinementdesign is both analysis and implementation. This goes further than simply re-expressing the reflectionconstruction cycle. Where some would see construction as the primary trigger for recording DR, we would argue strongly for the proactive (as opposed to reactive) use of argumentative reasoning for breadth-first analytical design, closer to Rittels original conception. The balance needs to be maintained between being concrete and construction-based, and more analytic and issue-based; between being driven bottom-up by the concrete implementational concerns, and working top-down to understand the 24
underlying issues at stake. Moreover, an important part of being able to build cumulative design knowledge is the process of abstracting from the details of a particular design to learn the general lessons. Reflecting on what have emerged as the key issues or dimensions following a period of construction is a natural way to preserve the important elements of reasoning for future use. Furthermore, if constructing DR is not considered to be a valid engine to drive design reasoning (as the artifact is), an implication is that the DR representation used does not in fact need to be especially suited to ill-structured problems. Since it is not being used to explore the problem space, but simply to document incidents or decisions arising from construction, free text notes are in some ways preferable, since they detract as little attention as possible from the main activity (coding; sketching; using CAD; writing a specification). Indeed, this move to unstructured DR, with few or no constraints on form or content, is exactly what we see in approaches which place construction at the centre. Conversely, those who continue to argue for argumentative DR as a mode of designing, use the more structured issue-based notations needed to support initial exploration and articulation of the problem space, and subsequently, management and checking of the DR. In brief, is a DR a legitimate artifact in its own right, whose construction will itself talk back ( Schn) to designers, and so inform the concrete artifact under construction? At present, the answer to this question seems to place DR representations and methods into two camps. These have yet to be integrated in a design environment which supports analytic issue-based deliberation (as supported by gIBIS, QOC, or SIBYL), but which also allows rationale of different degrees of formality to be linked to design artifacts of different sorts, as it is triggered by construction activity (such as JANUS, XNETWORK, or the VERDI environment Potts, 1994).
3.4. CLAIM 1 (UTILITY)CONCLUSIONS
Claim 1 asserts that expressing design ideas as argumentation is useful in different ways (addressed by evidence classes E1-3). In reviewing the literature, certain trends were identified, and the strength of the evidence assessed in design and non-design domains. We shall summarise these in turn, beginning with the trends. The Claim 1 analysis identified the work of Rittel et al., on wicked problems as a complementary root to that sparked by Englebart and Toulmin, which contributed to the adoption of argumentative notations in design. It was concluded that evidence for the existence of a distinctive class of wicked, or ill-structured design problems is good. In addition, a trend in argumentation-based DR work which was noted is towards integrating DR with the software artifact being constructed, to encourage a tight coupling between reflective and constructive modes of designing. Proponents of such approaches assert that without construction driving the DR, argumentation-based DR is of little use. However, there has as yet been no analysis reported of the way in which such environments are used, leaving this claim about the role of argumentation-based DR unsubstantiated. To balance this trend, it has been suggested that Rittels original insight into the power of argumentative deliberation as a proactive strategy for framing and shaping problems should not be rejected wholesale in favour of more recent insights into reflective designing, which relegate DR to what in the first instance is a passive, reactive role. The implications of each perspective need to be combined within a common environment which supports argumentation-based DR in both roles.
25
Turning in more detail to the strength of the empirical evidence, Figure 8 summarises the results of examining this literature.6 TABLE 1 Summary of the semiformal argumentation literature surveyed in this paper under Claim 1.
E1 evidence Using an argumentation scheme augments reasoning non-DR Observational experimental Hair, 1990 Storrs, 1989 (Toulmin form adequate & usable, but not clear that it was useful) Reader & Hammond, 1993 (educational concept mappingdescrib ed under Claim 2) Self-reported experimental Hair & Lewis, 1990 (failed to demon-strate) Newman & Marshall, 1991 (no effect reported) Detailed anecdotal VanLehn, 1985 (NoteCards graphs and matrix) Shum et al., 1993 Bellotti, 1993 Hair & Lewis, 1990 (failed to demonstrate) Newman & Marshall, 1991 (no effect reported) McKerlie & MacLean, 1994, in press Shum et al., 1993 (failed to demonstrate) Burgess Yakemovic & Conklin, 1990 McKerlie & MacLean, in press Burgess Yakemovic & Conklin, 1990 McKerlie & MacLean, 1994, in press McKerlie & MacLean, 1994, in press Newman & Marshall, 1991 Lewis et al., 1991 DR Buckingham Shum, 1994 Buckingham Shum & Hammond, this paper E2 evidence A record of previous argumentation is useful non-DR DR Shum, 1993 E3 evidence Using an argumentation scheme impedes reasoning non-DR Hair, 1990 DR Fischer et al., 1991 Buckingham Shum, 1994
General anecdotal
Citing of a source indicates that at least one incident arose of the specified class. Sources have only been included if the notation was evaluated in use. Studies are italicised and comments have been added if the results were negative, or inconclusive.
Sources will be used in describing citations in Figure , since not all can be described as studies.
26
The matrix shows clearly the empirical weakness of argumentation research in nondesign domains. It is of course a relatively coarse-grained indicator, and it should be remembered that citation of a study indicates that at least one incident was reported in the particular class. However, the density, as well as complete absence of sources in some cells provides a useful snapshot of the state of research at present. If we consider the non-design literature first, there is strikingly little evidence that using an argumentation scheme augments reasoning (E1 has one source). Three sources failed to demonstrate or report an E1 effect, though two others do report effects using semiformal knowledge structuring tools, but not argumentation schemes as studied in this paper. There is no evidence that argumentation schemes are useful for communicating with others (E2 two sources failed to show effects), but two sources present E3 evidence of argumentation impeding work. The situation is more optimistic for argumentation-based DR, in that six sources provide E1 evidence, though two of them are general anecdotal (i.e. the weakest form of evidence). Four sources support the E2 claim that DR can be a reusable resource, although one source reported failure to create such a DR. Finally, there is relatively strong E3 evidence illustrating conditions when argumentation in design can be obstructive (four sources, two of them observational experimental studies, and neither of the others general anecdotal reports). Such results are valuable for understanding the scope of the representations being developed. In most scientific research, the absence of properly replicated studies is regarded as significant weakness in the science base. This is an evident feature in the argumentation literature, and reflects much HCI research in general. In response, we, like many others, would be prepared to make the case that applied research into design domains like HCI can legitimately emphasise other, more pragmatic concerns, such as satisfying users, maintaining ecological validity, and the reality of a more product driven research agenda, which leaves few resources to meticulously replicate others results. However, given this weakened state, even more care needs to be taken in accepting, and building on theoretical claims. Analysis of semiformal argumentation as an approach has demonstrated that one of its foundational claims that explicit structure assists reasoning and communication is empirically much weaker than many assume. 4. Claim 2: Argumentation-based DR is usable Analysis of Claim 1 focussed on evidence relating to the end-product and benefits gained from using argumentation-based DR approaches. However, substantiating Claim 1 on its own is insufficientwhat is the cost of using argumentation-based DR, in order to reap these benefits? Utility and usability are clearly related, so it is not surprising that some of the discussion about E1-3 evidence has already overlapped with issues of ease of use (for instance, failure to communicate via a DR is clearly related to the ease with which information can be extracted). However, this section considers some important but as yet unaddressed implications for the user of such notations. Claim 2 asserts that designers are able to use argumentation-based DR representations. Compared to Claim 1, this claim is made less explicitly by most proponents of argumentation-based DR. It seems to be more of an assumption that semiformal notations do not pose significant overheads on designers. In Carroll et al.s sense, therefore, this class of notations makes the claim of being usable because little is said or done to suggest otherwise. Earlier discussion has indicated that the issue of overheads is recognised, but the solution adopted by some has been simply to abandon structured representations for informal notes. If the advantages of semiformally
27
structuring DR are to be gained, it falls to proponents of such approaches to ensure that designers can use them.
4.1. CLAIM 2 (USABILITY)NON-DESIGN EVIDENCE
In this section, we make the case that expressing ideas in the language of semiformal notations is a non-trivial task, and that there is highly relevant work in fields not specifically concerned with design, such as writing and concept mapping, from which DR research can learn. We identify an important theme which could be characterised as bridging the representational gulf. This gulf is the difference between the form and order of DR-material generated by designers, and that required by DR notations. The implications of this gulf will become clearer as the discussion progresses. Hypertext systems, the software technology most commonly used to represent and manage argumentation-based DR, were much acclaimed early on as the ideal representational tool, since they support processing by both humans (rich, informal node content) and machines (operations across formal entity and relation types). However, soon after systems such as NoteCards (Halasz, et al., 1987) and gIBIS (Conklin and Begeman, 1988) began to be used for structuring ideas, reports began to emerge of phenomena such as cognitive overhead and premature structuring (Conklin, 1987; Halasz, et al., 1987; Marshall, 1987; Fischer, 1988; Halasz, 1988; Conklin and Begeman, 1989) . For many users, the representational demands of parsing ideas into discrete nodes, with distinctive names and types seemed to impede the flow of thought, and moreover, the resultant structures were hard to change. However, these early warning signs tended to go unheeded amidst the enthusiasm and excitement surrounding hypertext at this time, and many developers assumed that hypertext designers would be able to structure content without undue difficulty. Substantially fewer reports appeared which analysed in detail the congruity of semiformal representations with the cognitive and conversational structures used by the targetted users of these systems. Could users speak the languages for thought being proposed? A field in which perhaps more effort than usual has been devoted to understanding users requirements is computer-supported writing. This research area already had a strong tradition of cognitive analysis, and the emergence of hypertext resulted in attempts to relate and adapt existing models of the writing process to the new media (e.g. Smith, Weiss and Ferguson, 1987; Sharples and Pemberton, 1988; Neuwirth and Kaufer, 1989; Streitz, et al., 1989). The challenge of providing computational support for externalising creative ideas in a structured form overlaps significantly with that facing proponents of argumentationbased DR. Writing has been frequently described as a design task (Thomas and Carroll, 1979; Streitz, et al., 1989), and the distinction between writing and argumentation-based DR is blurred even further by writing tools which support explicit argumentation via notations like Toulmin and IBIS (Schuler and Smith, 1990; Hashim, 1991). As writing processes have been studied far more extensively than DR creation processes, it is instructive to consider some of the analyses and solutions developed, especially those motivated by cognitive analyses of writing. To begin, consider the following extracts, which illustrate the related concerns facing hypertext authoring, computer-supported writing, and argumentation-based DR:
28
ARGUMENTATION-BASED DESIGN R ATIONALE We will need tools for massaging, organizing, browsing something more akin to a stream of consciousness than a carefully thought out structure. Brown (1982: p.2), on requirements for a future writing environment In early problem solving stages the enforcement of structure may get in the way. Fischer (1988: p.223), on current problems with hypertext One common but subtle difficulty in hypertext systems is that it is sometimes unnatural to break ones thoughts into discrete units, particularly when the problem is not well understood and those thoughts are vague, confused, and shifting. ...the cognitive overhead of having to segment the muck into discrete thoughts, identify their types, label them and link them is prohibitive. Conklin and Begeman (1988: p.148), on authoring with gIBIS
The key feature of research into writing-support tools is that it is dominated by a concern to facilitate the smooth externalisation of internally represented structures, without constraining the author counterproductively. The emphasis is on providing multiple representations of the ideas to be organised, covering the whole representational process from recording initial ideas, to restructuring networks, to finetuning surface form. Writing and design rationale clearly have different end goals, but in this important insightthat users need to be supported as they recruit and construct different representations depending on the state of development of ideasthe ideastructuring task faced by the user of one of these writing tools is very close in essence to that faced by a designer wishing to represent design arguments. To concretise our point, we now briefly describe two writing support systems which exemplify the cognitively motivated approach. Streitz et al., (1989) describe a writers tool which emphasises the mapping between cognitive models of the writing process and the tools user interface and functionality. The user interface presents four windows in which to work, which are termed activity spaces, corresponding to modes of idea structuring derived from models of writing. The spaces are planning (a meta-level for organising authoring), content (linked notes to be incorporated into the document), argumentation (Toulmin based organisation of arguments), and rhetoric (shaping the document to the audience and purpose of writing). The aim is to enable the writer to externalise ideas of different sorts, at any point, through one of the activity spaces. A close relationship is drawn between writing and design, viewing the former as an example of the latter, as both involve constraint management. Their stated aim of creating an environment for structured thoughtdumping could easily apply to DR. Schuler and Smith (1990) describe the implementation in more detail, which in particular uses PHI (McCall, 1991), an extension of IBIS, to organise arguments, and Toulmin to justify Position-Argument links. SEPIA has since been extended to support collaborative work (Haake and Wilson, 1992; Haake and Haake, 1993). However, despite the task-centred principles underlying the design, there is no evaluative data of single or multi-user versions in use. Verification of the strong claims embedded in the design would be interesting to see. The second example to consider is an analysis conducted in the development of the editing system (Neuwirth, Kaufer, Chandhok and Morris, 1990). Neuwirth and Kaufer (1989) discuss what makes a good external representation? They begin by listing some of the problems which even skilled writers have (all of which could equally be applied to open-ended conceptual design):
PREP
focus on details at the expense of larger goals, and distraction by irrelevant information; forgetting of useful information; searching for information for which there is only a partial specification; 29
selection of incorrect paths leading to backtracking; losing track of goals.
Drawing on the ACT* cognitive architecture (Anderson, 1983), Neuwirth and Kaufer present some criteria by which to evaluate writing representations (which, again, are clearly generalisable to other domains): encoding: what is the number of elements encoded internally, which are relevant to the task, and how easy is it to operate on the internal representation produced by the external representation? (This differentiates between informational content and the cognitive tractability of its presentation); storage and retrieval: what is the likelihood that appropriate information will be chunked in, and retrieved from, declarative memory? (This depends on how taskrelated information is grouped in the external representation. Retrieval as a chunk reduces the need for subsequent searches for related information); controlling cognition: how effectively can the user maintain current goals, the overall goal stack, and store intermediate results of operations?
This level of analysis is much more detailed than that often presented as the rationale for representational support tools, focussing as it does on the cognitive processes which may underlie the more general claims of supporting reasoning. Moreover, their generic form also makes it possible to consider their implications for design rationale. To summarise, computer-supported writing research provides a stimulating parallel research stream and resource for argumentation-based DR research. It highlights the challenge for DR research to bridge the conceptual gap between early ideas and semiformal arguments, by acknowledging the need for multiple representations. An interesting complement to this can be found in the concept mapping literature, which emphasises that although expressing ideas within a structure takes effort, this need not necessarily be counterproductive. Concept mapping is an approach, developed in the educational research domain, in which students , perhaps instead of writing an essay, produce a graphical representation of the conceptual structure of a topic, such as a hierarchy, a spider diagram, or semantic net (JRST, 1990; Novak, 1990; Kommers et al., 1992). Relationships are often labelled (e.g. is a, has bodypart, obeys the law), and by analysing maps qualitatively, and to some extent quantitatively, comparisons can be made between students, and assessments made of development over time. More recently, software support to ease the mechanics of constructing and editing maps has emerged (Fisher, 1992; Kozma, 1992; Twidale, Rodden and Sommerville, 1993b). Concept mapping is intended to deepen understanding of a domain through the discipline of expressing knowledge within a structural framework, forcing the student to articulate important distinctions and relationships. Similarly, argumentation schemes aim to clarify reasoning by forcing parties to make explicit important assumptions, distinctions, and relationships as they construct and critique arguments. Intuitively, therefore, one might hypothesise that similar processes are involved in the two activities. A study which lends support to concept mapping claims is reported by Reader and Hammond (1993), in which users of a concept mapping tool showed evidence of better learning than a control group using a simple textual notetaking tool (observational experimental E1 evidence). The precise reasons for effects such as this are not deducible from the experiments design (and indeed are a matter of keen debate within the area, e.g. Trapp, Reader and Hammond, 1992). One would therefore be wary of inferring that effects on the learning of student material will directly transfer to the design context, and enable designers to 30
benefit from structuring DR as argumentation. Learning may be a peculiar domain with unique priorities and task demands. However, studies like this strengthen claims that semiformal knowledge structuring can be beneficial to the user, and will encourage DR research in its pursuit of this goal. To summarise, there is a fine line between providing a structure which helpfully constrains the expression of ideas by forcing the user to think about the nature of, and relationships between ideas, and cognitively straitjacketing the user by providing too elaborate (or constrained) a vocabulary which holds up the flow of ideas. This became clear as a general problem within the early idea processing hypertext community. Writing tools research illustrates how this insight can be combined with domain-specific analyses of the different ways in which writers think and represent their ideas, and concept mapping research has tried to exploit the Procrustean discipline of fitting ideas into a limited structure. Each of these serves as a warning for the developers of DR notational schemes of the new cognitive tasks which they will be introducing to the design context.
4.2. CLAIM 2 (USABILITY)DESIGN RATIONALE EVIDENCE
Claim 2 asserts that argumentation-based DR does not make excessive representational demands on designers. The non-DR research surveyed above focusses attention on precisely this issue, strongly indicating that semiformal schemes for expressing knowledge introduce extra demands. Given this historical and conceptual context, we turn now to examine what is known about the cognitive representational demands of argumentation-based DR approaches. The only observational experimental studies of the representational process associated with a DR notation are by Buckingham Shum (1994), who trained groups of designers in QOC, and then studied them working in different combinations, for several kinds of software design. These studies documented the strategies used by designers to manage the core representational tasks of QOC (classifying, naming, linking, and restructuring ideas), and the ways in which they expressed ideas which QOCs notation did not support. (As described earlier in the context of DRs utility, the studies also documented situations in which QOC proved to both facilitate, as well as obstruct, design). Based on this evidence and the related hypertext literature considered in the previous section, it was proposed that designers need more informal representations to ease the construction of QOC. By making what was termed rough QOC a legitimate representation which relaxed the notational constraints on how argumentation could be recorded, the designer could, for instance, record isolated Questions or untyped things, and reorganise them until understanding had developed to allow them to be named, typed, and linked. The implications for tool support are significant, and are discussed later (these studies were conducted using pen and paper). It is instructive to note a parallel between the results of these QOC studies and the computer-supported writing research discussed above. Although there are clearly many issues yet to be resolved in the latter field, five years ago, Sharples and Pemberton (1988) characterised a consensus model of writing which had been consolidated over a decades research. If we consider the main features of this generic writing model, the QOC studies demonstrate that argumentative DR-construction displays similar characteristics, as summarised in Figure 9.
31
B UCKINGHAM SHUM & HAMMOND Consensus model of writing Results of QOC observational studies
Writing models must cover the whole process of A QOC DR moves from the relatively writing, from initial mustering of ideas, to final unstructured mass of ideas in initial idea presentation on paper. generation and debate, to a crafted structure which organises those ideas to different degrees (depending on the DR approach). The act of writing triggers new insights, leading to The process of reifying reasoning as QOC leads new intentions, and plan revision. designers to reformulate the initial expression of ideas in a number of beneficial ways. There are many strategies to writing. A tool should Designers adopt different, sometimes explicitly enable writers to choose, and subsequently support declared strategies in constructing QOCs, the process which best fits their mode of working. depending on which ideas suggested themselves first. Writing is managed by setting constraints; writers An initial styleguide describing attributes of can be taught well-defined techniques of constraint well-structured QOC has been developed, with the management. aim of assisting designers in conducting accurate, elegant, succinct design space analyses, as well as enhance QOC DR as a vehicle for communicating reasoning. Elements have been included in QOC training materials.
FIGURE 9: Parallels between a consensus model of the writing process, and the process of using an argumentation-based DR notation. Lewis, Rieman and Bell (1991) have proposed that design is inherently problemcentred, and moreover, that based on their experiences, it is too distracting for designers to abstract from concrete to more general characterisations of those problems as required by most DR notations. Consequently, in a design project of their own they expressed DR informally as a series of problems and alternatives. Several patterns were observed in their own deliberations as to how problems, subproblems, and alternatives interacted to move the design forward (e.g. micro problem derived from raw problem, design alternatives spawned by micro problem). The observation that design is often problem-centred provides insight into the ways in which Questions, Options and Criteria can productively interact to move a design forward. Notationally, it is simple to re-express the patterns reported by Lewis et al., as dynamics in a semiformal DR scheme (e.g. focussed Question derived from general Question, and Options generated for this consequent Question). Evidently, the key question is whether problems can be easily or usefully re-expressed in these terms during design. The QOC studies described above indicate that they can, but emphasise that reformulation is often necessary as an unknown design space is explored. Ultimately, it is the importance which the designers place on having a DR which (i) aids the reasoning process (E1 evidence), and (ii) which others can use (E2), which will dictate the extent to which this effort is judged worthwhile. Whilst Lewis et al., might argue that their DR assisted their own reasoning because of the lower overheads, it is unlikely that it was intelligible as a design reasource for others. Although there are no other empirical analyses of the demands of DR notations and the subsequent authoring strategies devised by designers, anecdotal reports coupled with growing awareness within parts of the hypertext community (see also Shipman and Marshall, 1993), has led to an interesting trend towards the informalisation of 32
notations which were originally presented as ready-to-use semiformal schemes. There are several examples of this. Firstly, Lee, who has developed DRL, one of the more expressive argumentation notations (described earlier) has more recently emphasised the capabilities of SIBYL, DRLs tool, for incrementally formalising DR, starting with informal natural language descriptions (Lee, 1992). Secondly, as reported above, a simplified indented-textual form of IBIS (itIBIS) was used in a field study (Burgess Yakemovic and Conklin, 1990), in contrast to the more cryptic, graphical gIBIS representation. However, even the itIBIS format had to be paraphrased into normal English when communicating with parties outside the IBISliterate group. Efforts to develop a collaborative version of gIBIS (rIBIS Rein and Ellis, 1991) have also encountered problems with the notation. All but one of a series of 16 meetings was described as mostly unsatisfying and frustrating by their participants, with significant difficulties encountered in using the IBIS method to structure discussions. It was concluded that the main causes of the problems were participants inexperience with IBIS notation, and the complexity of the rIBIS user interface. Thirdly, more recent efforts to utilise PHI (McCall, 1991 see earlier discussion) have emphasised the importance of beginning with informal textual rationale, with the option to formalise this using a semi-structured form (Reeves and Shipman, 1992a; Reeves and Shipman, 1992b). There are no reports as to whether and under what circumstances designers find it worthwhile to formalise DR in this way. We would hypothesise that they will only go to the trouble when there are clear, short-term benefits to be gained.
4.3. CLAIM 2 (USABILITY)CONCLUSIONS
This section has examined evidence pertaining to the claim that the overheads of representing argumentation-based DR are tractable for designers. An initial assumption was that designers would be able to directly express rationale in terms of semiformal node-link structures. The emerging picture from research into hypertext, writing, concept mapping, and idea processing points strongly to the need for more sophisticated understandings of the way in which designers can be expected to use argumentation schemes, based on the cognitive constraints involved in using such notations. 5. Agenda for future research We reflect now on the implications of the research surveyed. These are discussed on two fronts: approaches to bridging the representational gulf between designers and DR representations, and the need to strengthen the conceptual foundations on which argumentation-based DR approach has been built.
5.1. BRIDGING THE REPRESENTATIONAL GULF: NOTATIONS, TRAINING, AND TOOLS
The difference between the internal, cognitive forms in which rationale is utilised by designers, and the form in which a DR notation requires them to externalise this rationale, has been termed the representational gulf. Somehow this must be bridged in order to express and interpret DR, the challenge being to make crossing this bridge as efficient and smooth a process as possible. The term efficiency is used here rather than simplicity, since it takes into account both the effort required, coupled with the benefits which exerting this effort may provide (e.g. minimal effort might be simply to 33
record spoken DR, but sharing, maintaining and reusing such a DR would be extremely difficult). As in all ergonomics, the gulf can be effectively narrowed in three ways: by modifying the language of communication (the notation/dialogue), modifying the human (training), and modifying the representational medium (tool support). The distinction between the notation and the tool environment is blurred, but as will be seen, is useful to make (cf. Green, 1989). These are considered briefly in turn in the context of DR.
5.2. NOTATIONS
Notations, when understood as languages for thinking, are extremely powerful ways of manipulating views of the world. A language, whether for human or machine, makes certain aspects of the world salient, and hides others. Notations encourage the user to fit the content of their thoughts into a particular structure, which invariably involves categorising in a particular way. Whilst the social and political dimensions of categories have been extensively studied (e.g. Sacks, 1979) and are beginning to be considered in the context of HCI (Suchman, 1993), this paper has limited discussion of DR notations to the dimension of the cognitive. In this respect, therefore, a semiformal notation places the explicit requirement on its user to categorise ideas into the node and link types which it provides. Designing a notation to be usable and effective at this level would take into account at least the following factors: (i) How closely does the vocabulary of the notation (its node and link types) match that of the designers at which it is aimed? For example, can designers understand what a Claim or Warrant is? Does the extra work introduced by using particular categories or grammatical constructions assist reasoning in any way? For example, do guidelines on what counts as a Criterion (MacLean, et al., 1991), or what a good Question is (Bellotti, MacLean and Moran, 1991) encourage designers to reflect more deliberately, or improve their analysis of problems?
(ii)
(iii) Related to the above, to what extent does the notation recognise the process which designers may follow in formalising their ideas? An example of this would be provision for entities which are not named, linked, or classified, and conversely, specialisations of types. (The ability to define the latter by example would be a tool environment feature see next section7). Developers of argumentation-based DR have addressed points (i) and (ii), albeit briefly. For example, Burgess Yakemovic and Conklin wonder, Is there any magic in IBIS? (...) Would simply organizing the same kinds of information in an outline format, without the IBIS classifications, have been just as valuable? (p.116). They conclude simply that the structuring of problems into its constituent IBIS components is a challenging but useful intellectual exercise. Similarly, Lee (1990), in presenting DRL, records that it was decided not to refine the supports relation into causal support and evidential support, because the difference was too fine for general use. It would be extremely valuable if studies investigated which notational constructs designers were able to work with most easily, or how they tailored a standard notation to their own needs.
Rogers and Marshall (1992) report the demand for and subsequent provision of a facility to redefine representational schemas from an in situ example.
34
Since design invariably involves meetings, an analysis of cognitive compatibility should also cover group cognition as well as that of individuals. What role do DR representations play in such collaborative settings? From the literature and our own work with QOC, it has become clear that a DR notation can shape design discussion and hence, presumably, the design to a greater or lesser extent depending on the way it is used. At one end of the continuum, the state of the DR plays a major role in driving discussion, for instance, the meeting agenda is to debate and resolve a set of IBIS Issues, and to record that process. In this case, the discussion will be largely conducted in terms of the notations rhetorical moves, and the DR is owned by the whole group. The DR representations influence can then be gradually weakened in the extent to which the recorded argumentation shapes subsequent deliberation, examples of which are shown in Figure 10.
Prescriptive DR Developing a good DR representation is a central, collaborative activity in the meeting; ideas are developed through informal DR, and then edited and restructured as ideas evolve, to create a coherent summary. Creating DR is still a central, shared activity, but its structure is shaped by the discussion process, not the conceptual structure of the ideas (less rationalisation); DR used to monitor progress and shape discussion to some extent, but largely documentary. DR scribe records discussion, but other designers use the DR as a shared representation to monitor their progress and shape discussion to some extent; largely documentary (a single author version of the one above) DR scribe records discussion, which is then reviewed later for forgotten ideas, weaknesses in reasoning, action items, and so forth; largely documentary DR scribe records discussion, which is only reused if information is needed; no restructuring; purely documentary
Descriptive DR
FIGURE 10: Ways in which DR can be used, depending on the extent to which creating the DR shapes the meeting. The empirical and conceptual basis of this analysis is presented elsewhere (Buckingham Shum, MacLean, Bellotti and Hammond, 1993). Its relevance to the present discussion should, however, be self-evident: the context in which a notation is used will determine the expertise present, and the time available to structure deliberationthese what, when, and how factors cannot be ignored when designing a notation.
5.3. TRAINING
Research and practice in organisational change have established that technological intervention requires preparation on the ground, in the form of communication with, and training of those it will affect (e.g. Dawson, 1986; McLoughlin and Clark, 1988; Orlikowski, 1992). The importance of training in DR is now beginning to be recognised in DR research (Burgess Yakemovic and Conklin, 1990; MacLean, Bellotti and Shum, 1993), and is an integral part of the CM/1 gIBIS-based product (CMSI, 1992). Whilst most DR approaches are relatively easy to grasp in terms of simply understanding why a DR might be useful, and how rationale might be expressed using a 35
particular notation, from our experiences in communicating our work to design teams, it has proven quite a different matter to actually use it in design. Both DR studies cited above make it clear that there is no substitute for hands-on practice. The development of a tutorial for the QOC Design Space Analysis approach is described in detail elsewhere (MacLean, et al., 1993). The important point in the context of this paper is that it is based not only on in-house experience with QOC on the part of the developers of the approach, but on studies of its use, as reported in the claims analysis above. The challenge of communicating the approach to practitioners encouraged us to consider the practical implications of our research analyses, which were then codified in what we hoped would be appropriate forms. As the demands of conducting productive QOC analyses became clearer from empirical studies, the tutorial materials used to train designers in those studies was revised to sensitize the next set of designers to important features of QOC authoring (MacLean et al., 1993). Consequently much greater emphasis was placed on the following points: The notation can be used to produce poor representations
The tutorial included several examples of poor QOC due to unfocussed Questions, poorly differentiated Options, and common confusions between QOC node types. A set of heuristics (MacLean, et al., 1991) for constructing QOC representations of the design space was explained, and made available during their design sessions.
Rough, evolving QOC representations must often precede coherent analyses

Rough QOC was introduced as a concept earlier in this paper. A new feature in the tutorial was its emphasis on rough QOC as a stage in the representational process in its own right. During the design exercises, an A3 sheet was provided expressly for the purposes of drawing up first-pass rough QOC, making incomplete DRs legitimate structures to work with.
Developing QOC structures is a non-linear process

In order to provide some process support for designers using QOC for the first time, a model of the typical stages was devised, from initial gathering and structuring of potential DR, to the fleshing out and restructuring of the QOC design space, ending in decision making.
To summarise, a community of workers who adopt a new representation or method for working must adapt; by definition, a new way of working requires learning, and DR is no exception. It is anticipated that the serious adoption of DR in the development process will have quite profound consequences at cognitive, group, and organisational levels (cf. Grudin, 1994). Effective training facilitates an individual, project, or organisation to change the relevant structures (e.g. cognitive; social; managerial) more quickly, and highlights pitfalls to avoid at each level. The successful transfer from being a research vehicle to a realistic tool often depends on early successes within the client organisation, and it is here that training has a critical role to play.
5.4. TOOLS
A persistent theme to emerge from this analysis of current DR research is that of the need for a flexible representational environment which can support the crucial process elements of the task of rationalising decisions. In short, designers can rarely pour well articulated, coherent arguments direct from their heads into a semiformal notational structure. The process is one of refinement, and when in a group context, negotiation. Within the hypertext and collaborative design research communities, it is therefore encouraging to see a recent emphasis on the importance of allowing designers to express themselves ambiguously, rather than precisely and explicitly. For instance, Marshall and Rogers (1992) emphasise the importance of being able to work with unlinked, but meaningful clusters of nodes in the early stages of developing a 36
representational schema in the Aquanet hypertext system (Marshall, Halasz, Rogers and Janssen, 1991). The requirements for a knowledge structuring tool which have been specified for Aquanet could provide an environment which meets many of the likely requirements for a DR tool. In another project, Twidale, Rodden and Sommerville (1993) describe the Designers Notepad (DNP) for supporting the earliest stages of recording and organising ideas. DNP has been used experimentally for early brainstorming in writing, concept mapping, and software design/design rationale. Much effort has been devoted to minimising the intrusiveness of the tool, by providing flexible schemes for organising nodes and links, such as colour, shape, and spatial arrangement. The Inspiration brainstorming tool (I.S.I., 1992) is a commercially available example with similar functionality and ease of use. Another promising effort in this direction is the Tivoli electronic whiteboard (Pedersen, McCall, Moran and Halasz, 1993). Tivoli provides pen-based manipulation of groups of text and graphic objects in order to support common representational tasks in group work around a whiteboard. Delaying commitment to explicit structure comes at a cost: avoiding typing and linking in favour of supporting the gradual emergence of coherence from ill-structured material, trades off against the provision of powerful abstractions and the explicit articulation of relationships. These concerns have been succinctly expressed by Green (1989), who identifies them as dimensions germane to any environment for structuring information (respectively: premature commitment, role expressiveness, viscosity, and hiddendependency). A link between requirements such as these, and tools which might satisfy them was proposed by Shum (1991), who illustrated how these dimensions could be operationalised with respect to DR notations. The trade-offs incurred as one moves from the informal to the semiformal is illustrated schematically in Figure 11.
37
Rationalisation of ideas
undeveloped ideas
ideas, status & relationships unclear
how to do Y?? how to do Y?? how to do Y??
Cognitive Dimensions
no commitment to classify ideas status of ideas is implicit (untyped) local changes are simple to make few abstractions make global changes hard many relationships are implicit
? ?
how to do Y?? how to do Y??
how to do Y??
intermediate rough DR
increasing number of ideas status of ideas becoming clearer refinement of entities/restructuring
O: pop-up Q: what kind of menus? O: pull-down C: user orientation to position of menu options C: speed of access
O: top Q: where should cursor be on pop-up menu? O: middle
C: consistency of position
C: adapt to user
xxx?
?
how to do Y??
O: qqqqq
coherent, reusable DR emerging

Q: what kind of menus? O: pop-up O: pull-do wn C: user orientation to position of menu options C: speed of access C: context sensitivity of menu options
O: ?? top Q: where should cursor be on pop-up menu? O: middle O: last used option
C: consistency of position C: minimise average cursor distance to selection C: adapt to user
commitment to classify ideas status of ideas is explicit (types) tool support needed to restructure locally abstractions make global changes possible more relationships are explicit
FIGURE 11: Trade-offs between cognitive dimensions at different stages in the process of rationalising argumentation. The curved arrows indicate that the process is non-linear: designers need to switch between representations opportunistically. In sum, as representational tools become increasingly able to support the subtleties of the process outlined in Figure 11, the use of semiformal notations becomes a more tractable prospect. As the basic mechanics of externalising and organising ideas become less of a problem, other more subtle effects may become apparent, such the impact of different vocabularies. Given the development of very low constraint representational tools such as DNP and Inspiration, the next challenge is to support the designer in formalising the results of a brainstorming session into argumentative form (the transition shown in Figure 11). So for instance, how easy is it to select three unclassified DNP notes, declare them to be QOC Options, and link and position them neatly with another note which is declared to be a Question?; or to select a Criterion, and change it to a new Question, or convert two Criteria into one, with the associated link-management? These are examples of routine QOC restructuring tasks (Buckingham Shum, 1994). Without support for structural manipulations of this sort, the designer will find it easy to externalise initial ideas, but then face a tedious rewriting or copying task to convert them to more rigorous argumentation. With the addition of structural search facilities, automatic verification of certain sorts becomes possible. For example, one could check that no decisions have been made without considering negative assessments against Criteria (an indication of 38
confirmation bias), or that there are no Questions which address more than one important issue.8 In summary, the challenge for tool developers is to make shifting between degrees of formality as smooth and transparent as possible.
5.5. STRENGTHENING DESIGN RATIONALE RESEARCH
This paper has examined research which address the claims of utility and usability of argumentation-based approaches to DR. These are foundational claimsif they are demonstrated to be invalid or pragmatically impossible to achieve in the software development context, then there is little point in pursuing this particular strategy to representing DR. The analyses of research pertaining to Claims 1 and 2 have already been summarised. Analysis of Claim 1 (DRs utility) demonstrated that there are surprisingly few well documented examples of semiformal argumentation schemes proving useful for argument formulation or communication. Analysis of Claim 2 (usability) concluded that research in areas closely related to DR (hypertext; knowledge structuring; computersupported writing) point unwaveringly to the cognitive effort of parsing thoughts in order to fit them into a semiformal notation. This effort may be beneficial or an obstacle, depending on the task context, and whether one is trying to brainstorm and record new ideas, or make sense of an existing body of ideas. From the DR perspective, more sources were found which report incidents of argumentation supporting design reasoning. Of equal importance in circumscribing the scope of application, are reports which describe contexts in which such formalisms proved inappropriate and unhelpful. Of some concern for DR supporters will be the absence at present of realistic, longitudinal studies which track the utility and usability of DR in supporting the recovery and reuse of design reasoning. Thus far, there has been one experimental study, and reports of isolated incidents from field studies. Although this may be because DR research is relatively young, it is of prime importance that longitudinal studies are run in real development environments in order to understand DR re-use issues empirically. Readers will be only too aware of the difficulties in setting up a longitudinal DR study, requiring as it does a motivated design team and supportive management. There appears, however, to be no alternative. Interestingly, design knowledge re-use in engineering design appears to be more advanced than in software design, and may provide pointers for directions in which software DR research could productively move. In particular, there are now several studies which have looked in detail at the information requests which designers make when modifying or maintaining an existing design (Baudin, Sivard and Zweben, 1990; Kuffner and Ullman, 1990; Baudin, Gevins, Baya and Mabogunje, 1992). Understanding the target communitys interests in this way is a priority for software DR research. A recent report by Herbsleb and Kuwana (1993) is a good example of the kind of close analysis which is needed in order to better understand DRs niche in design practice. In our analysis, a number of themes were identified within ongoing DR research activity. More recent developments in DR are indicative of a trend towards tighter
If there is more than one issue embedded in the Question, different Options often respond to different requirements, and the phenomenon of Criterion bunching occurs, which can be detected as a structural patternsome Criteria are relevant to and assess only one Option, whilst the rest assess another.
39
integration of DR with the artifact under design. This was applauded, but also treated with caution lest the principle insights of Rittel are completely lost, and argumentative process, instead of being a proactive strategy for analytic design, became passive and reactive. Also noted was the increased awareness within the idea processing/knowledge structuring communities of the importance of supporting the opportunistic, messy process which often characterises the design of coherent, concise products (whether they be hypertexts, written documents, programs, concept maps, or DRs). This theme is further strengthened by links with ethnographic and other computersupported cooperative work which emphasise the social dimensions to heavily cognitive processes like design (e.g. Minneman, 1991). The claims analysed in this paper were essentially cognitive in nature, with the primary focus of attention being on notational fit to designers. A facet not addressed at this level was the role of negotiated conventions for using DR notations within a local design community, a process touched on in other argumentation work (Newman and Marshall, 1991; Marshall and Rogers, 1992). We venture to suggest that the absence of a strong methodological tradition such as that found in the social sciences, has some part to play in contributing to the current situation within argumentation research. Computer sciences tradition of proving-bybuilding is still dominant in much HCI work. Empirical demonstration and analysis of claims for usability and effectiveness is vital if interactive technology is to be built on principled grounds, rather than a mutually constructed and self-perpetuating folklore. Intriguingly, this trend persists even when researchers have been concerned enough to motivate their work by task-driven analyses. Basing a tool around a model derived from an analysis or meta-analysis of the work process as performed without computational support (e.g. writing or collaborative drawing) is a powerful first step to designing appropriate technology. Two examples would be computer supported writing tools based around writing models (e.g. Hayes and Flower, 1980; Scardamalia and Bereiter, 1987), or design environments based around a model of expert design practice (e.g. Schn, 1983). However, it cannot be assumed that this automatically leads to a usable tool; it is critical that the theories embedded within these artifacts are then verified as appropriate, since the theory may have been misrepresented in the process of concretising it, or the task may have been changed in important respects by the replacement of traditional media with computational media. A good example of iteration around this cycle is the work of Tang (1989; Tang and Leifer, 1988) . Studies of design groups use of drawing spaces led to the development of several tools which validated his recommendations for group sketching tools (e.g. Tang and Minneman, 1990; Greenberg and Bohnet, 1991; Lu and Mantei, 1991; Tang and Minneman, 1991). To broaden DRs scope a little, a new context for DR use may be that of participatory design (e.g. Floyd, Mehl, Reisin, Schmidt and Wolf, 1989; Muller and Kuhns, 1992). DRs emphasis on articulating the design process resonates closely with those who argue for a fundamental shift in software engineering to greater process-orientation (e.g. Floyd, 1987), and the dialectic process encouraged by Rittel is echoed by perspectives which emphasise the central importance of change and contradiction in system development (Matthiassen, 1987), and the reconciling of conflicting interests (Nygaard and Srgaard, 1987; Sandberg, 1979). It is therefore not surprising to find that participatory design perspectives and argumentation-based DR are beginning to be combined in fruitful ways (Goldkhl, 1991; Timpka, Nyce, Sjberg and Johansson, 1993). In conclusion, argumentation-based DR has been shown to be resting on rather more unstable conceptual and historical foundations than is often assumed by proponents of such approaches. However, systematic analyses of argumentation-based DR in use, in 40
conjunction with research in related areas, are beginning to allow substantiated claims to be made about the conditions under which explicit design argumentation is useful, and how it can be more easily expressed. These clarify the requirements for notational, instructional, and computational design to bring designers and argumentation-based DR closer together.
We are grateful to Allan MacLean, Jean McKendree, Will Reader and the journal reviewers, whose detailed comments helped to strengthen this paper. This analysis was conducted during projects funded by Rank Xerox EuroPARC, UK Science and Engineering Research Council (CASE Award 88504176), and Esprit Basic Research Actions 3066 and 7040 (AMODEUS Projects 1 & 2).
6. References
Anderson, J. R. (1983). The Architecture of Cognition. Cambridge, MA: Harvard University Press. Arango, G., Bruneau, L., Cloarec, J.-F. and Feroldi, A. (1991). A Tool Shell for Tracking Design Decisions. IEEE Software March, 75-83. Baudin, C., Gevins, J., Baya, V. and Mabogunje, A. (1992). Using Domain Concepts to Index Engineering Design Information. Proceedings of CogSci-92: 14th Annual Conference of the Cognitive Science Society, Bloomington, Indiana, Lawrence Erlbaum Associates: Hillsdale, NJ. Baudin, C., Sivard, C. and Zweben, M. (1990). Recovering Rationale for Design Changes: A Knowledge-Based Approach. IEEE Proceedings for Man, Machines and Cybernetics, Los Angeles, CA, IEEE. Bellotti, V. M. E. (1993). Integrating Theoreticians and Practitioners Perspectives with Design Rationale. Proceedings of InterCHI93: Human Factors in Computing Systems, Amsterdam, Holland, 101-106, ACM Press: NY. Bellotti, V. M. E., MacLean, A. and Moran, T. (1991). Structuring the Design Space by Formulating Appropriate Design Rationale Questions. SIGCHI Bulletin 23 (4), 80-81. (Extended version available as: Amodeus Working Paper RP6/WP6, Rank Xerox EuroPARC) Bernstein, B. (1992). MacEuclid. Department of Computer Science, University of Colorado at Boulder, University of Colorado Box 430, Boulder, CO 80309-0430. (Available by anonymous ftp from: info-mac@sumex-aim.stanford.edu) Bobrow, D. G., Stefik, M., Foster, G., Halasz, F., Lanning, S. and Tatar, D. (1990). The Colab Project Final Report. Xerox Palo Alto Research Center, Technical Report SSL-90-45 Brown, J. S. (1982). Notes Concerning Desired Functionality, Issues and Philosophy for an Authoringland. Xerox Palo Alto Research Center, CIS Working Paper Brown, J. S. (1983). Process Versus Product: A Perspective on Tools for Communal and Informal Electronic Learning. Report from The Learning Lab: Education in the Electronic Age, by the Educational Broadcasting Corporation; Reprinted in Journal of Educational Computing Research 1 (2), 179-202 (1985). Brown, J. S. (1986). From Cognitive Ergonomics to Social Ergonomics and Beyond. In D. A. Norman and S. W. Draper, Ed. User Centered System Design, 457-486. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Buckingham Shum, S. (1994). Analyzing the Usability of a Design Rationale Notation. In T. P. Moran and J. M. Carroll, Ed. Design Rationale: Concepts, Techniques, and Use, Hillsdale, NJ: Lawrence Erlbaum Associates. (Earlier version available as Amodeus-2 Working Paper TA/WP6) Buckingham Shum, S., MacLean, A., Bellotti, V. M. E. and Hammond, N. (1993). Learning to Use Argumentation-Based Design Rationale. Amodeus-2 Project, Working Paper TA/WP19, December, 1993. (Submitted for publication) Burgess Yakemovic, K. C. and Conklin, J. (1990). Report on a Development Project Use of an IssueBased Information System. Proceedings of CSCW90: Computer Supported Cooperative Work, 105-118, ACM: New York.
41
B UCKINGHAM SHUM & HAMMOND Carey, T. T. and Crentsil, J. (1992). Integrating Widget Design Knowledge with User Interface Toolkits. Proceedings IEEE CASE Workshop, Montreal, July, 1992, Department of Computing and Information Science, University of Guelph, Canada. Carroll, J. M. and Kellogg, W. A. (1989). Artifacts as Theory Nexus: Hermeneutics Meets TheoryBased Design. Proceedings of CHI89: Human Factors in Computing Systems, ACM: New York. Carroll, J. M. and Moran, T. P. (1991). (Eds.) Human-Computer Interaction (Special Issue on Design Rationale) 6 (3 & 4), Carroll, J. M. and Rosson, M. B. (1991). Deliberated Evolution: Stalking The View Matcher in Design Space. Human-Computer Interaction 6 (3&4), 281-318. (Reprinted in: Moran T.P. and Carroll J.M. (eds.) Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates (1994)) CMSI (1992). CM/1 Product Description. Corporate Memory Systems, Inc., 8920 Business Park Dr., Austin, Texas 78759. Conklin, J. (1987). Hypertext: An Introduction and Survey. IEEE Computer 20 (9), 17-41. Conklin, J. (1989). Design Rationale and Maintainability. Proceedings 22nd Hawaii International Conference on System Science, 555-561, IEEE Computer Society Press. Conklin, J. and Begeman, M. L. (1988). gIBIS: A Hypertext Tool for Exploratory Policy Discussion. Transactions on Office Information Systems 6 (4), 303-331. Conklin, J. and Begeman, M. L. (1989). gIBIS: A Tool for All Reasons. Journal of the American Society for Information Science May, 200-213. Conklin, J. and Burgess Yakemovic, K. C. (1991). A Process-Oriented Approach to Design Rationale. Human-Computer Interaction 6 (3&4), 357-391. (Reprinted in: Moran T.P. and Carroll J.M. (eds.) Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates (1994)) Dawson, S. (1986). Analysing Organisations. London: Macmillan. Englebart, D. C. (1963). A Conceptual Framework for the Augmentation of Mans Intellect. In Vistas in Information Handling, 1-29. London: Spartan Books: Washington, DC. (Reprinted in: Greif, I., Ed. (1987). Computer-Supported Cooperative Work: A Book of Readings, pp. 35-65. Morgan Kaufmann Publishers: San Mateo, California) Fischer, G. (1988). Position Statement on Panel Session: A Critical Assessment of Hypertext Systems. Proceedings of CHI88: Human Factors in Computing Systems, 223-227, ACM: New York. Fischer, G., Lemke, A. C., McCall, R. and Morch, A. I. (1991). Making Argumentation Serve Design. Human-Computer Interaction 6 (3&4), 393-419. (Reprinted in: Moran T.P. and Carroll J.M. (eds.) Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates (1994)) Fisher, K. (1992). SemNet: A Tool for Personal Knowledge Construction. In P. A. M. Kommers, D. H. Jonassen and J. T. Mayes, Ed. Cognitive Tools for Learning, 63-75. Berlin: Springer-Verlag. Floyd, C. (1987). Outline of a Paradigm Change in Software Engineering. In G. Bjerknes, P. Ehn and M. Kyng, Ed. Computer and Democracy A Scandinavian Challenge, Aldershot: UK: Avebury. Floyd, C., Mehl, W.-M., Reisin, F.-M., Schmidt, G. and Wolf, G. (1989). Out of Scandinavia: Alternative Approaches to Software Design and System Development. Human-Computer Interaction 4 (4), 253-350. Goel, V. and Pirolli, P. (1989). Motivating the Notion of Generic Design Within InformationProcessing Theory: The Design Problem Space. AI Magazine (Spring), 18-36. Goel, V. and Pirolli, P. (1991). The Structure of Design Problem Spaces. Cognitive Science 16 (3), 395429. Goldkhl, G. (1991). Information Systems Design as Argumentation: An Investigation into Design Rationale as a Conceptualization of Design. 14th IRIS Conference (Information System Research In Scandinavia), Ume (11-14 August) Green, T. R. G. (1989). Cognitive Dimensions of Notations. In A. Sutcliffe and L. Macaulay, Ed. People and Computers V, 443-460. Cambridge: Cambridge University Press.
42
ARGUMENTATION-BASED DESIGN R ATIONALE Greenberg, S. and Bohnet, R. (1991). Groupsketch: A Multi-User Sketchpad for GeographicallyDistributed Small Groups. Proceedings of Graphics Interface91 Grudin, J. (1988). Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces. In Proceedings of ACM CSCW88 Conference on Computer-Supported Cooperative Work, 85-93. Grudin, J. (1994). Evaluating Opportunities for Design Capture. In T. P. Moran and J. M. Carroll, Ed. Design Rationale: Concepts, Techniques, and Use, Hillsdale, NJ: Lawrence Erlbaum Associates. Haake, A. and Haake, J. M. (1993). Take CoVer: Exploiting Version Support in Cooperative Systems. In Proceedings of InterCHI'93: Human Factors in Computing Systems, 406-413. Addison Wesley. Haake, J. M. and Wilson, B. (1992). Supporting Collaborative Writing of Hyperdocuments in SEPIA. In Proceedings of CSCW92: ACM Conference on Computer-Supported Cooperative Work, 138146. Toronto, Canada: ACM: New York. Hair, D. C. (1990). Using the Interface to Assist in the Construction of Legal Arguments. Department of Computer Science, University of Colorado, Technical Report Hair, D. C. (1991). LEGALESE: A Legal Argumentation Tool. ACM SIGCHI Bulletin 23 (1), 71-74. Hair, D. C. and Lewis, C. (1990). Are Argumentation Representation Schemes Useful? Department of Computer Science, University of Colorado At Boulder, Boulder, Colorado 80309-1592, Technical Report CU-CS-475-90 Halasz, F. G. (1988). Reflections on Notecards: Seven Issues for the Next Generation of Hypermedia Systems. Communications of the ACM 31, 836-852. Halasz, F. G., Moran, T. P. and Trigg, R. H. (1987). Notecards in a Nutshell. Proceedings of CHI and GI87: Human Factors in Computing Systems and Graphic Interface, 45-52, ACM: New York. Harrison, S., Minneman, S. and Stults, B. (1988). The Media Space - Experience with Video Support of Design Activity. Proceedings of the International Workshop on Engineering Design and Manufacturing Management, Melbourne, Australia (Nov. 1988), 114-126 Hashim, S. H. (1991). WHAT: An Argumentative Groupware Approach for Organizing and Documenting Research Activities. Journal of Organizational Computing 1 (3), 275-302. Hayes, J. R. and Flower, L. S. (1980). Identifying the Organisation of Writing Processes. In L. W. Gregg and E. R. Steinberg, Ed. Cognitive Processes in Writing, 3-30. Hillsdale: New Jersey: Lawrence Erlbaum Associates. Herbsleb, J. D. and Kuwana, E. (1993). Preserving Knowledge in Design Projects: What Designers Need to Know. In Proceedings of InterCHI'93: Human Factors in Computing Systems, 7-14. Addison Wesley. I.S.I. (1992). Inspiration. Inspiration Software Inc., 2920 S.W. Dolph Court, Suite 3, Portland, Oregon 97219, USA. Jarczyk, A., Loffler, P. and Shipman, F. (1992). Design Rationale for Software Engineering: A Survey. Proceedings of 25th Annual Hawaii International Conference on System Sciences (8-10 January, 1992) Johnson, P. (1992). Collaborative Software Review for Capturing Design Rationale. Working Notes, AAAI92 Workshop on Design Rationale Capture and Use, July 15, 1992, San Jose: CA(Also available as Technical Report ICS-TR 92-01, Department of Information and Computer Sciences, University of Hawaii) JRST (1990). Special Issue on Concept Mapping. Journal of Research and Science Teaching 27 (10), Kozma, R. B. (1992). Constructing Knowledge with Learning Tool. In P. A. M. Kommers, D. H. Jonassen and J. T. Mayes, Ed. Mindtools: Cognitive Technologies for Modelling Knowledge, Berlin: Springer-Verlag. Kuffner, T. A. and Ullman, D. G. (1990). The Information Requests of Mechanical Design Engineers. Proceedings of 2nd American Society of Mechanical Engineers Conference on Design
43
B UCKINGHAM SHUM & HAMMOND Automation (Design Theory and Methodology Track)., Chicago,Illinois, 167-174, ASME: New York. Kunz, W. and Rittel, H. W. J. (1970). Issues as Elements of Information Systems. Institut fr Gundlagen Der Planung I.A, Universitt Stuttgart, Kleperstrae 11, 7000 Stuttgart 1, Germany, Technical Report S-78-2 Lee, J. (1990). SIBYL: A Tool for Managing Group Design Rationale. Computer Supported Cooperative Work, Los Angeles, CA, 79-92, ACM Press: New York. Lee, J. (1992). Incremental Acquisition and Formalization of Design Rationale. Working Notes, AAAI92 Workshop on Design Rationale Capture and Use, July 15, 1992, San Jose: CA, 179-188 Lee, J. and Lai, K. (1991a). A Comparative Analysis of Design Rationale Representations. Center for Coordination Science, Massachusetts Institute of Technology CCS Technical Report 121 Lee, J. and Lai, K. (1991b). Whats in Design Rationale? Human-Computer Interaction 6 (3&4), 251280. (Reprinted in: Moran T.P. and Carroll J.M. (eds.) Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates (1994)) Lowe, D. G. (1985). Cooperative Structuring of Information: The Representation of Reasoning and Debate. International Journal of Man-Machine Studies 23, 97-111. Lowe, D. G. (1986). SYNVIEW: The Design of a System for Cooperative Structuring of Information. Proceedings of CSCW86, Austin, Texas (Dec. 3-5) Lu, I. and Mantei, M. (1991). Idea Management in a Shared Drawing Tool. Proceedings of European Computer Supported Cooperative Work Conference (ECSCW91) MacLean, A., Bellotti, V. and Shum, S. (1993). Developing the Design Space with Design Space Analysis. In P. F. Byerley, P. J. Barnard and J. May, Ed. Computers, Communication and Usability: Design Issues, Research and Methods for Integrated Services, Amsterdam: Elsevier: North Holland Series in Telecommunication. MacLean, A., Young, R. M., Bellotti, V. and Moran, T. (1991). Questions, Options, and Criteria: Elements of Design Space Analysis. Human-Computer Interaction 6 (3&4), 201-250. (Reprinted in: Moran T.P. and Carroll J.M. (eds.) Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates (1994)) MacLean, A., Young, R. M. and Moran, T. (1989). Design Rationale: The Argument Behind The Artifact. Proceedings of CHI89: Human Factors in Computing Systems, 247-252, ACM: New York. Marshall, C. C. (1987). Exploring Representation Problems Using Hypertext. Proceedings of Hypertext87, 253-268, ACM: New York. Marshall, C. C. (1989). Representing the Structure of Legal Arguments. Proceedings of the International Conference on AI and Law, Vancouver, B.C., Canada (June, 1989) Marshall, C. C., Halasz, F. G., Rogers, R. A. and Janssen, W. C. (1991). Aquanet: A Hypertext Tool to Hold Your Knowledge in Place. Proceedings of Hypertext91, ACM: New York. Marshall, C. C. and Rogers, R. A. (1992). Two Years before the Mist: Experiences with Aquanet. In Proceedings of the Fourth ACM Conference on Hypertext, 53-62. Matthiassen, L. (1987). Systems, Processes, and Structures: A Contribution to the Theoretical Foundation of Information Systems Development and Use. In P. Docherty, K. Fuchs-Kittowski, P. Kolm and L. Mathiassen, Ed. System Design for Human Development and Productivity: Participation and Beyond, Amsterdam: North Holland. McCall, R. J. (1986). Issue-Serve Systems: A Descriptive Theory of Design. Design Methods and Theories 20 (8), 443-458. McCall, R. J. (1991). PHI: A Conceptual Foundation for Design Hypermedia. Design Studies 12 (1), 30-41. McCall, R. J., Bennet, P., DOronzio, P., Ostwald, J., Shipman, F. and Wallace, N. (1990). PHIDIAS: A PHI-Based Design Environment Integrating CAD Graphics into Dynamic Hypertext. In A.
44
ARGUMENTATION-BASED DESIGN R ATIONALE Rizk, N. Streitz and J. Andr, Ed. Hypertext: Concepts, Systems and Applications, Cambridge: Cambridge University Press. McDermid, J. A. (1991). Safety Arguments, Software and System Reliability. Proceedings of the International Symposium on Software Reliability Engineering, Los Alamitos, CA, IEEE Computer Society. McKerlie, D. and MacLean, A. (1993). QOC in Action: Using Design Rationale to Support Design. Video Program, InterCHI93: Human Factors in Computing Systems, Amsterdam, Netherlands, ACM SIGGRAPH Video Review Series. McKerlie, D. and MacLean, A. (in press). Reasoning with Design Rationale: Practical Experience with Design Space Analysis. Design Studies , McLoughlin, I. and Clark, J. (1988). Technological Change at Work. Milton Keynes: Open University. Minneman, S. (1991). The Social Construction of a Technical Reality: Empirical Studies of Group Engineering Design Practice. Xerox Palo Alto Research Center, Technical Report SSL-91-22 Moran, T. P. and Carroll, J. M., Ed. (1994). Design Rationale: Concepts, Techniques, and Use. Hillsdale, NJ: Lawrence Erlbaum Associates. Muller, M. and Kuhns, S. (1992). Proceedings of. Participatory Design92: Second Conference on Participatory Design, Boston, Mass. Naur, P. (1989). The Place of Strictly Defined Notation in Human Insight. Proceedings of the Workshop on Programming Logic, Gteborg, Sweden (May, 1989), 429-443, Programming and Methodology Group, University of Gteborg and Chalmers University of Technology, Gteborg, Sweden. (Reprinted in: Naur, P. (1992). Computing: A Human Activity. ACM Press: New York. Addison Wesley Publishing Company. pp.468-478) Neuwirth, C. M. and Kaufer, D. S. (1989). The Role of External Representations in the Writing Process: Implications for the Design of Hypertext-Based Writing Tools. In ACM Hypertext89 Proceedings, 319-341. Neuwirth, C. M., Kaufer, D. S., Chandhok, R. and Morris, J. H. (1990). Issues in the Design of Computer Support for Co-Authoring and Commenting. In Proceedings of ACM CSCW90 Conference on Computer-Supported Cooperative Work, 183-195. Newell, A. (1969). Heuristic Programming: Ill-Structured Problems. In J. Aronofsky, Ed. Progress in Operations Research, Vol.3, 360-414. New York: Wiley. Newman, S. and Marshall, C. (1991). Pushing Toulmin Too Far: Learning from an Argument Representation Scheme. Xerox Palo Alto Research Center Technical Report Novak, J. D. (1990). Concept Maps and Vee Diagrams: Two Metacognitive Tools to Facilitate Meaningful Learning. Instructional Science 19, 1-25. Nygaard, K. and Srgaard, P. (1987). The Perspective Concept in Informatics. In G. Bjerknes, P. Ehn and M. Kyng, Ed. Computers and Democracy A Scandinavian Challenge, Aldershot: UK: Avebury. Olson, G. M., Olson, J. S., Carter, M. R. and Storrsten, M. (1992). Small Group Design Meetings: An Analysis of Collaboration. Human-Computer Interaction 7 (4), 347-374. Orlikowski, W. J. (1992). Learning from Notes: Organizational Issues in Groupware Implementation. CSCW92: Proceedings of the Conference on Computer-Supported Cooperative Work, Toronto, Canada (Oct.31-Nov.4), 362-369, ACM Press: New York. Orr, J. (1991). Talking About Machines. An Ethnography of a Modern Job. Xerox Palo Alto Research Center, Technical Report SSL-91-07 Ostwald, J., Burns, B. and Morch, A. (1992). The Evolving Artifact Approach to System Building. Working Notes, AAAI92 Workshop on Design Rationale Capture and Use, July 15, 1992, San Jose, CA Pedersen, E. R., McCall, K., Moran, T. P. and Halasz, F. G. (1993). Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings. In Proceedings of InterCHI'93: Human Factors in Computing Systems, 391-398. Addison Wesley.
45
B UCKINGHAM SHUM & HAMMOND Potts, C. (1994). Supporting Software Design: Integrating Design Methods and Design Rationale. In T. P. Moran and J. M. Carroll, Ed. Design Rationale: Concepts, Techniques and Use, Lawrence Erlbaum Associates. Potts, C. and Bruns, G. (1988). Recording the Reasons for Design Decisions. Proceedings of 10th International Conference on Software Engineering, 418-427 Reader, W. and Hammond, N. (1993). Computer-Based Support for Access and Learning Strategies in Hypertext. Psychology Teaching Review 2 (2), 89-95. Reeves, B. and Shipman, F. (1992a). Making It Easy for Designers to Provide Design Rationale. Working Notes, AAAI92 Workshop on Design Rationale Capture and Use, July 15, 1992, San Jose: CA Reeves, B. and Shipman, F. (1992b). Supporting Communication Between Designers with ArtifactCentered Evolving Information Spaces. CSCW92: Computer-Supported Cooperative Work, Oct 31-Nov 4, 1992, Toronto, Canada, 394-401, ACM Press: New York. Rein, G. L. and Ellis, C. A. (1991). rIBIS: A Real-Time Group Hypertext System. International Journal of Man-Machine Studies 24 (3), 349-367. (Also in Greenberg S. (Ed.) Computer Supported Cooperative Work and Groupware, 223-241. (1991) Academic Press: London) Reitman, W. R. (1964). Heuristic Decision Procedures, Open Constraints, and the Structure of IllDefined Problems. In M. W. Shelly and G. L. Bryan, Ed. Human Judgements and Optimality, New York: John Wiley & Sons. Rittel, H. W. J. (1972). Second Generation Design Methods. In N. Cross, Ed. Developments in Design Methodology, (1984), 317-327. Chichester: J. Wiley & Sons. ((Originally an interview in) the DMG 5th Anniversary Report: DMG Occasional Paper No.1, 5-10) Rittel, H. W. J. and Webber, M. M. (1973). Dilemmas in a General Theory of Planning. Policy Sciences 4, 155-169. Ryan, M. (1992). An Interactive Tool for the Construction, Modification and Browsing of Design Rationales of Smalltalk Programs. Masters Thesis , Queen Mary and Westfield College, University of London Sacks, H. (1979). Hotrodder: A Revolutionary Category. In G. Psathas, Ed. Everyday Language: Studies in Ethnomethdology, 7-14. New York: Irvington. Sandberg, . (1979). Computers Dividing Man and Work. Stockholm: Swedish Centre for Working Life. Scardamalia, M. and Bereiter, C. (1987). Knowledge Telling and Knowledge Transformation in Written Composition. In S. Rosenberg, Ed. Advances in Applied Psycholinguistics: Vol.2. Reading, Writing, and Language Learning, 142-175. Cambridge: Cambridge University Press. Schn, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. New York: Basic Books. Schuler, W. and Smith, J. (1990). Authors Argumentation Assistant (AAA): A Hypertext-Based Authoring Tool for Argumentative Texts. In A. Rizk, N. Streitz and J. Andr, Ed. Hypertext: Concepts, Systems and Applications, Cambridge: Cambridge University Press. Sharples, M. and Pemberton, L. (1988). Representing Writing: An Account of the Writing Process with Regard to the Writers External Representations. School of Cognitive and Computing Science, University of Sussex, Cognitive Science Report 119 Shipman, F. M. and Marshall, C. C. (1993). Formality Considered Harmful: Experiences, Emerging Themes, and Directions. University of Colorado Department of Computer Science, Boulder, Colorado, Technical Report CU-CS-648-93(Submitted for publication) Shum, S. (1991a). A Cognitive Analysis of Design Rationale Representation. Doctoral Dissertation , HCI Group, Department of Psychology, University of York, YO1 5DD, U.K., (Available by a n o n y m o u s f t p f r o m : f t p . m r c apu.cam.ac.uk::/pub/amodeus/related/Shum_DesignRationale_PhD.hqx).
46
ARGUMENTATION-BASED DESIGN R ATIONALE Shum, S. (1991b). Cognitive Dimensions of Design Rationale. In D. Diaper and N. V. Hammond, Ed. People and Computers VI: Proceedings of HCI91, 331-344. Cambridge: Cambridge University Press. Shum, S. (1992). Empirical Studies of the Representational Process in Using Argumentation-Based Design Rationale. Working Notes, AAAI92 Workshop on Design Rationale Capture and Use, July 15, 1992, San Jose, CA Shum, S. (1993). QOC Design Rationale Retrieval: A Cognitive Task Analysis, & Design Implications. Rank Xerox EuroPARC, Technical Report EPC-93-105 Shum, S., MacLean, A., Forder, J. and Hammond, N. V. (1993). Summarising the Evolution of Design Concepts Within a Design Rationale Framework. Adjunct Proceedings of InterCHI93 (Short Papers), Amsterdam, 43-44(Also available as Amodeus-2 Project-Working Paper ID/WP5) Simon, H. A. (1973). The Structure of Ill-Structured Problems. Artificial Intelligence 4 (181-201), Smith, J. B., Weiss, S. F. and Ferguson, G. J. (1987). A Hypertext Writing Environment and its Cognitive Basis. In ACM Hypertext87 Proceedings, 195-214. ACM: New York. Smolensky, P., Bell, B., Fox, B., King, R. and Lewis, C. (1987). Constraint-Based Hypertext for Argumentation. In ACM Hypertext87 Proceedings, 215-245. ACM: New York. Smolensky, P., Fox, B., King, R. and Lewis, C. (1988). Computer-Aided Reasoned Discourse Or, How to Argue with a Computer. In R. Guindon, Ed. Cognitive Science and its Application to HumanComputer Interaction, 109-162. Norwood, New Jersey: Ablex. Storrs, G. (1989). Group Working in the DHSS Large Demonstrator Project. ECSCW89: First European Conference on Computer-Supported Cooperative Work, London (13-15 Sept.) Streitz, N., Haake, J., Hannemann, J., Lemke, A., Schuler, W., Schuett, H. and Thring, M. (1992). SEPIA: A Cooperative Hypermedia Environment. Proceedings of the Fourth International ACM Conference on Hypertext, Milan, Italy (Nov. 30 - Dec. 4), 11 - 22, New York: ACM Press. Streitz, N., Hanneman, J. and Thring, M. (1989). From Ideas and Arguments to Hyperdocuments: Travelling Through Activity Spaces. Proceedings of Hypertext89, 343-364, ACM: New York. Suchman, L. (1993). Do Categories have Politics? The Language/Action Perspective Reconsidered. 3 rd European Conference on Computer-Supported Cooperative Work, Milan, Italy (13-17 Sept.), 1-14, Kluwer Academic Publishers. Tang, J. C. (1989). Listing, Drawing and Gesturing in Design: A Study of the Use of Shared Workspaces by Design Teams. Xerox Palo Alto Research Center, Technical Report SSL-893(Doctoral Dissertation, Stanford University) Tang, J. C. (1991). Findings From Observational Studies of Collaborative Work. International Journal of Man-Machine Studies 34 (2), 143-160. (Also in Greenberg S. (Ed.) Computer Supported Cooperative Work and Groupware, 11-28. (1991) Academic Press: London) Tang, J. C. and Leifer, L. J. (1988). A Framework for Understanding the Workspace Activity of Design Teams. Proceedings of ACM CSCW88 Conference on Computer-Supported Cooperative Work, Portland, Oregon, 244-249, ACM: New York. Tang, J. C. and Minneman, S. L. (1990). VideoDraw: A Video Interface for Collaborative Drawing. Proceedings of ACM CHI90 Conference on Human Factors in Computing Systems, (April 1-5), Seattle, Washington, 313-320, ACM: New York. Tang, J. C. and Minneman, S. L. (1991). VideoWhiteboard: Video Shadows to Support Remote Collaboration. Proceedings of ACM CHI91 Conference on Human Factors in Computing Systems, New Orleans, Louisiana, 315-322, ACM: New York. Thomas, J. C. and Carroll, J. M. (1979). The Psychological Study of Design. Design Studies 1 (1), 511. Timpka, T., Nyce, J. N., Sjberg, C. and Johansson, M. (1993). Action Design: From Modelling to Support of the Software Process. Proceedings of Software Engineering and Knowledge Engineering93, San Francisco, IEEE Press. Toulmin, S. (1958). The Uses of Argument. Cambridge: Cambridge University Press.
47
B UCKINGHAM SHUM & HAMMOND Trapp, A., Reader, W. and Hammond, N. (1992). Tools for Knowledge Mapping: A Framework for Understanding. Proceedings of the East-West Conference on Emerging Computer Technologies in Education, ICSTI: Moscow, 306-312 Twidale, M., Rodden, T. and Sommerville, I. (1993a). The Designers Notepad: Supporting and Understanding Cooperative Design. 3rd European Conference on Computer-Supported Cooperative Work, Milan, Italy (13-17 Sept.), Kluwer Academic Publishers. Twidale, M., Rodden, T. and Sommerville, I. (1993b). Developing a Tool to Support Collaborative Dialogues and Graphical Representation of Ideas. Collaborative Dialogue Technologies in Distance Learning: Proceedings of NATO ARW, Segovia, Spain (April, 1993) VanLehn, K. (1985). Theory Reform Caused by an Argumentation Tool. Xerox Palo Alto Research Center, Technical Report ISL-11
48

Argumentation Based Design Rationale

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Argumentation Based Design Rationale

Uploaded by

Copyright:

Available Formats

International Journal of Human-Computer Studies, 1994, 40 (4), 603-652

Argumentation-Based Design Rationale: What Use at What Cost?

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

B UCKINGHAM SHUM & HAMMOND

Consequent Question Question

ARGUMENTATION-BASED DESIGN R ATIONALE

QOC from a Smalltalk design session:

low processor calculation required

low data throughput requirement during play

prestore all data patterns

low memory requirement

User interface QOC:

C: consistent C: minimise average cursor distance to selection C: adapt to user

B UCKINGHAM SHUM & HAMMOND

is a subgoal of is an alternative for

influences presupposes supports

is an answering procedure for

are possible answers to

FIGURE 3a: The DRL notation

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

Option 1.2 limited no. of inst.

Option of inst. inst. limited no. 1.3

low processor calculation required

Option 1.2.1 Option 1.1

Option 1.2.2.1 Option 1.2

* low data transferplay rate req. during

* low processor calculation req.

ARGUMENTATION-BASED DESIGN R ATIONALE

low (high-level) software calculation req. during play

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

B UCKINGHAM SHUM & HAMMOND

selection of incorrect paths leading to backtracking; losing track of goals.

ARGUMENTATION-BASED DESIGN R ATIONALE

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

Rough, evolving QOC representations must often precede coherent analyses

Developing QOC structures is a non-linear process

ARGUMENTATION-BASED DESIGN R ATIONALE

B UCKINGHAM SHUM & HAMMOND

O: top Q: where should cursor be on pop-up menu? O: middle

coherent, reusable DR emerging