This article was downloaded by: [Universidad de Buenos Aires], [Alejandro Pujalte] On: 12 September 2012, At: 13:56

Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

European Journal of Science Education

Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/tsed19

The Analysis of Qualitative Data

Joan Bliss , Jon Ogborn & Franois Grize
a b a a b

Chelsea College, University of London, UK Universit de Neuchtel, Switzerland

Version of record first published: 09 Jul 2006.

To cite this article: Joan Bliss, Jon Ogborn & Franois Grize (1979): The Analysis of Qualitative Data, European Journal of Science Education, 1:4, 427-440 To link to this article: http://dx.doi.org/10.1080/0140528790010406

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-andconditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

EUR. J. SCI. EDUC., 1979, VOL. 1, NO. 4, 427-440

The Analysis of Qualitative Data

Joan Bliss and Jon Ogborn, Chelsea College, University of London, UK, in association with Franois Grize, Universit de Neuchtel, Switzerland

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

Foreword by Richard Kempa

The recent shift in general educational research from the traditional psychometric approach to a more qualitative approach involving techniques such as loosely structured interviews, the recording of linguistic interactions, etc., is beginning to attract the attention of the science-education researcher. The feature of these approaches is that they lead to 'soft' data the evaluation of which has in the past often been limited to the presentation of quotes and selected passages. In this paper, Bliss, Ogborn and Grize outline a method for the analysis of qualitative research data which is based on the principles of network analysis. The Editors hope that the article will be of much interest to science-education researchers. R.F.K. 1. Introduction

One of the outstanding problems in educational research (indeed in social research generally) is the difficulty of performing an adequate content analysis of qualitative data such as interview transcripts, free questionnaire responses, observational material, or documentary evidence. We have begun to develop a method for handling such analyses, and this paper reports the present stage of development, current exploratory uses and possible future directions. Applications already made include an analysis of over 100 unstructured interviews with physics undergraduates about their reactions to learning (Bliss and Ogborn 1977); an attempt to describe teacher's questions (Ogborn 1977), and a scheme for categorizing some examination questions in physics (unpublished). We mention these uses to illustrate our view that the method may be of rather wide application and not be limited to only a few types of material. The method, and our terminology, derive from systemic linguistics. Accordingly, in Section 2 we introduce terms and notation with (intentionally naive) linguistic examples. In Section 3 we offer a miniature 'model' set of data to suggest how the terms and notation can be adapted for analysing educational or other data, and sketch a tentative theoretical framework for the description of such data. In Section 4 we give an account of past and present work using this or similar forms of analysis for a variety of purposes, and outline the computing system being developed to handle the information generated. Finally, in Section 5 we briefly review current progress and problems.
0140-5284/79/0104 0427 $0200 Taylor & Francis Ltd

428

RESEARCH REPORTS

2.

Systemic networks in linguistics

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

This section indicates the origin of the proposed method of analysis in systemic linguistics, and introduces terms and notation which we have taken over or adapted. Systemic linguistics originates largely with Halliday (Berry 1975, 1977; Halliday 1973, 1975, 1978; Kress 1976), and has been used in sociological studies (Turner 1972) and machine understanding of language (Winograd 1972). It has also influenced discourse analysis (Sinclair and Coulthard 1975, Coulthard 1977, Brazil 1975). Systemic linguists are interested in the description and representation of meaning; of the semantic resources of language. It is for just this reason that their ideas may have value for those of us who wish to be able to say what an interviewee meant, what an examination question is for, or what a teacher intends by a question. Since de Saussure, linguists have looked for meaning in the internal contrasts offered by language (as opposed to the relation of words and things); systemic linguists formalize this idea in the fundamental notion of system. By a system is meant a finite set of choices in an environment which permits that range of choice. The available options are called terms and give each other meaning by being just those contrasting choices which exist in a context. Starting a letter provides a commonsense example. To begin, Dear John has (roughly) the meaning 'normal informality between acquaintances' because that option is one out of a fairly small set ranging perhaps from Darling to Sirs and including Dear Jones, Dear Mr Jones, and Dear Sir. Each means what it does by being the choice it is, by not being the choices it is not, and by belonging to an environment (starting a letter) in which those choices are available or required. The terms in a representation of such choices would obviously include 'formality', 'informality', 'intimacy' etc. Systemic linguists have developed this basic idea considerably, and have introduced a convenient and powerful formalism for representing it. The simplest case is that of exclusive choice: a linguistic example would be the option open to the clause between statement and question. This is represented by terms written against a bar, as shown in figure 1.

I DECLARATIVE (Clause) 1 INTERROGATIVE

Figure 1. Bar to represent exclusive choice Next we consider the notion of delicacy. Questions can be more finely divided into YES/NO questions {Did he do it?) and WH-questions {When I Why IHow I Where did he do it?). Equally, the choice between question and statement can itself be seen as a finer choice amongst informationpassing (INDICATIVE) clauses as contrasted with requesting or exhorting clauses (IMPERATIVE). Delicacy is represented by linking options in a left-to-right tree, as in figure 2.

THE ANALYSIS OF QUALITATIVE DATA DECLARATIVE r- INDICATIVE (Clause) L- IMPERATIVE INTERROGATIVE WH Question r YES/NO Question

429

Figure 2. The notion of Delicacy

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

Binary choices are shown here only for simplicity: a system can have any finite number of terms. The options above act on clauses. But language is organized into units of more than one size: in English, these units include words, groups (e.g. verbal or nominal groups), clauses, and clause-complexes. Such units form a rank scale. Different options operate at different ranks: plural/singular operates on words, tense choice on verbal groups, and declarative/interrogative choice on clauses. There are also clause-co-ordination options which build units at the rank of clause-complex {I found it difficult but/when I learnt it). We have found it useful to think in terms of rank in devising suitable structures for describing data. The next notion is that of simultaneity, or allowed free combinations of choices. For example, verbal groups in all kinds of indicative clauses can vary freely in tense {He is doing it; He will do it; Will he do it? How was he doing it? etc. etc.), but imperatives do not vary in tense in the same way (one cannot tell someone to do something in the past!). The example introduces the further notation shown in figure 3.
PAST r NON-FINITE e.g. to go Verbal Group L- FINITE INDICATIVE PRESENT FUTURE r- DECLARATIVE j - YES/NO Question I MAJOR Clause IMINOR IMPERATIVE < INTERROGATIVE - W H - Question

Figure 3. Simultaneity, conditional entry and recursion The simultaneous options of tense and of types of indicative are shown by writing these systems against a right-facing bracket {I. Such a bracket is analogous to stating differing independent dimensions. It is an extremely powerful notion, because it allows the linguist to say how a segment of language means several different kinds of thing at one and the same time. It is precisely this aspect of the description of qualitative data that it is most difficult to handle in a system of simple exclusive categories. The example of tense further illustrates the useful notion of recursive choice, indicated by the letter R written against the tense system (our notation, notHalliday's). That is, in English, besides doing or did there can be will have done, was being done, will have been doing, etc., in which tense choices
E.J.S.E.
2F

430

RESEARCH REPORTS

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

are repeated to build complex tenses. In analysing qualitative material one frequently finds a need to allow for such repeated selection (e.g. 'the problem was both difficult and long'). Finally, figure 3 illustrates the use of entry conditions. It shows two types: The left-facing bracket Z } shows that for the option indicative/imperative to exist, the clause must be major (e.g. he learnt the work, but not, 'till he knew it), and that the verbal group must be finite (go or went but not to go or going). Thus a network can represent necessary entry conditions to a system. The second type is the left-facing bar]-, which in figure 3 shows that the system of tense is available either to (some) non-finite verbal groups (to go, to have gone, to be going to go) or to verbal groups in indicative clauses. J Thus a network can represent alternative entry conditions. Both types are useful in descriptive analysis, the first particularly so. We have now presented by examples the main components of networks. The examples are purely illustrative; they would not satisfy a linguist. But what is a network and what does it do? How does it relate to actual talk or writing? A network is a structure of possibilities, showing their dependence and independence. It allows certain configurations of choices (e.g. F I N I T E VERB, MAJOR CLAUSE, IMPERATIVE) but not others (e.g. PAST IMPERATIVE). Each possible allowed configuration is termed a paradigm of that network. Clearly a quite compact network can represent a very considerable number of paradigms. However, the paradigms are not yet actual language; they say only that certain choices exist, not how those choices appear in speaking or writing. To link the network to what is said or written we need the further notion of realization rules; the rules which say how the language encodes the choices. Obvious examples include adding 5 to make words plural or suppressing the subject and transposing the object to make clauses passive (he did the experiment, the experiment was done). As explained in the next section, we have modified the notion of realization rule so as to connect networks showing structures of possible features of data with descriptions of particular items of data. We have now introduced the terms: system, delicacy, rank, simultaneity, recursion, entry conditions, paradigm, and realization rule. We have also illustrated the network notation using right- and left-facing bars and brackets, summarized in figure 4. All are of course introduced here not for
--B
A-C -B A- -C -D ABC-D

A-B-D

--D

C--

If A, then enter one of B,C,D.

* If A, then enter all the systems B,C,D.

Enter D only if all of A,B,C.

Enter D if one or more of A,B,C.

Figure 4. Network notation

% It is a fault in the network not to allow tense variation for minor clauses.

THE ANALYSIS OF QUALITATIVE DATA

431

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

their linguistic interest, but for their value in describing educational or other data. The whole line of thought can be summarized as follows. A network is a structured pattern of interdependent options, showing by its structure the patterning of related descriptive features, and by the combinations of features it permits the particular complex groupings of features it accounts for and labels with those features. Actual instances of different meanings each correspond to just one configuration of choices out of the possible configurations or paradigms. All that we ever hear, say, read, or write are realizations of one paradigm at a time, but each instance gets its meaning because it exists as one possibility amongst a finite set of other (linked) possibilities. Different aspects of meaning are to be caught by describing simultaneously existing structures of meaning. Why trouble with all this? The reason is that the problem of describing qualitative data is, as we see it, very largely the problem of handling a complex of descriptive features at very varied levels of generality; of seeing in what ways items of data are alike and are different. When one speaks, one means more than one thing at once, and those who give us educational data normally take the same liberty. 3. Adaptation to non-linguistic analysis

This section illustrates, through a simple 'model' example, how we have been led to adapt and modify the linguistic framework just described, for the analysis of educational research data. It is essential to stress that the analysis is not itself linguistic, even when it deals with language data (such as interview transcripts). The problem is to extract, codify, and represent nonlinguistic information: thoughts, feelings, ideas, events etc. Data may be of any qualitative kind, including drawings or pictures. Suppose, then, that we are trying to analyse the following miniature 'model' set of data, consisting of fragments of stories about learning.}: It is offered purely for illustration: the real value of networks lies in their ability to deal with much larger and more complex bodies of information. Model data A '...when I started reading Landau and Lifschitz' mechanicsI followed the bookit is a beautiful classic book on mechanicsand things came out so well I felt like kissing the book...' B ' . . . I had a very good essay given back to me; that made me feel very good, because I had spent a long time on i t . . . for the rest of that term I felt so good I could have tackled anything...' C ' . . . in practicals we were supposed to be learning soldering... I got stuck and the supervisor thought it was very funny that there I was making a fool of myself...' D ' . . . I feel bound to go through as many problem sheets as I can, but I just couldn't do it. I found that I couldn't keep up with it and I kept getting very depressed and wouldn't speak to anybody .. .'
X Extracted and modified from interviews with physics students (Bliss and Ogborn 1977).
2F2

432

RESEARCH REPORTS

Model analysis Any analysis must start from the point of the research. Suppose that it is to relate students' reactions to the circumstances in which they arose: then an account of the data would want to describe reactions, situations, and if possible circumstantial detail. For this reason, the network shown in figure 5 begins at the left with these as independent descriptive dimensions. Consider the four situations: reading, an essay, a practical class, and doing problems. If the analyst thought that reactions might differ as between private, individual work and things done publicly in class (and if the data supported the distinction) he might sketch the simple tree of more delicate options shown following S I T U A T I O N S in figure 5, so grouping three situations together and one apart. One would try to have, as the most delicate options, situations which were specific enough to relate directly to individual items of data (e.g., READING) but general enough to apply to more than one (e.g., probably not READING LANDAU). Decisions would have to be made as to how delicately to distinguish situations, for example further dividing practical work into (say) techniques (as here), set experiments, and projects. The decisions would depend on the variety offered by the data, and the plausibility of the distinction mattering. Consider now the reactions or feelings expressed. After many attempts (and with more evidence than is given here) one might describe the reactions shown as, respectively: A B C D Elated, and 'beauty-of-ideas'. Pleased, and 'able-to-cope'. Felt foolish. Depressed, and 'not-able-to-cope'.

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

As before, one would try to find descriptions narrow enough to be able to be matched against the data itself, but wide enough to work on several items. Also as before, one would try to organize these (and other) descriptions into a network of less delicate and perhaps simultaneous options. To do so would naturally involve guesses about what kinds of difference might prove important and reliable as accounts of the data. Possibly the result might be the part of the network following REACTIONS in figure 5. Then it would be necessary to check that each of the described reactions corresponded well enough to one network paradigm: 'elated' to positive satisfaction; 'beauty-of ideas' to positive involvement, 'pleased' to positive satisfaction, 'able-to-cope' and 'not-able-to-cope' to positive and negative coping, etc. Doing so would immediately raise problems: for example, the network equates 'elated' and 'pleased' as both positive satisfactionsis that good enough? One might be led to distinguish strong versus moderate reactions as a result. Similarly, is 'depressed' well represented as negative satisfaction, as the opposite of 'pleased'? Might feeling able or not able to cope be better as feelings about the self instead of feelings within the self; indeed, can that distinction be maintained? As analysis proceeded, such questions would have to be resolved, judging always by how well the descriptions assigned by the network fitted the interpretations one could decently give the data.

THE ANALYSIS OF QUALITATIVE DATA

433

I ESSAY i - PRIVATE SITUATIONS READING IPROBLEMS

L P U B L I C _r
Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

PRACTICAL
I etc.

POSITIVE

:
< REACTIONS '

NEGATIVE

E
CIRCUMSTANCES V
U

OF INVOLVEMENT WITHIN MYSELF , ABOUT MYSELF

r- CONTROL L SATISFACTION L COPING

iDOING
r,NDIV.DUAL

~ L NOT DOING iTRYING L SUCCEEDING

r

B E LAUGHED AT

INTERPERSONAL-n
Ietc.

Figure 5. Network to represent the 'model' data Finally, figure 5 shows a fragment of network describing circumstantial detail, dividing it into interpersonal interaction (as in C) and more individual private circumstances as in A, B and D, together with a device for representing not trying and not succeeding as well as trying and succeeding. The whole network is clearly naive; at the same time it does perhaps capture a good deal of what one can see in the data. Its point here is to illustrate the process of network construction and its problems. Coding It is clear that the network describes, not the data, but an interpretation of the data for a purpose. Its paradigms are what the analyst wants to extract. For this reason, we have found it useful to make one very significant modification of the linguistic framework: to introduce the notion of an artificial coding language, which functions as the language in which the analyst says what he sees. The idea is simply to invent realization rules for the descriptive network which generate, from the chosen options, a simple code which reads like a direct analysis of each item of data; which reads as a summary of what is

434

RESEARCH REPORTS

there. But each term in the code is to be understood, not as having its normal range of meaning, but as having a fixed and definite meaning assigned by the network. For example, some simple rules might give as codes: A ABOUT READING. I F E L T ELATED AND BEAUTY-OF-IDEAS. HAVING SUCCEEDED. B ABOUT ESSAY. I F E L T PLEASED AND ABLE-TO-COPE. HAVING T R I E D AND SUCCEEDED. C ABOUT PRACTICAL-WORK. I F E L T FOOLISH. HAVING BEEN-LAUGHED-AT. D ABOUT PROBLEMS. I F E L T DEPRESSED AND NOT-ABLE-TO-COPE. HAVING T R I E D AND N O T SUCCEEDED. Rules might include using ABOUT, I FELT, and HAVING to mark respectively situations, reactions and circumstances, and using N O T to indicate negative rather than positive options. Many items of code are just copies of the most delicate option; in such cases the network automatically implies their having the less delicate features it assigns them. One value of coding like this is that the codes, which carry the analysis, are easily read and compared with interpretations of the data. Here, for example, one might note that D does not encode the feeling of 'being bound to do as many as possible', which might lead to augmenting the network by, and adding to the code, features to do with compulsion or freedom. Further, using the code, one can look to see how naturally the form chosen for the network expresses an interpretation of the data; is this data well represented by a 'situation-reaction-circumstance' format? The codes, of course, retain implicitly all the distinctions written into the network whose options they realize. Thus it is possible to count and correlate features at any level of delicacynormally there will be a great many. It is this dual property of being at once a description language which is rather close to the data, and a formal language with fixed and definite terms and meanings, which makes such codes and their associated networks useful in handling large bodies of complex and subtle qualitative data. In a sense, the code is what the analyst might write if he were asked simply to summarize what he saw in a given item of data, but written in a special language, developed from the data itself in a systematic way, designed to say as much but no more than is wanted. The network ensures that the codes have just the implications and 'hidden' meanings required, and provides the means of extracting them again when necessary. Theory of the analysis In the linguistic case, it is language itself which is the data. A systemic linguist tries to describe how aspects of it can be accounted for as realizations of options in a network he proposes. The network is his theory of language. In our case, the data is further filtered through the interests, preconceptions,

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

THE ANALYSIS OF QUALITATIVE DATA Realization ^ _ rules

435
L G E

()

N E T W 0 R K

^ ^ |

FERCEPTJCTNSJ
{b)

NETWORK

I Realization I _ _ ^ _ - ; - | rules I

L E

I Description ' _ ^ _ | | I

I DATA | I i

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

Figure 6. Network, language and data (a) linguistics (6) our analysis and perceptions of the analyst. Those perceptions he tries to codify andfixin an artificial description language. Figure 6 shows the difference between the two cases. No system of analysis can avoid the fact that the relation between perceptions of data and the analysis (whether as categories or as here) is problematic. What the present system aims to do is to make such intuitive relations more explicit, more accessible to discussion, and less fluid in their application.
A B C etc.

Figure 7. Categories as a system It is worth noting that the scheme can be seen as a generalization of orthodox analysis by categories. In the simplest case of mutally exclusive categories A, B, C, etc., the network is as in figure 7 and the descriptions are just the category names, 'realized' by quoting. Categories in a hierarchy (e.g. Bloom et al. 1956), or with sub-categories, are just tree networks. The essential innovation taken from linguistics is to provide for descriptions of several simultaneous aspects in one code. 4. Applications and developments

Socio-semantic networks The use of networks to represent educational or sociological data is not new. Turner (1972) has used systemic networks to describe structural differences in the ways parents of different kinds of families control their children, as part of a study of socialization (Bernstein 1972). Another suggestive application is that of Mohan (1969) to the language and situation in a card game, with reference to the nature of rules and instructions. One of us (J.O.) has attempted to use systemic networks to categorize differences between teachers' questions, using data from recorded university

436

RESEARCH REPORTS

tutorial classes in physics (Ogborn 1977). The networks represent styles of question, types of content, types of role, and types of interconnection between questions. In similar vein is a tentative study of teacher-pupil interactions, using published classroom transactions (Monk 1977). In all these applications, there is nothing corresponding to the artificial coding language introduced in Section 3. The networks function as a classificatory device, each paradigm being a structured complex of classificatory features.
Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

Analysis of interview data As part of the Higher Education Learning Project (Physics) (see Black and Ogborn 1977) we, with a team of university teachers, interviewed 115 physics undergraduates in 10 universities (Bliss and Ogborn 1977). The students were asked to talk about times when they had felt especially 'good' and 'bad' about their learning. The interviews produced some 300 stories for analysis: a formidable bulk of complex material, and a situation only too familiar to all those who have conducted interviews on any scale. Further, the teachers involved showed a healthy practical interest in matters of detail and were not inclined to accept any too general or global set of categories to describe the material. They felt that it mattered if an incident involved an unexpected word of praise, for example, and we were inclined to agree that 'motivation' is best understood as related to the particular character of events rather than to any general 'level of interest' or of involvement. We alsoas so oftenfound that simple categories failed because every item needed to belong at once to overlapping categories along several dimensions. We turned to systemic networks as offering the possibility of accommodating complex interrelated classifications. In addition, partly because of the bulk of the data, we developed the notion of coding as previously described. A typical story, encoding some two to five pages of transcript, might look like this: STORY CONCERNS PARTICULAR INDIVIDUAL LEARNING T H A T IS W R I T I N G ESSAY WHEN (I F E L T PLEASURE AND PLEASED-WITH-MYSELF AND I-HAVE-DONE-IT) BECAUSE (I DID A L O T OF PREPARATION) SO (I UNDERSTOOD IDEAS) ALSO (TEACHER PRAISED ME) ALSO (TEACHER GAVE ME GOOD MARK) ALSO (I D I D W E L L WHICH IS UNUSUAL) ALSO WHEN (I F E L T IT-WAS-MY-WORK) BECAUSE (I WAS W O R K I N G INDEPENDENTLY) We gave the codes a rank structure consisting of a story made up of clauses linked by relating terms (WHEN, BECAUSE etc.). The first clause had to describe the situation, the next the reaction, and further clauses the reasons for the reaction. Dependency between clauses was indicated by indentation.

THE ANALYSIS OF QUALITATIVE DATA

437

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

In this way the networks allowed the code to express the content and structure of the stories, all to be further analysed in terms of frequencies. At the rank of clause, each had to have a subject or topic (in first place) and a comment (in TEACHER PRAISED ME, the topic is the teacher, and 'praised me' is the comment). A very large network stored all the types of comments we found we needed, grouped by the network under classifying features such as actions or appearances, cognitive or affective, personal or situational, interpersonal or individual, etc. etc. The network grouped items under clusters of such features, keeping apart things to do with the teacher, the student, or the situation as topic, for example. This meant two things. First, each story had a code close in form and expression to the story itself, making it easier to check for adequacy, while at the same time being assigned by the network a cluster of features at all levels of delicacy. Second, the existence for each story of this cluster of network features made it possible to count and correlate features at various levels of generality. We could ask, without having lost the particularity of individual stories, how many concerned (for example) interpersonal interactions with a teacher, or the student's feelings about himself and the amount of praise he got or his perception of how well he worked. In the event, limitations of time restricted the number of issues we could look at. Typical outcomes were the relatively high frequency in 'bad' stories of reactions of withdrawal (notably in lectures), and the relatively high intensity of involvement felt in individual work as well as in projects. We felt a need, however, for a better way of doing all the counting than by hand. A systemic network computing system Recently, F. Grize has begun work with us on developing a computing system to accept an arbitrary network specified by the user; to accept, check and 'understand' codes representing data, written as realizations of network options; and to answer questions about numbers of items with arbitrary configurations of network features. At present, the system is still under development. It allows only the crudest realizations (copying most delicate terms); conditional entry is developed but not yet implemented fully. Items can be listed, counted or cross-tabulated. The computing system is told the network for given data by a fairly obvious translation of network symbols into a symbolic language. Figure 8 gives some examples. The system is written in PASCAL and runs interactively at a terminal. Problem-solving protocols A group at Chelsea College Centre for Science Education is interested in students' problem-solving processes. This work is at an early stage, but the idea of analysis using networks has shown some promise. In one case, F. Mujib has developed a network to describe how the student's attention shifts from one aspect of the problem to another, how the type of approach varies in different parts of a problem, and the type of cognitive operations used. H. Elliot has a prototype network to describe the range and interconnection of knowledge potentially implicated in the solution of a particular

438

RESEARCH REPORTS

- A -

F A = BAR(BC). A-

A = BAR(B C). B = BAR(EF).

-c

L-c

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

B
_ A A=BRA(BC). A

Lr

-c

A = BRA(BAR(B C) BAR(D El)

Figure 8. Computer translations of network symbols chemistry problem. One aim of the group is to develop further the idea of using networks to represent knowledge, especially partial knowledge. Examination questions Related to the problem of describing knowledge is that of describing test items or examination questions. Existing category systems (Bloom et al. 1956) together with lists of syllabus topics are much used, but we hope that the systemic network representation will offer the chance of representing more detail without losing higher classificatory features, and without deciding once and for all the level of delicacy at which to analyse. The particular material being investigated is, at present, some hundreds of multiple choice items for the Nuffield Advanced Physics examination, and networks describing the items are being explored for their potential in recording and extracting information about this question bank. 5. Conclusions

We do not wish to claim too much for the ideas sketched in this progress report. In particular, the preliminary stage of many of the applications makes it hard to evaluate the real promise of the approach. At the same time, we feel that there is here an important problem, namely the handling in educational and other research of complex qualitative material, where the difficulty is to retain its detail and subtlety without losing control and, at the same time, to be able to look at it from several general points of view without deforming the individuality of particular pieces of data. Just because these problems exist, it is not easy to deal with the important but 'soft' data one often has, and such data often fails to carry the weight it might otherwise have had. Researchers are reduced to quotation, which if selective is in danger of bias, and if comprehensive leads to enormous and indigestible reports. New approaches to evaluation, which seek to mitigate the limitations of 'hard' test data, have all too often been dogged by such methodological impasses.

THE ANALYSIS OF QUALITATIVE DATA

439,

Many problems remain to be resolved or clarified. One is the problem of establishing criteria for 'good' network features and structures. Where the aspect studied is well understood, as for example the scientific knowledge implied in a question, the understood logic of the material is a great help. In handling the interview material, we found that it was the harder to establish meaningful features and structures the less well we understood the situations to be described; fairly hard for (say) students' feelings despite some insights from psychology, and very hard for the complex situations of laboratory and classroom where, as the network revealed, our ideas tended to be superficial. Thus the existence of some background theory, and the knowledge and understanding of the analyst, turn out to be very important. Another problem is that of establishing ways of testing the reliability of codes and their associated features. Here the method seems to have some advantages over simpler category systems, in that it is hard to get agreement when one category must contain much data, but easier when, as here, the code reflects rather closely detailed perceptions of the data. One analyst may not agree with another, but each knows more about where the disagreement lies. Further, since the network assigns to codes features at successively less and less delicacy, it is often possible to trace, when there is disagreement, the level of delicacy at which one can find agreement. That is, analysts may not agree about the exact descriptive term chosen, but they soon find that all the terms they favour have some higher level set of features in common. Lastly, one problem of classificatory reliability is simply that people forget what the categories mean, or allow their meaning to shift. Here the network, as a kind of semantic grammar, can help a good deal in maintaining the stability of meanings. A very real problem with the method is that one has to learn it. It looks complicated, perhaps absurdly so. We find that it is not too hard to begin, but that (in our limited experience) difficulties soon set in. The difficulties are sometimes the outcome of the power of the method, in that one tries to handle more complexity than one is ready for. It will clearly be necessary, if the method is judged to have any value, to develop exercises and problems which students could use in order to learn it. In conclusion, it must be stressed that the method in no way forces anything on the analyst. Quite the reverse: we feel that, whatever its problems and need of development, the central claim that might be made for it is that it both allows and forces the analyst to find out what he wants to say, and to make that explicit. It does not tell him how to analyse data. It is not a substitute for understanding and insight. All it does is to provide a framework for developing one's own analysis. Indeed, in using it, we have found to our cost that it exposes in a cruel light many of the inadequacies and superficialities in one's own thinking. That by itself may be enough of an advantage to be going on with. References
BERNSTEIN, BERRY,

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

B. (Ed) 1972, Class, Codes and Control, Vol. 2 (Routledge and Kegan Paul: London). M. 1975, 1977, Introduction to Systemic Linguistics Vol. 1: Structures and Systems, Vol. 2: Levels and Links (Batsford: London).

440 BLACK, BLISS,

RESEARCH REPORTS

P. J. and OGBORN, J. 1977, Inter-university collaboration in methods of teaching science. Studies in Higher Education, Vol. 2, No. 2, pp. 149159.

J. and OGBORN, J. 1977, Students' Reactions to Undergraduate Science (Heinemann Educational Books: London). BLOOM, B. et al. 1956, 1964, Taxonomy of Educational Objectives Vols. 1 and 2, (Longmans: London).
BRAZIL,

Downloaded by [Universidad de Buenos Aires], [Alejandro Pujalte] at 13:56 12 September 2012

D. 1975, Discourse Intonation. Discourse Analysis Monographs, 1 (University of Birmingham). COULTHARD, R. M. 1977, An Introduction to Discourse Analysis (Longmans: London). HALLIDAY, M. A. K. 1973, Explorations in the Functions of Language (Edward Arnold: London). HALLIDAY, M. A. K. 1975, Learning How to Mean (Edward Arnold: London). HALLIDAY, M. A. K. 1978, Language as Social Semiotic (Edward Arnold: London). KRESS, G. (Ed) 197'6, Halliday : System and Function in Language (Oxford University Press: Oxford). MOHAN, B. A. 1969, An investigation of the relationship between language and situational factors in a card game, with specific attention to the language of instructions. (University of London Ph.D. Thesis). MONK, M. 1977, The Verbal Behaviour of Teachers and the Self-Identity of Students in the Classroom Interaction. (M.Ed, thesis: Centre for Science Education, Chelsea College, University of London). J. (Ed) 1977, Small Group Teaching in Undergraduate Science (Heinemann Educational Books: London). SINCLAIR, J. McH. and COULTHARD, R. M. 1975, Towards an Analysis of Discourse (Oxford University Press: Oxford). TURNER, G. J. 1972, Social class and children's language of control at age 5 and age 7.
OGBORN, In BERNSTEIN, B. op WINOGRAD, cit.

T. 1972, Understanding Natural Language (Edinburgh University Press: Edinburgh). Summaries

English

A method for the analysis of qualitative data, based on the linguistic device of systemic networks used to represent a structure of possibilities, is reported. Some initial applications are described, and work in progress including a computing system is outlined.
Deutsch

Vorgestellt wird eine Methode fr die Analyse qualitativer Daten. Sie basiert auf der linguistischen Vorrichtung von Systemnetzen, die zur Darstellung einer Struktur von Mglichkeiten gebraucht werden. Einige erste Anwendungen werden beschrieben, und die fortlaufende Arbeit einschlielich eines Rechensystems wird skizziert.
Franais

Cet article expose une mthode pour l'analyse qualitative des donnes. Cette mthode se base sur un systme linguistique de rseaux, utiliss pour la description d'une structure de possibilits. De plus, les auteurs dcrivent les premires applications et prsentent le travail en cours, y compris un systme d'ordinateur.