How to start?

As you read an article you mark (highlight) the limitations and positive points. Please note
in the following sample the red font implies negative points and the blue font implies
positive points. For example, I highlighted the year 1967 in red because the reference is too
old.

This is a draft example of a research critique.

The Relationship Between Health and Stress

The relationship between the number of stressful life events and the incidence of illnesses.

Abstract

Our research group conducted a study of the relationship between health and stress. We
investigated the results of a previous study done by Holmes and Rahe (1967). Their work
related the total number of stressful life events experienced by individuals during one year
to fluctuations in their personal health. Based on the study, it has been predicted that more
illnesses will be reported among those who score higher on the Life Events Scale than
those who have experienced fewer stressful events. Our research group distributed copies
of the Student Life Events Scale (Reference) and a health inventory (developed by?) to 25
undergraduate students. The scores of the questionnaires were evaluated and analyzed. Our
results did not demonstrate a strong correlation, however, we feel this was an effect of
experimental error. The study did exhibit some degree of connection between health and
stress, and research on past experiments has indeed supported the relationship.

The relationship between health and stress is an area in the field of psychology that has
been the object of much attention. (Why?) It has been logically reasoned that as the amount
of stressful life events increases, an individual's health is adversely affected. Many studies
have been conducted in the past to prove that the relationship exists. Several support the
hypothesis, but there are also some studies that have been conducted to challenge it or have
simply failed to support it.
Holmes and Rahe (1967) conducted a study in which a large group of people were asked to
assign points to different life events according to how much adjustment each event
required. They then related the total stress points reported to the individual's health.
According to the results, it was quite apparent that the greater the number of stress points,
the greater the incidence of illness. (Explain more about the results, e.g. the sample, the
mean and SD of stress and illness, the kinds of illnesses and stressors)

The study by Holmes and Rahe (Reference) ignited an interest in the health and stress
topic, and recently, quite a few studies have been done. In one related study by Cohen,
Tyrrell, and Smith (1993), volunteers were asked to report all stressful life events that
occurred during the previous year. Next, they were administered nose drops containing the
common cold virus. (When introduce a new study try to explain the difference e.g. this one
is an experimental while the Holmes and Rahe was survey). Those participants who were
experiencing a great deal of stress were more likely to contract a cold. Rawson, Bloomer,
and Kendall (1994) distributed four surveys and questionnaires to 184 undergraduate
students: the State-Trait Anxiety Inventory (Reference), the Life Expectancy Survey
(Reference), the North American Depression Inventory (Reference), and a health
questionnaire (developed by?). The results of this supported the hypothesis that stress,
anxiety, depression, and physical illness are interrelated. Weidner, Kohlman, Dolzauer, and
Burns (1996) examined the health behaviors of college students during times of high stress,
for example, midterm week. The researchers observed that the health behaviors of the
students deteriorated. The students exercised less, ate fewer nutritious food, and made less
of an effort to avoid substances such as drugs and alcohol. (what was their conclusion,
what is your conclusion, can we conclude that stress and health are indirectly related e.g.
less exercise?)
There is an interdisciplinary science known as psychoneuroimmunology devoted
specifically to the affects of psychological stress on the immune system. In this field it is a
common assumption that stress suppresses the immune system thus causing vulnerability
to stress. (this is another study that shows stress and health are indirectly related). In a
study by Evans, Clow, and Hucklebridge (1997) a contradiction to this assumption was
observed. Immediately after the introduction of a stressor, participants actually showed
signs of enhanced immunity. Overall, however, the immune systems of the participants
exhibited decreased immunity. (How do they/you justify this?) In general, experiments
done in the past have supported a few basic premises. Environmental stressors are
linked to stress. Health can be affected by stressful life events. Severity of chronic
illness is linked to stress. Also stress has a negative effect on the immune system.
Our research group conducted a study similar to that of Holmes and Rahe(Reference). Our
purpose was to find a relationship between the amount of stressful life events experienced
by college students and their general health. Based on previous work done in this area, we
reasoned that as the number of stressors in one's life increased, the health of that individual
would decrease.

Method

Sample
We conducted a correlational study in which we attempted to determine whether, and to
what extent, different variables (stress and health) are related to each other. Careful
observations of each variable were made (what does it mean?), and then a statistical
analysis was performed to determine the extent of correlation between variables. The
design included a single group. (This part should go to design or procedure & should
explain how (in detail) they evaluated the two variables e.g. did they do it during final or
on a holiday? ) Twenty-five undergraduate college students participated in the study. These
participants were average male and female college students. They were selected since
college students experience a significant amount of stress caused by academics, social life,
and a variety of other stressors (Proof?). They were randomly recruited by members of the
research group. The participants were representative of the population (only 25 students?
from how many in population?) since they were a random sample of typical college
students who experience stress.
The ethical treatment and anonymity of each participant was ensured. Each participant was
presented with an informed consent contract. In this contract, the participant was briefly
informed about what will take place in the experiment. They were notified that their
participation was to be strictly voluntary, and that they may have withdrawn from the
experiment at any time without prejudice or penalty. The contract guaranteed that the
participants' responses would remain anonymous. Finally, the contract was signed by the
participant and returned to the experimenter.

The Questionnaires
The apparatus consisted of two measures in this experiment. The first was the Student Life
Events Scale (Reference, Reliability, validity, Mean, SD,...more information. & should
explain the questions or mention that a copy of the questionnaire is attached in appendix)
The students were asked to indicate the events they had experienced in the last six months
or that they were likely to experience in the next six months. The events listed include
many stressors such as death of a close family member, divorce between parents, and
change of major. The second measure was the Health Inventory (if it is a well known test
should give Reference, Reliability, validity, Mean, SD,...more information). The students
were asked to indicate age and gender. They were then asked to rate the extent to which
they experienced various health problems during the past year, from no problem (0) to
significant problem (3) (is this your Health Inventory? If yes this is a very weak measure
of health problem if not why did you ask this question? & how did you used it in your
analysis?)

The procedure was executed by distributing the questionnaires. Each member of the
research group gave out the questionnaires to five different students. After the consent form
was signed and the measures completed, they were analyzed. The results were thoroughly
examined and explained. (this should go to design or procedure)

Results
The data collected showed that the mean, or average, score on the health Inventory was
14.32. The median, which is a measure of central tendency, was 13.00. The standard
deviation from the mean score was 9.64. The minimum score was 1.00 and the maximum
score was 42.00. The mean score on the Student Life Events Scale was 420.1 and the
median was 405.0. The standard deviation from the mean was 212.4. The minimum score
was 76.0 and the maximum score was 1144.0. The correlation coefficient (Pearson,
Spearman, phi,...?) obtained from our data (between what and what?) was 0.289. This
positive variable indicates that as stress increases, health increases (Wrong Conclusion,
illness increases not health) . However, the farther the distance of the coefficient from 0.00,
the greater the relationship between the two variables. Since 1.00 represents the strongest
correlation, our results did not strongly support a relationship between health and stress
(see attached graph). We predicted a strong relationship between these two variables, but
only a slight one was observed .(This goes to conclusions)

Conclusions & Discussion

Our experiment certainly did not indicate clear-cut results. Yet, it can support some of the
previous research demonstrating a relationship between health and stress since there was
positive correlation. It did not altogether fail to show a relationship, and does not coincide
with past research designed to challenge the correlation. (should compare the correlation
coefficients)
We think that the main reason for this finding is that Our study was not free of limitations.

(First Justification) Reliability is the extent to which any measuring device, such as a
psychological test, yields the same result each time it is applied to the same quantity. We
did not test for this consistency since each questionnaire was administered to each
participant only once. Validity is the extent to which a test actually measures what it is
intended to measure. Our experiment demonstrated a degree of content validity because the
items on each questionnaire were related in a straightforward manner to health or stress.
(How did you measure the content validity). Experiments similar to ours have been done
which demonstrate content validity.(Reference)
Another explanation for the results of our experiment could be that there was a great deal
of experimental error in the measurement. One type of error may be found in the test itself.
(Second Justification) Certain items on the questionnaire could be considered too broad,
such as dental problems, thus affecting the participant's response to that particular item.
(Third Justification) Other age groups could have been observed. (Fourth Justification)
Also, the participants could have been chosen more carefully to ensure a more equal
distribution of males and females. (did not explain the number of male and female in the
sample part) Another form of error is caused by the experimenter. (Fifth Justification) The
attitude of the experimenter towards the experiment may have affected the participant's
responses. Another (In other words) possibility is that the experimenter may have tried to
influence the participants to respond in a manner that is consistent with the predictions of
the experiment (HOW?). Lastly (therefore), the participants may have contributed as a
source of error. (Sixth Justification) The participants may not have taken the experiment
seriously and responded randomly to the items (Why?). They may have exhibited
dishonesty in their responses due to the personal nature of some of the items (which items
how and when did you realize this?).
The study of health and stress can be applied to everyday life. We encounter stress on a
daily basis, and unless we live a charmed life, we often suffer from some form of illness. In
learning more about the relationship between health and stress we can discover ways to
improve our general well-being, and live healthier, less stressful lives. (goes to the
beginning- importance of the problem) Research in this area has indicated the possibility
that stress may be combated, thus improving health conditions. (References & Explain
more) A scientific basis for this is necessary, however. It is quite likely that future research
it this area will continue because of the great benefits it may contribute to people in today's
stressful life.(Your Suggestions)

References
Cohen, S., Tyrrell, D. A., & Smith, A. P. (1993). Negative life events, perceived stress,
negative affect, and susceptibility to the common cold. Journal of Personality and Social
Psychology, 64, 131-140.

Evans, P., Clow, A., & Hucklebridge, F. (1997, July). Stress and the immune system. The
Psychologist, 303-307.

Holmes, T. H., & Rahe, R. H. (1967). The social readjustment rating scale. Journal of
Psychosomatic Research, 11, 213-218.

Rawson, H. E., Bloomer, K., & Kendall, A. (1994). Stress, anxiety, depression, and
physical illness in college students. Journal of Genetic Psychology, 155, 321-330.

Weidner, G., Kohlmann, C. W., Dotzauer, E., & Burns, L. R. (1996). The effects of
academic stress on health behaviors in young adults. Anxiety, Stress & Coping: An
International Journal, 9, 123-133.
 CRITIQUE

Hicks, R. B. & Laue, H. (1989) A Computer-Assisted Approach to Learning. Physics

Concepts, 57(9) , 807-811.

My summary of the article

Introduction

First-year physics is a difficult subject for many students. Large classes make for restricted
access to tutorial help from professors and graduate assistants. To address this problem, the
department of physics with assistance from faculty of science, has developed a computer
assisted learning resource called CALIPH ( Computer Assisted Learning In Physics ). The
resource consists of more than 50 tutorial lessons in first year physics, including
mechanics, fluid mechanics, kinetic theory, and thermodynamics.

Much of the available instructional software provides drill in numerical problem solving or
acquaints students with physical situations by means of simulations. However, there is a
lack of a comprehensive set of tutorial lessons to attack the most common cognitive
problems alluded to previously in a systematic pedagogical way. It is extremely important
that students be alerted to all relevant details and assumptions when presenting new
concepts and ideas. In the authors and their colleagues' experience, errors resulting from
incorrect preconceptions and a reluctance or inability to apply appropriate definitions and
physical laws are made by many students even though lectures cover the relevant material
repeatedly

Therefore individualized tutorial instruction is required for (a) the effective teaching of
details such as those which tend to get lost in a large lecture theatre; (b) students need to
act upon the information given to them at their own pace; and(c) students need individual
feedback to their thinking.

Assumption

Students in first year physics usually do not perform well. Poor performance may be due to
a variety of common misconceptions and difficulties. Apparent familiarity with the objects
under discussion may actually be one of the factors contributing to the malaise because one
is tempted to think about familiar objects in familiar way. Often students do not seem to
realize that they are being asked to make a significant change in their way of looking at the
material world around them. Therefore, the implementation of new instructional strategies
focusing on restructuring of the preexisting knowledge to produce a conceptual change is
an important problem in physics instruction.

Subjects
In total, 96 student volunteers from the current first year physics courses in the fall and
winter of 1987-88 participated in the study. No computer experience was stipulated.
Unfortunately, many students failed to follow through with the second tutorial session due
to workload or other pressures and were replaced by new volunteers where possible. As a
result, 46 students attended the first session only, 17 attended the second only and 33
attended both sessions.

Instrument

In this research only 5 of the 50 tutorial modules were used. (Two in kinematics and three
in dynamics ). The first module dealt with calculation of average speed for a simple one-
dimensional motion. The purpose of this tutorial is to make students aware that care must
be taken to distinguish between a simple arithmetic average and a time-weighted average,
as required by the definition of average speed.

The second kinematics module attacked the "speed comparison

task" in which a uniformly accelerating object (a bus) starting from rest is first passed by,
and then passes an object (a bicycle) moving with constant speed. Students are shown the
screen and are asked whether the two objects had the same speed (a) only once (b) twice or
(c) never.

The final three tutorials dealt with the frictionless block- and-pulley systems. These three
modules examine the relations among the weights and tensions in each system,
demonstrate how to draw correct "free-body" force diagram for each block in the system
and help the student to learn a systematic method of derivation to find expressions for the
tensions and accelerations.

One specification of the tutorial modules is that multiple choice questions are encountered
frequently, requiring the student to consider material in an active way before proceeding;
where possible, questions address typical student misconceptions. Qualitative questions are
presented first before proceeding to quantitative questions. Also some practice with
numerical problems is provided, but is secondary to the presentation of concepts.

Students are occasionally asked to perform simple derivations or moderately lengthy

calculations using pencil and paper; the computer provides the correct answer and method
as feedback.

Concepts and definitions are emphasized, graphics and animation are used liberally.
Tutorials are entirely mouse driven; no keyboard entry is required and considerable help is
available in the form of menu items and optional explanation sequences.

The other instrument used in this research questionnaire to measure the attitude of students
towards the tutorials which is not well explained in the article.

Hypothesis

Although the research hypothesis was not clearly stated but it could be inferred from the
introduction that the goal was to show that "it is possible to construct relatively simple,
attractive, and interactive computer program that facilitate the learning of fundamental
concepts and definitions in the absence of a human tutor while DISPELLING erroneous
preconceptions and other types of misunderstanding". It was also mentioned that
considerable emphasis would be placed on qualitative understanding and derivations.

Procedure

Each student agreed to attend two 2-hours sessions on Saturday mornings. The first session
involved the two kinematics modules only, while the second (3-4 weeks later) dealt with
the block-and- pulley tutorials. In all cases students sat at individual microcomputers with
help from the one or two instructors present being kept to a bare minimum.

To assess tutorial effectiveness, short written tests were provided at the beginning and at
the end of each session. Most test questions were aimed directly at the content of the
tutorials for the day. Test questions at the beginning and at the end of each session were
very similar, although modified in some case by changing input data. The initial test of the
second session also included a question on average speed in order to test retention of
material learned in the first session.

A brief questionnaire filled out at the end of each session was used to get the feedback
from the participants. Student anonymity on written work was maintained throughout.

Analysis

By comparing test scores on similar written questions immediately before and after the
tutorial sessions, it was possible to determine whether an individual student had improved
his conceptual understanding of material by exposure to the computer tutorial. Most
question asked were simple enough so that a binary scoring system ( 0= unsatisfactory ,
1=satisfactory ) was sufficient to characterize student responses. Other tasks, such as
derivations, required in addition an intermediate score ( 1/2 =substantially correct but
incomplete ). A table summarizes results for selected tasks tested in the study. Only
frequencies and percentages are computed and compared.

Results

It is apparent from the first column of percentages in the Table 1 that a sizable fraction of
the students in this study brought common misconceptions with them to the study even
though the relevant material had been covered in lectures. It is also evident that the
tutorials achieved significant success in attacking student deficiencies in a number of tasks
but were less successful with others. Specifically most deficient students (75% or more)
clearly learned the definition of average speed and its application and became familiar with
aspects of block-and-pulley systems, including derivations of acceleration and tension in
the two simplest systems discussed.

The fallacy that the tension in a string supporting an accelerating weight is equal to the
weight was clearly dispelled, and students appeared able to generalize this concept to
Atwood's machine. However they were less successful in performing the derivation for
Atwood's machine even though they had learned derivation for similar systems.

It was noted that the success rate for the speed comparison task was relatively low (37%)
even though an entire tutorial module was devoted to aspects of this task.
To test the persistence of knowledge acquired in the first session, students returning 3-4
weeks later for the second session were asked to define average speed. Of 20 students who
had been able to give a satisfactory definition at the outset of the study without any
exposure to the computer tutorials, 3 failed to do so at the second session. Among the
returning students were 8 who had learned the correct definition in the first session; 4 of
these could not give a satisfactory definition at the second session.

The results of the questionnaire showed that students experienced little difficulty in
operating the tutorials even where experience with computers was totally lacking.
Responses of the majority of students were supportive. Students liked the use of graphics
and animations. They also commented that the explanations found in tutorials were clearer
and more direct than those in the textbook.

Most students indicated that they enjoyed being able to work at their own pace. Negative
comment was rare. Several students asked for treatment of more complex situations,
stating that the material covered by tutorials was "too easy".

Conclusions

The study shows that tutorial software constructed along fairly simple lines can be
effective in improving student performance in the virtual absence of the human tutors.
Although many the results of the study could have been predicted by experienced teachers,
pilot studies of this type are essential to test strategies prior to massive development of
materials. Although no effort was made to control the selection of students entering the
study, favorable student responses and the success rate achieved more than 80% for a
number of important fundamental tasks have convinced the authors that the development
of further tutorials is desirable as perhaps the only way in which one can actively engage
many students in the thought process required to master basic physics concepts.

Critiques
Importance of the problem investigated in the article.

An important problem in physics instruction is "how to help students achieve a deep

conceptual understanding of the subjects and how to help them develop powerful problem
solving skills". The importance of the problem can be easily realized by comparing the
number of studies in physics instruction with other sciences such as chemistry, biology or
social sciences. So I think the goal of this research is clearly stated and its importance is
well justified.

Although the topic is not a new ground but it is important. It should be mentioned here that
most of the research studies which are reported in different physics instruction journals, are
not really experimental. However this one is at least a quasi-experimental one.

The scope of the problem is reasonable but the conclusions are over generalized. Actually,
the theoretical bases for the effectiveness of CAI in physics is not conclusive enough to
give the researcher a good theoretical work frame. Therefore, this study and most of
studies in this field are more application oriented rather than theoretical based.

Review Of Literature

Given the fact that the CALIPH is made by the authors of this article and that this study is
the first study on its effectiveness, one can not expect to find a similar or even a related
study on CALIPH. Furthermore, generally the effectiveness of CAI in physics is a new
field in science education.

However, there are enough related literature which could have been mentioned in the
present study. For example, there are several classical and meta-analytical studies
(Kulik,1980, Mc Neil, 1991) which are worth mentioning to justify the importance of the
problem and to give the reader some information about the controversies in the field. Only
two not very related 11 studies ( Trombridge & Reife, 1982 ) were mentioned, which is not
sufficient and also they were not well explained in this article.

Hypothesis.

The hypothesis "the CALIPH can dispel common student misconceptions in the first year
physics" is not clearly stated. One must search through the whole article to find the
indirectly stated hypothesis. The dependent and independent variables are also stated
implicitly but not appropriately. No operational definition and even a simple definition of
variables were stated. For example, several words (e.g. significant, effective, sizable,
majority, conceptual understanding, several, most ... ) are not defined. Therefore, one can
not conclude, for example, what proportion of students are called majority or what the
conceptual understanding means.

The underlying assumption was that "physics students come into class with some
preexisting misconceptions". This assumption had been made on the authors' and their
colleagues' experience. It is not based on research or theoretical models. It should be
mentioned here that the content of tutorials is based on this assumption and so it was
expected that the authors would have tried their assumption and to clarify what these
misconceptions were.

No pilot study was done before the construction of CALIPH and before starting the present
study. However at the end of the study the authors realized that it had been necessary and
that they should had done it.

Sample

The subjects of this study were 96 volunteer students from first-year physics courses. It is
not stated that the subjects were just physics majors or other students who take physics
courses were also included ( for example, engineering or other science students). Also
demographic characteristics of the subjects (for example, gender, age, SES) was not
reported in the article.

Furthermore, a volunteer group is not a representative sample because they may be higher
motivated or higher achiever students. This problem specially pertains to this study
because previous researches have shown that motivation has a crucial role in effectiveness
of computer assisted instruction. The result of this study also supported such a possibility
because it is reported that students enjoyed the tutorials and stated that they are "TOO
EASY." This shows that mainly top students participated in this research.

Another major problem with the sample was that many students failed to follow with the
second tutorial session and they were replaced by new volunteers. It is obvious that when
we add a new group to an experiment, any conclusion should be considered with caution.
Although it is claimed that 96 students participated in the study, only 33 attended both
sessions. It would have been better to analyze the mixed and the original subjects
separately.

Procedure & Instruments

Three instruments were used in this study. One was the short written test provided at the
beginning and at the end of each session. These short tests are briefly described but their
reliability and validity is not mentioned at all.

The second instrument was the questionnaire to measure the attitude of the students
towards the tutorials and the research. This questionnaire and its content and its validity
and reliability was not explained. Only the results of some of its questions were mentioned

The third instrument which actually was the independent variable (the tutorial modules )
was well explained in the article. It is not clear which kind of test was used (pretest and
posttest) to measure the effectiveness of the tutorials, but it seems that they were not
standard or even validated tests.

A major problem with this report is that it is not clear exactly how many subjects were
studied in each session and how the new group was entered into the experiment. Actually
the numbers given in the page 809 and the numbers in the given table are different.
Generally the goal, the procedure and the results of the second session is not well
explained and well justified.

Another problem is that the content of the tutorials was a repetition of the materials that
had been covered in the lectures. In fact any gain after working on the tutorials can not be
merely the effect of the independent variable but also can be the result of repeating the
lectures in a new form. Furthermore, this repetition may have caused a considerable
reduction in the students' motivation and may have had a grate impact on the results of the
study. This methodological problem could have been removed if two different groups (one
with no experience with the subject matter and one with such an experience) were studied.

Analysis

The major part of the results of the research is presented in a table, a copy of which is
enclosed with this paper. As it could be seen, the effectiveness of different tutorials is
measured by computing the percentage of students that were classified as unsatisfactory
before working on tutorials with the percentage of those that were improved by working on
tutorials.

For example, in the first row ( definition of average speed ) 48% of students did not know
the definition before working on tutorials. But 81% of them were improved by using
tutorials. No statistical test has been used to measure the extent of the effectiveness of the
tutorials. Therefore the conclusion that the tutorials achieved significant success in
attacking student deficiencies is statistically meaningless.

The important analytical problem with this research is that the researchers used a binary
scoring system to measure students achievement before and after the treatment. It was
concluded that about 80% of students who were classified unsatisfactory in pretest were
classified as satisfactory in posttest. Why was a binary or a nominal scale used instead of
an interval scale? We know that the statistical analyses such as t-test or ANOVA are much
more powerful than non-parametric analyses. It is possible that if they had used a
parametric analysis probably the difference between the pretest and posttest means would
not have been not significant.

The table used to present the results also had some problems. First it is not clear from the
table which result belongs to the first session and which is belong to the second session.
Secondly as mentioned earlier the number of students in the first session is reported (75) in
the table, while it is reported differently (79= 46+33) in section three of the article. It is not
clear whether the different

numbers is the result of missing values or other factors.

Furthermore, given that the research had been done in two sessions it would have been
better to use another table to show the rate of retention of learning during the time interval
between two sessions. If an interval scale was used in both pretest and posttest the rate of
retention could have been measured by a t-test.

Results and Conclusions

The authors concluded that the tutorials had been effective in improving students'
performance in the virtual absence of human tutors. So they believe that their hypothesis is
supported by the reported results.

It was hypothesized that the tutorials can dispel some preexisting misconceptions of the
physics students. These misconceptions are not clearly explained and systematically
measured. After reading the results of the article one can not find whether these
misconceptions were observed or measured. Furthermore, there is no evidence to show that
after working with tutorials these misconceptions were dispelled. It should be noticed that
the improvement of students from pretest to posttest does not necessarily imply the
existence and the dispelling of such misconceptions. A wrong answer to a question may
have several reasons, not necessarily one, which the authors predicted.

Furthermore, as mentioned earlier, several methodological and analytical problems with

this research imply that such conclusions are not well justified. The absence of a control
group, the invalid measurement of achievement (pretest and posttest), the inappropriate
scale of measurement and statistical method, the non-representativeness of the sample, and
lack of motivation of subjects all in all does not permit one to conclude that the hypothesis
is supported.
Therefore this research not only did not show the existence and dispelling of specific
misconception, but also it did not show any statistically significant and meaningful
improvement in the students' achievement.

An important limitation of the results of this research is due to lack of attention to the
demographical characteristics of the subjects. For example if the sex, age, major, SES, and
other characteristics of the subjects were asked in the questionnaire it would have been
possible to get much more valuable information about the differential effectiveness of the
tutorials. Some previous research studies on other software programs have reported
different effects for different groups.

One of the advantages of this article is that whenever there was a shortcoming or a
limitation in the procedure or results it was frankly stated. For example, when the results
showed that the confusion of average and instantaneous quantities was evident in many
responses, the authors stated that the tutorials did not specifically address the concept of
instantaneous speed. Or when a negative effect of the tutorial was noticed for 3 of the
subjects it was honestly mentioned in the results. Also in the conclusion part it is stated that
"Although many of the results of the study could have been predicted by experienced
teachers, pilot studies of these type are essential to test strategies prior massive
development of material "development of materials"

Some good suggestions are offered as a conclusion of the research. For example, when a
low success rate with the concept of equal tension on both sides of a massless, frictionless
pulley was noticed, it was suggested that this notion must be placed on an equal footing
with other topics in the tutorial rather than being relegated to a "HELP" item only.

Generally it seems that the authors have paid a particular attention to show the
effectiveness of the "CALIPH" and have not noticed the limitations and weaknesses of the
research method. Even the simplest and the most fundamental outline of research method
was not noticed carefully in this research. For example, the general format of the research
report (introduction, procedure, results...) was not in accordance with the standard format.

Furthermore, something, which should be mentioned in conclusion, was in fact mentioned

in introduction and something, which should be mentioned in the introduction, was
mentioned in the conclusion.

Some personal impressions and opinions can be seen in different parts of the article, which
are not data based or research based. For example, in the conclusion part it is stated that
"the use of attractive graphics and animation sequences engages students' interest and
makes dynamic physical situations more intuitively accessible."

All in all this study has several methodological and report problems, however, in
comparison with other research articles in the field (applications of computer in physics
instruction ) is better designed and more conclusive. It should be noticed that in this
research, only 5 of more than 50 tutorials were considered and these five tutorials were not
randomly selected. Actually they were the simplest ones.

Although the is paper is criticizing the Hicks and Laue's (1989) article it should not be
concluded that CALIPH is not effective. All I have to say is that their article has
methodological problems and therefore, does not reflect the effectiveness of CALIPH.
I have seen and worked on many tutorials of the CALIPH. I think that it is one of the best
tutorials in first-year physics. I have talked with its authors about the popularity of
CALIPH in Canada. I know how well they have done on this 5-year project. But I think
this report is not a good report on a very good project. A much more comprehensive and
well-designed research is required to evaluate its effectiveness.