Professional Documents
Culture Documents
le 6
Assess
ment of Student Learning
Module 6
ITEM ANALYIS
Objectives
Introduction
The preceding two lessons dealt with the different methods of assessment that
can be used to assess varying learning targets. After writing the items, a test
cannot be considered good unless it is tried out because the ultimate judge of the
test is the user. More often than not, teachers prepare, administer, and score a test,
return the papers to their students, possibly discuss the test, and then either file or
discard the test.
teachers is that they do not check on the effectiveness of the their test because of
the following reasons: a) teachers feel that test analysis is too time consuming, b)
they are not aware of the methods of analyzing tests, and 3) they do not always
understand the importance of accurate evaluate evaluation.
Item analysis
Page 1 of 13
Modu
le 6
Assess
ment of Student Learning
Remember that tests are used for instructional purposes and other important
decisions, it is therefore important to conduct item analysis. Item analysis is the
process of evaluating the quality of the items and of the tests as a whole. Item
analysis investigates the performance of items considered individually in relation
to the remaining items on the test (Thompson & Leviton, 1985). It is the process
of examining the pupils responses to each test item.
The key aim of the item analysis is to improve the whole test and
eventually increase the reliability and reliability of the test. The tools to test the
validity and reliability of the test include item difficulty, item discrimination, and
effectiveness of distracters. There are many methods that can be used for item
analysis. One method is the U L Index Method (Stocklein, 1957). The steps
are:
1.
Score the papers and rank them from highest to lowest according to
total score.
2. Separate the top 27% and the bottom 27% of the papers.
3. Tally the responses made to each test item by each individual in the
upper 27% group.
4. Tally the responses made to each test item by each individual in the
lower 27% group.
Page 2 of 13
Modu
le 6
Assess
ment of Student Learning
5. Compute the percentage of the upper group that got the item right and
call it U.
6. Compute the percentage of the lower group that got the item right and
call it L.
7. Average U and L percentage and the result is the difficulty index of
the item.
8. Subtract the L from the U percentage and the result is the
discrimination index of the item
Modu
le 6
Assess
ment of Student Learning
A good item discriminates between those who do well on the test and those who
do poorly (Susan Matlock-hetzel, 1997).
denoted by D refers to the degree of the ability of an item to distinguish those who know
from those who do not know. It can be measured by comparing the number of people
with high test scores who answered that item correctly with the number of people with
low scores who answered the same item correctly. The top and bottom 27% are used for
analysis because 27% has shown that this value will maximize differences in the normal
distribution while providing enough cases for analysis (Wiersma and Jurs (1990). Other
methods use 30% while others use 50% (no middle group).
The higher discrimination index, the better the item because such a value
indicates that the item discriminates in favor of the upper group, which should get more
items correct. If more students in the lower group get an item correct than in the upper
group, the item will have a negative D value and is probably flawed. Similarly, if there
are equal number of students from the upper group and lower group, the item cannot
discriminate because D value is 0.
Computing for the difficulty index and discrimination index of an item is a lot
easier than interpreting it. Remember that the purpose of the analysis is to determine
which are good items; that is there are items to be retained or accepted, rejected or
discarded, and revised. The table below will serve as guide in interpreting the results of
the item analysis.
Page 4 of 13
Modu
le 6
Assess
ment of Student Learning
Table for interpreting difficulty index (p)
Range of Difficulty Index (p)
0 - .20
0.21 0.40
0.41 0.60
0.61 0.80
0.81 1.00
Interpretation
Very difficult
Difficult
Moderately difficult
Easy
Very easy
Interpretation
Questionable
Not discriminating
Moderately discriminating
Discriminating
Very discriminating
In interpreting the results, we have to consider not only whether how easy or how
difficult the item is, but also its ability to discriminate students who know and those who
do not know the answer. In other words, both the p values and D values are taken into
consideration. The decision rule is to retain or accept the items that are not so easy or too
difficult, and at the same time can discriminate bright from poor students.
After interpreting the difficulty and discrimination indices, the table below will
help us what to do with the test item
Difficulty Index
Difficult
Discrimination Index
Not discriminating
Moderately discriminating
Discriminating/Very
Suggested Action
Discard/Reject
May need revision
Accept/Retain
discriminating
Page 5 of 13
Modu
le 6
Assess
ment of Student Learning
Moderately difficult
Easy
Not discriminating
Moderately discriminating
Discriminating/Very
Needs revision
May need revision
Accept/Retain
discriminating
Not discriminating
Moderately discriminating
Discriminating/Very
Discard/Reject
Needs Revision
Needs Revision
discriminating
Note that whether an item is easy, moderately difficult or difficult, there are
always three categories of discrimination indices that can be obtained, that also leads to
three different actions. For example, an easy item could either be not discriminating,
moderately discriminating or discriminating, and then it could be rejected, revised or
retained.
However, care and caution musts be followed in using the table in interpreting the
results of an item analysis. Judgment of the test constructor sis very important. For
example, what will be done with an item that is easy and not discriminating. Using the
table, we should reject the item. But there will be an instance when that kind of item can
be revised. When? When that particular item is the only item left to test a very important
concept. So, we have no choice but to revise or improve it. On the other hand, what will
be done with an item that is moderately difficult and discriminating? Normally that item
should be retained because it has good indices. But there will be instance when that kind
of item may be discarded or rejected. That will happen if there are already enough items
to test the particular concept or skill that it assesses.
Page 6 of 13
Modu
le 6
Assess
ment of Student Learning
The table below shows a sample result of an item analysis, illustrating the steps of
the U-L Index method.. Study the results and focus on the interpretation of the test item.
Table showing a summary of the Item Analysis
Ite
Uppe
Lowe
No.
1
27%
14
27%
12
Interpretation
Interpretation
Action
.88
.75
.82
Very easy
.13
Moderately
Revise
Retain
10
.63
.38
.51
Moderately
.25
discriminating
Discriminating
11
.69
.44
.57
difficult
Moderately
.25
Discriminating
Retain
.43
.37
Discriminating
Discriminating
Retain
Retain
4
5
9
12
2
6
.56
.75
.13
.38
.35
.57
difficult
Difficult
Moderately
6
7
6
13
14
4
.38
.81
.88
.25
.63
.53
difficult
Easy
Moderately
-.50
.56
Questionable
Discriminating
Discard
Retain
-.44
Questionable
Discard
10
.19
.63
.41
difficult
Moderately
13
12
.81
.75
.78
difficult
Easy
.06
Moderately
Revise
10
.50
.38
.44
Moderately
.12
discriminating
Moderately
Revise
difficult
discriminating
from the upper group and another 16 students for the lower
group. The middle group (28 students) will not be included in the analysis.
Page 7 of 13
Modu
le 6
Assess
ment of Student Learning
Step 3. The number of students from the upper group was tallied. For example, out of 16
students, 14 students got item 1 correctly, 10 students for item 2, and so forth.
Step 4. The number of students from the lower group was tallied. For example, out of 16
students, 12 students got item 1 correctly, 6 students for item 2, and so forth.
Step 5. The percentage of the students from the upper group is obtained by dividing the
number of students who got the item right and the total number of students; that is
14/16 = .88, or 88%., 10/16 = .63 or 63 % for item 2, and so forth. The results are
placed under Column U (upper group)
Step 6. The same procedure like in step 5 was performed. For example, 12/16 = .75 0r
755 for item 1, and so forth. Results are placed under Column L (lower group)
Step 7. The difficulty index (p) was computed by getting the average of U and L
percentages. For example in item 1 U = .88, L = .75. Therefore, p = (U+L)/2 =
(.88+.75)/2 = .82. This means that 82 % of the students got item 1 correctly.
Step 8. The discrimination index (D) is obtained by subtracting L from U, that is D = U
L, or D = .88 - .75 = .13
So for item number 1, the difficulty index is .82 and the discrimination index is .
13. Using the tables presented earlier, a difficulty index of .82 is said to be very easy, and
a discrimination index of .13 is moderately discriminating. The item therefore needs
revision. The item was too easy because there were many students, even from the lower
group who got the item correctly.
Modu
le 6
Assess
ment of Student Learning
An item distractor analysis is performed to review items judged to be problematic
during difficulty and discrimination analyses, to evaluate the plausibility of distracters or
incorrect options, and to identify areas in which instruction needs to be revised. For
example, item number 1 in the previous item analysis needs revision. Why? Because the
item is very easy, though it can moderately discriminate. In selection-type of test, like the
multiple choice, an item may be very easy or very difficult because of the options or
alternatives.
Options
C
2
4
Upper
Lower
A
0
0
B
14**
12**
D
0
0
Upper
Lower
13**
12**
1
2
1
2
1
0
Upper
8**
Lower
6**
1
9
10
N = 16
10
** correct answer/key
For example, item number has p value of .82 and D value is .13. The item is easy
but discriminating, which means that the item needs revision. If we will revise the item,
Page 9 of 13
Modu
le 6
Assess
ment of Student Learning
we might look as well at the distracters. Not one of the respondents choose A and D.
This may have two explanations. The distracters are impossible or illogical to them. You
may have included distracters that are obviously wrong. However, if you really think that
A and D represent logical alternatives to the correct answer, then it is more probable that
B reflects common misconception held by large portion of the class. This requires
revising your initial instruction and includes information on how to avoid the problem.
The same thing applies to item number 9. For item number 10, the item needs revision as
well. Though the item can moderately because there were more students from the upper
group than the lower group who got the correct answer, the same number of students
from the upper group chose a distracter as the correct answer (option B). Similarly,
looking at the lower group, it seems that option B is the correct answer for item 10.
In summary, item analysis provides relevant information about the individual
items of the test, and the test as a whole. Always remember that you can maximize the
ability of the test to help you assess students learning by making sure that you construct
high quality test items.
Page 10 of 13
Modu
le 6
Assess
ment of Student Learning
Activity 1
A. Complete the table below.
Ite
Uppe
Lowe
Interpretation
Interpretation
No. 27%
27%
1
20
18
2
19
12
3
17
11
4
10
20
5
21
11
6
9
2
7
24
14
8
18
13
9
9
19
10
22
15
11
26
24
12
25
13
13
6
3
14
23
12
15
11
19
Number of pupils tested = 10
Activity 2
A. Answer the following questions.
Page 11 of 13
Action
Modu
le 6
Assess
ment of Student Learning
1. When do we obtain a difficulty index of 0.0?; 1.0?
2. When do we get a negative discrimination index?; positive discrimination index?;
zero discrimination index?
3. If the item has a perfect discrimination index, what is the difficulty index of that
item?
4. In as far as item analysis is involved, what is a good item?
B. Analysis of Distractors
Below shows the students answers in 5 items. Study the information and
in each item, indicate (under remarks row) whether each option is a good, poor or
fair option. Figure with asterisk indicate the correct response.
Item 1
Upper
Lower
Remarks
A
10*
7
B
4
0
C
1
4
D
1
5
Item 2
Upper
Lower
Remarks
A
5*
16
B
5
0
C
5
0
D
1
0
Item 3
Upper
Lower
Remarks
A
2
8
B
3
2
C
4
5
D
7*
1
Item 4
Upper
Lower
Remarks
A
2
5
B
8*
5
C
1
1
D
5
5
Page 12 of 13
Modu
le 6
Assess
ment of Student Learning
Item 5
Upper
Lower
Remarks
A
16*
16
B
0
0
C
0
0
D
0
0
Activity 3
Page 13 of 13