What is Item Analysis?

Statistical properties of the item score are examined through a variety of procedures
known as item analysis. Item Analysis furnishes a variety of statistical data regarding
how subjects responded to each item and how each item relates to overall performance.
If an item analysis is conducted on a preliminary form of a test, the next step is to select
the best items to be used on the final version. The most basic results are the
proportions of responses to each alternative. 1) Be suspicious of any item if a distractor
is chosen more often than the correct alternative, 2) Distractors that are hardly ever
chosen are too transparently incorrect, 3) The proportion choosing the correct
alternative is the classical index of item difficulty. Items with extreme p-values should
be generally excluded since they do not discriminate among individuals (Information
taken from Nunnally and Bernstein, 1994; Crocker and Algina, 1986)

An item analysis must describe how each item relates to overall test performance and
thereby provide discrimination indices, of which there are several. (The best items are
the most discriminating). 1) Ordinary PM item-total correlation 2) the covariance
between an item and total score 3) the average correlation between a given item and all
other items and 3) the proportion of people passing the item in the top half of the class
minus the proportion of people passing the item in the bottom half of the class.

Another way to look at it:

The pool of items is administered to a screening sample comprised of subjects similar to
those for whom the scale is intended, who respond to each item on a scale indicating
varying intensities of approval/disapproval, agreement/disagreement, and the like. The
responses of the screening sample are subjected to an item analysis in order to
determine the adequacy of the individual items so that the best ones may be selected
for the inclusion of the scale. Two approaches to item selection is proposed by Likert: a)
items that discriminate between groups scoring high and low; or b) items whose
correlations with total scores are relatively high are selected (NB). (Pedhazur &
Schmelkin, 1991)

Item analysis is the computation and examination of any statistical property of an item
response distribution. For dichotomously scored items, the best known descriptor is
probably item difficulty (p), which denotes the proportion of examinees answering the
item correctly. For dichotomous items value close .50, for multiple choice items p-value
greater than .50 are ideal. (CA) NB warns us not to use items with too high p scores
though, as they will not have any distinguishing value attached to them. The best items
on a test are the ones best discriminating. The simplest discriminating index is the
ordinary PM item-total correlation (r) between each item and the total test score (NB).
(Crocker and Algina, 1986; Nunnally and Bernstein, 1994)