testing memos
Memo 1: Guessing | Memo 2: Difficulty | Memo 3: Essays
Memo 4: Multiple Choice 1 | Memo 5: Multiple Choice 2
Memo 6: Averaging Grades | Memo 7: Assigning Grade
Memo 8: Reliability | Memo 9: Missed Test
Memo 10: Multiple Choice 3 | Memo 11: Absolute/Relative Grading
Testing Memo 1:
Guessing on Multiple-Choice Tests
Robert B. Frary
Ever since multiple-choice tests became widely used, in the 1920s,
there has been concern over the fact that guessing affects the scores on
these tests. At first the phenomenon was not well understood, and score increases due
to guessing were uncritically viewed as ill-gotten gains even though
these score components usually reflected partial knowledge - the
ability to eliminate some wrong choices before guessing.
The reaction of some educators was to admonish students against all
guessing, directly or indirectly condemning it as dishonest.
Of course, admonishing students against guessing was ineffective as well
as unfair to those who refrained, so long as the tests were scored on the
basis of the number of right answers.
As a result, many educators avoided use of multiple-choice tests.
However, multiple-choice tests became indispensable for mass testing and
were found to have other virtues which argued for their inclusion in the
educational process, such as broader coverage of instructional topics,
accuracy of scoring, and provision of statistical feedback at the item
level.
Hence, neither admonishment against guessing nor avoidance of
multiple-choice tests was a satisfactory approach to resolving what was
perceived as a problem.
One approach which gained wide acceptance was the use of a scoring
formula which "corrects" for purely random guessing.
The conventional correction formula subtracts a fraction of the
wrong answers from the number-right score.
A mathematically equivalent procedure is to award partial score credit
for omitted questions rather than deduct score credit for wrong answers.
The later approach has a psychological advantage
over the former method since it rewards the desired behavior, not
guessing completely at random, rather than penalizing undesired
behavior.
Regardless of which correction formula is employed, ethics require that
students be encouraged to answer all questions on which one or more
choices can be eliminated as incorrect.
Only if the answer would represent a sheer guess among all choices
should the examinee be directed to omit the questions when formula
scoring is to be used.
Either of the procedures just described may be desirable when many
examinees are expected to be unable to finish or to be completely
ignorant on large proportions of a test.
However, there are several reasons why either method of "correcting for
guessing" is likely to be undesirable in a typical college academic
setting:
1. Very few examinees will be so ignorant or so slow that they will
fail to attempt or be completely unable to eliminate a single wrong
choice on any substantial proportion of questions.
Hence the effort of "correcting for guessing" is largely wasted.
The few who legitimately should omit substantial proportions of
questions under formula scoring will be so low in achievement that very
low scores will result regardless of whether random guessing is
suppressed.
2. The admonishment not to guess in the absence of information may be
interpreted differently by each examinee and thus may introduce score
variance associated with personality or background factors.
This phenomenon has been confirmed in numerous published studies.
Other published studies have shown that when students do omit questions
under conventional "correction for guessing" instructions, they are (on
the average) able to choose significantly more correct answers to these
questions than under chance expectation.
3. Individuals may choose to disregard the instructions since, on the
average, "correcting for guessing" does not penalize for random guessing
but only removes the resulting expected score gain.
In fact, if a student's knowledge is inadequate for obtaining a needed
score, the best strategy for that student is to guess on all questions,
hoping that luck in the short term will be favorable.
Since this action is contrary to the instructions never to guess
randomly among all choices, the instructor is in the questionable
position of giving directions which some students may ignore to their
benefit.
In balance, then, it is difficult to recommend any scoring procedure to
control guessing for typical college multiple-choice testing.
In the absence of this practice, the only fair procedure is to advise
all students that it is to their advantage to answer every question
regardless of knowledge.
Page last updated
August 7, 2008
.