Home

Office Information
Location
Hours
Contact Information
Staff
Opscan Forms

Services
Exams
Evaluations
Research
Our Archives

Help
FAQ
Opscan Forms
Understanding Output
Excel

Other Information
Alternatives to Opscans
Test Design & Analysis
Testing Memos

testing memos

Memo 1: Guessing | Memo 2: Difficulty | Memo 3: Essays
Memo 4: Multiple Choice 1 | Memo 5: Multiple Choice 2
Memo 6: Averaging Grades | Memo 7: Assigning Grade
Memo 8: Reliability | Memo 9: Missed Test
Memo 10: Multiple Choice 3 | Memo 11: Absolute/Relative Grading

 

Testing Memo 1:
Guessing on Multiple-Choice Tests

 

Robert B. Frary

Ever since multiple-choice tests became widely used, in the 1920s, there has been concern over the fact that guessing affects the scores on these tests. At first the phenomenon was not well understood, and score increases due to guessing were uncritically viewed as ill-gotten gains even though these score components usually reflected partial knowledge - the ability to eliminate some wrong choices before guessing. The reaction of some educators was to admonish students against all guessing, directly or indirectly condemning it as dishonest. Of course, admonishing students against guessing was ineffective as well as unfair to those who refrained, so long as the tests were scored on the basis of the number of right answers.

As a result, many educators avoided use of multiple-choice tests. However, multiple-choice tests became indispensable for mass testing and were found to have other virtues which argued for their inclusion in the educational process, such as broader coverage of instructional topics, accuracy of scoring, and provision of statistical feedback at the item level. Hence, neither admonishment against guessing nor avoidance of multiple-choice tests was a satisfactory approach to resolving what was perceived as a problem.

One approach which gained wide acceptance was the use of a scoring formula which "corrects" for purely random guessing. The conventional correction formula subtracts a fraction of the wrong answers from the number-right score. A mathematically equivalent procedure is to award partial score credit for omitted questions rather than deduct score credit for wrong answers. The later approach has a psychological advantage over the former method since it rewards the desired behavior, not guessing completely at random, rather than penalizing undesired behavior. Regardless of which correction formula is employed, ethics require that students be encouraged to answer all questions on which one or more choices can be eliminated as incorrect. Only if the answer would represent a sheer guess among all choices should the examinee be directed to omit the questions when formula scoring is to be used.

Either of the procedures just described may be desirable when many examinees are expected to be unable to finish or to be completely ignorant on large proportions of a test. However, there are several reasons why either method of "correcting for guessing" is likely to be undesirable in a typical college academic setting:

 

1. Very few examinees will be so ignorant or so slow that they will fail to attempt or be completely unable to eliminate a single wrong choice on any substantial proportion of questions. Hence the effort of "correcting for guessing" is largely wasted. The few who legitimately should omit substantial proportions of questions under formula scoring will be so low in achievement that very low scores will result regardless of whether random guessing is suppressed.

2. The admonishment not to guess in the absence of information may be interpreted differently by each examinee and thus may introduce score variance associated with personality or background factors. This phenomenon has been confirmed in numerous published studies. Other published studies have shown that when students do omit questions under conventional "correction for guessing" instructions, they are (on the average) able to choose significantly more correct answers to these questions than under chance expectation.

3. Individuals may choose to disregard the instructions since, on the average, "correcting for guessing" does not penalize for random guessing but only removes the resulting expected score gain. In fact, if a student's knowledge is inadequate for obtaining a needed score, the best strategy for that student is to guess on all questions, hoping that luck in the short term will be favorable. Since this action is contrary to the instructions never to guess randomly among all choices, the instructor is in the questionable position of giving directions which some students may ignore to their benefit.

 

In balance, then, it is difficult to recommend any scoring procedure to control guessing for typical college multiple-choice testing. In the absence of this practice, the only fair procedure is to advise all students that it is to their advantage to answer every question regardless of knowledge.

 

 

Page last updated August 7, 2008 .