Memo 1: Guessing | Memo 2: Difficulty | Memo 3: Essays
Memo 4: Multiple Choice 1 | Memo 5: Multiple Choice 2
Memo 6: Averaging Grades | Memo 7: Assigning Grades
Memo 8: Reliability | Memo 9: Missed Test
Memo 10: Multiple Choice 3 | Memo 11: Absolute/Relative Grading
Although these memos are offered for your information, Testing & Data Services no longer has the expertise to support them.
During the 1978-79 academic year, a Testing Advisory Committee served the Research and Measurement Division of the Learning Resources Center. In response to faculty inquiries about various aspects of testing, the Committee prepared a series of seven Testing Memos, which were circulated every few weeks to users of Research and Measurement's test scoring and analysis service.
Since 1991, four additional Memos have been circulated. Below are the complete set with slight editing changes.
Members of the Committee were Robert B. Frary (Chairman), Lawrence H. Cross, Jerard F. Kehoe, and Larry J. Weber. Frary wrote MEMOS 1 and 2, Cross wrote MEMOS 3 and 6. Kehoe wrote MEMOS 4 and 5, and Weber wrote MEMO 7. Subsequent MEMOS 8-10 were written by Frary, and MEMO 11 is by Cross.
Robert B. Frary
Ever since multiple-choice tests became widely used, in the 1920s, there has been concern over the fact that guessing affects the scores on these tests. At first the phenomenon was not well understood, and score increases due to guessing were uncritically viewed as ill-gotten gains even though these score components usually reflected partial knowledge - the ability to eliminate some wrong choices before guessing. The reaction of some educators was to admonish students against all guessing, directly or indirectly condemning it as dishonest. Of course, admonishing students against guessing was ineffective as well as unfair to those who refrained, so long as the tests were scored on the basis of the number of right answers.
As a result, many educators avoided use of multiple-choice tests. However, multiple-choice tests became indispensable for mass testing and were found to have other virtues which argued for their inclusion in the educational process, such as broader coverage of instructional topics, accuracy of scoring, and provision of statistical feedback at the item level. Hence, neither admonishment against guessing nor avoidance of multiple-choice tests was a satisfactory approach to resolving what was perceived as a problem.
One approach which gained wide acceptance was the use of a scoring formula which "corrects" for purely random guessing. The conventional correction formula subtracts a fraction of the wrong answers from the number-right score. A mathematically equivalent procedure is to award partial score credit for omitted questions rather than deduct score credit for wrong answers. The later approach has a psychological advantage over the former method since it rewards the desired behavior, not guessing completely at random, rather than penalizing undesired behavior. Regardless of which correction formula is employed, ethics require that students be encouraged to answer all questions on which one or more choices can be eliminated as incorrect. Only if the answer would represent a sheer guess among all choices should the examinee be directed to omit the questions when formula scoring is to be used.
Either of the procedures just described may be desirable when many examinees are expected to be unable to finish or to be completely ignorant on large proportions of a test. However, there are several reasons why either method of "correcting for guessing" is likely to be undesirable in a typical college academic setting:
1. Very few examinees will be so ignorant or so slow that they will fail to attempt or be completely unable to eliminate a single wrong choice on any substantial proportion of questions. Hence the effort of "correcting for guessing" is largely wasted. The few who legitimately should omit substantial proportions of questions under formula scoring will be so low in achievement that very low scores will result regardless of whether random guessing is suppressed.
2. The admonishment not to guess in the absence of information may be interpreted differently by each examinee and thus may introduce score variance associated with personality or background factors. This phenomenon has been confirmed in numerous published studies. Other published studies have shown that when students do omit questions under conventional "correction for guessing" instructions, they are (on the average) able to choose significantly more correct answers to these questions than under chance expectation.
3. Individuals may choose to disregard the instructions since, on the average, "correcting for guessing" does not penalize for random guessing but only removes the resulting expected score gain. In fact, if a student's knowledge is inadequate for obtaining a needed score, the best strategy for that student is to guess on all questions, hoping that luck in the short term will be favorable. Since this action is contrary to the instructions never to guess randomly among all choices, the instructor is in the questionable position of giving directions which some students may ignore to their benefit.
In balance, then, it is difficult to recommend any scoring procedure to control guessing for typical college multiple-choice testing. In the absence of this practice, the only fair procedure is to advise all students that it is to their advantage to answer every question regardless of knowledge.