Because research into the effects of test-item order on exam performance has produced equivocal results, the editor recently suggested that “it is not a bad idea for instructors to systematically order test items and analyze the results to see how test-item order might be affecting students,” in particular to examine “how … [students] feel when they come to a questions they can’t answer.”
The issue turns on the “fairly widespread belief among faculty that putting … easier questions first reduces exam anxiety [presumably by building confidence], … [which] can improve performance (The Teaching Professor, February 2013, p. 6). Logically, then, putting difficult questions first would be counterproductive. But is this in fact the case? Common sense and past experience with hard-to-easy test-item order suggests to me that most students, feeling increasingly overwhelmed by question after question to which they don’t know the answer, tend to say to themselves, “Well, I don’t know the answer to those first few questions, so I’ll move on and see if I have better luck with the next ones.”
If I’m correct, then previous studies involving students doing exams in the hard-to-easy order all contain the same methodological flaw. Simply put, researchers may indeed construct exams in which the hard questions occur at the beginning and the easy items at the end, but they have no guarantee that students actually proceed through such exams in the hard-to-easy order.Indeed, it seems quite likely that, left to their own devices, students encountering nothing but hard questions at the beginning of an exam would move fairly quickly toward the easy questions at the end of the exam, do those easy questions first, and attempt the hard questions last. Consequently, students would in effect have done the exam in an “easy to hard” order—exactly the opposite of what was intended by researchers—thereby rendering moot any comparison to performance on an actual easy-to-hard exam.
In research I conducted a few years ago, I did not leave students to their own devices! Rather, I required them to hand in the first half of the exam before receiving the second half.In other words, half the students completed easy items first and were only then given the hard questions; students in the other half of the class were required to finish the hard questions before they received the easy ones. Consequently, easy/hard and hard/easy item orders could be compared meaningfully (especially because students were responding on a “scratch and win” answer sheet, which gave them immediate question-by-question feedback as to whether or not each answer was correct). The results were compelling. “Students doing difficult items first obtained a mean score of 85% on later easy items, whereas student doing those same Easy items in the first half of the test averaged about 70%… (p is less than .01).” In sum, the answer to the question in the February Teaching Professor title is clear: YES, test-item order DOES matter!”
Reference
Skinner, N. F. (1999). When the going get tough, the tough get going: Effects of order of item difficulty on multiple-choice test performance. North American Journal of Psychology, 1 (1), 79-82.
Contact Nicholas F. Skinner at nskinner@uwo.ca.
