Eleven years ago, I discovered a life-changing pedagogy called team-based learning. It let me do things in large classrooms that I didn’t think was possible. I found that the key to successful team-based learning was writing really good multiple-choice questions. I would like to look at the multiple-choice format overall, including some of the vocabulary we use when looking at the literature on writing multiple-choice questions.
Over the years my beliefs around multiple-choice questions have evolved. Bob Bjork’s article called Multiple-Choice Questions Exonerated At Least of Some Charges, points to multiple-choice questions as really good vehicles to enhance learning. We often think that short-answer questions are the answer because in a multiple-choice question, the student needs only to recognize the right answer rather than retrieve it. Bjork’s studies have revealed that multiple-choice questions strengthen the student’s understanding of all the presented options, not just the single correct retrieval. His main idea is that multiple-choice questions have a really valuable place in learning. While they might not have a valuable place in assessment all of the time, in learning, multiple choice questions are really quite excellent.
Anatomy of a multiple-choice question
The question at the top, the actual question part of a multiple-choice question is called the stem, or it is sometimes called the question leader. The various answers, A, B, C, D, are known as the options. The option contains the correct answer and the incorrect answers. The correct answers are often referred to as keyed responses in the literature. The incorrect answers are referred to as distractors or foils.
Developing stems
The first step in writing good questions is deciding what is test-worthy knowledge. This is a place where instructors often fail when writing multiple-choice questions. We will start writing questions without thinking about what the students truly need to know. If you jump straight to writing questions, you tend to write questions at a very low level. If you step back a bit and identify the most important test-worthy knowledge, you can start writing questions at a higher level.
In developing stems, it’s helpful to write a standalone question. There are many examples of excellent multiple-choice questions that aren’t standalone, but that’s the gold standard, to try to make it a standalone question. A student should be able to put his or her hands over the options and successfully answer the question if it’s well-written. The stem should be grammatically complete and as clear as possible for the student.
Also, avoid negative stems if possible. If using negative words in the stem, you’ll want to bold them or underline them. The question should not be tricky. You want to make sure you’re testing the student’s understanding of the contents, not their ability to read the questions, so make bold or underline any “not” or negative statements.
Developing the options
There are a number of rules regarding option development. First, keep the options as short as possible. What you may often find is that, as you write the options, a phrase will consistently appear in the options. You don’t want long, rambling options because you can make the question tricky just by the length of the option.
Make sure there is a correct answer that’s clearly and defensibly the best, as well as incorrect answers that are clearly incorrect. There is a line you may be straddling there, as we want to make options plausible enough that we are testing the student’s understanding.
If options start to cluster around the correct end of the spectrum, you’ve created a question that will likely make the student mad. Try to avoid using the “all” or “none of the above,” mostly because the question should test higher-level understanding and nothing is absolute once you get to higher levels.
We also want to keep the options similar length so we don’t kind of give away any hints on what might be the right answer. Just like when writing lists, use parallel construction. Ensure grammatical consistency and make sure that there are not any cues in that question stem that are going to help students eliminate some of the options.
Construction errors
There are some very common errors that we all make as we write questions. The most common errors are grammatical cues. For example, you write a singular stem with a singular answer and a plural option. A test-wise student, if they don’t know the answer, will use that grammatical cue to guess more effectively.
Another one we often see, is in writing a series of options, the correct answer and some incorrect answers, and you’ll write a little more detail around the correct option so it really truly is correct. Again, a test-wise student will pick up on this. And if one of the options is longer than the others, they’ll guess that one, hoping that you have fallen into this trap.
Another place we can get ourselves into trouble is having logic cues called overlapping options. Repeating words is another one we often do. We’ll take a phrase or a word from the stem, and we’ll reuse it in one of the options.
There’s another human tendency as we develop multiple-choice questions to put the correct answer as option C or D. This is especially true when we write numerical questions with a numerical list. For some reason, we often don’t like to make the answer the first one or the last one because that’s “too obvious.” So be aware of this as you write your questions.
Take the time to write good questions
So have a talk with yourself, and make sure you spend the time to write good questions. The good news? You can write good questions with effort. The bad news: The better you get at writing multiple-choice questions, the longer it will take to write good questions. But in the end, writing good questions is well worth the time and effort involved.
Dr. Jim Sibley is director of the Centre for Instructional Support at the Faculty of Applied Science at the University of British Columbia (UBC) in Vancouver, Canada.
