As many classes and exams migrate online, many professors are increasingly concerned about an uptick in cheating. Preliminary numbers indicate those concerns are valid. In August 2020, Derek Newton in The Hechinger Report disclosed that at North Carolina State University, for example, 200 students in a single statistics class of 800 were caught cheating.
The unfortunate reality is that whenever exams are required, there will be some students who try to cheat. Teachers should work to create exams that reduce students’ ability and incentives to do so. Randomizing the order of questions and answers is a good start but insufficient by itself. Some learning management systems (LMSs), such as Canvas, have additional built-in tools that can make exams more secure.
One of those tools, question banks, can be a reliable way for professors to create flexible and secure exams. We define a question bank as a collection of multiple-choice questions and answers. They can be created in a variety of ways—by individual professors, by teaching teams, or, in prepackaged form, by textbook authors, for example.
We recommend that professors consider creatively organizing their question banks, especially if their LMS empowers them to randomly select a specified number of questions from individual banks. To illustrate the flexibility and increased security multiple question banks can provide, let’s consider several exam formats that each consist of 50 questions selected from a set of 100 total questions. There are many ways those 100 questions might be chosen for an exam. The only real difference between the examples discussed below is the way these questions are organized into various banks.
“A normal exam”
This is the standard midterm or final exam prepared by most professors. The professor selects which 50 of the 100 available questions will appear on every exam. Thinking in terms of question banks, this exam format has a single bank with 50 questions, and the exam generator selects all of them in random order. As a result, each student receives the same 50 questions. The concern for this setup is that it provides a high incentive as well as a high reward for cheating. If Student A tells Student B that a specific question appeared on their exam, Student B has a 100 percent chance of receiving that same question.
One expanded test bank
Consider, though, a slightly modified exam format. It also uses a single question bank, except this time all 100 of the available questions are included in that bank. Rather than having the professor preselect which questions will appear on every exam, the automated exam generator selects 50 questions at random from the 100 questions in the bank for each student. It is unlikely that any two exams would be the same. Student A may still tell Student B that a specific question was on their exam, but there is a good chance Student B will not receive that same question. Suffice it to say that doubling the number of questions in a single bank has an astounding effect on the number of possible question combinations. Doubling the number of questions from 50 to 100—but still selecting 50 questions—increases the number of possible exam formats from 1 to an incredibly large number. (Believe it or not, that number is 100,891,344,545,564,000,000,000,000,000.[1])
A possible downside to using a “single question bank” approach is that some exams generated may overemphasize certain learning goals or units while deemphasizing or eliminating others. Students may also receive exams with varying degrees of difficulty because of the way questions from the bank were randomly selected. There is an easy way to correct this weakness, however.
Multiple question banks
The solution is to use several narrowly defined question banks. The professor might create separate question banks for each unit or learning objective to test the course content more evenly. Let’s imagine our hypothetical 100-question set contains questions about five units in our course. We could create five different question banks that include 20 questions from each unit. The exam generator would randomly select 10 questions from the five unit-level question banks, thereby guaranteeing that each student would receive the same number of questions from every unit in the course.
Since questions are randomly chosen, there would be over 13 trillion ways questions from each bank could be selected. When using five banks in this manner, the number of possible tests that could be generated increases to 13 trillion multiplied by 5!
If you wish to carry this concept to the extreme, you could organize those same 100 questions into 50 different banks with two questions each. It is important, of course, to ensure that questions within each bank are of equal difficulty.
Hybrid approach
To create an ideal customized exam, you might use a combination of the approaches mentioned above. Your 50-question exam might look something like this:
- Ten questions chosen from one bank that contains exactly 10 questions. (Use this approach if you have questions you want every student to receive.)
- Ten questions selected from 1 bank that has 20 or more questions. (You could use this approach for unit-level questioning or testing general course concepts.)
- Thirty questions selected from 30 banks that have a varying number of questions in them. (This approach could be appropriate for testing individual learning goals or specific topics in your course.)
As remote learning continues, you may wish to consider using a variety of question bank strategies to enhance the flexibility and security of your exams.
[1] Here is a brief math review: A factorial, written with an exclamation point, is the result of multiplying the whole numbers from 1 to the number in question. For example, 3! = 1 * 2 * 3 = 6. 4! = 24, 5! = 120, and so on. When you have x questions in a question bank and select fewer questions than that number, there are multiple ways to select the questions. With a three-question bank, you could pick “1,2,” “2,3,” or “1,3.” The formula defining how many question combinations there are in an individual question bank is “C(n,r) = n! / (r! ( (n – r)!),” where n is the number of questions in a question bank and r is the number of questions chosen for an exam.
Reference
Newton, D. (2020, August 7). Another problem with shifting education online: Cheating. The Hechinger Report. https://hechingerreport.org/another-problem-with-shifting-education-online-cheating
Michael L. Shamo, PhD, has worked as a public history researcher at the University of Utah’s American West Center and for the Church History Department of The Church of Jesus Christ of Latter-day Saints. He currently teaches courses on American history at Utah Valley University and Brigham Young University.
Kenneth L. Alford, PhD, is a professor of church history and doctrine at Brigham Young University. After almost 30 years in the U.S. Army, he retired as a Colonel in 2008. He previously served as a professor at the U.S. Military Academy at West Point and a department chair at the National Defense University in Washington, DC.
