Those in higher education have long used assessments to measure learning, whether for a grade, to determine prior knowledge, or to see how well students are following a current lesson. But assessments can also serve as learning devices. One way to put assessment in the service of learning is by using “integrated testlets” as companions to lessons (Shiell & Slepkov, 2015). These consist of simple, auto-graded multiple-choice questions that are interspersed throughout a lesson. But unlike with traditional assessments, students’ answers add to their understanding, and that new understanding is needed to answer subsequent questions. Thus, the assessments are themselves lessons.
To ensure that a lack of understanding at one step does not carry forward, causing students to miss subsequent content and questions, students have unlimited attempts at each question but must get it correct before moving on. When students submit an incorrect answer, they receive robust feedback that helps them think through the problem and try it again.
Integrated testlets in action
Shiell and Sletkov use physics examples to illustrate how testlets work, but as these require specialized knowledge, I will illustrate the idea with an example from my own philosophy background that should be accessible to all readers. Imagine that I am using Socrates to teach a lesson on a logical device called Proof by Cases. I might first discuss Socrates’s famous argument in The Apology that he does not fear death because either (a) there is an afterlife, in which case he can discuss philosophical questions with others and so has nothing to fear, or (b) there is no afterlife, in which case it would be like an eternal sleep that, again, is not to be feared. Thus, while he does not know what happens upon death, in either case, there is nothing to fear, which is a Proof by Cases argument.
Now I can test student understanding with this interesting puzzle from the Numberphile YouTube channel. Imagine that there are three people in a room and we know the following about them:
- Jack is looking at Anne, but Anne is looking at George.
- Jack is married, but George is not.
The question: Is a married person looking at an unmarried person?
- Yes
- No
- Cannot be determined
This example illustrates how a question can extend a concept just learned to a new situation. The student needs to think about how to apply the concept, and many will fail, but that is fine because they get feedback and an opportunity to try again. Failure can also elicit the student’s curiosity to figure out why they got it wrong. If you got it wrong—the answer is A—you are likely right now trying to figure out why you got it wrong.
For those that got it wrong, I might give the following feedback: “Remember the principle of Proof by Cases and think about how that would apply to Anne.”
Ideally, a student who still cannot see the answer would get a further hint, such as “What can we say about Anne’s marital status using the Proof by Cases method?”
Notice how Socrates’ argument about not fearing death provides a simple understanding of the Proof by Cases method but figuring out why the answer to the question is A provides a deeper understanding of how to apply this method to different situations. Students could then move on to more complex examples of the method and ever harder questions that would continue to build their knowledge. In this way, the question is itself a learning device as part of a lesson about the Proof by Cases method.
Ways to implement integrated testlets
To incorporate integrated testlets into your online courses, you can start by seeing whether your learning management system’s quizzing function allows you to add questions within content. Unfortunately, no system that I know of can be set to require the correct answer before moving on, but as a workaround, you can explain to students that they must get all the questions right—as demonstrated by LMS quiz analytics—to receive credit for completing the lesson at all.
A second option is to load the content into a tool that combines content with questions. Nearpod and Pear Deck are the two most popular apps for this purpose. Both use slides to hold the content, so you would upload the content to slides and then add questions where desired. They can be used either live, with all students viewing the same slide on their computers at the instructor’s control, or in a self-paced mode that can be taken online. Both apps also provide all the analytics you need to ensure that students do the work.
Finally, PlayPosit is a good option for adding questions to videos. It allows you to insert questions into designated places in a video, whether it’s one that you have created or from an outside source, such as YouTube. You can set the questions to allow for multiple attempts and the video to require that the student get all questions correct for the activity to be marked complete. It allows you to view and download analytics and integrates into common LMSs.
Integrated testlets transform assessments from just a way to measure knowledge, to a way to produce knowledge. They are a great way to both ensure and deepen student understanding as they move through learning material.
Reference
Shiell, R. C., & Slepkov, A. D. (2015). Integrated testlets: A new form of expert-students collaborative testing. Collected Essays on Learning and Teaching, 8, 201–210. https://doi.org/10.22329/celt.v8i0.4244
