Shortly after 2000, higher education institutions started transitioning from paper and pencil student-rating forms to online systems. The online option has administrative efficiency and economics going for it. At this point, most course evaluations are being conducted online. Online rating systems have not only institutional
I started composting at our summer place in 2009, and now I’m a convert. In the summer, we live on an island that’s mostly rock covered with something the locals call “organic matter.” Growing anything this far north on this soil base is challenging, but
Right before the end of the academic year when the promise of summer stretches warmly ahead, many of us are making lists that anticipate other kinds of tasks. If you’re considering some pedagogical reading, I’ve got just the recommendation.
There hasn’t been a lot written recently about test anxiety, but that doesn’t mean it’s no longer an issue for a significant number of students. Those of us who don’t suffer from test anxiety—and I’m betting that’s most faculty—can find it hard to be sympathetic.
When teachers tell me about some new strategy or approach they’ve implemented, I usually ask how they found out about it and almost always get the same response: “Oh, a colleague told me about it.” I continue to be amazed by the amount of pedagogical
“First and last class sessions are the bookends that hold a course together.” I heard or read that somewhere—apologies to the source I can’t acknowledge. It’s a nice way to think about first and last class sessions. In general, teachers probably do better with the
I haven’t found too many pedagogical articles worth a regular re-read. Christa Walck’s “A Teaching Life” is a notable exception. It’s a soul-searching, personal narrative that confronts the difference between what a teaching life can be and what it is.
I’ve been rethinking my views on quizzing. I’m still not in favor of quizzes that rely on low-level questions where the right answer is a memorized detail or a quizzing strategy where the primary motivation is punitive, such as to force students to keep up
On a more-or-less regular basis, I find myself looking for something that I’ve written about in the newsletter or blog, which I only vaguely remember. Inevitably, my search leads me to something else that I’ve completely forgotten… and it is such a good idea or
Now here’s an argument I haven’t heard before: Improving your instruction makes it easier for students to learn. If it’s easier for them to learn, they won’t work as hard in the course, and that means they could learn less. It’s called offsetting behavior and
Magna Publications © 2023 All rights reserved
But not all the news is good. Response rates drop significantly when students do the ratings online, from 70–80% for paper and pencil forms to 50–60% online. A 2008 review of nine comparison studies reported that online response rates averaged 23% lower than traditional formats. These low response rates raise the issue of generalizability. What percentage of students in a course need to respond for the results to be representative? The answer depends on a number of variables, most notably class size. For a class of 20 students, one expert puts the minimum at 58%. As class size increases, the percentage drops. Despite some disagreement as to the percentages, there is consensus that online response rates should be higher than they are right now.
Goodman, Anson, and Belcheir surveyed 678 faculty across a range of disciplines asking them to report how they were trying to boost online response rates. Among those surveyed, 13% reported that they did nothing to improve the rates and that, on average, 50% of their students completed the forms. Those who did something to encourage students to complete the evaluations generated response rates of 63%. The most common approaches faculty reported were the ones we’d expect. They reminded students to complete the forms, which upped the response rate to 61%, and they explained how the results helped them improve instruction, which bumped the rate up to 57%. But what improved response rates the most (roughly 22%) was to provide students with incentives.
The incentives were grouped as point-based or non-point-based and individual or class-wide. Points were by far the most common incentive, used by 75% of the faculty who reported offering incentives. Points given for completing the evaluation ranged from 0.25 to 1% of the total grade. The most common class-based incentive involved setting a target response rate—say, 80%—and then rewarding the class if the target was reached. For example, students could use a crib card during the final (non-point-based) or got a designated number of bonus points. In an earlier blog post, I described an institutional incentive in which those who completed course evaluations got early access to their grades.
This study of incentives explored other interesting issues (which makes it worth reading), but I want to use the rest of today’s post to focus on using incentives to increase response rates. I can understand why faculty do it. Ratings are a part of promotion and tenure processes, and they affect adjunct employment and sometimes merit raises too, so I’m not interested in moral judgments about individuals who have decided to do.
But regardless of what we do in our courses, all of us need to think about the implications of the practice. What messages do we convey to students when we offer incentives to complete course evaluations? Does it change the value of the feedback? We also should consider why response rates are so low. Is it because once students reach the end of the course, they just want to be done and aren’t really interested in helping to improve the course for students who will take it after them? Or have they grown tired of all these course evaluations and don’t think their feedback makes any difference anyway?
Perhaps we all can agree that offering incentives to complete the evaluations doesn’t get students doing ratings for the right reason. Students should offer assessments because their instructors benefit from student feedback the same way students learn from teacher feedback. They should be doing ratings because reflecting about courses and teachers enables students to better understand themselves as learners. They should be doing these end-of-course evaluations because they believe the quality of their experiences in courses matters to the institution.
The bottom line question: Is there any way to get students doing ratings for the right reasons? Please, let’s have a conversation about this.
Reference: Goodman, J., Anson, R. and Belcheir, M., (2015). The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates. Assessment & Evaluation in Higher Education, 40 (7), 958-970.
© Magna Publications. All rights reserved.
