Since course evaluations started being collected online, response rates have plummeted. In one study they ranged from 23 percent to 47 percent with the mean at 33 percent, compared with a range of 33 percent to 75 percent with a mean of 56 percent for paper evaluations. Low response rates raise the issue of representativeness.
Incentives, like extra credit, do improve those percentages, but do they encourage students to go through the motions to get the credit without providing accurate or useful feedback? There is also the lingering concern as to who evaluates the course: the students who loved it or those who hated it. Faculty opinions are mixed as to which group it might be, and research results to date haven’t resolved the issue. Who is evaluating the course matters because if it’s more of one group than the other, the results are biased.
These issues motivated a group of faculty researchers to see if they could identify those course factors that motivated students to submit online evaluations and whether the motivation to evaluate the course were influenced by negative or positive experiences associated with it. Their study design was unusual. Using student focus groups and relevant research, they identified five positive and five negative course feature pairs and then had students differentially weight all the pairs in relation to making the decision to submit a course evaluation. The course pairs included good grade vs. bad grade, valuable information vs. trivial information (a measure of the perceived quality of the course content), extra credit options vs. no extra credit options, high standards vs. low standards, and easy tests vs. challenging tests. Each characteristic was rated against all the others, making a 50-question forced-choice survey.
The total group chose extra credit in the course over all other course characteristics as a reason for submitting evaluations. “Thus, having extra credit options within a course presumably promotes submission of course evaluations” (24). A high grade was also high on the list of motivational characteristics. The characteristics rated the lowest were no extra credit options and a course with trivial information.
The total group also rated easy tests, high standards, and valuable information high. However, subgroup analysis revealed that students with As and Bs on the exams and As in the course rated hard tests higher than easy ones. Rating easy tests highly does not jive with faculty views of exams, and high student ratings for course characteristics that do not promote good learning are one of the criticisms faculty regularly level against course evaluations. These researchers observe that “they pressure faculty members to cater to student desires at the expense of student learning” (28).
There was one bright spot in the findings: “Our results are contrary to the view that students are more likely to submit course evaluations when they have had a bad rather than a good course experience” (28).
The finding that extra credit motivates students to submit evaluations is interesting given the opposition to extra credit expressed by many faculty. Some faculty don’t like to give extra credit because they feel it encourages students to do less than they need to on regular assignments because any grade deficiencies can be made up with extra credit. But extra credit can be designed as extra assignments, not busy work but additional opportunities to master the material or explore part of it more deeply. If it’s designed as a robust, intellectually challenging learning opportunity, perhaps it can find a place in the course. In many courses there are few things that motivate students; if extra credit does, perhaps faculty thinking about it merits a revisit.
Reference: Jaquett, C. M., Van Maaren, V. G., and Williams, R. L. 2017. “Course Factors that Motivate Students to Submit End-of-Course Evaluations. Innovative Higher Education 42(1).
