Using Evaluation Results to Improve: What Does It Take?

Teachers don’t always have the best attitudes about student rating results, and for reasons that are clearly understandable. Institutions often don’t evaluate teaching in the most constructive and useful ways. However, feedback from students is an essential part of any effort to grow and develop as a teacher. Here are some research results that shed light on ways teachers can approach rating results that make them a useful part of improvement efforts.

It pretty much starts and ends with having the right attitude, and these researchers offer an array of alternatives that describe what they mean by “attitude.” They suggest the term “approach”—how you approach evaluation results. They also propose “perception,” “perspective,” “stance,” “belief,” or “conception” (p. 1) “The question we address is: what attitude or approach do teachers take to their student evaluations so that they can use them for formative or developmental purposes, to improve their teaching rather than to give a final assessment of performance?” (p. 2)

To answer the question, they gathered information from experienced, award-winning teachers during two-hour focus group sessions. They used a semistructured interview process during which they asked questions like these: What overall attitudes do you need to use evaluation data to improve teaching? How do you interpret or analyze the evaluations? How do you decide what to do on the basis of the evaluations? How do you decide what changes to make on the basis of the evaluations?” (p. 4).

What emerged out of these focus group discussions was what the researchers labeled an “improvement attitude . . . an overarching approach, perspective, orientation, or way of looking at evaluation data” (p. 5) which they organized into four “interrelated, overlapping, smaller categories for ease of discussion” (p. 5)

Reflective Approach

This part of the approach begins with a simple question: “How can I improve?” The question is asked regularly; the pursuit of improved teaching is ongoing. Often what drives this question is basic curiosity—the need to find out what’s going on in one’s courses or always being on the lookout for ways to improve student learning.

The drive for improvement is not predicated on notions of remediation or deficiency. It’s not about needing to improve because the teaching still isn’t good enough. Too often, evaluation results are perused in order to identify what needs to be fixed. Another author cited in this article calls this the “fire alarm” approach, the belief that rating results are relevant only when there’s a fire in the course that needs to be put out. The alternative proposed by those interviewed in the study is the notion that there’s always room for improvement—it doesn’t matter how good you are, or if you aren’t all that good. The reflective approach is also tied up in thinking about current practice and not being so vested in what you do that you can’t possibly consider changing it.

Data Viewed as Formative Feedback

From this perspective, the feedback students provide is not a judgment of the teacher or the teaching. It’s simply feedback. Two questions illustrate the different perspective here: “How can I develop and improve my teaching?” rather than “How good is my teaching?” (p. 7).

If the focus is on the second question, then there’s often a need to explain away the results or to blame them on students, the course, or one’s teaching circumstances. A focus on the second question also makes it easier to simply ignore the results—the teaching is good enough, there’s no need to improve. In contrast, participants with an improvement attitude “focused on how they could use the data to improve, rather than paying attention to how they could explain the data away” (p. 7) All student feedback should be taken seriously and given careful consideration. As one teacher put it bluntly, “I’m right and 1,300 students are wrong? I don’t think so” (p. 7)

Avoiding excuses did not mean these teachers ignored reasons associated with the results. Rather, they could be used to explain some of the results. Required courses often generate student complaints. But that’s not where these teachers focused their attention. They were trying to figure out what the results meant, what they could learn from them about their teaching and about student learning experiences in the course. “If students give less than perfect ratings or negative feedback, this indicates that something is not working for them and so there is room for improvement.” (p. 8)

Belief in the Ability to Improve

“When teachers adopted the improvement attitude, they thought they could improve their teaching, they could make a difference for student learning, and they could improve their evaluations, rather than these all being unchangeable” (p. 8)

Participants in the study did agree that some of this confidence comes with experience. It’s not as easy to take negative feedback and to believe in your ability to make things better when your teaching experience is limited.

Student-Centered Approach

Again, two questions differentiate the improvement attitude from an attitude that becomes a barrier to improvement: “How well have my students learned?” versus “How well have I taught?” (p. 9). According to the study authors, “The teachers saw the evaluation data not just as shining a light on themselves and what they have done, but as shining a light on their students: what benefits and blocks their learning? How well are they enabled and supported to learn?” (p. 9). Fundamentally, these teachers’ thinking is more about learning and less about teaching.

However, it’s not as simple and straightforward as teachers doing whatever students suggest. Sometimes what students want isn’t what they need. What they suggest won’t improve their learning in the course. But what students want and what they think they need still merits consideration and response. If students are largely opposed to essay questions on exams, but the teacher has good reasons to believe that’s the best way to promote deep learning of course concepts, then it would be irresponsible to give students what they want. In such a case, the teacher must explain the educational rationale behind this choice of exam question and may want to explore with students what else could happen in the course that would help them prepare for essay questions.

This is a very fine article. It clearly articulates an attitude that makes rating results useful. Attitudes are not immutable. They can be changed and developed, and if ever there were a place in higher education where attitudes could benefit from change, faculty response to course evaluations is it. For a variety of reasons (many justifiable), many teachers don’t have good attitudes about student ratings. This work shows a clear path to a more useful, positive, and constructive way of thinking about course evaluation feedback from students.

Reference: Golding, C., and Adam, L. (2016). Evaluate to improve: Useful approaches to student evaluation. Assessment & Evaluation in Higher Education, 41(1), 1–14.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles

Love ’em or hate ’em, student evaluations of teaching (SETs) are here to stay. Parts <a href="https://www.teachingprofessor.com/free-article/its-time-to-discuss-student-evaluations-bias-with-our-students-seriously/" target="_blank"...

Since January, I have led multiple faculty development sessions on generative AI for faculty at my university. Attitudes...
Does your class end with a bang or a whimper? Many of us spend a lot of time crafting...

Faculty have recently been bombarded with a dizzying array of apps, platforms, and other widgets that...

The rapid rise of livestream content development and consumption has been nothing short of remarkable. According to Ceci...

Feedback on performance has proven to be one of the most important influences on learning, but students consistently...

wpChatIcon

Teachers don't always have the best attitudes about student rating results, and for reasons that are clearly understandable. Institutions often don't evaluate teaching in the most constructive and useful ways. However, feedback from students is an essential part of any effort to grow and develop as a teacher. Here are some research results that shed light on ways teachers can approach rating results that make them a useful part of improvement efforts.

It pretty much starts and ends with having the right attitude, and these researchers offer an array of alternatives that describe what they mean by “attitude.” They suggest the term “approach”—how you approach evaluation results. They also propose “perception,” “perspective,” “stance,” “belief,” or “conception” (p. 1) “The question we address is: what attitude or approach do teachers take to their student evaluations so that they can use them for formative or developmental purposes, to improve their teaching rather than to give a final assessment of performance?” (p. 2)

To answer the question, they gathered information from experienced, award-winning teachers during two-hour focus group sessions. They used a semistructured interview process during which they asked questions like these: What overall attitudes do you need to use evaluation data to improve teaching? How do you interpret or analyze the evaluations? How do you decide what to do on the basis of the evaluations? How do you decide what changes to make on the basis of the evaluations?” (p. 4).

What emerged out of these focus group discussions was what the researchers labeled an “improvement attitude . . . an overarching approach, perspective, orientation, or way of looking at evaluation data” (p. 5) which they organized into four “interrelated, overlapping, smaller categories for ease of discussion” (p. 5)

Reflective Approach

This part of the approach begins with a simple question: “How can I improve?” The question is asked regularly; the pursuit of improved teaching is ongoing. Often what drives this question is basic curiosity—the need to find out what's going on in one's courses or always being on the lookout for ways to improve student learning.

The drive for improvement is not predicated on notions of remediation or deficiency. It's not about needing to improve because the teaching still isn't good enough. Too often, evaluation results are perused in order to identify what needs to be fixed. Another author cited in this article calls this the “fire alarm” approach, the belief that rating results are relevant only when there's a fire in the course that needs to be put out. The alternative proposed by those interviewed in the study is the notion that there's always room for improvement—it doesn't matter how good you are, or if you aren't all that good. The reflective approach is also tied up in thinking about current practice and not being so vested in what you do that you can't possibly consider changing it.

Data Viewed as Formative Feedback

From this perspective, the feedback students provide is not a judgment of the teacher or the teaching. It's simply feedback. Two questions illustrate the different perspective here: “How can I develop and improve my teaching?” rather than “How good is my teaching?” (p. 7).

If the focus is on the second question, then there's often a need to explain away the results or to blame them on students, the course, or one's teaching circumstances. A focus on the second question also makes it easier to simply ignore the results—the teaching is good enough, there's no need to improve. In contrast, participants with an improvement attitude “focused on how they could use the data to improve, rather than paying attention to how they could explain the data away” (p. 7) All student feedback should be taken seriously and given careful consideration. As one teacher put it bluntly, “I'm right and 1,300 students are wrong? I don't think so” (p. 7)

Avoiding excuses did not mean these teachers ignored reasons associated with the results. Rather, they could be used to explain some of the results. Required courses often generate student complaints. But that's not where these teachers focused their attention. They were trying to figure out what the results meant, what they could learn from them about their teaching and about student learning experiences in the course. “If students give less than perfect ratings or negative feedback, this indicates that something is not working for them and so there is room for improvement.” (p. 8)

Belief in the Ability to Improve

“When teachers adopted the improvement attitude, they thought they could improve their teaching, they could make a difference for student learning, and they could improve their evaluations, rather than these all being unchangeable” (p. 8)

Participants in the study did agree that some of this confidence comes with experience. It's not as easy to take negative feedback and to believe in your ability to make things better when your teaching experience is limited.

Student-Centered Approach

Again, two questions differentiate the improvement attitude from an attitude that becomes a barrier to improvement: “How well have my students learned?” versus “How well have I taught?” (p. 9). According to the study authors, “The teachers saw the evaluation data not just as shining a light on themselves and what they have done, but as shining a light on their students: what benefits and blocks their learning? How well are they enabled and supported to learn?” (p. 9). Fundamentally, these teachers' thinking is more about learning and less about teaching.

However, it's not as simple and straightforward as teachers doing whatever students suggest. Sometimes what students want isn't what they need. What they suggest won't improve their learning in the course. But what students want and what they think they need still merits consideration and response. If students are largely opposed to essay questions on exams, but the teacher has good reasons to believe that's the best way to promote deep learning of course concepts, then it would be irresponsible to give students what they want. In such a case, the teacher must explain the educational rationale behind this choice of exam question and may want to explore with students what else could happen in the course that would help them prepare for essay questions.

This is a very fine article. It clearly articulates an attitude that makes rating results useful. Attitudes are not immutable. They can be changed and developed, and if ever there were a place in higher education where attitudes could benefit from change, faculty response to course evaluations is it. For a variety of reasons (many justifiable), many teachers don't have good attitudes about student ratings. This work shows a clear path to a more useful, positive, and constructive way of thinking about course evaluation feedback from students.

Reference: Golding, C., and Adam, L. (2016). Evaluate to improve: Useful approaches to student evaluation. Assessment & Evaluation in Higher Education, 41(1), 1–14.