A Case Where Rubrics Worked!

Teachers are giving students rubrics to help improve the quality of their work, but do they? Does student work, say, writing a paper, improve when students are given the criteria that will be used to assess their work? Kathleen Greenberg notes in her article that there is “very little empirical research on the value of rubrics in enhancing student learning.” (p. 211) And what research there is offers a mixed bag of results. Greenberg's work contributes to the research with two studies exploring whether rubrics improved writing in research methods courses in psychology. In both studies she used a rubric she created (and tested in previous research) based on “a deconstruction of an APA-style research report into a set of 60 learning outcomes” that define the essential features of these papers. (p. 212) The first study explored whether students who used the rubric prescriptively to guide preparation of their reports wrote better papers than students who did not use the rubric. Six sections of an introductory course and six sections of an advanced course provided the cohorts for this study. Three sections of introductory students and three sections of advanced students were told to use the rubric as a guide when writing their reports. Students in both of these courses were familiar with rubrics, and the features of this particular instrument had been discussed in class. The papers were graded by another instructor familiar with the rubric but blind to whether or not the student had used it in writing the research report. “Independent measures t-tests showed that in both the introductory and the advanced course, the mean score for the students who used the rubric was significantly higher than the mean score for students who did not.” Greenberg believes the success of the rubric in this case may be its focus on behavioral components. “It appears that the rubric . . . can serve as an easily readable ‘blueprint' for helping students reconstruct the report that the rubric has essentially taken apart.” (p. 215) The second research question explored whether using the rubric to “grade” a classmate's paper resulted in improvement when the student grader revised his or her own paper. Forty-six students in the introductory-level course received the rubric before they started writing their research reports. They did not know that they'd have the opportunity to rewrite their papers. With names replaced by a code, each student received another student's paper, and using the rubric, they had one week to grade it. The graded papers were submitted, but not returned to the student author. Students then were given a week to rewrite their papers. Greenberg's objective was to help students become better at seeing the strengths and weaknesses in their own papers. “The value of this skill cannot be understated, as this type of metacognition is fundamental to the writing and editing process.” She explains further, “Just as in the visual arts where it is said that learning to draw is actually about learning to see, the ability to see in one's writing what is well said and what needs to be said differently is an integral (perhaps defining) part of the writing process . . . how can one fix what one cannot see?” (p. 218) Results for this second study were equally positive. Mean scores on the rewritten reports were significantly higher than mean scores on the original reports, and the analysis of variance (ANOVA) showed that the improvement was evident throughout the report, not just on the easier sections like the title page and abstract. However, not all the report scores were higher on the rewritten version. Over a third of the students received a lower score on the rewrite than on the original. For the 60 percent who did receive higher scores, on average their scores improved by 10 points, and those who scored lower lost on average four points. These results are encouraging but should be considered with caveats. The study looked at the impact of rubrics on writing a particular kind of research-based report. The rubric improved writing on this assignment. Whether it improved students' scientific writing generally is not established by these results. And whether rubrics are valuable pedagogical tools across the board remains to be seen. However, results like these are enough to cause us to take them seriously and further explore their role in our courses.  Reference: Greenberg, K. P. (2015). Rubric use in formative assessment: A detailed behavioral rubric helps students improve their scientific writing skills. Teaching of Psychology, 42 (3), 211-217.