The importance of small group dynamics is hard to overstate. Group functioning directly influences how well students learn the content, what they learn about working with others, and the attitudes they take with them from the experience. Most groups experience problems; while they usually start small, they’ll likely grow larger if left unaddressed. Being new to collaborative work, students don’t always recognize unhealthy group dynamics or know how to respond to them. Fortunately, researchers (O’Neill et al., 2018) have developed a measure that students can use to assess the health of their team. It “defines effective team health in terms of communication, adaptability, relationships and education,” or CARE (O’Neill et al., 2020, p. 1122). The first iteration of the instrument contained 64 items. The goal of this research was to create a shorter version (dubbed Bare CARE) that still produced stable and robust results.
The study
O’Neill, T. A., Pezer, L., Solis, L., Larson, N., Maynard, N., Dolphin, G. R., Brennan, R. W., & Li, S. (2020). Team dynamics feedback for post-secondary student learning teams: Introducing the “Bare CARE” assessment and report. Assessment & Evaluation in Higher Education, 45(8), 1121–1135. https://doi.org/10.1080/02602938.2020.1727412
The research questions
- Will psychometric analyses and content validity identify any instrument items that the researchers can eliminate?
- Will correlations with teamwork variables and team performance show that a shorter a version of the CARE instrument remains reliable and valid?
Interesting background information
The original CARE instrument measured groups’ communication, adaptability, relationships, and education (or learning) with 13 previously validated scales. “These scales were taken directly from other publications that offered reliability and validity evidence, thereby ensuring that the items were accurate, reliable and valid indicators of the intended constructs” (p. 1125).
Here are a few examples of how the instrument operationalized each of the four CARE areas: Healthy communication in a group involves cooperative conflict management—that is, if team members treat conflict as a mutual problem to solve. The group’s adaptability is illustrated when they monitor their progress toward the group’s goals. Relationships in healthy groups depend on members contributing equitably; the presence of healthy, fact-driven conflict; and trust. Those in the group educate themselves and learn whether they evaluate diverse options and work for decisions everyone can accept.
Study cohort
Data collected from those using the original instrument gave the research team access to more than 200,000 assessments. From that database they pulled 1,369 classes involving 61,549 students who participated in 14,601 teams and used this cohort to compare the pared-down instrument with the original.
Methodological overview
The team began by attempting to identify items that they could remove. They explored their psychometrics by using various tests and analyses, all thoroughly explained in the article. To further aid understanding of the process, they offer several examples showing what the analysis revealed about particular items and how they dealt with that data. They then compared the shortened version, Bare CARE, to the longer one in terms of intra-class correlations, concurrent validities, and criterion validities.
Findings
Did the psychometric analyses and content-validity exploration identify instrument items that could be eliminated?
- Yes, psychometric analysis uncovered 19 items that could be eliminated, thereby reducing the length of the instrument by 32 percent.
Did the shorter a version of the CARE instrument remain reliable and valid when correlated with teamwork variables and team performance?
- “The current research provides a large step forward in building the validity case in support of the original and Bare CARE assessments. The large samples provide strong evidence that the scales in the Bare CARE are valid and reliable indicators of team functioning” (p. 1132).
Cautions and caveats
Other research (reviewed by Gabelica et al., 2012) offers mixed results as to the effects of peer feedback on group functioning. Both versions of the CARE instrument measure a group’s health, but the effects of that feedback on group functioning remain unclear. Does knowing how well a group is functioning motivate members to change their behavior? Does up-front instruction on providing peer feedback increase its effectiveness? Does the feedback motivate groups to talk about group dynamics? Does it encourage them collectively to make changes? It makes sense that not discussing the feedback will minimize its effects. Moreover, some research verifies that group reflection activities do improve group performance (Gabelica et al., 2014).
Practical implications (what you might want to do about this research)
Here’s research that empirically developed an instrument (now available in both long and short forms) that students can use to assess group health. Even better, the instrument is available free of charge! Access it at ITPmetrics.com (it’s the Peer Feedback and Team Dynamics one), which the researchers describe as an “evidence-based, feedback report generating software system that is available without any permissions to anyone in the world” (O’Neill et al., 2020, p. 1131). In other words, teachers can direct teams to the site, where, after registering, they fill out the questionnaire and subsequently receive a report of the results. The well-organized site also offers a variety of supporting materials for teachers and students. It’s a first-rate resource that saves teachers time and develops students’ understandings of what group dynamics entail.
Related research
Gabelica, C., Van den Bossche, P., de Maeyer, S., & Segers, M. (2014). The effect of team feedback and guided reflexivity on team performance change. Learning and Instruction, 34, 86–96. https://doi.org/10.1016/j.learninstruc.2014.09.001
Gabelica, C., Van den Bossche, P., Segers, M., & Gijselaers, W. (2012). Feedback, a powerful lever in teams: A review. Educational Research Review, 7(2), 123–144. https://doi.org/10.1016/j.edurev.2011.11.003
O’Neill, T. A., Deacon A., Gibbard, K., Larson, N., Hoffart, G., Smith, J., & Donia, B. L. M. (2018). Team dynamics feedback for post-secondary student learning teams. Assessment & Evaluation in Higher Education, 43(4), 571–585. https://doi.org/10.1080/02602938.2017.1380161
One Response
I have students complete peer evaluations in some of my courses and the workload for me to compile all the info is a lot sometimes. Therefore, I was very excited to see the survey on ITPmetrics.com. It looks amazing and very useful for the instructor as well as the student. However, upon signing up to use the site, I read over their privacy information and was surprised how much information they collect when you are using their site. I don’t feel comfortable now having my students use this site. How disappointing 🙁