Many instructors feel that they need to be experts in mathematics in order to understand analytics. But according to John Vivolo, director of online and virtual learning for New York University, every faculty member can learn to use the course analytics available through their LMS to improve student learning.
Vivolo’s aim is to help faculty “use analytics to proactively reach out to students.” Vivolo talks about what he calls “pocket data analytics.” These are small, easy-to-use pieces of data that are readily available to instructors through their LMS.
Pocket data analytics are a way to leverage the data that is collected, often automatically, by looking at smaller bits of data that show discrete happenings and student behaviors in a class. This allows instructors, deans, and instructional designers to move beyond simple surveys and student grades as metrics into more information that is easily understood and responded to.
In a paper for the International Conference on Analytics Driven Solutions in 2014, Vivolo explains the concept this way: “Rather than looking at large scale data, the purpose of this method is to get instructors to focus on smaller patterns within a single course, during a specific time period, such as a week. The intent is to have a method in which to introduce the concept of academic course analytics as a practical tool….”
Vivolo highlights three types of analytics that are readily-available and easily-used:
Time-based
Time based measures are probably the type of analytic that faculty members are most familiar with. For example, data may show that students log into the class more often on weekends or more often late at night or at lunchtime. The faculty member may then find ways to tailor her own schedule to the needs of the students and the goals of the course.
For instance, a faculty member may decide that it would be best to encourage students to log in at times other than the weekend, and so she may decide to release assignments and discussion board comments mid-week. Or a faculty member may realize that her class is made up of many working professionals and may elect to give the majority of the information on the weekends when more students are logging in.
In a paper on the subject, Vivolo says, “When looking at these numbers, an instructor should consider the demographics of their students. One cannot ignore the variety of student populations when comparing online to on-campus learning. Are they working students who have less time to access the course during the week and must access it during the weekends? Are they more traditional students who can access material during the day (assuming they do not have a full-time job)? Are they international [students], national students, or local students?”
Individual assignments/content
Another helpful pocket analytic is the individual item or assignment analytic, such as how often students view a particular course element, such as a video. For example, a single student may view a video many more times compared to the rest of the class. This may indicate that the instructor should intervene in some way, perhaps by asking if there are any questions about the content.
It’s important, however, that instructors not use the pocket analytics in a way that makes students feel uncomfortable. In the above example, asking the individual student why he or she viewed the video so many times would probably be inappropriate. “I won’t recommend doing things with a Big Brother mentality,” Vivolo says. Instead, data points like this might cue the instructor to send a general invitation for any student with questions so that the answer may be discussed. Additionally, data that suggests that more than one student is viewing a particular piece of course information repeatedly may be an indication that the information is difficult or unclear. In this case, the faculty member “can host a review or re-record the lecture,” Vivolo says. He also advises that instructors refrain from moving on in a course until they are convinced their students understand the material.
Discussion boards
Discussion boards are another common tool that generates usable data. “Discussion board forums are the most commonly-used interactive tool and have been around since online learning started. Its purpose is usually to simulate an in-class discussion, but in an asynchronous method. But how do instructors impress upon students the importance of contributing to the discussion without forcing them to participate?” Vivolo asks.
Once again, data such as log-in times and frequencies are readily available to the instructor. The challenge is in how to use such data. One way is to use log-in and posting data to keep track of attendance. Vivolo cautions against this practice. He quotes an academic article that suggests that activity is not a proxy for monitoring attendance, because students can refrain from posting in a discussion forum and still be reading and thinking about the material.
Vivolo lists techniques to improve discussion board activity, including:
- “Post follow-up questions (randomly) when the replies are the lowest. The students may be more likely to answer follow-up questions from the professor at random times during the week.”
- “Create a Discussion Board Interaction Policy, requiring students to post before Wednesday and then another time before another day.”
- “Connect [the] discussion board to a major part of the course topics, and thus to a significant part of the student grade.”
- “Make the discussion board topics extend over more than one week to allow for additional topics.”
Overall, pocket data analytics allow instructors to respond preemptively to student problems and issues before they impact student learning and performance. This is opposed to more reactive approaches in which instructors make course changes only at the end of the course when they see final student results. By using pocket data analytics, instructors can respond in real time to the problems that students have.
On Tuesday, March 24, 2015, John Vivolo, director of online and virtual learning for New York University, will present a 40-minute online seminar titled “Using Student Analytics for Online Course Improvement.”
For further information, go to: www.magnapubs.com/online-seminars/using-student-analytics-for-online-course-improvement-13352-1.html.
