Successes and Challenges with Implementing Personalized Adaptive Learning

Personalized adaptive learning (PAL) is a software platform approach that provides each student with an individualized learning experience by allowing them to progress along their unique learning path through the course content based on their knowledge, skills, and learning needs. Adaptive learning systems customize the presentation of the content or present new concepts to the student based on their individual activities and responses.

The PAL team at University of Central Florida (UCF) has leveraged adaptive learning to improve student success since 2014 (Bastedo, 2016). The team’s work has resulted in 56 PAL courses designed and developed with faculty from such disciplines as anthropology, biology, math, physics, statistics, foreign languages, and hospitality. Approximately 600 PAL course sections have been delivered over the years, affecting over 35,000 graduate and undergraduate students across online, blended, and face-to-face modalities. This breadth of implementations has allowed the PAL team to learn several lessons that could benefit others in similar adaptive learning design.

The Realizeit software system has been the primary platform for the design and delivery of PAL courses at UCF as Realizeit provides students with individualized learning paths that tailor learning content and assessment to students’ prior knowledge and current performance. This learning content is typically presented to students in small chunks and followed by practice exercises or formative assessment questions or both. Upon completing the learning pathways, the system encourages students to review and practice to improve their mastery of a topic before moving to subsequent lessons. Although instructors can integrate PAL across an entire course, most of them use it for only a portion of their online course activities, compiling the remainder of the course materials and assessments in Canvas.

In terms of student perceptions of PAL experiences, students have responded positively to the real-time feedback and increased opportunities to test their knowledge and prepare for exams. The data analytics within the PAL system have also revealed specific areas where students might be struggling, allowing instructors to support student achievement (Cavanagh et al., 2020).

Time investment

While the opportunity to accelerate through PAL content and assessment might initially sound appealing to students, they quite often spend more time engaging with the learning experience in a PAL system than they do in a non-PAL system. The reason is that PAL systems encourage students to review content and assessments until they achieve mastery, whereas non-PAL courses often record a grade only once and allow students to move forward regardless of their performance. So, while it is true that some students may be able to move through the PAL material at an accelerated pace, many others will spend more time learning the material due to system-recommended repetition of key concepts. Such repetitions may come as a surprise to students who have never used a PAL system.

Instructors may be surprised by the exorbitant amount of time—not to mention content, design, and technical expertise (Cavanagh et al., 2020; Chen et al., 2017)—required to build a PAL course from scratch. While instructors typically possess adequate subject matter expertise, which provides a solid base to get started, it requires considerable effort to reimagine how concepts might flow nonlinearly and determine the narrowest scope within which each concept can be delivered and assessed independently within a more holistic content framework. Doing so comes naturally for some instructors but can prove challenging for others, especially those who have relied on textbooks that present material in a fixed, sequential order. Creating a successful PAL experience also requires having a firm grip on the implications of system features and configurations, which are critical to student and instructor satisfaction.

Faculty training

To address any potential challenges with PAL implementations, the PAL team created a professional development program titled PAL6000. This course focuses on PAL course design and delivery as well as technical and logistical matters particular to Realizeit. Before starting PAL6000, the PAL team instructional designers (IDs) consult with interested faculty to manage expectations around the PAL course development process. This initial meeting is useful to elicit buy-in from the faculty and get them excited about the effect PAL can have on student learning and engagement. Faculty members can (and often do) start small by redesigning just a few modules and continue to add to their course each semester. Several PAL courses at UCF started with this process and the instructors and designers are still revising the course content years later to continue to improve the student experience.

Results

Between 2019 and 2021, UCF selected faculty to participate in a course redesign initiative, and most redesigned PAL courses showed an increase in student success compared to the same class without PAL (Table 1). UCF measured student success by perceptions of student satisfaction, decreased withdrawal rates, and an increase in students who have achieved a final course grade of A, B, or C.

As one example, UCF has reported increased student achievement for a redesigned PAL courses in Biology for Majors (O’Sullivan et al., 2020). Before the redesign, 73 percent of students successfully passed the course; however, after implementing PAL, that percentage increased to 84 percent. Student satisfaction also increased after the redesign, and the course’s withdrawal rate dropped from 4 percent to 2 percent. Students indicated a consistent desire for additional practice problems, which highlighted student perception of value and engagement within the PAL course.

UCF has seen additional success in a PAL redesign of a two-course Spanish sequence. The pass rate increased by 23 percent in Spanish I, and the withdrawal rate decreased from 10 percent to 3 percent. In Spanish II, the pass rate increased by 22 percent, while the withdrawal rate decreased by 13 percent. Student perception, measured in end-of-course evaluations on a scale of 1–5, increased from 4.00 to 4.46. These positive results have led to conversations encouraging a program-wide redesign of Spanish language course sections and provided a model for implementing PAL instruction in the areas of Italian, German, French, and Portuguese.

While implementing an adaptive learning program requires considerable effect, the results show improved student achievement, making the program a worthwhile investment.

References

Bastedo, K. (2016). Getting started with adaptive learning. Online Classroom, 16(3), 4, 7. https://qa.teachingprofessor.com/topics/online-teaching-and-learning/course-design-online-learning/getting-started-with-adaptive-learning

Cavanagh, T., Chen, B., Lahcen, R. A., & Paradiso, J. (2020). Constructing a design framework and pedagogical approach for adaptive learning in higher education: A practitioner’s perspective. The International Review of Research in Open and Distributed Learning, 21(1). https://doi.org/10.19173/irrodl.v21i1.4557

Chen, B., Bastedo, K., Kirkley, D., Stull, C., & Tojo, J. (2017). Designing personalized adaptive learning courses at the University of Central Florida. ELI (EDUCAUSE Learning Initiative) Brief. https://library.educause.edu/resources/2017/8/designing-personalized-adaptive-learning-courses-at-the-university-of-central-florida

O’Sullivan, P., Voegele, J., Buchan, T., Dottin, R., Goin Kono, K., Hamideh, M., Howard, W. S., Todd, J., Tyson, L., Kruse, S., de Gruyter, J., & Berg, K. (2020). Adaptive courseware implementation: investigating alignment, course redesign, and the student experience. Current Issues in Emerging eLearning, 7(1), 101–137. https://scholarworks.umb.edu/ciee/vol7/iss1/6

Baiyun Chen, PhD, James R. Paradiso, MEd, Joseph Lloyd, MEd, and Rebecca McNulty, PhD, are instructional designers at the University of Central Florida.