Adaptive Learning: Implementation, Scaling, and Lessons Learned

When faculty, academic, and executive leadership at Colorado Technical University (CTU) made the commitment in 2012 to develop an adaptive learning strategy for teaching and learning as a part of CTU's long-range academic programming plan, the goal was for adaptive learning technology to support a primarily nontraditional student body—the majority of whom are over the age of 25—with a vast array of professional and academic backgrounds. Because CTU is an open-enrollment institution and includes "education supported by technology" in its mission statement, supporting students and faculty using adaptive learning technology that personalizes each student's learning path was a natural fit. When we wrote about our implementation strategy for EDUCAUSE Review in 2016,¹ CTU was well on the way to successfully implementing adaptive learning courseware at scale in a number of programs and in courses from undergraduate to master's degrees.

Table 1 provides data about the number of courses, faculty, and students who have participated in courses that include adaptive learning components (branded at CTU as "intellipath"). Courses are primarily delivered online, and the adaptive content is included as part of the total course content. Courses including adaptive technology are offered in 5.5-week and 11-week formats, both at the undergraduate and graduate levels; the largest number of students using adaptive learning is in general education courses.

Table 1. Adaptive learning overview: 2012–2019

Nov. 2012–Sept. 2019	Total
Total Courses Since Inception	218
Total Unique Students Since Inception	129,287

Approximately 600 CTU faculty have engaged with adaptive technology in the classroom, and these instructors are required to successfully complete asynchronous training prior to teaching a course with adaptive learning. CTU has been able to harness adaptive learning content in hundreds of sections at a time for thousands of students per quarter (as shown in table 2) through the support of academic and faculty leadership, the Teaching and Learning Team, and our curriculum department.

For the most part, faculty members' comments about their experience with the adaptive learning technology are positive. We believe that this is due to a very intentional focus on and engagement with the hundreds of faculty who are engaging with adaptive technology in their courses.²

Table 2. CTU adaptive learning courses, Summer 2019

July–September 2019	5.5 week Session A	5.5 week Session B	Total
Total Courses	152
Total Sections	817	840	1,637
Total Faculty	580
Total Unique Students	24,563

What Did We Learn?

As we have moved along this journey of developing and implementing adaptive learning, we have learned many important lessons, including those outlined below. A central part of this effort has been to solicit feedback from students, and selected quotes from students are included here—along with some quotes from faculty—to provide a window into their perspectives on these adaptive technologies.

• Adult Learners Enjoy Technology: In formal surveys and through engagement with their instructors, a majority of CTU students report that they enjoy adaptive learning classroom content, which they find conducive to learning course content and demonstrating content mastery. If there is a writing assignment and an adaptive learning assignment, students generally will prefer the adaptive learning assignment. However, as students take more adaptive learning classes, comments also include suggestions for improvement.

  "Maybe add more work in intellipath. The technology use is outstanding. I was able to access every single thing from my phone and my computer; therefore I never missed an assignment."

  "Integrate discussion boards. Students have different life experiences and viewpoints on a wide range of topics that can be beneficial for learning. Just intellipath week after week doesn't help facilitate any meaningful learning."

  "Intellipath and discussion boards offer different value to the learning process. Please don't remove one in a given week because the other is on the agenda. They provide different perspectives and learning opportunities and are not a substitution for each other."

While students generally express positive reactions to the adaptive learning content, equally important feedback has highlighted the desire to engage in additional content to supplement adaptive learning. In particular, students have communicated a desire to engage in discussions, case studies, and video content.

• Students want more engagement: Conjoining adaptive learning content to the other instructional areas of the classroom often translates into higher student engagement, with greater interest in topics and increased enjoyment when completing course tasks.

Critical to student engagement is deliberate integration of the adaptive learning content with the rest of the course areas, including the assigned readings and activities such as discussion boards or written assignments. Assigned readings should complement and align with adaptive learning content, and assignments should build on and provide opportunities to practice skills and demonstrate content mastery gained within the adaptive learning environment. Students might engage more readily in discussions and other activities after developing greater comfort with the content through adaptive learning lessons.

"Thank you for your advice. I just posted two responses on discussion board after complet[ing] my map on intellipath. I will try to improve my score on one topic when I get a chance."

"The directions and assignments given were a well-planned-out strategy for success with great efficiency. There were many useful and, when needed, very helpful resources given throughout the course. …the intellipath learning system encouraged and made readily available the … discourse with our instructor."

• Course design matters! We have revised our course design to include other digital tools to support adaptive learning. As students master content, feedback from students indicate that other media, including videos and case studies, would provide welcome supplement to the content addressed in adaptive learning lessons.

Student feedback strongly supports additional material to supplement adaptive content. Additionally, faculty supported the integration of additional course media to supplement the adaptive learning content. After we made the requested edits in a number of courses, course completion and D/F/W rates improved. To increase engagement, several courses piloted the use of avatars to provide relevance to the course content (see the sidebar "CTU Avatars"), Articulate/Storyline 360 to increase presentation of course material using infographics, and other tools to enhance course design.

Also, CTU implemented a cross-functional team to review courses, including adaptive learning maps, which represent the connection of individual lessons in each week's adaptive learning assignment. As noted by Kilgore, Prusko, and Gogia, institutions must discuss how to "better facilitate flexible, cross-functional, interdisciplinary teams for improved leadership, research, and course development."³ Our new course-review process has assisted faculty subject-matter experts and instructional designers in creating a more streamlined process for developing dynamic and engaging adaptive learning content. Participants in this process identified needed improvements that, when implemented, led to positive student feedback and, in many cases, improved student outcomes.

• Instructor presence is important! Adaptive courseware is not "plug and play." CTU has developed instructional models and approaches specific to effective instruction in adaptive learning environments.⁴ When faculty engage and provide guidance to students in the courseware, students report a more positive experience in the course.

We recently conducted a multiple regression analysis on Academic and Career Success—which is the first course that students take at CTU—to understand which areas of instructional behavior have the greatest impact on student success in the classroom. This course primarily uses adaptive learning technology for course content delivery; we looked at Quarters 1 and 2 of 2019. The analysis found that the faculty behavior most strongly correlated with student success in this course was instructor presence within the classroom. The better the faculty member scored on presence within the classroom during Faculty Observations, the more likely the students were to successfully pass the course. ("Instructor presence" here is defined as timely, regular, and routine involvement in key instructional areas of the classroom.) While we acknowledge that this finding is indicative of correlation as opposed to causation, the analysis supported the student narrative. Students generally express appreciation to faculty who are regularly engaged in the course technology, especially with targeted guidance on course content or through providing encouragement on student progress.

"Thank you so much. I really do appreciate the messages from you to let me know how I am doing and where I stand from your point of view in class. I will continuously work on my assignments until the last day. I hope you teach other classes because I would love to have you as another instructor for future classes."

"Thank you for reaching out. I am going to get started on my next lesson as soon as possible. If I have any questions or run into any obstacles I will be sure to turn to you for advice or any help that you may be able to provide. Again, thank you. You helped me understand how to navigate and complete my assignments."

CTU has long emphasized the importance of faculty members' engagement with the experience of digital tools. In the words of faculty who teach science courses that use adaptive digital tools:

"The technology platform used in SCI203 including intellipath and adaptive learning are well developed. Intellipath tracks student progress and helps the instructor adjust the instruction to train students more effectively. Adaptive learning technology helps the professor to facilitate evidence-based instruction and discovery-based learning."

"Technology-wise, the course is well done—intellipath continues to be a great resource and interactive tool for students."

Equally important are faculty members' suggestions for improvement with adaptive content:

"Review the videos and extra reading within intellipath. I am not sure they are all necessary."

"I think the course needs a bit of a revamp; some of the intellipath sections are too long. There is a lot of information in this course for an introductory-level course."

In CTU's experience, faculty engagement increases when their feedback is acknowledged and addressed. Survey questions requesting suggestions for improvement are administered to faculty after every course, and this feedback is reviewed by the college or department for follow up. In our experience, if faculty perceive that feedback is not being reviewed or acted upon, they become discouraged and might stop providing feedback for course improvements as well as course successes.

• Mobile access to adaptive learning increases student use. CTU's mobile application, highlighted in an EDUCAUSE Review article,⁵ provides a student access to adaptive technology to engage in course content on demand (see figure 1).

Currently, approximately 57% of students use the mobile application to access adaptive technology. Students indicate the importance of engaging with course content through CTU's mobile application:

"Keep making those wonderful upgrades. I enjoy the option of doing my intellipath from my mobile app on my phone, while I'm eating my lunch, in my car, at work…"

"For people that have busy, active schedules, intellipath is the best tool for them. Especially, when you are able to use the mobile app."

Although not all assignments can be effectively completed via CTU's mobile application, the addition of adaptive content to the mobile application has mattered to students and made a significant impact in the pursuit of mobilizing our classrooms.

Conclusion

Ultimately, integrating adaptive learning content into our programs at CTU has created what we believe is a more easily accessed learning environment for our students. The implementation of adaptive digital tools has also provided CTU the opportunities to collaborate in research with other institutions, including the University of Central Florida.⁶ However, adaptive learning is not a one-size-fits-all solution, and it is most effective when combined with strong faculty engagement, quality course design, and alignment to other instructional tools. And let's not forget that instructional design, including the use of engaging content and avatars to involve students in the course content, can influence students' experience with adaptive course content.

Notes

1. Constance Johnson, "Adaptive Learning Platforms: Creating a Path for Success," EDUCAUSE Review, March 7, 2016. ↩
2. Constance Johnson and Emma Zone, "Achieving a Scaled Implementation of Adaptive Learning through Faculty Engagement: A Case Study," Current Issues in Emerging eLearning 5, issue 1 (2018). ↩
3. Whitney Kilgore, Patrice Prusko, and Laura Gogia, "A Snapshot of Instructional Design: Talking Points for a Field in Transition," EDUCAUSE Review, August 8, 2019. ↩
4. Amy Sloan and Lindsey Anderson, "Adaptive Learning Unplugged: Why Instructors Matter More than Ever," EDUCAUSE Review, June 18, 2018. ↩
5. Connie Johnson, "Student-Driven Mobile App Design: A Case Study," EDUCAUSE Review, September 19, 2016. ↩
6. Charles Dziuban, Colm Howlin, Connie Johnson, and Patsy Moskal, "An Adaptive Learning Partnership," EDUCAUSE Review, December 18, 2017. ↩

---

Connie Johnson is Provost and CAO at Colorado Technical University.

Amy Sloan is Director of Academic Operations at Colorado Technical University.