Adaptive Courseware at Portland State University: A Question of Scale

Adaptive courseware delivers content to the student, assesses the student's understanding of that content, and adjusts which content to present next based on that level of understanding. Analytics from the student's engagement with the courseware can help instructors modify instruction based on what individual students have learned and what they still need to learn. As with any innovation, moving to scale produces deep and consequential changes in practice. It requires evaluation and research to understand and enhance the causes of effectiveness, enabling faculty to take ownership and learn from their course modifications and to maintain these changes over time.¹

In this article we share how the Office of Academic Innovation (OAI) at Portland State University (PSU) has found different ways to support faculty with their first adoption of adaptive courseware and how OAI learned about the factors that influence the spread and sustainability of its use on campus. This work takes place in the context of an Association of Public and Land-grant Universities (APLU) grant, "Accelerating Adoption of Adaptive Courseware at Public Research Universities," with funding from the Bill & Melinda Gates Foundation. (See "Active and Adaptive" for more information about PSU's work for the adaptive courseware grant.)

Getting Started

In fall 2016 we targeted high-enrollment, gateway courses and identified faculty in business, math, and chemistry who were already using adaptive courseware, which included off-the-shelf products such as McGraw-Hill's ALEKS and LearnSmart.² The objective at this stage was strengthening faculty use of that courseware. We were focused less on course design and more on course coordination and faculty training required to scale the adaptive courseware across multiple sections of a course.³ We invited other faculty in these departments to attend lunch-and-learn events where they could discover firsthand how their colleagues were implementing adaptive technology in their respective courses and hear about the results they found.

During this process we quickly learned that faculty who expressed interest in trying adaptive courseware had numerous questions. They were concerned that an off-the-shelf (publisher) product would limit their flexibility to modify instruction and that student cost for course materials would remain high after implementation. They also showed an interest in the algorithms behind the adaptive products and wondered whether the algorithms could be adjusted.⁴ Rather than using a publisher's adaptive courseware as a textbook replacement or supplemental homework system, they were aiming for a redesign of their entire course, which would blend online adaptive modules using open course content with active learning in the classroom.

Adjusting Our Approach

To accommodate this new use case of completely redesigning courses, we used the Courseware in Context framework to identify content-agnostic vendors (those providing adaptive technology without published content) and spoke with other grantee institutions for advice, including the University of Mississippi.⁵ With faculty from biology, physics, and statistics, we vetted these vendors of adaptive technology, looking at initial development fees, cost for students, functionality of their systems, data usage policies, and ability to integrate with the PSU LMS (D2L).⁶ Based on these criteria, we narrowed the list to CogBooks and Realizeit. OAI negotiated agreements with each of these vendors and dedicated staff to support these STEM faculty in a full content development and redesign of their courses starting in fall 2017.

We also adjusted our support structure at OAI. Together with instructors, we helped redesign courses based on best practices in blended learning, which has been the focus of the APLU grant. An instructional designer (ID) was assigned to each faculty member to understand the course context and objectives. OAI also added a UX (user experience) specialist for technical assistance and to assess the student user interface. In the process, OAI developed important competencies to facilitate the building of whole courses within an adaptive technology platform such as CogBooks or Realizeit:

• Curation of open content such as text, problem sets, videos, and simulations from various sources, including OpenStax and PhET Interactive Simulations
• Ingestion and authoring of content, as well as building large problem sets within each adaptive platform
• Eliciting student feedback through surveys and focus groups to inform a preferred blend between the adaptive courseware and classroom activities
• Patience and persistence required to redesign a full course in an adaptive platform, which can take between six months and a year

We developed a collaborative relationship with our faculty partners to design and implement adaptive technology with the goal of improving student outcomes and learning experiences. The customization process allowed us to explore the effective combination of course design with adaptivity using student feedback to iterate and refine. Based on this work, OAI contributed its lessons learned and authored sections of an adaptive implementation guide published by APLU and Every Learner Everywhere in fall 2018.⁷ These lessons learned included the following:

• Faculty customizing the content within an adaptive technology platform need to include sufficient prerequisite or supplemental content within the system for it to provide effective, real-time support for students who struggle with specific problem sets.
• Unlike a traditional course textbook, which allows students to skip material, the adaptive courseware can require students to review content that the instructor identifies as essential to the course.
• The content and problem sets presented in the customized adaptive courseware need to align with in-class lectures, group activities, and high-stakes exams.
• Prior to delivering the customized adaptive courseware in the course, it is essential to conduct user-experience testing from the student's perspective.

While we deepened the work with the faculty who participated in the early stages of this work, we struggled to engage other PSU faculty who were teaching gateway courses. Through discussions and outreach to these faculty, we gained important insights into their reluctance to participate in the initiative:

• Their time is limited. Redesigning a course from the ground up may be the right thing to do, but it is intensive and potentially risky.
• Most faculty are cautious about the possibility for overreach of the adaptive solution within their existing course; as with faculty in the earlier groups, these instructors seek flexibility in the application.
• For some faculty, the textbook used in their courses was selected by their department, and this can impede faculty's ability to redesign an entire course or shift entirely to an open resource textbook.
• They do not want to add to the current course material cost for students when testing a new adaptive solution.
• Many question the efficacy of the adaptive solution used in a discipline different from their own to solve an existing challenge in their course.

Refining Our Support Model

Using this feedback and recognizing OAI's capability to build within the two adaptive technology platforms, OAI constructed another support model, this one for creating course lessons in the adaptive courseware. The objective was to remove the barriers and create an experimental space for faculty that would minimally disrupt the current course and textbook while leveraging faculty members' ability to be flexible with the design and to reduce cost.

Rather than introduce adaptive as a complete replacement of existing practices or as an add-on to an entire textbook, we asked faculty to identify an existing pain point in their course. Was the students' transition to their course particularly difficult? Did a particular lesson or foundational concept hamper students' ability to be successful in the course? In essence, we were looking for an adaptive solution large enough to be impactful but small enough to be implemented within a short time frame. We added a stipend from the grant to compensate faculty for their time, and we negotiated with each vendor a bulk purchase of student licenses to cover the cost for students while faculty tested the new adaptive lessons.

We began the faculty outreach for this new adaptive support model in spring 2019 with the intent to start projects in the summer term. For targeted gateway courses we were looking for what we called faculty "anchors," instructors who had been teaching the course for several years and were scheduled to teach again in the fall term;  many of these were nontenured, full-time instructors.

As OAI hoped, the number of faculty who were open to explore the new approach increased. With direct outreach to faculty, we initiated projects in new disciplines: anatomy and physiology, art history, computer science, freshman inquiry, organic chemistry, philosophy, psychology, Spanish, and writing.⁸ OAI dedicated one instructional designer to support these faculty for a few reasons. First, the ID had worked in both adaptive technology platforms and knew their respective capabilities and constraints.⁹ Second, the ID could help faculty curate existing open content from one of the online repositories or from the faculty's own material. Additionally, the ID could act as a conduit for design choices between the faculty and thereby cross-pollinate ideas and disseminate best practices.

An important lesson we had learned from our earlier full-course redesigns was that these prototypes will only be as effective as their integration into the course. To this end, the ID offers tips to help with student onboarding of the adaptive courseware and coaches the faculty about assigning points or tying the adaptive lessons to high-stakes exams. Not only is a smooth student experience crucial, but the students will need to see the relevance to be sufficiently motivated to complete the adaptive lessons in a timely manner. The ID also introduces the faculty to the use of student performance analytics from the adaptive courseware to inform student interventions as well as classroom discussions and activities.

Conclusion

For PSU, the adoption and spread of adaptive courseware takes place through a variety of use cases and strategies. OAI plays the role of guide in the process, yet we are allowing faculty to investigate the value of adaptive courseware for themselves. What do they observe in their courses, and how have the adaptive lessons been beneficial? Do they see enough impact to want to continue these adaptive lessons? Do they want to expand the adaptive solution to the entire course? Do the existing platforms meet their needs? Are they interested in the further development of open resources to replace their existing textbook? This process increases faculty buy-in and sets the stage for scaling sustainable solutions.¹⁰

For now, we are seeing the scaling of adaptive courseware beyond the total enrollments of students impacted.¹¹ We are scaling a community of faculty across many disciplines who are reexamining their gateway courses and actively looking to improve them with adaptive courseware as a potential solution. While some faculty see the benefits of adaptive in combination with existing off-the-shelf resources, others seek to deeply understand adaptive courseware and modify it to their liking. Still others will only adopt adaptive on a small scale in a manageable space where they can test its impact. Through an iterative approach, we developed an agile support model to meet the diverse needs of our faculty, resulting in a wider community of innovators focused on student achievement.

Notes

1. Cynthia E. Coburn, "Rethinking Scale: Moving Beyond Numbers to Deep and Lasting Change," Educational Researcher 32, no. 6 (August/September 2003): 3–12. ↩
2. Lou Pugliese, "Adaptive Learning Systems: Surviving the Storm," EDUCAUSE Review, October 17, 2016. ↩
3. Karen Vignare, "Why Adaptive Courseware Will Scale in Higher Ed," EDUCAUSE Review, September 18, 2017. ↩
4. Dale Johnson, "Opening the Black Box of Adaptivity," EDUCAUSE Review, June 15, 2017. ↩
5. Patricia O'Sullivan, "Helping Students with Adaptive Learning: APLU and the University of Mississippi," WCET Frontiers, January 1, 2017. ↩
6. Megan M. Tesene, "Adaptable Selectivity: A Case Study in Evaluating and Selecting Adaptive Learning Courseware at Georgia State University," Current Issues in Emerging eLearning 5, no. 1 (October 22, 2018). ↩
7. Vignare, K., Lammers Cole, E., Greenwood, J., Buchan, T., Tesene, M., De Gruyter, J., Carter, D., Luke, R., O'Sullivan, P., Berg, K., Johnson, D., and Kruse, S., "A Guide for Implementing Adaptive Courseware: From Planning through Scaling," Association of Public and Land-grant Universities and Every Learner Everywhere (2018). ↩
8. David Raths, "New Frontiers of Adaptive Learning," Campus Technology, April 24, 2019. ↩
9. Thomas Cavanagh, Baiyun Chen, Rachid Ait Maalem Lahcen, and James Paradiso, "Constructing a Design Framework and Pedagogical Approach for Adaptive Learning in Higher Education: A Practitioner's Perspective," International Review of Research in Open and Distributed Learning 21, no. 1 (January 2020): 173–97. ↩
10. Anna Drake Warshaw, "What Does Change Management Really Look Like in Higher Ed?" University Innovation Alliance, April 16, 2019. ↩
11. Coburn, "Rethinking Scale,"; Patrick Corrigan, "Exploring the Process of Scaling Up," illustration in Threshold, Spring 2007. ↩

---

Johannes De Gruyter is Executive Director at the Office of Academic Innovation, Portland State University.

Kevin Berg is APLU Grant Program Manager at the Office of Academic Innovation, Portland State University.