Journey to Adaptive Learning at The American Women's College

As the name implies, adaptive learning systems tailor educational content to each learner's specific needs. They do this using various computer models that determine a student's knowledge level and strengths and weaknesses, and present the subject matter accordingly, varying the difficulty level, mode of presentation, and so on. Adaptive learning also reimagines students as active participants in what and when they learn within a given course. This empowerment is particularly important in large online courses, where faculty lack the physical cues that help identify struggling students in traditional classrooms and the time and information necessary to conduct personalized interventions.

In 2013, The American Women's College (TAWC) at Bay Path University was founded with a mission to expand access to higher education to the 76 million adult women in the United States who do not have a college degree. A core group of college administrators recognized that improving performance, retention, and graduation rates for these nontraditional learners would require nontraditional practices. From this was born the concept of Social Online Universal Learning (SOUL), an integrated platform of wraparound and personalized supports driven by learning analytics. Comprising SOUL are elements such as virtual communities, wraparound support services, data-informed advising interventions, and the use of predictive and learning analytics to proactively and appropriately respond to each student's needs. An adaptive learning platform, with its ability to personalize learning experiences and content in ways that traditional learning management systems (LMSs) cannot, was viewed as a critical element alongside the other components of SOUL.

Having decided to make a foray into adaptive learning, TAWC evaluated several potential solutions against required and desired features. The college selected Realizeit as its partner in 2015 and started with two pilot projects centered on high enrollment general education courses. Applying lessons learned from the pilot, the number of adaptive courses implemented increased from six courses in 2016 to nearly thirty courses in 2017. Major milestones along the way included creating institutional definitions for the roles of subject matter experts and course builders, designing a collection of onboarding materials and training courses, and establishing partnerships with major providers of openly licensed courseware. The final point is vital to the success of each project, as they require substantial volumes of content and alternative representations (text, video, audio, etc.) of component concepts to feed the adaptive functions of the system.

Why Adaptive Learning?

In the suite of support services TAWC provides to students, SOUL, the wraparound supports were designed to accelerate and promote degree completion for adult learners, particularly under-represented women. SOUL includes key features that research suggests are characteristics of effective learning environments for adult students: engagement with personalized learning; supportive mentoring relationships for students with peers, coaches, and instructors; immediate, constructive feedback on performance; and clearly delineated pathways to graduation.

One feature of the SOUL suite includes exposing students to adaptive technology in select courses. The adaptive platform, termed KnowledgePath, uses learning analytics to customize the student's learning environment and to provide targeted, individualized feedback based on her academic performance. Among the various definitions of adaptive learning, we like to use a simile with our stakeholders:

Adaptive learning is like a large brain that processes volumes of assessment and interaction data to figure out how to tailor and personalize the learning experience, as well as how to empower our faculty members to structure their teaching and support to better suit students' individual needs.

At this time, TAWC has 21 courses, predominantly in the Business and Psychology programs, developed and run within KnowledgePath. Our goal is to use it in all the core classes in these programs so that students experience adaptive learning throughout their course of study. Building adaptive assignments in a number of general education requirements also has expanded the impact to a much larger subset of students.

Big Data, Big Value

As we quickly realized, an adaptive learning platform generates a massive amount of data, and the value it provides is limited only by the questions that faculty, students, and staff members think to ask.

For example, instructors can access answers to questions such as: Which student is struggling with which concept? Which concepts are most difficult for the class, in general? Where can I intervene with targeted feedback or resources to achieve the biggest return on student learning?

Students can find answers to questions such as: Where am I struggling? and Where can I find remedial help for this concept? Empowering students to use the data to find answers helps them succeed and also offers opportunities for metacognition — that is, for students to understand how they are learning.

The course-building team and program directors can determine: Which version of content is most effective in teaching a concept? Which concepts are taking longer than expected to learn? Which assessment questions that we're asking drive the best possible learning?

Virtually the only limit to maximizing the impact of adaptive learning data is in the creative capacity of stakeholders to articulate questions.

Customized Learning

The adaptive learning system automatically collects data on student activity — clicks, time on task — and on student performance on the periodic assessments that take place after each major concept in the learning path. Data is turned into actionable information for the student and instructor in the form of streamlined dashboards. The system also uses the vast stores of data to create a learner profile for each student. Profiles can be used to ascertain which types of content — text, video, etc. — the student prefers and which drives the best learning outcomes for particular types of learning tasks. For example, a student might learn well in a history course when she encounters excerpts of an open text, but she might struggle when presented with the same type of resource while trying to learn math concepts. The system would automatically intervene to suggest to the student that she watch a video tutorial, instead. Her success enables the adaptive system to personalize future learning by triangulating information about the student, the concept at hand, and the content available.

This data access gives the system information similar to that possessed by a long-term tutor in a way that is scalable to thousands of students. And, because that information is readily available, faculty members can quickly get up to speed on any student as needed. Ethical uses of data are important in this context. TAWC provides information in online orientation and reminders in each adaptive course to inform students of what types of data are collected and how they are used to customize their learning experience. When sharing data beyond the course, such as when constructing aggregate learner profiles or feeding predictive models, the data are anonymized.

Adaptive Learning in Action: The SOUL Project

The SOUL Project was launched in 2015 to give women without a college degree — many of whom both work and have families — a flexible, supportive learning opportunity and experience. Using adaptive learning offers this flexibility and support, helping SOUL learners work at their own pace, own the experience, and increase their chances of academic success.

So far, the program is achieving remarkable results. In our first year of tracking entering students exposed to our SOUL model for re-enrollment in fall semester of our second year, we experienced a return rate that is roughly 10 percent higher than the comparable national average for retention calculated by the National Clearinghouse's most current Snapshot Report: "Persistence" (2016). We assert that these outcomes result from the blending of adaptive learning with the new roles that we have engendered for faculty and educator coaches (Bay Path University's equivalent of an academic advisor) through access to adaptive learning analytics.

The SOUL program's key features include the following:

• SOUL Connect. This on-boarding program helps orient students, who can strategize with an advisor to prepare for their academic journey.
• Responsive support. Each student is partnered with a dedicated Educator Coach. If students begin to struggle, SOUL's learning analytics and predictive models automatically alert the coaches through dashboards to facilitate immediate intervention and additional critical support. The intervention generally includes a discussion via telephone or e-mail to identify the reason for the student's academic hardships, followed by a review of all services available to the student that may assist them in improving their performance.
• Customized learning path. The SOUL KnowledgePath customizes instructions to student needs using a variety of modalities, including text, videos, audio clips, and interactive exercises.
• Community. SOUL provides students with virtual learning communities (VLCs) in the LMS to engage and network with other students who share their goals and professional interests. Subject matter experts including the degree program director and faculty members from their major facilitate the VLC, and students can learn about professional associations, read articles on hot topics in their chosen field, join practice-based discussions, and explore relevant job postings.
• SOUL Connect for Educators. Through this program, our faculty learn how to use analytics to inform both their instruction and support.

The keys to SOUL success are wraparound support and self-pacing; the former ensures that learners have what they need, when they need it, while the latter lets them control the pace and earn their degree quickly or at a more leisurely pace when demands increase in other areas of their lives.

The wraparound support services offered to students through the SOUL model allow the educator coaches to be proactive with customized outreach for each student. The learning and predictive analytics from the adaptive platform inform much of the outreach initiated. In a typical month, approximately 15 percent of students in adaptive courses will receive an advising case to generate a touchpoint. Educator coaches can connect these students with services such as technology support, online tutoring through Tutor.com, and mental health services. The range of support required by students varies greatly, but having the ability to use adaptive learning analytics empowers educator coaches to intervene early and often to increase the likelihood of student success.

Another aspect of the SOUL model includes an accelerated degree path. Students complete one or two courses every six-week session. KnowledgePath assists by enabling students to demonstrate mastery of concepts before learning begins so that they can proceed to new learning more rapidly. A health science student, for example, shared an experience in which she cut down her time reviewing materials by almost half. This enabled her to expend more of her effort in applying her knowledge in formative assessments. More generally, a survey of all students enrolled in adaptive courses in fall 2016 revealed that 77 percent of respondents felt that the pacing of adaptive learning was just right for their needs, and 78 percent agreed that their learning improved as a result of having used the adaptive system.

Advice and Considerations

In our experience, the two main considerations are the required investment of resources and the disruption entailed in launching adaptive learning.

Plan to Invest

The adaptive learning enterprise is incredibly resource intensive; you must consider the total cost of ownership, starting with licensing fees for the system. Here you will face a major decision point: will your institution foot the bill, or will students fund the system through course or technology fees? Regardless of the approach you choose for building an adaptive course — curating open resources or licensing proprietary resources — there also is a substantial cost for securing the broad and deep content you'll load into the system.

Adaptive learning is also disruptive of the pervading model of higher education. Course builders, subject matter experts, faculty, and students all require development and training to understand their redefined roles in this new world. For example, faculty must be supported as they pivot from the model of sage-on-the-stage to data-informed guide-on-the-side. This change requires not only embracing a new role, but also demonstrating a new set of competencies around navigating and using data sets and dashboards. Designing a training course to address these complexities can take months, while each faculty member can end up spending dozens of hours mastering the desired outcomes.

Plan for Disruption

Our biggest lessons learned from this process has been the need to plan intentionally for the disruption. Adaptive learning requires a full embrace of personalization and student centricity, a shift that necessarily redefines student, faculty, and staff roles. Communications with all stakeholders — administration, faculty, and students — about how this disruption will affect them, how learning will change, and how teaching will change is the first step to easing tectonic shifts. Information-rich communication channels — training courses, onboarding discussions, faculty meetings, etc. — are your best bet given the complexity of developing a shared lexicon around new approaches.

Another consideration to keep in mind is that implementing an adaptive platform, while complex, does not necessarily require the active input of all institutional departments. Providing these more disparate stakeholders with occasional updates can help ease the apprehension and answer the various questions that come with learning innovation. At TAWC, for example, we had to tackle conversations about allocating resources for adaptive learning within the larger framework of our strategic plan. An approach that worked well was pairing high-level overviews of the benefits of adaptive learning at "town hall" meetings of staff and faculty, and taking deeper dives in "lunch and learn" sessions. The former were structured as an hour when academics and technologists shared a lightning round of insights and then opened the floor for questions and discussion. The dialog that resulted helped to increase buy-in across the board.

Coming Next

A looming challenge at TAWC is figuring out how to develop more upper level adaptive courses. The open educational resources (OER) that form the backbone of many of the college's courses generally is lacking in advanced courses. Proprietary texts are not a viable option, as publishers largely have resisted or slowed the path to disaggregating and remixing their content. Possible solutions range from increasing self-publishing endeavors to teaming with other institutions to co-develop courses or partnering with OER purveyors to share the burden in building content that has a smaller market footprint. The order of the day will be creativity, flexibility, and resources.

TAWC is also looking to bring more programs into the adaptive platform, which means training more program directors and subject matter experts. This will test our ability to communicate processes, policies, and procedures in effective and replicable ways. The complexity of mapping learning also will increase as a result of more opportunities to develop cross-curricular connections. For example, students might benefit if modules on the writing process are bridged from introductory English courses into more advanced modules in major-specific research courses. More intentional collaboration, perhaps through retreats, should help to harness and direct this opportunity. Expanding the initiative to adopt adaptive course design in these ways will enable TAWC to continue serving the "new normal" student with the personalized supports she needs to persist and graduate.

---

Jeremy Anderson is deputy chief of Academic Technology, The American Women's College of Bay Path University.

Heather Bushey is director of SOUL and FIPSE project manager, The American Women's College of Bay Path University.