Adopting OER: A Case Study of Cross-Institutional Collaboration and Innovation

Nancy Pawlyshyn is special academic officer to the dean of the College of Professional Studies and full-time faculty in higher education leadership at Northeastern University, Boston, and an educational assessment and accreditation consultant. Dr. Braddlee is dean of Learning and Technology Resources, Northern Virginia Community College, Annandale. Linda Casper is assistant professor of Mathematics and Computer Information Science, and Howard Miller is professor of literacy education and chair of Secondary Education, both at Mercy College. They serve as Kaleidoscope Open Course Initiative Faculty Fellows.

In the winter of 2011, then-dean of academic technology Braddlee and associate provost Pawlyshyn at Mercy College met to review initiatives they had advanced collaboratively with faculty from the institution's Faculty Center for Teaching and Learning. While at a conference, the dean had learned about a potential opportunity to partner on an EDUCAUSE Next Generation Learning Challenges (NGLC) grant; the associate provost thought such a resource might help address attrition and failure rates in the Critical Inquiry freshman seminar course and College Mathematics. A focus on those courses aligned with current best practice that assessing outcomes in first year courses that are foundational to subsequent progress enables educators to address specific tasks toward improving success.¹

At Mercy, a minority-serving institution of 10,000 students, the Critical Inquiry seminar is a required first-year course focusing on three competencies: reading, critical thinking, and information literacy. Seminar faculty had formed a cohort faculty learning community, a faculty development model used effectively for other initiatives at Mercy² (see the video [49 seconds] where faculty discuss this), and met bi-weekly to address student engagement and achievement. However, in-house resources had not addressed the problem of low success rates in the seminar. After some coffee-fueled brainstorming that winter day, coauthors Braddlee and Pawlyshyn explored the option to engage in a national project with educators from seven other two- and four-year institutions with similar student populations. The Project Kaleidoscope Open Course Initiative (or KOCI; originally known as Project Kaleidoscope) would, they hoped, inspire faculty to revisit challenges they faced with first-year student achievement — challenges shared by many institutions. This was a logical step given that Mercy faculty were actively seeking a solution to an existing problem that they could not solve without additional support.

The initiators at Mercy sought to engage faculty colleagues in thinking creatively and working toward a larger purpose by creating and implementing open educational resources (OER) to improve student retention and learning. The phrase "free and better" became the cornerstone of project participants' vocabulary. This powerful idea served to motivate them through a somewhat messy and emergent process. Ultimately, however, their willingness to move through the morass of collaborative work across seven institutions on an impossibly tight timeframe resulted in turning the tide of student failure in these required courses.

The Implementation

The project implementation progressed quickly through several key steps, from conceptual development to implementation.

In May 2011, KOCI launched with funding from the Bill and Melinda Gates Foundation and the Hewlett Foundation through the NGLC program. Participants from the original seven partner institutions initially focused on developing new courses, but quickly realized that doing so would require months of negotiating with curriculum committees. At that point, there was a crucial shift from wholesale changes in course content to new approaches in instructional design.

Mercy faculty connected with cross-institutional faculty teams from the partner institutions in Cerritos, California, to identify common student learning outcomes; find effective, high-quality OER; and construct learning modules that would become embedded in a Sakai-based learning system with built-in analytics to track student progress. Learning materials adopted had to be free and openly available, either unlicensed or licensed through Creative Commons. Students would begin work in the course on day one, with textbooks and high-quality, openly available resources in hand.

In summer 2011, the KOCI Steering Committee, including Mercy's two project champions, held planning meetings with faculty teams in general education curriculum areas across the country using Skype and CCC Confer, the California Community College system's web conferencing system.

Teams created instructional designs with common formative and summative assessments. That fall, they launched Phase I instructional modules for 10 high-enrollment early academic program courses in Mathematics, English, Psychology, Business, and Natural Science. (For course designs visit the Project Kaleidoscope page.)

Implementing KOCI at Mercy

Back on their campus, Mercy faculty introduced KOCI-created modules targeting critical reading and math competencies collaborating with other institutions and among their colleagues in their learning community. The trajectory for implementation in each area progressed in different ways.

Reading Implementation

Coauthor Howard Miller, professor of Literacy Education at Mercy, participated in the weekly collaborative Skype sessions with faculty at other KOCI institutions. The team soon hit a roadblock, however, when members realized they were designing for students ranging from remedial to developmental to college entry; so, they worked to develop targeted materials accordingly. In this way, Miller noted, the faculty members "stumbled on the four Rs of the OER movement: reuse, revise, remix, and redistribute." These are the key attributes of materials created to share under Creative Commons licenses. One project leader, Brigham and Young University's David Wiley, said that the idea behind the four Rs of open content is to customize and tailor learning materials for different learning environments and diverse learners. As Miller put it, the realization of this was profound:

"This launched us in to a new direction, allowing us to be creative and unfettered in developing course materials that would meet the needs of the students at our respective colleges and that could also be made available to be freely and widely distributed as another set of OER materials."

Miller's designs encouraged students to become active critical readers of academic texts in mathematics, science, and history. His work focused on

• selecting excerpts of college-level readings in these three subject areas that were available under Creative Commons;
• developing four modules of introductory and explanatory materials and assessments in critical reading;
• piloting the four modules in fall 2011 with three seminar faculty members who were teaching 600 students; and
• integrating modules into the courses that faculty members determined would be most effective. This integration was a key aspect of success; use of the modules and holding regular cohort meetings helped professors share integration strategies.

  "The reading and exercises were succinct and aligned with the defined student learning outcomes. The highlight of this experience has been the improvements we as faculty have made in collaborating and making new and engaging learning materials."
  — Emily Seibert, seminar instructor and current seminar coordinator

Critical Inquiry Seminar faculty integrated the OER modules in different ways, selecting points within their existing course outlines where the integration made sense to achieve student learning objectives.  Faculty discuss these outcomes in this video (1:35 minutes) and also here (1:01 minutes).

The faculty's approaches were documented in four key outcomes:

• Assessments conducted at the end of each semester showed greater student learning gains in seminar classes that integrated the critical reading modules. Although the gains were modest at first, they indicated a trend — with a relatively small intervention — and sparked continued integration of instructional modules (see figure 1).
• The overall course success rate and persistence rate increased in the sections taught by KOCI project faculty, although reading is only 30 percent of the course content.
• These outcomes provided the impetus to expand and grow the OER approach; in the project's second phase, the plan is to more fully integrate modules into instruction for reading and other competencies in the course.
• Subsequent data collected showed that student learning gains continued to increase with further integration of KOCI OER modules (see figure 2).
• Based on feedback from seminar faculty using OER modules in 2012, Miller revised the existing modules and created a new module on reading skills for science content. Miller and faculty collaborators continue to work in Phase II of KOCI to improve the existing modules, while also creating new modules — for critical thinking and information literacy — and building a diverse library of OER resources. In fall 2013, short videos for instruction were launched on the Mercy Faculty Center for Teaching and Learning YouTube site.

Mathematics Implementation

Mercy's math faculty lead designated a KOCI Faculty Fellow — coauthor Linda Casper — who spent the summer and fall 2011 semesters conferring with KOCI project directors and math faculty from other participating institutions. The group examined many OER options for college math and discovered My Open Math (MOM), a tool created in 2005 by David Lippman, a community college professor in Washington State. Mercy adopted MOM, which runs on the free, open-source IMathAS platform and provides free, hosted use and free open texts.

The KOCI implementation also changed Mercy's basic math instructional model by "flipping" the classroom. The use of MOM's online videos to supplement instruction has both boosted students' confidence and improved their weaker math skills.

Mercy has achieved several results from the math OER implementation:

• Mercy math faculty reported a nearly 10 percent reduction in course failures in the first semester of using MOM.
• Math instructors cited the availability of the free online text on the very first day of class as key to their retention efforts.
• At pilot's end, Mercy's Mathematics Department chair announced that, starting in fall 2012, all 27 sections (695 students) in basic mathematics would use MOM.
• Between spring 2011 and fall 2012, the math pass rate increased from 48.40 percent to 68.90 percent (see figure 3).
• Algebra courses dropped their previously used licenses and costly math textbooks and resources, saving students a total of $125,000 the first year.

The plan for fall 2013 is to expand the MOM to pre-algebra and a course in math concepts and applications, which will affect nearly a thousand students per semester.

"This was a groundbreaking change in how introductory mathematics is taught and learned at Mercy," said Casper. "It serves to remove a significant barrier of a 'killer' course early in the students' academic program." Casper discusses the impact here (58 seconds).

Challenges and Resolutions

As with any innovation — especially one involving technology — the implementation brought both internal and external challenges. A look at those challenges, and also at what motivated faculty to participate, helps explain the project's success.

Motivations

Mercy faculty participated in this project for many reasons, including the opportunity to work together and meet colleagues at different institutions with fresh new ideas. Faculty participants interacted with OER leaders and KOCI advisors, which motivated them to work toward a solution to a problem beyond their own classrooms.³ In addition to a small stipend from the grant funds for individual participants, the grant also paid for faculty and team members to attend annual OER conferences, where they were invited to present their work and results. This proved an even bigger incentive than the individual stipends.

KOCI held its own gatherings of participants in conjunction with scheduled conferences, giving collaborating partners an opportunity to meet in the real world and build bonds within their community of practice. Faculty had the ongoing intensive support of both their institutional leaders and the KOCI project leaders, Kim Thanos, CEO of Lumen Learning, and M. L. Bettino, retired dean at Cerritos College, which proved integral to partnership success.

"What truly attracted me to this project, and allowed me to set aside my initial skepticism," said Miller, "was the obvious dedication of those involved in the OER movement to viewing accessibility to education as a matter of social justice, with world-wide implications."

Challenges

Mercy College's challenges were both conceptual and logistical. Internally, some faculty resisted raising issues of academic freedom, perceptions of corporate interference with education, and distrust of external influences. In the Critical Inquiry Seminar, faculty were eager for resources to explore a solution. In Mathematics, some department-level conflict erupted that was eventually assuaged by engaging more faculty in participating in OER events where they connected to the larger mission and purpose of the OER movement. The OER movement's focus on student success aligned with the college's mission. Externally, the dearth of OER was cited by faculty as a key challenge, despite the fact that funders such as The William and Flora Hewitt Foundation, the Mellon Foundation, and the William and Melinda Gates Foundation had been supporting OER development for the past decade.⁴

Another challenge was that KOCI needed a common technical interface so module design and analytic components could be standardized across partners. This proved problematic because institutions used different learning management systems (LMS). The short-term solution was to embed modules in a common LMS so outcomes data could be collected and analyzed effectively. Although this was efficient from a data-collection standpoint, it posed challenges both for faculty and students, who experienced the transition from one LMS to another as at times uncomfortable and disruptive. Phase II of the project moved to a different interface, and institutions were able to build within their own LMS.

Finally, Mercy faced the internal challenge of working with a group of innovators who were actively seeking solutions to an educational problem largely outside the institutional hierarchy. However, while the participating faculty were causing a disruption in practice, it produced positive outcomes and got noticed. As data were collected to provide evidence of the initiative's success, more faculty came on board and the project gained the attention of the upper level of administration. Now, faculty fellows are promoting KOCI as a recruitment tool to enable entering students to complete their first year of college courses without textbook costs.

Recommendations for Institutions Considering OER Integration

This case study's inherent value is that it demonstrates a process that increased student success with only a small investment of funds and faculty time. For institutions considering an OER integration, we propose five key recommendations:

• Introduce and facilitate OER efforts through faculty initiative rather than making a top-down institutional directive. Eventually, institutional policy must support emergent practice.
• Connect institutional faculty both to faculty at other institutions and to active OER leaders to motivate them to engage with ideas beyond your own institution.
• Provide professional development resources including those for faculty to attend and present at OER conferences.
• Support faculty with instructional design staff for module integration.
• Establish assessment models at the outset to document results and show trends. This is key to continued institutional support.

Impact, Known and Anticipated

The KOCI goals were to improve retention and reduce textbook costs. Assessments across the KOCI project and at Mercy College showed improvement in both areas, as well as in student and faculty satisfaction:

• In Phase I, Mercy's retention and success rates increased and textbook costs savings in math were $125,000 in the first year alone. Also, consistent with results from all seven partner institutions, courses showed improved student success in KOCI's first year compared with the prior year's course records.
• At Mercy, 1,295 students were affected in Phase I of KOCI (May 2011 through September 2012); across all participating institutions, that number was 9,327.
• Student survey findings administered by T. J. Bliss, a PhD student in Brigham Young University's Educational Inquiry, Measurement, and Evaluation Program, showed that 73 percent of students preferred the KOCI course sections.⁵
• Although cost savings played a significant role in students' preference for KOCI courses, students and faculty alike appreciated the improved learning environments. "Free and better," was the popular refrain. As one Mercy College freshman said, "Last semester, I took a math course that was very challenging and I did not succeed.... I did not understand anything the professor was saying… the pieces just did not connect. I am taking the same course this semester with MyOpenMath and it is set up in a way that I can learn."

A recent study of 22,000 college students by the Florida Distance Learning Consortium found that textbook costs take a toll on students both financially and academically and are a factor in lack of persistence.⁶ These findings suggest a relationship between textbook costs and student success; KOCI operates on that assumption as well. Indeed, the outcomes data collected show student learning gains and increased persistence in KOCI courses.

Next Steps

The next steps for KOCI are to scale participation and impact. There are three primary goals in the second phase of the KOCI supported by follow-on NGLC funding:

• Improve student success by more than 5 percent in each course.
• Eliminate textbook costs as a barrier to student success, with potential student savings of $4.5 million by the spring 2015 term.
• Create 30 open course frameworks and serve up to 75,000 cumulative students.

Looking back to May 2011, when the initial collaborative project leaders met to map out the approach to the project goals, that cumulative number seemed unattainable. Now, however, KOCI's clear success and popularity make that number seem like just the start. KOCI not only eliminates textbook costs, it improves course design and engages a collaborative community across the country in designing better and more accessible curriculum for students who are at risk for non-completion. As such, KOCI does more than save students' money — it might be just what students need for their future success.

Acknowledgments

Many people contributed to this project and article, including: Mercy College's senior instructional designer, Matt Lewis, who provided support in embedding modules in the LMS, along with Mary Lozina, Mercy's director of Online Learning. Lewis also worked closely with the team in creating the video clips noted within this article and with instructional videos in KOCI Phase II, which began with the award of continued funding in October 2013.

Mercy's seminar faculty and assessment specialist, Virginia Coleman-Prisco, served as the lead for collecting assessment impact data for Mercy's KOCI project involvement. Emily Seibert became the faculty seminar coordinator in fall 2013 and was instrumental in Phase I's freshman seminar implementation. Along with Linda Casper, the following people played key roles in KOCI's mathematics implementation: Zsuzsa Kosmane-Fejes, visiting instructor in Mathematics; and Dr. Nagaraj Rao, Mathematics and Computer Science Department Chair.

Co-authors Pawlyshyn and Braddlee are now with other institutions (Northeastern and NOVA, respectively), but are both involved with the KOCI on a national level. A video clip (59 seconds) introducing the collaborators is available.

1. Vincent Tinto, Completing College: Rethinking Institution Action (University of Chicago Press, 2012).
2. Based on the research and model by Dr. Milton Cox and Miami University, Ohio, Mercy College established faculty learning communities as a faculty development model in fall of 2009.
3. These leaders included Norman Bier, Associate Director of the Carnegie Mellon Open Learning Initiative; Vijay Kumar, Senior Strategic Advisor for Innovation and Technology at MIT; Cable Green, Director of Global Learning, Creative Commons; and David Wiley, Associate Professor of Instructional Psychology and Technology at Brigham Young University.
4. The investment in OER development has resulted in increasingly rich repositories such as MIT's Open Courseware Project, Carnegie Mellon's Open Learning Initiative, Washington's Open Course Library, and Rice University's Openstax.
5. T. J. Bliss, John Hilton, David Wiley, and Kim Thanos, "The Cost and Quality of Open Textbooks: Perceptions of Community College Faculty and Students," First Monday, Vol. 18, No. 1 (January 7, 2013).
6. "2012 Florida Student Textbook Survey," Florida Virtual Campus, Tallahassee.