Crowdsourcing the Curriculum: A MOOC for Personalized, Connected Learning

Many current efforts to provide personalized learning at scale involve plugging analytic data about learners into automated technological systems and using algorithms to generate customized and adaptive paths for each student. Yet personalized learning at scale need not rely solely on predictive analytics, quantitative data, and automated platforms. What is more, some fear these approaches may "reduce metacognition and student agency, encouraging students to become dependent on the system rather than learning how to make adjustments to optimize their learning."¹ Another way to personalize learning at scale leverages technology to connect individuals across time and space, capitalizes on the collective wisdom distributed across communities, and empowers learners to co-create their learning experience. In fact, this approach gave rise to the original MOOCs in 2008 (termed by some cMOOCs),² which were based on connectivism.³ Many of the MOOCs that followed these initial efforts — which garnered much more notoriety⁴ — were not based on connectivist principles and instead resembled more static and didactic delivery mechanisms for content (termed by some xMOOCs). In this article, we describe our experiences designing and teaching a MOOC that attempted to bring back and even enhance personalized learning opportunities by implementing connectivist principles within an xMOOC environment and platform.⁵

OK MOOC

In fall 2014, we designed and facilitated a multi-institutional, multidisciplinary, bilingual, and openly licensed (CC BY-SA) MOOC, "Open Knowledge: Changing the Course of Global Learning," known as OK MOOC. As the course designers and instructors, faced challenges and opportunities in developing a massive, open, online, personalized learning experience. We had to make decisions regarding instructional design, technologies, and legal considerations in order to maximize personalization and student self-determination.

With four other co-instructors, we developed and led the 13-week OK MOOC with over 12,000 learners enrolled from 178 countries. Course content focused on issues of "openness" (open education, open access, open source, open science, open data, etc.). (Further information on the course, including the course content, can be found on the course site.) OK MOOC was designed to provide participants with opportunities to

• increase their understanding of the foundations of open knowledge creation, use, dissemination, and evaluation;
• collaborate with learners from other cultures and disciplines;
• work effectively in rapidly changing knowledge environments; and
• develop as independent, self-directed, lifelong learners.

To achieve these aims, we recruited instructors from Canada, Mexico, the United States, and Ghana with expertise in education, information science, business, and technology. We received seed funding from Simon Fraser University and Stanford University, which included in-kind support from educational designers and technologists.

On the surface OK MOOC resembled many other xMOOCs in that it followed a traditional course format, used an xMOOC learning management system, and featured instructor-selected video content and readings.⁶ However, the course deviated from a traditional model by leveraging a "distributed flip" model that in part enabled learners to personalize their experience.⁷ In this model, MOOC content serves as a hub to facilitate and connect distributed communities of formal, for-credit students as well as noncredit, informal learners. The inspiration for the course design and technology decisions came from instructor experiences with the innovative cMOOC model,⁸ and the work of Cormier, Couros, Downes, Siemens [http://www.elearnspace.org/blog/], Wiley, and others. To facilitate connecting, OK MOOC was open to the world as an informal learning opportunity, but simultaneously offered for credit to registered students at the instructors' institutions.

Design Approaches

We chose several approaches to the design: a crowdsourced curriculum, module flexibility, multiple tracks to completion, and student-led discussions.

Crowdsourced Curriculum

The first component of personalizing the course was to collaboratively work with students to build a resource list of readings, lectures, and other materials for each module — to crowdsource the curriculum. All instructor-selected readings and videos were suggested but not required. Students could choose from the instructor-suggested list or the growing list of peer-selected resources. We wanted to enable students to recognize their existing knowledge; discover the multidisciplinary, multicultural knowledge of their peers; and share the power of setting the curriculum. We believed this approach would help develop learners' "personal knowledge mastery," a fundamental skill of professionals in the knowledge economy. Lastly, we wanted to collaboratively build an extensive repository of freely available "open knowledge" resources.

To facilitate the building of a shared list, we introduced Jarche's Seek, Sense, Share framework. This approach encouraged learners to seek resources from the world around them that the considered relevant to the course. We also asked learners to briefly describe and reflect on the resource (sensing) and put into practice what they learned through their writing and discussion posts. Lastly, learners were asked to share the resources with the OK MOOC community and, more broadly, the world.

Students appreciated the course-generated resource list in terms of broadening the resources available, enabling global voices to be heard, and empowering them as independent learners. For example, one student commented: "I have found many links suggested by fellow students to be extremely helpful in encouraging my own learning journey." Another noted: "So far I like the agency I have over choosing which subjects to review more thoroughly and respond to posts, etc."

Crowdsourcing the curriculum had its downsides. For example, many students felt overwhelmed by the volume of resources: "All channels are flooded with messages from hundred plus participants." Also, because much of the content was to be student contributed, some students felt the course was not a true MOOC experience. When asked in the midcourse survey what they found valuable in the course, a learner indicated: "Nothing. This is not a class. This is a website with different types of content. You're deceiving people by calling this a MOOC." This student's criticism encapsulates some of the tensions related to a crowdsourced curriculum, including that some students believe they can only "learn" in a course from instructor-driven content. This also illustrates how the "grammar of schooling" (instructor dissemination of knowledge, inflexible schedule, summative grading schemes) remains ingrained in many adult learners and can be a roadblock to self-directed and personalized learning.⁹ To some degree, we suspect that by adding recommended resources to each module, we might have inadvertently undermined the resource-sharing exercise, with many students opting to rely exclusively on our expertise rather than exploring the findings of their peers.

We came to realize the need to balance learner autonomy with guided instruction, as echoed in this student's comment: "Independent learning has to be based on some sort of structure. I have been working to provide a structured approach while keeping myself open to exploring new ideas." On reflection, we feel that increased messaging from the outset about the course structure and learners' roles and responsibilities might have allowed students uncomfortable with highly self-determined learning experiences an early opportunity to prepare themselves for something significantly different from previous learning experiences.

Module Flexibility

We attempted to foster personalized learning by encouraging learners to choose multiple paths through the course to fit their needs and interests. For example, if during the second week a learner was primarily interested in engaging with the content slated for week six, then she was welcome to explore those materials. Additionally, all OK MOOC content remained accessible following the formal course period, allowing learners to proceed at their own pace or return to those materials for just-in-time learning. Due to constraints of a traditional university calendar, for-credit students lacked similar flexibility, although they could choose to put less effort into some modules than others without penalty.

Despite encouraging learners to select modules of their choice, some students felt compelled to complete all materials before moving to the next module. One student noted: "I try to cover all the videos and core readings, and work on the activities. I wish I [could] cover more of the additional readings." For many students this caused frustration with the amount of content and an anxiety about course pace. "I am constantly a little behind so I don't really write as much as I could and should in the forums." Another student commented: "Since there is so much information on each topic...it's not possible in the time available to follow many of the interesting contributions from fellow students." Again, early instructor efforts to emphasize the personalized learning focus of the course might have reduced student anxiety.

Multiple Tracks to Completion

Learners could choose one of three tracks to earn a Statement of Accomplishment, which required completing a number of activities depending on the track they chose, including posting and commenting on the discussion forum; sharing resources; actively tweeting; evaluating resources; creating an open, online digital object; and writing a brief self-assessment of learning. Tracks included Connecting, with an emphasis on sharing and communicating; Evaluating, with an emphasis on critiquing resources; and Creating, with an emphasis on building and sharing a new open, online digital object of their choice. The course did not involve quizzes, tests, or essays. Completion rates for the tracks were Connecting, 54; Evaluating, 72; and Creating, 52.

Having multiple tracks allowed students to decide how much time they wished to devote to the course, with the Connecting track requiring approximately 1–2 hours per week, the Evaluating track 3–5 hours, and the Creating track 5–8 hours. Students only interested in one or two modules, while not eligible for a Statement of Accomplishment, were considered valuable course members. For the instructors, a successful MOOC experience was not one where students consume all of the learning materials but where they come away having learned something they consider of value. As mentioned, though, we could have been more explicit about the many ways learners could extract value from the OK MOOC experience.

Related to the tracks, we maximized personalized learning by allowing Creating track students to select both the format and topic of their major assignment. To complete the Creating track, learners needed to create an openly available, online object related to the course topics covered. Table 1 shows examples of student projects. On completing the assignment, students had to post their digital object to the course discussion forums, where instructors and students could engage with and respond to the work. Over time, several students have continued to work on their projects, signaling that this exercise was not a throwaway. For example, one student took on the digitization of the "forgotten fantasy" paperback Aladore. The student continues to make progress on this initiative and to document his work. Additionally, these projects were made publicly available to reinforce course ideals and extend their impact beyond the course.

Table 1. A sample of Creating student projects

Project	Format
Open Education Tutorial	Video
Mooctology	Blog
Open Knowledge	Pinterest board
Open Access Week at UBC [https://storify.com/lonelily/open-access-ubc-in-review]	Storify
Aladore	Digitized novel

Due to the formally structured nature of for-credit courses, credit students were much more constrained by course sequencing and had to review all course modules and complete all assignments to receive university credit. This highlights the difficulty of implementing connectivism and personalized learning in a traditional educational setting and again points to ways in which the pervasive grammar of schooling can be a roadblock to these methods.

Student-Led Discussions

We extended personalization through student-led discussions. The online forum was open to anyone to initiate a discussion or participate in an existing one. Over 2,400 unique learners wrote more than 15,000 forum posts during the semester, which were predominantly in English but also in Spanish. While some posts were instructor prompted, instructors limited their interventions, getting involved only to encourage participation, add ideas, or pose additional questions.

Despite probing instructor questions and encouragement in the course instructions, online discussions proved less successful in connecting learners. Many posts were monological, resulting in forum posts being a single author's voice instead of a participatory conversation. In addition, the large number of students, from a wide variety of backgrounds, made the development of meaningful connections challenging. Because much of the course value lay in the discussions, more effort would be needed to ensure its success for online students. This could include incorporating more-guided questions to promote discussion, actively linking students with similar interests, and ensuring optimal software options for encouraging dialogue.

In the physical classroom, students divided into small, permanent discussion groups and self-selected a weekly leader. Leaders determined the questions to ask their group and were expected to facilitate a meaningful exchange. Instructors circulated between groups, allowing discussions to organically unfold and intervening only when asked. The in-class discussions were largely successful, with students expressing a strong sense of community within and between the in-class groups. Despite our encouragement, however, interactions between the in-class and online students were not as successful as we expected. More attention to consciously building these connections through creating international discussion groups or offering more compelling reasons to collaborate with a peer from another institution might have helped students better interact with their wider peer group.

Technology Approaches

Personalized learning at scale requires the thoughtful and intentional integration of pedagogy, content, and technology throughout the design process. We recognize that the technological options are evolving rapidly; therefore, we do not make any specific recommendations, but instead provide examples of how we (as instructors and instructional designers) wrestled with integrating technology, course goals, personalized learning pedagogies, and institutional IT structures. Table 2 summarizes the pros and cons of the four technologies we used in the course, which are described in detail below.

Table 2. Technologies to support massive personalized learning: affordances and challenges observed in OK MOOC

Technology	Affordances	Challenges
Stanford Lagunita (Open edX)	Provides a central location for student participation Allows students to create new forum posts and comment on the posts of others Enables instructors to see student data to better meet the needs of individual learners	Unintuitive for structuring and organizing discussions No notifications of comments on own posts Difficult to integrate with third-party tools Reinforces a more teacher-centric approach
Twitter	Connects students to others around the world, students and nonstudents alike Breaks the artificial barrier between the classroom and the "real world" Students control who to follow and build their own personal learning network Students control what to tweet or retweet, and which links to select and read	Students must register with a third-party service, with data stored on a commercial server Loss of control over personal data High volume of content challenges some students Unstructured resource sharing Educational purpose not readily apparent for all students
Diigo	Structured resource sharing (tagging, etc.) Group-sharing feature Private or public bookmarking options Easy bookmarking and annotating of materials	Unfamiliar to most learners High initial learning curve for some students
Google Community	Familiar interface, often considered intuitive and easy to use by students Learners can decide what to post, when to reply	Students must register with a third-party service, with data stored on a commercial server Loss of control over personal data Unstructured resource sharing At larger scale, challenging to follow discussions

Lagunita (Open edX)

We selected Lagunita, hosted by Stanford, as our primary platform because it is open source and, in part, because of our partnership with and funding from the university. Lagunita contained the majority of instructor-curated content, including text-based orientation materials, video links, and readings. Lagunita captured and, for instructors, displayed learners' platform interactions. As a traditional xMOOC platform, Lagunita worked well as a central location for instructor-provided content and instructor communication.

Some aspects of Lagunita worked less well for OK MOOC and its more connectivist intent. Structurally, the discussion forums were a major concern. As instructors, we struggled to effectively embed prompts for weekly discussions and monitor forum activity, given the limited ability to receive alerts for posted content. Students expressed dissatisfaction with the display of posts and lack of ease in tracking their posts or any replies. "The overall structure of the discussion board is a nightmare for Type A organized folks. Often people end up starting a new thread instead of finding an existent discussion topic that fits their comments, resulting in this long stream of single-comment discussions, which is not helpful." Related to these concerns, one dissatisfied learner launched an in-MOOC discussion, "Help Wanted: Reinventing MOOC discussion boards," to generate ideas for improvement. This thread was one of the most commented on.

Twitter

We selected Twitter in the hope that learners' interactions as tweet writers and consumers would expose them to a global conversation on openness, teach them about information flow, provide opportunities to share resources and course experiences, and help develop their own personal learning networks. To facilitate Twitter use, we promoted #okmooc, pointed learners toward relevant hashtags and people to follow, and supplied instructional materials on Twitter. There were 2,237 tweets total, of which 1,854 were unique.

Harvard and nonprofit launch largest library of exceptional K-12 student work http://t.co/yptx3X1D6M via @educationdive #edchat #okmooc

Knowledge Unlatched, nuevo enfoque de las bibliotecas a hacer libros académicos de acceso abierto, consulta http://t.co/z9HSnvh8gf, #okmooc

Hey ex- #okmooc -ers! Keep up the exploration with this super cool discussion on Critical Pedagogy #moocmooc http://t.co/Mm0HH3YGO7

#OKMOOC how do we know we pass the course?

#OKMOOC classmates who just finished the @Stanford #OpenKnowledge course-any other #MOOCs you'd recommend? I'm suffering withdrawals here.

Are OER's the answer to equity What about issues of learner skills required to gain from OER's and scaling this #OKMOOC

Declaración de Lyon sobre el Acceso a la Información y Desarrollo (via #okmooc ) http://t.co/qpWmhVRPv

Some students voiced concern about digital privacy related to Twitter. For example, some learners were concerned about the longevity/stickiness of their posts and future employment. Resolute learners were able to meet the requirements of the Statement of Accomplishment without using Twitter or by creating an anonymous Twitter account exclusively for the course.

Diigo

To manage the "crowdsourced curriculum," we created a Diigo Group. Each week students were asked to discover, describe, tag, and share web resources, as well as explore and comment on peer-shared resources. This served as an additional resource list for the course, on top of the instructor-recommended resources. Over 2,500 resources are included in the collection, providing a record of the course's shared learning experience.

Diigo is feature-rich and easy to use (after some initial training), but in the end only 596 students registered with the group, although many more might have visited the site without signing up (registration is not required to view the group). The reasons why more students did not register included feeling overwhelmed by needing to learn another tool, finding the tool difficult to use, not being interested in actively participating in the overall process of finding and sharing resources, and not trusting one's peers to make useful recommendations.

Google Community

Shortly after the course launched, an OK MOOC student created a Google Community independent of the official course called "Stanford MOOC: Open Knowledge" and linked it to all course resources. All course participants and instructors were invited to join. The student chose this platform as "a structure that everyone can work with" and thus provided another platform for learners to further control their learning.

Of the learners, 578 joined the Google Community. Learners were encouraged to experiment with the platform, provide feedback and content, reflect on the course experience and materials, and share their expertise and unique global perspectives. Learners posted a variety of materials, including videos, graphics, and news stories, and commented on these materials in both English and Spanish. To support the Google Community, OK MOOC instructors allowed posts to count toward course requirements. Instructors played minimal roles in this independent online space, partially due to the inability to be present on all platforms but also to support and encourage learner autonomy.

From the instructors' perspective, the Google Community posed unique challenges. First, it stretched their time available to participate in another platform. Additionally, it introduced an inherent loss of "classroom" control. For example, a few weeks into the course, a learner proposed closing the group to newcomers as it neared 150 members to keep it small in accordance with Dunbar's number. It was difficult as instructors not to interfere and suggest that this went against the nature of the course; however, the group decided on its own to keep the course open.

Legal Considerations

In facilitating distributed, personalized learning communities, we provided options for learners to engage at different levels; select the modules of greatest interest; contribute new resources to the course materials; co-create, reuse, and share content for assignments; and connect over various social media platforms. The ability to use and share these activities — essential for students to personalize their learning — was enabled by the openness of materials. The course not only covered issues of "openness," the course materials were assembled from existing open educational resources. When possible, new material was made available under a Creative Commons Attribution-Share Alike (CC BY-SA) license. However, this meant specific legal considerations regarding open materials. For example, some content could not be included in the course or made accessible to all learners because of license and permission restrictions. In other instances, permission was obtained only after protracted correspondence with content owners, and this process consumed a significant amount of development time that could have been used to improve the course design. Special institutional permission was also needed to openly license instructor-created material. Finally, some students were concerned about agreeing to the platform's terms of service and, in particular, their privacy and ownership of any materials they contributed to the course. This was partially ameliorated by adding a component to the syllabus regarding learners' rights and allowing an option for them to access content without formally registering on the platform.

Conclusion

Our experience designing and teaching OK MOOC illustrates that personalized learning at scale can be supported without predictive analytics, and we encourage educators to consider integrating more open and connectivist principles in the learning space. The need to provide meaningful, personalized learning at scale will only continue to grow and will likely require the thoughtful integration of many pedagogies and emerging technologies, including learning analytics and connectivism. While there were certainly challenges to the pedagogical approach described in this article, we will continue to build personalized learning, self-direction, and open approaches into our future courses and hope other educators and instructional designers will consider our experience in their future teaching endeavors.

Acknowledgments

We'd like to thank our fellow instructors Arianna Becerril (UAEM), Smith Esseh (KNUST), Bozena Mierzejewska (Fordham), and John Willinsky (Stanford), and our teaching assistants Ashton Napier (Fordham) and Maryam Lucia Attai (Stanford) for their work on developing and delivering the course, as well as the students from our institutions and from around the world who participated in this unconventional learning experience.

Notes

1. 7 Things You Should Know About Personalized Learning ELI, September 8, 2015.
2. George Siemens, "MOOCs Are Really a Platform," [http://www.elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/] elearnspace blog, July 25, 2012.
3. George Siemens, "Connectivism: A Learning Theory for the Digital Age," [http://www.elearnspace.org/Articles/connectivism.htm] elearnspace blog, December 12, 2004.
4. Laura Pappano, "The Year of the MOOC," New York Times, November 2, 2012.
5. Michael Caulfield, "xMOOC Communities Should Learn From cMOOCs," EDUCAUSE blog, July 11, 2013.
6. Ibid. See also Todd Bryant, “Bringing the Social Back to MOOCs,” EDUCAUSE Review, June 22, 2015.  
7. Michael Caulfield, Amy Collier, and Sherif Halawa, "Rethinking Online Community in MOOCs Used for Blended Learning," EDUCAUSE Review, October 7, 2013.
8. See Tony Bates, "Comparing xMOOCs and cMOOCs: Philosophy and Practice," blog, October 13, 2014.
9. David Tyack and William Tobin, "The 'Grammar' of Schooling: Why Has It Been so Hard to Change?," American Educational Research Journal 31, no. 3 (September 21, 1994): 453–479.

---

Lauren A. Maggio is an associate professor of Medicine and the associate director for Distributed Learning and Technology for the Graduate Programs in Health Professions Education (HPE) at the Uniformed Services University of the Health Sciences. She holds a PhD in HPE from Utrecht University in conjunction with the University of California, San Francisco, and a master's degree in Information Science from Simmons Graduate School of Library and Information Science. Maggio's research examines how to effectively connect health professionals, learners, and patients with information through the design of educational initiatives and by facilitating access to knowledge for public and professional use.

Andrew Saltarelli is the director of Digital Learning Initiatives at Stanford University in the Office of the Vice Provost for Teaching and Learning. He leads a team of instructional designers who collaborate with faculty to design new and effective online, blended, and other innovative digital learning experiences for on-campus learners and beyond. Saltarelli holds a PhD in Educational Psychology and Educational Technology from Michigan State University and has published a number of studies that systematically investigate how instructional technologies affect the social psychological processes (e.g., belongingness, motivation, cooperation) underlying teaching and learning. 

Kevin Stranack is the Community Services & Learning Coordinator for Simon Frasier University Library's Public Knowledge Project. He is also an adjunct faculty member at the University of British Columbia's School of Library, Archival, and Information Studies. Stranack received his master's of Library and Information Studies from the University of British Columbia in 2002 and his master's of Adult Education from the University of Regina in 2013.