Today's LMS needs to be supplemented with (and perhaps later replaced by) a new digital architecture and new learning components—the NGDLE—to enable current transitions in higher education.
Malcolm Brown is Director of the EDUCAUSE Learning Initiative (ELI). Joanne Dehoney is Chief of Staff, EDUCAUSE. Nancy Millichap is Program Officer with Next Generation Learning Challenges (NGLC), EDUCAUSE.
The learning management system (LMS) is a remarkable phenomenon in higher education. On the one hand, the LMS has seen unprecedented adoption rates. Estimates of colleges and universities running an LMS are almost always near 99 percent. Of faculty, 85 percent use an LMS, with 56 percent using it on a daily basis, and 74 percent say it is a useful tool to enhance teaching. Among students, 83 percent use an LMS, and 56 percent say they use it in most or all courses.1 In an enterprise as highly individualistic as teaching and learning, these are remarkable numbers. No other academic application comes close to such adoption rates.
On the other hand, restlessness with the LMS is conspicuous. One measure of this is the finding that 15 percent of higher education institutions intend to replace their LMS in the next three years, which is far higher than typical for enterprise-class applications. Blog posts abound with expressions such as "LMS 3.0," "LMS 5.0," and "the LMS in a post-LMS world." Despite the high percentages of LMS adoption, relatively few instructors use its more advanced features: just 41 percent of surveyed faculty report using the LMS "to promote interaction outside the classroom."2
What is clear is that the LMS has been highly successful in enabling the administration of learning but less so in enabling learning itself. Tools such as the grade book and mechanisms for distributing materials (e.g., the syllabus) are invaluable for the management of a course, but these resources contribute only indirectly, at best, to learning success. Initial LMS designs have been both course- and instructor-centric, which is consonant with the way higher education viewed teaching and learning through the 1990s.
Higher education is moving away from its traditional emphasis on the instructor, however, replacing this emphasis with a focus on learning and the learner. Higher education is also moving away from a standard form factor for the course, experimenting with a variety of course models. These developments pose a dilemma for any LMS with a design that is still informed by instructor-centric, one-size-fits-all assumptions about teaching and learning. They also account for the love/hate relationship that many in higher education have with the LMS. The LMS is both "it" and "not it"—useful in some ways but falling short in others.
If current LMS designs are tied to a teaching and learning model that is being replaced with new approaches, then what should come next? Because the successor to the LMS needs to support a very different model of learning, it seems unlikely that we can get there by updating the current LMS with incremental improvements. Nor is the matter as simple as setting the current LMS entirely aside and developing another from scratch. Indeed, the instinct to "build from scratch" in the tradition of an enterprise application may be approaching obsolescence. Over time, the LMS needs to be supplemented with (and perhaps later replaced by) a new digital architecture and new learning components that contribute to and enable the transitions that higher education is currently experiencing. The challenge is to build on the value of the LMS as an administrative tool by retaining what works but not be bound to an outgoing model of teaching and learning. Another challenge is to make targeted investments that will bring about the environment of the next generation more quickly and coherently.
The Next Generation Digital Learning Environment
We have adopted the term next generation digital learning environment (NGDLE) for what should come next—after the LMS. The term pulls together several key themes. What comes next must be informed by the new learning-centered model that increasingly characterizes higher education practice (hence next generation). It must be digital, given that digital technology has become a component of virtually all teaching and learning practice. It must be about learning, since learning ties together learner and instructor. Finally, it must be an environment or ecosystem—a dynamic, interconnected, ever-evolving community of learners, instructors, tools, and content.
NGDLE is next generational in another sense as well. In traditional IT practice, when you have a problem to be solved, you write code to develop an application that will address the issue. But code, like the built environment, imposes a point of view. Classrooms, as one scholar puts it, are instances of "built pedagogy," and the LMS, in a similar way, imposes a pedagogical model.3 But any approach that posits a single chunk of new code is out of sync with the wide variety of postsecondary teaching and learning. This is particularly true today as higher education is transitioning from the transmission model of education to one built on concepts such as active learning, personalization, hybrid course designs, and new directions for measuring degree progress.
Although the NGDLE might include a traditional LMS as a component, it will not itself be a single application like the current LMS or other enterprise applications.4 Rather, the NGDLE will be an ecosystem of sorts, characterized by the following:
- At the built layer, it will be a confederation of IT systems, including content repositories, analytics engines, and a wide variety of applications and digital services.
- One key to making such a confederation work will be full adherence to standards for interoperability, as well as for data and content exchange.
- Instead of uniformity and centrality, it will need to support personalization as an option at all levels of the institution. The NGDLE will not be exactly the same for any two learners, instructors, or institutions.
- For users, it will be a cloud-like space to aggregate and connect content and functionality, similar to a smartphone, where users fashion their environments directly with self-selected apps.
- If the paradigm for the NGDLE is a digital confederation of components, the model for the NGDLE architecture may be the mash-up. A mash-up is a web page or application that "uses content from more than one source to create a single new service displayed in a single graphical interface."5 Hence it uses a heterogeneity of components to produce a homogeneity of function. The confederation-based NGDLE will be mashed up at both the individual and the institutional levels, as opposed to consortia forming to create open enterprise applications.
The challenge for the NGDLE is supporting this diversity while retaining the necessary technological coherence. But in this challenge also lies the opportunity. Clearly we need to invent new architectures that support a digital confederation. We need to invent a model for technological coherence for the NGDLE, consisting of standards and core services. Other components will also be necessary, such as new standards, tools, and user experience designs. Examples of how this might play out include the following:
- An institution may decide to forgo the current LMS altogether and instead set up a confederation of components that provide similar functionality.6
- An institution could retain the LMS as a core component, preserving its value as an administrative tool and a linchpin for learning data. But learning pioneers would be able to experiment and innovate by hooking apps and other functions onto the LMS. Contact with the LMS would be more indirect than direct for most users.
- Clusters of institutions could form consortia (such as Unizin or C-BEN)7 to set up co-ops to provide a buffet of apps and other tools, either purchased or donated. In this case, each institution's learning environment would be a unique blend of these components.
Five Functional Domains
As amorphous as the NGDLE may be from a traditional perspective, its realization will entail five critical domains of core functionality:
- Interoperability and Integration
- Analytics, Advising, and Learning Assessment
- Accessibility and Universal Design
Progress toward the full realization of the NGDLE is possible only if the whole set of five functional domains is addressed.
1. Interoperability and Integration
Finding: Interoperability is the linchpin of the NGDLE. The ability to integrate tools and to exchange content and learning data enables everything else.
Interoperability in the context of the NGDLE has four primary dimensions. The first concerns content: all components must be able to accept and exchange curricular content in common formats. This ensures that content can be exchanged, transferred, and utilized. Second, on the tool side, integration must be easy enough for end users to quickly and easily add tools to the environment, without help from the central IT organization. Third, the learning environment will continue to be the key source of learning data. The unimpeded exchange of data is imperative to be able to aggregate, integrate, and analyze learning data. Fourth, the NGDLE must enable the creation of new interoperability standards in ways that are compatible with its other standards so that overall coherence is maintained.
For example, an NGDLE might tie together an e-book application with a course syllabus and a separate quizzing tool, all of which could smoothly exchange data. The syllabus could link students to e-book resources, which could communicate with the quizzing tool to provide appropriate review questions based on the pages read, creating opportunities for adaptive learning. Data about responses could then be sent to a gradebook, providing a complete picture to students and instructors about progress and areas of weakness. Higher education has not always taken full advantage of available standards, and the NGDLE will be an opportunity to encourage full implementation of technical and quality standards that enable such interoperability.
Finding: Personalization is the most important user-facing functional domain of the NGDLE.
Personalization is highly dependent on interoperability. Whereas the mechanisms of interoperability (such as data standards) are largely invisible to the user, personalization is highly tangible and is the most important factor shaping the user experience. A learning ecosystem that enables learners and instructors to act as the architects of their environments is a powerful tool.
Personalization encompasses two aspects. The first is the outfitting and configuration of the learning environment, which is then used to construct pathways to accomplish learning tasks and attain learning goals. Typically we think of this as happening at the individual level, that of the learner and the instructor. But a configuration of this kind also needs to happen at the departmental, divisional, institutional, and consortium levels.
The second aspect is adaptive learning, in which an automated system provides learners with coaching and suggestions specific to each learner's needs. There has lately been considerable momentum around adaptive learning, much of it from textbook publishing companies, and it must be a feature of the NGDLE landscape. As with other NGDLE functional domains, integration of adaptive learning tools will be key, as well as the ability of the tools to contribute learner data to support analytics.
At the 2014 EDUCAUSE Annual Conference, 50 thought leaders from the higher education community came together to brainstorm NGDLE functionality. This group identified and prioritized 56 desirable NGDLE functions. Three of the top 10 functions pertain to personalization:
- Integration for discipline-specific apps (#1)
- Easy to configure or adapt to teaching styles and disciplines (#2)
- Clear, customized, self-paced learning/degree pathways (#9)
Note that the term discipline appears in two of these functions. Academic disciplines are defined in part by scholarly practices and pedagogical methods and so have their own "personalization" requirements. This illustrates that personalization needs to be supported at the collective level and not just the individual level.
The integration of tools and content is one of the more important NGDLE challenges/opportunities. Currently a great deal of third-party content (both open and proprietary) is available to learners and instructors, and the same is true to some extent of learning apps and tools. But it is far from easy for end users to integrate content and tools directly into the current LMS. By contrast, the NGDLE will need to empower all users of the environment to add, alter, and customize the components to directly support their individual needs.
3. Analytics, Advising, and Learning Assessment
Finding: The analysis of all forms of learning data—resulting in actionable information—is a vital component of the NGDLE and must include support for new learning assessment approaches, especially in the area of competency-based education.
In the context of the NGDLE, there are two primary dimensions to analytics:
- Learning analytics, defined as "the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs."8
- Integrated planning and advising systems (IPAS), defined as "an institutional capability to create shared ownership for educational progress by providing students, faculty, and staff with holistic information and services that contribute to the completion of a degree or other credential."9
These dimensions are much like the two sides of a coin. Both rely on the aggregation and analysis of learner data to produce actionable reports and information. Learning analytics tends to be focused at the course level, whereas the IPAS suite of analytics typically targets overall student success, especially degree completion. The thought leaders we consulted were unequivocal about the importance of these kinds of analytics for NGDLE. At the EDUCAUSE 2014 convening, 30 percent of the desirable NGDLE functions had to do with one or both of these kinds of analytics. The question, then, is not whether they should be included but how.
Today most major LMS platforms have proprietary learning analytics capabilities that use data from the LMS and the student information system. These modules can be considered first-generation attempts. Future analytics modules could sit outside the LMS, while their dashboards could be viewable within the LMS or other applications using the Learning Tools Interoperability (LTI) specification. In the transition to the NGDLE, several points are key:
- Widening the scope of the data. There are three kinds of learner data: dispositional (e.g., incoming GPA, biographic and demographic data), course activity and engagement, (e.g., keystrokes, selections, time on task), and learner artifacts (e.g., essays, blog posts, media products). All of these need to be incorporated into the NGDLE's approach to learning analytics.
- Integrating the platform, tools, and data. Standards are again key to this kind of integration.
- Enabling learning analytics for all stakeholders. As one thought leader told us, the next generation of learning analytics must address at least three levels: student empowerment, continuous instructional improvement, and institutional oversight. It must also furnish the basis for deeper, long-term research into the conditions that promote effective learning.
An ever-widening scope of data and integration is highly relevant for IPAS as well. IPAS applications are still an emerging technology, so a great deal of exploration lies ahead. Data standards will be of special importance. For example, one particular challenge is the need for a standard way to describe degree pathways for IPAS environments. Degree pathways are often complex and contain a host of exceptions, alternatives, and idiosyncrasies.
Like the other domains, analytics will require personalization in its features. Faculty, students, and administrators will want to configure reporting dashboards to reveal the information they deem most essential to tracking progress. Most often, users see analytics as a way to track student progress, but we found interest in what might be called "teaching analytics," the use of analytics to inform both the design and the conduct of a course. Employed in this way, analytics can play a role in providing evidence of the impact of pedagogical strategies and tactics.
Another important dimension is the assessment of learning. Assessment is foundational to learning and is therefore of central importance to any learning environment. The key aspects of NGDLE learning assessment include the weaving together of standard formative assessments, adaptive learning technology, and learning analytics, as well as the continued development and integration of portfolios. Competency has emerged as a particularly important way of assessing learning and mastery, calling attention to another disconnect in that the conventional LMS is organized around the course, whereas competency-based programs typically focus on smaller units of learning. Competency-based education (CBE) has already developed considerable momentum. For example, several LMS platforms have introduced support for competency-based approaches with tools such as a mastery-based gradebook, using the acquisition of skills as the way of measuring progress toward learning goals. Some institutions have now established CBE-based programs (e.g., Northern Arizona University, the University of Texas, and the University of Wisconsin). Key for the NGDLE is to integrate various ways to assess learning, moving away from tools that support only a single approach.
Finding: The NGDLE must support collaboration at multiple levels and make it easy to move between private and public digital spaces.
Collaboration is fundamental to many forms of learning. Even the relationship between the reader of a textbook and its author can be viewed as a kind of collaboration. Digital technology provides learners and instructors new opportunities to collaboratively construct unique pathways to accomplish learning goals. The tsunami of social networking in particular has enabled learners and instructors to organize learning collaborations at all levels, purposes, and group sizes. Learners are no longer restricted to forming collaborations with just their peers in a course. They can organize interinstitutional collaborations, discover content, and participate in MOOCs and other learning communities to augment their learning for a particular course. The breadth and depth of resources at the disposal of the higher education learner are unprecedented. The NGDLE must provide the resources to support learning collaborations of all kinds.
The support for collaboration must be a lead design goal, not an afterthought. The current LMS is often designed on the transmission model of education—a mechanism to transmit syllabi, content, and assessments. This process is important for the management of the course, but equal time must be given to collaboration, a true learning dimension. The NGDLE must provide learners with individual spaces that persist across entire academic careers (and possibly into professional lives), serving as a base for all learning operations. Tools such as portfolios and tools for content creation must also be fully integrated into the environment.
One issue that the NGDLE must address is the "walled garden" problem. Most of the current LMS systems were designed under the assumption that what happens in the course must stay in the course. As a result, within the LMS, the course is a private community—a walled garden. There are good reasons for this approach. It gives instructors use of content that might otherwise be unavailable to the course because of copyright laws. If trial and error is a core rhythm of learning, then a private setting makes it easier for learners to embrace a path of improvement, part of which is inevitably making mistakes and learning from them. But recent experience has shown that coursework in social settings, authentically situated, can have great value in the learning process. It has also shown the value of learning that is organized in units other than the course. So the issue is not that the walled-garden approach is entirely wrongheaded. The issue is that it is all too often viewed as a binary choice—a course is either public or private. A requirement for the NGDLE is to move past this either/or view and instead enable a learning community to make choices about what parts are public and what parts are private.
5. Accessibility and Universal Design
Finding: Efforts to realize the NGDLE should include working toward ensuring that all learners and instructors are able to participate, with access to content and the ability to create accessible learning artifacts. We should strive to address issues of accessibility from the start, based on a universal design approach.
The shift away from the LMS to the NGDLE is one of considerable magnitude. Instead of just altering a room or two in our digital learning environment "house," we are reimagining the whole house, including its foundation. A change of such scope brings with it many opportunities. We suggest that the NGDLE represents such an opportunity: to assist higher education in supporting the needs of people with disabilities by adopting a universal design approach in the context of a digital learning environment. A holistic, ground-up approach, addressing accessibility within the larger framework of universal design, has the potential to provide the most accessible digital learning environment possible.
The architect Ron Mace formulated a useful definition of the concept of universal design: "Universal design is the design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design."10 In the context of the NGDLE, universal design means, most simply, having the goal of a learning environment that works for all learners and instructors. A universal design approach means that accessibility is addressed in the initial design of all NGDLE components, as opposed to being integrated—often imperfectly and at considerable cost—after the fact. This changes accessibility from an exercise involving users and technology to one focused on people and experiences, with the goal of enabling everyone to be successful in the digital learning environment.
Addressing accessibility via universal design enables us to think about the dual role of any learner: both a receiver and a creator of content. Learning entails both reception and expression. NGDLE components need to address both aspects from a universal design framework. The opportunities are many. Consider the student producing a video in conjunction with a course assignment—the NGDLE toolset should enable and encourage that student to produce a version of the video that is broadly accessible. Universal design thus becomes integrated into core digital literacy skills that all students develop when interacting with the NGDLE. Starting from a universal design perspective, improvements in discussion boards and assessment instruments would focus on designs that support learners in independent, successful task completion, while reducing user interface clutter that may create severe usability challenges for people with certain impairments.
The confederated approach we propose for the NGDLE is the key to progress in accessibility and universal design. This approach would encourage the development of specialty tools that could potentially address the more difficult obstacles to accessibility. Embracing interoperability standards would enable faster, more effective integration of these tools into the larger learning environment, and including accessibility standards as part of interoperability will help produce components that support people with disabilities. Similarly, including accessibility in the support of personalization and adaptive learning helps balance the need for universal design of the learning environment with the opportunity to provide individually tailored experiences that are sensitive to accessibility requirements. Making progress in this functional domain is not just a matter of interoperability standards, since it is first and foremost a question of design. Incorporating universal design into the NGDLE framework will allow it to effectively address a wide range of accessibility needs and concerns, as well as encourage designers of NGDLE components to integrate accessibility as a core part of the design from the beginning.
Moving Forward: Lego Sets
If we look over the functional domains just discussed, and think too about the immense variety that higher education teaching and learning comprises, we might ask, "How could any application address all of that and do it well?" The answer, of course, is that no single application can, which harkens back to our earlier suggestion that no single chunk of code will constitute the NGDLE. This fact frees us to seek new, more effective approaches.
If the creation of an über application is not the path to the NGDLE, then we will need to take what might be called a "Lego approach." Indeed, if the mash-up is the way that individuals and institutions will assemble their own NGDLE, then one way to enable that model is to populate the landscape with a set of tools and resources that are NGDLE conformant. This would result in a toolbox of applications, content, and platforms that could be assembled in custom ways. The key is defining what is meant by "NGDLE conformance." Lego pieces work because of a design specification that ensures they will interlock, while enabling a wide variety of component parts. For the NGDLE to succeed as we describe here, we must define a similar set of specifications and services that constitute the conformance needed to make the Lego approach workable.
The Lego approach has two key advantages. One is that it enables communities to focus on realizing specific aspects of NGDLE functionality. If these components adhere to the necessary standards, they should be able to be interconnected into a single structure. Second, the approach addresses the key needs for personalization by enabling it at a variety of levels, from the individual to the institutional.
One could argue that some of this is happening today. We see work being done in a variety of NGDLE functional areas. Some of the more conspicuous efforts that are taking shape include the following:
- Incorporation of mastery-based models. LoudCloud has released a CBE platform called FASTRAK and is partnering with the University of Florida's Lastinger Center. Instructure, which offers the Canvas LMS, announced its mastery gradebook a year ago.
- Content tools. EdCast seeks to create a network of institutions so that students can efficiently find course content. Acatar offers a learning platform that incorporates findings from learning science.
- An app store. Instructure's EduApp Center has a set of open, LTI-compliant apps that should be usable with any LMS that also supports the LTI standard.
- Recommender systems. For course content, Brightspace LeaP can recommend supplementary course materials in an automated fashion. D2L Degree Compass, originally developed at Austin Peay State University and now owned by Brightspace, makes course recommendations based on predictive analytics.
- Interoperability standards and APIs. IMS Global has many standards that potentially apply to the NGDLE, including Common Cartridge, LTI, EDUPUB, Access for All, QTI (Question and Test Interoperability), Learning Information Services, and Caliper Analytics. The Experience API is an example of an API that "makes it possible to collect data about the wide range of experiences a person has (online and offline)."11
- Accessibility standards. Relevant accessibility standards include Web Content Accessibility Guidelines (WCAG) 2.0 and the Authoring Tool Accessibility Guidelines (ATAG) 2.0.
- Adaptive technology. Leading efforts include McGraw-Hill's ALEKS, Smart Sparrow, Carnegie Mellon University's Open Learning Initiative, and Cerego.
- Additional initiatives. Blackboard is developing a new LMS UX, currently called Ultra. Google (Google Classroom and Google Apps for Education) and Mobiliya Edvelop (based on Microsoft's Office 365) have stepped into the LMS arena.
These projects and activities are useful initiatives, but they are disparate. Some are open, whereas others are proprietary. Their allegiance to standards is variable. These factors work against the coherence needed to create the NGDLE "Lego set." This suggests an opportunity to expedite the development of NGDLE componentry. If the equivalent of the Lego specification could be articulated for the NGDLE, it would serve as the basis for the confederation we propose. We are suggesting an NGDLE-conformant standard or specification, which would be based on adherence to a coordinated set of component standards. Once such a standard is in place, future investments and development efforts could be designed around the NGDLE specifications.
Whereas an NGDLE Lego specification will need to include the appropriate technical standards to achieve interoperability, it will also need to address the other functional domains identified above. It must include specifications that address accessibility and universal design. It must also feature specifications that speak to collaborations and work with peers. The articulation of the NGDLE specifications will need input from community members with a range of expertise.
Having this as a common frame of reference, we can imagine a host of additional areas for the development of new NGDLE components. Here are some examples:
- Learning environment architectures: A set of exemplary NGDLE architecture designs, which could serve as models for the community
- Smart tools: A set of learning-tool designs that explicitly incorporate learning science and universal design and are fully NGDLE compliant
- Learning measurement rubrics: A set of designs to effectively integrate new rubrics for learning measurement and degree progress (e.g., competency) into the NGDLE
- Success promotion: A set of apps and components that directly address the important opportunities and challenges associated with the realization of the NGDLE, using priorities identified by the higher education community
- Collaboration: A set of NGDLE resources to enable collaboration and peer work among learners
- IPAS design challenge: A set of designs that effectively use the NGDLE framework to leverage IPAS development
- Portfolio 3.0: A set of applications and platforms that provide learners with an integrated set of portfolio tools
- Everybody all at once: A set of NGDLE designs, based on universal design precepts, that can be used and built on by the community
- Dual enrollment: A set of communities that use exemplary practices to bridge high school and higher education, using a single platform and supporting dual enrollments and a seamless transition
- Mobile-first: A fresh exploration of mobile-first designs, specifically addressing the NGDLE
- Authoring tools: A set of tools that are NGDLE compliant and relevant to the workflows and typical tasks that students and instructors undertake
There is perhaps one last "functional domain" for the NGDLE—one that has nothing to do with the technology. The culture of higher education teaching and learning must evolve to encourage and even demand the realization of the NGDLE. We need to adopt "NGDLE thinking," whereby the five functional domains described above feel like a natural fit for any learning environment. We must, as a community, encourage one another to move culturally and socially into the NGDLE mind-set. Allowing evolution in our thinking about the nature, purpose, and conduct of higher education teaching and learning is one of the best ways to ensure the arrival of the NGDLE.
This article is drawn from the EDUCAUSE Learning Initiative (ELI) white paper The Next Generation Digital Learning Environment: A Report on Research (April 2015), produced in partnership with the Bill & Melinda Gates Foundation. Background on the research and the full paper can be found here.
- Eden Dahlstrom, D. Christopher Brooks, and Jacqueline Bichsel, The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives, research report (Louisville, CO: ECAR, September 2014), 4.
- Ibid., 3, 10.
- Torin Monahan, "Flexible Space and Built Pedagogy: Emerging IT Embodiments," Inventio 4, no. 1 (2002).
- Michael Feldstein, "Dammit, the LMS," e-Literate, November 10, 2014.
- "Mashup (web application hybrid)," Wikipedia, accessed April 21, 2014.
- Carl Straumsheim, "Lynn University to Drop Blackboard Learn in Favor of iTunes U," Inside Higher Ed, January 8, 2014.
- Unizin is a consortium with seven founding member institutions, created "to exert greater control and influence over the digital learning landscape." The Competency-Based Education Network, or C-BEN, is a consortium of colleges and universities collaborating "to address shared challenges to designing, developing, and scaling competency-based degree programs."
- See 1st International Conference on Learning Analytics and Knowledge (LAK), Banff, Alberta, Canada, February 27–March 1, 2011.
- Ronald Yanosky, Integrated Planning and Advising Services: A Benchmarking Study, research report (Louisville, CO: ECAR, March 2014), 4.
- See the Center for Universal Design, NC State University.
- "What Is the Tin Can API?"
© 2015 Malcolm Brown, Joanne Dehoney, and Nancy Millichap. The text of this article is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
EDUCAUSE Review, vol. 50, no. 4 (July/August 2015)