Beyond the MOOC Model: Changing Educational Paradigms

Jim Mazoué is an educational consultant.

If 2013 was the Year of the MOOC Backlash, is 2014 the Year of Capitulation? Those cheering for MOOCs to disappear from the higher education landscape certainly hope so. Emboldened by Sebastian Thrun's comment that "MOOCs are a lousy product,"¹ anti-MOOC hypesters and the blogosphere's "I-told-you-sos" are reveling in a schadenfreude moment as they anticipate the blowback escalating into full-scale retreat followed by a raised white flag. The precipitous rise and ignominious downfall of MOOCs fits the standard "MOOC saga" narrative that has emerged over the past year, which goes something like this:

"Once, MOOCs were all the rage due largely to irrationally inflated claims circulating in the popular media. However, more sober reflection eventually revealed what MOOCs really are — a brazen attempt by Silicon Valley venture capitalists to palm off a cheap imitation of a college education — and all the hype died down. We are now in a post-MOOC recovery."

Having exposed MOOCs as a "sham," hardline critics feel vindicated in their repudiation of online education for the masses as a failed and discredited experiment. Among traditionalists, the declaration in Forbes that MOOCs are "yesterday's news" is hailed as a fitting and long overdue epitaph. It is time, they say, to put our flirtation with faux education to rest and move on.

Has the notion of a MOOC-based education finally reached a dead end?

Well, not so fast. Like the announcement of Mark Twain's death, a victory dance on the grave of MOOCs and similar disruptions to location-dependent learning is premature. It is wishful thinking to believe that either the much hoped-for demise of MOOCs has occurred or, even if it were to occur, that it would give traditional education safe harbor from forces threatening to upend its conventional ways of doing business. As I describe here, even if the current garden variety of MOOCs were to vanish, four drivers of change will continue to pose a serious challenge to the status quo.

1. MOOC-Based Degrees

First, the notion that MOOCs are dead is an overly hyped exaggeration. The experiment is not over; in fact, it has just begun. We are now transitioning from a period of speculation to one of evidence gathering.² In terms of gauging MOOCs' impact on traditional education, perhaps the most important question has yet to be answered: Are MOOC-based degrees a viable alternative to degrees from residential campuses? Thanks to Georgia Institute of Technology's Online Master of Science in Computer Science (OMS CS), which enrolled its first students in January 2014, we will find out. As the first accredited MOOC-based program, Georgia Tech's initial offering serves as a test case for the feasibility of MOOC-based degrees.

Although most institutions are cautiously taking a wait-and-see stance toward MOOCs, many will quickly move off the sidelines if the OMS CS is successful. Among the success indicators are

• the scalability of large-enrollment online degree programs;
• whether academic rigor and student learning outcomes are comparable to on-campus programs;
• retention and completion rates; and
• the marketability and employer acceptance of the OMS CS degree.

According to Georgia Tech's recent survey, initial reviews from the first cohort of OMS CS students are positive: 93 percent recommend the program to others and nearly two-thirds said their experience exceeds their expectations. If data from the OMS CS show that MOOC-based degrees are viable, others will follow with an array of offerings that will compete directly with on-campus programs.

Some may quibble that the $6,600 OMS CS is not modeled on real MOOCs because of its price tag. However, this misses the larger point: namely, that a quality online degree offered at scale for a nominal or greatly reduced cost is a more attractive alternative for many students than an on-campus degree. In deference to purists who might balk at calling a degree program that charges tuition a MOOC, we can call it a MOD (for Massive Online Degree). Whatever we call it, it will be bad news for on-campus degree programs. With competition, we can expect a MOD's cost to go down; it is not unreasonable to think that it might go down to a negligible amount if cost recovery shifts from charging students for the acquisition of knowledge to a model based on learning assessment and credentialing. In the end, students — if we let them — will be the ones who decide whether a MOD's value outweighs the additional cost of an on-campus degree.

Apart from the prospect of the spread of MOOC-based degrees, however, the notion that MOOCs in general are languishing in a trough of disillusionment reflects a provincial perspective from within the anti-MOOC echo chamber. Far from fading into oblivion, data show that MOOCs are in fact increasing in global popularity.³ The case for dismissing MOOCs as an educational alternative, therefore, has yet to be made.

On closer analysis, the fundamental problem with MOOCs might not be MOOCs themselves but rather the underlying hostility among those campaigning against their mainstream acceptance. Like a Rorschach test that dredges up and projects unresolved conflicts, the MOOCs controversy might actually say more about us and our deep-seated attitudes toward educational change than it does about what is objectively true of MOOCs themselves.

2. Competency-Based Education

Although competency-based education (CBE) is not new, the recognition of its advantages over a time-bound model of credit-hour accumulation is gaining momentum. A consensus is emerging among both educators and the general public that the focus of education should be on what students know and are able to do, rather than on how long it takes them to do it.⁴ A clear majority of the American public, for example, favors changes to the credentials-for-credits model that has been used as the common currency of higher education for over a century: A 2012 Gallup/Lumina Foundation poll found that 75 percent of respondents were more likely to enroll in a postsecondary program if they could receive credit for what they already know.⁵ With few exceptions, students in traditional programs not only receive no credit for what they already know, but they are forced to learn within a system of arbitrary temporal limits and often move through a curriculum without sufficient mastery. This "bucket brigade" mentality serves only to compound student failure.⁶ As others have noted, students are not well-served by a system of "Swiss cheese achievement" that routinely passes them along to the next level without an adequate understanding of what they are expected to know.⁷

By comparison, CBE is more

• transparent, because it clarifies the connection between learning outcomes and their assessment;
• effective, because the process of achieving proficiency is based on sound pedagogical practices; and
• efficient, because students advance only upon demonstrating mastery of program-defined competencies.

CBE has the potential to reduce waste and inefficiency by crediting students for demonstrable proficiency without penalizing them for when, where, or how they acquired it. Among its advantages, CBE:

• Clarifies the identification and assessment of learning outcomes.
• Uses granular metrics for monitoring and improving incremental academic achievement.
• Accelerates progress to proficiency.
• Improves the alignment between program curricula and employable skills.
• Reduces the cost of education.⁸

CBE not only effectively enables individualized learning but shifts the overall power differential in education from institutions to students. Whereas institutions control the awarding of credit hours — and, by implication, the conditions that determine the awarding of academic credentials — CBE gives students control over their competency development. A principal way colleges exercise control over students and maintain their brand's selectivity is by using grades to sort and differentiate talent rather than develop it — a practice that is "ineffective and potentially harmful to students."⁹ None of the justifications commonly given to support either differentiation's benefits or the notion that grades enhance academic achievement are plausible; such justifications include the claims that grades reflect innate intelligence; competition for grades motivates students to learn; and grades accurately indicate what students know.¹⁰ If anything, grading is counterproductive because it reinforces and perpetuates academic stratification based on the unfounded belief that grades reflect inherent differences in students' abilities and aptitudes for learning. With greater personal responsibility for developing their own competencies, students find CBE empowering and a motivational impetus for continuous improvement in their learning.¹¹

3. The Formalization of Learning

Despite progress in educational research, educational praxis in the classroom remains wedded to folk pedagogical practices based on intuition and improvisation. As Graham Nuthall notes, "much of what we do in schools is a matter of cultural tradition rather than evidence-based practice."¹² Based on his findings from decades of research on classroom teaching and learning, Nuthall concludes that these traditions have led to "no discernable difference" between how experienced or novice teachers teach or what their students learn. "Being an experienced expert teacher apparently made no difference."¹³ This discomfiting conclusion is also supported by research from others showing that the factors that promote improvement in student learning depend on how subject matter knowledge is deployed, not by whom or by what. ¹⁴

 Nuthall's explanation for this seemingly counterintuitive conclusion is that much of the activity in classrooms — even its "focused busyness" — consists of behavioral routines concerned more with classroom management than genuine learning.¹⁵ According to Nuthall, classroom interactions are more scripted and less pedagogically fruitful than we might think. Both experienced and novice teachers alike rely on classroom routines based on "folklore" rather than on research-based practices, which results in a disconnect between what teachers think students are learning and the students' actual cognitive experiences. Because relying on ritualized classroom behavior creates the false impression that real learning is occurring, conclusions about what students are actually learning are unreliable.¹⁶ The popular notion that classroom teachers can recognize when and what their students are learning from telltale signs of "engagement" as they participate in brainstorming sessions or group discussion, for example, might be subjectively appealing, but it is often mistaken.

The Phenomenology of Teaching

We can refer to what Nuthall calls "folklore" as the phenomenology of teaching. If Nuthall's findings are correct, then confidence in the phenomenology of teaching as a method of obtaining insight into whether learning is occurring is largely misplaced; surface-level generalizations based on casual observation do not reveal the underlying causal processes that explain individual episodes of student learning. The observation that certain activities promote "engagement" or "interaction" is a folk pedagogical description, not an explanation of why certain instructional methods are effective or how they work. It would be fair to say by analogy that it is as prudent to rely on a phenomenology of teaching in eduation as it would be to rely on a phenomenology of healing in medicine.

In his work on mastery learning, Benjamin Bloom provides the underlying pedagogical rationale for Nuthall's insight into why the phenomenology of teaching lacks efficacy — namely, the observation that less instructional variation results in greater disparity in student achievement.¹⁷ Citing Bloom's research, Thomas Guskey points out that, given the differences among students' readiness to learn, there is an inverse relationship between instructional variation and learning outcomes: the less instructional variation, the greater the variation among student learning outcomes; conversely, the greater the instructional differentiation attuned to the actual learning needs of individual students, the less disparity in learning outcomes.¹⁸

The phenomenology of teaching contributes to disparities in student learning because the generic routines it employs are not aligned to the instructional needs of individual learners. Because a phenomenology of teaching lacks the precision and specificity necessary for instructional differentiation, its methods lack the instructional variation learners need. Instructional methods might vary from teacher to teacher but not, more importantly, from student to student. Folk pedagogical approaches that engage in undifferentiated instruction should therefore be abandoned because their lack of instructional variation not only cannot effectively remedy disparities in student achievement, they serve to perpetuate them.

Micro-Level Formalization

Research countering the phenomenology of teaching has been underway for decades; the effort is to formalize how students learn. The pedagogical justification for formalizing learning is that it creates individual pathways to proficiency. Because formalization provides granular insight into what a student knows and is ready to learn, it can create differentiated pathways to mastery within a knowledge domain. Using learning progressions, for example, has proven effective in identifying and closing individual achievement gaps.¹⁹ Effectively using formative assessment to identify and close gaps in understanding also depends on precisely locating where gaps occur in relation to defined learning outcomes.²⁰ "The formative assessment process," W. James Popham writes, "will be far more successful if teachers systematically collect evidence of a student's progress toward mastery of each key building block in a learning progression."²¹ Thus, compared to traditional approaches that rely on the phenomenology of teaching, sequencing learning into developmentally differentiated progressions can more effectively guide students through prerequisite steps that lead to mastery.

The formalization of learning is based on the assumption that adopting effective learning strategies and instructional methods should not be a happenstance occurrence, but rather reflectively adopted and systematically implemented to optimize each student's academic development. By modeling a knowledge domain's organizational structure, formalization helps learners incrementally build cognitive structures that serve as reference points for progressively modifying their conceptual schemas. Research on the differences between how novices and experts solve problems, for example, indicates underlying differences in how they use internalized cognitive models to organize and retrieve knowledge.²² As John Bransford and others note, "The key attribute of expertise is a detailed and organized understanding of the important facts within a specific domain."²³ Creating individualized pathways to proficiency within a curriculum better enables learning by providing:

• a detailed architecture of how a domain organizes knowledge,
• a coherent set of standards for assessing progress towards proficiency, and
• research-validated methods that optimize individual progress toward mastery.

A number of terms are used to describe models that organize and structure the conceptual and functional relationships among key ideas and principles that constitute a knowledge domain: knowledge map, progress map, knowledge space.²⁴ These models are something like an epistemic genome, that is, a map of a knowledge domain's conceptual structure. Such a construct constitutes as complete and comprehensive a representation of knowledge states within a domain as possible.

By analogy, we could use the term cognitive genome to refer to a learner's complete set of knowledge states. Mapping a learner's knowledge states at any given time would be analogous to sequencing the learner's cognitive genome in relation to a domain's optimally structured set of knowledge states. On this model, learning would be the process of cognitively mapping to an epistemic genome.

The cognitive mapping or sequencing process consists of four stages:

1. Constructing a knowledge map, that is, as complete a representation of a knowledge domain as possible.
2. Mapping a learner's cognitive states to a knowledge domain through instruction and assessment.
3. Assessing learner-response data to construct a progressively more adequate representation of the organizational structure of the learner's cognitive states.
4. Updating and recalibrating the learner's cognitive profile until an optimal congruence is achieved between the learner's knowledge states and a knowledge domain.

Throughout this process, the objective is learning with understanding and developing the ability to know how and when to apply one's knowledge.²⁵ Appropriately applying one's knowledge would be analogous to demonstrating proficiency in navigating a conceptual terrain — not simply by following directions —but in virtue of having constructed an internalized guidance system modeled on a domain's epistemic structure. Not using a learner's cognitive genome as a more precise model to map and guide their learning — and detect and debug errors — would be like ignoring genomic information for medical diagnosis and treatment.

Admittedly, the amount of detail and complexity involved in the formalization of learning that this model envisions poses a daunting Grand Challenge problem. The extent to which such formalization is possible is an empirical question, and it might be impossible to achieve in disciplines that favor creativity and a diversity of perspectives over consensus. Even in high-consensus disciplines amenable to formalization, compiling and evaluating extensive data from studies that replicate and validate the instructional methods' effectiveness will prove difficult. The alternative, however, is even less palatable: conceding the very possibility of understanding how the learning process works by treating it as something irreducibly mysterious and inscrutable.

Although attempts to improve learning through formalization are challenging, the results thus far are nevertheless modestly successful. For example, meta-analyses of the use of computer-aided instruction in mathematics and statistics and evidence-based research in physics education consistently show a moderately positive effect size.²⁶ Kurt VanLehn's survey of the research literature indicates that well-designed intelligent tutoring systems in STEM disciplines have even greater effect sizes that might potentially approximate Bloom's Two Sigma.²⁷ The claim that formalization can improve learning, therefore, is based on more than just faith in a remotely possible Utopian future. By applying the learning sciences, empirical support is growing for the feasibility of constructing cognitive maps in at least some knowledge domains that can effectively guide students along optimal learning pathways toward mastery.²⁸ A process of formalization based on the scientific method, not the phenomenology of teaching, is the most reliable way to conduct a systematic assessment of pedagogical methods and identify practices that improve learning.²⁹

Macro-Level Formalization

As formalization delineates micro-level pathways that more effectively diagnose and guide student learning, support is also growing for the specification of macro-level standards that articulate the qualifications associated with discipline-specific credentials. Qualifications frameworks such as the Degree Qualifications Profile (DQP) and the Tuning process [http://tuningusa.org/] are gradually building agreement around standards that serve as normative benchmarks of what students should be learning. The DQP specifies a common set of outcomes and competencies that describe what students should know and be able to do upon the completion of a degree. Tuning establishes core competencies and learning outcomes at a discipline-specific level. Both approaches demonstrate the practicality of crafting commonly accepted standards that define the knowledge and skills that a credential should represent.

Although those leading the DQP and Tuning initiatives make it clear that they are not attempting to standardize curricula,³⁰ there is no reason why the resulting qualifications frameworks could not be used to define a curriculum's core components. Reaching consensus within a discipline on student learning outcomes at different degree levels can be viewed as a prerequisite to learning's formalization and the validation of institution-neutral proficiencies. Eventually, a seamless coordination of standards-based qualifications in higher education could be aligned with K–12 Common Core State Standards (CCSS) to create a more cohesive model of academic progress across the educational spectrum.³¹

4. Regulatory Reform

Pedagogical and curricular innovations alone are unlikely to bring about change without regulatory reform. Regulatory barriers favor incumbent higher education institutions and give them a competitive advantage over new entrants. The result? "Higher education," Andrew Kelly and Frederick Hess point out, "functions more like a cartel than a dynamic marketplace."³² Accreditation, state authorization regulations for distance education, and federal financial aid eligibility rules have contributed to a "cartelization" that both protects entrenched institutional interests and preemptively stifles innovation.³³

Recent actions by the Obama Administration and U.S. Department of Education, however, could signal a rollback of protectionist policies and movement toward regulatory reform.³⁴ Redefining the credit-hour to include the "amount of work represented in achieving learning outcomes"³⁵ is a significant policy change that has helped level the playing field for CBE. In a more hospitable regulatory environment, the pace of change would likely accelerate among some of the principal drivers of innovation: MOOC-based degrees, CBE, learning-optimized courseware, federal financial aid for learner support services, micro-credentialing, and the use of performance indicators to reward institutions that improve student outcomes. These innovations and reforms might put colleges in an increasingly untenable position similar to that of other industries that bundle services in a closed, fee-based system.

For example, the growing interest in video-on-demand (VOD) services that are challenging the hegemony of commercial broadcasting and cable TV conglomerates could be a bellwether of higher education's future. Notwithstanding the U.S. Supreme Court's recent 6–3 decision against Aereo, differently structured start-ups such as Simple.TV are providing user-defined services to consumers who are abandoning cable and satellite TV in favor of media streaming via personal devices.³⁶ Data show that "cord cutters" and "cord nevers" are increasingly younger adults who are foregoing subscription-based television in favor of free or low-cost video streaming via mobile devices and digital media players such as Google's Chromecast.³⁷ Colleges that engage in the credit indenturing of students by bundling and tethering their services to closed forms of institutional dependence might face a similar fate from "campus-cutters" — students who control not only what and when they learn, but how and from whom they learn it.

Clearly, efforts to erect protectionist barriers to innovation are not unique to education.³⁸ In health care, for example, a similar obstructionist stance toward innovation has blocked access to consumer genomics.³⁹ The medical establishment's opposition has halted the sale of tests that identify genetic risk information that individuals could use to take preemptive action in preventing a disease or illness to which they are genetically predisposed. U. S. Food and Drug Administration regulations currently prevent consumers from purchasing genomic tests for diagnostic purposes on the grounds that the public cannot be trusted to make important medical decisions for themselves.⁴⁰ Some experts predict, however, that consumer interests — not the medical establishment — will ultimately prevail and open wider access to genomic medicine as a new and more effective diagnostic and treatment paradigm.⁴¹ The lifting of protective regulatory barriers in education might similarly expand new options for students.

Colleges are also facing competition, not only from new entrants into the educational market, but also from feeder schools such as two-year colleges that have traditionally funneled transfer students into four-year institutions. In some states, community colleges are now receiving legislative authorization to offer four-year degrees in select subject areas; the cost is about one-fourth of that for a public four-year college degree. Such colleges are also dropping the term community from their name and rebranding themselves as simply colleges. Moving up-market in this way is a predictable outcome of Clayton Christensen's theory of disruptive innovation and further proof that higher education institutions are subject to the same disruptive forces as other businesses. How they respond could be critical to their survival. As Christensen points out, incumbents that defend against disruption by doubling down on the services they provide by adding more value (and cost) to differentiate their sustaining business model from peer competitors almost always fail in competition against innovators.⁴² For at least some colleges — especially those already on a financially unsustainable path — adopting this tactic could precipitate an institutional death spiral.⁴³

An even more radical alternative to regulatory reform that educational innovators might find effective is to work outside the existing regulatory framework altogether.⁴⁴ One way to accomplish this would be for nontraditional providers to work directly with employers to establish their services' credibility as evidence of workforce preparation, thereby undermining the rationale for seeking students with degrees from accredited institutions. The ultimate act of disintermediation might be to eliminate the need for regulatory oversight entirely through an alternate process that directly verifies employable credentials.

Changing Paradigms

It is ironic that at a time when higher education is seemingly more interconnected than ever, deep divisions exist over how best to carry out the core mission. The recent rancor over MOOCs reveals just how sharply divided the disagreement is between those who wish to preserve educational orthodoxy's traditions and those who seek to abandon legacy practices in favor of more progressive alternatives. What MOOCs and other emergent trends might signify is a developing schism within education between traditionalists and progressives.

• Traditionalists see the path to the future through the past. A good education, in their view, essentially requires a process of personal formation in close quarters — a process that is distorted by technology-mediated forms of distance learning.
• Progressives view learning-enabling technologies as part of education's evolution as a science. They view new models of location-independent learning as having a transformative impact that cannot be assimilated into conventional practices without compromising their effectiveness.

Finally, a third group of outliers elects to remain outside the fray. They are off-put by constant talk of disruption and weary of predictions about how so-called "innovations" will change — for better or worse — our educational future. Many are suffering from MOOC fatigue; they are tired of superficial bickering over what they view as simplistic and artificially contrived dichotomies. For this third group, being presented with a stark choice between tradition and progress is a false alternative. Because they believe there is some truth to the claims of both sides, they aspire to a middle ground of compromise and reconciliation. From the outlier perspective, we should break the logjam of conflict by simply finding common ground between educational traditionalists and progressives.

However, as much as we might desire a reconciliation between these divergent orientations, the trends of MOOC-based degrees, CBE, formalization of learning, and regulatory reform could indicate the early onset of paradigm change. If so, the divisions are far from trivial. Rather, such divisions are precursors of fundamental change in educational practice signaling several key shifts, from:

• Intuition to science
• Place to process
• Time to competency
• Handcrafted instruction to the formalization of learning
• Dispositional to structural explanations of learning
• Undifferentiated instruction to personalized learning

Given that teaching and learning are complex, evolving practices, it should be neither surprising nor unwelcome when disagreement emerges over their nature and purpose. Glossing over discord as either misguided or superficial trivializes the very possibility of having a substantive dialogue about methodological differences that underlie opposing views. Rather than being divisive and adversarial, such conflict can serve as an opportunity to reflect on why we are committed to a particular ideological path. A rift over which path to pursue can be useful, therefore, in clarifying whether we are headed toward a dead end or moving toward the fuller realization of a desired educational destination. If the paths of traditionalists and progressives represent a fork in the road that leads in divergent directions, we must choose which of the two to follow — or blaze an entirely new trail.

It may well be that our arriving at a settled judgment about which course to take has a long way to go before catching up with a changing reality.

1. Max Chafkin, "Udacity's Sebastian Thrun, Godfather of Free Online Education, Changes Course," Fast Company, December 2013/January 2014.
2. Recent examples include the Bill & Melinda Gates Foundation's MOOC Research Initiative Carnegie Mellon University's Simon Initiative and Google Focused Research Award, and the Ithaka S+R report, "Interactive Online Learning on Campus: Testing MOOCs and Other Platforms in Hybrid Formats in the University System of Maryland," July 10, 2012.
3. See, for example, the European Commission's MOOCs Scoreboard [http://openeducationeuropa.eu/en/european_scoreboard_moocs] and the Global MOOC Stats [https://magic.piktochart.com/output/1747660-moocs] infographic; and Julia Sufrin, "Report: Global MOOC Market Continuing to Grow," Campus Technology, July 22, 2014.
4. Amy Laitenin, "Cracking the Credit Hour," New America Foundation, September 5, 2012.
5. Sally M. Johnstone and Louis Soares, "Principles for Developing Competency-Based Programs," Change: The Magazine of Higher Learning, March/April, 2014.
6. Lumina Foundation and Gallup, Inc., "America's Call for Higher Education Redesign," February 5, 2013, 8.
7. Camille A. Farrington and Margaret H. Small, "A New Model of Student Assessment for the 21st Century," [http://www.aypf.org/documents/ANewModelofStudentAssessmentforthe21stCentury.pdf] American Youth Policy Forum, 2008, 4.
8. Chris Sturgis, "Progress and Proficiency: Redesigning Grading for Competency Education," CompetencyWorks, January 2014: 10.
9. "Experimental Sites Concept Paper: Competency-Based Education," a joint response to the U.S. Department of Education's request for comments by 17 colleges and universities, January 2014.
10. Thomas R. Guskey, "Five Obstacles to Grading Reform," Educational Leadership, vol. 69, no. 3 (November 2011): 19.
11. Emily Richmond, "Student-Centered Learning," Stanford Center for Opportunity Policy in Education, April 2, 2014.
12. Laura Shubilla and Chris Sturgis, "The Learning Edge: Supporting Student Success in a Competency-Based Learning Environment," CompetencyWorks, December 2012.
13. Graham Nuthall, "The Cultural Myths and the Realities of Teaching and Learning," The Jean Herbison Lecture, New Zealand Association for Research in Education, December 2001, 1.
14. Ibid., 3.
15. Robert J. Friedrich and Stanley J. Michalak, Jr., "Why Doesn't Research Improve Teaching?," The Journal of Higher Education, vol. 54, no. 2 (1983): 145–163; John Hattie and H. W. Marsh, "The Relationship Between Research and Teaching: A Meta-Analysis," Review of Educational Research, vol. 66, no. 4 (1996): 507–542; John Hattie and Herbert W. Marsh, "The Relation between Research Productivity and Teaching Effectiveness: Complementary, Antagonistic, or Independent Constructs?," Journal of Higher Education, vol. 73, no. 5 (2002): 603–641; Margaret Heritage Jinok Kim, Terry Vendlinski, and Joan Herma, "From Evidence to Action: A Seamless Process in Formative Assessment?," Educational Measurement: Issues and Practice, vol. 28, no. 3 (Fall 2009): 24–31; Louis Deslauriers, Ellen Schelew, and Carl Wieman, "Improved Learning in a Large-Enrollment Physics Class," Science, vol. 332, no. 6031 (2011): 862–864; Ittima Cherastidtham, Julie Sonnemann, and Andrew Norton, "The Teaching-Research Nexus in Higher Education," Grattan Institute, October 2013.
16. Graham Nuthall, "The Cultural Myths and the Realities of Teaching and Learning," 16: "Being busy is not a cause of learning unless you know exactly what information or knowledge the student is getting out of being busy." In How People Learn: Brain, Mind, Experience, and School: Expanded Edition (National Research Council, 2000, 24), John D. Bransford, Ann L. Brown, and Rodney R. Cocking note that there is an important distinction between "hands-on-doing" and "doing with understanding." Engagement is not "the primary index of successful teaching," but rather a knowledge-centered environment that promotes learning with understanding. It is also worth noting that the most pervasive form of learning management system — the classroom — escapes the notice of digital LMS critics. As a medium for scripted behavior, classrooms serve as venues in which students and faculty engage in a "giant game of favor exchanges" (see Jeffrey J. Selingo, College (Un)bound: The Future of Higher Education and What It Means for Students, New Harvest, 2013, 21).
17. Graham Nuthall, The Hidden Lives of Learners, NZCER Press, Wellington, New Zealand, 2007.
18. Benjamin S. Bloom, "Learning for Mastery," Evaluation Comment, vol. 1, no. 2 (1968): 1–11.
19. Thomas R. Guskey, "Closing the Achievement Gap: Revisiting Benjamin S. Bloom's 'Learning for 'Mastery'," Journal of Advanced Academics, vol. 19, no. 1 (2007): 8–31.
20. Margaret Heritage, "Learning Progressions: Supporting Instruction and Formative Assessment," The Council of Chief State School Officers, Washington, DC, 2008.
21. John Hattie and Helen Timperley, "The Power of Feedback," Review of Educational Research, vol. 77, no. 1 (2007): 81–112.
22. W. James Popham, "All About Accountability/The Lowdown on Learning Progressions," Educational Leadership, April 2007, 83.
23. Bransford et al., How People Learn, 237–238; Paul A. Kirschner et al., "Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching," Educational Psychologist, vol. 41, no. 2 (2006): 75–86.
24. Bransford et al., How People Learn, 239.
25. Jean-Paul Doignon and Jean-Claude Falmagne, Knowledge Spaces, Springer-Verlag, 1999; Jean-Paul Doignon and Jean-Claude Falmagne, Learning Spaces, Springer-Verlag, 2011; Jean-Paul Doignon and Jean-Claude Falmagne, "Spaces for the Assessment of Knowledge," International Journal of Man-Machine Studies, vol. 23, no. 2 (1985): 175196; Jean-Claude Falmagne, Eric Cosyn, Jean-Paul Doignon, and Nicolas Thiery, "The Assessment of Knowledge in Theory and in Practice," Formal Concept Analysis, LNCS 3874, Springer-Verlag, 2006, 61–79.
26. Bransford et al., How People Learn, 2000, 137.
27. Carl Wieman, "Why Not Try a Scientific Approach to Science Education?," Change: The Magazine of Higher Learning, September/October 2007; Rana M. Tamim, Robert M. Bernard, Eugene Borokhovski, Philip C. Abrami, and Richard F. Schmidt, "What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study," Review of Educational Research, vol. 81, no. 1 (March 2011): 4–28; Giovanni W. Sosa et al., "Effectiveness of Computer-Assisted Instruction in Statistics: A Meta-Analysis," Review of Educational Research vol. 81, no 1 (March 2011), 97–128; and Saiying Steenbergen-Hu and Harris Cooper, "A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on College Students' Academic Learning," Journal of Educational Psychology, vol. 106, no. 2 (May 2014): 331–347.
28. Kurt VanLehn, "The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems," Educational Psychologist, vol. 46, no. 4 (2011): 197–221.
29. Practical applications of knowledge-mapping and system-generated guidance are emerging in the learning analytics capabilities embedded into publisher-produced digital learning systems; examples are Pearson's partnership with Knewton and McGraw-Hill's recent acquisitions of ALEKS and Area9.
30. Among those calling for the application of the scientific method to improve teaching are James M. Fraser et al., "Teaching and Physics Education Research: Bridging the Gap," Reports on Progress in Physics vol. 77 (2014): 1–17; Jane Jackson, Larry Dukerich, and David Hestenes, "Modeling Instruction: An Effective Model for Science Education," Science Educator, vol. 17, no. 1 (2008): 10–17; and the Online Learning Initiative's blog dedicated to Herbert Simon.
31. Cliff Adelman, Peter Ewell, Paul Gaston, and Carol Geary Schneider, The Degree Qualifications Profile 2.0, Lumina Foundation, January 2014, 8.
32. David T. Conley and Paul L. Gaston, A Path to Alignment: Connecting K-12 and higher education via the Common Core and the Degree Qualifications Profile, Lumina Foundation, October 2013.
33. Andrew P. Kelly and Frederick M. Hess, Beyond Retrofitting: Innovation in Higher Education, Hudson Institute, June 14, 2013, 10.
34. Ibid., 3.
35. The White House, "Fact Sheet on the President's Plan to Make College More Affordable: A Better Bargain for the Middle Class," August 22, 2013; President's Council of Advisors on Science and Technology, Report to the President, Office of Science and Technology Policy, December 18, 2013.
36. U.S. Department of Education, "Program Integrity Issues; Final Rule," Federal Register, vol. 75, no. 209 (October 29, 2010): 66846.
37. Cecilia Kang, "Why So Many Want Aereo to Beat Broadcasters in the Supreme Court," The Washington Post, April 3, 2014; Pema Levy, "Aereo Supreme Court Case Could Change How Americans Watch TV," Newsweek, April 21, 2014.
38. "Cross-Device Video Analysis," Experian Marketing Services, 2014, 6.
39. Another recent example of industry opposition to policies that protect consumer interests at the expense of corporate profit is the reluctance to install a "kill switch" on cell phones to deter theft; see, for example, Brian X. Chen, "Carriers Reject a 'Kill Switch' for Preventing Cellphone Theft," Bits Blogs, New York Times, November 19, 2013.
40. Eric Topol, The Creative Destruction of Medicine, Basic Books, 2013.
41. Christina Farr, "UPDATE 1-Gene Startup 23andme Casts Eyes Abroad after U.S. Regulatory Hurdle," Reuters, May 7, 2014.
42. Topol, The Creative Destruction of Medicine, 114.
43. Clayton Christensen, Disrupting Class: How Disruptive Innovation Will Change the Way the World Learns, McGraw Hill, 2008, 50; Clayton Christensen, "Key Concepts: Disruptive Innovation," Clayton Christensen Institute for Disruptive Innovation, 2012.
44. Jeff Denneen and Tom Dretler, "The Financially Sustainable University," Bain & Company, 2012; see also, Emily Schwarz, "One-Third of US Colleges Facing Falling or Stagnant Tuition Revenues," Moody's Investor Service, January 10, 2013; and Richard Vedder, "The Higher-Ed Bubble Starts to Pop," Minding the Campus, May 4, 2014.
45. Michelle R. Weise, "Disrupting College from the Margins: Innovations in Lifelong Learning," keynote address delivered at the 10th Anniversary Annual Meeting of the Presidents' Forum, 2014.