Bigfoot, Goldilocks, and Moonshots: A Report from the Frontiers of Personalized Learning

More than four years ago, a report from the National Center for Public Policy and Higher Education and Public Agenda noted: "In the view of many college and university presidents, the three main factors in higher education—cost, quality, and access—exist in what we call an iron triangle. These factors are linked in an unbreakable reciprocal relationship, such that any change in one will inevitably impact the others."¹ In other words, every positive improvement comes with an equal, but negative, trade-off. Unfortunately, that "iron triangle" remains strong, encapsulating a challenge that continues to face higher education today.

Fortunately, the resolve and creativity of higher education innovators are producing a set of solutions that have the potential to break the "iron triangle." These solutions are not theoretical; they are reaching hundreds of thousands of students today. The detailed results are still emerging, but the initial results suggest we may be able to deliver high-quality education at an affordable price without sacrificing access. I refer to the three categories of innovations as Bigfoot, Goldilocks, and Moonshots—and will explain each of them below.

The Challenge

The first angle of the "iron triangle" is access. In the United States, we've actually done a very good job of promoting access to higher education. The last three decades have seen steady enrollment growth, and we've seen more low‑income students and first-generation students pursuing their education beyond high school. However, change is underway in terms of the types of students pursuing postsecondary education and whether there are seats for them when they arrive. Close to 75 percent of college students are now defined as "non-traditional" by the U.S. Department of Education: they are financially on their own, they have dependents of their own, they work full-time, and/or they are enrolled in a college or university part-time.² Access for these students is increasingly at risk. In California community colleges, for example, an estimated 470,000 students were put on waiting lists in the fall of 2012.³ Perhaps not surprisingly, higher education enrollments are dropping in some states, mostly in community colleges but in some four-year institutions as well.⁴ In addition, we have "hidden" capacity constraints (see Figure 2). A fall 2010 survey of nearly 1,500 community college students reported that nearly one in five had difficulty getting into required courses.⁵ So, even if students can get into college, their courses may not be available, leading to excess credits and longer time to degree.

The second angle of the "iron triangle" is quality. Looking at completion rates as one measure of quality, we see that 58 percent of first-time students who enroll in a four-year college or university will earn a credential within six years, whereas 30 percent of students who enroll in two-year colleges will complete in three years.⁶ That statistic varies by type of institution, not surprisingly, but overall students have just better than a 50-50 shot of reaching the goal of a degree. Digging a little deeper, we see that over 50 percent of community college students, and almost 20 percent of four‑year students, are starting college unable to do college-level math, writing, or reading.⁷ Finally, when students are in college, what are they learning? According to "Academically Adrift," 45 percent of students "demonstrated no significant gains in critical thinking, analytical reasoning, and written communications during the first two years of college."⁸ Meanwhile, employers note that graduates are not well-prepared in "global knowledge, self‑direction, writing, and critical thinking"⁹ (see Figure 3).

Lastly, the third angle of the "iron triangle" is cost. Tuition and fees have risen more than 400 percent since the early 1980s, nearly twice the pace of the increase in health care costs and about four times the rate of inflation.¹⁰ A key problem is that state spending on public higher education is declining—dropping an average of 13 percent from 2006 to 2011.¹¹ The prospect of state funding recovering as the U.S. economy improves is muted by states increasing long-term commitments to health care, prisons, and K‑12 education. Thus, in many ways, tuition increases are being driven by state cuts. The net effect is that students are shouldering more of the load. In early 2010, the nation's total student loan debt surpassed credit card debt, and is now three to four times what it was a decade ago.¹²

How ready are we to deal with the iron triangle of cost, quality, and access? It might be surprising to hear that I think the news is actually good. Some exciting examples in personalized learning are quite promising. People are taking these challenges very seriously and are addressing them systemically, with thousands of students. These innovative strategies fall into three groups: Bigfoot, Goldilocks, and Moonshots.

Bigfoot

In the Pacific Northwest, we call Bigfoot by the name Sasquatch. But whatever you want to call him, Bigfoot represents our search for a mythical thing that many people believe exists, but where we lack indisputable evidence. In higher education, I would translate this question as: "Can we really find a way to produce undeniably better learning outcomes at consistently lower cost?" If we can indeed do that, we can start to unlock new ways of allocating our resources for student success.

I think we will soon be able to answer this question with "yes." We have a few plaster foot castings and some grainy photos and a little video. So while I'm not ready to declare that our Bigfoot exists, I think the evidence is promising.

This evidence comes from people such as Bill Bowen, former president of the Andrew W. Mellon Foundation and of Princeton University, who has stated: "I have come to believe that 'now is the time'—that far greater access to the internet, improvements in internet speed, reductions in storage costs, and other advances have combined with changing mindsets to suggest that online learning, in many of its manifestations, can lead to good learning outcomes at lower cost."¹³ We are starting to see examples that the technology has matured enough, that the solutions are there, that we really can deliver quality learning outcomes at an affordable price. This is not about technology first, however; it's about pedagogy and what we know about learning.

In 1984, the educational psychologist Benjamin Bloom compared traditional classroom instruction mastery-based learning, and one‑on‑one tutoring (also with mastery-based learning). Student achievement in one‑on‑one tutoring was two standard deviations higher than the traditional classroom instruction.¹⁴ So we've known for decades that traditional classroom instruction is not the most effective way to learn. We just have not been able to afford to do anything different.

How do we close the learning feedback loop to reach the achievement of one-on-one tutoring? How do we determine what students intend to learn and represent that in terms of a unique student profile that is individual to each student? How do we provide the right type of learning experience and environment for each student and use assessment to figure out if we are making progress and what else the student might still need to learn?

Let me give some examples of people and institutions doing just this. One is Carnegie Mellon University's Open Learning Initiative (OLI). OLI uses a "learning curve" that shows the number of hints students needed or wrong answers given in their digital tutor versus the number of opportunities they had to demonstrate competency in a particular skill. If the number of wrong answers remains high as the number of opportunities increases, it means the student isn't mastering the concept being taught. At this point, faculty assess how they can best help. Faculty time with students is probably one of the most precious resources in higher education. How can we use that resource most efficiently and effectively? Faculty time should be spent shoring up concepts that students aren't understanding, as shown by the learning curve, or on the higher-order thinking skills that can't be captured in digital tutors.

In May 2012, ITHAKA S+R published the results of a study looking at Carnegie Mellon's OLI courseware in a blended or hybrid format and in a traditional format at six public universities. They looked to see if there was any difference in outcomes. They concluded: "We find that learning outcomes are essentially the same—that students in the hybrid format 'pay no price' for this mode of instruction in terms of pass rates, final exam scores, and performance on a standardized assessment of statistical literacy." In addition, they noted that "adopting hybrid models of instruction in large introductory courses" offers opportunities "to significantly reduce instructor compensation costs in the long run." This suggests that we may be getting close to better learning outcomes for a more affordable price.¹⁵

Another example is the work of Carol Twigg and the National Center for Academic Transformation (NCAT). In the NCAT Changing the Equation Initiative, 32 two-year institutions redesigned their entire developmental math sequence (involving approximately 100,000 students) using NCAT's Emporium Model and commercially available instructional software. As a result, most of the institutions experienced increased student learning, improved course progression, and reduced instructional costs (by about 20 percent). What this means is that one-size-fits-all developmental education may soon be an artifact of the past.¹⁶

A third example is the National Repository of Online Courses (NROC) from the Monterey Institute for Technology and Education. NROC is developing interactive engaging content and making it available in a growing library of free high-quality online course content for students and faculty worldwide involved in higher education, high school, and Advanced Placement courses.

Finally, in August 2011 Arizona State University partnered with Knewton to customize learning for 5,000 students taking remedial math. After just one semester, withdrawal rates dropped from 13 percent to 6 percent, pass rates increased from 66 percent to 75 percent, and half of the students finished their class four weeks early.¹⁷

To date, most Bigfoot sightings have occurred in developmental education, where teachers have to get students ready for college-level work as quickly as possible. What about college-level courses? Resistance is higher in these courses, which are driven more by individual faculty. One development that could push this type of instructional model forward may be MOOCs (massive open online course)—not the current version of MOOCs but perhaps version 2.0 or 3.0. The interactive and social resources in MOOCs could make them an effective tool for flipping the classroom. In fact, early results from a partnership between edX and San Jose State University showed that the pass rate for one course increased from 60 percent to 91 percent when the course was "blended" with a MOOC.¹⁸

Goldilocks

Next up is Goldilocks. Remember her? Not too hot, not too cold; not too big, not too small; not too soft, not too hard. She wanted everything "just right." Actually, she wanted just the right choice for her. In higher education, these are the structured pathways to graduation with some choices—but not too many—for students.

Students have a mind-boggling number of pathways to choose from, including credit hours for the four-year track, the five-year track, or the six-year track¹⁹ (see Figure 4). And courses: think about a course catalogue and the number of permutations and combinations that are possible within it. Now imagine that you are a first-generation student who doesn't have someone to help guide you through that maze, or that you attend an institution that has about a 1,000-to-1 student-to-advisor ratio. It's no wonder many students aren't getting degrees. More than half of courses taken at a community college do not contribute to a degree. These are courses that students drop, courses that are over students' credit limit, and courses that students fail and have to retake.²⁰

Institutions like The City University of New York (CUNY), Georgia State University, Florida State University, Arizona State University, and Tennessee's Tech Centers are leading the way. For instance, Arizona State University's eAdvisor provides "online advising and personalized student support" for degree planning. It helps students understand the various pathways available in their majors and the critical requirements they must meet for graduation. If a student enrolls in a particular course, the system might note, "You just moved yourself from the four-year plan to the five-year plan. Are you sure you want to do that? You might want to go see an advisor." The system knows which courses students need to pass for each major and it knows which courses cannot be failed twice. The result is that every student gets a degree plan, and every student is on a path to completion.

The eAdvisor system also helps ASU know—three semesters out—how many class sections it will need, which means that the university can be much more efficient with classroom space and faculty time. In addition, once the university knows who is on which path, it can use predictive analytics to determine which students will need help staying on the path. As a result, ASU went from 22 percent of students "on track" in their programs in 2007 to 91 percent of students on track in 2010. The university has increased its first-year retention by about 8 percentage points and improved its utilization of classrooms and faculty.²¹ But doing so wasn't easy and took many years.

I view choice like a college green in the middle of a traditional campus. There are hundreds of possible ways to cross that green, but generally only four or five paved pathways across, because those are the most efficient ways to get from the dining hall to the dorm or from the library to the classroom. We need to expose the "paths" to completion for our students. We need to help them get on those paths, we need to intervene if they get off-track, and we also need to let them opt out if their chosen path is not a good fit. This means taking out the guesswork. Social scientists have found that people do best when presented with seven to nine choices. Having only one or two choices can feel constraining, but having more than nine choices tends to be overwhelming and paralyzing.

That is the Goldilocks model: not too many choices, not too few, but a number that is "just right" for students.

Moonshots

On September 12, 1962, President John F. Kennedy said: "We choose to go to the moon. We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard." I like to edit his words slightly and say: "We choose to provide high-quality, affordable education at scale. We choose to provide high-quality, affordable education at scale in this decade, not because it is easy, but because it is hard."

Much of what is emerging in higher education represents new programmatic and institutional models that are combining different technology solutions, pedagogical approaches, and learning science to reimagine the teaching and learning moment and to reimagine a student's progression through his or her degree program. But they're also using these innovations to reimagine how they organize their institutions to support those changes. There's even a willingness to reimagine how we fund the institutional model. Fundamentally rethinking how we provide education for a significant portion of the higher education student body will not be easy. There are no shortcuts or silver bullets. In fact, it is very, very hard work — but it is imperative that we try.

The Bill & Melinda Gates Foundation has partnered with EDUCAUSE, the League for Innovation in the Community College, and others in the Next Generation Learning Challenges. We recently concluded the third wave of that program, which focused on breakthrough models. We are challenging institutions to meet the following target performance metrics:

• Completion Rate: 50 percent Associate degree completion rate within three years for Pell Grant–eligible students and a 75 percent rate within six years for Bachelor degrees for Pell students
• Price and Cost: $5,000 or less per year in student price and institutional costs
• Scale: 5,000 additional students by year five with path to 50,000 students
• Quality: Clear definition and monitoring process

Participating institutions include Altius Education, Ameritas College of Brandman University, Kentucky Community and Technical College System, New Charter University, Northern Arizona University, Rio Salado College, Southern New Hampshire University, Texas Higher Education Coordinating Board, University System of Georgia, and University of Washington. We've also worked directly with The City University of New York, Portmont College at Mount St. Mary's, University of the People, and Western Governors University, among others. For many of these institutions, this represents innovation from within. They are saying: "We need to try something different. Let's create a competency-based nursing program. Let's try an accelerated degree-completion program."

Some common features of these models include prior learning assessment; competency-based progression; diagnostics assessment and adaptive learning; badges, interim milestones, and motivational science; connective media and peer-to-peer learning; learning analytics and targeting scarce faculty and support resources; and/or online/blended delivery. No one is using all of these, and no one has figured out the ideal formula. Yet everyone is using at least half of these features.

The innovators in this last group are shooting for the moon, not because it is easy, but because it is hard.

Conclusion

Bigfoot, Goldilocks, and Moonshots represent the three leading groups of innovative strategies: (1) growing evidence that we are finding ways to consistently deliver better learning outcomes at lower costs; (2) "just right" structured pathways to help students reach their goals while balancing student choices with intrusive advising; and (3) entire programs that are shooting for the moon with fundamentally different delivery and cost structures.

Can we use these strategies to break the "iron triangle" of cost, quality, and access, and can we use technology to improve learning and student outcomes? Doing so will require a grand partnership between those who are focused on changing the finance side of higher ed and those who are focused on improving what happens in the classroom. We need to engage both promoters of innovation and protectors of quality. We must reimagine the instructional model and also encourage deep faculty engagement—high tech combined with high touch. We need to think not only about design and research and development, but also about scaling. Finally, we need both traditional institutions and the breakthrough models on the frontier.

Josh Jarrett talks about the spread of breakthrough models for student success and how these new ideas might just change our fundamental approach to education.

1. John Immerwahr, Jean Johnson, and Paul Gasbarra, "The Iron Triangle: College Presidents Talk about Costs, Access, and Quality," a report from the National Center for Public Policy and Higher Education and Public Agenda (October 2008),.
2. Paul Attewell, "The Other 75%: Government Policy and Mass Higher Education" (unpublished paper, 2008).
3. Carla Rivera, "Survey Offers Dire Picture of State's Two-Year Colleges," [http://articles.latimes.com/2012/aug/29/local/la-me-0829-college-survey-20120829] Los Angeles Times, August 29, 2012.
4. Richard Vedder, "Five Reasons College Enrollments Might Be Dropping," Bloomberg.com, October 22, 2012.
5. Pearson Foundation Community College Student Survey: Summary of Results.
6. "The Condition of Education 2012," Indicator 45, Table A-45-1 (p. 278) and A-45-3 (p. 280), U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary Education Data System.
7. Complete College America, "Remediation: Higher Education's Bridge to Nowhere," 2012.
8. Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College Campuses (Chicago: University of Chicago Press, 2011).
9. Association of American Colleges and Universities, "How Should Colleges Assess and Improve Student Learning? Employers' Views on the Accountability Challenge," January 9, 2008.
10. "Soaring College Tuitions," New York Times, December 3, 2008.
11. "Degrees of Difficulty," Time, October 18, 2012.
12. Data from the Federal Reserve Bank of New York, as cited in William G. Bowen, "The 'Cost Disease' in Higher Education: Is Technology the Answer?" Tanner Lectures, October 2012 (ITHAKA), p. 13.
13. Ibid., p. 26.
14. Benjamin S. Bloom, "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring," Educational Researcher, vol. 13, no. 6 (June–July 1984).
15. William G. Bowen, Matthew M. Chingos, Kelly A. Lack, and Thomas I. Nygren, "Interactive Learning Online at Public Universities: Evidence from Randomized Trials," May 22, 2012 (Ithaka S+R), summary.
16. National Center for Academic Transformation, "Changing the Equation: Redesigning Developmental Math," Outcomes Analysis.
17. Bruce Upbin, "Knewton Is Building the World's Smartest Tutor," Forbes, February 22, 2012.
18. Steve Kolowich, "How EdX Plans to Earn, and Share, Revenue from Its Free Online Courses," Chronicle of Higher Education, February 21, 2013.
19. Peter Kinsley and Sara Goldrick-Rab, "Weighing the Alternatives: Financial Constraints and Early College Enrollment Trajectories among Low-Income University Students," Affordability & College Attainment in Wisconsin Public Higher Education, July 8, 2011.
20. Nate Johnson, "The Institutional Costs of Student Attrition" (Delta Cost Project, American Institutes for Research, September 2012), and Byron G. Auguste, Adam Cota, Kartik Jayaram, and Martha C. A. Laboissière, "Winning by Degrees: The Strategies of Highly Productive Higher-Education Institutions" (McKinsey & Company, November 2010).
21. Complete College America, "Guided Pathways to Success: Boosting College Completion," 2013.