The Origins of Innovation in the Edtech Ecosystem

The learning management system (LMS) as we have known it is fading in its importance. Supplanting it are hundreds of tools and innovations, created by a plethora of vendors, institutions, students, and instructors in a frothy, bubbling world known as the edtech ecosystem. If the pace of innovation and experimentation persists, this could change instruction for the better. However, this transformation will not come about in the manner that many experts and pundits have predicted. Furthermore, IT leaders and chief information officers (CIOs) will need to dig in now and start contributing.

The word ecosystem, borrowed from its ecological and biological roots, here refers to the educational technology (edtech) market. This market is now sufficiently complex, with dynamic and constantly evolving interconnections between all participants: the edtech vendors themselves; their customers (educational institutions and students); investors in the market; industry trade groups; analysts; local, state and federal governments; think tanks; and vendor suppliers, including companies — like Amazon, Microsoft, and Google — that offer cloud infrastructure and software development tools. This market is diverse with participants big and small, players who have been deeply embedded in higher education for a long time as well as recent interlopers who are feeling their way. The relationships between the various players change over time and are often murky. For example, the largest edtech vendor today may be Amazon. So many edtech vendors have created their solutions in the Amazon environment that it is likely students, faculty, and staff find their digital interactions flowing through the Amazon platforms more often than in any other platform.

Unfortunately, the market and many IT leaders have mischaracterized IT tools, causing them to underestimate the importance of an ecosystem approach. In addition, external conditions, most noticeably the calls for improvements in teaching from our citizen stakeholders and now from many edtech vendors, continue to influence the ecosystem. In both cases, action from CIOs and IT leaders can help.

The Deeper Nature of IT Tools

Unlike all the tools that have come before, IT tools are much more malleable, almost obscenely so. Timeless tools like the hammer and the saw are made for a specific purpose and require just a bit of practice to master. The capabilities of other tools like the bow and arrow, the atlatl, and the sword become greatly enhanced with advanced skill, requiring dedicated and effortful practice. While designed to work with the human body, these tools still require that people change their behavior and their physical skill to adapt to the tool.

IT tools are quite different. They usually require little to no physical practice to master. While we spend a modicum of physical effort to learn how to hold, type, or otherwise physically handle these tools, we spend much more time in mental effort. We must learn what each tool, each component, each icon, each link, and each button does. We have to learn how very different parts of the software relate to each other. And we need to practice the mental routines needed in order to make good use of these software tools. Over time, this mental rehearsal, just like the physical one, improves our skill.

Unlike conventional tools, IT tools are not delivered in a fully completed form. All of them require some level of modification, configuration, or customization. The inherent pliability of software almost begs for this. In today's software tools we have all sorts of customization capabilities ranging from simple to elaborate settings; we can even go as far as writing code ourselves or mashing up a collection of smaller software tools.

Since the worlds we build as educators are mental worlds with tremendous complexity, diversity, and range of ideas, our relationship with the tool is even more fascinating than the usual business functions to which software is applied. Instructors have to master this complexity in such a way as to effectively share these different worlds of ideas with learners. This complex process of matching content to learners through either automated methods or good old-fashioned face-to-face instruction beckons instructors and their attendant IT helpers to highly customize learning tools. For some difficult teaching situations, the demands that the learning task places on instructors and learners can require a great deal of tailoring. Right now and for the foreseeable future, this tailoring process is necessary for both instructors and learners. The EDUCAUSE Learning Initiative (ELI) research into the next generation digital learning environment (NGDLE) identified this need as a critical one that the market needs to address.¹

Customizable and configurable IT tools give learners the opportunity to organize their own content and manage their own knowledge. Learners frequently invest their time into their mental scaffolding. Their tool use becomes connected to their learning and to course outcomes. This can be beneficial. When both students and instructors become facile co-creators or, at the least, co-tailors of their own tools, attachment both to the tools and to the knowledge delivered grows, building a cycle of engagement and mastery.

The richness of this emerging edtech ecosystem is spawning many thousands of micro-experiments in classrooms across the globe. While some critics may discount this seemingly endless faculty tinkering with tools as indulgent frivolity, I disagree. Instructors are learners too. They are performance artists and have to contend with all the pressures of being on the physical or the online stage. Adopting new teaching and learning tools requires dedicated practice and tool tailoring before stepping onstage.

This tailoring yields other significant benefits. While it can facilitate learning and mastery for all involved, the configuration of these tools is enabling better data integration of learner activities or clickstreams. This in turn catalyzes rapid experimentation with learning analytics. Many vendors and leading instructional researchers are instrumenting classrooms and capturing digital footprint data for the purposes of finding ways to improve instruction. As the technology continues to evolve and undergo speciation, these instructional research insights are likely to get deeper, more focused, and highly varied across disciplines, institutional contexts, and learners. These insights will lead to the further evolution of software tools. The emerging edtech ecosystem looks poised to enable this virtuous cycle of adaption, experimentation, insights, and community sharing.

Leading Up to This Point: External Conditions

A combination of both political and technical trends has pushed us to this point. Our citizen stakeholders have been, for many years now, exhorting us to adopt business-like approaches to achieve breakthrough solutions that will deliver great results at lower costs. Partially as a result of this pressure, the edtech sector has grown significantly with dozens and dozens of startups, with some that are fabulously funded and others that had "garage band" beginnings. This most recent edtech bubble has created much interest and excitement. As one experienced edtech company chief executive officer and investor said to me: "Venture capitalists want to transform higher education because of deep-seated emotions, probably growing out of personal experience. I think they are just angry and want to reform education as a matter of personal belief." Clearly there is a lot of emotion behind the pile of money.

This hope and hype is understandable. Since the Internet, information technology has enabled new levels of coordination and synchronization across the globe, ranging from large-scale computing systems talking to each other through very-high-speed networks to an enormously large collection of small devices on the Internet chatting with each other across wired and wireless networks. Software modularity and speed of global network communication are now capable of sub-second response times. In several cases, entire industries — from travel and entertainment to retail — have been remade. Everyone likes a good story, and the come-uppance and destruction of giants through the use of a special sword is an old and appealing one. Many critics have imagined higher education as a walled fortress that, with enough scaling of the walls and battering of the doors, will also fall at the feet of IT disruption. This metaphorical overuse has led to some misplaced expectations. The mental model of information technology as a disruptive tool (i.e., a sword) has caused many participants in the ecosystem to underappreciate the co-evolutionary processes at work. In this market, both tools and tool-makers undergo intertwined incremental evolution to help solve local and collective problems.

We can consider the edtech market as being comprised of (a) IT constructs such as integration standards and new approaches for analyzing learning data or components like a gradebook or an in-class student response system, and (b) collections of human practices for using and configuring these constructs to deliver improved digital learning activities. IT constructs bound together with practices for delivering educational experiences undergo both change and replication (sharing). This IT tool and behavior bundle is akin to an extended phenotype in the ecosystem. While individual participants in the market come and go, it is this ever-evolving bundle of tool and behavior that constitutes the enduring change people seek. This sounds complex, and it is. I believe it follows from the intrinsically complex nature of teaching and learning.

The difficulty of instruction, I contend, is just beginning to be appreciated by those closest to the instructors: the staff, the administrators, and the vendor community. How complex is instruction? As it turns out, it could be wildly complex. According to Kenneth Koedinger, Julie Booth and David Klahr, a learning environment yields 205 trillion (10^14.3) options to consider in crafting an appropriate digital learning experience.² The authors are perhaps carefully understating the size of the problem by choosing a very small set of operational definitions of a learning environment. Expanding their model to match a more realistic depiction of a digital learning environment yields a problem space of 10⁶²⁹ combinations to address. Exploring this problem space will require many decades of research insights, data, and advances in learning theory. Researchers studying how humans learn are faced with a plethora of overlapping and often conflicting theories and approaches. A single theory of the mind useful for designing detailed digital interactions may not be simply on a distant horizon; it may be a mirage on that horizon. In this regard, we are still very early in the study of learning.

In many ways, the health care community is ahead of the education community. The twentieth century planted the scientific method firmly into the process by which treatments are applied to patients. Peer-reviewed science and gold standards of empirical research are now standard practice before the medical community adopts changes. With the plethora of websites available, some of which provide high-quality information, many patients learn about the treatments they are going to receive and visit their doctors better informed. Many patients now understand the set of cause-and-effect relationships in treatments and the underlying human biology and physiology involved.

Over the past several years, when I give a presentation or talk I often start with four questions to the audience. The first question is: "Who can name a part of the human heart or explain roughly how the heart works?" I usually get a large show of hands and, in some cases, very accurate answers. I then ask: "Armed with this knowledge, how many of you would offer suggestions to a friend or neighbor about a heart problem he or she is experiencing?" I usually get no hands showing except those of people who would tell their friend to go see a doctor. With the third question, I pivot to learning science and ask: "Who can name one anatomical or functional part of the human brain involved with learning?" Unless there is the rare learning scientist in the room, no one answers. I probe with the final question: "How many of you have recommended to friends or family how they could improve learning for themselves or their children?" More hands go up. The typical answers I hear are common folk psychology answers. No one visits a website to learn about detailed scientific explanations for how learning occurs. Such a site does not exist due to the complexity of the learning research field.

Most vendors also struggle with insight into instructional science. If you poke beneath the punditry and the white papers the vendor community produces each year, you find little rigorous, peer-reviewed science in these solutions. While some vendors conduct usability studies internally, most vendors are not conducting rigorous research.³ In my discussions with learning technology vendors and investors, most acknowledged that there is a significant lack of understanding of learning research. Instead, many of these edtech entrepreneurs (and animated political leaders!) are using faith-based reasoning to argue that information technology should not just lead to tremendous improvements in learning for all but also usher in a radical reshaping of colleges and universities. Due to the lack of detailed knowledge about how learning actually occurs, most of the advice for how to cure what ails us resembles that of ancient medical doctors prescribing bloodletting to cure diseases — well-intentioned but terribly uninformed.

Earlier this year, George Veletsianos and Rolin Moe called attention to this issue of insufficient evidence as part of a larger technocentric view of educational reform that may be problematic.⁴ Many within the higher education vendor and reform communities are persistent in their belief that technology ought to provide compelling improvements in education just as it has in other industries. Our problem may be more fundamental and related both to the practically infinite nature of information itself and to the requisite complexity within the human brain that must absorb and utilize this information. For the foreseeable future, given the size and complexity of this research space, it is unlikely that any one person, team, or organization will discover an approach that will rapidly transform education. Making improvements in education through digital learning tools will require a community of experimentation and research and incremental discoveries and improvements over time. A modular, better-integrated, and flexible next generation digital learning environment will be needed.

Where CIOs Can Help

Every year as I interact with peers, my hope for the future grows. There are many ways we can help.

The first and perhaps easiest place for CIOs and IT leaders to start is by helping the IT organization learn more about how instruction and digital learning tools interact. We can shift investments to help bring in knowledgeable and skilled experts and embed them within the various technology teams that design and deploy solutions. IT leaders serve as critical boundary spanners, bridging between the IT staff and fellow administrators and the myriad of instructors who teach in the different disciplines. IT leaders represent their concerns in countless meetings with vendors and others who are building this new ecosystem. Getting learning theory and instructional design experts well-placed in front of both vendors and faculty can make a difference.

CIOs should challenge themselves on how they conceptualize the current LMS. As noted in the earlier-mentioned ELI research on the NGDLE, the LMS is more of a tool to help in the administration of learning, not in the actual promotion of learning. As discussed here and elsewhere, the edtech ecosystem has undergone a type of Cambrian explosion to get around the real and perceived limitations of the LMS. The better we understand both the broad trends and the nuances in this realm, the more we can help guide the ecosystem's evolution.

In most institutions, IT leaders and staff serve predominantly administrative and business functions. While most have an IT leader and some staff dedicated to the teaching and learning mission, these are usually a very small fraction of the whole. Few CIOs and IT leaders have sufficient training in or experience with the teaching and learning domain. This creates a blind spot where IT units inadvertently cut off teaching and learning initiatives by treating these systems as minor appendages to the mass of information technology supporting the institution. Valuable data from administrative systems (including student administration systems) fails to cross over to the teaching and learning realm and vice versa. Many IT staff members are not knowledgeable enough, not interested enough, or not trained enough in our teaching and learning missions. To properly design and integrate the next generation learning technology ecosystem, we need more IT leaders and IT staff engaged in learning theory.

IT leaders can demand better content and data integration from their vendors. The standards market is maturing rapidly with adoption steadily growing, but more needs to be done. CIOs and IT leaders can enforce these standards during procurement processes and can help rally support from instructors and educational staff for these standards. When institutions demand standards, vendors respond. But the compliance with good standards needs to be placed before a purchase decision; it is much harder to require vendors to adhere to standards after the purchase event. Here, IT discipline matters. These standards can ease data and content integration and help create a better experience for students and instructors. Just because we go digital doesn't mean we operate in the dark. We need to make sure faculty and institutions maintain a digital "line of sight" with their students. The integration, especially the data integration, will ensure that instructors and researchers have insight into the digital learning experience.

I also suspect that the edtech ecosystem may be dependent on these standards to ensure vibrancy. Unless there is a way to collect data for insight and to easily deploy tools, institutions may grow weary of the effort that ad hoc integration brings and may back away from supporting broader intra- and inter-institution collaboration.

IT leaders can do their part to enable sharing of data and teaching practices. This new edtech environment will need an analytic infrastructure to bring together all the data from each module in the edtech ecosystem so it can be easily analyzed, abstracted, and absorbed. Whereas all institutions are working for better collaboration across the academy, the difficult demands of each discipline often frustrate the reallocation of time for this purpose. To take advantage of these next generation learning technologies, institutions are going to need not just a community of instructors who want to advance the science and art of teaching in their own disciplines, but enough instructors who want to participate in building the human and technical systems that can share insights and teaching practices across the disciplines. CIOs can help by prioritizing IT projects than can advance both technical and human collaboration across the academy.

IT leaders need to encourage, coax, or otherwise mandate that vendors build solutions that help the community. I have talked to many vendors and several of their investors. All of them are curious and passionate about what they are doing. In some cases, however, they believe that both the data and the algorithms must be under lock and key and cannot be shared with the institution. CIOs and IT leaders can help set vendors' expectations. Higher education tends to have a culture of considerate collaboration between institutions. We regularly share our teaching methods and findings with each other in many venues, ranging from peer-reviewed journal articles to brown-bag lunch sessions. Vendors have a culture of intense competition as they seek significant, long-term profits. Many vendors place their algorithms and sometimes their data under lock and key, hoping this intellectual property will be a major source of future profits. While the complexity of instruction and the ease of reverse-engineering technical solutions make it very unlikely that any single vendor will invent such an algorithm, this culture difference creates a problem when those of us in higher education need the ecosystem to share data, algorithms, content, and insights more broadly. Our job is to manage the differences between these two cultures and bring collaborative, not overly competitive, learning solutions to our institutions.

The edtech ecosystem brings forth its own set of privacy, ownership, and security concerns. Learning analytics can reveal a lot about both learners and instructors, so this data needs to be carefully stewarded. IT leaders can help lead in establishing new or adopting existing privacy principles, policies, and guidelines and then advocating on behalf of faculty and students when engaging with the edtech vendor community.

IT leaders can also have a voice within industry and governance groups. By having conversations with these stakeholders across different and sometimes provocative dimensions, they can continue to advocate for a collaborative community of institutions and edtech solution providers who will collectively advance the art and science of learning and learning analytics for the good of all.

In the past several years, more CIOs are getting involved in innovation. Because the IT infrastructure tools have sufficiently advanced in the past decade or so, IT leaders can devote more time to innovation, encouraging local experimentation within and between institutions. They can help share best practices, especially around learning systems infrastructure, analytics, and integration techniques. And they can promote the right balance between the diversity of instructional technologies needed and the parsimony that will be cost-effective. Of all the areas of higher education technology, the NGDLE can provide some of the more meaningful and impactful innovations.

While edtech vendors build products, more importantly they construct marketing narratives around those products. What we sometimes fail to realize in higher education is that the narrative the vendors create has to fit within the confines of the institutions they serve first. These narratives must be understandable, nonconfrontational, digestible, and consumable. Often, edtech vendors merely mirror higher education's current average practice or that of their first adopters. Far from blazing into new territory, many edtech startups walk a familiar and comfortable trail. CIOs can help bridge the divide between true innovation and the status quo by connecting instructors and lead designers with the product design teams within these edtech companies.

Dare to Join the Revolution?

As colleges and universities continue to bring on additional technologies, including mobile and real-time learning analytics tools, those of us in higher education are going to be challenged to make dreams come true in ways we have not seen before. While many external stakeholders in government and industry are waiting for technical innovation to transform teaching and learning by itself, sans humans, that is like waiting for Godot. For the foreseeable future, tackling the enormous complexity in the art and science of learning will take a village. A really big village. We will need higher levels of human and technical integration. CIOs and IT leaders are well positioned to help.

These innovations in next generation digital learning environments — in the edtech ecosystem — are coming. What we collectively do next will determine the outcome.

Notes

1. Malcolm Brown, Joanne Dehoney, and Nancy Millichap, The Next Generation Digital Learning Environment: A Report on Research, an ELI white paper (April 2015).
2. Kenneth R. Koedinger, Julie L. Booth, and David Klahr, "Instructional Complexity and the Science to Constrain It," Science 342, no. 6161 (November 22, 2013).
3. Emmeline Zhao, "$8 Billion a Year on Ed Tech, but Does It Work? Experts Call for Better Research at Unique D.C. Symposium," The 74, May 4, 2017.
4. George Veletsianos and Rolin Moe, "The Rise of Educational Technology as a Sociocultural and Ideological Phenomenon," EDUCAUSE Review, April 10, 2017.

---

Vince Kellen is Chief Information Officer at the University of California, San Diego.