Exploring the Future of Extended Reality in Higher Education

During the winter break of January 2020, a group of forty educators, instructional technologists, and industry leaders gathered at Yale's Center for Collaborative Arts and Media to discuss the future of extended reality technologies (XR) in higher education. Attendees were mostly representatives of the institutions that participated in the HP/EDUCAUSE Campus of the Future research project. These individuals and institutions have been involved with XR deployment on campus for as long as anyone and longer than most—at least one year and as much as three. This XR in Education Summit was convened to create a venue for this group to reflect on their work with XR, identify the open questions and problems that need solving, and collectively create a vision for the future of the technology and its use in higher education. Over two days of the event, some shared agreement emerged, both about what the XR community in higher ed is doing and what remains to be done. The meeting was made possible through the generous support of HP.

Why XR? Why Now?

XR encompasses virtual reality (VR), augmented reality (AR), and mixed reality. Development of technology recognizable as virtual reality dates to The Sword of Damocles head-mounted display system, developed by Ivan Sutherland in 1968,¹ though non–computer scientists might be more familiar with Jaron Lanier's work at VPL Research in the mid-1980s.² Augmented reality technology dates back even further, to the military's development of heads-up displays for fighter jet pilots in the 1950s.³ Despite having existed for decades, these technologies have seen limited adoption in educational settings.

So, why now? Given the long history of XR, why are we now seeing more institutions leading the work to introduce XR on campus? Because now we have empirical evidence of the efficacy of XR technology for teaching and learning. EDUCAUSE has published three reports on XR, which demonstrate that XR is an effective technology for active and experiential learning, enabling users to gain concrete experience that might not otherwise be available.⁴ By providing "hands-on" experience, XR helps promote student engagement with learning materials and deepens student interaction with complex problems.

XR technology also promotes engagement with the XR technology itself.⁵ Trying out a new piece of VR content can be a revelation. As Nonny de la Peña found when she started developing journalistic stories in VR, the experience produced strong emotional reactions in users.⁶ Partly as a result of the depth of the XR experience, there has been a burst of discussion around accessibility, empathy, data collection, and the ethics of the technology.⁷ All of this attention to the technology has led the demand for XR developers to far outstrip supply.⁸

Someone on campus must advocate for XR technology. Some instructors will need to see XR in use or use it themselves before they can imagine how they might use it for their own courses. Others might want to see data on the efficacy of XR for learning and evidence of its impact. Before allocating resources to support XR initiatives on campus, campus leadership will want to see evidence of factors such as increased student engagement and greater achievement of learning outcomes. Communicating the value of XR for teaching and learning, showcasing successful use cases and evidence of impact, and providing access and space for users to use the technology are necessary steps in any action plan for promoting widespread adoption of XR.

The Emerging Technologies Consortium at Columbia University and the XReality Center at The New School provide good examples of campus units that have taken on this advocacy role. The ETC hosts regular meetings of Columbia faculty, students, and staff; hosts events bringing industry leaders to campus; and provides support to members of the Columbia community who are working on projects that involve emerging technology. The XReality Center produces XR projects developed by faculty and students, offers workshops and tutorials on XR-related topics, and collaborates with faculty to integrate XR into courses. Part of the function of these units is to promote the use of technology by reaching out to faculty and students, and another part is to host events and showcase the technology to the campus community. But a particularly important function of such units is understanding what data to collect and what metrics of success are important to instructors, campus leadership, and the campus community.

The XR Summit took place in January 2020, a few days before the first recorded case of COVID-19 outside China. It's become commonplace to say that it's a whole new world now, but that's especially true in higher education. If nothing else, COVID-19 has demonstrated that technologies that enable distance learning are critical to the survival of institutions of higher education. And so there is another answer to the question "Why now?" that we did not see during the Summit: although one of the most important uses of XR is to provide experience of environments that are remote or dangerous, we did not imagine that would include our campuses.

Need for More Educational XR Apps

One of the greatest challenges in deploying XR in education is a simple lack of educational resources. Every VR and AR development platform has its own repository, and although some of the apps in those repositories have educational value, they are in the minority. A few projects are under way by research groups at institutions of higher ed, to develop educational resources in XR, such as Cellverse teaching molecular biology and Electrostatic Playground to teach atomic physics, both at MIT, and HoloAnatomy to teach anatomy at Case Western University. Some traditional publishers have also made inroads into the XR market—Pearson launched and spun off its XR development team, while McGraw-Hill has partnered with an AR startup.⁹ The applications from publishers are relatively expensive, however, and those developed at institutions of higher ed are still under development.

At present, several XR development platforms are available. Each has many hardware-platform dependencies, however, some of which change with software or hardware updates, and it is difficult to justify scaling the learning curve of these platforms when their future is unclear. Emblematic, a company founded by Nonny de la Peña and Jamie Pallot, has made considerable progress toward solving this problem, having developed an XR platform that is both web-based and based on open standards. A platform developed on open standards addresses the issue of the transferability of content—across platforms and over time—thus lowering the cost of investing in that particular platform. If one or more platforms emerge as the choice for developing XR educational resources, that would go a long way to addressing the lack of educational resources by making it possible for more instructors to develop their own.

Students as Innovation Drivers

Institutions should make it possible for students to drive XR on campus,¹⁰ and at several institutions, students have independently developed projects in XR. Some of these projects were for courses, such as the development of an AR app for an assignment in a journalism course, or as part of research projects, such as the development of an app for learning anatomy. Many others were self-initiated and self-directed, and self-directed learning is often the most effective form of learning. For this kind of development to happen, institutions need to provide students with easy access to XR technology, including hardware such as VR headsets and computers capable of sophisticated graphics rendering, and software such as development platforms. Access to this technology might be in computer labs or studios, but technology lending programs are even more valuable, particularly when students cannot come to campus. Support for the technology is also important, but simple access is critical.

Student activities are a useful way to foster student interest in XR. Holding a VR game night, for example, or hosting a local user group on campus, is likely to bring in interested students. This new medium also requires new ways of evaluating student work, which in turn requires instructors to reconsider their assessment criteria for course assignments and program outcomes to encompass self-directed learning.

Rapid Pace of Change

Planning in an environment of rapidly evolving technology is a constant challenge. New, potentially game-changing devices come onto the scene while others are discontinued.¹¹ The standard higher ed budget cycle and the time frame of faculty course planning can make it difficult to project technology needs even a semester or two ahead. This difficulty is often compounded by concerns about purchasing hardware that is almost guaranteed to be outdated or obsolete in a few years. Application hardware dependencies, coupled with a software development cycle that can be measured in years, make for a risky endeavor.

Colleges and universities need to stay nimble by avoiding large technology deployments and significant technology investments. Purchasing one or two VR headsets presents much less risk than purchasing a lab full of them. Being nimble also means leading with institutional needs. Institutions should start by developing pilot projects, small-scale uses of XR in specific courses and research projects, and other such use cases. Developing the lightest-weight application that will still allow an institution to achieve its goals enables formative evaluation of an emerging technology. It then becomes possible both to backfill those needs with a range of possible technologies and to make decisions about how to backfill as late as possible, to allow for technological changes.

Empowering students to use XR for class projects is one way to accommodate the pace of change. The learning that takes place during the process of building an XR experience—for both the student and the instructor—is arguably more important than the final product. Rapidly developed, half-baked, "minimally viable" projects are a completely reasonable way to deal with the rapid evolution and uncertain futures of XR technology in an educational context.

One further challenge is that most XR hardware and software are designed for a small number of end users. Cellverse, under development by the CLEVR project at MIT to teach cell biology, is a two-user experience. Even the popular VR game Star Trek: Bridge Crew is designed for just four players. To be useful in a classroom, an XR experience must allow for multiple, often simultaneous, users. ClassVR, for example, enables the simultaneous delivery of a simulation to multiple headsets. Furthermore, campus deployment of XR technology in a classroom or in a lab requires an enterprise license. Vendors in the XR space that seek to gain a foothold in the education marketplace would do well to offer enterprise licenses for educational institutions.

Collaboration

XR technology benefits from—indeed, perhaps requires—work to be cross-disciplinary. An AR app that places anatomical structures before the user's eyes, for example, requires an understanding of space and lighting, skills more common in a school of architecture than a school of medicine. A VR simulation of an archaeological site might benefit from a narrative presentation to aid the user's understanding of the development of the site over time, thus bringing the skills of journalism and storytelling into history.

The interdisciplinary nature of XR technology, one of its greatest opportunities, is also one of its greatest challenges. Institutions of higher ed tend to be quite siloed. Campus units have their own priorities, culture, and budgets. Launching a collaboration can be difficult because it requires cutting across existing organizational structures and possibly developing entirely new ones.

Some institutions are taking the lead on this type of interdisciplinary collaboration. Part of the work of the ETC and the XReality Center is to foster this sort of collaboration. The fact that both the ETC and the XReality Center are lightweight and flexible makes them particularly effective at fostering interdisciplinary collaboration, but the fact that both have official status as campus units gives them credibility and agency and within and outside of their campus communities.

The Miami Beach Urban Studios, within the Florida International University College of Communication, Architecture + The Arts, is a 16,000-square-foot makerspace-like facility containing a wide range of equipment from computers and software to 3D printers and scanners to laser cutters, all of which is accessible to all members of the FIU campus community and to the local Miami Beach community. The MBUS is central to developing a new offering in the FIU First-Year Experience curriculum, developing a VR simulation called Community in partnership with the FIU College of Engineering & Computing and the office of Academic and Career Success.¹² At the other extreme, a few faculty members at Syracuse University with an interest in XR started meeting informally in the 2017–18 academic year. This grassroots group has grown and now includes collaborative grant applications and hosts events to showcase XR technology on campus.

XR challenges the traditional organizational models that exist in higher ed, and varying organizational structures and leadership will necessitate different strategies to foster cross-campus collaboration. XR technology changes rapidly, and the domain in which it is deployed might change from project to project. Interdisciplinary collaborations enable the development of proofs of concept, a proof of concept fosters successes, successes attract attention and often money. There might be power in having a center or initiative, but there is also power in having an agile and flexible, constantly shifting collaboration around XR.

External Partnerships

If it's difficult to collaborate across units within a single institution, it's often even more difficult to collaborate across institutions. Interinstitutional collaboration is the norm in research but is very much not the norm for institutional support for IT. Collaborative support structures are exactly what XR development needs, and community building must be part of this support. The XR in Education Summit was an example of this—an opportunity for a community of interest to meet, to brainstorm, and to discuss possible collaborations. Attendees included not just representatives from academia but also representatives from industry—from game design, workforce training, journalism, and hardware manufacturing.

Institutions of higher ed can benefit from collaborations with organizations in these and other sectors because these businesses can often bring resources to the table that higher ed lacks, including funding and hardware, expertise in software and hardware development, and a view into workforce training needs. By the same token, the business sector can benefit from such collaboration, drawing on higher education's research expertise, relying on the community as a constantly changing beta test site, and hiring their students and graduates. Each sector lends expertise and credibility to the other.

The current budgetary environment brought about by COVID-19 throws into sharp relief the significant differences in economic resources among institutions of higher ed. On one hand, it might seem as though the current budgetary environment would create a situation that simply reinforces the status quo. But emerging technologies cannot succeed if they're only adopted by a few; an emerging technology must be widely adopted or it will fail. The market for learning management systems (LMSs), for example, would not exist if only a handful of institutions had adopted an LMS when it was an emerging technology. The widespread adoption of the LMS was driven in part by some institutions being early adopters, in part by the development of standards for online learning applications, and in part by the rise of businesses and nonprofits in the broader market space (such as the IMS Global Learning Consortium, which guides community-building around standards). Similarly, if XR as an instructional technology is going to see widespread adoption, higher education must work together to make it happen—as a community, across institutions, and across sectors.

Building Community

Prior to the COVID-19 pandemic, one of the most important contributions that institutions of higher ed could make to building community around the use of XR in education would have been simply to provide space for communities of interest to gather. In the COVID-19 world, the ability and the willingness to gather side-by-side become uncertain. Even in such an environment, though, institutions of higher ed can still play a critical role as a convener and community-builder.

Part of the purpose of community-building is social. As we've rediscovered these past few months, merely being in a room with colleagues is enjoyable for its own sake. At the same time, part of community-building is to create leverage: as the saying goes, if you want to go quickly, go alone; if you want to go far, go together. The XR community wants to go far in developing the technology and integrating it into teaching and learning, and the best way to do that is together. COVID-19 has forced organizations of all types to spend money in unexpected sums and categories. Institutions of higher ed, in particular, are experiencing the double-whammy of unanticipated spending on technology and online courses and resources while also not receiving expected revenues from residence halls, rentals of campus spaces by external organizations, and other sources. New campus initiatives and projects must now compete for resources even more strenuously than before.

Building community must therefore include creating mechanisms for the XR community to collaborate on projects and leverage resources across organizations. Creating mechanisms for communities of interest to collaborate while remote from each other is fundamental to the way institutions of higher ed operate. Conferences and other mainstays of academic culture have been moving online, and there has possibly never been a more active burst of creativity in developing tools for fostering collaborative teaching and research than in the past decade. EDUCAUSE has long supported this collaborative and community-building work with the XR (Extended Reality) Community Group.

Looking beyond the Pandemic

In the short amount of time since COVID-19 closed campuses and moved teaching and learning entirely online, talk of "a new normal" has become commonplace. Colleges and universities have existed for more than a thousand years. Institutions of higher ed have always changed with the times while retaining many traditional structures and practices. The question that faces higher ed in the new normal is, what will stay with us after this time? What changes to colleges and universities, to teaching and learning, to collaboration, and to XR technology and related industries will be permanent?

One of the most obvious answers is that this pandemic might result in the end of shared VR headsets. No one is going to want to put on a headset that someone else just took off or share other hardware. Google Cardboard or other inexpensive, smartphone-based XR might be the future of this technology for educational and other group uses.

Technologies that enable distance learning are critical to the survival of institutions of higher education. The question has changed from how we can facilitate the use of XR technology on campus to how we can facilitate the use of XR technology off campus. As with laptops and phones and other technologies that move between campus and users' homes, institutions will need to develop IT infrastructure and security policies to manage XR hardware. And just as some institutions provide some software-as-a-service, as more educational XR apps are developed, these too might be provided in the cloud, perhaps using infrastructure similar to e-sports.

COVID-19 has changed the world and the role of institutions of higher ed in it. This situation makes clear the potential of XR for teaching and learning, but it also makes clear the fact that not all of the tools are in place for XR to be as useful in education as it could be. The goal of the XR in Education Summit was to create a vision and action plan for the future of XR technology and its use in higher ed. Little did we know that reality would make that vision more important than ever.

Notes

1. Howard Rheingold, Virtual Reality: The Revolutionary Technology of Computer-Generated Artificial Worlds—and How It Promises to Transform Society (New York: Simon & Schuster, 1992). ↩
2. Simon Parkin, "Virtual Reality Startups Look Back to the Future," MIT Technology Review, March 7, 2014. ↩
3. "Windshield TV Screen To Aid Blind Flying," Popular Mechanics, March 1955, p. 101. ↩
4. Jeffrey Pomerantz, Learning in Three Dimensions: Report on the EDUCAUSE/HP Campus of the Future Project, research report (Louisville, CO: ECAR, August 2018); Jeffrey Pomerantz, XR for Teaching and Learning: Year 2 of the EDUCAUSE/HP Campus of the Future Project, research report (Louisville, CO: ECAR, October 2019); and Jeffrey Pomerantz, Extending XR across Campus: Year 2 of the EDUCAUSE/HP Campus of the Future Project, research report (Louisville, CO: ECAR, March 2020). ↩
5. Marta Figlerowicz and Ayesha Ramachandran, "How an Experiment in 3-D Printing Illuminated Our Humanities Classroom," Chronicle of Higher Education, October 22, 2017. ↩
6. Nonny de la Peña, "The Future of News? Virtual Reality," TED Talk, December 15, 2015. ↩
7. See "Accessibility of Virtual Reality," W3C wiki; Paul Bloom, "It's Ridiculous to Use Virtual Reality to Empathize with Refugees," The Atlantic, February 3, 2017; and Emory Craig and Maya Georgieva, "VR and AR: The Ethical Challenges Ahead," EDUCAUSE Review, April 10, 2018. ↩
8. Shirin Ghaffary and Rani Molla, "How Tech Companies Are Trying to Make Augmented and Virtual Reality a Thing, Again," Vox, February 11, 2020. ↩
9. Tony Wan, "Pearson Sells Mixed Reality Learning Unit to a New Company, GIGXR," EdSurge, December 18, 2019, and Tommy Palladino, "McGraw-Hill Jumps into Educational Augmented Reality via Partnership with Startup Alchemie," Next Reality, May 21, 2019. ↩
10. Emory Craig and Maya Georgieva, "VR and AR: Learners as Creators and World Builders of Our Immersive Future," EDUCAUSE Review, December 15, 2017. ↩
11. Michael Kan, "Google Discontinues Daydream VR Headset, Drops Support in Pixel 4," PC Magazine, October 15, 2019. ↩
12. Tatianna Basanta, "Virtual Reality Project Offers 21st-Century Skills to First-Year Experience Students," FIU News, March 26, 2020. ↩

---

Jeffrey Pomerantz is Associate Professor of Practice, School of Library and Information Science, at Simmons University.

Randall Rode is Director of Campus IT Partner Relationships and Development at Yale University.