Using Augmented Reality to Promote Making with Understanding

Our research team at the Harvard Graduate School of Education's laboratory for Learning Innovation and Technology (LIT) is researching how emerging technologies have the potential to radically transform collaborative learning and informal STEM education. We are currently witnessing the birth of a cultural movement in which everyday people tinker with STEM concepts simply to satisfy creativity and curiosity. This "maker movement" is powered by the wide availability of low-cost electronics and manufacturing tools, which allow amateur student makers to create interactive objects while exploring scientific phenomena.

Dale Dougherty, a founder of the maker movement, characterizes makers as just "playing with technology…. They don't necessarily know what they're doing or why they're doing it. They're playing to discover what the technology can do and probably to discover what they can do themselves."¹ Such environments have great potential to engage people with STEM concepts and activities while empowering individuals to physically manifest their dreams.

But in these learning environments, the students' focus is often more on completing a technological project than on comprehending scientific concepts. While some learning does happen in such contexts, a large part of maker activities is about following a list of instructions and trusting that they are going to result into a functional project.

Our current research explores how collaborative learning unfolds in a new generation of makerspaces where we can enhance objects with augmented reality (AR) technologies. We think that emerging technologies such as AR have the potential to address this issue and radically transform STEM makerspace education by making challenging concepts accessible to students. AR headsets such as the Microsoft Hololens allow students to see virtual "holograms" in the physical world. These technologies enable us to design activities that enable learners to visualize and interact with the hidden forces involved in physical projects (e.g., electrons, magnetic fields, light or radio waves) or to see the inner workings of physical devices such as robots.

We currently have two projects pursuing this investigation. In one project we use Microsoft Hololens AR devices to research how AR technology can be used to show the invisible physics phenomena involved in audio speakers. (Another research project, not discussed here, features the ability to look at physical devices and visualize electronic sensors and programs.) Audio speakers are energy transformation devices, which convert electrical energy into magnetic fields; these magnetic fields move membranes to create physical vibrations that are perceived as sound. The majority of these phenomena are impossible to see with the naked eye, but AR can give them a physical form. To research the benefits of AR for this context, we built a large model of a speaker in which we amplify the signals of a smartphone and use them to create physical vibration in a cup (see videos below). When interacting with our system, students can see sound waves emanating from the cup, magnetic fields involved in creating vibrations from the cup, and electricity flowing through wires. They can also see how electric signals vary with music and how an amplifier strengthens them. This technology allows people to perceive otherwise invisible phenomena, which in turn helps them grasp the learning content and more easily discuss those concepts with peers.²

In this research we measure whether AR-enabled interventions can increase learning gains, ease deeper conceptual discussion among students, change student conceptions of the observed phenomena, or change student attitudes toward STEM and their own learning. We use traditional quantitative and qualitative measures, along with novel methods from the field of multimodal learning analytics (such as physiological sensors, eye trackers, and body posture trackers) to understand and classify students' behaviors across time and conditions.

From the preliminary data we have analyzed, students are excited about being able to see invisible concepts in AR, and this effect goes beyond simply being more engaged. We frequently get quotes such as "Fifth graders, fourth graders…they are in for a treat. I wish I had this when I was their age. I probably would have gone into physics instead of finance." Our measures of participant attitudes indicate that with AR, people's perceptions about their own abilities to understand physics is increasing. We also find that students in AR conditions learn more about the different shapes of magnetic fields, perform better at some transfer questions, and engage more easily in teaching behaviors where one participant is using the AR representations to teach another participant.

Our data also suggest some interesting detriments of AR technology. For example, we see that AR visualizations capture participants' attention so much that they can forget what they were discussing before we enabled the AR and/or forget to look at instructions sheets. Another interesting preliminary finding is that people in non-AR conditions focus more on the physical experience, such as the feeling of forces and vibrations, whereas people in AR conditions focus more on the visual aspects, such as how AR represents magnetic fields. This indicates that AR may not always be the best tool depending on the topic taught.

This work highlights the power of AR to engage people in STEM topics. It also provides some preliminary guidelines for the kinds of topics and activities for which AR is a better approach than non-AR techniques. This research contributes to our understanding of how we can enhance student education of STEM concepts with new technologies such as AR by contributing a set of reusable modules to visualize and simulate the invisible phenomena that are commonly encountered in maker activities, and by helping us produce guidelines to help the design of innovative learning environments.

Notes

1. Dale Dougherty, "We Are Makers," TEDtalk, January 2011. ↩
2. There is a slight misalignment between the physical objects and the AR holograms. This is due to the way that the video was captured and is not present from the student's perspective. ↩

---

Bertrand Schneider is an Assistant Professor at Harvard University.

Iulian Radu is a Postdoctoral Researcher at Harvard University.