The Myth about No Significant Difference

© 2006 Diana G. Oblinger and Brian L. Hawkins

EDUCAUSE Review, vol. 41, no. 6 (November/December 2006): 14–15

Diana G. Oblinger is Vice President of EDUCAUSE, where she is responsible for the association's teaching and learning activities and for the EDUCAUSE Learning Initiative (ELI). Brian L. Hawkins is President of EDUCAUSE. Comments on this article can be sent to the authors at [email protected] and [email protected].

It seems that someone always wants to know if technology "makes a difference." Sometimes it is a legislator, hoping that technology will reduce costs. Sometimes it is the college or university president, hoping for a competitive edge. Sometimes it is the provost, hoping to prove that students learn more in online courses than in face-to-face settings. Sometimes it is a faculty member, hoping to show just the opposite. One group will claim that using technology produces no significant difference; another will say that technology has transformed higher education.

The problem is that to receive a valid answer, one needs to ask a good question. To get an answer as to whether technology makes a difference, we need to ask: "Difference in what?" For example, asking whether technology makes a difference in student learning implies that learning is a high-tech or no-tech phenomenon. The issue is not that simple. Learning occurs as a result of motivation, opportunities, an active process, interaction with others, and the ability to transfer learning to a real-world situation.

Whether or not technology makes a difference depends on how it is used for motivation. Many students are motivated through connections with experts. Technologies, such as videoconferencing, allow students to interact with experts—on and off campus. Others are motivated by being able to work on real-world problems or by being part of a project team. Students use databases of astronomical data to make their own discoveries. Others join globally distributed teams in taking measurements, sharing observations, and testing hypotheses. A few students have always had the opportunity to interact with experts and work on real-world problems. But today, those opportunities are much more accessible for many more students. Does technology make a significant difference?

Opportunities are also affected by technology. For millions of learners around the world, online learning has provided them with the flexibility to make learning adapt to their schedule, not the other way around. Whether it is a part-time student taking a distance-learning course, an undergraduate fulfilling a prerequisite over the summer so that graduation is not delayed, or a graduate student accessing rich resources to augment an on-campus course, technology provides unprecedented capabilities. Does technology make a significant difference?

Learning is an active process. In fact, the more active the learning is, the more likely the student is to learn. Simulations allow students to learn by doing. Visualizations enable students to see information that may have been hidden in tables of numbers. Students use technology as a key enabler in problem-based learning, searching for background information, conferring with team members, and using the tools of the profession to develop solutions. Does technology make a significant difference?

Learning is also a social process that is not bounded by space or time. Learning ebbs and flows throughout the campus—real and virtual—whether it be part of a class discussion, a conversation over coffee, IMing with a friend, or extracurricular activities. Interaction with others provides almost endless learning opportunities stimulating personal and professional growth. With wireless access, any space can be a learning space. Does technology make a significant difference?

Still, learning may not be enough if that learning can't be transferred to new and unique problems. The ability to transfer learning to a real-world situation enhances the application of knowledge and leads to enduring understanding. Those problems can be simulated using techniques such as augmented reality. Reflection can occur through Web journaling. Achievements can be documented in e-portfolios. Does technology make a significant difference?

Perhaps another, related question should be asked: What is technology? It is all too easy to fall into the trap of using outmoded definitions of technology. Technology is more than presentation software or a browser. The Web doesn't just "bring the world to the desktop." The Web is a medium for participation. Users receive information, but they also comment, collaborate, and create their own content. Anyone can create and publish content on the Web.¹ Perhaps we don't find a significant difference when using technology in student learning because we aren't looking in the right places or using the correct definition.

In thinking about the "no significant difference" phenomenon, the CIO and members of the executive team should ask themselves the following strategic questions:

1. Do we think of technology as a solution in itself or as a means to an end? Using technology does not necessarily produce change. A lecture may be enhanced when the lecturer uses presentation software, but if everything else remains the same—the interaction, the examples, the opportunity to problem-solve—the learning won't change. Colleges and universities should not expect learning outcomes to change if the pedagogical approach does not change. Learning is an active process; technology can provide a wide range of active learning opportunities and can enable those to scale to reach more learners. But adding technology without altering pedagogy is not a solution.
2. Do we assume that using technology is an either/or proposition? It is all too easy to assume that using one technology replaces another. But in many instances, technologies are blended rather than substituted. For example, research has verified that although students use technology in their personal lives, they aren't interested in replacing human contact with online content.² When asked, students typically respond that they came to college to be with people—faculty and other students. Being with others is now multimodal, involving face-to-face and online communication, often simultaneously.
3. Have we identified those processes and activities we want to improve and looked at how technology can facilitate those actions? Students learn through interaction with others. Interaction can occur more often, with more individuals, and over a more extended period of time when it is online rather than face-to-face. Students learn from authentic experiences. Although access to laboratories and instruments is limited, they can be shared via the Web, such as using remote instruments to collect data. Students learn from using the tools of the profession, such as CAD software. Students learn from applying what they know to solve a new problem, such as might occur in a simulation. Students learn through self-expression—whether text, audio, video, or images. Technology can make these learning opportunities more readily accessible and more flexible to accommodate the schedules of busy students—and faculty.
4. Are we doing the same things with technology, or are we taking advantage of the unique capabilities of technology and redesigning our activities? Although completing a writing assignment using a word-processor is faster and may encourage more revision than when using longhand, it doesn't necessarily change the activity. However, allowing students to listen to a podcast of a lecture while reviewing their class notes and watching an animation on the screen permits the integration of multiple inputs. In the early phases of technology adoption, old uses tended to be replicated with new tools. Perhaps rather than just assembling more data with technology, we should be using technology to interpret that data through visualization techniques. Patterns emerge; insights occur.

Does using technology produce a significant difference? The answer depends on how the question is asked.

1. George Lorenzo, Diana Oblinger, and Chuck Dziuban, "How Choice, Co-Creation, and Culture Are Changing What It Means to Be Net Savvy," EDUCAUSE Learning Initiative (ELI) white paper, October 2006, http://www.educause.edu/ir/library/pdf/ELI3008.pdf.

2. Joel Hartman, Patsy Moskal, and Chuck Dziuban, "Preparing the Academy of Today for the Learner of Tomorrow," chapter 6 in Educating the Net Generation, ed. Diana G. Oblinger and James L. Oblinger (Boulder, Colo.: EDUCAUSE, 2005), e-book, available at http://www.educause.edu/ir/library/pdf/pub7101f.pdf.