In direct contradiction to Betteridge’s Law, we believe the answer is yes. Analytics in higher education is in the trough of disillusionment.
The trough of disillusionment refers to a specific stage of Gartner’s Hype Cycle. It is that moment when, after a rapid build up leading to a peak of inflated expectations, a technology’s failure to achieve all that was hoped for results in disillusionment. Those who might benefit from a tool perceive a gap between the hype and actual results. Some have rightly pointed out that not all technologies follow the hype cycle, but we believe that analytics in higher education has followed this pattern fairly closely.
Gartner currently locates analytics at the peak of the hype cycle. We disagree. In October 2016, the Campus Computing Project released their annual IT trends survey. In addition to survey data about low satisfaction/efficacy with analytics, the phrase ‘analytics angst’ was used in the report. We see this as the first real signal that analytics has started its natural descent into the trough.
Source of the Hype
1. Media Hype
While ‘blame the media’ is an easy dart to throw, it’s very relevant in this case. We have a perfect storm of factors that make this area ripe for media attention:
- A hotbed topic: student success in higher education
- A sexy, futuristic, ‘cyberpunk’ technology: analytics and predictive modeling
- The potential for financial returns: for both institutions and vendors
Given these factors, there have been numerous cases of media hype around data/analytics in higher ed over the last eight years. There was Knewton claiming to offer robot tutors in the sky (famously called out by Mindwires), InBloom's demise in K-12, and every vendor who has marketed their products using the phrase "big data in education." Even the granddaddy higher ed analytics use case--Purdue's Course Signals--has had challenges.
In reality, not all analytics initiatives are vaporware. Institutions have made measurable advances by leveraging data. It’s just that evidence of success has been harder to come by than many would have hoped. It is very reasonable to draw the conclusion that sustained media attention combined with a much lower output of measurable results has added to the hype.
2. Institutional Capabilities
The second root cause of disillusionment is a bit less obvious. We posit that there is an understandable gap between what an analytics technology provides and the capabilities that institutions need to have in order to use the data effectively.
Let’s explore by looking at a hypothetical use case for student success. Let's say our analytics tool suggests that there's a 41% chance that a specific student will pass their current class. The big question is, so what do we do about it? Who receives this information (Dean, Student Services)? Who is responsible for acting on it (Faculty, Coach/Advisor, Student)? What action should be taken? These are very challenging questions with institution-specific (or program-specific) answers. Even with the success of initiatives like what's been done at Georgia State University, it's estimated that as recently as September 2015 only 2% of institutions [http://www.ithaka.org/news/testing-impact-proactive-advising] used proactive, data-driven advising, so the risk of not being able to act on the data is real.
A good analytics implementation should be led by process and culture, not by technology. Unfortunately, most of the attention is paid to the technology because it is much more tangible than process or culture. The resulting disconnect that appears after implementing the technology is a significant contributor to the disillusionment.
Editor’s Note: This is part one of a two part series. Be on the lookout for Part II: Where do we go from here?
Mike Sharkey is the Vice President of Analytics for Blackboard. He has spent years as an active implementer and collaborator in the higher education analytics space.
Timothy D. Harfield, PhD is Senior Product Marketing Manager for Analytics at Blackboard. Dr. Harfield has published and presented widely on how learning analytics can be used in a variety of contexts to promote student success.