Solutions, or Data-Driven Decision-Making?

Business analytics is quickly infusing its way into the decision-making processes of higher education. We all regularly hear the siren call "show me the data." This is an opportunity-rich step beyond the emotionally or empirically based decisions that have long been the cornerstone of decision-making. Under this "tyranny of the empirical," if student X has a particular problem, it must be universally true for all students. The natural response, then, is "Let's fix this problem!" The so-called problem, though, is frequently something that we don't truly understand and that pulls resources from other issues that arguably have a higher need for those resources. Resources that are increasingly scarce at institutions of higher education certainly cannot be spent fixing problems that may not exist in the first place. Using data to inform rather than drive decisions is one incremental step forward—and a giant evolutionary step for administrator-kind.

Analytics turns out to have many gradations, and each step up the pyramid of data use leads to wisdom. At the base of the pyramid is simply pulling and compiling data—describing an issue. Descriptive data do not provide much insight and certainly don't predict anything. This step is simply an accounting of data. The next step is to analyze data against other data and try to identify relationships—insights. The insightful step identifies that something is happening and helps us identify possible reasons why it is happening—the drivers, correlations, and relational influences. The final step up the wisdom pyramid is predictive. The insight and descriptive information allow the forecasting of what will happen and its probability of occurring: "The conditions are right for X to happen, and there's a 60% likelihood that it will happen—slightly better than a coin toss."

Each of these steps toward wisdom requires experienced managers and analysts to thoughtfully review the information, identify gaps in knowledge, challenge—or "truth"—the data, and incorporate other information to develop a full picture before making a final call. Decisions should never be made solely on the basis of "what the data say." Good decision-making should not be management by formula, lest we replace "tyranny of the empirical" with "tyranny of the data."

When I hear administrators proudly state that they have created a "data-driven culture," I cringe. Although being driven by data can be motivated by good intentions and a sincere recognition of the numerous opportunities that analytics provides, "data driven" suggests loyalty to the formula and discounts the role of the thoughtful analysis that goes into all good decision-making. My dean and mentor, Shane Burgess, espouses being "data informed," and here's why that makes sense.

A data-informed approach recognizes the role that capable leaders and managers play in interpreting data and shaping understanding, including known weaknesses. Data are agnostic. Data only become useful when put into context, compared against another piece of data, or further "crunched" to a new form. All of this work is the human process that turns an otherwise meaningless factoid into a useful and insightful or predictive competitive advantage. This is the value that analytics provides over database management.

Being data informed respects the potential usefulness of the information. Data informed is one step up the evolutionary ladder from data driven. Take, for instance, an effort to raise the retention rate of first-generation students. A data-driven approach to improve this metric might start by pulling information from central data warehouses and discovering that first-generation students often struggle with English 101. The logical next step is to increase the use of English tutoring services by first-generation students. Money and time are spent marketing tutoring services. When the improvement is only marginally successful, the next step is to add more tutors and spend money on professional development and training for tutors. Still, the improvement is only marginally better, but the tutors are happy at the new resources and attention, and the first-generation students may feel singled out by the extra attention.

In contrast, the process with a data-informed decision would go deeper before a decision is made and money is spent. English 101 would be identified, but then more questions would be asked based on the experience and wisdom of the analysts and decision makers. This would lead to discussions with the tutors, advisors, instructors, and the students themselves. The result is a more nuanced and richer view of the underlying relationships and challenges. The data-informed approach might lead to the realization that English 101 has nothing to do with the underlying issues but is simply a symptom of the true challenge: that first-generation students face many hurdles that multigenerational college families don't, including financial challenges, family pressures and perceptions, part-time jobs, a lack of resources and knowledge to draw on when financial challenges present themselves, language barriers, and so forth. This investigation of letting the data guide the exploratory process can lead to an improved suite of support services, such as a micro-emergency financial loan program to help students focus on learning.

I am both heartened and fearful at the rapid adoption of data in the decision-making of higher education. On the one hand, data will certainly improve decision-making processes. On the other hand, we could be creating a new tyranny that does not solve problems. As human beings, we are prone to believing that we truly know what's occurring, regardless of whether we really do. "It looks and sounds like a winning solution, so let's decide already." But is it a winning solution? Each of us, and our perceptions of the world, is a sample size of one, which is not enough to make decisions except what we want for lunch today.

How many new practices and tools have been introduced in higher education that have proven only marginally successful or have been abandoned during the implementation phase? A lot rides on our ability to get decisions right—student success, the next medical research breakthrough, who can enjoy the social mobility afforded by a higher education, and on and on. Decisions are more critical now due to shrinking budgets and greater administrative accountability. Let's slow down our thinking. What are we trying to solve with data? Apply the scientific method to identify relational drivers as opposed to just spurious correlates. Do the data actually solve the problem? Do they raise more questions? Or are they just factoids? The difference between data driven and data informed is not trivial. It can be the difference between an illusion of knowing how to address a problem and a solution that solves the root cause.

---

Jeffrey Ratje is Associate Vice President, Finance, Administration and Operations, Agriculture, Life and Veterinary Sciences, and Cooperative Extension at the University of Arizona.