Culture, Care, and Predictive Analytics at the University of South Florida

Case Summary

From 2008 to 2018, the six-year graduation rate of students at the main campus of the University of South Florida (USF) increased from 48% to 73%. This 25-point gain is the largest improvement in this metric among all public doctoral-granting institutions during this ten-year period. However, in the middle of that ten-year span, retention and graduation rates plateaued, compelling university leaders to find new ways to boost persistence and completion rates.

Institutional Profile

USF is a large public research university that serves more than 50,000 students at campuses in Tampa, St. Petersburg, and Sarasota. The performance plateau occurred at the largest campus in Tampa, which serves more than 43,000 students, about 31,000 of them degree-seeking undergraduate students. The 3,500 First Time in College (FTIC) students who enrolled in Tampa for the fall 2019 semester entered with an average SAT score of 1287; 43% of the class graduated in the top ten percent of their high school class, and about 43% identified as underrepresented minority students. Over the past ten years, as USF has raised the academic profile of the FTIC cohort, undergraduate student enrollment has actually become more diverse, as measured by race, ethnicity, and socioeconomic status. More importantly, the university eliminated the graduation rate achievement gap by race, ethnicity, and income.

In 2015, with the retention rate from year one to year two stuck at 88% and the six-year graduation rate holding at 68%, the university was just shy of state performance benchmarks. To quality for higher levels of funding, the university had to hit a 90% retention rate and 70% graduation rate. To get there, the university had to utilize new analytical tools in new ways, breaking down silos between academic affairs, student affairs, information technology, and institutional research.

Overview

Every institution implements a broad range of programs, practices, and policies to promote student success, including revising admissions standards, deploying financial aid leveraging models, adding living learning communities, redesigning gateway courses, promoting student engagement, enhancing career counseling, deploying student success coaches and advocates, applying case management, professionalizing academic advising, and much more. USF has deliberately attempted to change institutional culture and behavior as part of its university-wide student success movement, launched in November 2009.

USF built its student success initiative on the principle that every student admitted to the university will succeed when given the opportunity to do so. We launched promotional campaigns to spread this message across campus, knowing that there were too many faculty, staff, and even some students who did not believe that every student had the potential to succeed. We promoted this message to counter conventional thinking that limited-income, first-generation, and minority students just couldn't be expected to persist and graduate at rates equal to those of other students. Instead, we wanted our community to focus on the many assets that our students brought to the institution—the determination and resilience that often characterize students who see a college degree as a ticket out of poverty and into the middle class.

Without an institutional culture that places students at the center of our work and believes in them, no digital transformation effort will contribute much to student success. Although artificial intelligence can help us identify struggling students, and while technology might empower us to deliver high-touch solutions, the intervention must be done by a caring professional who has been specially trained to assist students.

In 2014, the university deployed an analytics platform from Civitas Learning. I once thought that as soon as that platform went live, all I would need to do is turn it on and learn what I needed to do to "fix" our problems, but the platform by itself didn't solve anything. All it did was shine a light on certain issues, and it took my team over a year to figure out how to act on the insights.

Along the way, we broke down what was left of several institutional silos, including the traditional barrier between student affairs and academic affairs. We also eliminated the barriers that too often exist between those units and the offices of informational technology and institutional research. USF's president and provost made a firm and non-negotiable commitment to student success, and without their support, we would have had a hard time getting all units to apply the insights of predictive analytics.

We formed a Persistence Committee in January 2016, by which time we had spent at least six years creating the institutional culture that made cross-functional collaborations possible. The members of this committee knew, from the start, that they were expected to work together to boost retention and graduation rates, the highest strategic priority of the university.

As this committee developed its own practices, the members contributed to and accelerated a change in institutional culture regarding the use of data. Like other universities, we had typically reported out the official retention rate for a given fall cohort about sixteen months after the cohort first enrolled. Once we had that, we would speculate on the reasons why certain students were not retained. We would often look for insights in "dead" data—a list of non-retained students who subsequently re-enrolled at another institution. By the time we had collected and analyzed the data from the National Student Clearinghouse, those students were hopelessly irretrievable and the reasons why they left might have nothing to do with the behavior or expectations of the currently enrolled FTIC students—who were, by that time, halfway into their first academic year.

We had to find ways to use "live" data to identify struggling students—in real time—so that we could provide them with the support they needed in and out of the classroom. We couldn't wait for mid-term grades. We needed to act on insights given to us on the first day of the fall semester. Plus, that data had to be refreshed daily.

We found a great many of those insights in the Civitas Learning platform, which draws hundreds of data points from our student information system (Banner) and learning management system (Canvas) to sort all students in a given cohort based on the likelihood that they will persist into the next semester or academic year. From that platform, each week we extracted lists of students who were least likely to persist and distributed them to the Persistence Committee. I simply asked that committee to "triage" the list of students and identify those they believed were most likely to benefit from timely and appropriate support.

As the Persistence Committee began to work with these students, we realized that we were applying a case management approach to student success, similar to the structures and processes of our behavioral intervention team. Unfortunately, our case managers (Academic Advocates) were trying to coordinate interventions by a "Care Team" of several hundred people using email and spreadsheets. Our CIO and vice president of information technology, Sidney Fernandes, learned of our interest in devising a case management system and offered to help. He came to his job with a refreshing vision for how the IT unit should operate. IT considered the Student Success team a power user, a particularly important client leading the university's efforts to boost retention and graduation rates. Fernandes's job, as he saw it, was to figure out what the Student Success team needed and then provide them with the tools they needed to do the job. He told me that he could develop a case management platform in twelve weeks. More significantly, he offered our staff the chance to design the tool, meaning that they would end up with a functionality that they—rather than IT or an external vendor—needed.

The mission and vision of this remarkably different CIO made a world of difference. His attitude toward and relationship with Student Success already reflected the cultural change this institution had gone through. The work he and his team would do over the next few months and year would accelerate our transformation. Within three months, our Student Success team had a minimally viable product to utilize in the development of case management. That tool, which we dubbed Archivum Insights, allowed us to coordinate the work of support personnel spread across three campuses. Academic advocates housed in undergraduate studies could refer student "cases" to academic advisors, financial aid counselors, success coaches, career counselors and more. They could share information, post reports, schedule appointments, and close cases.

Lessons Learned

Using predictive analytics—including some home-grown tools—and technology, USF moved off its performance "plateau" and raised its retention rate from 88% to 91% (and climbing). The six-year graduation rate increased from 68% to 73% and then to 75% with the most recent cohort. With these gains, USF qualified for higher levels of state funding and established itself as a preeminent university. As we continue to aim for ever-higher rates of success for our students, we will always remind ourselves of the lessons learned along the way:

• Big data and technology didn't solve any of our problems. People solved our problems.
• Creating an institutional culture supportive of student success is a prerequisite for the application of any analytical or technological solution. If the culture isn't there, the new tools will not likely make an impact.
• Without a team of trained professionals who care about the success of students, we could not provide the right support to the right students at the right time.

Where to Learn More

• Thomas E. Miller and Melissa Irvin, “Using Artificial Intelligence with Human Intelligence for Student Success,” EDUCAUSE Review, December 9, 2019.
• Kevin Carey, "Why More Colleges Should Treat Students Like Numbers," Washington Monthly, September/October 2018.
• Civitas Learning, "From Plateau to Preeminence with a Culture of Care" [https://www.civitaslearning.com/outcomes/university-of-south-florida/], Learning Brief.

Paul Dosal is Vice President for Student Success at the University of South Florida.