Using Information to Drive Meaningful Change

Leaders from higher education information technology (IT), business operations, and institutional research (IR) met in Orlando in March 2018 as part of a first-ever collaboration of EDUCAUSE, the National Association of College and University Business Officers (NACUBO), and the Association for Institutional Research (AIR). Presentations and discussions examined practical ways to use information derived from analytics to drive meaningful change at colleges and universities.

The 2018 Enterprise IT Summit examined three types of success for analytics initiatives: collaborative success, operational success, and student success. It also focused on the ethics of analytics as it plays out across all analytics initiatives. This report describes the recommendations that emerged from summit presentations and discussions in these four areas.

Analytics Overview

Colleges and universities typically have access to vast stores of data from both their ever-increasing enterprise systems and their access to external or informal data sources. However, most institutions have yet to make the best use of those data to derive the information required for strategic decision-making and meaningful change.

Previous enterprise IT summits have considered how institutions could advance the maturity of their analytics capabilities and the importance of cross-enterprise partnerships for institutional change. The 2018 summit extended that partnership focus to include IR and examined how that collaboration might further analytics-related institutional goals.

For analytics endeavors to succeed, collaboration among leaders from IT, IR, and business operations is essential, as each leader brings an important and distinct voice to the analytics table. In their opening remarks, the CEOs of the three participating associations described the importance of this collaboration.

Christine Keller, AIR's executive director and CEO, said that the time has come for leaders from these areas to play a more proactive and visible leadership role on our campuses. In her words, "We can longer be silently awesome in our corner offices."

John Walda, NACUBO's president and CEO, described how the position of chief business officer has evolved from a highly technical role to one that is strategically focused. "That means that the CBO must work collaboratively with others on campus, with other institutions, and with other associations to solve problems."

EDUCAUSE President and CEO John O'Brien added, "The skills that got us here won't get us to the place where we need to be. We need to develop new skills in cross-enterprise collaboration to help our institutions be successful. To be an effective leader means you are part of the strategic planning fabric of the institution and the key to innovation."

Keynote: Data Challenges

In the summit's keynote presentation, Scott Jaschik, editor of Inside Higher Ed, addressed the urgency of this analytics work and the importance of coming together around data. Jaschik described the need for the three groups to collaborate as they help their institutions use data to meet the growing challenges faced by higher education.

Jaschik said that several recent Inside Higher Ed surveys of provosts found that most institutions collect data on outcomes, graduation rates, and so on but that few effectively use those data for informed decision making. So, while we have the data, he said, we have yet to figure out the best way to use those data to drive success on our campuses. Achieving this is complicated and requires effort; as Jaschik noted, we can't simply hand data to campus leaders and expect that they will make good use of them.

Jaschik went on to describe a recent conference attended by provosts and institutional researchers, who had a notable difference in their viewpoints on data use on their campuses. The researcher attendees, he said, viewed the situation as: "I give you all these data, but you don't look at them." In contrast, the provost view was: "You give us too much data; you need to understand the importance of the one-pager."

Further, Jaschik suggested that, while higher education assessment technology shows promise, too few colleges and universities have invested the time and energy required to train and support people in using the data.

Three Perspectives on Analytics Initiatives

Throughout the summit, presenters and attendees discussed how colleges and universities can make progress on data and analytics initiatives. They considered these initiatives through three lenses: collaborative success, operational success, and student success.

Collaborative Success

The importance of collaboration was highlighted in presentations and discussions across all categories, emphasizing collaboration as a requirement for success in any type of analytics initiative.

Collaboration is a necessary foundation for analytics efforts. It starts at the beginning of an initiative with the need to foster a shared vision; it continues throughout the project in areas such as data governance, communication, and change management.

The related topic of governance also emerged as important across all categories. Data are increasingly recognized as institutional strategic assets, but to realize the benefits, data must be effectively managed. An effective data governance program helps institutions maintain data integrity, control data access, secure data, and ensure best practices related to data use. (For more information about data governance, see the EDUCAUSE Working Group paper The Compelling Case for Data Governance.)

Many attendee discussions here focused on restarting stalled governance work — or even simply beginning that work. Data governance is an area in which many institutions appear to be struggling.

Collaborative Success Recommendations

• Bring people together to build a shared vision.
  • Convene representatives from IR, IT, academic affairs, and the business office to develop and prioritize goals.
  • Consider devoting time and resources to a retreat aimed at developing a shared vision.
  • Establish focus groups to increase understanding of stakeholder needs.
  • Hold discussions with key stakeholders to understand the gap between their information needs and what is currently available to them.
  • Involve all stakeholders in developing common definitions.
  • Use collaboration when determining who has access to data.
• Get out of your silos.
  • Attend events (such as the summit) that bring together people from different areas and participate in other cross-functional conversations to get new ideas.
  • Get exposure to ideas outside areas of expertise — not only for institutional leaders, but also for staff members who do the day-to-day work.
  • Go to events outside your own association, such as having IT leaders attend the NACUBO annual meeting or the AIR Forum to develop a deeper understanding of other areas.
• Focus on the value of teamwork.
  • Keep in mind that we need each other. With a strong team of IT, IR, and business operations leaders, for example, it is possible to represent or underscore each other's viewpoints in meetings.
  • Take advantage of the fact that IT, IR, and business operations each bring different perspectives and are involved in different conversations on campus. Each person's work complements the work of others; each has different networks and different professional organizations.
• Focus on data governance efforts.
  • Establish a data governance committee if none exists or reconvene the committee if it has languished.
  • Get support from senior leadership for data governance work.
  • Develop an understanding about the relationship between data governance, ethics, and privacy, and don't expect your data governance committee to also take care of the ethics work.

Operational Success

As with collaborative success, the operational success presentations and discussions focused on the importance of developing a shared vision for analytics initiatives. The presentations also highlighted issues related to culture change, training, and convening stakeholders.

Operational Success Recommendations

• Bring the right people to the table from across the institution to agree on a vision.
  • Include senior leadership in planning for operational analytics so that the president and cabinet fully understand the importance of data.
  • Take the conversation beyond the leadership level so a shared vision permeates the organization.
  • Understand stakeholders' visions for success.
  • Start conversations with stakeholders by asking which problems need to be solved and determine the gaps in data related to meeting their needs.
• Provide training where needed.
  • Use training to raise the level of understanding and professionalism around data and analytics across all staff levels.
  • Understand the value of analytics and each staff member's strategic connection to analytics work, and ensure that staff members understand how their work contributes.
  • Develop common terminology and training to ensure understanding.
  • Identify resources for professional development related to data governance.
  • Empower users by teaching them how to find and interpret data.
• Focus on change management.
  • Shift the thinking away from meeting data requests and toward solving business needs.
  • Develop positive change management techniques to help foster a data-informed decision-making culture.

Student Success

Similar to the other areas, student success presentations and discussions emphasized the need for a shared vision across the institution. They also focused on the need to foster a culture change to raise awareness of the benefits of analytics. Presentations and discussions described the importance of asking good questions — suggesting that all the data in the world won't tell you anything unless you know which questions you're trying to answer.

Student Success Recommendations

• Develop a shared vision for student success and use it in change management efforts.
  • Develop strong support from senior leadership and ensure that support comes from a position of knowledge and understanding.
  • Look for data champions among faculty to help spread the vision.
  • Focus work on the end game: What makes a difference to students?
• Foster an understanding of the role and possibilities of analytics.
  • Help stakeholders understand how analytics can be used in strategic decision-making.
  • Understand the questions being asked, and then provide synthesized, interpreted information rather than just data.
  • Host regular gatherings on special data topics (retention, graduation rates, etc.) and invite faculty and staff to learn about how analytics can address the target issue.
  • Work with all stakeholders to develop common definitions.
• Work to develop a culture of evidence within the institution.
  • Provide training and guidance on how to use data and information, rather than simply distributing data.
  • Make sure people understand why they are receiving information.
  • Be intentional about the goal of developing a culture of evidence.
  • Provide training on ethics and ways to use data effectively and ethically.

Panel Discussion: The Ethics of Analytics

The summit's panel discussion on the ethics of analytics generated lively discussion. Panelists and attendees agreed that IT, IR, and business operations leaders face a key challenge: They must educate themselves about these issues while simultaneously leading their colleges and universities forward on them. The ethics discussions focused on three key topics.

Explaining Data Collection and Use

Panelists stressed the importance of institutional transparency about data collection, helping students understand which data are being collected about them and how the data are being used.

Several attendees and panelists further suggested the importance of giving students an opt-in choice whenever possible regarding the collection and use of data about them or their activities. Panelists agreed that student consent is an important consideration, something that many students (as well as faculty and staff) don't fully understand. Some panelists recommended that institutions hold campus discussions about data use and include a variety of student voices in those discussions.

Because laws regarding student data vary from state to state — particularly when it comes to underage students — panelists also recommended that attendees familiarize themselves with their own state's laws related to student data.

Raising Awareness of Ethical Issues

Panelists agreed on the need to work toward a cultural shift around ethics. In particular, institutions should not expect their faculty, staff, administrators, and students to understand the issues involved in the ethical use of data and analytics.

As an example, a common reaction to a concern about ethics is, "I sign off on the FERPA agreement not to release grades, so I'm good." In reality, FERPA (the Family Educational Rights and Privacy Act) represents only a small piece of the data-ethics puzzle.

Institutional leaders must increase awareness of ethics issues among all campus stakeholders to help them understand what the institution is trying to achieve through an analytics initiative and how ethics awareness is part of that work.

Distinguishing Ethics and Governance

Presenters and attendees stressed that data ethics and data governance have a strong connection but should not be mistaken for each other. In fact, panelists argued that an ethics group or committee should be separate from the data governance group. That is, the ethics group should serve as an independent voice related to and informing data use, and both groups should contribute to the conversation about data management, include the right people, and represent their stakeholders appropriately.

Further, an ethics group should include representation from the people whose data are being collected so they can question and understand the purpose of the data collection and how the data will be protected. Getting those concerns resolved prior to implementation can help an analytics initiative succeed. The ethics group should also include IT security staff to inform discussions about actions in case of a data breach.

Collaboration and Community Are Key

As O'Brien said in his opening remarks, "The future of IT and IT leaders isn't about professionally talking more intensely to ourselves. Our future is about talking effectively, intensely, and collaboratively with our colleagues across the institution." The same is true for IR and business leaders. Clearly IT, IR, and business leaders can play a positive and powerful role in advancing college and university goals — if they work collaboratively.

Guideposts for this work can be found in the themes featured repeatedly in the 2018 Enterprise IT Summit's presentations and discussions:

• Develop a shared vision for analytics initiatives.
• Think carefully about stakeholders and get them involved early and often.
• Work to mature an effective data governance practice.
• Provide training and support to help constituents understand information.
• Educate yourselves and others about data governance.
• Remember that this work requires culture change.

---

Betsy Tippens Reinitz is the Director of Enterprise IT Programs at EDUCAUSE.