Efficiencies, Learning Outcomes Bolstered by Analytics, Data-Informed Decision Making

Over the past decade, credit enrollment at Pennsylvania's Montgomery County Community College (MCCC) has grown by more than 35 percent. Last year alone, more than 34,000 unduplicated credit and non-credit students enrolled in courses online and at MCCC's Blue Bell and Pottstown campuses. The combination of tremendous growth and increased demand for services, coupled with shrinking resources and facility and parking constraints, led the college to investigate potential operating efficiencies that would also enhance student-learning outcomes.

A Focus on Data

With a system-wide structure in place that regularly collects and analyzes data, MCCC's culture has long encouraged continuous improvement through data-informed decision making. In fact, the institution is an Achieving the Dream Leader College that is recognized nationally for leveraging data to align budget decisions and strategic planning efforts with student success goals.

MCCC's data-informed culture emerged more than a decade ago, when a strong focus on data, analytics, and reporting was built in to the 2001–2005 Strategic Plan. Led by the president and executive leadership team, the plan called for the incorporation of quantitative and qualitative data analysis into the consideration process for new initiatives and for program changes. To support this analysis, a department of Institutional Research (IR) was established, and a data warehouse for student registration and demographic data was developed. Today, MCCC has an extensive data warehouse for student, human resources, and finance information, and IR and IT work collaboratively with Academic and Student Affairs faculty and staff to refine reporting and analysis processes that measure student learning outcomes.

The primary challenge to creating a data-informed culture at MCCC was the inability of personnel outside of the IR and IT departments to articulate their reporting and analysis needs. To address this concern, IT trainers began working with end users to demonstrate examples of reports that might benefit them and to promote understanding of data elements. Overall, there was great interest from the end-user community to be self-sufficient in reporting, and through training and support, they began to appreciate the value that data analysis could lend their programs. Through this effort, IT has made data and analytics tools available to a broad population of decision makers, including faculty, academic coordinators, deans, and vice presidents.

The data itself is collected from a variety of sources, including the Community College Survey of Student Engagement (CCSSE), Institutional Effectiveness Model (IEM), Noel-Levitz Student Satisfaction Inventory, and Survey of Entering Student Engagement (SENSE), along with MCCC's Fact Book, High School Yield Report, cohort data, and custom reports. This data is then made available to decision makers in MCCC's data warehouse.

Tools for Decision Makers

MCCC has adopted tools — such as Blackboard Analytics, Ellucian Colleague, Astra Schedule, and Career Coach — to investigate trends in enrollment, learning outcomes, and student engagement, as well as to analyze the use of facilities, parking, and technology services. These tools let decision makers generate custom, live reports that can be shared with faculty coordinators for further discussion and analysis.

There is no "one size fits all" or perfect reporting/analytics tool. Instead, institutions should employ a variety of tools and strategies that help them reach their data goals. For example, MCCC's IR and IT departments included end users in the tool selection and evaluation process that resulted in the purchase of Blackboard Analytics (formerly iStrategy). In addition to ease of use, flexibility is an important consideration when selecting a tool. MCCC's IT team works collaboratively with end users to customize tools in terms of data elements and reports. Of note, all significant reporting and analytics projects at MCCC follow detailed project plans that outline the responsibilities of IT and IR, as well as for the end users.

Regardless of the tool used, MCCC encourages end users to "drill down" when the initial data and analysis inevitably prompt additional "cause to wonder" questions. In so doing, users often discover additional information that leads to more questions and deeper analysis of programs and initiatives.

Scheduling, for example, is one area in which analytics is helping to provide solutions. Given MCCC's tremendous enrollment growth, a shortage of classroom, laboratory, and parking space — especially at the urban Pottstown campus — has become critical. As figure 1 shows, the Astra Schedule tool lets academic administrators examine student enrollment by program, course, and section to identify opportunities for cost savings and cost containment. Low-enrolled sections can be eliminated or redistributed to different days and times to generate additional space and teaching resources for high-demand courses. The goal is to reduce the overall number of course sections by increasing the utilization of existing sections, thereby increasing faculty and facility efficiency.

Figure 1. The Astra Schedule tool

For technology services, such as the technical help desk, the college uses Microsoft Service Manager to track the number, time, and type of call; the areas or individuals to which the help desk ticket is assigned; the status of the ticket; and end-user feedback (see figure 2). These reports let IT managers predict call volume and adjust personnel schedules for optimal customer service and efficiency. The reports also identify common problem areas that can be turned into college-wide professional development and training opportunities.

Figure 2. The Microsoft Service Manager tracking interface

Key Factors in Addressing Challenges

Faculty buy-in is key when introducing data-informed decision making into existing college processes (figure 3). To achieve this, MCCC uses a team approach in its exploration of curriculum development. These teams are generally composed of faculty within a specific discipline — mathematics, for example — with support from the Office of Provost. The collaborative team structure has proven especially beneficial when completing periodic academic program reviews, which are presented to Curriculum Committee of the Board of Trustees and which must be approved by the Board as a whole. IT and IR, in collaboration with Academic Affairs, identified 35 data/analytical reports that are now being used in the academic program review process.

Figure 3. Faculty buy-in to data-informed decision making

This type of collaboration led to significant redesigns in developmental math, English, and reading courses, with the ultimate goal of improving student success. For example, through data analysis, Assistant Mathematics Professor Barbara Lontz discovered that student success rates in Basic Arithmetic (MAT 010), the most basic, pre–college level math course at MCCC, were between 38 and 45 percent. However, students who place into the course (based on their Accuplacer test scores) must successfully complete MAT 010 before they can progress with their education. Lontz and her colleagues took a different approach to teaching MAT 010, structuring the course by concepts rather than topics (discussed in the video below). A few pilot sections were taught in 2008–2009, and in spring 2010, 257 students were enrolled in 13 sections, which had a success rate of 60 percent — 20 percent higher than the success rate in sections that used the traditional approach. Today, 100 percent of MAT 010 sections at MCCC are taught by faculty trained in the "Concepts of Numbers" approach.

Video of Assistant Mathematics Professor Barbara Lontz discussing the redesign of basic arithmetic (4:31 minutes):

Using Analytics to Improve Course Offerings

MCCC is committed to fostering an institutional culture in which decisions are consistently informed by data. As the following examples show, the institution's use of analytics has made a significant impact on a wide cross-section of departments, programs, and courses.

Liberal Studies

MCCC's liberal studies program has the highest enrollment of any degree program. The 60-credit curriculum is highly customizable and lets students enroll in courses that transfer to a diverse array of programs at four-year colleges and universities. However, because of its flexibility, some students might not select course electives that best position them to meet specific program goals upon transferring to a four-year institution.

Stephen Grieco, dean of Arts and Humanities — the division in which liberal studies resides — used Blackboard Analytics to get a clear picture of the program's enrolled students. He discovered that a significant number of liberal studies majors were enrolled in music, dance, and theater courses, and that a large percentage of these students planned to continue their education or pursue careers in these fields.

Based on this information and using a collaborative process that included faculty, IR, and institutional leaders, MCCC introduced three new Associate in Arts degree programs in music, theater, and dance in 2009. Today, the programs are thriving and their students have transferred to renowned conservatories, colleges, and universities across the country.

Stephen Grieco discussing the analytics used to develop new programs in music, theater, and dance (3:19 minutes):

Specialty Certificate in Public Relations

To ensure that its degree offerings are in line with current industry demand, MCCC uses employment projections. In 2011, the college launched Career Coach on its website. Career Coach is an online tool that helps students and community job seekers learn about employment trends, income potential, and required education and training for specific types of jobs.

Using Career Coach, Grieco learned that employment prospects for public relations specialists were trending upward, with a projected 222 annual job openings through 2013 within 25 miles of MCCC's Blue Bell campus. He shared this "cause to wonder" information with Assistant Professor Tom Donlan, who teaches an introductory public relations course in MCCC's Speech Communications department. Donlan and his colleagues then analyzed current course offerings in the Speech Communications program. This led to the development of a second public relations course, Writing for Public Relations Campaigns, and to the development and fall 2012 launch of an 18-credit specialty certificate in public relations.

Creating a Data-Informed Culture

The key to establishing a data-informed decision-making culture is to secure access to an institution's data. Each institution can use different tools to do this. It's not necessary to purchase an expensive analytics tool — although such tools do simplify the process. Most importantly, for a data-informed decision-making process to thrive, institutions must place value and emphasis on reporting and analytics. MCCC achieved this value shift by building data and analysis into the strategic plan as a means to measure the institution's impact on student access and success.

Next Steps: Analytics and Learning Management

Over the next 12 to 18 months, MCCC will seek to expand the use of analytics in its learning management system, moving beyond institutional data and enabling faculty members to track student progress themselves without the need for IT and IR to run reports. Students will also benefit from this data, as they can compare themselves to their peers as a group, thereby learning from and replicating the academic habits of high-achieving students.