Adrift No More: Improving Student Outcomes in Math, Ethics, and Communication

According to the 2011 book Academically Adrift by Richard Arum and Josipa Roska, higher education in general fails to achieve student competency in subjects such as writing.¹ Successful approaches do exist, however. Through a course-level outcomes assessment framework, a large university has demonstrated statistically significant undergraduate student learning gains in communication, ethics, and mathematics. Three large sample-size quantitative studies found:

• A cross-institutional cohort of students showed successive gains in communication.
• Psychology students in 100, 200, and 400-level courses demonstrated gains in ethics and communication.
• College algebra students showed learning and retention gains after a course revision.

While these results might seem expected, they are precisely what Arum and Roksa argue rarely occurs. The studies' results demonstrate the value of analytics in measuring achievement of competency by students, in this case through a course-level outcomes assessment framework.

Institutional Context

Kaplan University is a private, proprietary university accredited by the Higher Learning Commission of the North Central Association. It offers professionally focused associate, baccalaureate, and master's degree programs. Its students are typically older, working, have personal/professional issues that prohibit them from pursuing higher education in a more traditional setting, and have numerous other risk factors (e.g., first-generation college attendance).

The institution's assessment model is outcomes-driven course- and program-level assessment. Faculty in each course measure their students' proficiency using common rubrics to assess assignments ranging from essays to examinations to discussions and more.

In 2008, the university implemented a general education program comprising both a core curriculum and the embedding of standardized general education outcomes across the curriculum in art and humanities, critical thinking, communication, ethics, mathematics, research, science, social science, and technology. These outcomes, which support the core curriculum of two courses in communication (composition) and one each in math, the humanities, science, and social science, are embedded across the remaining major requirements and electives. One third of the major requirements include an ethics outcome, one third a critical thinking outcome, and one third a research outcome. Among elective courses, one quarter include a math outcome, one quarter humanities, one quarter science, and one quarter social science. All courses include a communication outcome and require the application of technological skills. Embedding the instruction and assessment of these outcomes in students' major disciplines helps them gain proficiency in the vital skills needed to succeed in college and beyond.

In the curriculum, faculty assess student competency in each of a given course's outcomes — discipline-specific and those tied to general education — using standardized rubrics and identifying student progress towards mastery on a six-point scale (no progress, introductory, emergent, practiced, proficient, and mastery). Faculty members enter these assessment scores directly into the online class grade book, where students can view them immediately. These scores are then bulk uploaded into a centralized institutional database. Scores on outcomes are analyzed to ensure that students are gaining the desired mastery of all learning outcomes. Results from the assessment of student learning are examined to determine whether students demonstrate progress towards mastery in general education fields across programs. Results are reviewed as part of the continuous improvement process.

Student Learning Gains Demonstrated

The purpose of the quantitative studies summarized in this article was to measure the impact on student learning gains of the model described. In the first study, 679 students completing three courses in sequence were assessed on communication. In the second study, 2,581 psychology students were evaluated on their attainment of ethical and communicative skills. In the final study, the outcomes attainment of 3,574 college algebra students was measured before and after a major revision to the course.

No. 1: Communication Cohort Study

The goal of any Writing across the Curriculum program is to help students not only practice writing continually but also connect the critical skill of effective communication to their major and future profession. Through this reinforcement, our students have come to understand the importance of writing and communicating well, as captured by one information technology (IT) major:

"Writing is not just for writers. Every profession, even IT, places importance on writing."

Even a student who was dissatisfied with her performance made the connection:

"My professor is an excellent teacher. My grades do not reflect what I have gained from this course. I already am applying the knowledge at work. I know that what I have learned will pay forward in both my personal and business life for years to come."

This study analyzed the communication proficiency of undergraduate students across degree programs (N = 679). The students, who were completing three courses in sequence, were assessed on the communication outcome: "Demonstrate college-level communication through the composition of original materials in Standard English." This study reviewed the percentage achieving "practiced" or higher (3 or more) as the students advanced through the sequence.

Students achieving "practiced" or better increased from 77.5 percent in the first course in the sequence to 84.5 percent in the second course and 88.5 percent in the last course in the sequence — a significant result demonstrating a documented improvement of this core academic skill (figure 1).

Figure 1. Improvement in communication skills over the three-course sequence

No. 2: Ethics and Communication Study

General education ethics and communication outcome data was analyzed from undergraduate students (N = 2,581) in the psychology program. General education outcome scores were analyzed from the first term of 2010 from 100-, 200-, and 400-level courses. The analysis established a significant difference in the average score on an ethics outcome ("Identify the ethical issues within a field") across course level (F(2,2462) = 261.32, p < .001). Average scores were 2.72, 3.54, and 3.64 for students in course levels 100, 200, and 400, respectively. A similar trend was found for the communication outcome, with significantly different average scores of 3.20, 3.49, and 3.54 across the same course levels (F(2,2578) = 52.01, p < .001). Figure 2 shows both trends.

Figure 2. Improved outcomes in ethics and communication competencies

Incorporating ethics into the students' major-specific course clearly helped them connect the principles of ethics to their profession:

"I enjoyed this course primarily because it opened my eyes up to the strategies involved with making conclusions and diagnoses in a clinical setting. Most disorders and issues that need attention can be properly assessed by following ethical guidelines while administering specific assessments. Great course."

No. 3: College Algebra Course Revisions

A revision of MM212 College Algebra was undertaken in late 2010. The revision incorporated adaptive learning techniques, low-stakes practice, and individualized learning. Student comments from before and after the revision underscore the change in approach.

A student comment prior to the revision of MM212:

"This course had way too much information to absorb in certain units. The last 2 units had way too much. We were not able to go through all of the material in the one hour seminar. The course needs to be reworked AGAIN!!"

A student comment made after the revision to MM212:

"I was initially terrified to take this course and had to withdraw from another school as I could not keep up. I am so thankful for the format of this class as it allowed me to focus on the work and not just passing a test. I do not do math well under pressure and have severe anxiety when tested. This class format for me was ideal and actually allowed me the opportunity to learn the material. This is a class I will never forget and academically speaking is my pride!"

This study compared student retention (defined as passing the course) and learning outcome attainment (CLA: Course Level Assessment) pre- (N = 2,805, 2010) and post-revision (N = 769, 2011). The results demonstrate that given the right set of changes (adaptive learning techniques, low-stakes practice, and individualized learning) and the opportunity, a high percentage of students can achieve mastery of course-level outcomes in introductory, college-level mathematics courses. Figure 3 shows results for students passing the course, and figure 4 shows learning attainment.

Figure 3. Retention of students in math; red circle marks revised course launch date

Figure 4. Assessed learning in math; red circle marks revised course launch date

Conclusion

All undergraduate institutions grapple with the best way to teach their students the core competencies of general education. The model discussed here has many advantages:

• It provides the ability to directly measure students' performance across their programs.
• It integrates general education skills, thus assisting students in their understanding of the broad applicability of this knowledge, specifically to their future professions.
• It drives continuous curricular improvement because the exact degree of student learning can be captured and acted upon whenever the institution desires.

The quantitative studies briefly described here support the concept of embedding core competencies across the curriculum and measuring their impact at the course level.

1. Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College Campuses (Chicago: University of Chicago Press, 2011).