Using Design Statements in AI-Enhanced Composition: A Small Institution's Approach

The dogmas of the quiet past are inadequate to the stormy present. The occasion is piled high with difficulty, and we must rise—with the occasion. As our case is new, so we must think anew, and act anew. We must disenthrall ourselves, and then we shall [succeed].

—Abraham Lincoln, "Annual Message to Congress: Concluding Remarks," December 1, 1862

Artificial intelligence (AI) has been a significant disruptor in higher education. Following the release of ChatGPT and other generative AI tools over the last several years, the debate surrounding the use of AI tools has intensified. Due to its ability to take direction from the user and instantly generate a document of arguably similar quality as one created through the traditional writing process, many people in the academic community believe that AI use represents abject academic dishonesty and plagiarism and will serve to negate the significance of an academic degree. Others regard AI tools as simply that: tools to be used with care and discretion. Just as using a calculator helps students to be more efficient in their math courses (without taking away critical thinking), AI can help students in other disciplines understand their assignments better or help them get started, increasing their productivity while still requiring them to use critical thinking skills to create a quality product.

When ChatGPT was released in late 2022, institutions like Tiffin University (TU) had to decide how to address students' use of AI tools to complete their coursework. As the use of AI expanded, TU chose not only to allow but also integrate AI directly into its courses and curriculum. During the summer of 2023, the provost and chief operating officer at TU sent faculty a series of emails explaining the institutional stance on AI, outlining expectations for faculty, and describing initial interventions that would be implemented at the start of the fall 2023 semester. The official statement from the university regarding the use of AI takes the position that it is the responsibility of TU to "ensure [students] graduate with the ability to think critically, demonstrate creativity, and evaluate information," which includes using generative artificial intelligence tools (GAITs) as learning resources. This statement stressed that part of TU's responsibility in preparing students for life beyond their undergraduate experience and into their careers is the "obligation to prepare [them] for the ethical and legal use of AI."^Footnote1

In addition to this message, faculty were given a series of directives to implement in their teaching during the upcoming academic year. First and foremost, faculty were instructed to include the university's official statement on AI use in their syllabi and refrain from adding any language that prohibits the use of GAITs in their classes. Faculty were also given a list of specific expectations about handling AI use, including furthering their familiarity with GAITs, anticipating students' use of these tools, and adjusting their coursework to embrace GAITs. In addition to these guidelines, pre-semester meetings held in August 2023 included dedicated time to train on-campus faculty on when and how to use AI tools in the classroom. Full-time and adjunct faculty generally responded positively to these sessions and were receptive to the provost's messages. Despite some uncertainty regarding the scope and capabilities of generative AI tools, most faculty members were on board and curious about ways they could use GAITs to improve their classes.

While communication from the provost's office made the university's intentions regarding the use of AI  clear to faculty, the question remained: what would be the best way to handle AI use in an online setting, where the composition-heavy nature of courses made students' use of GAITs all but certain? Given the short timeframe and the significant number of online courses involved (more than four hundred), the Center for Online and Extended Learning (COEL) implemented short- and long-term interventions to integrate AI into the curriculum. In the long term, the COEL team will review and redesign assignments within courses to be more authentic and pivot the focus of assessment from the finished assignment to the completion process. One of the short-term actions was adding a brightly colored dialog box at the beginning of every online course that explains the institutional stance on AI use. The box contains a link to a webpage with videos and other resources discussing the appropriate use of GAITs, including a general overview, tutorials, an ethical guide, and more. In addition to introducing students to the idea that they are welcome and encouraged to use GAITs, these resources are intended to build students' trust that instructors will not penalize them for ethical AI use. Working within the historically punitive world of higher education, building trust between students and instructors is arguably the most challenging but important objective amid the rapid advancement of generative AI tools. Without consistency among faculty in honoring this no-policing policy, students are unlikely to trust that their use of GAITs will be accepted, and, as a result, instructors will not get a clear picture of how their students complete assignments.

Another short-term but pedagogically significant intervention that the COEL team incorporated into online courses was requiring students to include design statements at the end of their written assignments. A design statement is a short, three- to four-sentence paragraph explaining the process the student used to complete an assignment. Creating these statements is intended to be a form of metacognition for students: they are instructed to be specific and detailed about their process, describing how they came up with their ideas, researched the necessary information, etc. The instructions explaining how to write a design statement encourage students to use a less formal writing tone than they would when completing an academic assignment. While students are not expected to list the sources they use in their assignments, they are expected to list the specific tools they use. For example, a student might write, "I used TU's library database to find two scholarly articles that supported my thesis," or "I used ChatGPT to summarize an article that I was having trouble understanding." In these examples, the tools used (the library database and ChatGPT, respectively) are listed rather than the specific articles in which the information was found.

A study done at the University of Michigan (UM) helps frame what the COEL team observed among TU students. The authors of the UM study explained the importance of metacognition, stating, "Research in education has identified the importance of helping students develop the ability to monitor their own comprehension and to make their thinking processes explicit to teachers."^Footnote2 Specifically, the researchers sought to determine if using metacognition would lead students to "become better disciplinary thinkers," and whether instructors' focus on metacognition would "help students better connect diverse disciplinary writing tasks and develop more versatile identities as disciplinary writers."^Footnote3 While the UM study focuses more on specific metacognitive practices, it provides some suggestions that are applicable to the design statement approach.

Because design statements require students to think about and document how they complete an assignment from start to finish, they facilitate conversations between instructors and students about how to find and use accurate information. This skill serves students beyond their years at TU and fulfills the obligation to prepare students to use AI ethically and legally. Like work shown on a math problem, design statements are intended to give instructors insight into how students create and polish their assignments without adding significantly to students' workloads, thus providing an opportunity for instructors to teach students to use GAITs responsibly.^Footnote4

Process

To evaluate the efficacy of the design statement concept, the COEL team examined submitted materials from four disparate courses: a general education English course that all TU students are required to take (two sections); an undergraduate course that is part of the core curriculum for business students; and a course that master of business administration (MBA) students at TU are required to take. For each course, the COEL team examined design statements from early in the seven-week term and late in the term. These weeks were selected intentionally to ascertain whether the quality or depth of the design statements improved from the beginning of the term to the end.

The chosen assignments consisted of essays, research papers, presentations, proposal outlines, and analyses. For the four total assignments examined in the two undergraduate courses, 150 design statements were submitted by seventy-six students; in the graduate course, fifty-four design statements across two assignments were submitted by thirty-two students. The design statements were anonymized, randomized, and reviewed for word count and content. The reviewers kept a tally of the tools and resources students mentioned in their design statements. Grammar and citation tools, course materials, various online databases, and unspecified internet sources appeared most frequently. Finally, COEL team members contacted and interviewed the instructors of the courses to determine if and how they engaged with and used design statements. Despite the limited scope of this study and the nascent nature of the AI interventions at TU, a qualitative analysis of the data revealed some actionable insights into the relative success of the effort and the work left to be done.

Findings

Initial findings were both informative and, in many ways, predictable. In general, submitted design statements indicated that students did not fully understand the purpose of the statement or the expectations related to it. Overall, the quality of the statements did not improve from the beginning of the course to the end. Some design statements demonstrated obvious use of GAITs, even when such tools were not cited as resources. Design statements included with assignments from the later weeks of the term often showed little to no growth and remained steadfastly in the "ambiguous" category. For example, one student wrote, "To create this essay, I used multiple outside sources in order to support my argument. I also provided the reader with opposing viewpoints and rebutted them to the best of my ability. I also added the title page and sources." In this example, the phrase "multiple outside sources" is not specific enough to identify the tools the student used. What were the outside sources? Did they include AI tools? Without specificity, faculty are unable to get a clear picture of the tools that students used on their assignments and are, therefore, unable to intervene should the need arise.

Beyond vague responses, reviewers also noted a prevalence of "fluffy" writing—with high word counts and an overreliance on advanced vocabulary (often used incorrectly)—in the design statements submitted by students in both the undergraduate and graduate courses. Since these types of responses were common both early and later in the term, the COEL team wondered if students had received constructive feedback or other guidance on their early design statements. Two instructors noted that they had read the design statements, but only one of the two provided feedback on them in the early weeks of the course. The instructors also noted that the expectations for the design statements were unclear, specifically whether the statements should be included as part of the students' final assignment grade and if instructors should provide feedback at all. These comments suggest that while the ultimate goal of the COEL is to review and revise the online courses at TU to include more authentic assignments, the short-term interventions should be surrounded by more discussion between students and instructors. This effort starts with instructor training.

Although the student-submitted design statements fell short of their pedagogical potential, this outcome was unsurprising after a pilot term. However, the implementation of the design statements demonstrates that students are willing to explain the process they use to complete their assignments, even if they have not been fully trained to do so. As the design statement initiative continues, instructors will work with students to help them understand the pedagogical purpose of these statements, ideally fueling the creation of meaningful statements that will help instructors and inform their feedback.

An important side benefit of the design statement process is building trust between students and instructors. By engaging students in an open dialogue about these statements and the use of GAITs in assignments, students will begin to understand that instructors do not intend to police their AI use. As a result, they are more likely to be transparent about the tools they use on assignments and less fearful of being penalized for using GAITs.

Next Steps

As part of a recent training session for online instructors, Dan Clark, dean of the School of Education and Extended Learning at TU, summarized AI efforts at the university in the following way:

If you think AI is changing the world, you haven't been paying attention; it has already changed it, and there is no going back. Students who graduate in the near future who are not fully versed in the ethical and effective use of generative AI tools will be at a disadvantage in an increasingly competitive job market. To say that AI is a disruption in higher education is an understatement, but rather than fear it, Tiffin University has pledged to embrace it as an opportunity, an equalizer, and an indispensable tool to ensure student success. Though some might consider our approach somewhat daring and ambitious, in reality establishing this philosophy and direction was the easy part; now begins the hard work of building the structures and supports for students and faculty that embody this philosophy.^Footnote5

To ensure the relevance of online education in the face of the advancing capabilities of GAITs, the entire online course catalog at TU will need to be heavily modified to integrate more authentic, AI-enhanced assignments. Even as this massive undertaking is begun, implementing short-term interventions and practices, such as the design statements initiative, is a logical and critical first step.

While initial returns may have fallen short of hoped-for results, the institution must take advantage of students' willingness to engage in the design statement process. Relying solely on a webpage and an institutional statement to train students on how and why they engage in this task is insufficient. Instructors must initiate open conversations through discussions and feedback to gain students' trust and hone their inclination to share their processes. Because instructors play a critical role in communicating and facilitating new practices and policies for students, they must be well-informed about them.

To ensure that instructors fully engage in their part of the design statement process and understand the pedagogical importance of this approach, the COEL team must provide ongoing faculty training. In addition to understanding the significance of the design statement process, instructors should know how to respond to and grade design statements. If instructors can give students more meaningful feedback on their design statements, there is a better chance that the statements will fulfill their intended purpose. Design statements have the potential to provide the institution with a clear picture of the tools (GAITs or otherwise) students use to complete their assignments and help instructors put interventions in place to ensure students use these tools appropriately. The design statement approach will also help the institution adjust to the ever-changing capabilities of AI tools.

The most challenging, and arguably most important, next step goes beyond logistics and new practices and focuses on shifting from a product-oriented assignment paradigm to a process-oriented one. Returning to the math analogy, it is common practice for math instructors to require students to show their work, and students typically earn points for their process, even if their answer is incorrect. GAITs present an opportunity for non-math instructors to adopt a similar practice by examining the process students use to create an assignment and intervening when necessary rather than looking only at the finished product. Design statements are a window into students' processes. They are not meant to catch students out but to ensure that students learn to use the tools they are given ethically and responsibly.

Notes

1. Peter Holbrook, PhD, email to faculty, Tiffin University, June 12, 2023. Jump back to footnote 1 in the text.↩
2. Naomi Silver et al., Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy (Sterling, VA: Stylus, 2013), 2. Jump back to footnote 2 in the text.↩
3. Silver, Using Reflection and Metacognition, 3. Jump back to footnote 3 in the text.↩
4. "Design Statements," TU Online Resources, Tiffin University (website), accessed December 14, 2023. Jump back to footnote 4 in the text.↩
5. Dan Clark, with Jason Bock and Luis Galarza, "Embracing Artificial Intelligence (AI) at Tiffin University" (lecture, AVENU Learning International Faculty Seminar, Virtual, 2023). Jump back to footnote 5 in the text.↩

Madelyn Iannantuono is Project Coordinator, Center for Online & Extended Learning, at Tiffin University.