The Effects of COVID-19 on Learning Space Rating System Scores

During the fall 2020 semester, the University of Notre Dame, like all other colleges and universities, had to make adjustments to its classrooms and traditional models of teaching in order to accommodate learning in a world with COVID-19. When the COVID-19 pandemic hit in spring 2020, Notre Dame transitioned to completely online learning, with no students on campus. For fall 2020, Notre Dame adopted multiple modes of teaching for its classes, with some online, some in person, and some hybrid learning where half the class attended in-person while the other half attended online. Every in-person class had the capacity to be "dual mode," with the professor and some students in-person while students who could not come to class attended live remotely. Students diagnosed with COVID-19, at high risk for COVID-19, in contact with someone who had COVID-19, or with symptoms of COVID-19 were able to stay isolated by attending class remotely. We investigated how changes made in response to the COVID-19 pandemic affected the Learning Space Rating System (LSRS) scores of various classrooms at Notre Dame.

Changes to Schedules and Classrooms

To accommodate social distancing and smaller class sizes, university officials expanded the locations and times of classes. Previously, classes were held Monday through Friday, 8:00 a.m. to 5:00 p.m. This semester, classes were held Monday through Sunday, 8:00 a.m. to 9:00 p.m. Forty-four classrooms with a pre-COVID-19 capacity of twenty-four people were closed for the year because their seating capacity became too low to hold a class while social distancing. Twenty-eight nontraditional venues were opened for classes, including the student activities convocation space, student theatre, ballrooms, music recital halls, and the museum theatre.

To allow for dual-mode delivery and COVID-19 safety precautions, 130 classrooms operated and assigned by the registrar's office were upgraded by ND Studios Audio Video Technologies group—with design consultation with stakeholders from the Teaching and Learning Technologies, registrar, and Center for Teaching and Learning units—with new technology and modified seating. A second computer monitor was added to the professor workstation, allowing instructors to manage remote students on one monitor and class materials on the other. Some rooms were given cameras mounted in the back of the class, which provided a wider view of the front, and some were given participant microphones throughout the room so students on Zoom could hear in-person students. In rooms with moveable chairs, some were removed so all seats were six feet apart. In rooms without movable chairs, seats were labeled with stickers instructing students where to sit to remain six feet apart. Students were expected to sit in the same seat all semester, and seats were reported for contact tracing. The front of each room was taped off to separate the professor's space from the students' space, and moving throughout the classroom except for entering and leaving was not allowed.

Investigating the Effects of Room Changes

For this research project, we chose ten rooms on the basis of their size, setup, and the technological upgrades they received for dual-mode learning. The sample includes two small seminar rooms (see figure 1), two medium lecture halls, two large auditoriums, two classrooms originally set up for flexible learning (typically called "active learning classrooms"; see figure 2), and two spaces that were transformed into classrooms to accommodate smaller, socially distanced classes during the pandemic (see figure 3). Of these two spaces that were converted into classrooms, nontraditional learning space 1 is part of the library typically set up with booths, chairs, and tables for student studying, and nontraditional learning space 2 is a ballroom in the student center.

Of the ten rooms discussed in this article, eight were used as classrooms prior to the pandemic. During the pandemic, they saw a reduction in capacity of 28–67%. The average seating density in this sample had previously been 18 square feet per person, and this changed to 40 square feet per person during the pandemic (see table 1).

Table 1. Reductions in Seating Capacity

Room	Seating Capacity: Pre-Pandemic	Seating Capacity: Pandemic
Small Seminar 1	49	19
Small Seminar 2	30	14
Medium Lecture 1	93	38
Medium Lecture 2	132	47
Large Auditorium 1	276	90
Large Auditorium 2	464	160
Active Learning 1	30	18
Active Learning 2	32	23
Nontraditional 1	n/a	100
Nontraditional 2	n/a	150

For the purposes of video, all of the learning spaces in this research featured these technologies:

• Rear-mounted pan–tilt–zoom (PTZ) camera
• Webcam on the computer monitor
• Dual instructor monitors

For audio, the learning spaces employed different kinds of microphones:

• Small seminar rooms, medium lecture halls, and the active learning spaces had ceiling array participant mics, with the exception of one of the medium lecture halls, which had a faculty mic only.
• The large auditoriums and the nontraditional spaces used lavalier mics for instructors.

Research Methods

The Learning Space Rating System (LSRS) provides a set of measurable criteria to assess how well the design of classrooms supports and enables multiple modalities of learning and teaching, especially that of active learning. The LSRS provides a scoring system to serve as an indicator of how well a classroom's design serves these goals. We performed a walkthrough in each sample room to give an LSRS score for the rooms during the COVID-19 pandemic and a retroactive score for the pre-COVID-19 technology and layout. (The nontraditional learning spaces were only given pandemic scores because they were not previously set up for classroom use.)

This study uses Part B of Version 2 of the LSRS (see the sidebar "LSRS: Version 2 versus Version 3"). Part B measures learning spaces across various criteria in three content areas: Environmental Quality, Layout and Furnishings, and Technology and Tools (see table 2). Each criterion adds 1–3 points to the classroom's total score.

Table 2. Categories and Criteria of LSRS Scoring

Environmental Quality	Layout and Furnishings	Technology and Tools
Daylight Views to Outdoors Interior Visibility Lighting Control Thermal Comfort Acoustic Quality Environmental and Cultural Inclusiveness Accessibility and Universal Design	Proximities within Space Movement Through Space Seating Density Furniture Configuration Flexibility Work Surfaces Seating Comfort Movable Partitions Transparency Access to Adjacent Informal Learning Areas Writable Surfaces Physical Storage Future Proofing	Electrical Power Network Connectivity Visual Displays Sound Amplification Audio/Visual Interface and Control Distributed Interactivity Session Capture and Access

This research project uses the LSRS to address two questions. The first asks how LSRS scores changed in each room after pandemic upgrades were made. For the pandemic-configured classroom, scores were given pragmatically for what could be used with social distancing and health measures in place. For example, even if a room had moveable chairs, it was not given a point for Furniture Configuration Flexibility because the seats could not be moved while maintaining social distancing. Pre-pandemic and during-pandemic scores were compared only for in-person learning—in which no students were attending on Zoom—even if other forms of learning also took place in those spaces. Before the pandemic, classrooms were equipped with a webcam and microphone on the monitor or house microphones; these were not judged to fit the criteria for session capture and access because they were not used to capture instruction. The limited equipment did not provide effective access to captured lessons if the instructor moved from the monitor or used any visual aids. Without upgrades, physical displays were not visible from these cameras, and remote sharing technologies were not standard practice before the pandemic.

The second research question compares the scores of each room based on the mode of learning (in-person, dual-mode, or online) taking place in those spaces during the pandemic. These scores used a subset of seven of the LSRS criteria that we selected because they can translate across the three modalities. Those seven LSRS criteria were assessed for how well dual-mode and purely remote environments fulfilled them compared with in-person instruction during the pandemic. Scores were determined in part by survey results from students. For rooms where surveys were not distributed, scores were given based on scores for rooms with similar size and technological capacities.

In the comparison of in-person, dual-mode, and remote scores, the criterion for Writable Surfaces was considered in both the physical and virtual environment. Physical writable surfaces were no longer available to students in the in-person classrooms to limit possible transmission of COVID-19 from high-touch surfaces. Online classes were considered to have writable surfaces available as a function of Zoom as a classroom space, which was the standard format of online classes for the fall term. Digital writing surfaces such as Google Docs that could have been used by both in-person and online students during dual-mode learning were not seen as fulfilling the criterion because this was not standard practice and because survey results revealed that they were rarely used.

Surveys were sent to students and professors in five of the sample rooms. Seminar 2, both of the flexible learning classrooms, and both of the nontraditional learning spaces were added to the sample after the semester concluded, so surveys were not administered for these rooms. A total of 1,215 students (of 3,590 invited) and 29 out of 82 faculty responded to the surveys. Students were asked to determine how well the rooms fulfilled the following criteria: visual displays, acoustic quality, seating comfort, sound amplification, and distributed interactivity for both the remote and in-person experience. Students were asked directly to rate how well their classroom fulfilled the LSRS description of the above criteria points on a 5-point scale, from strongly disagree (1) to strongly agree (5). Faculty were asked how well the classrooms fulfilled the LSRS criteria from their perspective, including how well they could see, hear, and interact with their in-person and remote students and how the upgraded technology assisted in interacting with remote students. Classrooms given an average score of at least 2.5 were considered to meet the criteria and gained the point(s).

Results

Figure 4 provides a visualization of the data from the first research question—the overall impact of the pandemic on the LSRS scores. Changes in the classrooms can be divided into three categories. Most of the traditional lecture rooms (seminar 1, lecture halls, and auditoriums) experienced a small net gain (2–3 points) from the COVID-19 changes. The flexible classrooms, which were previously designed to enhance active learning, experienced a moderate net loss (4–5 points). Seminar 2 represented a middle ground between the two, with no change in net score. Nontraditional learning spaces scored equal to the flexible learning classrooms after they were modified. The nontraditional learning spaces scored higher than any of the non-active-learning classrooms did, either during or before the pandemic.

With the changes made for the pandemic, all traditional rooms gained points for Session Capture and Access. Any that had points for Furniture Configuration Flexibility lost that point. Seminar 2 was the only traditional classroom that also lost points in other areas, losing points for Proximities within Space and Movement through Space, points that the other classrooms did not originally have. The flexible learning classrooms also gained points for Session Capture and Access, with active-learning room 2 gaining an additional point for Seating Density. Both active-learning classrooms lost points for all the same criteria—Proximities within Space, Movement through Space, Furniture Configuration Flexibility, Work Surfaces, and Writable Surfaces.

From our second research question, figure 5 displays the overall scores for the LSRS criteria applied across the in-person, dual-mode, and synchronous online learning environments during the COVID-19 pandemic. All classrooms scored highest for active learning potential in online learning situations, with all classrooms gaining the full eight points for all seven criteria. For dual-mode learning, the seminars and flexible learning rooms had the best scores, only 1 point lower than synchronous online and equal to in-person classes. The lecture, auditorium, and nontraditional rooms scored just 5 points for dual-mode learning, lower than in-person or online learning.

The dual-mode environment for all rooms except the seminars and flexible learning classrooms lost points for Interior Visibility and Distributed Interactivity because surveys revealed that students could not see or hear their peers in the opposite mode in these rooms. Nontraditional learning space 1 also lost the points for Interior Visibility and Distributed Interactivity for in-person learning because pylons divide the room, preventing students from seeing or interacting with each other, especially when students could not move around the room.

Discussion and Conclusion

Evaluating multiple types of classrooms demonstrates that the COVID-19 pandemic affected a wide variety of elements in the classroom. This is best demonstrated by the flexible learning classrooms, which had the highest LSRS scores to begin with and were the only classrooms with a net loss. These classrooms were affected in the most diverse ways, experiencing changes in six and seven different criteria.

Comparing scores across the sample rooms demonstrates the importance of space and flexibility in movement in the classroom. The largest increases in scores came from decreased seating density, allowing more space per person in almost every room. The largest decreases came from lowered scores for flexible configuration and movement and proximity in the rooms because social distancing prevented movement and physical interaction.

For the traditional rooms, the overall small change in scores indicates that the pandemic did not have a large impact on LSRS scores when all students are in-person. However, given the changes in classroom behavior that the LSRS Version 2 could not have foreseen, the scores may not reflect the full scope of classroom activity. The change in seating density, in particular, may reveal that the criteria used to assess a classroom's capacity for active learning before the pandemic do not fully capture the COVID-19 classroom. A large benefit from having more than 30 square feet per person in a classroom is that there is enough room for students to move around and interact with new people. During the pandemic, this is not possible with fixed social-distance seating. Preserving decreased seating density after the pandemic may increase interaction in a more powerful way when students and the instructor can safely move around the classroom space, except in fixed-seating and auditorium-style classrooms. Flexible furniture and movement make a large difference in the potential of a classroom to perform well in multiple pedagogical strategies, including active learning. Traditional lecture rooms were shielded from a loss in their LSRS scores because they had some form of fixed seating, be it rowed tables or auditorium-style fixed chairs.

All sample rooms received an additional point for Session Capture and Access that was added as a COVID-19 accommodation, which enabled students who otherwise might not have been able to go to class for up to two weeks at a time to attend and participate remotely. Using Zoom for live remote attendance, students can actively participate in class by speaking or typing their comments and asking questions during class. This made an immense difference in their education during the fall semester and was a large part of the reason the university could return to campus. Two-thirds (67%) of students across all classes appreciated having the ability to watch class live remotely or later asynchronously, and nearly as many (64%) responded that session capture upgrades would be useful after the pandemic. One risk of the enhanced lecture capture capabilities would be the technology pushing instructors to emphasize more traditional lecturing as opposed to interactive activities and discussion.

Comparing the regular classrooms to the nontraditional learning spaces demonstrates areas where the traditional rooms can be improved in their capacity for active learning. The nontraditional learning spaces received equivalent scores to the flexible learning classrooms by fulfilling criteria such as having a view outdoors, improved lighting control, movable partitions, transparency, future proofing, electrical power, and multiple visual displays—elements that some or all of the traditional classrooms did not have.

During the pandemic, the online learning environment compensated for some elements of the in-person environment that were lost and that dual-mode technology was not able to entirely remedy. This was reflected in the higher scores of the online environment compared to dual mode, especially for the criteria of Interior Visibility and Distributed Interactivity. The lower score received by dual mode in most classrooms revealed where technology must be improved for effective dual-mode learning, especially to increase visibility and interactivity among students of both online and in-person modes. The online environment allows all students to see each other, the professor, and any materials being presented. Breakout rooms and group assignments where students do not need to be in their assigned seats allow students to interact with more of their peers and change groups throughout the semester. Students also have better access to a writable surface, which is not available in the physical room during the pandemic and is not part of the standard practice of dual-mode learning. These benefits were echoed by some professors who stated during interviews that they moved some class periods fully online in order to facilitate active learning. Being in the physical classroom is still valuable, and the majority of students and professors in surveys and interviews appreciated the ability to be in the classroom during the pandemic. However, the best practice in future semesters of dual-mode learning may be to switch some class sessions online if dual-mode technology in the classroom cannot effectively allow for student and professor interaction.

Smaller classrooms, including the seminars and flexible learning rooms, performed better in dual mode than medium or large classrooms, even when the small classrooms were equipped with the same technology. Seminar 1 had the same technology as Lecture 2. The difference in scores in rooms with the same technology indicates that technology must be tailored to fit the room and that professors must use technology in ways that best improve learning. This could include displaying remote students on a screen and including a camera angle that allows remote students to see their in-person peers. Participant microphones were helpful for contributing to interactivity, but visual interaction was lacking. In order to improve the rooms to encourage active learning during future semesters of dual mode, technological upgrades and improved use of technology by professors should be geared toward improving interactivity between students of differing modes. Notre Dame's Office of Information Technology has already committed to making dual-mode upgrades part of standard setups beyond the pandemic.

For the immediate future, the LSRS shows a few relevant ways to improve while classrooms are still configured for learning during a pandemic:

• Implement multiple multi-input displays in the classroom in order to display remote students on one screen and visual aids (e.g., slide decks) on another. When possible, encourage remote students to keep cameras on for increased student–student and faculty–student engagement.
• Add camera capabilities to see in-person students. Place cameras in the front of the room, facing the student seating area, to help remote students better see their in-person peers.
• Improve in-person student mics for remote students to better hear their in-person peers, increasing remote students' ability to engage in class discussion and active learning.
• Add electrical towers throughout classrooms to improve electrical connectivity for students.
• Add culturally inclusive design features and emphasize multiple pedagogical strategies fit for learners with different styles.

Implementing these improvements may improve the dual-mode environment and would also add points to the LSRS scores for visual displays, distributed interactivity, electrical connectivity, and inclusivity.

Further investigation into the effects of the COVID-19 pandemic on a classroom's capacity for active learning can expand this research beyond the LSRS scores. Further research is currently focusing on surveys, interviews, and observations of classes during the pandemic to reveal aspects of dual-mode learning beyond the LSRS and measure how the learning space, teaching, and active learning continue to improve after the COVID-19 pandemic.

Acknowledgments

We want to express our deep gratitude to Linda Martellaro, Brian Burchett, and Kevin Barry for all their leadership through design consultations and assistance with assessment methods, analysis, and reporting to constantly improve our campus's learning spaces, and to Jason Railton and his team for their heroic and timely action to enhance classrooms in less than three months, which allowed the university to open in August for the fall semester after the onset of the pandemic.

---

Jess Staggs is a research assistant in the ND Learning Research Lab at the University of Notre Dame.

G. Alex Ambrose is the Director of Learning Research in ND Learning, Kaneb Center for Teaching Excellence, at the University of Notre Dame.