Pedagogy and Space: Empirical Research on New Learning Environments

In a previous EDUCAUSE Quarterly article,¹ we reported the results of quasi-experimental research on the University of Minnesota's new, technology-enhanced learning spaces called Active Learning Classrooms (ALCs). That investigation found — after controlling for potentially confounding factors such as instructor, instructional methods, assessments, and student demographics — that teaching in an ALC contributed significantly to student learning outcomes. In addition, our findings indicated that the type of space in which a class is taught influences instructor and student behavior in ways that likely moderate the effects of space on learning. Finally, we found significant cross-sectional differences between different subsets of our student sample in terms of how they perceived the ALC's contribution to their learning experience.

Here, we report on the next phase of learning-spaces research at the University of Minnesota (UMN), which had two components. First, to ensure that our earlier results were not simply fortuitous, we replicated the original investigation with a different instructor, student sample, and subject matter. Second, having shown that the type of learning space matters, we turned our attention to the pedagogy employed within the room. Using another quasi-experimental design, we investigated whether or not having our instructor adapt her instructional approach to fit the space would influence student learning outcomes and student perceptions of their learning experience.

Two specific research questions guided this phase of our research:

• Holding the pedagogical approach constant, what is the relationship between the type of learning space and (1) student learning outcomes, (2) instructor and student behavior, and (3) student perceptions of the learning experience?
• Holding the learning space constant, what is the relationship between the type of pedagogical approach and (1) student learning outcomes and (2) student perceptions of the learning experience?

Methodology and Methods

In the year since our last EDUCAUSE Quarterly article, interest in new learning spaces at colleges and universities across the country has continued to rise. In August 2011, UMN hosted ALC Forum, a national conference devoted to improving education through new learning spaces. About the same time, editors for the Journal of Learning Spaces, a new scholarly periodical, issued their inaugural call for submissions. And numerous higher education institutions are designing, building, using, and researching newly configured and technologically equipped learning environments — from Stanford University's Wallenberg Hall to the University of Iowa's TILE classrooms to Texas Wesleyan University's Classroom.NEXT.

Learning Environments Examined

UMN's ALCs feature large circular tables that seat nine students and can accommodate laptop computers for collaborative work. Typically, students bring their own computers to class and use the building's ubiquitous wireless capability. The classrooms are designed to facilitate hands-on activities and problems that require students to interact with each other to reach a solution. Students can display their work on large LCD screens mounted around the room to promote small- and large-group discussion. Each table is allotted three gooseneck microphones so that students can make themselves heard without shouting during group discussions. Additionally, students can press a button on their table console to light a signal lamp to indicate when they need help or wish to speak. To facilitate brainstorming and group reporting, the room is also lined with erasable marker-boards.

At UMN, teaching in the newly completed Science Teaching and Student Services (STSS) building began in the fall of 2010 (see Figure 1; a video of the STSS building's two-year construction is also available).

Figure 1. The STSS Building at the University of Minnesota

Prior to STSS, Minnesota had only two ALCs, where we conducted our original research. The STSS building houses 10 additional ALCs of various sizes, making it one of the largest collections of such spaces in the country. It also contains a host of informal learning spaces and more traditional, amphitheater-style classrooms. For this article, we replicated our original learning spaces experiment using only classrooms in the new STSS building (see Figures 2 and 3; more classroom images are available through the UMN's Office of Classroom Management).

Figure 2. A Large ALC (up to 126 Students) in the STSS Building

Figure 3. A Schematic Diagram of a Large ALC

The second component of our research — the investigation of pedagogy within new learning spaces — was conducted in UMN's first ALC, housed elsewhere on campus. Figure 4 shows that room, which has a capacity of 117 students.

Figure 4. The Original, Large-Enrollment ALC

As in our earlier research, we used a Learning Environments Research Partnership Model that involved a team consisting of undergraduate students, the faculty member teaching the course, and a research expert. The undergraduate researchers underwent training on the protection of human subjects and research methodology. They also worked with the other team members to establish the project's research design, create the research instruments, and collect and analyze data.

Data Collection Methods

We collected data using a variety of methods, including surveys, interviews, focus groups, achievement tests, and direct observations. We also made an explicit effort to triangulate the outcomes of interest using both quantitative and qualitative methods and analytical techniques.

Results and Key Findings: The Importance of Space in BIOL 1003

In spring 2011, we had the opportunity to replicate our original study. That study focused on a Post-Secondary Teaching and Learning (PSTL) 1131 science class that was moderately sized (N = 86) and contained a somewhat specialized sample of students. In contrast, Biology 1003 was a large introductory class (N = 263), which, because it satisfied an important curricular requirement for undergraduate students, drew a broadly representative sample of students from across the university.

Our faculty partner, Dr. Sehoya Cotner, taught two sections of BIOL 1003 — one in a traditional room, which featured standard row seating with a traditional podium at the front, and another in an ALC. Both course sections were taught in back-to-back sessions on Tuesday and Thursday mornings at 9:45 and 11:15, respectively. As in our earlier study, we were unable to assign students to the classrooms randomly, but assigning students to a lecture section based on their laboratory enrollment might mitigate any potential selection bias related to classroom-type preference. Regardless, we were able to control for a variety of other potentially confounding factors, such as day of the week, course materials, assignments, schedules, exams, and pedagogical approach. Furthermore, we were able to establish post-hoc demographic equivalencies between both course sections using institutional data. Because both sections were taught in the new STSS building, our design also controlled for any novelty factor that might have confounded our results. Taken together, these controls meant that the only factor allowed to systematically vary across sections was the type of classroom in which the course was held, which let us isolate the effects of the type of learning space in which the sections were taught.

Students in the ALC Outperformed Expectations

Given the lack of randomization in our study design, we examined a wide range of demographic variables to ensure equivalent student populations in both classrooms. The only variable for which a statistically significant difference between sections emerged was composite ACT score. The traditional classroom's students had an average ACT score of 26.36, while ALC students had an average ACT score of 25.32 (t = 2.09; p < .05).

Composite ACT scores are reliable and valid predictors of course grades,² particularly for first-year and sophomore students — the majority of those enrolled in BIOL 1003 — so it was reasonable to expect that students in the course's ALC section would underperform in terms of average final course grades relative to their peers in the traditional room. As in our previous study, however, we found that the ALC students outperformed expectations and ended the semester with an average final grade that was statistically equivalent to the average grade in the traditional section.³ Specifically, based on their average composite ACT score, the expected final grade for the traditional room's students was 78.52 percent, while the ALC room's students had an expected grade of 71.77 percent. But the ALC students did substantially better than expected, earning an average of 76.49 percent, a score that is statistically equivalent to the traditional room's average of 78.45 percent (see Figure 5).

* p < .0001
Figure 5. Expected Versus Actual Grades (BIOL 1003)

In both BIOL 1003 and in our earlier study of PSTL 1131, therefore, we found that even though the ALC  students' ACT scores would suggest that they would score lower than the traditional classroom students (who had higher ACT scores), their levels of achievement were statistically identical. Given our study's controls, these replicated results strongly indicate that teaching in an ALC had a positive effect on student learning outcomes as measured by course grades.⁴

Space Impacts Instructor and Student Behavior

One of the hypotheses guiding our learning spaces research has been that a new learning space's physical layout affects how people behave in the room. We believed that the arrangement of ALCs — which have no clear center or point of focus like traditional amphitheaters, and in which students sit at round tables and thus face their classmates rather than the instructor — would have ramifications for instructor and student behavior; indeed, this is what we found in our earlier study of PSTL 1131.

For our BIOL 1003 study, we used the classroom observation protocol that we developed and tested for the earlier study. This protocol let us systematically record and quantify classroom activities at five-minute intervals throughout each class period. Dr. Cotner explicitly tried to keep her pedagogical approach constant across the two sections of BIOL 1003, employing the same materials, syllabus, and learning activities.

Our findings were consistent with our previous study's results. Analysis of the data indicated that, despite Dr. Cotner's attempts to create identical instructional environments in both class sections, she behaved differently in the two rooms (see Figure 6). In the traditional classroom, she spent significantly more time at the podium, while in the ALC she spent significantly more time away from it, possibly reflecting the greater ease of instructor movement through the ALC. This increased mobility might also account for the fact that, in the ALC, Dr. Cotner consulted significantly more often with individuals and small groups of students. Finally, the ALC's collaborative layout might be responsible for the finding that ALC students spent significantly more time engaged in group activities than students in the traditional room.

* p < .05; ** p < .01; *** p < .001
Figure 6. Time Spent on Classroom Activity/Instructor Behavior in BIOL 1003

Student Perceptions of the ALCs

The final variables we examined in this study relate to how students perceived their own learning experiences in the different types of classroom. In particular, we constructed five theoretical variables from survey items that consistently have been shown to be affected by formal learning spaces: student engagement; enrichment of the learning experience; flexibility of the room; fit between learning space and course; and good use of the space by the instructor. The validity of each of these theoretical variables has been confirmed by principal component factor analysis. The scale reliability of each construct has also been confirmed with Cronbach's alphas of .85 or greater.

As in our previous study, the ALC students gave significantly higher marks to their classroom in terms of engagement, flexibility, and fit than students in the traditional room (see Figure 7). The mean score on the enrichment variable was somewhat higher for the ALC than the traditional classroom, but the difference was not statistically significant. Finally, students in both classrooms gave very high and near-identical ratings to Dr. Cotner's use of the classrooms, which might reflect the fact that she is a very experienced, comfortable, and popular instructor who is capable of using any learning space effectively.

* p < .01; ** p < .001; *** p < .0001 >>
Figure 7. Student Perceptions of the BIOL 1003 Classroom

Broadly speaking, the BIOL 1003 results in terms of how students perceive the learning experience confirm what we found in our investigation of PSTL 1131. In both courses, compared to students in traditional classrooms, the students in the ALCs said that they were more engaged, that the room was more flexible, and that the room and the course were a better fit. It is important to note that while our PSTL 1131 results might be explained by a novelty effect — in that case, the ALC was a very new and impressive classroom, while the traditional room was older and arguably not as exciting — the BIOL 1003 findings cannot be similarly explained, because both sections were taught in brand-new spaces in the same new building.

In our earlier research, we found that students from metropolitan Minnesota counties generally reacted more positively to the ALCs than students from rural counties, and that first-year and sophomore students responded more positively than juniors and seniors. In spring 2011, these cross-sectional differences had disappeared, possibly because the proportion of UMN students with experience taking classes in ALCs had sharply increased.

Qualitative Data: Students and Teachers

Narrative data gathered through interviews and open-ended survey questions reinforced our quantitative analysis, illustrating the ways in which both students and instructor perceived distinct differences between the traditional classrooms and the ALC.

Many students liked the cleanliness and audiovisual projection capabilities of the newly built traditional-style classrooms, stating that the room "was perfect for watching videos," easy for PowerPoint, and that "the seats are really comfortable so when we discuss, it was more relaxing and less intense." Other students faulted the configuration of the amphitheater, noting the limitations of the seating and that "it was sometimes difficult to have conversations when [our groups of five or six students] are all in a row." Others complained that the room's immensity and density made it difficult for the instructor to move among the students and for the students to feel encouraged to speak up: "Because we are a huge glob of students, not only is it hard for the professor to see you but it is hard to muster the courage to talk."

Those students in the ALCs, on the other hand, felt their room encouraged interaction and inclusion: "I speak a lot more in this class than in any other plain lecture hall." Another commented: "No one was left out during group work. I met new people! Exchanged good ideas!" Rather than the "relaxed" atmosphere of the traditional classroom, one student recognized the double-edged nature of the audio-enabled tables: "It helps us pay attention because we never knew when our table would be called on [and would have to talk into the microphone]." Indeed, students noted that the ACL table groupings were initially uncomfortable and that they had to overcome their early reticence: "At first it was a bit awkward, but now we all are tight!"

Dr. Cotner also saw a distinct difference between the rooms. She felt the traditional room worked well for lecture, but the ALC pointedly did not: "In the ALC, I feel like I almost have to be speaking in iambic pentameter, rhyming, or setting my hair on fire or something, so that they will actually be there, listening. [Lecturing in the room] is hard, it's really hard because your back is to some students. I wouldn't want to listen to somebody I couldn't see." However, the ALCs, in her opinion, produced richer discussions and more sophisticated questions. She conjectured that it might be because "they could really quickly establish what they knew already, right?" She said that students would ask each other the easy questions, allowing their line of questioning to quickly evolve. Thus, simpler concepts — like the difference between mitosis and meiosis — were quickly hashed out, with precedence going to unpredictable questions that might have "societal or legal relevance" and ideas that connected to the students' lives. From the instructor's viewpoint, this can be disconcerting when you are accustomed to anticipating most of the questions. Her advice? "Get ready to not know everything."

Results and Key Findings: The Importance of Pedagogy in FSoS 3101

Our studies of PSTL 1131 and BIOL 1003 indicate strongly that — while holding constant numerous, potentially confounding variables — teaching a class in an ALC has significant positive effects on student learning outcomes and student perceptions of the learning experience, and results in significant changes in faculty and student behavior.

But what about pedagogy? Does it matter whether instructors adapt their instructional approaches to the room they are teaching in? New learning spaces like the ALCs are designed to facilitate a more active, student-centered pedagogy, so what happens if an instructor explicitly redesigns a course to take advantage of the room's potential?

We utilized a longitudinal, quasi-experimental design to investigate these questions in a class on personal and family finance (FSoS 3101) taught by Catherine Solheim. Dr. Solheim taught the course twice, in fall 2008 and fall 2009, both times in the original ALC. The first iteration of the class was taught using the instructor's usual approach, which was largely expository and lecture-based. From fall 2008 through fall 2009, Dr. Solheim was enrolled in an 18-month faculty development program run by the Office of Information Technology that focused on transforming courses through the strategic and effective application of technology to teaching. In the second iteration of FSoS 3101, Dr. Solheim utilized a more team-based, student-centered approach to teaching. To accomplish this, she incorporated several simple classroom assessment techniques (CATs), including think-pair-share and scratch-off instant feedback assessment techniques (IF-ATs), to encourage cooperative learning and rapid feedback, as well as in-class collaborative problem-solving assignments.

Figure 8 shows a schematic view of our three quasi-experimental studies of learning spaces. As the colored bars indicate, in each study we have held constant one of the two main independent variables — pedagogical approach and type of room — while allowing the other to vary between the two course sections involved in each investigation. This enables us to isolate the effects of each variable on student reactions, instructor behavior, and student learning. As the diagram shows, in PSTL 1131 and Biol 1003, pedagogical approach was held constant while type of room varied; in FSoS 3101, type of room was held constant while pedagogical approach varied.

Figure 8. Learning Spaces Research: Quasi-Experimental Designs

Despite our inability to randomly assign students to the control and experimental versions of the FSoS 3101 course, we were able to establish statistical equivalency across both sections on a host of demographic, evaluative, and theoretical variables. The students enrolled in the fall 2008 and fall 2009 course versions were statistically identical in terms of their sex, age, year in school, ethnicity, geo-cultural origins (such as rural or metropolitan), and ACT scores. Also, both course sections were taught in the same ALC room, and students offered statistically identical evaluations of the room's features (lighting, chairs, tables, views of the instructor and screens, room layout, and so on).

Finally, students' perceptions of the ALC's impact on our five theoretical variables — engagement, enrichment, flexibility, use, and fit — were the same in both versions of the course. In fact, the only survey item that showed any significant difference between sections was related to participation: students in the fall 2009 course reported significantly higher levels of agreement with the statement "The classroom in which I am taking this course encourages my active participation" than students in the fall 2008 course (t = 2.03; p < .05).

Given that students in both sections were essentially the same on every meaningful category, including ACT composite scores, we would expect their grades in the same course, taught by the same instructor, in the same space, to be the same. However, as Figure 9 shows, students in the fall 2009 section clearly and significantly exceeded those expectations by 5.54 percentage points — the difference between a final grade of B- and B (t = 5.80; p < .0001).

* p < .0001
Figure 9. Expected Versus Actual Grades (FSoS 3101)

Although the aggregated student gains resulting from the shift from a lecture-based pedagogy to an active learning approach are impressive in their own right, our results also demonstrate that students of all ability ranges benefitted significantly from the changes (see Figure 10). Broken out into quartiles by composite ACT scores, we observe that student learning as measured by grades improved significantly across the board. The lowest (first) quartile gained the most (6.86 percentage points, t = 5.17; p < .0001), while the highest (fourth) quartile students gained the least (1.23 percentage points, t = 2.20; p < .05). Because educational technology interventions frequently benefit only one segment of a student population, the finding that aligning pedagogical practice with the course's physical space benefits all students, regardless of ability, is an important and noteworthy one.

* p < .05, ** p < .001, *** p < .0001
Figure 10. Aggregated Student Grades (%)

Finally, we analyzed grade data related to each assignment. Because the assignments for the fall 2009 course were altered somewhat to accommodate the instructor's new approach to teaching the course, we had to rule out the possibility of any one assignment being responsible for the observed differences in student grades. If the grades for all of the compared assignments were significantly different from one another, then the changes clearly would be attributable to the pedagogy. If, however, grades for only one or two assignments were significantly different from one another, the overall gains might have resulted from changes to those particular assignments, not from the alignment of pedagogy and space. After matching comparable assignments, weighting assignment grades based on their respective contributions to the overall grade, and making adjustments to compensate for the elimination of a midterm exam between the course's first and second iterations, our analysis of the assignment grades revealed support for the argument that it is the pedagogy, not any particular assignment, that is responsible for the observed changes in overall grade. That is, students in the experimental section performed significantly better — between 5.36 and 14.60 percentage points better — on each comparable assignment than their peers in the control section (see Figure 11).

* p < .001; ** p < .0001
Figure 11. Grades on Comparable Student Assignments (%)

The improvement in student participation grades is noteworthy given the difference in student perceptions noted earlier. Students in the active learning section of FSoS 3101 believed that the room encouraged their active participation to a greater degree than students in the control section, and their grades validate that belief.

Qualitative data

Once again, faculty interview data and student commentary in both the 2008 and 2009 courses were generally consistent and largely confirmed our findings. Students in the revised course more frequently mentioned positive classroom attributes associated with working in teams. For instance, one student commented, "When we take group quizzes it allows for discussion and everyone can participate and see what we're doing." Others said the availability and proximity of computers "made team projects faster and more efficient." Similarly, their complaints tended to critique pedagogical methods that countered the room design: "When the professor was lecturing and going through PowerPoint slides, the layout of the room made it harder to pay attention and engage in the material." One student said frankly, "I think this classroom makes lectures difficult but [it] is great for group work." The instructor agreed with the students, and in her first exposure to the ALC in 2008, she lectured often and delivered PowerPoint presentations. As she noted, "Even though you know that it's not the best pedagogy to use, you just kind of fall into that." In 2009, of course, she redesigned the class to take fuller advantage of groups and teams.

Understandably, both classes found some fault with the space. Some considered the technology "burdensome" and the time spent sorting out technical glitches to be "a waste of time." Others found easy distraction in their neighbors; the same proximity that encouraged engagement also created a climate for social diversion. Even the value of a closely knit table group was not sacrosanct, and it was sometimes contrasted with the loss of the wider classroom community. As one student noted, "I do not like [the room] because I feel it is 12 mini classes in one because of [the] multiple tables." As another student somewhat sadly framed it, "[I] didn't meet some of the girl students across the room; I would have liked to." Finally, another student reminded us, "It's still a big class, so I didn't get to know that many people..." and then quickly added, "but I loved the tables."

Implications and Recommendations

• Where local resources allow, administrators might consider refitting older learning spaces along the lines of the ALCs to improve student learning outcomes and increase student satisfaction.
• Instructors teaching in an ALC-style classroom should be aware that the space's de-centered nature can make traditional, expository teaching techniques difficult. They might wish to incorporate more active learning techniques and approaches into their repertoires.
• The beneficial impact of aligning pedagogy with the learning space indicates that faculty development programs designed to support the redesign of courses being taught in ALC-type spaces deserve continued or increased institutional support.

Further research is needed into the connections among learning spaces, pedagogical approaches, and student learning outcomes. In particular, future research should investigate the mechanisms underlying the positive effects of space and aligned pedagogy that our studies have found. Furthermore, as ALC-type learning environments become more common, it will be possible to conduct longitudinal studies of the impact of taking multiple classes in new learning spaces on broad strategic outcomes such as student retention, persistence, and failure rates.

Acknowledgments

We thank our University of Minnesota partners in this ongoing project, including the Center for Teaching and Learning, the Office of Classroom Management, the Office of Institutional Research, the Office of Measurement Services, the College of Biological Sciences, and the College of Education and Human Development. We also thank our faculty partners, Jay Hatch, Sehoya Cotner, and Catherine Solheim. Finally, we thank the administration within the Office of Information Technology — particularly Linda Jorn and Bradley A. Cohen — for their support and encouragement of our research.

1. Aimee L. Whiteside, D. Christopher Brooks, and J. D. Walker, "Making the Case for Space: Three Years of Empirical Research on Learning Environments," EDUCAUSE Quarterly, Vol. 33, No. 3 (July–September 2010).
2. ACT, Prediction Research Summary Tables, Iowa City, Iowa, 1998; ACT, The ACT Technical Manual, Iowa City, Iowa, 2007; Crystale M. Marsh, Michael A. Vandehey, and George M. Diekhoff, "A Comparison of an Introductory Course to SAT/ACT Scores in Predicting Student Performance," The Journal of General Education, Vol. 57, No. 4, 2008, pp. 244–255; Heinrich Stumpf and Julian C. Stanley, "Group Data on High School Grade Point Averages and Scores on Academic Aptitude Tests as Predictors of Institutional Graduation Rates," Educational and Psychological Measurement, Vol. 62, No. 6 (2002), pp. 1042–1052; Paula Wilhite, Bob Windham, and Robert Munday, "Predictive Effects of High School Calculus and other Variables on Achievement in a First-Semester College Calculus Course," College Student Journal, Vol. 32, No. 4 (1998), pp. 610–617; Robert L. Ziomek and Kevin M. Andrews, Predicting the College Grade Point Averages of Special-Tested Students from Their ACT Assessment Scores and High School Grades, ACT Research Report Series (Iowa City, IA: ACT, 1996), pp. 96–97.
3. D. Christopher Brooks, "Space Matters: The Impact of Formal Learning Environments on Student Learning," British Journal of Educational Technology, Vol. 42, No. 5 (2011), pp. 719–726.
4. Ibid.

© 2011 J. D. Walker, D. Christopher Brooks, and Paul Baepler. The text of this EQ article is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 license.