Making the Case for Space: Three Years of Empirical Research on Learning Environments

Key Takeaways

• Students attending classes in the University of Minnesota's new, technology-enhanced learning spaces exceeded final grade expectations relative to their ACT scores, suggesting strongly that features of the spaces contributed to their learning.
• First-year and sophomore students as well as students from metropolitan areas rated the new learning spaces significantly higher than their upper-division and rural counterparts in terms of engagement, enrichment, effectiveness, flexibility, fit, and instructor use.
• Different learning environments affect teaching-learning activities even when instructors attempt to hold these activities constant.
• Although assignment types greatly affect the study environments students select, in choosing informal study spaces students fall into routines early and are reluctant to deviate from them even if they are not meeting their study goals.

An article in last year's EDUCAUSE Quarterly special issue on learning spaces reported on the results of the University of Minnesota's pilot evaluation of its high-tech, state-of-the art Active Learning Classrooms (ALCs).¹ The pilot study addressed the need for evidence-based research on learning environments, focusing on student and faculty reactions to the ALCs based on the PAIR-up model, which values partnerships, pedagogy, assessment, innovation, and continually revisiting "current views on emerging technologies, diverse learners, strategic campus issues, new course design methods, and recent findings in learning science."² The data from the university's pilot study suggested that both instructors and students responded to these learning environments in positive ways:

• Instructors adapted their teaching techniques to the new learning spaces and frequently found themselves in the role of learning coach or facilitator.
• Students found the classrooms effective at promoting teamwork and collaboration.
• Both students and instructors reacted well to the technological and physical features of the ALCs, with the rooms' round tables singled out for special praise.³

The next phase of research on learning environments at the University of Minnesota involved a unique partnership with undergraduate researchers, faculty members with considerable classroom experience, and several collegiate and unit associates. In this new project, funded by the Bush Foundation (established in 1953 by Archibald G. and Edyth B. Bush), a team of university researchers conducted a more systematic investigation of the impact of formal and informal learning environments. We explored the effects of different types of formal learning environments on student learning outcomes and teaching and learning strategies, and we explored where, when, and how students completed their course assignments when they used informal study spaces.

More specifically, the research question and sub-questions that guided this project were:

• To what extent, if any, do formal and informal learning environments shape teaching and learning?
  • What is the relationship, if any, between formal learning environments and student learning outcomes?
  • What is the relationship, if any, between the type of learning environment and the teaching/learning activities employed by the instructor?
  • In what sorts of informal learning environments do students complete their course assignments? When, where, and how do students complete their course assignments?

Methodology and Methods

To address the research questions as thoroughly as possible, we employed a Learning Environments Research Partnership Model that involved undergraduate students, the faculty member teaching the course, and a research expert in the Office of Information Technology. The undergraduate student researchers were equal partners, a choice made because of recent successes from including undergraduate students at other educational institutions and in the Carnegie Foundation's Advancement of Teaching and Learning's Student Voice in the Scholarship of Teaching and Learning program.⁴ The undergraduate researchers went through training on the protection of human subjects and on research methodology. They worked with other team members to establish the research design for this project, to create the research instruments, and to collect and analyze data.

Data Collection Methods

Data were collected using a wide variety of traditional and innovative methods. We made an explicit effort to triangulate the outcomes of interest by using both quantitative and qualitative methods. Research on the formal learning environments involved course grade data as well as data collected using a class observation form, a student survey, an instructor interview, and student focus groups. The informal environments research used data gathered by means of student focus groups, student assignment logs, and photo surveys.

Learning Environments Examined

We focused on a number of formal and informal learning environments, including two ALCs and a traditional classroom. The two ALCs in the study consisted of one environment with a capacity of 45 students and a larger environment with a capacity of 117 students. Additionally, this study involved a number of informal study environments selected by the student participants.

In the ALCs, instruction typically involved hands-on activities and problems that required students to interact to reach a solution. The rooms featured a number of large circular tables, each of which accommodated nine students and had three laptop computer connections to large LCD screens to facilitate collaborative work. The ALCs featured 360-degree glass marker boards around the edges. PSTL 1131: Principles of Biological Science, with a course enrollment of 43 students, took place in the midsized ALC (see Figure 1), and FSOS 3101: Personal and Family Finances, with a course enrollment of 117 students, took place in the larger ALC (see Figure 2).

Figure 1. Midsized ALC (Capacity = 45) with a Schematic of the Space
Photo courtesy of Office of Classroom Management, University of Minnesota

Figure 2. Larger ALC (Capacity = 117) with a Schematic of the Space
Photo courtesy of Office of Classroom Management, University of Minnesota

In our study, a separate section of the biology class was taught in a traditional classroom on the University of Minnesota's East Bank Campus (illustrated in Figure 3), which featured standard row seating with a traditional podium at the front of the room. It was a projection-capable classroom with a VCR, a DVD player, and an overhead projector. This classroom featured a large whiteboard in the front intended mainly for instructor use; it also had wireless connectivity available.

Figure 3. Traditional Classroom
Photo courtesy of Office of Classroom Management, University of Minnesota

Results and Key Findings: Formal Learning Environments

Because one section of the biology course was taught in a traditional room and another was taught in the ALC, we had the rare opportunity to employ a quasi-experimental research design. The two sections were taught by the same instructor at the same time of day, but on different days of the week. While we could not assign students randomly to the classrooms to achieve full experimental conditions, we were able to control for a host of other potentially confounding factors, such as time of day, course materials, assignments, schedules, exams, and pedagogical approaches. These controls meant that the only factor allowed to vary systematically across the two sections was the classroom, thereby isolating the independent and relative impacts of the traditional classroom and the ALC on student learning for comparative purposes.

Students in the ALC Outperformed Expectations

Given the lack of randomized assignment of students to the two biology classrooms, we tested a number of demographic variables to ensure that the student populations under consideration were equivalent. The only variable for which a statistically significant difference between the biology sections emerged was students' composite ACT scores. Students in the traditional classroom had an average ACT score of 22.5, while students in the ALC had an average ACT score of 20.5 (d = 2.0; p < .05).⁵

Since composite ACT scores are known to be reliable and valid predictors of course grades, especially for first-year students like those enrolled in PSTL 1131, we would expect the significant difference in ACT scores between sections to be reflected in students' average final course grades. Specifically, students in the traditional classroom would be expected to earn an average of 502.2 points, while students in the ALC would be expected to earn 454.6 points (d = 47.6; p < .05). For students in the traditional classroom setting, composite ACT scores predicted with considerable accuracy the actual average course grade of 499.3 points. However, students in the ALC earned approximately 29.8 more points on average than their ACT scores predicted, an amount that renders the average difference between the sections' final grades statistically insignificant (d = 14.9; p = .26). Thus, as illustrated in Figure 4, our evidence strongly suggests that when controlling for nearly every other factor, the ALC had a significantly positive effect on student learning outcomes as measured by course grades.⁶

Figure 4. Expected Versus Actual Course Grades in Biology, by Section

Qualitative data from our student surveys substantiated these findings. Biology students cited the round tables in the ALC as being important for group work and for helping them get to know more of their classmates. One student went further in her praise of the ALC, stating, "I thought it was a wonderful class[room in which] to take this course. The nonconventional [sic] way of learning worked real[ly] well and forces discussion and thinking." Conversely, students in the traditional classroom were mostly ambivalent about their formal learning environment, suggesting simultaneously that the "set up was good," but that "it is awkward to talk in groups because the desks are long and rectangle-shaped." Another student's evaluation summarized the general opinion of the traditional classroom space: "It's a fine classroom but doesn't really positively or negatively affect my interest in the course topic."

Space Impacts Instructor and Student Behavior

For this study we developed, revised, and tested a classroom observation protocol for reliability and validity. This protocol helped us document the classroom activities at five-minute intervals throughout a class period.

The data gathered in this way from the traditional and ALC sections of the biology class indicated that despite the professor's explicit attempts to conduct the same learning activities in both sections, he behaved quite differently in the two classrooms, lecturing significantly more in the traditional room and conducting discussion significantly more in the ALC (see Figure 5). Additionally, given the physical constraints of the traditional classroom, the professor remained at or near the instructor's podium significantly more than he did in the ALC, which afforded considerably greater freedom of movement throughout the space. The instructor also consulted (discreetly) with individual or small groups of students significantly more in the ALC than in the traditional classroom. Finally, students in the ALC participated in group activities about nine percent more than students in the traditional classroom.

Figure 5. Frequency of Selected Biology Classroom Activities and Faculty Behaviors in Observed Intervals, by Section

While the differences in classroom activities and instructor behavior appeared to be associated with room type, a test of statistical independence is required to make such a claim. Employing tests of statistical independence, we correlated the variable ALC (ALC = 1; traditional classroom = 0) with the dichotomous variables measuring the frequency of classroom activities and faculty behaviors. For each of the variables except group activity, we found that what took place in a classroom was dependent upon classroom type (see Table 1). Specifically, the ALC was strongly correlated with infrequent lectures and frequent discussion, the instructor walking around the room, and the instructor engaging individual or groups of students in discreet conversation. Conversely, the traditional classroom was strongly correlated with frequent lecture delivered mostly from the podium and infrequent class discussion and student consultation.

Table 1. Correlations of Room Type with Selected Classroom Activities and Faculty Behavior

* Cell entries are correlation coefficients (phi)
** p < .0001

Drawing from these results, we employed a lagged, fixed effect logistic regression model that demonstrates causally the impact of the physical environments on instructor and student behavior (see Table 2). Specifically, in the traditional classroom (Model 1), the only instructor behavior that predicted significantly high levels of on-task behavior from students was lecturing; the classroom discussion variable dropped out of the model due to its lack of variation in the traditional classroom. Conversely, in the ALC (Model 2), both group activities and classroom discussion predicted high levels of on-task student behavior at statistically significant levels. From these findings, we conclude that the type of learning environment produces different types of instructional activities and instructor behavior that, in turn, produce appropriate levels of on-task student behavior. Stated more simply, we find that in spaces designed for lecture, only lecture significantly elicits high levels of on-task behavior from students; in ALCs, only active learning approaches to teaching do the same at significant levels.

Table 2. Fixed Effects Logistic Regression Models: Biology Instructor's Behavior and Classroom Activities' Impact on On-Task Behavior, by Section

+ Cell entries are odds ratios and standard errors (in parentheses)
* p < .05
** p < .01

Distinctions in Student Perceptions of the ALCs

To assess the impact of the ALCs on students' learning experiences, we crafted five scales comprised of items designed to assess their perception of the degree to which a classroom promotes engagement, enhances their learning experiences, affords flexibility in approaches to learning, is a good fit to the course being taken, and is used well by the instructor. Each of the scales was identified via factor analysis, met the criteria for construct validity, and was highly reliable (Cronbach's α > .85 for each of the scales).

Our initial comparisons tested whether there were any differences in student perceptions between the traditional classroom and the ALC for the biology course. Compared to students in the traditional classroom setting, students in the ALC thought their classroom contributed significantly more to their engagement (t = 2.22; p < .05), enrichment of their experiences (t = 2.41; p < .05), the flexibility of learning (t = 3.55; p < .001), and the fit of classroom to course (t = 3.58; p < .001).

We then compared differences in student perceptions of the ALCs along demographic lines. Using survey data from the courses held in ALCs (biology and finance), we found no statistically significant differences based on gender or ethnicity. We did, however, find significant differences in how students evaluated the rooms based on two other major distinctions.

First, students from metropolitan Minnesota counties rated the ALCs significantly higher than rural Minnesota students on all five of the dimensions (see Figure 6). There are a few possible explanations for why the metro-rural distinction produces such stark differences in student perceptions of the ALCs, including K–12 exposure to technology-enhanced learning and differing value systems. On this last point, research shows that metropolitan students are more likely to emphasize the importance of autonomy and self-expression, while rural students place greater currency on economic and physical security.⁷ These differences may translate into a greater appreciation for the educational opportunities afforded by the ALCs (metro) and a perception of the ALCs as excessive and a waste of valuable resources (rural). In fact, independent analysis of student comments about the ALCs lends support to this explanation. For example, while a metro student proffered that the ALC "works well because the tables are round and you can see all the students [at] your table, making it better to connect and discuss with you[r] fellow schoolmates," rural students suggested that "the technology is impressive but impractical" and that the ALCs are a "waste of money."

Figure 6. Student Perceptions of the Impact of the ALC on Key Dimensions of Interest, Metro Versus Rural

Second, difference of means tests revealed that juniors and seniors rated the ALCs at significantly lower levels than first-year and sophomore students on all five dimensions (see Figure 7). One plausible explanation for this discrepancy is that upperclassmen may be socialized largely into lecture-based modes of learning and therefore were disconcerted by the introduction of more constructivist approaches to learning supported by the space. As one student remarked, the ALC is "only good for collaborative working. Forcing a class to change its normal means of conveying information for [the] sake of a 'High Tech' room is ridiculous." Another explanation might be that juniors and seniors, who probably have witnessed increases in their tuition over the years, might perceive the ALC as an undesirable waste of scarce university resources. Again, independent analysis of qualitative data supports this position, as demonstrated by one senior who declared, "The design aesthetic [of the ALC] is a waste of money and space. We don't need fancy equipment or flat screens to foster creative energy."

Figure 7. Student Perceptions of the Impact of the ALC on Key Dimensions of Interest, Lower-Division Versus Upper-Division Students

In sum, our formal learning environments research yielded several key findings:

• First, the data showed that students in our new, technology-enhanced learning spaces exceeded final grade expectations relative to their ACT scores, which suggests that features of the spaces significantly contributed to their learning.
• Second, we learned that different learning environments affect teaching-learning activities even when the instructor attempts to hold these activities constant.
• Finally, the data showed that first-year and sophomore students as well as students from metropolitan areas rated the new learning spaces significantly higher than their upper-division and rural counterparts in terms of engagement, enrichment, effectiveness, flexibility, fit, and instructor use.

Results and Key Findings: Informal Student Study Environments

To answer our research questions about informal student study environments, we developed a study assignment log, asked students to complete a photo survey of their selected learning spaces, and conducted student focus groups. The study assignment log focused on the spaces where students study, the characteristics of those spaces, students' study preferences, and whether students would select that space again. Participants in the study were asked to complete an assignment log for each study session, and many participants completed more than one log. We had 91 total participants from a possible 219 students for a 41.6 percent response rate; participants completed a total of 190 logs. From these logs as well as photo surveys and student focus groups, we pinpointed a few key findings.

Study Session Length, Place, and Preferences

Our data suggested that students study mainly in the early evening for an average of 116 minutes per session (s.d. = 75.5), with the plurality of sessions logged lasting between 1 and 1.5 hours. Figure 8 illustrates the length of each study session.

Figure 8. Length and Number of Study Sessions

In each study log, we asked participants to fill in their study location. These locations were then classified into the following main categories:

• Classroom
• Computer lab
• Home or dorm
• Informal campus study
• In transit (bus, train, airplane)
• Library (on campus)
• Off-campus spaces

Figure 9 illustrates that 60.0 percent of the students' logs noted home or dorm room as their selected study location; 10.5 percent noted coffee shops or off-campus spaces; 10.5 percent said libraries; 10.0 percent indicated on-campus informal spaces; 4.7 percent said computer labs; 3.7 percent noted classrooms; and 0.5 percent said train (in transit).

Figure 9. Study Location Selected by Participants

In addition to the student assignment logs, we included photo surveys to gain a richer, deeper answer to our research question. Figure 10 provides two student photos of their study spaces, while an accompanying Animoto video shows more examples.

Figure 10. Examples of Student Study Spaces

In the following excerpt, the students discuss their basic needs and preferences for their study environments (the names are pseudonyms):

Technology Is an Integral Part of Studying

We also learned that the average student in our study uses technology 12.9 times for a total of 50.6 minutes of their 116-minute study session. Table 2 tallies students' average technology use for phone calls, text messages, e-mail messages, and web surfing per study session.

Table 3. Average Technology Use and Duration per Study Session

Study Goals and Habits

Additionally, our study assignment log asked whether students achieved their study goals. In 69.2 percent of the assignment logs completed (133 out of 190 logs), students reported that they achieved their study goals. Interestingly, there were almost no significant relationships between a student's achieving his or her study goals and the other variables measured by the assignment logs. The lone exception was studying at a desk: 86.8 percent of students who studied at a desk achieved their study goals, as compared to 70.2 percent of students who did not (Chi-square = 6.232, p = .013).

Students commented on their ability to concentrate, being comfortable, and having everything they need as the key components to accomplishing their study goals. One student noted, "The space helps because I am able to concentrate and be left alone in this environment." Another said, "Love the environment, the café was pretty low key and my relaxing music helped me focus on my schoolwork." Another mentioned, "It helped because I am comfortable in the space and have everything I need to accomplish my goals." The availability and proximity of food and drink was mentioned frequently. One student stated, "This space help[s] me…without distraction[s] from other thing[s]. Also, I can get food to eat whenever I need to."

In 15.1 percent of the assignment logs completed (29 out of 190 logs), students reported that they did not achieve their goals due to several factors. The top four distractions were roommates, television, telephone, and noise level/crowded spaces. One student commented on "distractions of roommates, temptation of Facebook, and my bed." In addition, students noted they were distracted by uncomfortably cold spaces, messy spaces, the Internet, uncomfortable chairs, or not having all the materials and resources they needed to complete their assignments. One student commented, "It was a bit too cold and the colors were not warm (white walls)."

Interestingly, even though students reported in 29 logs that did not achieve their study goals, they noted in 24 of those 29 logs that they would still select the same study location again. Students cited convenience, comfort, and routine among the main reasons. Student comments included, "I study at my desk all the time," "normally can get stuff done," "convenience," "how I always study," "good lighting; convenience," "comfort," "it's the place I study," and "like coffee and background music."

Assignments and Study Environments

Students indicated that they worked on a several different types of assignments, including, but not limited to, the following tasks and course assignments:

• Calculus problems
• Case studies
• Chapter notes
• Essays
• Group problems
• Lab reports
• Memos
• Reading
• Research
• Research papers
• Statistics assignment
• Study for exams
• Technical instructions

Students noted that they select different types of spaces depending on the type of work assigned and the environment they need to meet their goals. One student mentioned in the assignment log, "For group work, yes! Maybe not for individual studies because other people may be talking." Another student said, "No/yes because it's convenient and a good place to sit and meet with people, but it's hard to concentrate."

The focus group data also suggested that the type of assignment can dictate students' choice of study environment. The following excerpt is taken from the transcript for the PSTL 1131 student focus group (the names are pseudonyms):

Thus, to answer our research question about the types of informal learning environments where students complete their course assignments, we found the data suggested:

• Students select a wide variety of study spaces, mainly their home or dorm room, library spaces, coffee shops, and computer labs.
• Students study, on average, nearly two hours per session, mainly in the early evening.
• Technology is ubiquitously integrated into their study time.
• Students are reluctant to change their study location, even if they do not meet their study goals.
• The type of assignment greatly affects a student's choice of study environment.

The student researchers on our team presented our learning environments research at the National Conference on Undergraduate Research in April 2009, as recorded on this video.

Concluding Implications and Recommendations

Overall, our research findings suggest the following implications and recommendations:

• Understand that critical, invested, and wide-ranging partnerships are the key to successful learning environment endeavors, including empowering undergraduates as equal partners and engaging faculty members who are truly invested in the scholarship of teaching and learning.
• Feel comfortable investing in high-tech, state-of-the-art learning spaces. We know that students in our new, higher-tech learning spaces exceeded final grade expectations relative to their ACT scores, suggesting strongly that features of the room contributed significantly to their learning. Additionally, we recommend investing in faculty development to help instructors maximize the potential of these new learning environments.
• Know that learning environments affect which teaching-learning activities occur in a class. Instructors need to consider the physical attributes of their classrooms as they prepare and deliver their courses.
• Understand that different groups of students respond differently to the ALCs. In particular, students' year in school and whether they come from an urban or rural background might affect their perception of an ALC and thus its influence on their learning experience. We recommend scheduling first-year and sophomore students in the ALCs to maximize their effectiveness and to garner long-term buy-in from students for technology-enhanced classrooms.
• Promote campus study environments before students develop study routines. It is important to introduce students to study spaces during first-year orientation, welcome week, transfer orientations, and any other departmental-, college-, and university-level welcoming sessions or celebrations.
• Know that technology is an integral part of students' study sessions. Students prefer study environments with ready access to a variety of technologies and fast, reliable connectivity.
• Realize that students' assignments and their study choices play an important and understated role in the future design and redesign of informal learning spaces. By understanding the curricular tasks assigned to students as well as their study needs and preferences, we may be able to "encourage students to spend more time on campus, increase engagement, and improve retention."⁸

Further research is needed to fully understand the relationships among different types of learning spaces and the complexities of teaching and learning. In particular, the connection between new, technology-enhanced spaces and improved student learning outcomes should be replicated in future studies, and the role of moderator and mediator variables should be explored. Meanwhile, the University of Minnesota continues to innovate in this area in its construction of the new Science Teaching and Student Services building, which will include several innovative technology-enhanced learning environments.

Acknowledgments

Our sincere thanks to the Archibald G. and Edyth B. Bush Foundation for its financial support of this research, to Linda Jorn for her vision and leadership in the research process, to Andrea Nixon for inspiring us to use innovative research methods, and to Jeremy Todd for his vision in the design and development of the Active Learning Classrooms. We would also like to thank our faculty partners Lee-Ann Kastman Breuch, Jay Hatch, and Catherine Solheim, as well as our undergraduate research partners Tabinda Hasan, Tim Quan, Kevin Race, and Lexi Schmidt. Finally, photo credits go to Jeremy Todd, the student participants, and the University of Minnesota's Office of Information Technology and Office of Classroom Management.

1. Aimee Whiteside, Linda Jorn, Ann Hill Duin, and Steve Fitzgerald, "Using the PAIR-up Model to Evaluate Active Learning Spaces," EDUCAUSE Quarterly, vol. 32, no. 1 (January–March 2009).
2. Linda Jorn, Aimee Whiteside, and Ann Hill Duin, "PAIR-Up," EDUCAUSE Review, vol. 44, no. 2 (March/April 2009).
3. Aimee Whiteside and Steve Fitzgerald, "Designing Learning Spaces for Active Learning," Implications, vol. 7, no. 1 (January 2009), pp. 1–6.
4. Nancy Foster and Susan Gibbons, "Studying Students: The Undergraduate Research Project at the University of Rochester," 2007; Andrea Nixon, "Aligning Learning Space Design and Student Work: Research Implications for Design Processes and Elements," EDUCAUSE Quarterly, vol. 32, no. 1 (January–March 2009).
5. D. Christopher Brooks, "Space Matters: The Impact of Formal Learning Environments on Student Learning," British Journal of Educational Technology, 2010, (DOI:10.1111/j.1467-8535.2010.01098.x).
6. Ibid. Since students were not randomly assigned to groups in this study, it is theoretically possible that a confounding factor is responsible for the learning gains observed in the ALC group. Our position is that, given the controls in place in this quasi-experimental study, this is highly unlikely.
7. Ronald Inglehart, Human Values and Social Change: Findings from the Value Surveys (Boston: Brill, 2003); and Ronald Inglehart and Christian Welzel, Modernization, Cultural Change, and Democracy: The Human Development Sequence (New York: Cambridge University Press, 2005).
8. Cyprien Lomas and Diana Oblinger, "Student Practices and their Impact on Learning Spaces," in Learning Spaces, Diana G. Oblinger, ed. (Boulder, CO: EDUCAUSE, 2006).

© 2010 Aimee L. Whiteside, D. Christopher Brooks, and J. D. Walker. The text of this article is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 license.