Hotseat: Opening the Backchannel in Large Lectures

Key Takeaways

• Despite their flaws as teaching and learning environments, large lectures remain a standard teaching approach because of their relatively low cost, ease of preparation, and long tradition in education.
• Research shows that active inquiry yields positive results in learning outcomes, and one way to make the large lecture class more active and participatory is to create and leverage a backchannel.
• Hotseat is a backchannel management tool developed to connect students to each other and their instructors in these large lecture conditions.

"Instructors who incorporate mobile devices are actively engaging us in the class — they are using the medium of communication that we most often use… [W]e need to find new ways to present material; we don't need more cut-and-dry lecture formats. Engage us in video, ask us questions, challenge us to think for ourselves while applying the class information in our answers, and we will likely not disappoint. Lectures are just one way to present material, but with the prevalence that instructors are using this format today, one would think it was the only way to teach."
— Hotseat survey response

Never before have mobile devices been so widespread, accessible, or widely used. The ECAR undergraduate student study for 2009 found that 51.2 percent of undergraduates surveyed owned an "Internet-capable handheld device, and another 11.8 percent planned to purchase one in the next 12 months."¹ CTIA, the Wireless Association, reports that at the end of 2009, 91 percent of homes had a wireless device, up from 69 percent at the end of 2005 and 38 percent at the end of 2000.²

In higher education, mobile devices in large lectures seem to be the source of growing frustration among instructors. In 1986 Maryellen Gleason presented a set of challenges with the large lecture format,³ which have remained the same for decades. Her points serve as a useful framework to pinpoint potential problems with the format given the increasing number of mobile devices in class:

• Large space. Large lectures take place in a big room, and interaction doesn't feel like personal communication because the instructor is so far away. Tight rows make group discussion difficult.
• Isolation. Large lectures are full of people, mostly strangers, creating a sense for students that what they say and do doesn't matter, leading them to care less about seemingly small distractions (like talking to a neighbor or reading a newspaper), and creating an inhibition about participating in front of a large audience.
• Group size. The sheer number of students makes discussion during a regular lecture that includes everyone (or everyone willing to chime in) impossible.
• Sage on the stage. The instructor appears impersonal, remote, and inaccessible, and the communication gap between the students and the instructor feels (and may be) very real.
• Theater setting. A seating arrangement that feels more like a theater than a class induces student passivity.

These characteristics of large lectures do not describe a learning environment conducive to ideal teaching. Despite this, large lectures continue as a standard approach — perhaps because lecture is an economic instructional solution (it's among the cheapest ways to teach, and easiest to prepare for) and has a long tradition in education (centuries, literally). Research indicates, however, that more active inquiry yields positive results in learning outcomes⁴ and creates a more motivating, stimulating learning environment.⁵ A potential way to make the class more active and participatory is to create and leverage a backchannel.

A classroom can divided into two channels: the frontchannel, the official channel for the class, consisting of interactions with the instructor at the front of the room, and the backchannel, the unofficial channel for the class, consisting of interactions among the audience, or perhaps with those outside the class. The backchannel could include whispering, side conversations, or passing notes. In our current technological environment, the backchannel has progressed from paper notes to texts and Facebook updates for students, and Twitter for professionals. Jeremy Roschelle makes a distinction between the old and new, and describes this difference as a social layer (in this case whispering or notes) and an informatic layer (in this case texts or Facebook).⁶ The shift to the informatic layer moves the conversation from the physical space, where the instructor has influence and access, to a private information space the instructor is locked out of. One approach, then, is to create a backchannel for the course and have the instructor check in with it or use it explicitly during class. We developed Hotseat as a tool to connect students to each other and instructors in these conditions.

Effective technology integration has two parts: a good tool that does what it's meant to do, and a good implementation that meets the instructional goals of the course. Over time, new technology finds a niche or fills a gap, improving the existing structure. Our experience with Hotseat followed this pattern — it provides a way to artificially decrease the classroom size, giving a larger number of students the chance to participate in conversation in class. In this article we will describe Hotseat and the features designed to make it useful in instructional situations. We will then describe how it was implemented at Purdue University and how it affected the large lecture learning environment Gleason described.

What Is Hotseat?

Hotseat at Purdue University

Hotseat was developed at Purdue University as a way to include and improve on existing backchannel solutions for learning environments. It is a social networking tool that creates a backchannel of collaborative discussion both in and out of the classroom. It allows students to provide near real-time feedback during class, potentially enabling faculty to adjust course content and enrich the overall classroom experience. Hotseat makes this interaction possible by taking advantage of the mobile devices that students already bring into the classroom and by using popular social tools such as Facebook, Twitter, and text messaging.

Instructors provide a question, comment, or framework, and students contribute a short response using their preferred mobile device. These responses, or "thoughts" within Hotseat, are automatically updated as submitted by the class. Students can read, vote, and comment on posts made by other students, thus identifying common questions or areas of discussion (see Figure 1). Access to Hotseat is granted by the student's standard university login and password and is typically based on course enrollment. Some faculty also choose to broaden the conversation by allowing open contribution to the discussion topic from people outside of the class. These features allow for rapid community building in the course.

Figure 1. Hotseat Topic View

Unlike typical threaded discussions, Hotseat enables rapid-fire group communication, or microdiscussion. The microdiscussion format consists of short-form comments or questions similar to those popularized by text messaging or microblogging tools such as Twitter. Hotseat further facilitates the discussion by presenting a simplified interface for quickly reading posts, voting, responding to relevant posts, and marking favorite posts for later review. Each discussion can also be sorted based on posts that are "fresh" (most recent), "hot" (most popular), and "deep" (most discussed).

Developing Social Tools for the Classroom

Delivering any new technology to the classroom requires significant forethought and planning. Delivering a faculty-endorsed classroom distraction requires mitigating a unique combination of challenges that, for some faculty, border on the extreme. Through developing Hotseat, we identified four core challenges that needed to be overcome to create an open backchannel within the classroom. While these concepts guided our product development, the same considerations could be used to evaluate any potential classroom backchannel technology.

Technology Challenge

Most of us can imagine the ideal scenario for student technology in the classroom. This vision would typically include some type of standard device. Some institutions use a low-cost clicker solution, while others have adopted a standard laptop, phone, or tablet. Deploying an open and easy-to-use backchannel within the classroom requires using an "un-clicker," or taking advantage of whatever technology the students already own.

In developing Hotseat, we adopted a philosophy of "run what you brung." Whatever device the students carry into the classroom, whether phone, netbook, or laptop, this should be the technology used to engage the class in discussion. Hotseat makes this possible by providing interfaces to a wide range of devices. Many students use laptops or netbooks to monitor the classroom backchannel, while others use a mobile web interface designed for the iPhone (Figure 2), Android (Figure 3), and Opera Mini. For students who didn't have or didn't want to use either device, the system accepts tweets created using Twitter's text messaging capabilities. As of fall 2010, Hotseat also accepts direct text messages via SMS. Figure 4 shows a projected view of Hotseat as students submit their thoughts concerning the topic of investment bonds.

Figure 2. Hotseat iPhone Screen

Figure 3. Hotseat Droid Screen

Figure 4. Hotseat Projector Display

Access Challenge

A key consideration for IT organizations that provide student technology is, how do you support student access to the technology on a regular basis? Put another way, how do you get students into the system, and how do they know what to do when they get there?

Backchannel communication is typically an informal and spontaneous conversation among a group of people related to a live event; providing traditional IT call center and training support seems contrary to providing the functionality. Nearly every student uses social networking tools such as Facebook,⁷ and many of these tools include some type of short-form status update. Implementing an effective backchannel means not breaking this already understood model of online chat and taking advantage of the tools that students are already familiar with.

For the Hotseat project, we designed the system to take advantage of the social and mobile technologies students already use. Open APIs provided by Facebook and Twitter make it possible to create one aggregate interface in which conversation can occur. This aggregation establishes a single source of information, but grants the students their choice of access.

While it is difficult to measure ease of use, one common indicator is user adoption. In the initial Hotseat pilot with three different large lecture courses, 86 percent of the students enrolled used Hotseat, and 60 percent used it at least once a week.

Social Challenge

Of the common faculty concerns, most center on content credibility and content relevance. The best — and worst — feature of any backchannel discussion is that anyone can say anything they want on any topic. While effective in bringing out a diversity of opinions, in a learning context this openness can give weight to inaccurate information and threaten the overall credibility of the content. In addition, it provides an opportunity for sharing information that might not be relevant to the discussion — for example, the "comment-shaped grenade." If anyone can post anything, what prevents someone from lobbing an off-topic comment into the class with the intent to harm or, at the very least, distract the class? For these reasons, adopting social tools within the classroom requires accountability.

Students also have concerns about using social tools in their coursework because they can find themselves in a socially awkward position with both the instructor and the class. This is one reason to avoid using only Twitter or Facebook and requiring students to permit access to their existing social networks. Hotseat acts as a separate, class-related social network accessible from Facebook that accepts content from Twitter, but does not push content back into either space. Because engagement within large lecture classes can be difficult for many students, the goal is to increase communication and decrease students' perceived risks of participation. For these reasons, adopting social tools within the classroom requires the option of anonymity.

Accountability and anonymity are seemingly opposing forces. Using a backchannel tool makes it possible to accomplish both. One of the principal technical issues in developing Hotseat was creating a means to connect institutional credentials with social credentials. Using the Facebook and Twitter APIs, we developed a technique to combine accounts in such a way that students using Facebook or Twitter to engage the classroom backchannel could be identified by their institutional identity. This process makes it possible to provide faculty with the accountability needed to ensure a relevant discussion and reward those students making a contribution to the classroom discussion, no matter how they access it or what technology they use.

In addition to linking with social credentials, students can display individual posts anonymously in Hotseat. Within the system these posts are still attributed to the student, but displaying posts without making their owner's identity visible makes it easier for a student to ask difficult questions or offer a controversial opinion. Of the 38,000 comments and questions that students submitted to Hotseat in our pilot project, 53 percent were displayed anonymously. Figure 5 shows the display options.

Figure 5. Hotseat Display Options

Assessment Challenge

For some, introducing a backchannel into the classroom holds allure — but how do we determine its effects? Capturing analytics to use in assessing impact is an important aspect of any technology implementation. Open tools, such as Twitter, do an excellent job of creating an environment for backchannel communication. However, they require students to use applications hosted externally, with limited access to analytics information. While some usage data can be mined through open APIs, matching it with the student's course information can be an arduous manual process.

As part of providing a system of accountability within the backchannel, Hotseat captures and stores analytics related to each student's use of the system (see Figure 6). Faculty have access to real-time reporting where they can review a student's contribution to one discussion or a history of discussions during the semester. This data is also available for download, where it can be combined with the learning management system gradebook data to evaluate student participation in relationship to course performance. By automating the process of collecting and reporting assessment data, the system enables faculty to assess the backchannel's impact on learning outcomes.

Figure 6. Hotseat Usage Report

In addition, instructors can download the entire input for a course space, the content and order of replies, posts, and votes, identified by student. We have used this feature extensively in our research, but it also has been helpful for instructors who want to collect data on student responses in one place. All reporting features are within the tool itself, and the instructor can navigate to them when logging in.

How Does Hotseat Differ from Clickers and Twitter?

Three main differences distinguish Hotseat from clickers:

1. Clickers require students to purchase either standardized hardware or a license for existing mobile devices. Hotseat uses existing devices, including mobile devices.
2. For students who do not have mobile devices in class, Hotseat continues the conversation outside of class. In the spring 2010 semester, over half of all posts were made outside of class, and posting patterns ranged throughout the day, from early in the morning to early the next morning.
3. Hotseat shifts the locus of control from instructor to students by allowing open-ended input. Like microblogging tools, Hotseat builds community. Open responses, seen not just by the instructor but also by other students, allow development of ideas and let students see what other students are thinking. With Hotseat, the instructor can quickly get an accurate response to the question, "What was the toughest concept today?" or a host of other questions more meaningfully answered by students in an open format.

Another advantage of the microblogging/microdiscussion format is the brevity of the messages. One instructor informed us that Hotseat was faster and cheaper than the focus groups she had planned to use to get student input on the course and determine where additional strategies are needed to improve students' understanding of the concepts taught.

Twitter is the most common tool for enabling backchannel discussion, and some institutions have had success using Twitter as a learning environment. With a strong development community, a plethora of free tools, and a wide range of input options, Twitter is a viable alternative for classroom discussion. However, it is not without trade-offs.

Twitter is the new word-of-mouth — but with the entire world eavesdropping. This would also be true of a classroom discussion within Twitter. The common method of identifying tweets for a specific conversation is with a hashtag. This ad-hoc functionality doesn't afford any level of privacy or restriction of the conversation — anyone in the world with a Twitter account can not only watch but also participate. Some instructors might see this as a benefit because it extends the backchannel outside of the institution, but Hotseat can also limit the conversation to students enrolled in the class.

Another common challenge in using Twitter is that it requires everyone to have and use Twitter accounts. In addition, instructors may need to record which students are connected to each account — a cumbersome process for large-enrollment classes. Hotseat handles identification of students for accountability purposes, as already noted.

Hotseat Implementation

Hotseat was used with three large classes having roughly 700 undergraduate students in the fall of 2009 and with eight large classes having roughly 2,000 undergraduate students in the spring of 2010. This piece will focus on data from the spring. During both semesters we gathered Hotseat participation data and deployed a student perception survey, which evolved in the spring based on responses from the fall; in the spring we added a social presence survey. We also gathered grade data for each course. We received variable responses to the survey, so when reporting grade and survey data will focus mainly on three large lecture courses with well-rounded data with roughly 650 responses. A brief description of the implementations we observed follows:

• An instructor in a large lecture family development class (more than 250 students) used Hotseat to have discussions in class, asking questions related to lecture material. After posting the question, he gave students time to get into a group and answer, then brought up the responses on screen and reviewed them in front of the class. Students could also answer outside of class.
• An instructor in a large lecture finance course (more than 250 students) used Hotseat to field student questions during lecture. Two graduate teaching assistants on laptops answered questions as they came in. Every 15 minutes or so the instructor paused his lecture for a few minutes and took questions gathered by the teaching assistants.
• An instructor teaching human sexuality (roughly 70 students) used Hotseat to help students feel more comfortable talking about the subject. He asked broad, open-ended questions related to the day's topic and got a sense for students' opinions and concerns.
• An instructor in a large lecture biology course (more than 450 students) used Hotseat outside of class as a way to diagnose scientific misunderstandings. She would post a question, ask students for an explanation, then use their responses and discussion to find common misconceptions she needed to address in class.

We'll use Gleason's solutions for the challenges of large lecture classes to frame the implementation and findings.

Large Space

If interpersonal communication seems difficult in a large lecture and students can't easily physically turn and talk to each other, Gleason's solution is, "A space is made to feel small by treating it as if it were. In other words, instructors ought to do things in large course spaces that they do in smaller ones."⁸ She suggested literally moving closer to students, so that it feels more like personal communication. We suggest that one way to shrink the physical space is to superimpose over it an informatic layer, not bound by the limitations of the room, where communication is rapid and available to everyone and the instructor can communicate with students by responding to their questions or comments. This can move students and instructors closer together virtually and mentally, to overcome the gap in space.

Isolation

Gleason suggested that to reduce the feeling of isolation, instructors can create a supportive learning environment where students feel individually validated. One instructor in our pilot stood out as an example in this area. He taught a course on sexuality to 70 students. He wanted to both decrease isolation and help students overcome natural inhibitions that arise when discussing the topic. For example, he asked, "What kind of insecurities do you think men might have about their bodies and their sexuality?" This question received 36 responses, 35 from unique users. It's difficult to imagine a classroom in which 35 students (half the class) would feel comfortable sharing on this topic face-to-face, no matter what the student age. Roughly half the responses were made in class, and half were made out of class over the next week. Used this way, Hotseat allowed more participation during class and more cohesion among the class community outside of class.

This instructor in particular was good at creating a supportive environment in which student comments were validated and addressed. One way he did this was by paying attention to students and by telling them how many points the test would be worth and the fraction of the participation points that would be attributed to Hotseat. He posted 12 times, replied 31 times, and voted 15 times. The class posting average was 27 — he was active in his own discussions, validating student opinions and encouraging ideas. He logged in 111 times, at least once a week during weeks when class was held and often more than once a week.

Across all courses, students reported Hotseat helped them cut through the isolation of the large lecture learning environment: 76 percent (n = 792) agreed or strongly agreed that "Hotseat helped me see what other people in the class were thinking" (Figure 7), and 68 percent (n = 790) agreed or strongly agreed that "Seeing what other people in the class were thinking improved the discussion" (Figure 8).

Figure 7. Responses to "Seeing What Other People Were Thinking"

Figure 8. Responses to "Improved the Discussion"

Group Size

Gleason suggested that the way to overcome the problem of group size is for "receivers [to] become senders." That is, have the students, previously passive, share information using active learning strategies. She continued, "Communication works best when senders and receivers depend on each other for the construction of mutually agreed upon meanings. It frequently breaks down when the exchanges become linear, as in something a sender does to a receiver. Lectures tend to be linear."⁹ Jane Caldwell determined that one-way communication and difficulty in concentrating are major drawbacks of a large lecture.¹⁰ One of the hallmarks of backchannels on Web 2.0 tools like Hotseat, Twitter, and Facebook is that as the number of participants increases, the user experience improves. Moreover, the direction of communication is immediately multidirectional — anyone can read what's being said, and anyone can chime in. In direct contrast to the limited time and space of direct instruction, backchannels seem to work best when used by a large number of people with a variety of engagement styles.

Two courses on family relationships shifted the student role from receiver to sender. The instructor wanted to increase engagement and discussion, but had severe hearing loss in one ear and used Hotseat as a way to check in with students on their understanding of course topics. Since the course dealt with family and relationships, he regularly addressed controversial topics like gay marriage or assisted suicide, so he also wanted a way for students to feel comfortable sharing their thoughts. His strategy was to bring up a question, give students time to respond in class (usually a couple of minutes), then bring up Hotseat on the screen in front of the class. As the responses came in, he would review them; as students finished, he would scroll through some responses, commenting on the posts and using them as a springboard to highlight specific aspects of the topic.

Our survey indicated that overall 71 percent (n = 790) of students agreed or strongly agreed that "Hotseat helped me get my opinion out without having to answer in front of the whole class." This response indicates success in getting students to participate in large numbers without disrupting the frontchannel of class and overcoming barriers of inhibition.

Students Explain Hotseat

Sage on the Stage

To appear less distant, instructors can share personal information, for example the way they understood a difficult concept they are teaching, or address student questions in a supportive way. In the finance course, at one point the instructor shared an inaccurate historical fact. A student pointed this out on Hotseat, the TA pointed it out to the instructor, and the instructor acknowledged his mistake and commended the student for letting him know. Hotseat might have lowered the bar for students' participation, and in this case the instructor chose to reinforce the behavior, making himself appear more human and approachable. Instructors also achieved this end by replying to student posts and "coming down" to the students' level. Even here it is interesting to note the "chain of command" that emerged.

Theater Setting

To overcome the naturally passive state students seem to slump into in large lecture settings, Gleason suggested giving them a role in the course through active learning activities. Active learning in large lectures can take many forms, but enabling strategies that work in small classrooms, like discussions and giving students opportunities to ask questions, is a start. Hotseat makes this easier by enabling multiple students to respond simultaneously in class and by allowing students to ask questions without feeling like they are "interrupting the show." We also found that students are more likely to participate when instructors give them structured activities.

For example, in the spring of 2010, in three courses with over 200 students each, with comparable point incentives for participation (1–3 percent extra credit), one type of implementation of Hotseat yielded dramatically more responses than the other. The courses where the instructor asked students to answer questions on Hotseat generated significantly more posts than the course where students asked the instructor questions. Table 1 shows student participation based on the instructor's approach, while Figure 10 shows average posts per course.

Table 1. Student Participation Through Hotseat

Figure 10. Average Student Posts per Course

Student Experience

To measure the feeling of connectedness in the large classes, we used a measurement of social presence. According to Chih-Hsiung Tu, social presence refers to "the degree of feeling, perception, and reaction of being connected" to others using a specific technology, in this case Hotseat. We used the Computer-Mediated Communication (CMC) Questionnaire Tu developed¹¹ to measure social presence.

Hyo-Jeong So and Thomas Brush performed an exploratory factor analysis to refine the instrument. The Cronbach's alpha coefficient for 17 items for the Social Presence Scale was .85.¹² We used their instrument with those 17 items and replaced CMC with Hotseat (see "Social Presence Survey"). We received completed surveys from the Finance course (162), Family Science 1 course (276), and Family Science 2 course (215).

Overall, we found that students agreed or strongly agreed that it was easy to express what they wanted to communicate through Hotseat (65 percent), that where they access Hotseat doesn't affect their ability or desire to participate (57 percent), and that using Hotseat was a pleasant way to communicate with others (59 percent). Other items had overall agreement, but differences showed up based on the course. Overall, for example, students agreed or strongly agreed that Hotseat was confidential (59 percent), but students in Family Science had higher percentages of agreement (FS 1 = 61 percent, FS 2 = 68 percent) than students in Finance (48 percent). Various factors might have influenced that response, of course. A more dramatic example is that 58 percent of students agreed or strongly agreed that Hotseat messages convey feeling and emotion, with 30 percent coming from Finance students, 64 percent from Family Science 1, and 81 percent from Family Science 2. One explanation could be the course content. Family Science 1 dealt with factual information related to families and development; Family Science 2 dealt with more controversial topics such as conflicts in relationships and gender roles; and Finances dealt with factual content, procedures, and current events. Perhaps the Hotseat activities related to emotional content in the Family Science courses influenced student perceptions of Hotseat.

For the two Family Science courses, students who received A's for the courses posted more on average than students who received B's, B's posted more than C's, and so on (see Figures 10 and 11). For those two courses, statistical correlations between total points in the class and posts for students who passed (received a grade of D or higher) were positive and significant, which means that students who did well in the course generally posted more — it was part of their course activities. For the Personal Finance class there appears to be no correlation, possibly due to the student question approach, which does not include all students. Interestingly, for the Finance course, B students had the highest posting average (see Figure 12). This could be because students who received A's had fewer questions, since student questions were the content of most posts. Table 2 shows the correlations of passing students.

Figure 10. Average Number of Posts and Course Grade for FS 1

Figure 11. Average Number of Posts and Course Grade for FS 2

Figure 12. Average Number of Posts and Course Grade for Finance

Table 2. Correlations of Passing Students (D or Higher)

* Significant at the .01 level (2-tailed)

Instructor Perspective

Instructors who used Hotseat regularly reported that they'd use it again. The three instructors who used the tool in the fall of 2009 used it again for the same courses in spring 2010, and again in fall 2010. It seemed to work most effectively when it solved a particular instructional need for the course – getting students to participate more, ask questions they were reluctant to ask initially, or give feedback en masse.

However, three specific facets of the backchannel seem to concern instructors. First, they worry that students do not know how to pay attention or regulate themselves. Of students in our study, 64 percent (n = 789) agreed or strongly agreed that "When I use my mobile device in-class for non-course-related activities (like texting friends), I'm less engaged in the class." Students have a sense of when they're distracted. In addition, not all mobile devices appear to be equally distracting — qualitative comments from the student survey indicate that laptops are more distracting than smaller devices because of the effect they have on the students around them. To be clear, one of the goals of Hotseat is to help develop an in-class community and to reduce distracting use of mobile devices. We also ask instructors to consider their own habits of backchannel or mobile device use; most students have been using mobile devices as long or longer than we have and likely have developed patterns that work for them. We expect a great deal from our students and in many ways treat them as adults; perhaps this is another way that we can extend them the courtesy of responsibility, with some clear guidance and support. As one student put it, "I feel like it's a students [sic] choice to use a computer, phone, or any other mobile device. If they want to fail the course because they go to class but spend the majority of the time on Facebook, so be it."

Second, instructors perceive mobile device use as disrespectful. To address this concern, we found that when asked if they disrespected the instructor when they used mobile devices in class, responses were balanced but leaned in the direction of "yes": 33 percent agreed, 20 percent disagreed, and 31 percent stayed neutral (n = 786). These were large lecture, introductory courses with many freshman and sophomore students who did well in traditional high school settings, and that influence persists. It's a shift for them as well as the instructor to feel comfortable using these tools in a productive way in class, and they will probably respond to cues from the instructor.

Third, instructors seem to worry that the online conversation will devolve into jokes and distraction at best, and criticism of the course and instructor at worst. Naturally, most students played around when testing the new tool, but their use quickly became more functional. Even though comments might be anonymous to peers, they were not anonymous to the instructor. The bulk of messages consequently were content or course related. Again, students will often take direction from the instructor in this respect.

As with most technology, it critical to implement the tool well: setting up realistic expectations and consequences, adjusting to feedback, and using the tool to maximize student learning.

Best Practices

While we're reluctant to prescribe usage so soon, we do recommend some general best practices:

• After students understand how Hotseat actually works (which tends to be very fast), they care about two things: how serious the instructor is about using the tool to help them learn, and whether or not they get points for it (and how they get them). Decide before class starts how the tool will help your students reach their learning goals, explain that to the students, and be clear about what they need to do to get the participation or extra credit points.
• Additionally, a small amount of extra credit or participation points will go a long way in encouraging students to use the tool. We've seen success with as low as 1–3 percent of the total points.
• Use the tool regularly and predictably. If it will be used every class, some students may get accustomed to bringing their laptops or other mobile devices to every class. However, those who don't prefer to know ahead of time when to bring them and whether or not to check the topic after class.
• Students indicated that Hotseat helped them see what other people in the class were thinking (76 percent agreed or strongly agreed, n = 792). When looking for topics to get student input on, questions about class management, controversial topics, frequently asked questions, or answers to challenging (or apparently too easy) questions are good fodder for situations where lowering the bar for "safe" participation can pay off for the group.
• If the tool is used during class for students to ask questions, give students who prefer not to use mobile devices during class the chance to go back in later and participate. This may mean checking in on the posts and voting or replying to other posts after class. Students tend to notice when instructors read their posts (and reply) and when they don't. One advantage of Hotseat is that the limited number of characters per post makes reading them a quick task.
• If they feel they won't be punished for it socially (by their classmates or the instructor), students will often be honest about what's working and what isn't. We suggest getting feedback about the how things are going with the tool through the tool itself. When that feedback is addressed (through acknowledgement or action), students are likely to appreciate the sincerity of the instructor's approach.

Next Steps

Feedback from the 2009–2010 school year indicated that accessibility was an issue with laptops and smartphones, and that the Twitter implementation for cell phones was not an option (perhaps because the students don't use Twitter, or because of the extra steps involved). For the fall 2010 semester, Hotseat was enhanced by the developers at Purdue so that students could text thoughts into it directly from their cell phones. They can also vote on instructor-selected thoughts via cell phone.

In this article the research related to course outcomes, specifically grades, is descriptive. During this early stage in the development and implementation of Hotseat, we are seeing some positive results, but we are still trying to make sense of what they might mean, how they might be influenced, and how Hotseat can improve learning. We're continuing to do research in this area and look forward to seeing the effects of specific implementations of Hotseat and other tools and how they relate causally, if at all, to learning outcomes. We suggest that it's perhaps as important to question current teaching techniques as it is new tools that supplement or challenge them.

Conclusion

Because so many students have cell phones, text with them so often, and will probably have better and faster cell phones every year, we're more likely to succeed in engaging (and teaching) them if we take advantage of the devices they already use. In many ways it's an ideal situation because most students will not need to buy new devices. Instructors can have a better sense of what questions students have and what they're thinking — a dream scenario for many teachers, no matter what they decide to do with that information.

Acknowledgments

We would like to thank faculty who participated (and continue to participate) in this effort, in particular Sugato Chakravarty and Charles Calahan, who were pioneers and innovators. We also would like to thank Purdue's Informatics team for their nimble and timely development of Hotseat.

1. Shannon D. Smith, Gail Salaway, Judith B. Caruso, The ECAR Study of Undergraduate Students and Information Technology, 2009, vol. 6 (Boulder, CO: EDUCAUSE Center for Analysis and Research, 2009), p. 85.
2. CTIA — The Wireless Association, "Wireless Quick Facts, Year-End Figures."
3. Maryellen Gleason, "Better Communication in Large Courses," College Teaching, vol. 34, no. 1 (1986), pp. 20–24.
4. Adam Butler, Kayah-Bah Phillmann, and Lona Smart, "Active Learning Within a Lecture: Assessing the Impact of Short, In-Class Writing Exercises," Teaching of Psychology, vol. 28, no. 4 (2001), pp. 257–259.
5. Jennifer K. Knight and William B. Wood, "Teaching More by Lecturing Less," Cell Biology Education, vol. 4 (2005), pp. 298–310.
6. Jeremy Roschelle, "Unlocking the Learning Value of Wireless Mobile Devices," Journal of Computer Assisted Learning, vol. 19 (2003), pp. 260–272.
7. Silvia Ruiz, Jessica Sharkness, Kimberly Kelly, Linda DeAngelo, and John Pryor, "Findings from the 2009 Administration of the Your First College Year (YFCY): National Aggregates," HERI.
8. Gleason, "Better Communication in Large Courses," p. 20.
9. Ibid., p. 22.
10. Jane E. Caldwell, "Clickers in the Large Classroom: Current Research and Best-Practice Tips," CBE Life Sciences Education, vol. 6, no. 1 (2007), pp. 9–20.
11. Chih-Hsiung Tu, "The Impacts of Text-Based CMC on Online Social Presence," Journal of Interactive Learning, vol. 1, no. 2 (Fall 2002), pp. 1–24.
12. Hyo-Jeong So and Thomas A. Brush, "Student Perceptions of Collaborative Learning, Social Presence and Satisfaction in a Blended Learning Environment: Relationships and Critical Factors," Computers & Education, vol. 51 (August 2008), pp. 318–336; see p. 324.

© Hans Aagard, Kyle Bowen, and Larissa Olesova. The text of this article is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 license.