A Rubric for Selecting Active Learning Technologies

The evaluation of active learning technology is fraught with a number of pitfalls associated with the broad scope of the term "active learning." Charles Bonwell and James Eison's early work in this area describes active learning as "instructional activities involving students doing things and thinking about what they are doing."^Footnote1 Given the breadth of this definition, a holistic approach is necessary when selecting active learning technologies. Such an approach requires an understanding that active learning exists on a spectrum and in a context that includes, but is not limited to, the space in which the active learning is being conducted, the technology used as part of active learning, the course modality, the activities intended to be supported, and the desired end product of that activity.^Footnote2

In this article, we outline a framework for evaluating technologies for active learning, and two resources serve as the underpinnings of our method. The EDUCAUSE Learning Space Rating System (LSRS) extensively addresses the needs of the physical space for active learning. For the most part, we defer to the LSRS in areas of space design and instead attempt to address the holistic interaction of space and active learning as it relates to the choice of technology. Not all technologies are digital—a whiteboard is a technology, and its suitability could be evaluated using the proposed method along with technologies such as virtual whiteboards or wireless projection. Many technologies—both analog and digital—have as much to do with space planning as they do with active learning.^Footnote3

The second important contribution in this space can be found in the EDUCAUSE Review article "A Rubric for Evaluating E-Learning Tools in Higher Education" by Lauren Anstey and Gavan Watson.^Footnote4 Their Rubric for eLearning Tool Evaluation serves as a model for our work simply because the criteria they use can also apply to active learning. However, Anstey and Watson's rubric emphasizes pedagogical concepts that don't need to be included in a discussion on active learning technology. One example is metacognition. We contend that the inclusion of metacognition is unnecessary in our method due to the fact that it is often included as part of active learning tasks in the form of discussion or sharing of the final product of an active learning session. It is part of the activities themselves, not a function of the technology.

Furthermore, the technology itself should be kept as simple as possible, and new features should be included only if they can be added without impacting usefulness and the ability to participate in class activities. Anstey and Watson's rubric occasionally highlights capabilities that aren't necessary for active learning. In contrast, our work focuses on class participation and ease of use in the learning environment over all else in hopes that these factors will promote technology adoption. Other items within Anstey and Watson's rubric such as Hypermediality and Customization have been reframed into active learning concepts and incorporated into our method.

It is crucial that the evaluation of an individual technology be viewed through the lens of workflow or interaction with other technologies, some of which might not exist or might not yet be part of the institutional environment. Consequently, our approach emphasizes workflow and flexibility within the classroom so that students and instructors are free to choose how they interact within a space.

Rubric for Active Learning Technology Evaluation

In practice, a rubric is useful to help quantify the various aspects of a technology review and allows for a standardized method of comparison between technologies in an objective way. Additionally, a rubric provides an artifact that can act as documentation regarding the selection of a specific technology and serves as a quick reference should a decision need to be revisited or updated. This is especially common in the world of modern technology in which updates are frequently made on a rolling basis and licensing decisions are made as a matter of routine. To support our process for evaluating technologies, we have developed our own rubric that anyone may make a copy of and use:

Rubric for Active Learning Technology Evaluation

Heuristics for Technology Evaluation

The assessment of active learning technologies can be broken down into six separate categories of heuristic evaluation, as illustrated in our rubric. Two of these categories—Easy to Use and Participation—are viewed as foundational. Any technology that fails to move beyond these initial categories in the evaluation is not acceptable, and the evaluation can be halted. These foundational categories were chosen as the basis for the evaluation because they provide a strong indication that the technology respects the ability to enter into an educational experience and participate in activities with minimal resistance.

The use of the term "heuristic" here is intentional in that these evaluation methods require participation from a diverse set of users and/or experts familiar with the context in which these technologies are to be used. For instance, a technology in one context may be easy to use because all individuals who are likely to use the technology have significant previous experience with it due to the institutional requirements. At another institution, the same technology might be completely foreign to prospective users.

Some examples of context include the following:

• An institution's single sign-on (SSO) system: Technology that integrates with the SSO provider of the institution is more likely to be easily accessed by all users.
• Existing presenter amplification: All rooms that the technology will be used in could have an existing presenter microphone and speakers that could be used to interrupt group work, and, as such, this criterion may be deemphasized in the rubric.
• An institution's videoconferencing platform: Because this article is being written during a pandemic, nearly every instructor has some competence with the videoconferencing technology provided by the institution. Preexisting literacy implies a smoother onboarding process for instructors and students.
• Requirements for student devices: Are students expected as a matter of institutional policy to have their own laptop/tablet present, or are physical alternatives such as pen and paper necessary?

Our rubric includes an assumptions field to capture some of these contextual factors. The rubric also includes a column to explain the rationale for the points awarded. This rationale column is to provide documentation around the contextual reasons a particular technology was selected. Should the context of the institution change, this documented evidence will help develop an understanding of the ways a particular technology may or may not fit into the larger active learning context any longer.

Foundational Criteria

Our rubric sets two criteria as foundational, meaning that a technology must satisfy these before it is evaluated against the other criteria.

Easy to Use

If the instructor has to spend noticeable time teaching how to use the technology or setting up the technology for an activity, then this is time lost for meeting learning objectives. Ideally, students and instructors can become skillful with the technology with minimal exposure.

Instructors are more likely to implement active learning strategies if the methods to facilitate the strategies are easy to prepare for and easy to use during the class time.

Participation

By nature, many active learning strategies require the interaction of more than one person. Collaboration between remote and in-person learners might not be necessary, but multiple people do need to be engaged with the same activity during the activity time period.

Additional Evaluation Criteria

If a technology meets the first two criteria, it can then be evaluated against the other four.

Broadly Available

To allow for the most students to be able to participate in an active learning activity, the technology must be usable on whatever types of devices or with whatever materials students already have access to, whether those are personal devices or materials within the learning space. In addition, any barriers for accessing the technology itself must be low, so cost, account access, and software/app installation are considerations. Being broadly available also supports an equitable experience for students.

Equitable

The concept of equitable technology can be broken down along two broad lines: accessibility and student identity inclusiveness.

• All technology must be usable by students with disabilities. The technology can either meet accessibility guidelines (e.g., W3C WCAG 2.1 AA, Universal Design for Learning) or facilitate the use of accommodations.
• It is important for students to be able to express their diverse personal perspectives and learn about other cultures from each other.^Footnote5 Technology that allows students to showcase themselves can help improve equity. Therefore, technology should be designed to address the needs of diverse users—their various literacies, capabilities, and cultural and other identities.

Furthermore, the article "Encouraging Equitable Decision-Making in Academic Technology" clearly establishes that any technology chosen for an academic context must take into account a diverse set of viewpoints and not exclude individuals.^Footnote6 Those who select technologies can improve inclusivity through the selection process itself by engaging a range of stakeholders, adopting effective change management practices, and ensuring transparency. The use of our rubric should improve transparency by providing an artifact that documents the outcome of the process and some areas to document rationale.

Flexible

To best support the widest range of instructional domains and active learning activities, technology must be flexible. Instructors and students should be able to customize the technology to fit their specific needs, and the technology should allow for multiple types of media to be used if applicable (audio, visual, textual). Because group work is heavily used in active learning, technology should support different group sizes and the changing of those groups for different purposes. Flexibility also supports an equitable experience for students.

Suitable for Active Learning Activities

By nature, active learning activities are performed by students without significant instructor involvement. However, for the purposes of facilitating the activity, the instructor needs a method to understand how students are progressing with a task. Instructors also need a way to easily recapture the attention of the class while the active learning activity is taking place. Often active learning activities produce a lot of noise. When students are engaged in the task, they may not realize it is time to proceed to another phase of the activity. Many active learning activities include a sharing phase in which students share their activity results with the whole class or other groups to synthesize ideas, discuss perspectives, or obtain feedback. Instructors might also want a way to understand a student's or group's contributions for the purposes of grading.

Conclusion

Because the use of active learning is characterized by a broad range of activities in the classroom, comparing technology and determining which option provides more benefit to an active learning classroom can be difficult. The Rubric for Active Learning Technology Evaluation can provide some differentiation when comparing technology offerings. It has been designed to reveal subtle but impactful differences between technology in the context of active learning. The rubric was designed to be a tool for comparative technology evaluation and as such should be quick to use when comparing similar technologies. It is freely available to use and adapt under a Creative Commons license.

Notes

1. Charles C. Bonwell and James A. Eison, "Active Learning: Creating Excitement in the Classroom," ERIC Digest, September 1991. Jump back to footnote 1 in the text.↩
2. University of Minnesota Center for Educational Innovation, "Teaching in an Active Learning Classroom (ALC)," n.d., accessed November 30, 2021. Jump back to footnote 2 in the text.↩
3. Gina Sanchez Gibau, Francia Kissel, and Modupe Labode, "Starting with the Space: Integrating Learning Spaces and Technologies," Journal of Teaching and Learning with Technology 8, no. 1 (August 6, 2019), 17–32. Jump back to footnote 3 in the text.↩
4. Lauren Anstey and Gavan Watson, "A Rubric for Evaluating E-Learning Tools in Higher Education," EDUCAUSE Review, September 18, 2018. Jump back to footnote 4 in the text.↩
5. Tanya Joosten, Harness Lindsey, Russ Poulin, Van Davis, and Margaret Baker, "Research Review: Educational Technologies and Their Impact on Student Success for Racial and Ethnic Groups of Interest," The National Research Center for Distance Education and Technological Advancements (DETA), WCET - the WICHE Cooperative for Educational Technologies, May 2021. Jump back to footnote 5 in the text.↩
6. Courtney Plotts and Jenae Cohn, "Encouraging Equitable Decision-Making in Academic Technology," EDUCAUSE Review, September 15, 2021. Jump back to footnote 6 in the text.↩

---

Katie Bush is Senior Instructional Technologist at Rochester Institute of Technology.

Monica Cormier is Instructional Technologist at Rochester Institute of Technology.

Graham Anthony is Manager, Instructional Technology and Media Services, at Rochester Institute of Technology.