Proctoring Software in Higher Ed: Prevalence and Patterns

Driven by the proliferation of online learning and by institutional use of remote and blended forms of teaching and learning due to the COVID-19 pandemic, higher education institutions are placing an increased emphasis on remote proctoring technologies. But do we know how common the use of remote proctoring is among North American colleges and universities?

The answer to this question matters for students, faculty, staff, and administrators and is important for practical, scholarly, and ethical reasons. Practical reasons include the fact that academic integrity is at the core of the operations of higher education institutions. In addition, since higher education practitioners often learn from, reflect on, and evaluate the activities of their peers, understanding the preponderance (or not) of such tools may inform institutional practices. The answer is critical for scholarly reasons because it enables researchers to develop a better understanding of the landscape of educational technology use in higher education and because the evidence for the effectiveness of these technologies—or for the degree to which academic misconduct may be more likely in online settings—is inconsistent. Equally importantly, the answer to this question is important for ethical reasons that go beyond the discipline of educational technology. Specifically, many faculty, students, and administrators have critiqued proctoring tools and have expressed significant concerns about their use to monitor students' behavior. For instance, the Electronic Frontier Foundation has gathered a long list of student petitions against the required use of these technologies, and the students' concerns have been reported in the popular press.^Footnote1

Meanwhile, the proctoring software industry advances a narrative—evident in the websites of these proctoring companies—that such tools ensure academic integrity by deterring academic dishonesty and verifying that test and assessment results are valid and reliable.^Footnote2 In short, as Steve Kolowich argued in 2013, proctoring software companies are "hired by universities to police the integrity of their online courses."^Footnote3 Critics of such tools view them as not only unethical but also largely unnecessary, pedagogically bereft, and inequitable and as something that causes various stressors for students. Critics further argue that proctoring software is better described as a set of surveillance tools that foster a culture of distrust and are grounded in bad-faith views of students' honesty and integrity.^Footnote4 Much of this important critique has been presented in detail in a recent teach-in, Against Surveillance, "about surveillance, educational technologies, academic freedom, and student care."

If proctoring software is truly harmful, understanding the extent of its adoption—the prevalence and patterns—is imperative, since this information may allow estimates not just of expenditures but also of potential harm.^Footnote5 Yet very little literature is available to indicate how many colleges and universities make use of these technologies. A November 2020 Washington Post article noted that "thousands of colleges in recent months" had been using proctoring software.^Footnote6 The proctoring company Examity indicates that "more than 500 colleges and employers use its services," and another proctoring firm, ProctorU, notes that it works with over 1,000 institutions.^Footnote7 Other proctoring companies state, on their websites, that they have proctored many millions of tests.

In April 2020, EDUCAUSE conducted a QuickPoll about grading and proctoring. Findings showed that about half of the 312 respondents reported using online/remote proctoring tools. Five services dominated the market for proctoring software: Respondus (65%), ProctorU (23%), Proctorio (17%), Examity (12%), and HonorLock (12%).^Footnote8 While these findings are valuable and enable us to begin to make sense of the range of popular solutions, and potentially of the scale of use, the representativeness of the survey, sample size, and granularity (e.g., at the state or provincial level) may limit the usefulness of its results.

A Wider View

We wanted to take a wider view. Building on the work of EDUCAUSE, we used the Google Custom Search Application Programming Interface and a list of 2,155 college and university websites in the United States (n=1,923) and Canada (n=232) to determine how widely these tools and services were being used. If a college or university subscribed to one of these proctoring services, we assumed that it would reference the service somewhere on its website, so between November 13 and November 18, 2020, we searched each website for unique terms associated with the top five services (e.g., "proctorio," "respondus") and noted the number of results. We also checked the first ten results for each institution to ensure representativeness of the query—in either page titles or summary snippets—to confirm precision, and we considered each site that returned at least one validated reference to a proctoring service as representing some level of adoption. Each site returned between 0 and 3,290 results for each term, showing that in some cases, mentions of proctoring software at educational institutions seem to be highly active.

To evaluate whether results actually indicated adoption, we read through 100 randomly returned results and found that they included the following:

• Explanations of an adopted service (e.g., "Honorlock is an online ID verification and proctoring tool.")
• Explanations of how to use a service (e.g., "Guide for Selecting Settings for Proctorio")
• Announcements for training webinars on a service (e.g., "Accessing Examity")
• Information on how students can sign up or pay for a service (e.g., "Examity. Proctoring Fee: $25; all major credit and debit cards accepted.")
• System requirements for a service (e.g., "Students interested in taking the Accuplacer through Examity must have a computer that meets Examity/Accuplacer requirements.")
• Links to a service (e.g., "Blackboard Tests; Proctoring with Respondus; Blackboard Journals")
• Mentions of a service on a purchase list (e.g., "Examity" being included on a list of "Available Services for Faculty")

Of these 100 results, 4 did not clearly reveal that the institution had adopted the service, but upon further examination of these exceptions, we discovered that all 4 institutions had indeed adopted the mentioned service. We took this to indicate that results were reasonably reliable to show institutional adoption.

We also noted that of these 100 results, none took a critical stance toward proctoring tools or addressed the ethics of student surveillance. To check this, we also did a keyword search on all results to explore how many included the words "privacy" or "surveillance" in their title or summary snippet. We chose these two keywords because their use might indicate a critical approach to these tools. We found that less than 1% included one of these keywords, which suggests that virtually all results approached these tools in a neutral, detached, or supportive manner.

Of course, institutional websites are often massive, representing tens or hundreds of thousands of pages and various internal groups with their own subscriptions, licenses, and approaches to educational technology software. Thus, the mention of "ProctorU," for example, on an institutional website may suggest adoption of the service by one or more entities within the college or university, but it does not mean that the entire campus uses the service or that it has purchased a site license. In fact, in many cases, costs for proctoring services are directly paid by students to the servicer.

Context for Further Discussion

Our results show that nearly 63% of colleges and universities in the United States and Canada mention proctoring software, indicating use. Our numbers generally align with the QuickPoll results from EDUCAUSE, but they also provide more nuanced results by state or province. Overall, colleges and universities in the United States were more likely to use one of these services than their counterparts in Canada (see table 1).

Table 1. Mentions of Proctoring Software on US and Canadian College and University Websites

In Canada, of the four largest provinces, Quebec seems to be an outlier. While mentions in Ontario, British Columbia, and Alberta range from about 69% to around 77%, Quebec's use is only around 8.9%. On the other hand, in nearly 90% of US states, more than half of the institutions mention these services (see figure 1).

These results indicate that proctoring tools and services are becoming increasingly ubiquitous among higher education institutions in North America—though more so in the United States than in Canada. Yet although they confirm broad use, these results do not reveal the degree to which these tools are used within institutions or other nuances around their adoption. For instance, while some institutions might adopt these tools for courses that require external accreditation, others may adopt them for all courses that require exams. Furthermore, while these technologies share similarities and concerns, choices between them also encompass ethical dimensions. The University of Victoria in British Columbia, for example, reports that it does not approve "the use of artificial intelligence / surveillance proctoring tools, such as Respondus Monitor or Proctorio," but it does support the Respondus LockDown browser in its broader efforts to secure examinations.^Footnote9

Since the ongoing COVID-19 pandemic seems to have caused a spike in adoptions of proctoring tools,^Footnote10 the higher education community should take extra care when implementing these tools as part of normal practice. Martin Weller explains that when institutions invest significant resources (e.g., money, expertise, processes, training) to embed a technology into their operations, such software becomes integral to operations, resulting in software sedimentation—a term he borrowed from Jaron Lanier.^Footnote11 He notes that an unintended pedagogical outcome of such sedimentation is "tool-focused solutionism," which encourages individuals to look to particular technologies for solutions (e.g., "How can [SurveillanceTool] aid integrity in our classrooms?"), rather than process-oriented and practice-oriented solutionism (e.g., "How can we adjust our pedagogies, assessments, and relationships with students to aid integrity in our classrooms?"). Importantly, sedimentation also makes it very difficult for institutions to extract themselves from particular technologies. Simply canceling annual contracts is not enough, since the internal processes designed to support such technologies amount to long-term investments in them (e.g., training hundreds of faculty to use particular tools means that an institution is now invested in that tool in numerous ways).

Given the prevalent nature of these technologies as indicated by the results of this study, we urge individual faculty and institutional leaders to consider the long-term implications of these adoptions in response to the challenges of the COVID-19 pandemic and the very real possibilities of long-term frustrations. We also encourage further research to investigate the ways in which—and the degree to which—these technologies are being used. Data for this research will likely need to come from the higher education institutions, since the proctoring software industry, similar to the broader educational technology industry, lacks transparency in this respect. For instance, even basic metrics (e.g., lists of institutional adoptions) are unavailable on proctoring company websites, which also do not provide access to third-party researchers who might want to evaluate the company claims and technologies.

Our review is an early look. Our findings should provide ongoing context for debates around student surveillance, security, rights, and privacy and should offer added urgency for institutional leaders and policymakers to take these matters seriously. As proctoring tools and services are being adopted at institutions serving millions of students, the higher education community needs to responsibly grapple with the implications of this use, reflect on how these shifts respond to actual needs, evaluate the costs of these shifts (in terms of money, privacy, and distrust toward students), and consider whether adopting such tools so quickly and broadly is the best solution to the problems we are trying to solve.

Notes

1. Jason Kelley, "Students Are Pushing Back against Proctoring Surveillance Apps," Electronic Frontier Foundation, September 25, 2020; "Students Revolt over Remote Test Monitoring," Washington Post, November 12, 2020; Avi Asher-Schapiro, "'Unfair Surveillance'? Online Exam Software Sparks Global Student Revolt," Reuters, November 10, 2020. Jump back to footnote 1 in the text.↩
2. See Colleen Flaherty, "Big Proctor," Inside Higher Ed, May 11, 2020. Jump back to footnote 2 in the text.↩
3. Steve Kolowich, "Behind the Webcam's Watchful Eye, Online Proctoring Takes Hold," Chronicle of Higher Education, April 15, 2013. Jump back to footnote 3 in the text.↩
4. Bonnie Stewart, "Why Higher Ed Needs Data Ethics," Inside Higher Ed, November 10, 2020; Albert Fox Cahn, Caroline Magee, Eleni Manis, and Naz Akyol, "Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services," November 11, 2020; George Veletsianos and Shandell Houlden, "Radical Flexibility and Relationality as Responses to Education in Times of Crisis," Postdigital Science and Education 2, no. 3 (October 2020). Jump back to footnote 4 in the text.↩
5. Shea Swauger, "The Calculus of Harm," Shea Swauger (blog), October 31, 2020. Jump back to footnote 5 in the text.↩
6. "Students Revolt over Remote Test Monitoring," Washington Post, November 12, 2020. Jump back to footnote 6 in the text.↩
7. Paul Sawers, "Examity Raises $90 Million for Online Proctoring Platform That Thwarts Exam Cheats," VentureBeat, April 30, 2019; ProctorU website (accessed February 15, 2021. Jump back to footnote 7 in the text.↩
8. Susan Grajek, "EDUCAUSE COVID-19 QuickPoll Results: Grading and Proctoring," EDUCAUSE Review, April 10, 2020. Jump back to footnote 8 in the text.↩
9. "Invigilating Online Exams at UVic," Division of Learning and Teaching Support and Innovation, University of Victoria, October 21, 2020. Jump back to footnote 9 in the text.↩
10. See Jeffrey R. Young, "Pushback Is Growing Against Automated Proctoring Services, But So Is Their Use," EdSurge, November 13, 2020. Jump back to footnote 10 in the text.↩
11. Martin Weller, 25 Years of Ed Tech (Edmonton, Alberta: Athabasca University Press, 2020). See also Jaron Lanier, "The Complexity Ceiling," in John Brockman, ed., The Next Fifty Years: Science in the First Half of the Twenty-First Century (New York: Vintage Books, 2002). Jump back to footnote 11 in the text.↩

Royce Kimmons is an Assistant Professor of Instructional Psychology and Technology at Brigham Young University.

George Veletsianos is a Professor at Royal Roads University in Victoria, BC Canada. He holds the Canada Research Chair in Innovative Learning and Technology and the Commonwealth of Learning Chair in Flexible Learning.