Scholarship beyond the Word

We live in a world of intensifying visuality. Something as mundane as a ride in a taxicab brings multiple windows into play, from the view that zips by outside to the meter that chirps advertisements to the smartphones that we obsessively stroke as we ride. A walk through any downtown area is mediated by screens large and small, public and private. Although the mediation of daily life has been in play since the early days of photography and cinema, this process accelerated with the advent of the World Wide Web. If the early web tended to privilege text, today's web is deeply multimodal—full of image, sound, and video. The online photo-sharing site Flickr hosts billions of images, with roughly 1.83 million images uploaded daily in 2014; Instagram, launched in 2010, now surpasses 70 million photo uploads a day. An average of 300 hours of video is uploaded to YouTube every minute. These user-generated sites are joined online by many other institutional websites and archives hosted by libraries, museums, and higher education institutions, sites that are increasingly opening up their visual content to the public. From the growing resources of the Digital Public Library of America (DPLA) to the New York Public Library's vast Digital Collections to the 109,729 hours of video testimony in the Visual History Archive at the University of Southern California (USC) Shoah Foundation's Institute for Visual History and Education, expansive online resources push far beyond text.

Scholarship has not kept pace with this explosion of audiovisual material. Like the early web, scholarship is still largely textual. There are many reasons for this, of course. Words carry arguments well. The systems that have developed in support of scholarly publishing depended on and reinforced the circulation of print. Yet signs of change are afoot. College and university researchers are increasingly interested in creating scholarship that uses visual materials in dynamic ways. From art history and cinema studies to many areas of science, scholars are producing digital scholarship that includes sound, still and moving images, interactive data, and more. For instance, in partnership with MediaCommons and the Society for Cinema and Media Studies, [in]Transition publishes peer-reviewed, open-access video essays. The College Art Association's caa.reviews published an online exhibition review [http://scalar.usc.edu/hc/caa.reviews-bernini/index] that included a video walk-through of a major Bernini exhibition along with scholarly commentary, additional images, floor plans, and other materials. University presses are also expanding into multimedia content, although this work typically requires a good deal of additional labor from authors. Examples include the Civic Media Project, an expansive online compendium of case studies edited by Eric Gordon and Paul Mihailidis; a companion print volume of additional materials will follow from MIT Press.

Even though this growing array of multimedia scholarship takes many forms, posing many challenges for stewardship and preservation, these practices collectively point the way toward new directions for scholarly publishing. Consider scholars who are interested in conducting research in an archive such as Shoah's. Traditionally, they would study the videos, develop analyses, and present their findings in print. These publications would be read within a fairly small scholarly community. We are now developing digital platforms that allow us to present such research in richer online formats that include the testimony clips. This also means that we can ask new types of research questions and reach new audiences. We might imagine the scholars' role as providing a kind of guided tour of a selection from the archive using their analytic skills and historical knowledge. Another benefit of working online within an academic context is that several scholars can work together on a particular theme or set of testimonies, providing their different interpretations. In addition, the research is not static: other communities of users—such as students, archivists, or interested members of the public—can provide comments or add new clips from the testimonies. This, in turn, creates new forms of online publication and new paths through the archive. The line between edited collection and archive might blur, as in Performing Archive: Curtis + "the vanishing race", a jointly authored project on the photographs of Edward S. Curtis. Readers of this piece can follow one of many pathways to learn about the photographer and his work or can instead browse collections of photographs and other materials in extensive media galleries comprising nearly 2,500 digital objects.

One can imagine a future in which users of the Shoah archive might not only watch the testimonies but also view the scholars' research, which could be accessible from within the collection itself. For example, if a scholar has studied a group of testimonies about, say, hunger in the ghetto, whenever one of those testimonies is watched, the archive will be able to tell the viewer that there is interesting research connected to it. That way, archive and analysis would live together within the digital realm, each enriching the other.

Or consider a scholar who wants to demonstrate the effect of facial expression on how we understand the spoken word. She would investigate meaning in the testimonies that exceeds a text transcript. She could use the video testimony and include the nuances of the visual directly in an online publication. This will allow readers to engage the face and voice of the survivor through the video while reading (or hearing) the research analysis. The flick of an eye, the sweep of a hand, and the timbre of a voice would bring the analysis to life, communicating beyond the word. Projects such as this point toward new possibilities for visual scholarship, making rich use of the audiovisual nature of many online resources. This type of scholarship will hold scholarly analysis and visual evidence much more closely together. It will also embrace the human elements of testimony, underscoring that the entire person remembers history and illustrating the power of video to capture that embodied experience. Visual archives, including Shoah's Visual History Archive, are eager for scholars to engage the testimonies as audiovisual material, but very little support exists for scholars to undertake such research. For this reason, in close partnership with several institutions and with support from the Andrew W. Mellon Foundation and the National Endowment for the Humanities, the Alliance for Networking Visual Culture (ANVC) is working with archives, colleges/universities, and publishers to address the challenge. ANVC aims to enable scholars to work more closely with online archival materials, creating new forms of born-digital analysis and publication.

One outcome of ANVC's work is an easy-to-use software platform called Scalar, which is being developed at USC. Scalar allows scholars to author, with relative ease, long-form multimedia projects that incorporate a variety of digital materials while also connecting to digital collections, utilizing built-in visualizations, exploring nonlinearity, supporting customization, and more. This work is often undertaken with leading university presses, scholarly societies, and humanities centers. The platform has also been taken up for a number of pedagogical projects, many of which work closely with archival materials. ANVC is currently in a prototyping phase for Scalar, working with a select group of institutions to test our ideas and to develop appropriate infrastructure. In addition to Shoah's Visual History Archive, our archival partners include a video database of performance art addressing cultural memory (the Hemispheric Institute Digital Video Library), the nonprofit Internet Archive, and the USC-based research and teaching archive Critical Commons, among others. Each of these archives differs from the others, but across their differences, a great deal is being discovered about how scholars might best use still and moving images within new forms of scholarly writing.

Researchers are excited to engage archival materials in digital form, and those scholars committed to analyzing video or still images see particular advantages in being able to work "close up" with these materials, allowing annotation and detailed interpretation. They see the distinct gains to be had from being able to "pull" these materials directly into new authoring platforms, especially when they can draw material from multiple sources. Nonetheless, they are also concerned about how this work with archival materials will circulate and how it will count within their institutions when they are evaluated for promotion and tenure.

There are several tensions in the academic community around digital scholarship. University presses want to undertake this work, but they are operating on very limited budgets that make experimentation difficult. Scholars value the ability to use digital media in their research, but they often need support to author in new digital platforms. Archives and libraries want to provide such support, but they also want to work in tandem with other archives and libraries so that each need not reinvent the wheel. College and university administrators worry about how best to evaluate emerging online research formats that are often deeply collaborative and that also circulate to new audiences outside the academy.

Over time, the academic community has carefully developed guidelines for assessing more traditional forms of research, and digital scholarship sometimes challenges these methods. Rather than working in a piecemeal fashion, ANVC aims to address these many tensions as part of the same process, building both a rich technological and a nuanced human infrastructure for new modes of scholarly communication. Working with the tools of the digital era is exciting, but the digital should be a means to an end, not an end in and of itself. At ANVC, our larger goal is to deploy the digital in ways that let us bring our richly mediated world more palpably to life while fomenting new forms of analysis and pedagogy.

Tara McPherson is associate professor of critical studies at the University of Southern California's School of Cinematic Arts.