What do you do when the world outside the campus seems much more interesting to language students than what you have to offer in your functional but outdated language lab? How do you engage jaded students in such a way that they leave your class wanting to know more about the language and culture of the people they’ve been studying? How do you know whether keeping up with the latest in digital technology will be worth the price and the effort?
Three years ago, the language lab at the University of Wisconsin–Madison encountered the same challenge many campuses face. Older analog technologies were getting harder—in some cases impossible—to maintain, thus making the move to digital technologies a necessity. A typical solution would have been to digitize existing materials and substitute digital equipment (computers) for analog equivalents (tape recorders), all the while continuing to maintain a fixed-location language lab.
Instead, our college administration challenged us to explore more innovative ways to leap into the digital world. Fortunately, we obtained a substantial equipment grant from HP that had as its focus the implementation of wireless (networking) and mobile (portable equipment) technologies.
Crafting the Proposal
At first, we had a difficult time getting our minds around the idea of mobile language learning, but the more we examined the possibilities, the more excited we became. We put together a proposal we called "Creating a Mobile Language Learning Environment." Unlike many instructional technology projects that focus on one particular course or activity, ours proposed to address the needs of foreign language instructors, in and out of the classroom, and with or without the traditional language lab space. The resulting diversity of subprojects makes difficult a comprehensive description of "the project." We wanted to change the whole learning environment.
The proposal called for a range of initiatives. First, we planned to replace two of our traditional language labs with laptop-based labs, writing our own language-learning and language-testing software. Second, we planned to implement the use of wireless handheld and laptop devices for use in classrooms. We intended to make it easier for instructors and students to access the Internet and to use digital materials in the classroom, without having to take up valuable class time by going to a fixed-location lab. Third, we planned to make digital materials available to all students through our online virtual digital language lab. With wireless coverage throughout our building, and by taking advantage of campus wireless coverage in many other locations, students could use handheld and laptop computers at any time and any place. Finally, we planned to determine whether the use of mobile technology helped further the cause of foreign language learning.
Applying the Grant
We were actually somewhat surprised to receive the grant, primarily because our proposal focused on humanities courses, while HP typically provides equipment grants to engineering and computer science projects. We have learned since that it is precisely because of our humanities background and our ability to affect literally thousands of students that HP viewed our proposal favorably. In fact, subsequently HP invited us to submit two proposals extending our grant, and we received both extensions. The grants consisted primarily of equipment, with some cash for student programmers and minor incidentals.
The fixed-lab parts of our project proved to be the easiest to implement, albeit not entirely without challenges. Our fixed-location language labs let learners listen to native speech, record their own responses, and compare their recordings with the originals. This method, which has proven successful over many years, contrasts with a simpler approach where students merely listen to native speech, with no opportunity to speak themselves. Although turnkey digital language labs on the market accomplish these more-complex tasks in a prepackaged combination of hardware and software, they seemed extremely expensive to us and often were tied to specific vendor equipment and software, making the systems useful for little else. For example, outfitting a new proprietary digital language lab would have cost us nearly $180,000 for 30 student stations.
Instead we chose to use normal laptop computers (HP Pavilion and Compaq Evo) and write our own software to replicate traditional language-lab functions. These computers could also be used for other common computing activities, such as Internet access or word processing.
Further, we used one of our labs to test English as a Second Language (ESL) students for speaking proficiency. Because international students must pass this test before the university permits them to become graduate teaching assistants, the pressure to perform well is intense. We needed to create a secure environment where the technology would not be so difficult to use that it would adversely affect student performance. The new technology also needed to be as thoroughly reliable as tape recording technology.
The challenging yet fun part of any research grant is trying to determine what you need and how you’ll use it, and then making adjustments along the way as you find out what works and what doesn’t. Such was the case with the mobile parts of the project. Although we consulted with faculty and brought our best instructional technology experience to bear in selecting equipment and software, we soon discovered that while some things worked as planned, some never got off the ground, and several new and unexpected uses and results came about.
We acquired four different types of mobile equipment with the intention of using them in our foreign language classrooms. Because of the size of our university—several thousand students were enrolled in beginning language courses—we knew we could not provide devices to students for long-term loan. Instead we focused on the classroom, recognizing that we could extend beyond the classroom somewhat because some students had begun to bring their laptops and handheld computers to campus.
The first equipment we acquired was the Jornada 728, a "clamshell" handheld computer that featured a horizontal screen and a small but full-featured keyboard. Faculty and students loved this easy-to-use device, although we were concerned about the cost (nearly $1,000 each). Unfortunately, about two weeks after we received these units, HP stopped producing them, and we could not find an equivalent replacement.
The second type of equipment acquired was the Jornada 568, which looks like a Palm PDA but uses the Windows PocketPC operating system. Perhaps the largest challenge in using this device is its form factor (a small, portrait-oriented screen) and the lack of keyboard input. We purchased add-on keyboards, which have been used heavily, and we worked closely with faculty to design Web pages or instructional activities suitable for the small screen. Originally we intended to write listening/recording software for these devices despite their relatively weak recording capability, but we found students and faculty had little interest or need for this functionality. Instead, typed chat sessions and Web access are the primary activities conducted on these handhelds.
All our early Jornada and iPaq models also required adapters and PCMCIA or Compact Flash wireless networking cards, which added significantly to the overall cost. For example, a typically configured Jornada with adapter, wireless card, and keyboard cost $200–$300 more than the base model.
Subsequently, we purchased iPaq 3950 handhelds, but the keyboards available for these devices handle foreign character input poorly. Most recently we acquired the iPaq 4350, which includes internal wireless capability and an integrated thumb keyboard. These work well, since students don’t have to set up external keyboards. Generally, students have had little trouble adapting to the thumb keyboard. A totally integrated hardware solution also reduces the cost of the devices to just over $400 each.
Third, we acquired several laptop computers that we made available on a mobile cart. While these computers support the broadest range of PC activities, most of our classrooms are equipped with tablet-arm chairs barely large enough to hold a laptop. Thus, using full-sized laptop computers has been difficult.
Finally, we acquired several tablet PCs, which weren’t available for the original grant. They offer a nice compromise in terms of size, and we’re still exploring ways to take advantage of the Tablet PC interface for language learning. For example, we discovered that the Tablet’s journal software may work well in teaching students to write non-Roman scripts (Chinese, Japanese, Urdu, and so forth).
The principal lesson learned is that equipment decisions are rarely completely satisfactory, especially when dealing with developing, sometimes non-mainstream technologies such as handheld PCs. These require constant adjustments of expectations and objectives as available devices and features change.
Critical to our mobility efforts was creating a wireless environment for the entire classroom wing of our building, which includes more than 60 classrooms. Although our campus has begun to move in the direction of wireless computing, with this grant we were at least two years ahead of the campus implementation. Thus we had little expertise to call on, and we learned a great deal about making sure wireless signals are strong enough for transparent Internet connections. Weak or dropped signals make classroom activities a real adventure, and early faculty adopters deserve thanks for recognizing the research aspect of this grant and sticking with activities even when things didn’t go particularly smoothly. We realize, however, that reliability and transparency are critically important for continued success.
Finally, the grant also provided funding for hiring undergraduate student programmers. We intended to use student employees in creating our special lab and testing software.
Implementing Our Goals
Our first step, even before receiving the equipment, was determining how to get faculty to buy into the program. Fortunately, we already had substantial experience in working with faculty in exploring innovative uses of technology. Coinciding with the beginning of the HP grant, and with modest funding from the college, we created a grant program that invited faculty to explore the use of mobile technology in their courses. Our concerns about faculty buy-in quickly dissipated as the number of quality faculty proposals outpaced our ability to fund them.
During the fall 2002 semester we began implementing 30 faculty projects in language courses ranging from Spanish and French to Norwegian and Ojibwe. Some projects employed relatively traditional uses of computing and Internet access. Others involved activities not previously tried: online chats, instant messaging with students from another country, use of a custom database of medieval Spanish texts to discover words or phrases in context, and online and real-time collaborative writing using wikis (editable Web pages).
Our second step was to deploy the equipment. We knew up front that the grant would require matching investments on our part. Fortunately, we obtained additional funding from the college for furniture, electrical and network wiring, security cables, and more.
We also began our software development projects. Our first objective was to find inexpensive but functional software that would meet some of our needs. For the labs, we decided to use commercial lab management software that cost approximately $35 per student seat. This software enabled instructors to monitor and communicate with students, distribute and collect student exercises and tests, and control student access to applications and Internet sites. Then, in addition to traditional application software such as word processing and Internet browsers, we looked for open source software to meet instructors’ requests, such as instant messaging and Internet chat software.
Our second objective was to write software to handle specific language-lab tasks, such as audio-comparative recordings. We publicly stated our intention to provide the results as open source software. After assembling a team of student programmers, we began defining objectives. Unfortunately, we soon learned that working with student programmers presented some challenges.
First, they’re students—school seems to get in the way of their programming projects. Second, the good ones tend to be in demand and gravitate toward new and perhaps more interesting projects, not always finishing their current projects. As a result, by the time we were ready to move forward, the software for the labs still had not been written. Fortunately, one of our full-time staff was able to quickly write a piece of software to handle the secure testing functions for our ESL project. In addition, in the preceding few months similar open source language-lab-type software had begun to appear, and the need to write our own became less important. By spring 2003, we were ready to move forward, and faculty projects began in earnest.
An Early Experience
We anticipated many learning opportunities as we made the leap from analog to digital and from fixed location to mobile equipment. Nevertheless, some of the things we learned, both positive and negative, surprised us.
Our first experience also provided one of our initial key findings. Our Tagalog (Filipino) instructor wanted her students to use online Tagalog quizzes that were developed and run over the Internet by Northern Illinois University. None of our regular labs was available, so we decided to try using our Jornada 728 handheld computers for the exercise. We wheeled a cart full of the devices into her classroom, plugged in a wireless access point, and jumped right in. Here we learned there are three stages to using new technologies for instruction, typified by student comments:
- "How does this darned thing work?"
- "Hey, this is way cool!"
- "Hey, we didn’t learn that in class!" (They were taking a quiz.)
In other words, we found that very quickly, usually within one or two sessions, students and instructors passed through the first two stages (apprehension, then excitement) with the focus on the technology to the third stage, where they focused on the learning activity and materials without regard for the technology. This experience held true for nearly every faculty project we helped implement over the next two years.
Assessing Objectives
A primary objective in obtaining the grant was to discover the impact technology might have in foreign language learning. However, we knew that trying to measure actual performance skills would require more resources than we had. For that reason, we chose to focus instead on student and instructor attitudes. Our proposal stated:
Part of our challenge in assessment was that our projects and the activities within those projects were so diverse. For example, a Spanish Medieval Literature course used a database of old texts to assist students in researching various topics, while beginning French students used Pocket PCs to practice communicating using written chat sessions. A Japanese instructor used Pocket PCs to let students view live Japanese TV programs, while a linguistics professor used Pocket PCs and memory cards to let students practice oral Ojibwe exercises.
Nevertheless, we tried to create an assessment tool that would measure student and faculty satisfaction with the technology and whether it enhanced or detracted from the language learning experience. We were fortunate to collaborate with another HP grant recipient, Toni Doolen and her staff at Oregon State University, who already had some experience in conducting a similar survey, albeit with engineering students. With their help we devised a prototype survey.
In the spring of 2003 we conducted a pilot survey to test our assessment instrument. Based on that experience, we drastically reduced the number of questions and focused on the information we really wanted. We also learned to administer the survey so as not to interfere with classroom activities. For example, students did not like taking a paper-based survey in class instead of having a final review in preparation for final exams. Thus we switched to online surveys that students completed outside of class.
In the fall of 2003 we implemented the online survey in as many courses as possible of those using mobile technology in the classroom. Although we did not survey students using fixed location labs, anecdotal reports from instructors indicated that attitudes of students using these facilities paralleled those of students involved with the mobile technologies.
The first survey was for attitudes prior to using the technology, while the second was a post-use survey. Thanks to the diligence of our staff, we generated a fairly high response rate. Total participation in the first survey of the 667 students enrolled was 463 (67 percent), and in the second, 360 (51 percent). Some 265 students participated in both surveys, providing a substantial measure of before and after attitudes.
Again, the primary things we wanted to learn had to do with student attitudes toward technology, their attitudes toward language, and how they perceived technology to fit with learning languages, literature, and culture. Figures 1 and 2 show typical questions.
Click image for larger view.
Click image for larger view.
Our preliminary examination of the results (see Table 1) showed improvement in all areas where the differences were statistically significant on a scale of 1 (negative) to 7 (positive).
Click image for larger view.
When we viewed student perceptions on the value of technology as it applies to specific language-learning activities, however, we were surprised to find that students felt strongly that the PDA and laptop devices helped them with their reading and writing activities, but not as much for listening and speaking activities. Figure 3 shows the results for a survey of all the courses in response to the question, "What aspects of your learning were aided by use of the PDA and laptop?"
Click image for larger view.
Not until we examined the types of activities conducted in the surveyed courses did we begin to understand why students felt the way they did. A large majority of those taking the survey were beginning students in French and Italian courses that conducted classroom chats (19 sections of 20 to 25 students each), implemented primarily as a reading and writing activity. Fewer courses used listening or speaking/recording exercises. A few other courses focused on cultural activities.
We then separated the results into three groups: courses primarily doing chats (19 sections of French and Italian); the remaining language courses, which conducted more traditional classroom language learning activities (7 sections including Danish, Filipino, French, Japanese, Norwegian, and Spanish); and the Comparative Literature courses (9 sections), which had problems quite apart from language learning issues (described later). We believe the results show high levels of student satisfaction with the mobile technology used in the language learning courses, with positive indications in nearly all categories that corresponded with the specific type of activity. For example, the students in classes using chat (see Figure 4) indicated high levels of satisfaction with the vocabulary, grammar, reading, and writing activities. Students in the other, non-chat, language courses (see Figure 5) rated vocabulary, listening, speaking, and cultural activities most highly.
Click image for larger view.
Click image for larger view.
The Comparative Literature courses require a separate explanation. The activity in those courses involved a unique method of collaborative commenting where students viewed a film clip and then, before discussing it as a class, wrote individual comments and submitted them to a Web site that collected and displayed them. The instructor could display the collective commentary to the class, so each student had to think critically, not just sit back and let others in the class do the thinking. Some instructors also used typing chats to have pairs of students comment to each other.
Although the course professor had a clear vision of what he wanted to accomplish, not all of his teaching assistants or discussion leaders did. Their inconsistent attitudes affected their students’ attitudes toward the technology, and the survey results were largely negative (see Figure 6). In addition, the chat activity seemed pedagogically pointless. Students complained that it was "stupid" to write to each other in pairs, when they could more easily communicate orally. To add to the problems, instructors provided little or no preparation for either type of activity.
Click image for larger view.
From this experience and other responses (discussed later) we learned how critical it is to design appropriate pedagogical activities and to get instructor and student buy-in. Both are necessary to ensure that implementation of this kind of technology in a classroom setting can succeed.
Additional Survey Results
In addition to measuring success in meeting language-specific objectives, we also collected information about the respondents and their attitudes toward the instructors and the courses in general. We thought we might find correlations between certain instructors and positive attitudes, and vice versa.
Conventional wisdom holds that younger instructors, including teaching assistants, show greater enthusiasm for technology and integrate it more readily into their courses. Our survey indicated something quite different. Student satisfaction with instructors was significantly higher for those 40 and older, and significantly lower for those 18 to 39. Those with 10 years’ or more teaching experience scored much higher than those with less than 10 years’ experience. In addition, graduate teaching assistants scored significantly lower than all three ranks of professors.
Another telling result was that instructors who used technology more than five times in their classes scored four times higher in satisfaction with technology than those who barely used technology or used it only as required (fewer than five times).
Have we disproved conventional wisdom? Probably not. We surveyed instructors, not only with objective short-answer questions but also with open-ended narrative responses. Their answers gave us insight into the above correlations. In nearly all cases, the grant participants were older and more experienced not only in their subject matter but also with instructional technology and were enthusiastic about it. Graduate assistants often were required to conduct experimental activities, regardless of their previous experience with or interest in technology. Further, less experienced instructors often received little guidance in how to prepare students to take full advantage of the learning activities conducted using technology. For some of them, adding these activities was quite stressful, occasionally resulting in negative, disapproving attitudes toward the technology when communicating with students.
When comparing and correlating student attitudes with instructor attitudes, therefore, we clearly saw that instructor buy-in is a critical factor in determining the success or failure of technology activities in the classroom. This assumes, of course, that the instructional activity itself is pedagogically sound.
Comments from a Norwegian-language professor whose students responded favorably to the use of technology illustrate how a well-planned strategy coupled with instructor enthusiasm seem to be the key ingredients for success. The survey results from her class back up her own assessment:
Other Lessons Learned
Although the survey provided compelling data on whether we had achieved our primary objectives, we also learned many other things about implementing a wireless language-learning environment, some of which we never anticipated. Going into the project, we had certain notions as to which kinds of learning activities instructors would implement. It surprised us that traditional listening and viewing activities, even on the Web, received relatively little attention. On the other hand, a very popular activity that never crossed our minds was the use of handhelds to have students type (chat) with each other in the classroom.
In retrospect, it seems evident that some key technical issues inhibited traditional uses of audio and video:
- Limited wireless bandwidth made it difficult to stream video to the handhelds.
- Handheld-based player/recorder software for language learning is virtually nonexistent.
- The built-in Windows media player is only a limited version of the player found on full version PCs.
- Audio input methods for handhelds are quite primitive (the devices have no microphone input jack).
A Japanese professor, for example, wanted her students to access a Japanese Web site to listen to authentic news broadcasts. While they could view video on one or two devices, when a whole class logged on, the video speed and quality deteriorated rapidly. We devised a workaround where we placed digitized video clips on secure digital (SD) memory cards and made these available to students to use with their handheld PCs.
One of the most surprising lessons we learned was the sheer amount of logistical detail required to implement wireless technology in the classroom. When all sections of French 101 wanted to conduct chats, for example, all on the same day and one right after another, we simply couldn’t recharge the devices fast enough. Thus, we learned the need to spread out the scheduling of devices.
We were also disappointed when what should have been a simple, 15-minute, in-class exercise turned out to take the entire class period, due to student unfamiliarity with the handheld devices and sporadic problems with wireless networking connections. We soon found ourselves sending two of our staff members to each classroom to set up the devices and to make sure students could connect to the network. Some instructors even began requesting use of our fixed labs instead of hassling with the setup and use of handhelds in the classroom.
Since then we have taken steps to minimize these problems. We created relatively simple student tutorials intended to help them become familiar with the device they would be using, how to connect to a strong wireless signal, and how to use any specific application required for their class. Instructors require students to go to the language lab and take these tutorials prior to the class in which the devices will be used. With this advance student training, we hope to help instructors realize that they now can call on their students to help set up their own equipment in the classroom and assist their fellow students in getting connected and in using wireless applications.
We also discovered that not all instructors understood the procedures themselves, and thus they were not adequately preparing students for the specific learning activity they would be using. We have created brief training materials that help instructors learn how to use the devices and that assist them in more proper integration of the activities with their learning objectives.
Another important lesson learned involved the size of the PDA screen. For example, a professor of a French culture course used an exercise that required students to compare two cultural scenes or paintings side by side. Having to scroll side to side and not being able to view both at the same time was a real problem. The solution was simple: arrange the pictures vertically on the Web page, enabling them to view both pictures simultaneously. We also created our own narrow, vertically oriented Web page with links grouped by the courses that use them, making it easier for students to jump to the appropriate Web locations. Surprisingly, video material works quite well in the PDA form factor, since player software enables users to rotate the video 90 degrees.
Of special concern for our language learning environment is the ability to input foreign language characters quickly and naturally. Doing so with the soft (on-screen) keyboards is cumbersome at best. What we have learned has involved a step forward, a step backward, and so on, as we experimented with different keyboard solutions, the latest being iPaqs with integrated thumb keyboards, which students seem to quickly master and which provide reasonable foreign language support.
We feel fortunate to have had this opportunity, but we definitely feel we’ve been on the "bleeding edge" of learning about using mobile technology for instruction. Essentially, few standards exist, and we had to experiment all along the way. Further, we had to implement a wide array of projects, both in terms of dramatically different types of hardware and specialized software applications. It was also quite frustrating to figure out how to deploy a given device, only to find out when we tried to purchase additional units that the company had already moved on to a newer model that did not always improve on what we needed.
Finally, we have to assess whether the costs were justified. Grants can sometimes be like the Trojan Horse: you receive what appears to be a wonderful gift, but with serious hidden costs and impacts. In our case, we are convinced that the costs for the wireless infrastructure were well worth our investment. In terms of the devices used to access that network, although laptops are a bit more expensive than desktop units, in lab and classroom settings the space savings were also well worth the costs. And handhelds, great when desk space is limited, are still rather expensive—nearly $500 each, including wireless capability.
All in all, we wouldn’t trade this experience for an easier, more traditional route. What we have learned, coupled with genuine faculty and student enthusiasm and successes, adds up to a very satisfying project.
Scalability and Sustainability
Although the grant allowed us to implement a successful wireless environment in one building, we wonder whether we can implement the use of mobile language-learning technology more broadly on our campus. It seems clear that the wireless infrastructure is robust enough for more widespread use. The cost of devices will make it difficult to provide more of them for students, however. At some point many students might have their own handhelds, or even Internet-capable wireless phones, but we can’t expect that yet.
An even stickier issue is whether we can sustain the momentum we have created. How many faculty will bother to implement this technology in their classrooms without the incentives we provided to these initial rounds of projects? Our instructor surveys indicate that generally they are pleased and encouraged by the potential of these mobile devices. We are also happy to note that many instructors have continued with their projects even after supporting funds expired. Although it might be difficult to bring in new instructors without incentive programs, we are optimistic about the potential of building on our successes.
Extensibility
By this point you might be wondering, "Could we do this?" If you can come up with the money, you could. The costs vary widely, depending on the number of students and courses you want to support, the types of devices you plan to use, and how much infrastructure (networking, facilities, and technical support) are already in place. For example, a simple cart of wireless iPaqs for 30 students, along with a wireless access point, could cost less than $15,000. A similar setup using Tablet PCs (nearly twice as expensive as laptop computers) could easily exceed $50,000.
We hope the data and the experiences provided in this article are compelling enough to help you argue for such funding, either from your own institution or from another grant-funding agency. You might also be able to capitalize on existing devices on your campus, perhaps those already owned by your students.
The equipment and software, however, are only part of the solution. Depending on the size of your institution, you’ll also need good staff to make this work. You probably can’t add more people, but you can make sure instructors and students get involved in deployment and implementation.
The Bottom Line
What we can offer through our language lab’s extensive collection of instructional materials can never fully match what students can find on the Internet. However, when we create an environment that enables students to go to original sources, supplementing and complementing our language-learning-specific resources, then the world itself becomes a language lab without walls. When students get a chance to hold the world in their hands, language learning becomes more exciting, more interesting, and more relevant.