The Pennsylvania State University (Penn State) has actively promoted e-portfolio use among undergraduate students at its University Park campus since May 2002. The primary objective is to foster consensus among faculty members and students about the nature and potential benefits of portfolio practice. Guiding this effort is the vision of a university culture centered on students’ evidence of, and reflections upon, their curricular and cocurricular achievements.
Since 2001 Penn State has provided a faculty-centered enterprise course management system through which instructors at all 24 campus locations can deliver course content, conduct quizzes, and manage course communications. The university does not yet provide a comparable student-centered learning management system to help students collect, select, reflect upon, and publish evidence of their learning and personal growth, however. A secondary objective of the promotional effort described here is to help create the conditions within which the functional requirements of such a system can be specified reliably.
In lieu of such a student-centered learning management system, Penn State currently provides a free 200 megabyte personal Web space account to any student on request. To use these accounts, students need to master transferable Web publishing skills such as HTML document editing and secure file transfer. No standardized e-portfolio template is imposed; individual students and academic programs are encouraged to develop e-portfolio formats that best suit their particular goals.
In this article we review the results of several research studies focused on Penn State’s efforts to promote e-portfolio development among undergraduate students. We use these results as a basis for considering the value and viability of a generic Web publishing approach as an alternative, or prelude to, an enterprise learning management system that enables students to create and maintain e-portfolios.
In the two years since the start of the promotional effort, the proportion of the approximately 32,000 undergraduates at Penn State’s University Park campus who have published evidence of their academic work in personal Web space accounts has increased from less than 10 percent to nearly 30 percent. The proportion of personal Web space accounts that include the reflective commentary that characterizes e-portfolios has increased from approximately 1 percent to 9 percent. Survey results indicate that Penn State’s Web publishing approach is satisfactory to a substantial majority of these students and that the existing support infrastructure is adequate. We estimate that additional support infrastructure will be required, however, as the rate of participation approaches and exceeds about 50 percent.
Background
As at many colleges and universities, individual faculty initiatives related to student e-portfolios took place at Penn State through the late 1990s.1 In an attempt to bring coherence to these disparate activities, the John A. Dutton e-Education Institute of Penn State’s College of Earth and Mineral Sciences hosted faculty seminars in 2000 and 2001 that focused on the theory and practice of portfolio assessment.
Seminar participants acknowledged that student participation in e-portfolio activities depends on faculty perceptions of the value of such activities. Very few students are likely to create and maintain e-portfolios voluntarily; most will do so only when required by faculty members as part of a class assignment or as a degree requirement. A survey of Penn State faculty attitudes about instructional computing2 revealed that while more than 90 percent of respondents expected students to use the Web for research and communication, only 7 percent expected their students to create Web pages. Penn State faculty members seemed content that their students be consumers, but not necessarily producers, of Web-based information.
An early Dutton Institute survey of student Web usage confirmed the impact of low faculty expectations: as of December 2001, fewer than 10 percent of Penn State undergraduates (at University Park and other campuses) had published examples of their academic work in their personal Web space accounts.3 More encouraging was a finding of a 2002 survey of Penn State students in which "creating Web pages" was the computing activity cited most frequently as desirable.4 This disjuncture between faculty and student perceptions of publishing on the Web is one of the problems that our e-portfolio initiative aims to address.
Participants in the e-Education Seminars on e-portfolio assessment issued a report of their findings and recommendations in February 2002. The report urged support for an "energetic and sustained promotional effort" to raise awareness of the potential benefits of e-portfolio practice.5 "To encourage instructors to refine their expectations about how the Web can contribute to learning," the report advised, "the university needs to create pedagogical and support infrastructures that complement the existing technological infrastructure" (that is, personal Web space accounts).6 Specifically, seminar participants recommended development of a new, centrally maintained Web site for e-portfolio developers (http://portfolio.psu.edu) that includes the following features:
- A new default home page for personal Web space accounts that includes a link to <http://portfolio.psu.edu/>
- A gallery of best practices in student e-portfolios
- An online tutorial and self-assessments on how to create an effective resume
- An online tutorial and self-assessments on file management
- A set of online tutorials and self-assessments on how to capture evidence in digital formats compatible with Web browsers
- A set of online tutorials and self-assessments on reflective writing
- A set of online tutorials and self-assessments on privacy and free speech
In addition, the report encouraged:
- First-year seminar instructors to require students to activate their personal Web space accounts, to develop resumes, and to link their resumes to their personal home pages.
- The development and offering of practicum courses in professional e-portfolio development for second-, third-, and fourth-year students.
- Instructors to request technology learning assistants (coordinated by Information Technology Services) to help develop and test e-portfolio-related assignments and to recruit undergraduate teaching interns to provide peer technical support.
- Academic advisors to consult advisees’ personal Web space accounts and to advise that students use their accounts to document and reflect upon their university experiences.
In May 2002 the Teaching and Learning with Technology group within Penn State’s Information Technology Services and Division of Student Affairs agreed to provide support for a part-time instructional design specialist, based in the Dutton Institute, who would be responsible for spearheading the promotional effort and developing and maintaining the portfolio Web site.
Two years later the primary goal of Penn State’s e-Portfolio Initiative remains the same: to promote widespread understanding of the instructional and educational opportunities associated with e-portfolio development and to support those involved in e-portfolio activities. The e-Portfolio project manager continues to offer informational and motivational presentations, hands-on e-portfolio workshops, and consulting services for students, faculty members, and academic units. He also continues to maintain and refine the portfolio Web site. To evaluate the impact of these activities, the project manager gauges student usage of personal Web space accounts twice a year and polls student and faculty about Web publishing and e-portfolio practice every fall. The following sections summarize the findings of these surveys.
Surveying Personal Web Space Use at Penn State
Since 1996 Penn State’s Information Technology Services has provided free personal Web space accounts to students on request. The current storage allocation is 200 megabytes. To submit a request, students must first pass a short quiz that assures their familiarity with relevant university policies and procedures. Activated accounts are assigned the URL http://www.personal.psu.edu/xyz123, where "xyz123" is the student’s Penn State user ID. A default personal home page, like the one shown in Figure 1, is automatically generated when an account is activated. This default page includes links to the university’s e-portfolio Web site.
Click image for larger view.
To create and maintain personal Web sites, students need to master basic Web publishing skills including HTML document editing, graphics processing, and file transfer and management. Students upload and download files to and from their Web space accounts using one of several secure file transfer tools. The e-portfolio Web site provides guidance on how students can develop these skills, along with a gallery of e-portfolio examples and commentary on the rationale on which the initiative is based.
Dutton Institute Surveys
Since July 2000 the Dutton Institute has monitored students’ usage of personal Web space accounts through semiannual surveys.
Survey Methodology
Web space usage surveys involve visiting random samples of students’ personal URLs, noting whether or not sampled accounts have been activated, and characterizing the contents of active sites. Samples are drawn from two populations: the approximately 700 students in residence at Penn State’s University Park campus who pursue baccalaureate degrees within one of the five academic units that make up the College of Earth and Mineral Sciences (EMS, the Dutton Institute’s home college) and the larger population of more than 32,000 University Park undergraduates pursuing degrees in other colleges (non-EMS).
The Dutton Institute’s Web systems administrator extracts from the university student data warehouse lists of the EMS and non-EMS students enrolled in the most recently completed semester. He then sorts the lists alphabetically by last name and assigns ascending record numbers. Microsoft Excel’s random number function is used to generate lists of random numbers with replacement for each of the two lists. Two samples of 100 students each are then selected from the EMS and non-EMS lists by selecting the first 100 unique record numbers from the random number lists. The Web system administrator then delivers two lists of 100 randomly selected student user IDs to the e-portfolio project manager.
Survey Criteria
The e-portfolio project manager uses the database management system interface illustrated in Figure 2 to record whether each account has been activated. The blue "Web Space" button provides ready access to a student Web space account, opening it in a Web browser. The project manager visits all 200 sampled student URLs (100 EMS and 100 non-EMS). He also evaluates the following characteristics of the publicly available content of active sites:
- Does the site include academic content (such as assignments)?
- Does the site include a resume?
- Does the site include personal information (for example, about hobbies or music)?
- Does the site include evidence of cocurricular achievements (like participation in clubs or volunteer activities)?
- Does the site include reflective commentary on the student’s experiences at Penn State?
Clicking the "Yes" or "No" radio buttons assigns a value of 1 or 0 for each characteristic. These values are then exported to a spreadsheet for statistical analysis. The question mark button opens a small window containing the definition of the corresponding characteristic. Quick referral to characteristic definitions helps to ensure consistent evaluation.
Click image for larger view.
Survey criteria have evolved considerably since the initial survey in June 2000. To streamline the review procedure, we reduced the number of criteria from nine to five. Ambiguous criteria—those associated with the lowest inter-rater reliability—were clarified. For example, personnel who conducted the first surveys were asked to judge whether active sites constituted "portfolios." Since 2003, the project manager simply notes the presence or absence of reflective commentary. Following Linn and Gronlund,7 Yancey,8 and other commentators, we define "student learning e-portfolios" as "purposeful collections of examples of student work annotated with students’ reflective commentary."9 Thus, a student’s personal Web space account that includes evidence of curricular or cocurricular achievement and reflection constitutes an e-portfolio. The change in this criterion reflects a shift from a product-oriented emphasis to one that emphasizes process.
Survey Results
Figure 3 reveals trends in findings of four Web space usage surveys conducted between December 2001 and December 2003.10 Four trends are shown: rates of activation of personal Web space accounts and inclusion of academic content in Web space accounts for the EMS and non-EMS populations.
Click image for larger view.
The thicker pair of lines in the lower half of Figure 3 represent trends in account activation rates (triangles) and rates of academic Web publishing (diamonds) among all Penn State undergraduate students at the University Park campus except those pursuing degrees in the College of Earth and Mineral Sciences (EMS). The thinner pair of lines in the upper half of the graph represent activation and academic Web publishing among EMS undergraduates.
The overall rate of Web space activation among Penn State undergraduates (non-EMS) increased at a rate of 43 percent, from 37 percent in 2001 to 53 percent in 2003. The percentage of non-EMS student sites including examples of academic work increased from 8 percent to 29 percent over the same period—a more than threefold increase. Account activation rates among students in the EMS population are higher than those of the university overall, but are increasing at a slower pace. Whereas 63 percent of EMS students had activated their Web space accounts in 2001, 77 percent had done so in 2003, an increase of more than 22 percent. The percentage of EMS students who had published academic content increased only slightly overall, from 48 percent in 2001 to 50 percent in 2003.
In all, by December 2003, approximately 10,000 Penn State University Park undergraduates had published examples of academic work in their personal Web space accounts.
Not shown in Figure 3 is the rate at which EMS and non-EMS students developed e-portfolios (personal sites that include evidence of curricular or cocurricular achievement and reflection upon that evidence). Due to incommensurable survey criteria, it is possible only to estimate that by December 2003, about 10 percent of the approximately 33,000 undergraduate students at Penn State University Park had published e-portfolios in the personal Web space accounts. Four examples drawn from the more than 3,000 Penn State student e-portfolios are illustrated in Figure 4 (captured May 2004). Links to exemplary e-portfolios appear at <http://portfolio.psu.edu/>folio.psu.edu/>.
Click image for larger view.
Triangulating Web Space Use: The ITS Student Survey
Penn State’s Office of Information Technology Services (ITS) periodically polls students and faculty members on topics related to information technology. In fall 2003, e-mails containing a Web link to the student computing survey were sent to a random sample of 5,000 full-time students with active access accounts at all campus locations.
The results of the ITS survey were consistent with findings of the Dutton Institute’s Web space usage survey. Of the 903 responses from students at the University Park campus, 24 percent indicated they had "posted assignments or other evidence of academic achievement" in their Penn State Personal Web Space. Thirty-five percent reported instructors required them to create a Web page for one or two courses during the summer and fall 2003 semesters, while 15 percent reported instructors required them to create an e-portfolio during this same period.11
Clearly, a substantial and growing minority of Penn State instructors are beginning to expect their students can master the basic Web publishing skills needed to create and maintain e-portfolios. Is this expectation reasonable? If so, to what extent can this approach scale to serve a majority of students?
Pros and Cons of Web Publishing Approach
Three advantages accrue to the current Penn State approach of relying on students to develop basic Web publishing skills to create and maintain their e-portfolios. First, providing Web servers and file spaces is cheaper and easier for the university than supporting an enterprise learning management system. Second, students who develop e-portfolios also develop transferable IT skills. Third, students and academic units are not bound by a rigid template, but are free to develop formats that best serve their particular requirements and tastes.
Two important disadvantages also accrue, however. First, students who have trouble acquiring the requisite skills might find it difficult to participate. Second, faculty members who feel responsible for helping students develop Web publishing skills might be less likely to require e-portfolio activities.
To judge the viability of the current Penn State approach, Johnson surveyed more than 400 students enrolled in classes requiring e-portfolio–related activities. As discussed in the following sections, survey results lead us to conclude that the existing human infrastructure is sufficient to support participation by about half, but certainly not all, Penn State undergraduates.
Feedback from Students
During academic year 2003–04 the e-portfolio project manager held 69 consultation sessions with faculty regarding the implementation of e-portfolio–related activities within the courses or programs they teach. Of these, he worked most closely with instructors and students associated with five undergraduate classes. For each class the project manager offered one-hour lab sessions in which he introduced tools and techniques involved in HTML document editing, graphics processing, file management, and file transfer. In some classes he also hosted optional "open" lab sessions during which he assisted students individually or in small groups. For technical support beyond those sessions he referred students to the University Learning Center, whose staff of one full-time director and approximately seven student technology tutors assists students on a drop-in basis. He also pointed out the extensive help available at the university’s e-portfolio Web site.
Near the end of the fall 2003 semester, the project manager administered surveys to 407 students enrolled in the five classes in which e-portfolio–related activities were assigned. All students delivered a complete personal e-portfolio. The classes and number of participants appear in Table 1.
Click image for larger view.
The surveys asked students to characterize their previous Web publishing experience (see Table 2), describe the amount and source of technical support they needed to complete their e-portfolio assignments, and identify what they valued most and least about those assignments.
Click image for larger view.
Despite the fact that 87 percent of respondents reported little or no previous Web publishing experience, only 16 percent reported they needed more than three hours of help outside class. Of the 225 students who reported no previous experience, 56 (26 percent) reported that they needed little or no help to complete the e-portfolio assignment (see Table 3). In sum, while some support needs existed for all, few students required extensive support.
Click image for larger view.
Where did students get the help they needed? The survey provided a list of possible support resources, including friends, open lab sessions, individual tutoring sessions at the University Learning Center (ULC), online resources, classmates, and other. Participants were asked to identify and rank the kinds of support they used. Support resources considered most important were to be ranked "1." Thus, the lowest scores in Table 4 indicate the kinds of support considered most important. If students did not use a resource, they were asked not to rank it. About 8 percent of students used the "Other" category to list a family member, teaching assistant, ITS help desk personnel, or faculty member as their most important source of help. "Friends" was selected most often (286 responses) of the separate options listed and ranked highest in importance. In general, students seem to rely most on fellow students for help in acquiring the Web publishing skills they need.
Click image for larger view.
After friends, open lab sessions, individual tutoring, online resources, and classmates followed in perceived importance. While fewer students selected "Individual tutoring in the University Learning Center" (118), those who did tended to rank it highly. The ULC’s student technology tutors provide walk-in, one-on-one support helping students learn basic Web publishing and other technology skills. Students with no previous Web publishing experience were most likely to choose this service.
In addition to students’ high ranking of technology tutors, the ULC's own statistics indicate a sharp increase in the number of requests for help over the past five semesters beginning with 47 sessions fall 2001 to 442 sessions fall 2003. Sixty-two percent of students who consulted the ULC in fall 2003 and filled out feedback forms reported they were working on e-portfolios.
In general, students seem to rely heavily on fellow students (including friends and peer tutors) for help in acquiring the basic Web publishing skills needed to publish a Penn State e-portfolio.
Finally, participants were asked to identify the aspects of their e-portfolio assignments they considered valuable to the course and to rank the relative value (1 = most important). If the reason did not apply to them, students were asked not to rank it. These rankings appear in Table 5.
Click image for larger view.
Seventy-seven percent of respondents selected the Web publishing skills they learned and the e-portfolio they created as the most important reasons why Web publishing activities were a worthwhile part of the course. Getting their resumes on the Web and self-reflection seemed to have secondary importance. Although approximately 25 percent of students included the statement "this assignment added little or no value to the course" among their rankings, only 8 percent ranked the statement highly. The remaining 18 percent of respondents ranked this statement least important.
Additional Student Feedback
Results of a similar survey administered to students enrolled in the general education course GEOG 121: Mapping Our Changing World corroborated the results reported above. Since 1998 GEOG 121 has required students to publish four to six project reports in their personal Web space accounts. Unpaid undergraduate teaching interns drawn from the best students of preceding semesters earn one internship credit by helping students in the current semester master the necessary skills. Although these project assignments did not require students to create comprehensive e-portfolios, they did require them to develop the skills needed to publish academic products online.
A student survey conducted at the conclusion of the fall 2003 semester revealed that although 81 percent of 68 respondents reported little or no previous Web publishing experience, 61 percent indicated that it was "not very difficult" or "easy" to complete their assignments. More than half reported they "needed a little help" or none at all. Like their counterparts in other academic programs across campus, GEOG 121 students welcomed the Web publishing requirement; 72 percent of survey respondents characterized the Web publishing part of the course as worthwhile. Seventy-five percent expected they would continue to develop an e-portfolio of their academic work. Interestingly, when asked if they would willingly accept a $5 increase in fees to invest in a system-wide application that would make e-portfolio development easier, 49 percent responded, "Yes, it would definitely be worth it."
Discussion
The survey results demonstrate that the promotional effort begun formally in May 2002 has had considerable impact. Between December 2001 and 2003, the activation rate of personal Web space accounts among University Park undergraduates increased 43 percent, and the number of undergraduates who have published academic content in their Web space accounts has increased more than 360 percent. Nearly 30 percent of the 33,000 University Park undergraduates—about 10,000 students—have now published evidence of their academic achievement online. About 10 percent—probably more than 3,000 students—have published combinations of evidence and reflection that are characteristic of a proper e-portfolio.
ITS surveys indicate that learning to create and publish a Web page is the IT skill most Penn State undergraduates consider desirable. Many also reported that they value publishing resumes and engaging in self-reflection. Few (less than 10 percent) rated Web publishing and e-portfolio development not worthwhile. Future studies will investigate the ways in which e-portfolio development enriches teaching and learning from the perspectives of participating faculty members as well as students.
Most of the undergraduate students who participated in the studies analyzed here, including students with little or no prior experience, reported that Web publishing does not provide an insurmountable barrier to e-portfolio development. The technical support available from the e-portfolio project manager, the staff and students of the ULC, and volunteer student teaching interns is adequate for the current rate of participation. The existing support infrastructure may not suffice for university-wide participation in e-portfolio activities of 50 percent or higher, however. If the current rate of increase in e-portfolio–related activity continues, the university may need to expand its support infrastructure or deploy an enterprise learning management system that facilitates e-portfolio development—or both—within the next two years.
Conclusion
We have portrayed the current stage of Penn State’s approach to student e-portfolios as an effort to "keep the horse before the cart." This homely metaphor refers to our conviction that a coherent demand for e-portfolio services, informed by an institution-wide consensus about the nature and potential benefits of portfolio practice, ought to precede the design and deployment of an enterprise e-portfolio or learning management system. This conviction is consistent with Love, McKean, and Gathercoal’s argument11 that institutions should not fall prey to the temptation to "begin at the end" by skipping important incremental steps that are critical to understanding the pedagogical and maturational issues related to e-portfolio implementation.
Consensus about e-portfolios at Penn State, not to mention commitment to fostering an e-portfolio culture, remains a destination on the far side of a tall hill. Our experience demonstrates, however, that a sustained promotional and educational effort, backed up by a modest support infrastructure, can generate substantial interest and involvement in e-portfolio use, even when students must master basic Web publishing skills to participate. This approach can be replicated at other institutions at modest cost.
Before long, we believe the cart will have ascended the hill and gained momentum of its own. At that point Penn State’s e-portfolio project will have begun a new phase. Steering the cart, rather than pulling it, will become the challenge. When and if that time comes, the experience gained in our e-portfolio project will, we believe, prepare the university to choose wisely among the e-portfolio systems offered by commercial vendors or through open source initiatives. The effort will also have justified Penn State’s investment in such a system and set the stage for its widespread adoption.