Many universities use electronic portfolios (e-folios) to collect student work and assessment data for accreditation purposes at either the program or unit level. Programs within teacher education can include early childhood, elementary, middle level, and secondary education. For accreditation, each program is responsible for meeting a specific standard set for that program. Teacher-education programs must align their curricula and student outcomes with state and national teacher-education standards. These standards provide direction for the selection and inclusion of artifacts in the e-folio and affect its organization. With e-folios, students can enter evidence such as lesson plans, parent communication samples, and implemented assessment strategies. Evaluation data are aggregated and disaggregated to demonstrate that a program meets the necessary standards and to highlight the program's strengths and weaknesses. Colleges of education can use the data to guide changes in their curricula.
The research described here investigated the experiences of teacher education programs at four universities that use a common commercial e-folio system.1 Students in these programs purchase the system, which is comparable to other e-folio systems, from the vendor. Like most e-folio systems, this one provides students with opportunities to create lesson plans, portfolios, and other projects, along with options for these items to be assessed by instructors using common rubrics. This assessment allows for data analysis and aggregation, the common objective of most e-folio systems. The e-folio coordinators at these four institutions, which are major users of the system, shared issues of implementation, faculty training, and faculty buy-in in response to an e-mail questionnaire.
Literature Review
E-folio systems provide a tool for collecting artifacts that can be used for program-wide assessment. Students demonstrate they have met various university goals, and the university can use this data to document its program's successes and shortcomings.2 In 2002, approximately 90 percent of all teacher education programs employed a portfolio system.3 Fifteen years later, a majority of this portfolio usage is electronic. Schools of education are good candidates for e-folios because of accrediting agencies' demands for better-organized and easily accessible student work.4 In addition, accreditation agencies have encouraged the use of e-folios because they are more manageable than paper copies, display appropriate evidence, and demonstrate a student's performance and mastery.5 The ability to manipulate data electronically allows teacher preparation programs to use the same evidence in determining whether they have met national accreditation standards, state certification requirements, program goals, or institutional objectives.6
E-folios have the potential to change the face of higher education, especially teacher education.7 However, successful implementation requires strong support from university administration and the broad spectrum of stakeholders involved in decision making and planning.8 Strudler and Wetzel9 believe that part of leadership is communicating and selling the vision to others. Inclusive, productive faculty governance can be key to faculty participation in and satisfaction with an e-folio program. Strong dean-level leadership that includes substantial allocation of resources to support faculty and students in their use of e-folios is also beneficial.
As colleges bring the e-folio from idea to implementation, a number of decisions must be made. Universities must determine which tool to use, which assignments will become part of the student e-folios, and who will make these decisions. Considerable thought needs to go into introducing the e-folio system to students and assessing items included in e-folios.10 Implementation of an e-folio system also requires sufficient access to technology, an adequate campus technology infrastructure, and continual user support.11
Many institutions have indicated that it is not easy to successfully implement an e-folio system. A plan should be in place first. It is essential to implement an e-folio system in steps instead of all at once, and finding faculty to test the system and push its limits is a necessity.12
Chappell and Schermerhorn13 suggested five rules for implementing e-folios:
Following implementation of the e-folio system, the next steps should include streamlining requirements, ensuring access to technology, and addressing the technology's reliability. For example, faculty and administrators should decide what will be required of students and how the college will provide the technology needed (software, hardware, and support). Next, the college should plan for data aggregation and program evaluation. A large group of stakeholders should participate in the continued planning process, clarifying the purpose of the e-folios and continuing to improve usage of the e-folio system.14
Implementation of an e-folio system requires a large amount of time and effort from both faculty and students. Faculty who will be reviewing and assessing artifacts within the e-folios must be willing to take the time needed to do so. Faculty must also realize that e-folios can benefit them by providing a tool to "manage, review, reflect, and comment on student work."15
One of the most evident benefits of an e-folio system is the elimination of physical storage problems. A second benefit is accessibility. Faculty can tap into the e-folio wherever there is an Internet connection.16 And, e-folios give administrators the capability of tracking students, aggregating and disaggregating data, and organizing curricula around professional standards more easily than doing these tasks manually, especially at institutions with many students and large amounts of data.17
Nevertheless, an assessment tool is only as useful as the overall assessment system used. Rubrics used for e-folios must provide the data necessary for programmatic or unit-wide accreditation and program improvement.18 As the integration of an e-folio system continues, assessments must change so that data collected are useful for the specific program. Many colleges of education do not use commercial e-folio systems that include data collection and analysis tools. Personnel at colleges of education often use their own assessment systems, enter data into their own databases, and aggregate data from evaluation rubrics without the use of an embedded tool,19 which is appropriate for smaller programs.
The important questions of when, how, and how often to assess performance deserve attention. Two popular options include incorporating requirements within specific courses and requiring evidence at specific points throughout the program. With requirements for a specific course, faculty specify the artifacts to include, and evaluation of the artifacts occurs as part of the regular course grading procedure.20 When students choose which items to place in their e-folios, usually the items are evaluated at given checkpoints.
Wetzel and Strudler21 offered many recommendations for implementing and using an e-folio system, some of which follow:
- Provide appropriate leadership.
- Provide appropriate faculty support, including training and technology.
- Be clear about the purpose for using the e-folio.
- For those groups that resist using e-folios, meet with them individually to explain the benefits and provide individual demonstrations.
- Have a coordinator available to resistant faculty and others.
- Make sure the e-folio system is a requirement for all teacher education candidates.
- Develop a portfolio system that would be used even if the technology aspect disappeared.
- When implementing, start with smaller steps that will provide a firm foundation for the larger steps ahead. Full implementation should be at least a two- or three-year process.
Using these guidelines during implementation should reduce problems. They also served as a conceptual framework through which we examined the results of our study.
The Four Schools
The four universities implemented the e-folio system in different, although similar, ways and on varying schedules. We have labeled them Southeast University, Metropolitan University, Southern University, and Midwest University to preserve their privacy.
Southeast University
Southeast University is located in a state where the state department of education has adopted an e-folio system and required it of all teacher education programs. Approximately 1,250 students and 100 faculty members use the e-folio system on campus for portfolios, assessment data collection, vitae collection, meeting notes, organization, and explanation. The university's most recent National Council for Accreditation of Teacher Education (http://www.ncate.org/) visit was in fall 2005, and the next NCATE visit will be in fall 2013.
Southeast University phased in use of the e-folio system beginning with the undergraduates admitted in fall 2004. The university had used a hard-copy portfolio system previously. Training sessions for both faculty and students began in summer 2004 and continue to be offered every semester. The technology office also provides on-demand personal training, according to the questionnaire response, although they hope to eliminate the need for this type of support because it is so time-consuming.
Metropolitan University
Metropolitan University has five different campuses in a large metropolitan area, all with the same missions and a similar curriculum. Just under 1,900 students are registered to use the e-folio system, and just over 90 faculty accounts are available, some of which are shared by multiple faculty for assessment purposes only. Metropolitan University's last NCATE visit was spring 2004, and its next NCATE visit will be in 2014. This institution provided no information regarding e-folio implementation.
Southern University
Southern University is the largest producer of teachers in its state, and among the top 50 in the country. The university uses the e-folio tool for portfolios, lesson plans, standards searching, assessment, data analysis, and posting of committee minutes. There are 3,150 students and 200 full-time and part-time faculty using the e-folio system. The university's last NCATE visit was in 2005, and its next NCATE visit will be in 2013.
Southern University began by requiring all student teachers to complete a portfolio using the e-folio system. The student teachers were not happy with the new requirement for their last semester, but as the coordinator stated, "We had to start somewhere!!" The next semester Southern University required students entering the teacher education program to purchase and use the e-folio system in their classes. Training was incorporated into the Orientation to Teacher Education course. The final group to begin using the e-folio system was those students in teacher education who had not yet done student teaching. Training for the students took place during the required instructional technology course. For fall 2006, graduate education students were required to purchase the e-folio tool and attend a graduate orientation class where they received training.
Midwest University
Midwest University is one of the top-ten producers of teachers in the country. One of every seven teachers in the state graduated from this university. Approximately 5,000 teacher education students and about 400 full-time and part-time faculty use the system during any given year. The university's last NCATE visit was in 2002, and the next visit will be in 2011.
This institution uses the e-folio system for mandated portfolio assessment of unit requirements, including 37 accreditation programs. Some programs also use the system for program requirements. Their use of the e-folio tool incorporates data collection and analysis at the program and unit level. Some programs use the tool for collecting vitae and meeting minutes, but this is not a unit requirement.
At Midwest University, the e-folio system, along with training, was introduced to faculty in fall 2002 and required in fall 2003 for all preservice teachers, whose training began in spring 2003. Student teaching requirements were implemented in fall 2004. The e-folio coordinator formed a group of faculty from across campus to work with the e-folio system, which helped with implementation and communication throughout the university in the various teacher education programs. Student training became a requirement in fall 2005. A group of advanced student users began to meet in spring 2006 to help push the limits of the e-folio system and provide examples of portfolios with more than the minimum requirements.
Data Collection and Analysis
The four universities participating in this study were selected because of the large number of faculty and students using this specific e-folio tool on each campus. These institutions also occupied different points on the continuum of implementation and use, making their feedback and experiences valuable to others. We e-mailed the e-folio coordinators from each university seven open-ended questions pertaining to their e-folio usage:
We used a qualitative research method based on e-mail interviews. Completed surveys were categorized and analyzed for common responses. Basic descriptive information was used to better define the study population and occurrence of responses. For example, it provided the context and background information on each of the teacher education programs.
Information about these universities and their teacher education programs was obtained not only from the questions but also from their websites. We collected this information while awaiting responses to the e-mail survey. Upon receipt of responses from the coordinators of the electronic portfolio system, we examined the data for common themes.
We met on several occasions to analyze the data and developed categories by segmenting the information. As Bryman and Burgess noted, "categories are rarely known in advance of data exploration, and the relationships between categories must always be discovered during data analysis."22
The process of analyzing the data for common themes began with questions four, five, and six. These questions provided narrative responses that dealt specifically with benefits, challenges, and advice for implementation. Questions one, two, three, and seven provided descriptive data for participating institutions, which was not included in this categorical analysis. We designated a category that captured each comment, categorizing each sentence in a participant's form separately. In some sentences more than one category was designated. For example, one participant provided the following benefit response: "Efficient means for developing a richer portfolio than hard-copy format." This sentence received three categories: student (richer portfolio), programmatic/administrative (efficiency), and accreditation (efficient for data collection and analysis). This process of theme identification resulted in five categories used for questions four, five, and six.
Findings
Question four asked coordinators to identify what they believed to be the benefits of e-folio use for students, faculty, and administrators. That question resulted in 23 responses, to which the 5 established categories were applied; some responses received multiple categories. Established categories for analyzing benefits and challenges were student, faculty, programmatic/administrative, accreditation, and miscellaneous. The student category incorporated those items relating specifically to student benefits and student challenges. Similarly, the faculty category included items relating specifically to faculty benefits and faculty challenges. The programmatic/administrative category was used for all mid-level and higher-level administrative responses, including those at the program level. The accreditation category included responses regarding the use of electronic data collection and organization for accreditation purposes as well as required reporting. The miscellaneous category included the two responses that did not fit in the above four categories: "ecological friendliness" and "easy accessibility via the Internet."
As described in Table 1, when the e-folio coordinators at the participating institutions described benefits of an e-folio tool, the highest percentage of responses (27 percent) fell into both the programmatic/administrative and the accreditation categories. One response in the accreditation category included:
| Table 1 | ||
| Benefits of E-Folio Use by Research Category | ||
| Category of Benefits | No. of Responses | Percentage |
| Student | 8 | 24% |
| Faculty | 4 | 12% |
| Programmatic/administrative | 9 | 27% |
| Accreditation | 9 | 27% |
| Miscellaneous | 2 | 6% |
Responses in the programmatic/administrative category included:
Responses that focused on both accreditation and programmatic/administrative benefits included:
When coordinators were asked to think about benefits of e-folio usage, both Southern and Metropolitan Universities reported high percentages of benefits to students. Southeast and Midwest Universities reported the highest benefit percentages in the programmatic/administrative arena.
When analyzing responses involving challenges observed in e-folio usage, coordinators reported few challenges to students. However, challenges to faculty and programmatic/administrative challenges were high (42 percent and 52 percent, respectively). See Table 2.
| Table 2 | ||
| Challenges in E-Folio Use by Research Category | ||
| Category of Challenges | No. of Responses | Percentage |
| Student | 1 | 5% |
| Faculty | 8 | 42% |
| Programmatic/administrative | 10 | 52% |
| Accreditation | 0 | 0% |
| Miscellaneous | 0 | 0% |
One response in the faculty category included:
Programmatic/administrative challenges centered on administrative issues. Participants described them as follows:
An analysis of the challenges reported by each institution demonstrates that all the challenges reported by Southern and Southeast Universities were faculty-centered. Metropolitan and Midwest Universities had the highest percentages of challenges in the programmatic/administrative arena, with some faculty challenges as well. One area of challenge at Metropolitan University focused on communication issues between the administration and faculty and e-portfolio tool requirements:
Other challenges identified were related to the e-folio platform or vendor, including:
In question six of the survey instrument, e-folio coordinators were asked to make suggestions for implementing an e-folio tool. Many of the responses seemed to be administrative suggestions for implementation and management of the e-folio system that would assist faculty in its use and acceptance, including:
Suggestions for students included making their e-folios more useful. Southeast University has students create scavenger hunts, web quests, online newsletters, lesson plans, and rubrics using the e-folio tool. This not only provides for university assessment requirements and data with appropriate evidence but also gives each student a database of classroom activities and strategies for teaching. Other student/faculty suggestions included planning a method of communication to all faculty and students, providing training sessions, and having someone with a "can do" attitude available for class visits.
Suggestions for new users correlated with the benefits and challenges experienced by the e-folio coordinators. A frequent theme was the importance of understanding the difference between the e-folio system and the assessment plan. An e-folio platform does not constitute an assessment plan. Institutions should work to develop an authentic assessment plan to meet accreditation standards and provide meaningful feedback at all levels. The choice of an e-folio platform should serve as a tool in the implementation of that assessment plan, not as the plan itself.
For those universities and teacher education programs looking at implementing an e-folio system for the first time, communication is key. This communication should take place in more than one format: e-mails, web pages, announcements during faculty meetings, posters, and workshops. In addition, after implementation, a method of evaluation and change must be in place so that the information needed becomes available. One cannot expect to get it all right the first time. Depending on the circumstances of implementation, all methods can succeed, as shown through the different methods of implementation employed at these four universities.
Discussion
When examining the data, it is interesting to consider the programmatic/administrative category. This category had the highest number of responses for both benefits and challenges. Programmatic and administrative benefits seemed to focus on tool use support, while challenges centered on organization and communication. There was a clear divide between benefits and challenges in e-folio use. In this study, benefits to e-folio use were linked to the accreditation process and student benefits, and challenges seemed primarily linked with faculty issues.
The popularity of portfolios—more specifically, e-folios—evolved because of accreditation's role in teacher education.23 However, e-folios have demonstrated more benefits than just providing data for accreditation purposes. Such benefits include organizing course information, maintaining programmatic and teacher-education information, collecting evidence and providing support for program improvement, and organizing student work.
In reflecting on the differences in benefits at the four schools, and noting that two schools found more benefits in accreditation and two in program and administration, we concluded that many reasons might underlie the results. E-folios are just a tool. How teacher education programs choose to implement and use the tool can vary. For example, the two schools that found accreditation support most beneficial might have used the assessment and data reporting function to its potential, while the schools that considered the programmatic/administrative functions most beneficial might have found the organization of portfolios, syllabi, and other coursework more useful for their needs.
In looking at the challenges, program and administrative challenges appear to stem from communication about the tool's purpose, use, and capabilities, although this also proved one reason for some faculty to buy in to the benefits of the system. Students evidently do not encounter much challenge in working with the system, but when faculty members unhappy with the system discuss its failings in the classroom, students may question the e-folio system's purpose and usefulness.
Challenges with programs and administrations could arise from a need to reorganize coursework and assignments, not to mention finding the time to include faculty in the process. Trying to decide how to obtain the data needed for accreditation, deciding which performances to include in the e-folio and which should be included elsewhere, and providing training for faculty and students can challenge programs and administrations.
The suggestions for implementation provided by Wetzel and Strudler24 offer an appropriate conceptual framework for this model. Wetzel and Strudler gave high importance to leadership in implementing an e-folio system, and all four universities in this study reported appropriate leadership and support from their administrations. Faculty support was provided in a variety of ways including workshops and one-on-one, online, and phone support. As for clarity in purpose, the participating institutions began to use the e-folio system for accreditation purposes, but some teacher education programs expanded on those uses.
Wetzel and Strudler25 stressed the importance of countering resistance by explaining benefits and providing demonstrations to small groups and individuals. Although contagious resistance is one of the biggest derailers of any new process, participants in this study did not discuss specific strategies used to meet resistance.
Schools implementing an e-folio took specific steps to promote the system's success, as follows:
- Having a coordinator who is available and accessible to faculty was important to the study participants. Each university had a coordinator who helped with faculty, student, and e-folio implementation issues.
- Consistency in e-folio requirements is a necessity in implementation. Each institution in this study required e-folios for all teacher education undergraduate students.
- Wetzel and Strudler also emphasized the importance of having a portfolio assessment system in place regardless of whether technology is used as a tool. Only Southeast University reported having a portfolio system in place for all teacher education students prior to implementing the electronic version.
- Finally, all participants in this study reported taking implementation slowly, in manageable increments. Wetzel and Strudler suggested a three-year implementation process; the participating institutions reported a two- to three-year implementation cycle.
Considering Wetzel and Strudler's suggestions as a conceptual framework, the one area not discussed among the four institutions involved working with faculty who resisted e-folios. The institutions cited this as an issue, and they did include one-on-one meetings to help the resisters work through perceived problems, but they did not hold meetings specifically for this group to describe benefits and demonstrate system operation.
Our findings examined through this framework would apply not just to teacher education but also to other programs and university-wide e-folio implementations. Universities are accredited in order to grant degrees, so e-folios serving university-wide as a data collection tool would provide administrative and accreditation benefits. This study demonstrated the obvious challenges of implementation, which would be compounded by sheer volume in a university-wide implementation.
Conclusion
Despite its importance, this study's limitations are evident. The participants were experts in the given e-folio system who do not necessarily hear students' thoughts on the program and, in many cases, may hear faculty comments only when faculty members need help, such as when problems arise. A similar study interviewing various teacher education faculty and students on their use of and satisfaction with the e-folio system would provide additional information. It would also be useful to conduct an analysis of university policies and procedures, conceptual frameworks, training evaluations, and actual e-folio content.
Another area for further research is the use of inter-rater reliability, especially when dealing with multiple assessors using the same rubric. As the literature suggests, many universities using e-folio systems have not established inter-relater reliability.26 How do e-folio coordinators and other teacher education administrators ensure that all assessors use the same measurements for different groups of students? Do the methods work well? What are future plans to guarantee the reliability of these methods of assessment and data analysis? If universities have not addressed inter-rater reliability, why not? And how can they guarantee their data are appropriate for accreditation? Institutions considering an e-folio tool in their assessment process should think about not just the platform choice but also implementation issues, specifically organization of information and effective communication between all stakeholders.
A final area of research could examine the similarities and differences between e-folio implementations in large and small teacher education programs. Although implementation of an e-folio tool in a large teacher education program is more challenging because of its size, smaller programs can learn from what larger institutions have done despite some distinct differences in implementation: choosing a system, determining required assignments, training, and data collection.
E-folios have become popular in teacher education programs because of accreditation requirements. As teacher education and other higher education programs continue to need proof of student performance, e-folio systems will grow in popularity. Commercial systems will better accommodate universities' needs, or they will lose their customers.
As more higher education programs become performance based, they will need e-folios. It is possible that e-folios will become part of the PK–12 school system as those schools focus more on student performance and alternative assessment as a supplement to standardized testing.
We recommend that any program interested in performance-based assessment systems and e-folio tools explore various options after they know what kind of data and evidence they want. Communication, training for everyone, and continuous improvement in system use are imperative for successful implementation of an e-folio system. The four universities surveyed have provided words of wisdom to help in the process.