A Dialogical Approach to Learning Technology Success

Leif Nelson, Interim Director of Learning Technology Solutions, and Daniel L. Gold, Blackboard/LMS Coordinator, Office of Information Technology, Boise State University

In spring 2014 Boise State University leadership worked with Associate Professor of Educational Technology Ross Perkins to realign instructional technology support at the university, which led to the creation of a Technology for Teaching and Learning shared governance group, an Instructional Design and Educational Assessment department, and a Learning Technology Solutions department. This article summarizes how Learning Technology Solutions has developed tools and strategies to improve learning technology management and support. —The Authors

Educational technology lies at the intersection of different institutional cultures. One culture represents the financial, operational, and administrative aspects of a higher education institution, while the other emphasizes effective pedagogy, research, and scholarship. Simply put, one might categorize these cultures as "business" and "academics." Granted, these cultures are not mutually exclusive, nor do the terms "business" and "academics" capture the diversity and nuance of institutional complexity. Yet, differences manifest at various levels of an institution. There is evidence of this phenomenon in the literature. For example, an article about academic analytics described how faculty members would bristle at language like "business intelligence."¹ In one study, the intentional avoidance of IT project management methodologies was, according to the author, the success factor in an LMS implementation.² Leeriness is common when cultures comingle. Meanwhile, collaboration is essential to institutional success. Departments that support educational technology often reside at the intersection of business and academic institutional cultures; these departments have a unique opportunity to advocate for common frameworks and productive dialogue. The convergent evolution of project management and instructional design reveals such common frameworks. Embracing processes that are sensitive to cultural differences will promote dialogue.

Understanding Cultural Differences

To move to a more unified model for managing educational technology, an institution must first acknowledge differences reinforced through training and experience in both business and academic environments. Within the IT and project management disciplines, formal training generally skews toward private sector communication styles and organizational structures.

A fundamental aspect of academic culture is debate. Faculty members and academic leaders embrace the universal language of critique and argument. This passion for debate manifests frequently in meetings. Even an idea that is generally supported by the group will be taken apart and discussed in detail. For project managers familiar with business review meetings in the private sector, this can cause culture shock. Most project managers do not expect to approach a project kick-off meeting like a thesis defense, and they may leave the meeting wondering what it accomplished.

Words can create barriers. Both IT and academic lexicons are rife with discipline-specific terms that do not translate well. Common language can lead to mutual understanding, but due to the importance of articulating specific requirements for software design and development, IT staff members sometimes resist adopting a more universal tone in order to accommodate nontechnical audiences. IT encodes needs and requests in a way that is unrecognizable to the requestors. In a 2010 survey, non-IT stakeholders were more likely to see requirements gathering as a way to create "detailed user interface designs," while IT teams characterized this work as "capturing specifications."³ In the same survey, only 23 percent of respondents stated they were always in agreement about when a project is truly done, meaning the majority leave the door open for "rework, scope creep, and disappointment."³ When there is confusion about technical needs, projects take longer than expected and are not considered a success.

Regardless of the methodologies used within an IT organization, certain foundational documents are required to move a project forward. These templates abound with unfamiliar language and writing styles that may make it difficult for academic partners to interpret and review. This can lead campus stakeholders to withdraw in project meetings and allow the project's momentum to carry it forward, often resulting in planning or design documents that are not aligned with the needs of stakeholders. Similarly, academic stakeholders may lack a command of the technical terms necessary to express their needs in terms technical staff can accurately interpret. Objectives may be expressed by academics in abstract or verbose language that is interpreted literally by technical teams desiring to accurately accommodate needs. This can be a challenge when selecting the best solutions. The outputs of discussions, rituals, and artifacts make up the blueprint for our educational technology ecosystem. A systems approach to reconciling academics and business could cultivate a healthy operating rhythm within a university, because accuracy and mutual understanding would be supported.

Boise State University has established a Project Management Office within the Office of Information Technology to successfully manage complex projects related to the enterprise and administrative systems. A new department, Learning Technology Solutions, is now integrating project management methodologies with academic research and instructional design methods.

Where Does Technology Fit in Academia?

Technology plays an instrumental role in promoting effective pedagogy. In a recent meta-analysis, active learning strategies were shown to produce higher student performance outcomes compared with other, more traditional instructional methods.⁴ Audience polling tools, interactive multimedia, games and simulations, and learning spaces incorporating collaboration technology exemplify how educational technology supports active learning strategies.

In higher education, certain barriers inhibit successful incorporation of both active learning strategies and the technologies that support them. Faculty members expect educational technologies to be not only stable and reliable but also to work flawlessly and to have a high level of available institutional support.⁵ At the same time a strategic shift has occurred in many academic technology units: from "how-to" technical support and training to a "big picture" focus in multiple areas of campus, including faculty development and IT infrastructure planning.⁶ The level of support faculty members require remains at the "how-to" level.⁷ If "how-to" support exists solely in units not trained in educational theory or methods, then there may be a missed opportunity to address the pedagogical factors related to successful technology integration.

Technology systems should support a diverse range of pedagogical benefits. To promote effective pedagogical practices, instructional designers and faculty developers use research and evidence to help faculty members incrementally adopt new tools and techniques. A dialogue between technology managers and instructional developers will lead to a more cohesive approach to improving effective teaching and learning with technology. Strategies and models conducive to dialogue can lead to successful integration of both reliable technology and effective pedagogy.

Melissa Pierson's TPACK model articulates the relationship between technology, pedagogy, and content knowledge.⁸ Pierson's model suggests that a faculty member should command a certain level of fluency in all three domains to be most effective. At an institution, the academic departments contain the content knowledge of a particular discipline, yet the departments require an institutional support infrastructure to be effective in the areas of technology and pedagogy. This is especially true for hybrid or fully online instruction.⁹ IT units support technology, and faculty development units (e.g., teaching and learning centers) support pedagogy. The faculty member — the content expert — will have variable levels of pedagogical and/or technology knowledge. A dialogue among all three groups leads to enhanced techno-pedagogical effectiveness.

Climbing the SAMR Ladder: Progress and Pitfalls

Ruben Puentedura developed a model illustrating a trajectory toward highly effective pedagogy — the Substitute, Augment, Modify, Redefine (SAMR) model.¹⁰ Envision a pyramid with increasing levels of effectiveness at each stage and with potential value adds and potential pitfalls at each level. The base of the pyramid is the substitution level, where faculty members might simply digitize content such as syllabi or lectures. Efficiency, not effective pedagogy, often prompts faculty to begin using a new technology tool,¹¹ and pedagogical strategies at the substitution level tend to be designed for students to simply receive and remember information.

At the augmentation level, assessments may be added to encourage engagement, make formative improvements, or begin personalizing the delivery of instruction. The pitfall to overcome at this level is the perceived problem of cheating. Despite the impulse to design better and more complex mechanisms to subvert cheating, effort should instead be focused on using assessment formatively to improve student learning.

The modify level incorporates formative assessment data to personalize the learning experience by delivering differentiated instruction, promoting collaboration among learners, or promoting personal meaning making in the learning activities through authentic and experiential activities. A potential pitfall at this level is that personalized learning mechanisms that are decontextualized from the social construction of meaning, or that are purely algorithmically generated, might be overly prescriptive or preclude serendipitous discovery of unrelated content and the formation of divergent connections among disparate sources of information.

The pinnacle of instructional effectiveness, according to a plethora of theories and models, empowers students to actively create meaning and knowledge. Lorin Anderson and David Krathwohl's revised Bloom's taxonomy places creation and synthesis at the apex of their pyramid,¹² while Ference Marton and Roger Sāljö suggest that learning should be "deep" and self-directed as opposed to superficial and driven by purely extrinsic motivators.¹³ And, as discussed, active learning is preferable to passive, didactic content delivery.¹⁴ An integrated view of the SAMR and TPACK models appears in figure 1.

To maximize the potential afforded by these integrated models, a dialogue must exist between IT professionals and pedagogical consultants. A synthesis of project management and instructional design methodologies may be the mechanism to avoiding the pitfalls and achieving educational effectiveness.

Project Management and Instructional Design: A Case Study in Convergent Evolution

A deeper look into project management and instructional design reveals similarities in their models and processes. Project managers commonly coordinate resources to improve standard technology systems, and instructional designers often participate in training, best practices, and effective use of standard educational technology systems. A dialogue between these groups helps ensure effective adoption of educational technologies for instruction. The history of project management and instructional design as professional disciplines reveals a convergent evolutionary path, suggesting they have developed similar characteristics due to similar environmental conditions.

Project management and instructional design both systemize and apply processes to ancient human activities: namely, doing and learning, respectively. Figure 2 shows citations of the two terms and their growth over time.

Much of the literature about project management and instructional design deals with applying project management methods to instructional design, rather than recognizing similarities and striving to synthesize these two fields. For example, Shahron Williams Van Rooij wrote, "The absence of project management from instructional design curricula can be viewed as a consequence of the divergent perspectives of institutional subcultures."¹⁵ The article title, "Project Management in Instructional Design: ADDIE Is Not Enough," suggests inefficiencies in instructional design methodologies that only project management methodologies can supplement. John LeTourneau suggested that project management best practices (as articulated by the Project Management Institute) can be instrumental in the success of change management and innovative IT initiatives in a higher education institution.¹⁶ The hypothesis in the literature is so far a dialectical one — that one model dominates the other. We propose a more dialogical approach, one that allows for equal footing, assimilation, and evolution.

Learning Technology Solutions: Bridging Institutional Cultures

The first step in any project, whether academic research or IT, requires the initiator to articulate in clear terms the proposed initiative. In project management, this results in the creation of a project charter that outlines a problem statement, proposed solution, and implementation plan. In academics, a research proposal outlines the purpose, rationale, objectives, and methodology. Table 1 shows the comparison.

Table 1. Comparison of common components of a project charter and research proposal

Project Charter	Research Proposal
Emphasis on delivery	Emphasis on evidence
Statement of purpose	Statement of purpose
Problem statement	Rationale
Scope/deliverables	Objectives
Methodology	Methodology
Resources	Resources
Other sections: stakeholders, risks, etc.	Other sections: lit review, limitations, etc.

 

Each document has the same goal: secure approval and authority to move forward. Despite sharing a common cause, the style and tone of the documents often differ significantly. In a project charter the methodology section emphasizes delivery and output, while in academic research the methodology emphasizes evidence and rigor. When creating an academic technology proposal, trying to appeal to both IT and academic affairs leadership poses a special challenge. The Learning Technology Solutions (LTS) department at Boise State University has developed an integrated initiatives proposal document template with the following sections:

1. Statement of Purpose: a concise description of the initiative
2. Rationale: an explanation of the initiative's importance or value
3. Background: institutional context, triggers for the need for this initiative, literature review, etc.
4. Objectives: listed as SMART (specific, measurable, attainable, relevant, and time-bound) goals
5. Methodology: describes the initiative process
6. Constraints and Assumptions: further elaborates on factors such as resource needs, related initiatives, scope, and scheduling constraints
7. Works Cited: cites any literature or other resources used in the proposal document

The proposal template represents a blending of project management and instructional design methodologies (figure 3). The model intentionally borrows from IPECC and ADDIE, yet generalizes both the concepts and the language to make them more consumable for diverse audiences. Importantly, the model emphasizes both delivery and evidence.

Boise State University's LTS has augmented a project scoring rubric first developed by Montana State University. The augmented version includes a section for pedagogy that was co-created by instructional designers. Once initiatives are scored, a bubble chart visualizes the strategic (including pedagogical) alignment of the initiative as well as its probability of success and potential value (see figure 4).

Both the proposal and the rubric seek to improve communication among IT and academic teams. The initiative proposal template uses terminology that is largely process and discipline agnostic, while the rubric aims to provide an objective instrument for prioritization.

As initiatives move beyond planning to implementation, stakeholders need to stay informed about progress. Templates for effective status reports abound within the project management discipline, and while the layout and content vary, the templates all tend to have a push approach to communication, usually in the form of a weekly or monthly e-mail. When a question arises, the stakeholder may e-mail the project manager, but rarely the entire distribution list, thus limiting the discussion to just two individuals. If a question could lead to multiple e-mail exchanges, the stakeholder might wait until the next project meeting to bring it up. This lag reduces the time the project team has to react to a problem. Boise State has experimented with blogs, focus groups, interviews, and case studies — effectively leveraging tools from both project management and research methods — to address communication challenges. Where the one-way communication format of status reports can leave stakeholders feeling that something is being done for them or to them, blogs, surveys and interviews can leave stakeholders feeling that something is being done with them.

An indirect goal of these activities is to create a framework that encourages staff members who are not formally trained as project managers to easily pick up a set of tools and techniques to ensure the successful delivery of campus initiatives. Future efforts will include further research about the convergent evolution of project management and instructional design, as well as continued refinement of the models and methods used.

Acknowledgments

We would like to thank Brian Bolt, Shad Jessen, and Jeffrey Oliver for their proofreading and feedback. Thanks also to EDUCAUSE and Nancy Hays for editorial and publishing assistance.

Notes

1. Philip J. Goldstein, "Key Findings," Academic Analytics: The Uses of Management Information and Technology in Higher Education (Boulder, CO: EDUCAUSE, 2005).
2. Alan Lawler, "LMS Transitioning to Moodle: A surprising case of successful, emergent change management," Australasian Journal of Educational Technology, Vol. 27, No. 7 (2011): 1111–1123.
3. Doomed From the Start -Why a Majority of Business and IT Teams Anticipate Their Software Development Projects Will Fail . (2011): 3-5
4. Scott Freeman, Sarah L. Eddy, Miles McDonough, Michelle K. Smith, Nnadozie Okoroafor, Hannah Jordt, and Mary Pat Wenderoth, "Active learning increases student performance in science, engineering, and mathematics," Proceedings of the National Academy of Sciences of the United States of America, Vol. 111, No. 23 (June 10, 2014): 8410–8415.
5. Kate Hudson, Working Together: The Role of Collaborations in Promoting the Use of Academic Technologies in Higher Education, Dissertations, Paper 288, University of Massachusetts–Amherst (September 2010).
6. Ibid.
7. Stuart A. Karabenick, "Classroom and Technology-Supported Help Seeking: The Need for Converging Research Paradigms, Learning and Instruction, Vol. 21, No. 2 (April 2011): 290–296.
8. Melissa E. Pierson, "Technology Integration Practice as a Function of Pedagogical Expertise," Journal of Research on Computing in Education, Vol. 33, No. 4 (Summer 2001): 413–430.
9. June Lee, "Instructional Support for Distance Education and Faculty Motivation, Commitment, Satisfaction," British Journal of Educational Technology, Vol. 32, No. 2 (2001): 153–160.
10. Ruben Puentedura, "Technology in Education: An Integrated Approach," Ruben R. Puentedura's Weblog, December 12, 2014.
11. Nike Arnold, Technology-mediated learning 10 years later: Emphasizing pedagogical or utilitarian applications?. Foreign Language Annals, (2007) 40, 1, 161-181.
12. Lorin W. Anderson and David R. Krathwohl, A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives (New York: Longman, 2001).
13. Ference Marton and Roger Sāljö, "On Qualitative Differences in Learning: I - Outcome and Process," British Journal of Educational Psychology, Vol. 46, No. 1 (Feburary 1, 1976): 4–11.
14. Ibid.
15. Shahron Williams Van Rooij, "Project Management in Instructional Design: ADDIE Is Not Enough," British Journal of Educational Technology, Vol. 41, No. 5 (2010): 852–864.
16. John LeTourneau, Institutional Change in a Higher Education Environment: Factors in the Adoption and Sustainability of Information Technology Project Management Best Practices, PhD thesis, Michigan State University, 2012.