Agentic AI and Change Management Lessons for Modernizing Systems

Sophie White: Hello everyone and welcome to EDUCAUSE Shop Talk. This is a really interesting episode that we have ahead for you and if you're interested in agentic AI for increased efficiency related to things like software development and enterprise systems, this is a great episode for you. In this one we talk to Brian Zahn and Eric Markle from the George Washington University about a project that they are in the middle of related to modernizing their student information system and human resource system through the Banner SaaS platform, but they actually use an agentic AI solution that they co-developed to do that implementation more efficiently. And so this is kind of a case study of their work. We get into some really technical elements. You will see the limits of my technical expertise in talking about these kind of things, but then we also talk about the important change management elements, how they worked with different stakeholders at the institution to set guidelines and how they've been able to use this solution in other even unexpected areas of the institution.

So I hope you will check it out and enjoy.

***

Hello everyone, and welcome to EDUCAUSE Shop Talk. I'm Sophie White. I'm a content marketing and program manager at EDUCAUSE and one of your co-hosts for today's discussion.

Jenay Robert: And I'm Jenay Robert and I'm your other host.

Sophie White: Great. So today we're talking about agentic AI and higher education and we're really excited to be having this discussion. It feels like a topic that's bubbling up quite a bit in our community. And we're thrilled to be here with two guests today who've been working really closely at their institution with agentic AI. So they're going to talk to us about the larger landscape as well as a really fascinating project that they're working on. So first we have Brian Zahn. Brian is a senior enterprise systems analyst at the George Washington University in Washington DC. He's focused on digital transformation initiatives, including a major ERP modernization effort and the rollout of agentic AI development workflows across the IT organization. With more than ten years of experience in higher ed IT spanning multiple public universities in Ohio, Brian specializes in enterprise systems, workflow redesign and practical approaches to using emerging technologies to accelerate institutional transformation.

Thanks so much for being with us, Brian.

Brian Zahn: Thank you. It's great to be here. Thank you for having us.

Sophie White: Next up we have Eric Markle. Eric is an enterprise systems developer three at the George Washington University. Eric has twenty-five years of experience in higher education with fourteen years at George Washington. During his career, he's worked in web development, content management system administration, continuous integration, continuous deployment workflows and enterprise systems development. Eric is currently a technical lead for the Banner SaaS modernization project at George Washington and has begun to work extensively with agentic AI to streamline and enhance the development of system integrations. Thanks so much for being here, Eric.

Eric Markle: Thanks for having us. I'm happy to be here.

Sophie White: Great. So you all have some big projects going on. We're not going to talk too much about ERPs and the Banner SaaS modernization, but you have a lot of big projects on your plate it seems.

Brian Zahn: That's putting it lightly. Yeah, for sure. Yeah. I think what you mentioned there, we'll talk a little bit about it just because it kind of was actually the catalyst and the impetus for our work with agentic AI. So yeah, we'd be happy to get into that.

Sophie White: Yeah. Do you want to just jump in with what is the connection or the catalyst to this work? I'd love to hear that history.

Brian Zahn: Yeah, for sure. So yeah, as mentioned during the bio, we are going through currently a major ERP modernization project. So we've had Banner on-prem for over thirty years. So it's been pretty deeply embedded, heavily customized, as you can imagine, being around for that long. And we've been going through the process of getting everything migrated into Banner SaaS. So as part of that, there's a lot of workflow redesign, business process re-engineering, but also a lot of technical debt, meaning technical processes that we've had that need converted into Banner SaaS safe solutions. So as part of that, we had a backlog of hundreds of integrations and reports that needed to be converted. And Eric can speak a little bit more to that. I'm relatively new to GW. I've been here for about a year, so by the time I joined, we were already pretty far deep into the project. And so we were looking for anything we could do, any levers we could pull to really accelerate that development process. It was taking a lot of time. It's a pretty complex tool to do some of these conversions. And so that was kind of the impetus for our foray into agentic AI coding. Eric, do you want to touch a little bit on some of the challenges?

Eric Markle: Yeah, I can talk a little bit about the technical part of it. So we've had an on-prem Banner for years, I don't even know, twenty years or so. And so the project is moving from on- prem Banner to SaaS Banner and you think you're moving to the same system, it's pretty straightforward. But when you're going to the SaaS safe year, we're moving from Oracle database to Postgres. So queries need to change. We're going from coding customizations in PL/SQL to now we're going to a low-code method that they have in place in SaaS to be able to build these integrations. So there's a lot of different ways that longtime Banner developers are having to change how they develop and work with the system. So that was a big challenge moving from these old school ways of doing development for Banner to more modern web development kind of workflow.

Sophie White: Oh, go ahead, Brian.

Brian Zahn: Yeah. I was going to say, and to expand on that, so some of these tools, like Eric was saying, there's the change management part of this. So these tools were complicated to learn. They're not simple and they're low code. So it was really a lot of manual work to do those conversions. So it wasn't just learning the new tool, but it was also just the aspect of, okay, instead of being able to hand code PL/SQL programs, now you're really switching the paradigm to a completely low code platform. So that was kind of another challenge that we were experiencing there.

Sophie White: What were some challenges with the change management element that you found in this? Are there any that stick out to you as maybe the most difficult to manage in this transition?

Brian Zahn: Eric, do you want to talk a little bit about what that was like when you guys first were learning Data Connect and Insights and some of the Banner SaaS technologies?

Eric Markle: Yeah, I think like I mentioned, one of the biggest things is most of the developers that we have been longtime PL/SQL developers. The low-code way that we're doing it is kind of more like JavaScript and switching into a completely different mindset of how you are developing a program, losing direct access to the database and having to use APIs to make those calls and things like that. And also this is not so much for the IT part of it, but our business partners have become accustomed to us being able to customize the application directly to add additions to it, to make changes to baseline functionality. And that's something that we don't have the option to do anymore in the SaaS environment. So that has been a big challenge. How can we rethink how we can do customizations in Banner and also do without them?

Brian Zahn: Yeah. So I think that gives some really good context in terms of what the challenge was that we were facing. So now we're going to fast forward to last fall. So I had been at GW for a few months and actually our agentic AI initiative actually initiated with a proposal that I submitted directly to our deputy CIO and this was born out of a conversation that we were having of anything we could do to really accelerate this conversion process. So like I said, we were already into it, but we had a significant backlog and these integrations, just to give you an idea of the timeframe that it was taking, we're talking six to eight weeks for a single integration to be ported over into this technology for Banner SaaS safe technology. So it was really a significant lift. So the initial proposal that I had put together, I was calling it an intelligent migration agent. I wasn't exactly sure what it was going to look like, but in my previous role, I had had a lot of experience coding with various AI tools, was using things like Copilot, ChatGPT, and Claude. And when I joined at GW, what I saw was that the developers were using some of the web-based LLMs. Gemini, Microsoft Copilot to small translations. So maybe you have a small stipend of code, you want to have it converted from Oracle to Postgres. We should also note that that's part of this transition. So not just going from on-prem to SaaS, but we're also changing our database flavors. So a lot of conversion work that would have to be done manually. So the proposal, as I mentioned, was for kind of an intelligent migration agent and our deputy CIO, and I have to say we have really great leadership and we've had a lot of support from the beginning. And I think the executive sponsorship was really one of the key drivers that allowed us to take this forward. And so we had a series of meetings with a couple of our vendors. So we met with Ellucian and we met with AWS to kind of show them this proposal and what our vision was, which was we should be able to have an agent that we can give what our source programs look like on-prem and it should be able to get us pretty close to that end result without that six to eight weeks of manual work. So that was really what our vision was. And when we met with Amazon and we showed them this, they came back with a couple of tools that they wanted us to evaluate that they thought could help. And really that pushed us into tailored tools for agentic coding.

And so in the industry right now, there's things like Cursor, things like Claude Code, OpenAI Codex. Their flavor of agentic AI coding tool is something called Amazon Q Developer and then something called Amazon Kiro. And they take a pretty unique approach. They encourage you to go through what they call a proof of value. So once we saw the tools, we decided to embark on this initiative, with a proof of value team. What Eric? I think there were four or five of us on that original team and we took a very practical hands-on approach to it. So we set up meetings twice a week for about thirty minutes and we started playing with these tools and the original part was experimentation. So what can we do with it? We weren't exactly sure. So we tried a variety of different use cases. We mentioned the integration work, we looked at that, we had some reporting work.

We tried working with that as well, a couple other tools in the SaaS toolkit. And we were quickly starting to see that there was definitely some value here, but there were still challenges because when you think about how these models work, they're trained on their corpus. They have general programming knowledge, which is very useful. So it knew about JavaScript, it knows about the languages and the frameworks, but what it was really missing was the specific context that we needed. So it was missing some of that GW specific context that was missing some of that Ellucian-specific context. And so that kind of took us into our next phase. Before I get into that, Eric, anything you want to add from the early days of the proof of value and from your experience?

Eric Markle: No, like you said, it was just a lot of evaluating the tools, figuring out how we could move from just general asking chatbots, how can I do this to a more comprehensive approach to it?

Brian Zahn: Yeah. So from there, we then started to look at MCP. So MCP kind of getting into some of the technical details here, but MCP is model context protocol and that provides ways that it's really a mechanism for allowing an agent, which is in one of these tools to be able to interact with some third-party system. And so this was really the key to unlocking the value to be able to get that context that we were talking about that we're missing. So that GW-specific context and that Ellucian-specific context. And so we set up a few MCP servers and this all of a sudden now allowed the agent to not just have access to the knowledge about coding, but it had access to the Ellucian documentation. And once we did that, we started to have some of what we called, in the early days, we were referring to them as “aha” moments.

So different things that we didn't think were possible that now all of a sudden we knew it was. So I'll tell you one of the challenges we ran into with DataConnect, which is the tool for creating these integrations. It's proprietary framework. So we don't have general knowledge. The LLMs didn't know about DataConnect. And we had some skepticism during the early days of the proof of value where we weren't exactly sure. So I'm kind of an optimist and I had been using the AI coding tools for quite a while, I was all in. But we also, as Eric mentioned, some folks who have been doing this for a long time, they were a little bit more skeptical and rightfully so to be honest, but we had some of these moments where all of a sudden we saw it was able to generate a pipeline that was syntactically correct.

At the beginning, we didn't even think that that was possible. Now it wasn't getting us from zero to 100 percent, but just the fact that we proved it could do syntactically correct pipeline development, that was getting us from zero to 25 percent. At that point, we knew we were onto something and we knew that we could potentially really start to streamline this process.

Sophie White: Question, just thinking about the change management element, do you feel like when you were able to prove that proof of concept that 25 percent in, people were seeing that it was successful and accurate, was that helping build trust in this migration or were you still dealing with some skepticism at this point?

Brian Zahn: You want to take that one, Eric? In the early days?

Eric Markle: I mean, definitely from the leadership, we got a lot of excitement about it. I think we saw a lot of excitement from the other developers as well too. We've started to have some other developers that are really trying to get on board with what we're doing with this and see the value in it now.

Sophie White: Okay. Just trying to understand culturally where we are in this journey at this point. So thank you for that pause.

Eric Markle: I think there's always going to be some skepticism from certain developers in the use of it because they've been doing this the same way for a long time. It's hard to change that. And so we're dealing with not only changing how they work with the system, now we're also throwing on top of it having to use AI tools to do that as well. So you're changing how you're working in multiple different ways.

Sophie White: Right.

Brian Zahn: And I think one thing I would add to that is we shared. So from the beginning, we were not doing this as a secret. So this was a proof of value team and as we were getting different wins, and again, I want to share some of the practical things we did. So we were having twice-a-week meetings. We also set up a Microsoft Teams just as a collaboration space and we invited people to it. And as we were having wins, and I could think back, Eric would copy and paste some of the prompts he put in and he's like, wow, look at what this did. I was having trouble with this DataConnect pipeline. I was able to throw this in and I got the answer, something that would've taken me hours before I was able to get back in five or ten minutes.

So I think part of that change management was really kind of evangelizing what we were seeing and socializing some of that. And I think that the more that people saw that, then there was started to get some buzz about Kiro. People were like, okay, well, we kind of want to get involved in this now. We're seeing this and there's some pretty good results. So I think it was sharing some of those early wins. We tried to be very intentional about that from the beginning.

Eric Markle: Yeah. And we get all the time now people are like, "Hey, can you throw this in Kiro? Let's see what it does." I'm like, "Yeah, let's do it."

Sophie White: That's a good proof of concept in itself that people are now coming to you to ask to use it. Great.

Brian Zahn: Yeah. And so one other note I want to make about that is we also had some what I would call unexpected wins along the way. So keeping in mind that our main goal from the beginning was how can we speed up integration development? We were also looking for some other repeatable patterns of things that we could roll out, just things that were, they were happening frequently, they were kind of painful. How could we accelerate some of that? And I'll give you a really simple one, which was just doing DIFs on files. So doing data comparisons. When you're doing a migration like this, one of the ways you have to validate that your conversion was correct is you're going to run it out of SaaS, you're going to run it out of on-prem and you're going to compare that and see if they match. And so it was kind of being done in an inconsistent way previously where some people were counting the number of records, some people were spot checking certain data points and we realized that that's a very repeatable pattern that AI can handle that.

Throwing that into these agentic tools and being able to say, "Go through, write a script that will do a comparison of these files, all of a sudden you're saving quite a bit of manual work and you're applying consistency. And so that was one of those kind of unexpected wins. We didn't really think that that was something we were going to use it for, but we found it. And then I want to share one second unexpected win. As part of this migration, we also had an on-prem job scheduler that needs to be migrated to a SaaS safe version of that. And it's a tool that's a bit, I would say, clunky to use, a bit difficult to use, and we weren't even targeting that with our initial proof of concept here. But as we realized what the tool was capable of, all of a sudden we found this also works for the job scheduler and we were finding tasks that were taking, again, about eight hours of manual work. Somebody might spend a day trying to troubleshoot through this, we'll call them job chains, being able to export that as a text-based definition and feed that into the agent. The agent is able to parse through that with, of course, with the assistance of a human. And that's a core part. We'll get into that a little bit later on. I view AI as very much as an amplifier. So it's not going to solve all your problems if you're not knowledgeable of the existing technology, just giving you the tools, it's not going to solve all your problems. But that was a real big one for us just being able to say, wow, what we're learning, the lessons that we're learning with DataConnect with our initial goal, we can now apply to some of these other technologies that we weren't even initially thinking about. So those were some of those kind of early aha moments that we had and some of those quick wins back in the December time period.

Jenay Robert: Something I'm really appreciating about how this conversation is getting kicked off by talking about these unexpected efficiencies gained and so forth is that I think I've done a lot of research around how AI is impacting higher education and specifically published a report not too long ago about the impact of AI on work in higher education. And one of the big limitations of this research for me consistently is that when I ask people about how AI is impacting their work, it almost ubiquitously folks consider generative AI tools like chatbots. And it's really difficult for us to dig down deeper and find some generalizable findings that relate to these more nuanced, specific uses of AI. And I think as it goes with research, oftentimes when something is so new and so niche, you really have to get into case study basically. And so I'm just really appreciating that angle of this that I think people can perhaps use this story to see how agentic AI specifically can improve workflows that it is being done at your institution.

Brian Zahn: No, I definitely agree on that. So now I want to jump to the beginning of the year. So we've gone through our quick wins, so we had some of those aha moments, but we still wanted to get back to the integration development. That was our core problem. So going back to earlier in the conversation, it was taking six to eight weeks. And although we had added in some of these MCP servers, we were giving it some additional context. It was still only getting us, as I mentioned earlier, about 25 percent of the way. It was syntactically correct, but it wasn't the whole thing. And so this is where we started to really ... We took a step back and Eric and I spent a lot of time together just going through brainstorming, looking at what we were getting and what our challenges were. And we said," We need to refocus on DataConnect and see what can we do to speed this up.

So as we went through this initiative, we also met with other groups within our IT organization. We were not doing this in a silo. So of course, security, that's a big part as well. We brought them on early. We included them in the conversations, told them what we were doing. We showed them the tool. We also brokered some conversations with AWS, so they had a chance to ask them directly, made sure we got their approval and they gave us some recommendations. The second piece is we also have a technical review board. So we made sure to present in front of them and show them what we were doing at this point. We got some feedback and we just, again, wanted to make sure that we were vetting the architecture and trying to get as many different perspectives as possible. And one of the things that came out of that presentation to the technical review board is, again, we have great leadership on that side. We have a chief technical architect who had introduced the concept to me of a harness, which was new. So it was really this idea that we have the agentic AI coding tool, but where is it operating within? We need to give it some framework that grounds it in the GW-specific context. And this was where this concept of a DataConnect harness came to the picture, which really takes it a step further. So when I think about the maturity of it, when we really started, we were going through and it was simply prompting it, you're kind of doing vibe coding. Then we moved into the vibe coding plus MCP, so now we're giving it some external context sources, and then we move into the harness, which really allows us to give guardrails and to tell it and give it some rules. How exactly can you operate with these additional context sources? What do we want your output to look like? And so we spent some time architecting that and designing what exactly rules do we want to build in, what guardrails and things like that. And eventually we got to a point where we had a working harness, and I think I want to let Eric share the story once we got it built, what the end result of this was, and this has been just in the last month or so that we really hit our, I would say the tipping point where now we're at the point we're really getting a lot of value from the tool.

Eric Markle: See, I told you Brian likes to push things out when we hear all these questions.

Sophie White: Keeps you on your toes.

Eric Markle: Yeah, he does. So I guess what Brian was talking about is one of the big moments that we had recently was there was a pipeline conversion that I was working on. I was taking an original PL/SQL program that ran through a process and generated PDF letters. So this is something that I estimated would probably take me manually to do at least two solid weeks of work. After we had the harness in place with the knowledge base of how we have samples of working pipelines already, we have the information about all the APIs that are available to us and things like that, the documentation for the pipelines. After I would say about four hours of prompting back and forth using the Kiro tool, I had a 95 percent working DataConnect pipeline. It still has a little bit of work to go, but it took the Jasper report, which that in itself for me to recreate. So it's a very structured form letter that has to be in a specific format that they can print out and send to people. For me to be able to do that manually to convert it from a Jasper report to HTML that then generate into a PDF, that in itself would've taken me days of work to do to get it in the exact same format as before.

Sophie White: What's a Jasper report? Is that a GW-specific thing?

Eric Markle: No, it's kind of a tool they use for Java programs to-

Sophie White: Okay.

Eric Markle: Yeah. So it's like a markup language to do PDFs in Java basically.

Sophie White: Perfect. Thank you. You're seeing the limits of my technical expertise here, just trying to translate for folks who are less technical and maybe not be listening to this.

Eric Markle: And I said it was exciting to see something that would've taken me two, three weeks of manual work in four hours I had mostly done. It's incredible. Yeah, it is pretty incredible.

Brian Zahn: And to me, that's when we really, when we think about what this vision was, again, it started with this initial proposal of an intelligent migration agent. We weren't sure what we were going to get. And I just think to get to the point where we're able to consolidate roughly sixty hours of work down into four hours of focus development time, I mean, that was huge for us. And to me, that really showed us the power of what these agents can do, but it's not just about the agent. It's when you have an agent that's grounded in the specific context that you need for the particular use case. And I think that's, to me, that's the takeaway that I think other institutions can take back is that if you take the time to architect the environment correctly and you don't just say, "Okay, this is the agent. I'm going to ask it a generic question." If you really take the time to think through all the different contexts it needs. And one of the things when we were building this, I have to say, we spent a lot of time just going through and thinking through what are you currently doing manually, all of those manual steps because that was the key. You can't expect an agent to do what a human can do manually if it doesn't have access to everything that the human has. That was really the core part. So we had to map out all those steps and then make sure that the agent had access to all of those and it had those grounding contexts, those steering files, all of that defined just like the human would. And it was really, that was the value unlocked for us to where we were really able to see that.

And now that we have this capability, of course, this is where it gets more into change management. So we've been working with a small group as we've been developing this. So now how do we scale it? And one of the phrases that I kind of used from the beginning was we want to make this be a force multiplier. It shouldn't just be in the hands of a few people. So we want this to really accelerate all of us and bring us all toward the finish line. And so this, we call it templatization is kind of another way to think about the harness. We're looking at rolling that out to additional developers. And we took an interesting approach here. So we have given out about twenty to twenty-five Kiro licenses to our agentic AI tool. And from the beginning, I mentioned we weren't doing this in secret. So we've been very open. We took opportunities at our technology focus groups where we presented on this, some of those early ones, as I mentioned, we showed folks and then we would get requests after that somebody would say, "Hey, can we get a license for this? " And we granted that and then let them start to play around with the tool. So they got familiar with it without that harness. And now the next step, and I think this is where we're really going to see, again, more value be unlocked is folks already know the tool, now we're going to roll out the harness and get them on board with that. And that's when we're really expecting to see, again, continued acceleration of our development timelines.

Eric Markle: I just want to say, I think it's exciting too because we're only three, four months into this process. We've only been using the harness for three weeks maybe. So the fact that we're already getting this type of return with minimal training to it and examples of working pipelines, I'm kind of excited to see how far this can progress. I don't ever see that it's going to completely take over and just be like, "All right, go convert this for me." You still need developers and analysts who know what the programs do and be able to analyze them and make sure that they're operating properly, but it's a huge time saver.

Brian Zahn: And I would say one other thing also to keep in mind is that we don't see where the gains are going to be evenly distributed. And so what I mean by that is if we give five or ten developers access to this tool within this harness, we don't expect to see the same return from everybody because again, it really is an amplifier. So somebody like Eric, Eric really knows DataConnect well. He's been working on it prior to bringing in this tool, we're expecting to see better results from those folks. So that's been one of the challenges though, the change management is how do we bring everybody along for the ride? And just a couple other ideas and these things might be valuable to other institutions. We did partner with AWS where they came in and they actually gave a workshop and we opened it up to all of IT. So it wasn't just within our enterprise applications area. We posted it in one of our general channels, and we had a great turnout. Eric, what was it? Thirty-five, forty people? Might've even a bit more. Yeah, I mean, it was a pretty significant number of folks that were interested and they came and it was really just showing what the tool can do. We've tried that. We're also looking at setting up a community of practice in the future as well. We want to share some of these lessons.

One of the cool things about AI, and I love AI, you could probably see I get excited talking about it, but everybody uses it a little bit differently. And so I learned from Eric, Eric learns from me, a lot of these sessions that we've had, at least my methodology is I like to sit with somebody one-on-one and one of us will share our screen and it's kind of like going back to the days of pair programming. I like to watch exactly how people prompt. I can give suggestions. We all do it a little bit differently. So even that's been helpful.

I think what isn't effective is just giving a large presentation for an hour, letting people watch you, they hear you talk and they're really not following. What I've found with AI, especially with coding, because there is a pretty steep learning curve to these tools. They're not simple. I mentioned chatbot earlier. This is not just a chatbot that you can enter in and say, "Okay, convert this program." There's a steep learning curve. And I've found success in having these one-on-one sessions, really the pair programming approach of just let's talk through this together and let's see what we can get it to do. And it really seems to have worked and it gets people excited.

Sophie White: Yeah. I love that really practical collaboration that it sounds like you're doing with the tool of let's do a screen share and share prompts together to make sure that we're using it in the best way that we can. And I'm curious, Eric, about something you said earlier. Obviously there's a lot of discussion about AI replacing jobs and I could see from the development standpoint that that is a concern that's very valid of if you create this thing that is able to take some of the hours out of your previous work, that could be concerning, but it sounds like you're also kind of harnessing and unleashing this AI tool in a way that enhances the work that you're doing. What's your perspective here as a developer and what do you see other developers at GW saying if you feel comfortable speaking to that?

Eric Markle: I can't really say what the other developers are saying. I know for me personally, I don't see any time in the near future where AI is going to completely take over what developers are doing. It is going to fundamentally change how we code. I think just from this short time period of working with these tools, I can see that there's going to be very few instances where I just go in and just start writing code anymore. I'm going to go consult these AI agents and use that right from the beginning to help me build out specs. I'm very much a vibe coder at heart. So whenever I get a problem, I'll just go in and start knocking out code and not really think about it and plan it. And I think specifically with how AI has progressed, the spec-driven development has been a big game changer for me where I can go in and explain the problem to AI or have it analyze an existing program. It'll generate a spec for me that tells me exactly what the program does, what we're looking to accomplish and then it can generate a task list for me and then we can step through every task within that list to actually finalize the program and the work that we need to do. And that's something that I wouldn't have done previously as a developer just because that's not how I work, but I really see the value in that and that's really helped my coding out.

Brian Zahn: So if I can add a little bit to that, Sophie, I think you raise a good question, which is, so what's really the impact on the developers? So I think I have a little bit of a different view, which is that we have such a demand for coding and such a demand for new applications that I don't see this getting rid of jobs. I see this allowing us to deliver a lot more value and to deliver value quicker. I think the more that we're able to output, we're going to have more demand coming right in the backlog. So that's kind of how I see it. And then I think it also allows us to reposition some of those resources to the more important things, the things that the students care about. Eric's heard me give this example, but one of the processes that we had to convert, it was one of the payroll processes that generates these direct deposit files and that might take sixty or eighty hours and really the students don't care about that. The employees do, but the students don't care about that. So I think if we can get to the point where these coding agents can take some of that administrative burden off of us, that's going to really unlock value. And I think we could build, again, things that are going to be more relevant to the students. We can build experience cards within our portal that surface data to them in a more effective way. So I don't think it's about getting rid of the work and getting rid of jobs. I think it's about allowing us to operate and do more value-add work as opposed to just administrative work.

Jenay Robert: That kind of gets at the question I was going to ask, which is where this technology goes next for you and for other institutions. I mean, you've got a use case or two under your belt now. And so what are some of the specific things maybe already planned, maybe just dreams you hope you can someday accomplish with this? What's next?

Brian Zahn: So we've already started to form a backlog of additional harnesses that we want to build, and that's what's coming next. So we're really focusing right now on the integration development, but we want to build these for additional technologies in the SaaS environment. So that's kind of our short-term goal. The other goal that we're looking at is the model that we're building within enterprise applications we've been doing in conjunction with our chief technical architect, we want to expand that capability across GW IT. So not just within the enterprise apps, but we want to expand that to other use cases and to other teams and really allow what we're building to be a model. So that's where we see it, at least that's where I see it going next. Eric, what do you think? Any thoughts on that?

Eric Markle: Yeah. And I mean for me a little bit, it's hard to judge where it's going to go from here because part of the reason that we're doing this now is because we're helping us to get to an end goal of going live in the banner SaaS environment. So we're doing this work in conjunction with the huge backlog of work that we have to get to that endpoint. So I think, thinking of it larger term outside of that has been a little bit on the back burner for now, but hopefully we will start to plan that out a little more.

Brian Zahn: And so one thing I would say also, you asked about other institutions. So I think that this is something that's coming for everybody. I think that part of why we're early on and why we're kind of ahead of the pack is to Eric's point, we had a catalyst and our catalyst was we made a strategic decision to move our ERP to SaaS. And as part of that, we needed to accelerate that timeline to get to go live. I think a lot of other schools are probably slightly behind because they haven't had that catalyst. But from my perspective, the world has shifted. Eric touched on this earlier. I think the days of coding things manually, they're gone and everything's going to be shifting toward this. You still have to develop, you still need to have good, strong engineering background, strong engineering principles. None of that's going to go away, but it really is something that I think once institutions have the pain, they have that need, they have to accelerate, they have to develop quicker, that's when they're going to start using these tools.

What do they say? Necessity is the mother of all inventions. So I think that's the way things are shifting from my perspective. And also something that Eric touched on earlier, and I don't know if you want to expand on this, but spec-driven development. When we just think about modern development workflows, that's a whole kind of a separate piece from the agentic coding. Eric, do you want to share a little bit on that and what you've seen with your background in software engineering?

Eric Markle: I mean, just like I mentioned, a lot of the talk with AI, you hear about vibe coding. Anybody can come in and just start typing in prompts and it's going to generate an application for you. With the spec-driven development, it does a lot of the planning work upfront for you. So you'll give it information about your project and it will generate a plan for you. You can review that plan before you start any work. You can make modifications to it and make sure that the plan is sound based on what your knowledge of the project is. It can generate test cases. So if it knows what the outcomes are supposed to be, it can say, "Okay, here's what we're expecting at the end of this project that this program will do.” And then it can use that to not only build the program for you, but it can also build the test cases to validate that the program is doing what you wanted it to do from the beginning.

So it's been a really helpful way to develop for me to organize my thoughts because that's not really how I think.

Sophie White: Yeah. The task list I thought was really interesting here. And I'm curious, you've talked about some of the internal stakeholders. Obviously, security is a big concern here and I'd love to dive into that in a bit, but I'm also curious about, this is a really complex project that it sounds like involved Amazon and Ellucian as well. Do you have any advice for institutions on how best to partner on a big initiative like this if you have a third-party vendor that is really integral to the project?

Brian Zahn: Yeah, I think that's a really good question. So when we put together the initial proposal, one of the things that was included in there was a value matrix and it was really showing what's the benefit to us, what's the benefit to GW, what's the benefit to Amazon? What's the benefit to Ellucian? What I see is this problem that we're solving is not unique to GW. And as I mentioned before, I think we're early on because our hand was kind of forced by being an early adopter to SaaS. So I would say those partnerships, I think it's critical. And we had developed and Eric has too, a really good relationship with our solutions architect, and we had a lot of meetings from the beginning where we were just, I think some of those initial meetings were just whiteboarding and just brainstorming, thinking out loud, what's the problem that we're having?

I always go back to it's really important to map out the workflow. That was so critical. And once we map that out, we share that with our vendor partner. They need to be able to see that so that they can tell us what tools do they have available that could fit that. So I think for us that that was definitely key in getting things moving forward. And then just as the project has continued, we've kept our solutions architect involved. As I mentioned, he was willing to come in and gave a workshop to GW IT. We meet at least, I would say usually once every week or so just to kind of touch base on how things are going. We've even thrown back some problems that we've run into where, we're trying this, we're not quite getting the results we're expecting. And he's been able to bring back some other solutions consultants that they have who are more knowledgeable about Kiro. So I just think keeping that relationship and open line of communication has really helped us because that's definitely something I would recommend for other schools. But I think those vendor partnerships keep them close and keep them active.

Eric Markle: Yeah. And I don't know what ... No, go ahead. Go ahead.

Sophie White: I was just saying it sounds like you kind of co-created that workflow together with them from the beginning so they had an expectation of what you were doing instead of bringing them in later. We have this vision, can you support it? So I really love that framework of working on the foundations first together. And sorry, Eric, I'd love to hear what you want to say.

Eric Markle: I don't remember now.

Sophie White: Oh, no.

Eric Markle: I will say, and institutions should not be afraid to reach out to peer institutions to find out what they're doing with these things as well, because I think we've all worked in higher ed for a while and other institutions are generally pretty good about sharing what they're doing and how they're working with other peers.

Jenay Robert: You may have just opened the floodgates when we publish this. We'll make sure that your email addresses and your personal cell phone numbers too, right, Eric? You want people to just text you with their questions? That's what I was hearing.

Eric Markle: We'll give them Brian. He likes to talk.

Jenay Robert: There you go.

Sophie White: All hours of the night. Perfect.

Brian Zahn: But I mean, so I think to that point though, Eric's right. And I would say again, not just externally but internally. So make sure you're not living within silos within your IT organization. And I think about the success of this, I think and part of that was the vendor partnerships, but also partnering within other groups within IT. As I said, the harness concept originated. It was really a result of having that presentation at the TRB and being able to bring in some external perspectives. So I would also encourage folks talk across your organization, your IT organization, and see what people are doing, share and collaborate. I mean, that's when innovation happens. When you get differing viewpoints, you get people working together. So I think that was one of the things that really helped us.

Eric Markle: And I'm going to be interested to see, I mean, we're a larger university, so we have a pretty large IT staff compared to a smaller school. How is the proliferation of AI and agentic tools going to differ for us as opposed to a smaller IT shop that only has maybe ten people where they can all be on board working the same way. We have a large variety of people and how they work and what they do. So we're definitely not going to have a one-tool-fits-all for every part of our IT department. So it's going to be interesting to see how this kind of flows out across the entire organization.

Brian Zahn: And one thing I'd say that too about tools, we've even seen this internally where in terms of the specific agentic coding tool, there's different tools that are being used. And I've talked to some other universities. Everybody's using different things, but the model that we've shared during this podcast, this idea of harnessing it, giving it access to this different context, building guardrails, this can apply to any tool. And I think that's one of the beauties of this model and that's why we're trying to share this is that it's really modular and it's plug and play. So if things change in the future, want to use different models, we need a different tool, that can happen. And so I think that just keeping that in mind. Don't get yourself too attached to a specific tool, a specific technology. Think bigger picture. Again, I go back to the engineering principles, architecting things. You have to build a solid foundation. These tools, one of the dangers of them, Eric's probably going to laugh and we saw this is they can be very confident. And even at the beginning, we saw where they were very confident. So we're asking if they convert this pipeline, come back and say, sure, it's done, but it's not done. It does not look correct. So it's maybe 20 percent of the way there. And we even see, again, I find it kind of funny, but sometimes we'll catch it where it just makes something up. We'll say, well, why do you put that in the code? Well, honestly, I just guessed. I wasn't sure, but we've been able to get away from that again by giving those steering documents, harnessing it. That was really the game changer and we can reduce some of those hallucinations and guessing. So that's something else I would definitely encourage folks to do as they move forward.

Some skepticism is good. We talked about that at the beginning. I'm all for being skeptical. I've used these tools quite a bit. So you have to keep that in mind. You don't want to be overconfident, but it's a balance. I mean, developing trust in the tools, it comes with time and again, with that proper foundation.

Jenay Robert: And it just underscores what we talk about so much here and in other conversations about AI literacy being kind of the foundation of this. You're not going to be able to embark on a project like this without really understanding what those limitations are and continuing to maintain that the human is smarter than the machine and you have to play the role that you're supposed to play. This is not a tool that you just offload all of your work onto. That's so key. And I wonder if that played a big part in some of the conversations with other stakeholders. Do you feel like people perhaps peripheral to the core team that was working on this project understood that element of it, that the time it would take to properly train the model, the importance of how you're working with the model, or is that something you had to bring people along with?

Brian Zahn: That is a very good question. And Eric and I were actually just recently chatting about this where it requires time. And so that's been one thing that we've continued to communicate as we've given these presentations and communicating to the leadership is that there's a learning curve and you have to make that investment to get up to speed with this. The world's shifting very quickly. These tools are coming out fast. Some of these things we're talking about, spectrum and development, that's its own topic, very large, even separate from the agenic AI. So I would say you need to make that case to your leadership to be able to have some of that time that you can spend investigating these tools and the experimentation and being able to just do some personal learning to get up to speed with this tech. Anything else to add there, Eric?

Eric Markle: I would say I don't think when we went into it, we even knew how much time it was going to take or what we were getting into. We didn't know what direction it was going to go, how well it was going to work. I think getting into building the harness and the knowledge of that was a big deal for us and that's something that will take time for you to build properly, but it pays off in the end the time that you put into it.

Sophie White: Fantastic. I think this feels like a pretty good place to wrap up, unless there's anything ... Is there anything else, any other advice you have for other institutions that you feel like we haven't covered at this point and you want to leave us with?

Brian Zahn: I have a couple key takeaways. So I think the first thing I would say is you've got to identify your catalyst. That's the biggest thing. So you've got to find what's causing you pain and where do you want to apply this? And I think once you have that, the second thing is the executive sponsorship. And that was critical. We had the support from the leadership from the beginning. They're the ones that can help broker some of those conversations with security. They're the ones that can approve the funding to get the tools, and they're the ones that can help protect some of your time to be able to push forward with the initiative. So those would be my two big things. Identify the catalyst, find what your pain point is, and then get that secure the executive sponsorship early on.

Jenay Robert: Great.

Eric Markle: And I was just going to say don't be afraid to adapt and accept different ways of doing things. Don't stick with the way you've been doing things for twenty years because that's how you know how to do it and that's how it worked. Be open to trying out these AI tools and learning how to use them properly and you'll see the benefits of it.

Sophie White: That's a fantastic insight. And thank you, Eric, I think for your honesty of just being open to this change. It's not always easy and we know that change management is really sticky work in higher ed sometimes. So thank you for sharing that mindset firsthand. And one thing you mentioned just before we wrap up, I think I have this cybersecurity hat on because we were just at our cybersecurity conference with EDUCAUSE and I'm thinking if I were a cybersecurity professional listening to this, I might be a little queasy of we're working on this solution that's managing really sensitive information like payroll documentation, et cetera. Do you have any, I think, advice for folks who are in security and might be worried about something like this about how you address those guidelines in the development and the harness that you ultimately came up with?

Brian Zahn: So I would say I think a couple of things to that. So one thing is if you don't do anything and you just let people use their web-based tools, then you're lacking enterprise control overall. So then you're lacking visibility, people are going to go and do what you essentially do what they want. So I think when you do roll out an enterprise tool such as Kiro in the way that we did, we get some of those enterprise controls. So it's all through the identity center, so they're logging in through their institutional credentials. We have auditability, we have logging enabled through that. And I think the other part is just encouraging and teaching developers safe ways to interact with the models. There's always that education part of it and we always push that. You shouldn't be copying and pasting API keys into your prompts or pasting in sensitive data. So I just think that education piece is really important. And then I would also say the tools themselves do provide some security measures and security controls built into them. I mentioned some of them earlier, but also assurances that any of your data that you're sending, they're not training their models on your data. We get that with our professional subscription to these tools. So I just think getting in front of it, being able to roll out an enterprise tool like this, you do get some of those data privacy and data security controls that you otherwise wouldn't if you get on this and people were just using the web-based tools. So that's kind of my perspective on it.

Sophie White: So that's helpful. So it's an education piece and then it also sounds like you're intentional in the procurement of the tools and the way that you work with these solution providers as well to make sure it's fitting the needs that you have. Okay.

Brian Zahn: Absolutely. And we've also asked our partner, vendor partners for different, again, guardrails is kind of what we call them. So inside that harness, we tell it certain things it can do, certain things it can't. So that's another way you can harden your coding tool.

Sophie White: Awesome. Thank you for that. Another example of change management of I'm putting my skepticism in and then you're helping walk me through how to address the skepticism in the name of security. So thank you.

I think that is all the time we have, but thank you so much for sharing your lessons with us and how exciting this project is. I think this will be really valuable to other institutions as we're looking at how AI folds into the ways that we work in higher education and technology. So thank you so much for being here.