Chance Favors the Prepared

Benjamin Franklin observed, "In this world nothing can be said to be certain, except death and taxes." When dealing with higher education institutions, we can also add a lack of mature, tested disaster recovery plans (DRPs) to this proviso.¹ Often viewed as a heavy lift for an organization, disaster recovery planning can be a gateway to a broader institutional understanding of IT, a tool to clarify crisis decision making and help foster organizational cohesion.

It can be challenging for even the most successful CIOs and senior managers to engage campus leadership in conversations about the cost of redundant systems, or cloud storage versus on premises. Disaster recovery planning puts a fine point on these issues: how long can you function without access to your primary network file share? What is the impact of your admissions system failing during the spring? Can you maintain NCAA compliance if your athletics department's systems are destroyed by a natural disaster? Raising these questions — and the ensuing discussion — helps form the basis for a DRP.

It is not enough, however, to merely prepare a plan. Plans must be reviewed regularly, kept current with changing conditions and personnel, and tested from time to time. In fact, testing a plan might be one of the most important things you can do after the plan is initially drafted. Testing the plan gives you an opportunity confirm (or deny) any assumptions you made during the initial planning process. Testing also gives you a chance to identify and address any gaps that you find in your plan.

You can test a DRP in a number of ways. One of the most extreme methods is to conduct a full interruption, where an organization acts as if an actual disaster had occurred and moves from all of its primary systems to its backup systems. Such a test is usually not possible for most institutions. Another, more lightweight way to test a disaster recovery plan is to conduct a tabletop exercise.

A tabletop exercise sounds just like what it is: A scenario-based exercise where key personnel discuss the actions they would take when responding to a simulated event. These exercises can be engaging and surprisingly easy to develop. At Brandeis University, our own IT-focused tabletop exercise involving the loss of the learning management system initiated a host of conversations that contributed to a complete refresh of our university's crisis communications plan. It also led to the creation and implementation of a formal incident command structure based on FEMA's guidance.

If you want to plan an IT-related tabletop exercise at your institution, below are some considerations:

• Be sure to plan plenty of time for an on-campus event. The exercise could generate a lot of conversation and interesting discussions.
• Be sure to scope the exercise appropriately. You might want to say that some scenarios are "out of bounds" for the purposes of the event, like secondary IT systems or business processes.
• Place university leadership in one location and operations staff in another during the exercise. This separation will give you a chance to simulate how communications between those groups might take place, and it will help identify communication breakdowns.
• "Drill, baby, drill!" Run microtabletops often to make sure that key personnel are familiar with the institution's disaster plan processes and procedures.
• Leverage your campus emergency response plan format and structure so that IT operations feed into that structure seamlessly.

Louis Pasteur wrote that "chance favors the prepared mind." While he was referring to complex scientific problems, this concept also applies aptly to disaster recovery planning. Preparing, testing, and refining your institutional disaster recovery plans will serve you well in the event that a disaster strikes.

Note

1. Only 35% of U.S. institutions have tested their data center disaster recovery site in the past fiscal year, according to the 2015 EDUCAUSE Core Data Service, Module 5, Question 14.

---

Michael Corn is the deputy chief information officer for Brandeis University.

Joanna L. Grama is director of cybersecurity and IT GRC programs for EDUCAUSE.