A Disaster Recovery Discussion: Power Outages Lessons Learned.
The recent power outages in and around the Dayton area sparked an extended disaster period for many companies and many of our clients. DTG feels that is our responsibility to address this issue and hopefully inspire all of us to take some time to evaluate what happened and how we can avoid big problems in the future. What did these outages do to your business? This event warrants a second look and follow-up meetings should be conducted in a “Lessons-Learned” format in order to promote new discussions (or renew old discussions) about Disaster Recovery.
This recent event hit hard for a lot of you and points to the importance of a solid and tested Disaster Recovery plan. Some of you already have these Disaster Recovery plans in place but the recent power outages have brought us all into conversations about a new version of a “disaster”.
The power outages were fairly unpredicted (as the weather predictions underestimated the affects of the remnants of Hurricane Ike) and showed us a “Gray Area” of Disaster Recovery.
Many of the outages came with extended periods of UNKNOWN recovery. The worst part of this event was NOT KNOWING when the power would be restored. This facilitated confusion on whether or not to declare a disaster or not.
The gray area that I keep referring to stands as follows: Some of you have Disaster Recovery and data backup plans that hit in 3 main areas:
- Protect the data EXISTENCE with tape or off-site backup plans;
- FAILOVER or REDUNDANT servers/services; and/or
- LONG TERM Disaster recovery plans that planned for a KNOWN long term outage.
Being that power restoration was unknown, it was unclear whether declaration of long term disaster should be declared. A lot of these long term outage plans require multiple day turn around for getting services back up and running either at hot sites or other locations. Doing so when power came back in a day or two would have meant your business would be down for longer than necessary.
So we all saw that gray middle area of medium term Disaster. This should spark a new conversation about DR and this medium term should be taken into consideration.
Some questions that should now be asked:
Is it worth the risk that you faced recently to justify you implement a plan for this type of medium-term disaster?
Should your current Long-term Disaster Recovery plan be redesigned?
Should you redesign ALL disaster recovery plans to cover more scenarios….all scenarios ….of short, medium and long term disasters?
If you are one of the many who do not have ANY Disaster recovery plans….this exposed the value of those plans and we were shown first-hand how important they can be for your business. DTG can help with this problem and get you on track and fully deployed with a Disaster Recovery SOLUTION and a Disaster Recovery POLICY / PLAN.
If you are those who have DR solutions but no Policy/Plan…… you saw firsthand how having a DR solution is one thing but not having a tested PLAN of ACTION meant that the Disaster Recovery was chaotic at best. Without a clear plan the disaster can be chaotic and ineffective even with a good backup solution.
Lessons Learned from these outages is something that every local business should conduct and DTG can help you conduct these meetings so that you come away with clear direction on what you should NOW do in moving forward with Disaster recovery plans.
Remember that Disaster recovery is different from business to business – we can help you evaluate what you should target, how you can target, solutions to solve Disaster Recovery and plans that you can follow WHEN this happens again.
So, in conclusion here is a summary of some of the global and overall lessons learned:
- Management/Executives need to meet collectively and discuss lessons learned and disaster recovery needs
- Need to have a DR policy/plan and solutions to implement them
- Medium range or unknown outages should be planned for if they negatively impact your business
- The DR plans need to be tested and everyone needs to be properly informed of their roles
- If Medium range or unknown outages are important, you should consider redesigning your current DR policy/plan to cover all scenarios
- If you do not have ANY DR policy/plan it is important to have one. This is a “process” that you must go through…..DTG can assist you with this process, the policy/plan and the solutions.
Please let us know and we can meet and discuss these Disaster Recovery must-haves!
By: Brock Kiser
Dayton Technology Group, Inc.