Absolutely Erik. I want to help others as much as possible as it really is a horrible thing to have to go through.
We have recovered very quickly imo and the systems we did have in place helped us to do that. But we could have avoided huge amounts of pain if we had better DR systems and plans in place. From: Erik Goldoff [mailto:egold...@gmail.com] Sent: Monday, 31 January 2011 9:54 PM To: NT System Admin Issues Subject: RE: Some thoughts for your DR Plan I'm sorry you had to experience this, but glad you didn't experience personal catastrophic harm. I belong to the Southeastern Continuity Planners' Association here in Georgia ( www.scpa-us.org<http://www.scpa-us.org> ) and I'd like to share the content of your message below, but wanted to ask your permission first. What say you ? Erik Goldoff IT Consultant Systems, Networks, & Security ' Security is an ongoing process, not a one time event ! ' From: James Hill [mailto:james.h...@superamart.com.au] Sent: Sunday, January 30, 2011 9:06 PM To: NT System Admin Issues Subject: Some thoughts for your DR Plan We now have the majority of things restored and up and running. Below are just some initial thoughts and ideas that I wanted to share with the list. It is in no way any form of DR plan nor is it meant to indicate what we did or didn't have. It's simply my experiences from our recent DR experience written down for the benefits of others. Some or none of this may apply to you. I certainly do not regard myself as any form of DR expert nor am I the first to have been through a real DR experience. However if I am able to provide any info that can assist others than I am more than happy to do so. * Don't ever think it can't happen, it can. * You do need a DR location, a live one if possible. Convince management of this! * Build redundancy into your designs of everything. Thanks to this all our stores were able to continue to trade even though the data centre was under water. * If you have something in your environment that isn't in your backup schedule, add it now, no matter how small it may be. * Consider that staff with specific duties in your DR plan may not be able to assist as they are tending to their own personal issues or physical access is simply not available. * Services you take for granted may simply be not available. There were power outages (some for weeks) and communication network outages. Phone systems quickly become overloaded in a Disaster, especially mobile/cell networks. * Make allowance for the following in your DR location(for relocation of office staff) o Furniture for staff o Computers and comms o Power, can the circuits handle the extra load you will be adding to the site? o Bandwidth o Air conditioning/heating * Have remote visibility of your data centre and its surroundings o A camera or two would have shown us the level of the water and we could have saved much more equipment. * Add sensors to your data centre that shuts off the power if water is detected. * Exchange cached mode and offline files provide quick access to much critical information. * Keep critical infrastructure/server build/networking documentation in multiple places. o I had a recent backup at my personal residence. It was invaluable in the early stages of our Recovery. * Data restores o Do test restores regularly. Environments change all the time and maybe something hasn't been added to the backup list for that server. o Ensure that you can retrieve critical data quickly. Restores take time. o Tapes - do anything to avoid them, if you have to use them have multiple tape drives available so that restores can be conducted more quickly. o Have backup backup servers. Especially with the tape catalogues available. We saw cataloguing of tapes take 14 hours plus. o Have an offsite location authorised as a delivery point with your Offsite Tape holder. * Check your emotions at the door. Remain calm and logical, consider others needs. The people that are true leaders(that doesn't necessarily mean all Managers) should be running the show. Everyone else will be looking to them for guidance. * Fire and water make fantastic servants, they are horrible masters. James. ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
