Hot Diggety! Seay, Paul was rumored to have written: > Ask them where they were on 9-11-2001. Are they totally brain dead?
Ahhh, so that's what you referred to in passing in the other post. That's all right, and understandable. I have a first rate appreciation of this. If you'll allow me to indulge briefly on a tangentially related (but not completely) issue on this list, just once... I used to be a VMS admin. Best, most robust OS that I ever worked with - probably true for the IBM mainframes but didn't work much with them, alas. (A little OS/400, DOS/VSE, and one or two other related OSes) Anyway, come post-9/11, a *lot* of financial firms were in a world of hurt. The ones who planned and re-tested over and over again, each year, for an alternate site a good distance away from NYC, was able to reopen for business only a few days later. Many were based in NJ or about an hour west/north of NYC... one was even based not too far from home, their DR site being about 4-5 hours northwest of NYC. Around this time, I heard that Compaq (company that bought out DEC) was making a lot of frantic calls all around the country seeking out high end machines such as the AlphaServer 8400s and VAX 7000s...that had been discontinued for perhaps 10 years since, because a lot of customers were suddenly calling in for warranty replacements (under their expensive support contracts) in NYC and DC -- you can guess what kind of "customer" it was in DC. How desperate was Compaq? They were calling up even third level resellers of used equipment that they would normally never ever think of talking to. Compaq was in a nasty hole, because they had run out of set-aside reserve spares. Fab plants *long* since shut down...they can't just "take the original plans and re-fab", since the engineers no longer there... I'm not sure how they eventually resolved that... probably offered newer machines to customers and provided migration assistance at Compaq's cost, is my guess. But what the bean counters don't realize is that it doesn't take a catastrophic national event to mean a bad effect on the business bottom line, which I find unfortunate. Can be all sorts of more 'mundane' (albeit not very common) events such as that train which burned in a Baltimore tunnel and closed a part of downtown near Oriole Park at Camden Yards. My company (used to also own a telco) was personally affected by an homeless man burning something in a former abandoned railroad tunnel that melted fiber optics and took out OC-12 to the area for 12+ hours, with a nice number of servers based out of here. It doesn't have to be a corporation for a nasty disaster to mean bad things for their bottom line. I am very well reminded of a colossal failure at an academic institution almost a decade ago that was a chain of events ultimately resulting in failure of a critical drive in a RAID-5 array, and the tapes weren't really usable for recovery...which they found out the hard way. An entire semester of classwork was effectively disrupted, with much data lost, before they were finally able to convince DEC to send out the very best people to recover about 80% off the RAID-5 array through some custom work. So many classes, projects, research papers, etc. were affected that it just simply isn't funny. Same place where if the IBM mainframe ever went down, school was closed for the day. (Happened only once ever, to best of my knowledge.) ...and that is truly unfortunate, that the people who are actually tasked to make things happen, like us, understand and appreciate, whereas others higher up may not share the same view, knowledge, and experience. In a D/R scenario, it also behooves you to know your power sources, how they kick in, at what levels, how fast/when, evacuation plans, how to config PBXes, have emergency equipment handy (eg flashlights), and a million other details. Hardware that can be quickly hooked up/activated, written step by step plan nearby, software CDs handy if needed, dry runs done, backups/restores/app operation verified, and all of this tested once or twice a year depending on level of need and impact, etc. Still, I resolve to do my best to do whatever I can realistically do. :) With that said, I now return you to the normal *SM discussions. ;) (with the reason for copy stgpools driven home ;) ) -Dan Foster IP Systems Engineering (IPSE) Global Crossing Telecommunications