--------------------------------------<snip>-----------------------------------------
Patrick Lyon wrote:
On Fri, 3 Dec 2010 11:45:12 -0600, Darth Keller
<darth.kel...@assurant.com> wrote:
<snip>
You're now on battery power and have 23 minutes to power everything off as
gracefully as possible.
</snip>
I, unfortunately, have lived it. Minus the 23 minutes.
We had a normal power bump to our building. Happens once in awhile, right?
Lights go off for a second and back on, PC reboots, you've recovered from
that sudden shock it gives you.
So you decide to go into the computer room, "just to check", like you always
do...
You enter to a dark room with only emergency lights on. Dead silence... Well,
almost dead, but dead compared to what the room usually sounds like.
You walk through the computer room, through the spotty darkness. Wait -
you hear something! Something is running! You hear fans! You walk towards
the sound... It's your disk drives thank goodness! You remember they have
their own internal battery backup! And within minutes you hear them slowly
spin down to silence. Now what? What do I do?
The Scenario: The building took a power bump. The UPS's had not been
maintained by building services appropriately. They note the power hit,
switch from local to battery. Only, the batteries fail.
Building services manually switches the UPS's over to local, you check all the
breakers. You have management standing over your shoulder asking when
everything will be back up again. You don't know - you don't know if there
was any data corruption. You don't know if there will be a hardware failure.
You just. Don't. Know.
If anyone else knows that feeling, you know what I'm talking about, half
wondering if you should do "something" or go blind, I hear ya. You wonder if
you are going to work for a week straight trying to fix things. You wonder
where that copy of your resume is.
Luckily we did not suffer any issues. Apparently the disk drives were able to
de-stage all the data from cache to disk before it's battery ran out.
Everything else, 3480's, 3590's, 3745, 9672, all dead.
Luckily (again) we are a somewhat small community, and you know your IBM
CE personally. You hunt and fish with him, know his family, invited to their
kids weddings. You call him up directly and tell him to get his ___ over here
pronto.
He shows up and starts powering up everything. The disk drives are already
spinning and alive. Everything initializes. The 3480 auto mounts click in
succession. You have finally POR'ed the CPU. You IPL. DB2 and IMS do their
backout. You restart batch jobs that were running.
We may have expirenced a christmas miracle, I don't know. Just know I
thanked the big man upstairs that night when I went to bed.
But to answer your question Darth, no, we have no real "T minus what the
heck to do" sort of document created. Absolutely should.
Hope you have enjoyed Friday story time!
--------------------------------------------<unsnip>-------------------------------------------
I had almost the exact same experience during the "CHICAGO FLOOD" of
1992, only our power was down for 2 weeks! Edison dropped the power
without warning to keep water leakage from causing transformer
explosions and we didn't HAVE a UPS. Edison had to inspect every
transformer vault in the Loop before they would restart power and there
were still a number of problems.
Rick
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
