Here's the update of our events last night...sorry I didn't respond back
last night, a bit busy :-)
EMC came back and stated that there were bad sectors in the luns...9
volumes were 'toast' (900 gig overall in volume group)
EMC rebuilt the luns effected
Brought TSM back up, 9 volumes marked off-line
jfs logs rebuilt
disabled all sessions, except for a couple of effected 4 servers that had
to have data restored (TSM Server down for some time)
We ended up with the decision to delete volumes involved, then create new
volumes
finally started backups
AIX Engineers and EMC evaluating the cause of issue 2 drives down.
Thank You to all who responded!
Nancy Backhaus
Enterprise Systems
[EMAIL PROTECTED]
Office: (716) 887-7979
Cell: (716) 609-2138
TSM_User <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU>
05/04/2005 06:54 PM
Please respond to "ADSM: Dist Stor Manager"
To: ADSM-L@VM.MARIST.EDU
cc:
Subject: Re: CX700 ATA failure - Audit Storage Pool Volumes
We lost 2 drives in the same array on a CX600 and they wouldn't rebuild.
Our problem did not cause a loss of data. EMC had to bring down the whole
thing and when it came back up the rebuild kicked off and everything was
OK. We were told that if we had a more recent version of microcode then
the rebuild would have worked from the start.
Bottom line wait until EMC is done before you do anything because you may
not have lost any data.
Steve Schaub <[EMAIL PROTECTED]> wrote:
First, make sure the drives were really bad - we had this happen to us on
a
cx500 and it turned out that the microcode was actually bad, and the 2nd
raid5 disk failure was phony. Have your ce double check this - we lost
tons
of data from this little glitch biting us several days in a row before
they
figured it out (when it did come back up the volumes were toast).
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Nancy L Backhaus
Sent: Wednesday, May 04, 2005 1:09 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] CX700 ATA failure - Audit Storage Pool Volumes
TSM Server 5.2.3.5
AIX Operating System 5.2
We have lost two drives on a single ATA RAID array. EMC CE is on site and
working with the EMC SAC to resolve the problem. We brought TSM Server
down while EMC is working on the issue. I started receiving the
following errors before we stop all processes, backups, migrations marking
the effected diskpool volumes read-only.
05/04/05 09:06:36 ANR1411W Access mode for volume
/tsmpool39/diskpool22a now
set to "read-only" due to write error. (SESSION:
125991)
My question is >
When we bring the server back up. I want to ensure that I am doing the
right thing, or if I am missing anything.
1. I will need to make sure the volumes affected are online and in a
read/write status.
2. Audit diskpools volumes, Inspect only first 3. If there are damaged
volumes,rerun audit volume, with Fix=Yes.
4. Suggestions?
Nancy Backhaus
Enterprise Systems
[EMAIL PROTECTED]
Office: (716) 887-7979
Cell: (716) 609-2138
CONFIDENTIALITY NOTICE: This email message and any attachments are for the
sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation
of law. If you are not the intended recipient or a person responsible for
delivering this message to an intended recipient, please contact the
sender
by reply email and destroy all copies of the original message.
Please see the following link for the BlueCross BlueShield of Tennessee
E-mail disclaimer: http://www.bcbst.com/email_disclaimer.shtm
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
