Ian, Thanks for sharing (seriously).
Igor Neyman, OCP DBA [EMAIL PROTECTED] -----Original Message----- MacGregor, Ian A. Sent: Monday, September 15, 2003 11:34 PM To: Multiple recipients of list ORACLE-L Last Friday was hot here, and rumor has it our 230 KV power line sagged and touched some tree branches. The local power company shut it off. Leaving our systems to depend on UPS. About 30 minutes afterwards one system produced these errors. This was jus before the system went dead Fri Sep 12 12:58:40 2003 Errors in file /opt/oracle/admin/BBRO/bdump/bbro_ckpt_1420.trc: ORA-00206: error in writing (block 3, # blocks 1) of controlfile ORA-00202: controlfile: '/u1/oradata/BBRO/BBROcntrl01.ctl' ORA-27063: skgfospo: number of bytes read/written is incorrect SVR4 Error: 5: I/O error Additional information: -1 Additional information: 8192 Fri Sep 12 12:58:42 2003 Errors in file /opt/oracle/admin/BBRO/bdump/bbro_ckpt_1420.trc: ORA-00221: error on write to controlfile ORA-00206: error in writing (block 3, # blocks 1) of controlfile ORA-00202: controlfile: '/u1/oradata/BBRO/BBROcntrl01.ctl' ORA-27063: skgfospo: number of bytes read/written is incorrect SVR4 Error: 5: I/O error Additional information: -1 Additional information: 8192 Fri Sep 12 12:58:42 2003 CKPT: terminating instance due to error 221 Instance terminated by CKPT, pid = 1420 ------------------------------------------------------------------------ ------------------------------------- Things look pretty shaky here. When things were restarted the following error was produced. Fri Sep 12 13:32:01 2003 ORA-00204: error in reading (block 1, # blocks 1) of controlfile ORA-00202: controlfile: '/u1/oradata/BBRO/BBROcntrl01.ctl' ORA-27091: skgfqio: unable to queue I/O SVR4 Error: 6: No such device or address Additional information: 1 The raid array had not been powered on ------------------------------------------------------------------------ ------------------------------- However Fri Sep 12 15:33:08 2003 ORA-00202: controlfile: '/u1/oradata/BBRO/BBROcntrl01.ctl' ORA-27037: unable to obtain file status SVR4 Error: 2: No such file or directory Additional information: 3 Fri Sep 12 15:33:11 2003 ORA-205 signalled during: alter database mount... Now the file system is available, but the file itself has disappeared. It was not corrupted, just disappeared. We duplex a copy to an internal disk. So recovery was easy. However once this was fixed Fri Sep 12 16:18:58 2003 Thread recovery: start rolling forward thread 1 Fri Sep 12 16:18:58 2003 Errors in file /opt/oracle/admin/BBRO/udump/bbro_ora_1804.trc: ORA-00313: open failed for members of log group 3 of thread 1 ORA-00312: online log 3 thread 1: '/u2/oradata/BBRO/redo0301.log' ORA-27037: unable to obtain file status SVR4 Error: 2: No such file or directory Additional information: 3 ORA-313 signalled during: ALTER DATABASE OPEN... ------------------------------------------------------------------------ ------------------------------------- These files are on a RAID 1 LUN. Both copies of the file are gone. Again not corrupted but gone. I don't know if using duplexing rather than RAID 1 would have mattered here, but I am changing things so that one group of redo logs is on internal disk and written via the duplexing method. Ian MacGregor Stanford linear Accelerator Center [EMAIL PROTECTED] -- Please see the official ORACLE-L FAQ: http://www.orafaq.net -- Author: MacGregor, Ian A. INET: [EMAIL PROTECTED] Fat City Network Services -- 858-538-5051 http://www.fatcity.com San Diego, California -- Mailing list and web hosting services --------------------------------------------------------------------- To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.net -- Author: Igor Neyman INET: [EMAIL PROTECTED] Fat City Network Services -- 858-538-5051 http://www.fatcity.com San Diego, California -- Mailing list and web hosting services --------------------------------------------------------------------- To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
