LSI MegaRaid non-hotspare
Hi, As mentioned before, I have a new server with the LSI MegaRaid SATA150-4 card. All works nicely at the moment, bar a slight problem with hot-spares. We configured a RAID-5 array with three 250Gb drives and one hot spare. We simulated a failure by yanking the cable out from drive 2, and the alarm went off, bioctl allowed us to silence it, and showed that the array was rebuilding, onto disk 3. The rebuild process took about 9 hours (64bit card in a 32bit slot). We put the drive back in, and bioctl showed the drive as Unused. So we try to promote that drive back to a hot spare, but the bioctl command: # bioctl -H 0:2.0 ami0 Seems to return nothing, nor does it make the change. We tried rebooting, but there's no change, and the command still does the same. When we boot into the MegaRaid config utility on the card's BIOS, it shows the drive as a hot spare, whereas bioctl still reports it as unused. # bioctl -Dhiv ami0 bioctl: cookie = 0xd2882ca0 bio_inq Volume Status Size Device ami0 0 Online 468G sd0 RAID5 0 Online 234G 0:0.0 noencl 'V5075JFG' 1 Online 234G 0:1.0 noencl 'V5075JVG' 2 Online 234G 0:3.0 noencl 'V5064EEG' ami0 1 Unused 234G 0:2.0 noencl 'V5075LQG' # bioctl -Dhiv -H 0:2.0 ami0 bioctl: cookie = 0xd2882ca0 bio_inq Volume Status Size Device ami0 0 Online 468G sd0 RAID5 0 Online 234G 0:0.0 noencl 'V5075JFG' 1 Online 234G 0:1.0 noencl 'V5075JVG' 2 Online 234G 0:3.0 noencl 'V5064EEG' ami0 1 Unused 234G 0:2.0 noencl 'V5075LQG' Any suggestions? In order to get the kernel to boot we had to disable pcibios using config, which we did on a copy of bsd.mp. We took a backup of the fresh bsd.mp. Here's a dmesg: OpenBSD 3.9 (GENERIC.MP) #598: Thu Mar 2 02:37:06 MST 2006 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class) 2.68 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID real mem = 2146541568 (2096232K) avail mem = 1952505856 (1906744K) using 4278 buffers containing 107429888 bytes (104912K) of memory mainbus0 (root) bios0 at mainbus0: AT/286+(00) BIOS, date 10/30/05, BIOS32 rev. 0 @ 0xf0010 apm0 at bios0: Power Management spec V1.2 apm0: AC on, battery charge unknown apm0: flags 30102 dobusy 0 doidle 1 pcibios at bios0 function 0x1a not configured bios0: ROM list: 0xc/0x8000 0xc8000/0x2200 mainbus0: Intel MP Specification (Version 1.1) (INTELPremium ) cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 133 MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class) 2.68 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID mainbus0: bus 0 is type PCI mainbus0: bus 1 is type PCI mainbus0: bus 2 is type PCI mainbus0: bus 3 is type PCI mainbus0: bus 4 is type PCI mainbus0: bus 5 is type ISA ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 "Intel 82955X MCH" rev 0x81 ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01 pci1 at ppb0 bus 4 ppb1 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01 pci2 at ppb1 bus 3 em0 at pci2 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 2 int 16 (irq 11), address 00:15:f2:c8:8e:10 ppb2 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01 pci3 at ppb2 bus 2 "CMD Technology SiI3132 SATA" rev 0x01 at pci3 dev 0 function 0 not configured uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 2 int 20 (irq 10) usb0 at uhci0: USB revision 1.0 uhub0 at usb0 uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 2 int 17 (irq 10) usb1 at uhci1: USB revision 1.0 uhub1 at usb1 uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 2 int 18 (irq 3) usb2 at uhci2: USB revision 1.0 uhub2 at usb2 uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self
Re: LSI MegaRaid non-hotspare
I fixed this in current. You can simply just upgrade the ami files to -current and build a 3.9 that is mostly RELEASE. Gaby vanhegan wrote: Hi, As mentioned before, I have a new server with the LSI MegaRaid SATA150-4 card. All works nicely at the moment, bar a slight problem with hot-spares. We configured a RAID-5 array with three 250Gb drives and one hot spare. We simulated a failure by yanking the cable out from drive 2, and the alarm went off, bioctl allowed us to silence it, and showed that the array was rebuilding, onto disk 3. The rebuild process took about 9 hours (64bit card in a 32bit slot). We put the drive back in, and bioctl showed the drive as Unused. So we try to promote that drive back to a hot spare, but the bioctl command: # bioctl -H 0:2.0 ami0 Seems to return nothing, nor does it make the change. We tried rebooting, but there's no change, and the command still does the same. When we boot into the MegaRaid config utility on the card's BIOS, it shows the drive as a hot spare, whereas bioctl still reports it as unused. # bioctl -Dhiv ami0 bioctl: cookie = 0xd2882ca0 bio_inq Volume Status Size Device ami0 0 Online 468G sd0 RAID5 0 Online 234G 0:0.0 noencl 6V250F0 VA11> 'V5075JFG' 1 Online 234G 0:1.0 noencl 6V250F0 VA11> 'V5075JVG' 2 Online 234G 0:3.0 noencl 6V250F0 VA11> 'V5064EEG' ami0 1 Unused 234G 0:2.0 noencl 6V250F0 VA11> 'V5075LQG' # bioctl -Dhiv -H 0:2.0 ami0 bioctl: cookie = 0xd2882ca0 bio_inq Volume Status Size Device ami0 0 Online 468G sd0 RAID5 0 Online 234G 0:0.0 noencl 6V250F0 VA11> 'V5075JFG' 1 Online 234G 0:1.0 noencl 6V250F0 VA11> 'V5075JVG' 2 Online 234G 0:3.0 noencl 6V250F0 VA11> 'V5064EEG' ami0 1 Unused 234G 0:2.0 noencl 6V250F0 VA11> 'V5075LQG' Any suggestions? In order to get the kernel to boot we had to disable pcibios using config, which we did on a copy of bsd.mp. We took a backup of the fresh bsd.mp. Here's a dmesg: OpenBSD 3.9 (GENERIC.MP) #598: Thu Mar 2 02:37:06 MST 2006 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class) 2.68 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID real mem = 2146541568 (2096232K) avail mem = 1952505856 (1906744K) using 4278 buffers containing 107429888 bytes (104912K) of memory mainbus0 (root) bios0 at mainbus0: AT/286+(00) BIOS, date 10/30/05, BIOS32 rev. 0 @ 0xf0010 apm0 at bios0: Power Management spec V1.2 apm0: AC on, battery charge unknown apm0: flags 30102 dobusy 0 doidle 1 pcibios at bios0 function 0x1a not configured bios0: ROM list: 0xc/0x8000 0xc8000/0x2200 mainbus0: Intel MP Specification (Version 1.1) (INTELPremium ) cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 133 MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class) 2.68 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID mainbus0: bus 0 is type PCI mainbus0: bus 1 is type PCI mainbus0: bus 2 is type PCI mainbus0: bus 3 is type PCI mainbus0: bus 4 is type PCI mainbus0: bus 5 is type ISA ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 "Intel 82955X MCH" rev 0x81 ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01 pci1 at ppb0 bus 4 ppb1 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01 pci2 at ppb1 bus 3 em0 at pci2 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 2 int 16 (irq 11), address 00:15:f2:c8:8e:10 ppb2 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01 pci3 at ppb2 bus 2 "CMD Technology SiI3132 SATA" rev 0x01 at pci3 dev 0 function 0 not configured uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 2 int 20 (irq 10) usb0 at uhci0: USB revision 1.0 uhub0 at usb0 uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 2 int 17 (irq 10) usb1 at uhci1: USB revision 1.0
Re: LSI MegaRaid non-hotspare
On 20 May 2006, at 16:28, Marco Peereboom wrote: > I fixed this in current. You can simply just upgrade the ami files > to -current and build a 3.9 that is mostly RELEASE. Was it a functional problem or just a cosmetic one? If I leave it as it is, is it going to cause any real problems for me? Gaby -- Junkets for bunterish lickspittles since 1998! http://www.playr.co.uk/sudoku/ http://weblog.vanhegan.net/
Re: LSI MegaRaid non-hotspare
> As mentioned before, I have a new server with the LSI MegaRaid > SATA150-4 card. All works nicely at the moment, bar a slight problem > with hot-spares. > > We configured a RAID-5 array with three 250Gb drives and one hot > spare. We simulated a failure by yanking the cable out from drive 2, > and the alarm went off, bioctl allowed us to silence it, and showed > that the array was rebuilding, onto disk 3. The rebuild process took > about 9 hours (64bit card in a 32bit slot). We put the drive back > in, and bioctl showed the drive as Unused. So we try to promote that > drive back to a hot spare, but the bioctl command: > > # bioctl -H 0:2.0 ami0 > > Seems to return nothing, nor does it make the change. We tried > rebooting, but there's no change, and the command still does the > same. When we boot into the MegaRaid config utility on the card's > BIOS, it shows the drive as a hot spare, whereas bioctl still reports > it as unused. Right. The card honours your request for the device to be a hot spare, but something was busted in reporting the new hot spare. Apparently this bug is now fixed: revision 1.156 date: 2006/05/12 20:51:25; author: marco; state: Exp; lines: +14 -22 Fix a misreporting bug after bioctl -H is used to create a hotspare. This was reported by several people. What happens is that the firmware sometimes misreports what SCSI type a device is. The driver was only allowing a create hotspare function when the type was set to hard disk. Since the firmware will, obviously, not allow the driver to create a hotspare on any other type of device the driver doesn't need these smarts and now will ignore the type. Tested by henning todd and Ben Lovett. ok dlg
Re: LSI MegaRaid non-hotspare
Depends on which bug you hit. If the BIOS shows it is a hotspare you're golden. If it does not show it as a hotspare you want to upgrade (or use the bios to create the hotspare). Gaby vanhegan wrote: On 20 May 2006, at 16:28, Marco Peereboom wrote: I fixed this in current. You can simply just upgrade the ami files to -current and build a 3.9 that is mostly RELEASE. Was it a functional problem or just a cosmetic one? If I leave it as it is, is it going to cause any real problems for me? Gaby -- Junkets for bunterish lickspittles since 1998! http://www.playr.co.uk/sudoku/ http://weblog.vanhegan.net/
Re: LSI MegaRaid non-hotspare
* Marco Peereboom <[EMAIL PROTECTED]> [2006-05-20 21:48]: > Depends on which bug you hit. If the BIOS shows it is a hotspare you're > golden. If it does not show it as a hotspare you want to upgrade (or > use the bios to create the hotspare). no, that is incorrect. even if the bios does hsow you teh drive as hotspare, it might not be used as one; you have to to put it back to unused using the bios and then mark as hotspare in the bios. that is the only 100% reliable way for the moment, unfortunately. -- BS Web Services, http://www.bsws.de/ OpenBSD-based Webhosting, Mail Services, Managed Servers, ... Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)