LSI MegaRaid non-hotspare
Hi,
As mentioned before, I have a new server with the LSI MegaRaid
SATA150-4 card. All works nicely at the moment, bar a slight problem
with hot-spares.
We configured a RAID-5 array with three 250Gb drives and one hot
spare. We simulated a failure by yanking the cable out from drive 2,
and the alarm went off, bioctl allowed us to silence it, and showed
that the array was rebuilding, onto disk 3. The rebuild process took
about 9 hours (64bit card in a 32bit slot). We put the drive back
in, and bioctl showed the drive as Unused. So we try to promote that
drive back to a hot spare, but the bioctl command:
# bioctl -H 0:2.0 ami0
Seems to return nothing, nor does it make the change. We tried
rebooting, but there's no change, and the command still does the
same. When we boot into the MegaRaid config utility on the card's
BIOS, it shows the drive as a hot spare, whereas bioctl still reports
it as unused.
# bioctl -Dhiv ami0
bioctl: cookie = 0xd2882ca0
bio_inq
Volume Status Size Device
ami0 0 Online 468G sd0 RAID5
0 Online 234G 0:0.0 noencl
'V5075JFG'
1 Online 234G 0:1.0 noencl
'V5075JVG'
2 Online 234G 0:3.0 noencl
'V5064EEG'
ami0 1 Unused 234G 0:2.0 noencl
'V5075LQG'
# bioctl -Dhiv -H 0:2.0 ami0
bioctl: cookie = 0xd2882ca0
bio_inq
Volume Status Size Device
ami0 0 Online 468G sd0 RAID5
0 Online 234G 0:0.0 noencl
'V5075JFG'
1 Online 234G 0:1.0 noencl
'V5075JVG'
2 Online 234G 0:3.0 noencl
'V5064EEG'
ami0 1 Unused 234G 0:2.0 noencl
'V5075LQG'
Any suggestions? In order to get the kernel to boot we had to
disable pcibios using config, which we did on a copy of bsd.mp. We
took a backup of the fresh bsd.mp.
Here's a dmesg:
OpenBSD 3.9 (GENERIC.MP) #598: Thu Mar 2 02:37:06 MST 2006
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class)
2.68 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID
real mem = 2146541568 (2096232K)
avail mem = 1952505856 (1906744K)
using 4278 buffers containing 107429888 bytes (104912K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 10/30/05, BIOS32 rev. 0 @
0xf0010
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 30102 dobusy 0 doidle 1
pcibios at bios0 function 0x1a not configured
bios0: ROM list: 0xc/0x8000 0xc8000/0x2200
mainbus0: Intel MP Specification (Version 1.1) (INTELPremium )
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133 MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class)
2.68 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID
mainbus0: bus 0 is type PCI
mainbus0: bus 1 is type PCI
mainbus0: bus 2 is type PCI
mainbus0: bus 3 is type PCI
mainbus0: bus 4 is type PCI
mainbus0: bus 5 is type ISA
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82955X MCH" rev 0x81
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01
pci1 at ppb0 bus 4
ppb1 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01
pci2 at ppb1 bus 3
em0 at pci2 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00:
apic 2 int 16 (irq 11), address 00:15:f2:c8:8e:10
ppb2 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01
pci3 at ppb2 bus 2
"CMD Technology SiI3132 SATA" rev 0x01 at pci3 dev 0 function 0 not
configured
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 2
int 20 (irq 10)
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 2
int 17 (irq 10)
usb1 at uhci1: USB revision 1.0
uhub1 at usb1
uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 2
int 18 (irq 3)
usb2 at uhci2: USB revision 1.0
uhub2 at usb2
uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self
Re: LSI MegaRaid non-hotspare
I fixed this in current. You can simply just upgrade the ami files to
-current and build a 3.9 that is mostly RELEASE.
Gaby vanhegan wrote:
Hi,
As mentioned before, I have a new server with the LSI MegaRaid
SATA150-4 card. All works nicely at the moment, bar a slight problem
with hot-spares.
We configured a RAID-5 array with three 250Gb drives and one hot
spare. We simulated a failure by yanking the cable out from drive 2,
and the alarm went off, bioctl allowed us to silence it, and showed
that the array was rebuilding, onto disk 3. The rebuild process took
about 9 hours (64bit card in a 32bit slot). We put the drive back
in, and bioctl showed the drive as Unused. So we try to promote that
drive back to a hot spare, but the bioctl command:
# bioctl -H 0:2.0 ami0
Seems to return nothing, nor does it make the change. We tried
rebooting, but there's no change, and the command still does the
same. When we boot into the MegaRaid config utility on the card's
BIOS, it shows the drive as a hot spare, whereas bioctl still reports
it as unused.
# bioctl -Dhiv ami0
bioctl: cookie = 0xd2882ca0
bio_inq
Volume Status Size Device
ami0 0 Online 468G sd0 RAID5
0 Online 234G 0:0.0 noencl 6V250F0 VA11>
'V5075JFG'
1 Online 234G 0:1.0 noencl 6V250F0 VA11>
'V5075JVG'
2 Online 234G 0:3.0 noencl 6V250F0 VA11>
'V5064EEG'
ami0 1 Unused 234G 0:2.0 noencl 6V250F0 VA11>
'V5075LQG'
# bioctl -Dhiv -H 0:2.0 ami0
bioctl: cookie = 0xd2882ca0
bio_inq
Volume Status Size Device
ami0 0 Online 468G sd0 RAID5
0 Online 234G 0:0.0 noencl 6V250F0 VA11>
'V5075JFG'
1 Online 234G 0:1.0 noencl 6V250F0 VA11>
'V5075JVG'
2 Online 234G 0:3.0 noencl 6V250F0 VA11>
'V5064EEG'
ami0 1 Unused 234G 0:2.0 noencl 6V250F0 VA11>
'V5075LQG'
Any suggestions? In order to get the kernel to boot we had to
disable pcibios using config, which we did on a copy of bsd.mp. We
took a backup of the fresh bsd.mp.
Here's a dmesg:
OpenBSD 3.9 (GENERIC.MP) #598: Thu Mar 2 02:37:06 MST 2006
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class)
2.68 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID
real mem = 2146541568 (2096232K)
avail mem = 1952505856 (1906744K)
using 4278 buffers containing 107429888 bytes (104912K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 10/30/05, BIOS32 rev. 0 @
0xf0010
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 30102 dobusy 0 doidle 1
pcibios at bios0 function 0x1a not configured
bios0: ROM list: 0xc/0x8000 0xc8000/0x2200
mainbus0: Intel MP Specification (Version 1.1) (INTELPremium )
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133 MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Pentium(R) D CPU 2.66GHz ("GenuineIntel" 686-class)
2.68 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,TM2,CNXT-ID
mainbus0: bus 0 is type PCI
mainbus0: bus 1 is type PCI
mainbus0: bus 2 is type PCI
mainbus0: bus 3 is type PCI
mainbus0: bus 4 is type PCI
mainbus0: bus 5 is type ISA
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82955X MCH" rev 0x81
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01
pci1 at ppb0 bus 4
ppb1 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01
pci2 at ppb1 bus 3
em0 at pci2 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00:
apic 2 int 16 (irq 11), address 00:15:f2:c8:8e:10
ppb2 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01
pci3 at ppb2 bus 2
"CMD Technology SiI3132 SATA" rev 0x01 at pci3 dev 0 function 0 not
configured
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 2
int 20 (irq 10)
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 2
int 17 (irq 10)
usb1 at uhci1: USB revision 1.0
Re: LSI MegaRaid non-hotspare
On 20 May 2006, at 16:28, Marco Peereboom wrote: > I fixed this in current. You can simply just upgrade the ami files > to -current and build a 3.9 that is mostly RELEASE. Was it a functional problem or just a cosmetic one? If I leave it as it is, is it going to cause any real problems for me? Gaby -- Junkets for bunterish lickspittles since 1998! http://www.playr.co.uk/sudoku/ http://weblog.vanhegan.net/
Re: LSI MegaRaid non-hotspare
> As mentioned before, I have a new server with the LSI MegaRaid > SATA150-4 card. All works nicely at the moment, bar a slight problem > with hot-spares. > > We configured a RAID-5 array with three 250Gb drives and one hot > spare. We simulated a failure by yanking the cable out from drive 2, > and the alarm went off, bioctl allowed us to silence it, and showed > that the array was rebuilding, onto disk 3. The rebuild process took > about 9 hours (64bit card in a 32bit slot). We put the drive back > in, and bioctl showed the drive as Unused. So we try to promote that > drive back to a hot spare, but the bioctl command: > > # bioctl -H 0:2.0 ami0 > > Seems to return nothing, nor does it make the change. We tried > rebooting, but there's no change, and the command still does the > same. When we boot into the MegaRaid config utility on the card's > BIOS, it shows the drive as a hot spare, whereas bioctl still reports > it as unused. Right. The card honours your request for the device to be a hot spare, but something was busted in reporting the new hot spare. Apparently this bug is now fixed: revision 1.156 date: 2006/05/12 20:51:25; author: marco; state: Exp; lines: +14 -22 Fix a misreporting bug after bioctl -H is used to create a hotspare. This was reported by several people. What happens is that the firmware sometimes misreports what SCSI type a device is. The driver was only allowing a create hotspare function when the type was set to hard disk. Since the firmware will, obviously, not allow the driver to create a hotspare on any other type of device the driver doesn't need these smarts and now will ignore the type. Tested by henning todd and Ben Lovett. ok dlg
Re: LSI MegaRaid non-hotspare
Depends on which bug you hit. If the BIOS shows it is a hotspare you're golden. If it does not show it as a hotspare you want to upgrade (or use the bios to create the hotspare). Gaby vanhegan wrote: On 20 May 2006, at 16:28, Marco Peereboom wrote: I fixed this in current. You can simply just upgrade the ami files to -current and build a 3.9 that is mostly RELEASE. Was it a functional problem or just a cosmetic one? If I leave it as it is, is it going to cause any real problems for me? Gaby -- Junkets for bunterish lickspittles since 1998! http://www.playr.co.uk/sudoku/ http://weblog.vanhegan.net/
Re: LSI MegaRaid non-hotspare
* Marco Peereboom <[EMAIL PROTECTED]> [2006-05-20 21:48]: > Depends on which bug you hit. If the BIOS shows it is a hotspare you're > golden. If it does not show it as a hotspare you want to upgrade (or > use the bios to create the hotspare). no, that is incorrect. even if the bios does hsow you teh drive as hotspare, it might not be used as one; you have to to put it back to unused using the bios and then mark as hotspare in the bios. that is the only 100% reliable way for the moment, unfortunately. -- BS Web Services, http://www.bsws.de/ OpenBSD-based Webhosting, Mail Services, Managed Servers, ... Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie)
