Dell PERC H330: no disks, no volumes
So after I managed to boot my new PowerEdge R6515, the next challenge is that I have no discs. The machine is equipped with a PERC H330 mini, a SCSI backplane and two SATA SSDs. I do see the discs in the BIOS's RAID controller configuration menu. Autoconfiguration says: mfii0 at pci1 dev 0 function 0: "PERC H330 Mini", firmware 25.5.9.0001 mfii0: interrupting at ioapic4 pin 26 scsibus0 at mfii0: 0 targets, 8 luns per target mfii0: unable to register with sysmon (rv = 86) mfii0: unable to create sensors [...] mfii0: physical disk inserted id 32 enclosure 32 mfii0: physical disk inserted id 0 enclosure 32 mfii0: physical disk inserted id 1 enclosure 32 (both with 8.2 and current), but bioctl mfii0 show says bioctl: no volumes available and bioctl show disks show a header and then bioctl: BIOCDISK_NOVOL: Inappropriate ioctl for device The BIOS configuration lets me set the controller mode from RAID to HBA an I can mark individual discs as "RAID elegible", but that doesn't seem to make a difference. I suppose it's something stupid. Anyone using a H330?
Re: Dell PERC H330: no disks, no volumes
It appears to me we have two drivers for the SAS3008: mfii(4) and mpii(4). Why?
Re: Dell PERC H330: no disks, no volumes
On Tue, 13 Sep 2022, Edgar Fu? wrote: It appears to me we have two drivers for the SAS3008: mfii(4) and mpii(4). Why? I know nothing about these drivers, but the man pages show that mfii works for MegaRAID devices, while mpii deaels with LSI devices. I have no idea if this makes a difference. FWIW, mfii appears tobe more recentin NetBSD, appearing in 8.0 (vs 6.0 for mpii. ++--+--+ | Paul Goyette | PGP Key fingerprint: | E-mail addresses:| | (Retired) | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com| | Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | ++--+--+
Re: Dell PERC H330: no disks, no volumes
p...@whooppee.com (Paul Goyette) writes:
>On Tue, 13 Sep 2022, Edgar Fu=DF wrote:
>> It appears to me we have two drivers for the SAS3008: mfii(4) and mpii(4)=
>=2E
>> Why?
>I know nothing about these drivers, but the man pages show that mfii
>works for MegaRAID devices, while mpii deaels with LSI devices. I
>have no idea if this makes a difference.
These controller chips can run two different kinds of firmware.
The mfii driver is for talking to the RAID firmware ("IR mode")
while the mpii driver is for talking to the vanilla SAS firmware
("IT mode").
Some of the cards can even be re-flashed and used with either firmware.
Re: Dell PERC H330: no disks, no volumes
> These controller chips can run two different kinds of firmware.
> The mfii driver is for talking to the RAID firmware ("IR mode")
> while the mpii driver is for talking to the vanilla SAS firmware
> ("IT mode").
Ah, and how do I know which mode my card runs?
mpii(4) explicitly mentions the Dell PERC HBA330, but the "R" in PERC
is for RAID.
The controller can be switched to RAID or HBA mode in the BIOS setup,
so does it run both firmware versions?
Re: Dell PERC H330: no disks, no volumes
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes:
>> These controller chips can run two different kinds of firmware.
>> The mfii driver is for talking to the RAID firmware ("IR mode")
>> while the mpii driver is for talking to the vanilla SAS firmware
>> ("IT mode").
>Ah, and how do I know which mode my card runs?
>mpii(4) explicitly mentions the Dell PERC HBA330, but the "R" in PERC
>is for RAID.
>The controller can be switched to RAID or HBA mode in the BIOS setup,
>so does it run both firmware versions?
The different firmware versions return different PCI-IDs, so that
the right driver attaches, e.g.:
mpii.c: { PCI_VENDOR_SYMBIOS, PCI_PRODUCT_SYMBIOS_SAS3008 },
mfii.c: { PCI_VENDOR_SYMBIOS, PCI_PRODUCT_SYMBIOS_MEGARAID_3008,
PCI_PRODUCT_SYMBIOS_SAS3008 = 0x0097
PCI_PRODUCT_SYMBIOS_MEGARAID_3008 = 0x005f
There is a PERC H330 and a PERC HBA330 and the Dell PERC9 user manual
(includes the H330) says you can boot it in HBA mode. Not sure if
that means that you can chose the firmware.
-> This is attaching a H330 (RAID version) and it gets the mfii driver.
mfii0 at pci1 dev 0 function 0: "PERC H330 Mini", firmware 25.5.9.0001
Re: Dell PERC H330: no disks, no volumes
Oh, I wasn't aware the H330 and HBA330 are different devices! > There is a PERC H330 and a PERC HBA330 and the Dell PERC9 user manual > (includes the H330) says you can boot it in HBA mode. Not sure if > that means that you can chose the firmware. Oh well. So the HBA330 is a PowerEdge RAID Controller that isn't a RAID controller? Thanks, Dell marketing! > -> This is attaching a H330 (RAID version) and it gets the mfii driver. > mfii0 at pci1 dev 0 function 0: "PERC H330 Mini", firmware 25.5.9.0001 OK, remains the question why I don't see any discs in bioctl. On startup, the machine utters the following: PowerEdge Expandable RAID Controller BIOS Copyright(c) 2016 Avago Technologies Press to Run Configuration Utility HA -0 (Bus 1 Dev 0) PERC H330 Mini FW package: 25.5.0.0001 0 Non-RAID Disk(s) found on the host adapter. 0 Non-RAID Disk(s) handled by BIOS 0 Virtual Disk(s) found on the host adapter. 0 Virtual Disk(s) handled by BIOS Is this normal? The only place I see discs being recognized is in the BIOS setup's controller setup.
Re: Dell PERC H330: no disks, no volumes
On Wed, Sep 14, 2022 at 11:07:23AM +0200, Edgar Fuß wrote: > Oh, I wasn't aware the H330 and HBA330 are different devices! > > > There is a PERC H330 and a PERC HBA330 and the Dell PERC9 user manual > > (includes the H330) says you can boot it in HBA mode. Not sure if > > that means that you can chose the firmware. > Oh well. So the HBA330 is a PowerEdge RAID Controller that isn't a RAID > controller? Thanks, Dell marketing! > > > -> This is attaching a H330 (RAID version) and it gets the mfii driver. > > mfii0 at pci1 dev 0 function 0: "PERC H330 Mini", firmware 25.5.9.0001 > OK, remains the question why I don't see any discs in bioctl. > > On startup, the machine utters the following: > > PowerEdge Expandable RAID Controller BIOS > Copyright(c) 2016 Avago Technologies > Press to Run Configuration Utility > HA -0 (Bus 1 Dev 0) PERC H330 Mini > FW package: 25.5.0.0001 > > > 0 Non-RAID Disk(s) found on the host adapter. > 0 Non-RAID Disk(s) handled by BIOS > > 0 Virtual Disk(s) found on the host adapter. > > 0 Virtual Disk(s) handled by BIOS > > Is this normal? The only place I see discs being recognized is in the BIOS > setup's controller setup. It's not normal, and this explains why NetBSD doens't see any disks. I don't remember the details (and it depends on the controller version), but you need to have physical disks assigned to one (or more) RAID volume, and then the RAID volume has to be exported as one (or more) virtual disks. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Dell PERC H330: no disks, no volumes
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes: >> -> This is attaching a H330 (RAID version) and it gets the mfii driver. >> mfii0 at pci1 dev 0 function 0: "PERC H330 Mini", firmware 25.5.9.0001 >OK, remains the question why I don't see any discs in bioctl. > 0 Non-RAID Disk(s) found on the host adapter. > 0 Non-RAID Disk(s) handled by BIOS > 0 Virtual Disk(s) found on the host adapter. > 0 Virtual Disk(s) handled by BIOS This means you do have a RAID controller (so much for marketing) but that you have haven't configured any RAID volumes. >Is this normal? The only place I see discs being recognized is in the BIOS >setup's controller setup. Yes, in the controller setup you can create "Non-RAID Disks" (aka JBOD) or "Virtual Disks" (aka RAID volumes) and at least the latter could then be visible to bootloader (BIOS) and kernel. Disks that aren't configured as either aren't visible. In theory you could use bioctl to create and manage volumes, but the driver doesn't implement it.
Re: Dell PERC H330: no disks, no volumes
> I don't remember the details (and it depends on the controller version), > but you need to have physical disks assigned to one (or more) RAID volume, > and then the RAID volume has to be exported as one (or more) virtual disks. But what if I want to pass the bare discs to NetBSD for a RAIDframe use?
Re: Dell PERC H330: no disks, no volumes
> Yes, in the controller setup you can create "Non-RAID Disks" (aka > JBOD) or "Virtual Disks" (aka RAID volumes) Where exactly are those Non-RAID Disks hiodden? > In theory you could use bioctl to create and manage volumes, but the > driver doesn't implement it. Ah, interesting. That was the way I was trying to use.
Re: Dell PERC H330: no disks, no volumes
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes: >> I don't remember the details (and it depends on the controller version), >> but you need to have physical disks assigned to one (or more) RAID volume, >> and then the RAID volume has to be exported as one (or more) virtual disks. >But what if I want to pass the bare discs to NetBSD for a RAIDframe use? If possible configure the disk as JBOD or as a 1 disk RAID-0 volume.
Re: Dell PERC H330: no disks, no volumes
> There is a PERC H330 and a PERC HBA330 and the Dell PERC9 user manual > (includes the H330) says you can boot it in HBA mode. Not sure if > that means that you can chose the firmware. When I set the H330 to HBA mode, it still attaches as mfii0, the only difference to RAID mode being that the attachment in HBA mode says scsibus0 at mfii0: 0 targets, 8 luns per target instead of scsibus0 at mfii0: 32 targets, 8 luns per target in RAID mode. I tried to force it to use mpii (by adding the PCI Id in mpii.c and disabling mfii in the kernel config, but that didn't work either (I had the faint hope the controller would use the MPT-2 protocol in HBA mode despite showing the RAID PCI Ids). What /does/ work is setting the controller to RAID mode and create two volumes with a one-element RAID-0. But that feels like crazy.
Re: Dell PERC H330: no disks, no volumes
e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes: >What /does/ work is setting the controller to RAID mode and create two >volumes with a one-element RAID-0. But that feels like crazy. That's a common setup. In particular it allows to present a "right-sized" disk so that you can replace a faulty disk with a similar model of slightly different size without changing the visible disk layout. It also means that the RAID controller could do media checks (which it will not do if you pass through a raw disk). N.B. the controller should be able to pass through raw disks (JBOD), but maybe your controller setup tool doesn't support this.
Re: Dell PERC H330: no disks, no volumes
Edgar Fuß writes: >> There is a PERC H330 and a PERC HBA330 and the Dell PERC9 user manual >> (includes the H330) says you can boot it in HBA mode. Not sure if >> that means that you can chose the firmware. > When I set the H330 to HBA mode, it still attaches as mfii0, the only > difference to RAID mode being that the attachment in HBA mode says > scsibus0 at mfii0: 0 targets, 8 luns per target > instead of > scsibus0 at mfii0: 32 targets, 8 luns per target > in RAID mode. > > I tried to force it to use mpii (by adding the PCI Id in mpii.c and > disabling mfii in the kernel config, but that didn't work either > (I had the faint hope the controller would use the MPT-2 protocol in > HBA mode despite showing the RAID PCI Ids). > > What /does/ work is setting the controller to RAID mode and create two > volumes with a one-element RAID-0. But that feels like crazy. In the foggy recesses of my memory this is Just How It Is Done. At my final $DAYJOB we had a set of systems that had some PERC controller in them. The desire was to present the raw disks to Hadoop and the only way that could be done was to create a virtual disk for each physical device. There was no other option available to us. -- Brad Spencer - b...@anduin.eldar.org - KC8VKS - http://anduin.eldar.org
Re: Dell PERC H330: no disks, no volumes
On Thu, 15 Sept 2022 at 19:27, Brad Spencer wrote: > > In the foggy recesses of my memory this is Just How It Is Done. At my > final $DAYJOB we had a set of systems that had some PERC controller in > them. The desire was to present the raw disks to Hadoop and the only > way that could be done was to create a virtual disk for each physical > device. There was no other option available to us. I was annoyed enough by this behaviour to swap out the PERC on my old T320 for another model, specifically one for which I could find generic LSI firmware, so it would expose the 8 disks directly to NetBSD (for ZFS use) mpii0: SAS9217-8i, firmware 20.0.7.0, MPI 2.0 David
Re: Dell PERC H330: no disks, no volumes
David Brownlee writes: > On Thu, 15 Sept 2022 at 19:27, Brad Spencer wrote: >> >> In the foggy recesses of my memory this is Just How It Is Done. At my >> final $DAYJOB we had a set of systems that had some PERC controller in >> them. The desire was to present the raw disks to Hadoop and the only >> way that could be done was to create a virtual disk for each physical >> device. There was no other option available to us. > > I was annoyed enough by this behaviour to swap out the PERC on my old > T320 for another model, specifically one for which I could find > generic LSI firmware, so it would expose the 8 disks directly to > NetBSD (for ZFS use) > > mpii0: SAS9217-8i, firmware 20.0.7.0, MPI 2.0 > > David I have totally forgotten the model numbers of any of this, but in another area at the last $DAYJOB place we did that too for SmartOS and for the same reason. Until somewhat recently, however, that wasn't a supported configuration by Dell... it is now. There was also a BIOS annoyance in that the simple HBA wasn't seen and would not be reported in the DRAC (at least the versions we had). -- Brad Spencer - b...@anduin.eldar.org - KC8VKS - http://anduin.eldar.org
