Hi all, I'm new to the list and OI, but I've been running a Solaris box with some version of ZFS on it for a couple years. I recently acquired a Supermicro 6047R-E1R36N. It has an X9DRi-LN4F+ motherboard, dual E5-2620s and 48GB of RAM. I soon learned that I needed a different HBA (it comes with an LSI 2108-based card that doesn't have any JBOD passthrough mode), so I replaced that with an LSI SAS9211-8i, flashed with the latest "IT" firmware.
Being new to SAS expanders (which this server has two of), I quickly learned that some older 2TB drives I had on my older Solaris box simply will not work reliably on the expander. (They would very quickly lock up the entire file system once the load got heavy.) However, I've had much better luck (so far) with some newer 3TB SATA drives (Seagate ST3000DM001). I'm still not convinced that these are going to be reliable in the long run, having now read about a few SATA-on-SAS expander horror stories (and my own experience with the 2TB drives), but this isn't a production system (yet) so I'm willing to see how it goes. But the one problem I can't seem to figure out are the Samsung 830 SSD drives I have for ZIL and ARC. I have 2 mirrored 128GB 830s as the boot pool, but they're on the built-in SATA ports, not the expander ports. Then I have four other 128GB 830s which I installed into 3.5" hot-swap adapters and these are the ones causing problems. The HBA sees the drives just fine—they're listed during POST and I can access them via the card's built-in BIOS. But once OI (151a5) is up and running, it only sees ONE of the 830s. And here's where it gets weird: if I remove all but one of the SSDs from the expander ports, it will see whichever one SSD is connected—it does not matter which port the drives are on. Also, regardless of the port, the attachment path always seems to be the same in the OS. It's like the OS thinks all four drives are the exact same physical drive. $ cfgadm -al Ap_Id Type Receptacle Occupant Condition c6 scsi-sas connected configured unknown c6::es/ses0 ESI connected configured unknown c6::smp/expd0 smp connected configured unknown c6::w5002538043584d30,0 disk-path connected configured unknown <-- this is the SSD c6::w5000c500513a0efe,0 disk-path connected configured unknown c6::w5000c5005168b2f0,0 disk-path connected configured unknown c6::w5000c5005169a612,0 disk-path connected configured unknown c6::w5000c5005169cc6a,0 disk-path connected configured unknown c6::w5000c500515199bd,0 disk-path connected configured unknown c6::w5000c500517042d9,0 disk-path connected configured unknown c8 scsi-sas connected configured unknown c8::es/ses1 ESI connected configured unknown c8::smp/expd1 smp connected configured unknown c9 scsi-sas connected unconfigured unknown sata0/0::dsk/c4t0d0 disk connected configured ok sata0/1::dsk/c4t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok sata0/4 sata-port empty unconfigured ok sata0/5 sata-port empty unconfigured ok $ format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c4t0d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56> /pci@0,0/pci15d9,626@1f,2/disk@0,0 1. c4t1d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56> /pci@0,0/pci15d9,626@1f,2/disk@1,0 2. c7t5000C500513A0EFEd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c500513a0efe 3. c7t5000C5005168B2F0d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c5005168b2f0 4. c7t5000C5005169A612d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c5005169a612 5. c7t5000C5005169CC6Ad0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c5005169cc6a 6. c7t5000C500515199BDd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c500515199bd 7. c7t5000C500517042D9d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB> /scsi_vhci/disk@g5000c500517042d9 8. c7t5002538043584D30d0 <ATA-SAMSUNG SSD 830-3B1Q-119.24GB> /scsi_vhci/disk@g5002538043584d30 This last one is the SSD. If I remove all the SSDs from the expander ports, disk #8 does not show up (nor does it show in cfgadm). If I put one SSD into any expander port, disk 8 is back. If I add additional SSDs, I still only have disk 8. If I remove the first SSD (while others are still attached), disk 8 still appears. In all cases, I can run a read test on the disk (format -> analyze -> test) without any errors, so I think it's actually accessing one of the physical drives. How do I get OI to see the other 3 SSDs? Any pointers are greatly appreciated. Thanks! -Scott _______________________________________________ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss