Re: SAS/SATA controllers: 8 port that support 8TB Drives
On Fri, Jun 17, 2022 at 05:55:50PM -0500, Larry Rosenman wrote: | On 06/17/2022 5:48 pm, Michael Gmelin wrote: | >> On 18. Jun 2022, at 00:31, Alexander Motin wrote: | >> | >> | >> | >>> On 17.06.2022 18:24, Alexander Motin wrote: | On 17.06.2022 18:16, Larry Rosenman wrote: | On 06/17/2022 5:08 pm, Alexander Motin wrote: | > On 17.06.2022 11:59, Larry Rosenman wrote: | >> I'm looking to upgrade the controllers in my TrueNAS box to | >> something that will | >> support 8TB drives because apparently my LSI 2108 controllers do | >> not support 8TB drives. | >> | >> What's the communities recommendation? | >> needs to support SFF connectors for a total of 4 SFF connectors, | >> as I have 16 slots. | > | > We at iX are still using LSI/Broadcom HBAs, just moved from long | > discontinued mps(4) to newer mpr(4). And I don't believe the | > problem | > is directly related to capacity. According to my observations it | > may | > be Seagate HDDs of/above certain (8TB) generation. We do not use | > Seagate HDDs in our products, so about that instability I only | > heard | > from forums and TrueNAS community user reports. | | This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) | drive. | | Is this a bad combo? | | mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not | supported | (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain | (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain | (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain | (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain | (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | (probe0:mfi0:0:0:0): Error 5, Retries exhausted | | mfi0 Physical Drives: | 0 ( 932G) UNCONFIGURED GOOD >>> serial=ZA1AC912> SATA E1:S3 | >>> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS | >>> due to problems with hot-plug, disk identification, etc. and so have | >>> limited experience with them. But I know some of LSI RAIDs can be | >>> reflashed into equivalent HBAs, so if they share the hardware, I can | >>> speculate that they may share some issues. | >> | >> I've just noticed "932G" instead of "8000G". It is obviously a bigger | >> problem than what we heard for HBAs. It looks like a kind of problems | >> that should not happen to HBAs, since they should not care about disk | >> capacity. | >> | > | > What does `smartctl -a ` report (especially sector sizes)? | > | > -m | > | > | >> -- | >> Alexander Motin | >> | It's not even making a mfid* node (it is a 4Kn disk) FYI, mfi never got support for 4Kn drives and is hard coded to 512. mrsas does have support so that is better to use for 4Kn and newer controllers. My day job isn't in that area anymore. I looked a bit as what it would take to support 4Kn in mfi and that required a bunch of things and structures to figure it out. Doug A.
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/18/2022 8:30 am, Michael Gmelin wrote: That certainly sounds promising. Best Michael got the new controllers, and no sweat -- saw all 8TB on the drives (modulo one bad drive -- seller is replacing). thanks all. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/18/2022 8:30 am, Michael Gmelin wrote: On 18. Jun 2022, at 15:10, Larry Rosenman wrote: On 06/18/2022 3:54 am, Michael Gmelin wrote: [SNIP] Subvendor is Fujitsu Siemens - so I guess this is integrated into a system by them. Seems like flashing the 2108 to an IT firmware isn't an option (based on what I found online). You could check if there are firmware updates available though. How did you configure the drives in the megaraid utility (ctrl-h after boot)? Did you create a RAID-0 for each disk? And what capacity is shown in there? Based on [0], 2108 based controllers don't support 4kn. IT mode would help (true passthrough), but as written above, I don't think it's an option for this model. -m [0] https://bitdeals.tech/blogs/news/4kn-lsi-compatibility-list as I said earlier in the thread, I've bought 2 of these: https://www.ebay.com/itm/194910024856 which if I'm reading that chart right should work with the 4Kn drives. That certainly sounds promising. Best Michael -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 And I realized I didn't answer the question about how stuff was configured, each disk is Raid0. the current pool is 10x3T disks, and I'm adding 6x8T since the pool is 70% full (bacula backups, Time Machine Backups, random other stuff). -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
> On 18. Jun 2022, at 15:10, Larry Rosenman wrote: > > > On 06/18/2022 3:54 am, Michael Gmelin wrote: > > [SNIP] > > > Subvendor is Fujitsu Siemens - so I guess this is integrated into a system by > them. > > Seems like flashing the 2108 to an IT firmware isn't an option (based on what > I found online). You could check if there are firmware updates available > though. How did you configure the drives in the megaraid utility (ctrl-h > after boot)? Did you create a RAID-0 for each disk? And what capacity is > shown in there? > > Based on [0], 2108 based controllers don't support 4kn. IT mode would help > (true passthrough), but as written above, I don't think it's an option for > this model. > > -m > > [0] https://bitdeals.tech/blogs/news/4kn-lsi-compatibility-list > > > > as I said earlier in the thread, I've bought 2 of these: > https://www.ebay.com/itm/194910024856 > > which if I'm reading that chart right should work with the 4Kn drives. > That certainly sounds promising. Best Michael > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: l...@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/18/2022 3:54 am, Michael Gmelin wrote: [SNIP] Subvendor is Fujitsu Siemens - so I guess this is integrated into a system by them. Seems like flashing the 2108 to an IT firmware isn't an option (based on what I found online). You could check if there are firmware updates available though. How did you configure the drives in the megaraid utility (ctrl-h after boot)? Did you create a RAID-0 for each disk? And what capacity is shown in there? Based on [0], 2108 based controllers don't support 4kn. IT mode would help (true passthrough), but as written above, I don't think it's an option for this model. -m [0] https://bitdeals.tech/blogs/news/4kn-lsi-compatibility-list as I said earlier in the thread, I've bought 2 of these: https://www.ebay.com/itm/194910024856 which if I'm reading that chart right should work with the 4Kn drives. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
> On 18. Jun 2022, at 01:31, Larry Rosenman wrote: > On 06/17/2022 6:20 pm, Michael Gmelin wrote: On 18. Jun 2022, at 00:57, Larry Rosenman wrote: >>> On 06/17/2022 5:48 pm, Michael Gmelin wrote: > On 18. Jun 2022, at 00:31, Alexander Motin wrote: > >> On 17.06.2022 18:24, Alexander Motin wrote: >>> On 17.06.2022 18:16, Larry Rosenman wrote: >>> On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: > I'm looking to upgrade the controllers in my TrueNAS box to something > that will > support 8TB drives because apparently my LSI 2108 controllers do not > support 8TB drives. > What's the communities recommendation? > needs to support SFF connectors for a total of 4 SFF connectors, as I > have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. >>> This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) >>> drive. >>> Is this a bad combo? >>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted >>> mfi0 Physical Drives: >>> 0 ( 932G) UNCONFIGURED GOOD >>> SATA E1:S3 >> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due >> to problems with hot-plug, disk identification, etc. and so have limited >> experience with them. But I know some of LSI RAIDs can be reflashed >> into equivalent HBAs, so if they share the hardware, I can speculate >> that they may share some issues. > I've just noticed "932G" instead of "8000G". It is obviously a bigger > problem than what we heard for HBAs. It looks like a kind of problems > that should not happen to HBAs, since they should not care about disk > capacity. What does `smartctl -a ` report (especially sector sizes)? -m > -- > Alexander Motin >>> It's not even making a mfid* node (it is a 4Kn disk) >> Ok, that’s sad (and explains the wrong size calculation as 4096/512=8). >> Is this in HBA mode? (Like Alexander suggested, re-/crossflashing >> using an IT firmware might be an option). What controller / firmware >> image version is it? >> -m >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 E-Mail: l...@lerctr.org >>> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 > > > mfi0@pci0:8:0:0:class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 > device=0x0079 subvendor=0x1734 subdevice=0x1176 >vendor = 'Broadcom / LSI' >device = 'MegaRAID SAS 2108 [Liberator]' >class = mass storage >subclass = RAID > > mfi1@pci0:3:0:0:class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 > device=0x0079 subvendor=0x1734 subdevice=0x1176 >vendor = 'Broadcom / LSI' >device = 'MegaRAID SAS 2108 [Liberator]' >class = mass storage >subclass = RAID > > mfi0: port 0xd000-0xd0ff mem > 0xfbc9c000-0xfbc9,0xfbcc > -0xfbcf irq 26 at device 0.0 on pci3 > mfi0: Using MSI > mfi0: Megaraid SAS driver Ver 4.23 > mfi0: FW MaxCmds = 1008, limiting to 128 > mfip0: on mfi0 > mfi0: 10014 (708822708s/0x0020/info) - Shutdown command received from host > mfi0: 10015 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID > 00 > 79/1000/1176/1734) > mfi0: 10016 (boot + 3s/0x0020/info) - Firmware version 2.130.353-2727 > mfi0: 10017 (boot + 6s/0x0020/info) - Package version 12.12.0-0174 > mfi0: 10018 (boot + 6s/0x0020/info) - Board Revision > > mfi1: port 0xc000-0xc0ff mem > 0xfba9c000-0xfba9,0xfbac00
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/17/2022 6:20 pm, Michael Gmelin wrote: On 18. Jun 2022, at 00:57, Larry Rosenman wrote: On 06/17/2022 5:48 pm, Michael Gmelin wrote: On 18. Jun 2022, at 00:31, Alexander Motin wrote: On 17.06.2022 18:24, Alexander Motin wrote: On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD serial=ZA1AC912> SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. I've just noticed "932G" instead of "8000G". It is obviously a bigger problem than what we heard for HBAs. It looks like a kind of problems that should not happen to HBAs, since they should not care about disk capacity. What does `smartctl -a ` report (especially sector sizes)? -m -- Alexander Motin It's not even making a mfid* node (it is a 4Kn disk) Ok, that’s sad (and explains the wrong size calculation as 4096/512=8). Is this in HBA mode? (Like Alexander suggested, re-/crossflashing using an IT firmware might be an option). What controller / firmware image version is it? -m -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 mfi0@pci0:8:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 vendor = 'Broadcom / LSI' device = 'MegaRAID SAS 2108 [Liberator]' class = mass storage subclass = RAID mfi1@pci0:3:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 vendor = 'Broadcom / LSI' device = 'MegaRAID SAS 2108 [Liberator]' class = mass storage subclass = RAID mfi0: port 0xd000-0xd0ff mem 0xfbc9c000-0xfbc9,0xfbcc -0xfbcf irq 26 at device 0.0 on pci3 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: FW MaxCmds = 1008, limiting to 128 mfip0: on mfi0 mfi0: 10014 (708822708s/0x0020/info) - Shutdown command received from host mfi0: 10015 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 00 79/1000/1176/1734) mfi0: 10016 (boot + 3s/0x0020/info) - Firmware version 2.130.353-2727 mfi0: 10017 (boot + 6s/0x0020/info) - Package version 12.12.0-0174 mfi0: 10018 (boot + 6s/0x0020/info) - Board Revision mfi1: port 0xc000-0xc0ff mem 0xfba9c000-0xfba9,0xfbac -0xfbaf irq 16 at device 0.0 on pci8 mfi1: Using MSI mfi1: Megaraid SAS driver Ver 4.23 mfi1: FW MaxCmds = 1008, limiting to 128 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
> On 18. Jun 2022, at 00:57, Larry Rosenman wrote: > On 06/17/2022 5:48 pm, Michael Gmelin wrote: >>> On 18. Jun 2022, at 00:31, Alexander Motin wrote: >>> On 17.06.2022 18:24, Alexander Motin wrote: > On 17.06.2022 18:16, Larry Rosenman wrote: > On 06/17/2022 5:08 pm, Alexander Motin wrote: >> On 17.06.2022 11:59, Larry Rosenman wrote: >>> I'm looking to upgrade the controllers in my TrueNAS box to something >>> that will >>> support 8TB drives because apparently my LSI 2108 controllers do not >>> support 8TB drives. >>> What's the communities recommendation? >>> needs to support SFF connectors for a total of 4 SFF connectors, as I >>> have 16 slots. >> We at iX are still using LSI/Broadcom HBAs, just moved from long >> discontinued mps(4) to newer mpr(4). And I don't believe the problem >> is directly related to capacity. According to my observations it may >> be Seagate HDDs of/above certain (8TB) generation. We do not use >> Seagate HDDs in our products, so about that instability I only heard >> from forums and TrueNAS community user reports. > This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. > Is this a bad combo? > mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported > (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 > (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error > (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain > (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 > (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error > (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain > (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 > (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error > (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain > (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 > (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error > (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain > (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 > (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error > (probe0:mfi0:0:0:0): Error 5, Retries exhausted > mfi0 Physical Drives: > 0 ( 932G) UNCONFIGURED GOOD > SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. >>> I've just noticed "932G" instead of "8000G". It is obviously a bigger >>> problem than what we heard for HBAs. It looks like a kind of problems that >>> should not happen to HBAs, since they should not care about disk capacity. >> What does `smartctl -a ` report (especially sector sizes)? >> -m >>> -- >>> Alexander Motin > It's not even making a mfid* node (it is a 4Kn disk) Ok, that’s sad (and explains the wrong size calculation as 4096/512=8). Is this in HBA mode? (Like Alexander suggested, re-/crossflashing using an IT firmware might be an option). What controller / firmware image version is it? -m > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: l...@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/17/2022 5:48 pm, Michael Gmelin wrote: On 18. Jun 2022, at 00:31, Alexander Motin wrote: On 17.06.2022 18:24, Alexander Motin wrote: On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD serial=ZA1AC912> SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. I've just noticed "932G" instead of "8000G". It is obviously a bigger problem than what we heard for HBAs. It looks like a kind of problems that should not happen to HBAs, since they should not care about disk capacity. What does `smartctl -a ` report (especially sector sizes)? -m -- Alexander Motin It's not even making a mfid* node (it is a 4Kn disk) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 17.06.2022 18:29, Larry Rosenman wrote: On 06/17/2022 5:24 pm, Alexander Motin wrote: On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD serial=ZA1AC912> SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. I bought 2 of these: https://www.ebay.com/itm/194910024856 to replace the 2 mfi(4)'s Hopefully I can just move the controllers and TrueNAS 13.0-RELEASE will just notice them. and pick up the new 8T disks. let me know if I'm setting myself up for failure. Not so obvious. They should work at least on a short run. But as I have told, I've heard number of rumors about mps(4) and 8TB Seagates. Though nothing confirmed, only some instability, that nobody is really motivated to debug now. Next 93xx line of mpr(4) HBAs still seems cost few times more on EBay thogh, so there is a choice. -- Alexander Motin
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/17/2022 5:24 pm, Alexander Motin wrote: On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. I bought 2 of these: https://www.ebay.com/itm/194910024856 to replace the 2 mfi(4)'s Hopefully I can just move the controllers and TrueNAS 13.0-RELEASE will just notice them. and pick up the new 8T disks. let me know if I'm setting myself up for failure. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 17.06.2022 18:24, Alexander Motin wrote: On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. I've just noticed "932G" instead of "8000G". It is obviously a bigger problem than what we heard for HBAs. It looks like a kind of problems that should not happen to HBAs, since they should not care about disk capacity. -- Alexander Motin
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 17.06.2022 18:16, Larry Rosenman wrote: On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD SATA E1:S3 mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. -- Alexander Motin
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 06/17/2022 5:08 pm, Alexander Motin wrote: On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. This is a mfi(4) set of controllers, and a ST8Nm0045 8TB (CMR) drive. Is this a bad combo? mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error (probe0:mfi0:0:0:0): Error 5, Retries exhausted mfi0 Physical Drives: 0 ( 932G) UNCONFIGURED GOOD SATA E1:S3 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: SAS/SATA controllers: 8 port that support 8TB Drives
On 17.06.2022 11:59, Larry Rosenman wrote: I'm looking to upgrade the controllers in my TrueNAS box to something that will support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. What's the communities recommendation? needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. We at iX are still using LSI/Broadcom HBAs, just moved from long discontinued mps(4) to newer mpr(4). And I don't believe the problem is directly related to capacity. According to my observations it may be Seagate HDDs of/above certain (8TB) generation. We do not use Seagate HDDs in our products, so about that instability I only heard from forums and TrueNAS community user reports. -- Alexander Motin
