Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On Fri, Nov 3, 2017 at 6:59 PM, Hannes Reineckewrote: > On 11/03/2017 04:38 AM, Gavin Guo wrote: >> On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guo wrote: >>> On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke wrote: On 10/27/2017 04:02 PM, Gavin Guo wrote: > Hi Hannes, > > Thank you for looking into the issue. If there is anything I can help > to test the patch? I appreciate your help. Thank you. > If you had checked linux-scsi you would have found this patch: '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is already scheduled for inclusion in 4.14. Anything else I could help you with? Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg) >>> >>> Really appreciate your help. I will proceed the testing and keep you posted. >> >> Hello Hannes, >> >> I've tested the MPT SAS device controller, the patch works perfectly. >> However, the MPT SPI still cannot work with the hotplugging. The lspci >> of the LSI SPI device is listed: >> >> 00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic >> 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01) >> Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976] >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- >> ParErr- Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >>> TAbort- SERR- > Latency: 64 (1500ns min, 63750ns max) >> Interrupt: pin A routed to IRQ 17 >> Region 0: I/O ports at 1400 [size=256] >> Region 1: Memory at feba (64-bit, non-prefetchable) [size=128K] >> Region 3: Memory at febc (64-bit, non-prefetchable) [size=128K] >> [virtual] Expansion ROM at 40008000 [disabled] [size=16K] >> Kernel driver in use: mptspi >> >> The vendor id/device id[1000:0030] can also be found in the >> mptspi_pci_table(drivers/message/fusion/mptspi.c). >> >> PCI_VENDOR_ID_LSI_LOGIC, MPI_MANUFACTPAGE_DEVID_53C1030 >> >> If there is anything I missed in the testing? I appreciate your help. >> > Don't. Ever. Try. > > SCSI Parallel hotplugging is so loosely defined that essentially every > vendor can (and did!) implement their own thing, and _still_ be spec > compliant. > Plus there is no guarantee that we even see events when a hot-add > occurs. So any thingie we put in here has a really high chance of > breaking other installations. > I wouldn't even bother to attempt this; it's virtualized anyway so you > can as well switch to the SAS emulation here. > Or use a different SCSI parallel emulation like the Symbios/LSI 53c800 one. > > Cheers, > > Hannes > -- > Dr. Hannes ReineckeTeamlead Storage & Networking > h...@suse.de +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton > HRB 21284 (AG Nürnberg) Hello Hannes, I totally understood your concern and appreciate your help again on the hotplugging patch for MPT SAS controller. :)
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On 11/03/2017 04:38 AM, Gavin Guo wrote: > On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guowrote: >> On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke wrote: >>> On 10/27/2017 04:02 PM, Gavin Guo wrote: Hi Hannes, Thank you for looking into the issue. If there is anything I can help to test the patch? I appreciate your help. Thank you. >>> If you had checked linux-scsi you would have found this patch: >>> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is >>> already scheduled for inclusion in 4.14. >>> Anything else I could help you with? >>> >>> Cheers, >>> >>> Hannes >>> -- >>> Dr. Hannes ReineckeTeamlead Storage & Networking >>> h...@suse.de +49 911 74053 688 >>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg >>> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton >>> HRB 21284 (AG Nürnberg) >> >> Really appreciate your help. I will proceed the testing and keep you posted. > > Hello Hannes, > > I've tested the MPT SAS device controller, the patch works perfectly. > However, the MPT SPI still cannot work with the hotplugging. The lspci > of the LSI SPI device is listed: > > 00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic > 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01) > Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976] > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >> TAbort- SERR- Latency: 64 (1500ns min, 63750ns max) > Interrupt: pin A routed to IRQ 17 > Region 0: I/O ports at 1400 [size=256] > Region 1: Memory at feba (64-bit, non-prefetchable) [size=128K] > Region 3: Memory at febc (64-bit, non-prefetchable) [size=128K] > [virtual] Expansion ROM at 40008000 [disabled] [size=16K] > Kernel driver in use: mptspi > > The vendor id/device id[1000:0030] can also be found in the > mptspi_pci_table(drivers/message/fusion/mptspi.c). > > PCI_VENDOR_ID_LSI_LOGIC, MPI_MANUFACTPAGE_DEVID_53C1030 > > If there is anything I missed in the testing? I appreciate your help. > Don't. Ever. Try. SCSI Parallel hotplugging is so loosely defined that essentially every vendor can (and did!) implement their own thing, and _still_ be spec compliant. Plus there is no guarantee that we even see events when a hot-add occurs. So any thingie we put in here has a really high chance of breaking other installations. I wouldn't even bother to attempt this; it's virtualized anyway so you can as well switch to the SAS emulation here. Or use a different SCSI parallel emulation like the Symbios/LSI 53c800 one. Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg)
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guowrote: > On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke wrote: >> On 10/27/2017 04:02 PM, Gavin Guo wrote: >>> Hi Hannes, >>> >>> Thank you for looking into the issue. If there is anything I can help >>> to test the patch? I appreciate your help. Thank you. >>> >> If you had checked linux-scsi you would have found this patch: >> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is >> already scheduled for inclusion in 4.14. >> Anything else I could help you with? >> >> Cheers, >> >> Hannes >> -- >> Dr. Hannes ReineckeTeamlead Storage & Networking >> h...@suse.de +49 911 74053 688 >> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg >> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton >> HRB 21284 (AG Nürnberg) > > Really appreciate your help. I will proceed the testing and keep you posted. Hello Hannes, I've tested the MPT SAS device controller, the patch works perfectly. However, the MPT SPI still cannot work with the hotplugging. The lspci of the LSI SPI device is listed: 00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01) Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR-
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reineckewrote: > On 10/27/2017 04:02 PM, Gavin Guo wrote: >> Hi Hannes, >> >> Thank you for looking into the issue. If there is anything I can help >> to test the patch? I appreciate your help. Thank you. >> > If you had checked linux-scsi you would have found this patch: > '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is > already scheduled for inclusion in 4.14. > Anything else I could help you with? > > Cheers, > > Hannes > -- > Dr. Hannes ReineckeTeamlead Storage & Networking > h...@suse.de +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton > HRB 21284 (AG Nürnberg) Really appreciate your help. I will proceed the testing and keep you posted.
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On 10/27/2017 04:02 PM, Gavin Guo wrote: > Hi Hannes, > > Thank you for looking into the issue. If there is anything I can help > to test the patch? I appreciate your help. Thank you. > If you had checked linux-scsi you would have found this patch: '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is already scheduled for inclusion in 4.14. Anything else I could help you with? Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg)
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
Hi Hannes, Thank you for looking into the issue. If there is anything I can help to test the patch? I appreciate your help. Thank you. On Fri, Oct 6, 2017 at 9:08 AM, Gavin Guowrote: > On Mon, Oct 2, 2017 at 10:32 PM, Hannes Reinecke wrote: >> On 09/27/2017 09:33 AM, Gavin Guo wrote: >>> There is a problem in the latest upstream kernel with the device: >>> >>> $ grep -i lsi lspci >>> 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios >>> Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01) >>> >>> The device is simulated by the VMware ESXi 5.5 >>> >>> When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel >>> cannot automatically probe the disk. However, on the v3.19.0-80.88 >>> kernel, the disk can be dynamically probed and show the following >>> info message: >>> >>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, >>> sas_addr 0x5000c29a6bdae0f5 >>> scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 >>> ANSI: 2 >>> sd 2:0:1:0: Attached scsi generic sg2 type 0 >>> sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB) >>> sd 2:0:1:0: [sdb] Write Protect is off >>> sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00 >>> sd 2:0:1:0: [sdb] Cache data unavailable >>> sd 2:0:1:0: [sdb] Assuming drive cache: write through >>> sdb: unknown partition table >>> sd 2:0:1:0: [sdb] Attached SCSI disk >>> >>> After looking up the message: >>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, >>> sas_addr 0x5000c29a6bdae0f5 >>> >>> I found it comes from the path: >>> mptsas_firmware_event_work -> mptsas_send_sas_event -> >>> mptsas_hotplug_work -> mptsas_add_end_device >>> >>> I'll appreciate if anyone can give the idea: If it's possible that the >>> irq from the simulated LSI SAS controller didn't come in to trigger >>> the event? However, it can work on the v3.19 kernel so if there is >>> any driver implementation issue in the latest kernel. >>> >> This is an issue with the mptsas driver, who originally assumed that no >> system will have direct-attached SAS devices. >> VMWare chose to implement exactly that, so the hotplug detection logic >> is flawed here. >> I'll be sending a patch. >> >> Cheers, >> >> Hannes >> -- >> Dr. Hannes ReineckeTeamlead Storage & Networking >> h...@suse.de +49 911 74053 688 >> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg >> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton >> HRB 21284 (AG Nürnberg) > > Thank you in advance. Please add me to the CC list when you send out > the patch and I can help to verify that.
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On Mon, Oct 2, 2017 at 10:32 PM, Hannes Reineckewrote: > On 09/27/2017 09:33 AM, Gavin Guo wrote: >> There is a problem in the latest upstream kernel with the device: >> >> $ grep -i lsi lspci >> 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios >> Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01) >> >> The device is simulated by the VMware ESXi 5.5 >> >> When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel >> cannot automatically probe the disk. However, on the v3.19.0-80.88 >> kernel, the disk can be dynamically probed and show the following >> info message: >> >> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, >> sas_addr 0x5000c29a6bdae0f5 >> scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 >> ANSI: 2 >> sd 2:0:1:0: Attached scsi generic sg2 type 0 >> sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB) >> sd 2:0:1:0: [sdb] Write Protect is off >> sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00 >> sd 2:0:1:0: [sdb] Cache data unavailable >> sd 2:0:1:0: [sdb] Assuming drive cache: write through >> sdb: unknown partition table >> sd 2:0:1:0: [sdb] Attached SCSI disk >> >> After looking up the message: >> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, >> sas_addr 0x5000c29a6bdae0f5 >> >> I found it comes from the path: >> mptsas_firmware_event_work -> mptsas_send_sas_event -> >> mptsas_hotplug_work -> mptsas_add_end_device >> >> I'll appreciate if anyone can give the idea: If it's possible that the >> irq from the simulated LSI SAS controller didn't come in to trigger >> the event? However, it can work on the v3.19 kernel so if there is >> any driver implementation issue in the latest kernel. >> > This is an issue with the mptsas driver, who originally assumed that no > system will have direct-attached SAS devices. > VMWare chose to implement exactly that, so the hotplug detection logic > is flawed here. > I'll be sending a patch. > > Cheers, > > Hannes > -- > Dr. Hannes ReineckeTeamlead Storage & Networking > h...@suse.de +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton > HRB 21284 (AG Nürnberg) Thank you in advance. Please add me to the CC list when you send out the patch and I can help to verify that.
Re: mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
On 09/27/2017 09:33 AM, Gavin Guo wrote: > There is a problem in the latest upstream kernel with the device: > > $ grep -i lsi lspci > 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios > Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01) > > The device is simulated by the VMware ESXi 5.5 > > When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel > cannot automatically probe the disk. However, on the v3.19.0-80.88 > kernel, the disk can be dynamically probed and show the following > info message: > > mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, > sas_addr 0x5000c29a6bdae0f5 > scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 > ANSI: 2 > sd 2:0:1:0: Attached scsi generic sg2 type 0 > sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB) > sd 2:0:1:0: [sdb] Write Protect is off > sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00 > sd 2:0:1:0: [sdb] Cache data unavailable > sd 2:0:1:0: [sdb] Assuming drive cache: write through > sdb: unknown partition table > sd 2:0:1:0: [sdb] Attached SCSI disk > > After looking up the message: > mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, > sas_addr 0x5000c29a6bdae0f5 > > I found it comes from the path: > mptsas_firmware_event_work -> mptsas_send_sas_event -> > mptsas_hotplug_work -> mptsas_add_end_device > > I'll appreciate if anyone can give the idea: If it's possible that the > irq from the simulated LSI SAS controller didn't come in to trigger > the event? However, it can work on the v3.19 kernel so if there is > any driver implementation issue in the latest kernel. > This is an issue with the mptsas driver, who originally assumed that no system will have direct-attached SAS devices. VMWare chose to implement exactly that, so the hotplug detection logic is flawed here. I'll be sending a patch. Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg)
mptsas driver cannot detect hotplugging disk with the LSI SCSI SAS1068 controller in Ubuntu guest on VMware
There is a problem in the latest upstream kernel with the device: $ grep -i lsi lspci 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01) The device is simulated by the VMware ESXi 5.5 When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel cannot automatically probe the disk. However, on the v3.19.0-80.88 kernel, the disk can be dynamically probed and show the following info message: mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x5000c29a6bdae0f5 scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2 sd 2:0:1:0: Attached scsi generic sg2 type 0 sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB) sd 2:0:1:0: [sdb] Write Protect is off sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00 sd 2:0:1:0: [sdb] Cache data unavailable sd 2:0:1:0: [sdb] Assuming drive cache: write through sdb: unknown partition table sd 2:0:1:0: [sdb] Attached SCSI disk After looking up the message: mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x5000c29a6bdae0f5 I found it comes from the path: mptsas_firmware_event_work -> mptsas_send_sas_event -> mptsas_hotplug_work -> mptsas_add_end_device I'll appreciate if anyone can give the idea: If it's possible that the irq from the simulated LSI SAS controller didn't come in to trigger the event? However, it can work on the v3.19 kernel so if there is any driver implementation issue in the latest kernel.