RE: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors
> -Original Message- > From: Johannes Thumshirn [mailto:jthumsh...@suse.de] > Sent: Monday, October 17, 2016 2:34 AM > To: Christoph Hellwig > Cc: martin.peter...@oracle.com; Don Brace; ax...@kernel.dk; linux- > s...@vger.kernel.org; linux-block@vger.kernel.org > Subject: Re: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors > > EXTERNAL EMAIL > > > On Sat, Oct 15, 2016 at 10:47:21AM +0200, Christoph Hellwig wrote: > > Which cleans up a lot of the MSI-X handling, and allows us to use the > > PCI IRQ layer provided vector mapping, which we can then expose to blk- > mq. > > > > Signed-off-by: Christoph Hellwig > > --- > > Reviewed-by: Johannes Thumshirn > -- > Johannes Thumshirn Storage > jthumsh...@suse.de+49 911 74053 689 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: Felix Imendörffer, Jane Smithard, Graham Norton > HRB 21284 (AG Nürnberg) > Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850 Acked-by: Don Brace -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors
> -Original Message- > From: Christoph Hellwig [mailto:h...@lst.de] > Sent: Monday, October 17, 2016 9:21 AM > To: Don Brace > Cc: Johannes Thumshirn; Christoph Hellwig; martin.peter...@oracle.com; > ax...@kernel.dk; linux-s...@vger.kernel.org; linux-block@vger.kernel.org > Subject: Re: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors > > EXTERNAL EMAIL > > > Hi Don, > > did you also have a chance to test the patch and verify that the > queues are properly set up with a smartpqi controller? I need more time to test, I need to apply some other patches to fully test. -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors
> -Original Message- > From: Don Brace > Sent: Monday, October 17, 2016 8:45 AM > To: 'Johannes Thumshirn'; Christoph Hellwig > Cc: martin.peter...@oracle.com; ax...@kernel.dk; linux- > s...@vger.kernel.org; linux-block@vger.kernel.org > Subject: RE: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors > > > > -Original Message- > > From: Johannes Thumshirn [mailto:jthumsh...@suse.de] > > Sent: Monday, October 17, 2016 2:34 AM > > To: Christoph Hellwig > > Cc: martin.peter...@oracle.com; Don Brace; ax...@kernel.dk; linux- > > s...@vger.kernel.org; linux-block@vger.kernel.org > > Subject: Re: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors > > > > EXTERNAL EMAIL > > > > > > On Sat, Oct 15, 2016 at 10:47:21AM +0200, Christoph Hellwig wrote: > > > Which cleans up a lot of the MSI-X handling, and allows us to use the > > > PCI IRQ layer provided vector mapping, which we can then expose to blk- > > mq. > > > > > > Signed-off-by: Christoph Hellwig > > > --- > > > > Reviewed-by: Johannes Thumshirn > > -- > > Johannes Thumshirn Storage > > jthumsh...@suse.de+49 911 74053 689 > > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > > GF: Felix Imendörffer, Jane Smithard, Graham Norton > > HRB 21284 (AG Nürnberg) > > Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850 > > Acked-by: Don Brace Tested-by: Don Brace -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/2] genirq/affinity: try to make sure online CPU is assgined to irq vector
> -Original Message-
> From: Laurence Oberman [mailto:lober...@redhat.com]
> Sent: Tuesday, January 16, 2018 7:29 AM
> To: Thomas Gleixner ; Ming Lei
> Cc: Christoph Hellwig ; Jens Axboe ;
> linux-block@vger.kernel.org; linux-ker...@vger.kernel.org; Mike Snitzer
> ; Don Brace
> Subject: Re: [PATCH 0/2] genirq/affinity: try to make sure online CPU is
> assgined
> to irq vector
>
> > > It is because of irq_create_affinity_masks().
> >
> > That still does not answer the question. If the interrupt for a queue
> > is
> > assigned to an offline CPU, then the queue should not be used and
> > never
> > raise an interrupt. That's how managed interrupts have been designed.
> >
> > Thanks,
> >
> > tglx
> >
> >
> >
> >
>
> I captured a full boot log for this issue for Microsemi, I will send it
> to Don Brace.
> I enabled all the HPSA debug and here is snippet
>
>
> ..
> ..
> ..
> 246.751135] INFO: task systemd-udevd:413 blocked for more than 120
> seconds.
> [ 246.788008] Tainted: G I 4.15.0-rc4.noming+ #1
> [ 246.822380] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 246.865594] systemd-udevd D0 413411 0x8004
> [ 246.895519] Call Trace:
> [ 246.909713] ? __schedule+0x340/0xc20
> [ 246.930236] schedule+0x32/0x80
> [ 246.947905] schedule_timeout+0x23d/0x450
> [ 246.970047] ? find_held_lock+0x2d/0x90
> [ 246.991774] ? wait_for_completion_io+0x108/0x170
> [ 247.018172] io_schedule_timeout+0x19/0x40
> [ 247.041208] wait_for_completion_io+0x110/0x170
> [ 247.067326] ? wake_up_q+0x70/0x70
> [ 247.086801] hpsa_scsi_do_simple_cmd+0xc6/0x100 [hpsa]
> [ 247.114315] hpsa_scsi_do_simple_cmd_with_retry+0xb7/0x1c0 [hpsa]
> [ 247.146629] hpsa_scsi_do_inquiry+0x73/0xd0 [hpsa]
> [ 247.174118] hpsa_init_one+0x12cb/0x1a59 [hpsa]
This trace comes from internally generated discovery commands. No SCSI devices
have
been presented to the SML yet.
At this point we should be running on only one CPU. These commands are meant to
use
reply queue 0 which are tied to CPU 0. It's interesting that the patch helps.
However, I was wondering if you could inspect the iLo IML logs and send the
AHS logs for inspection.
Thanks,
Don Brace
ESC - Smart Storage
Microsemi Corporation
> [ 247.199851] ? __pm_runtime_resume+0x55/0x70
> [ 247.224527] local_pci_probe+0x3f/0xa0
> [ 247.246034] pci_device_probe+0x146/0x1b0
> [ 247.268413] driver_probe_device+0x2b3/0x4a0
> [ 247.291868] __driver_attach+0xda/0xe0
> [ 247.313370] ? driver_probe_device+0x4a0/0x4a0
> [ 247.338399] bus_for_each_dev+0x6a/0xb0
> [ 247.359912] bus_add_driver+0x41/0x260
> [ 247.380244] driver_register+0x5b/0xd0
> [ 247.400811] ? 0xc016b000
> [ 247.418819] hpsa_init+0x38/0x1000 [hpsa]
> [ 247.440763] ? 0xc016b000
> [ 247.459451] do_one_initcall+0x4d/0x19c
> [ 247.480539] ? do_init_module+0x22/0x220
> [ 247.502575] ? rcu_read_lock_sched_held+0x64/0x70
> [ 247.529549] ? kmem_cache_alloc_trace+0x1f7/0x260
> [ 247.556204] ? do_init_module+0x22/0x220
> [ 247.578633] do_init_module+0x5a/0x220
> [ 247.600322] load_module+0x21e8/0x2a50
> [ 247.621648] ? __symbol_put+0x60/0x60
> [ 247.642796] SYSC_finit_module+0x94/0xe0
> [ 247.665336] entry_SYSCALL_64_fastpath+0x1f/0x96
> [ 247.691751] RIP: 0033:0x7fc63d6527f9
> [ 247.712308] RSP: 002b:7ffdf1659ba8 EFLAGS: 0246 ORIG_RAX:
> 0139
> [ 247.755272] RAX: ffda RBX: 556b524c5f70 RCX:
> 7fc63d6527f9
> [ 247.795779] RDX: RSI: 7fc63df6f099 RDI:
> 0008
> [ 247.836413] RBP: 7fc63df6f099 R08: R09:
> 556b524be760
> [ 247.876395] R10: 0008 R11: 0246 R12:
>
> [ 247.917597] R13: 556b524c5f10 R14: 0002 R15:
>
> [ 247.957272]
> [ 247.957272] Showing all locks held in the system:
> [ 247.992019] 1 lock held by khungtaskd/118:
> [ 248.015019] #0: (tasklist_lock){.+.+}, at: [<4ef3538d>]
> debug_show_all_locks+0x39/0x1b0
> [ 248.064600] 2 locks held by systemd-udevd/413:
> [ 248.090031] #0: (&dev->mutex){}, at: [<2a395ec8>]
> __driver_attach+0x4a/0xe0
> [ 248.136620] #1: (&dev->mutex){}, at: [<d9def23c>]
> __driver_attach+0x58/0xe0
> [ 248.183245]
> [ 248.191675] =
> [ 248.191675]
> [ 314.825134] dracut-initqueue[437]: Warning: dracut-initqueue timeout
> - starting timeout scripts
> [ 315.368421] dracut-initque
RE: [PATCH 0/2] genirq/affinity: try to make sure online CPU is assgined to irq vector
> -Original Message-
> From: Ming Lei [mailto:ming@redhat.com]
> Sent: Thursday, February 01, 2018 4:37 AM
> To: Don Brace
> Cc: Laurence Oberman ; Thomas Gleixner
> ; Christoph Hellwig ; Jens Axboe
> ; linux-block@vger.kernel.org; linux-ker...@vger.kernel.org;
> Mike Snitzer
> Subject: Re: [PATCH 0/2] genirq/affinity: try to make sure online CPU is
> assgined
> to irq vector
>
> EXTERNAL EMAIL
>
>
> On Tue, Jan 16, 2018 at 03:22:18PM +, Don Brace wrote:
> > > -Original Message-
> > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > Sent: Tuesday, January 16, 2018 7:29 AM
> > > To: Thomas Gleixner ; Ming Lei
> > > Cc: Christoph Hellwig ; Jens Axboe ;
> > > linux-block@vger.kernel.org; linux-ker...@vger.kernel.org; Mike Snitzer
> > > ; Don Brace
> > > Subject: Re: [PATCH 0/2] genirq/affinity: try to make sure online CPU is
> assgined
> > > to irq vector
> > >
> > > > > It is because of irq_create_affinity_masks().
> > > >
> > > > That still does not answer the question. If the interrupt for a queue
> > > > is
> > > > assigned to an offline CPU, then the queue should not be used and
> > > > never
> > > > raise an interrupt. That's how managed interrupts have been designed.
> > > >
> > > > Thanks,
> > > >
> > > > tglx
> > > >
> > > >
> > > >
> > > >
> > >
> > > I captured a full boot log for this issue for Microsemi, I will send it
> > > to Don Brace.
> > > I enabled all the HPSA debug and here is snippet
> > >
> > >
> > > ..
> > > ..
> > > ..
> > > 246.751135] INFO: task systemd-udevd:413 blocked for more than 120
> > > seconds.
> > > [ 246.788008] Tainted: G I 4.15.0-rc4.noming+ #1
> > > [ 246.822380] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> > > disables this message.
> > > [ 246.865594] systemd-udevd D0 413411 0x8004
> > > [ 246.895519] Call Trace:
> > > [ 246.909713] ? __schedule+0x340/0xc20
> > > [ 246.930236] schedule+0x32/0x80
> > > [ 246.947905] schedule_timeout+0x23d/0x450
> > > [ 246.970047] ? find_held_lock+0x2d/0x90
> > > [ 246.991774] ? wait_for_completion_io+0x108/0x170
> > > [ 247.018172] io_schedule_timeout+0x19/0x40
> > > [ 247.041208] wait_for_completion_io+0x110/0x170
> > > [ 247.067326] ? wake_up_q+0x70/0x70
> > > [ 247.086801] hpsa_scsi_do_simple_cmd+0xc6/0x100 [hpsa]
> > > [ 247.114315] hpsa_scsi_do_simple_cmd_with_retry+0xb7/0x1c0 [hpsa]
> > > [ 247.146629] hpsa_scsi_do_inquiry+0x73/0xd0 [hpsa]
> > > [ 247.174118] hpsa_init_one+0x12cb/0x1a59 [hpsa]
> >
> > This trace comes from internally generated discovery commands. No SCSI
> devices have
> > been presented to the SML yet.
> >
> > At this point we should be running on only one CPU. These commands are
> meant to use
> > reply queue 0 which are tied to CPU 0. It's interesting that the patch
> > helps.
> >
> > However, I was wondering if you could inspect the iLo IML logs and send the
> > AHS logs for inspection.
>
> Hello Don,
>
> Now the patch has been merged to linus tree as:
>
> 84676c1f21e8ff54b ("genirq/affinity: assign vectors to all possible CPUs")
>
> and it breaks Laurence's machine completely, :-(
>
> I just take a look at HPSA's code, and found that reply queue is chosen
> in the following way in most of code path:
>
> if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> cp->ReplyQueue = smp_processor_id() % h->nreply_queues;
>
> h->nreply_queues is the msix vector number which is returned from
> pci_alloc_irq_vectors(), and now some of vectors may be mapped to all
> offline CPUs, for example, one processor isn't plugged to socket.
>
> If I understand correctly, 'cp->ReplyQueue' is aligned to one irq
> vector, and the command is expected by handled via that irq vector,
> is it right?
>
> If yes, now I guess this way can't work any more if number of online
> CPUs is >= h->nreply_queues, and you may need to check the cpu affinity
> of one vector before choosing the reply queue, and block/blk-mq-pci.c
> may be helpful for you.
>
> Thanks,
> Ming
Thanks Ming,
I start working up a patch.
RE: [PATCH V2 8/8] scsi: hpsa: use blk_mq to solve irq affinity issue
> -Original Message- > This is a critical issue on the HPSA because Linus already has the > original commit that causes the system to fail to boot. > > All my testing was on DL380 G7 servers with: > > Hewlett-Packard Company Smart Array G6 controllers > Vendor: HP Model: P410iRev: 6.64 > > Ming's patch fixes this so we need to try move this along. > > I have a DL380 G8 as well which is also likely exposed here and I added > Don Brace for FYI to this list. > > Thanks Ming Running some tests now.
RE: [PATCH V2 8/8] scsi: hpsa: use blk_mq to solve irq affinity issue
> -Original Message- > From: Laurence Oberman [mailto:lober...@redhat.com] > Sent: Monday, February 05, 2018 9:58 AM > To: Ming Lei ; Jens Axboe ; linux- > bl...@vger.kernel.org; Christoph Hellwig ; Mike Snitzer > ; Don Brace > Cc: linux-s...@vger.kernel.org; Hannes Reinecke ; Arun Easi > ; Omar Sandoval ; Martin K . > Petersen ; James Bottomley > ; Christoph Hellwig ; > Don Brace ; Kashyap Desai > ; Peter Rivera ; > Paolo Bonzini > Subject: Re: [PATCH V2 8/8] scsi: hpsa: use blk_mq to solve irq affinity issue > > EXTERNAL EMAIL > > > On Mon, 2018-02-05 at 23:20 +0800, Ming Lei wrote: > > This patch uses .force_blk_mq to drive HPSA via SCSI_MQ, meantime > > maps > > each reply queue to blk_mq's hw queue, then .queuecommand can always > > choose the hw queue as the reply queue. And if no any online CPU is > > mapped to one hw queue, request can't be submitted to this hw queue > > at all, finally the irq affinity issue is solved. > > > > Cc: Hannes Reinecke > > Cc: Arun Easi > > Cc: Omar Sandoval , > > Cc: "Martin K. Petersen" , > > Cc: James Bottomley , > > Cc: Christoph Hellwig , > > Cc: Don Brace > > Cc: Kashyap Desai > > Cc: Peter Rivera > > Cc: Paolo Bonzini > > Cc: Mike Snitzer > > Tested-by: Laurence Oberman > > Signed-off-by: Ming Lei > > --- > > drivers/scsi/hpsa.c | 51 ++- > This is a critical issue on the HPSA because Linus already has the > original commit that causes the system to fail to boot. > > All my testing was on DL380 G7 servers with: > > Hewlett-Packard Company Smart Array G6 controllers > Vendor: HP Model: P410i Rev: 6.64 > > Ming's patch fixes this so we need to try move this along. > > I have a DL380 G8 as well which is also likely exposed here and I added > Don Brace for FYI to this list. > > Thanks Ming Tested-by: Don Brace P441, P431, P830i, H240 Acked-by: Don Brace
RE: [PATCH V2 7/8] scsi: hpsa: call hpsa_hba_inquiry() after adding host
> -Original Message- > From: Ming Lei [mailto:ming@redhat.com] > Sent: Monday, February 05, 2018 9:21 AM > To: Jens Axboe ; linux-block@vger.kernel.org; Christoph > Hellwig ; Mike Snitzer > Cc: linux-s...@vger.kernel.org; Hannes Reinecke ; Arun Easi > ; Omar Sandoval ; Martin K . > Petersen ; James Bottomley > ; Christoph Hellwig ; > Don Brace ; Kashyap Desai > ; Peter Rivera ; > Paolo Bonzini ; Laurence Oberman > ; Ming Lei > Subject: [PATCH V2 7/8] scsi: hpsa: call hpsa_hba_inquiry() after adding host > > EXTERNAL EMAIL > > > So that we can decide the default reply queue by the map created > during adding host. > > Cc: Hannes Reinecke > Cc: Arun Easi > Cc: Omar Sandoval , > Cc: "Martin K. Petersen" , > Cc: James Bottomley , > Cc: Christoph Hellwig , > Cc: Don Brace > Cc: Kashyap Desai > Cc: Peter Rivera > Cc: Paolo Bonzini > Cc: Mike Snitzer > Tested-by: Laurence Oberman > Signed-off-by: Ming Lei > --- > drivers/scsi/hpsa.c | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c > index 287e5eb0723f..443eabf63a9f 100644 > --- a/drivers/scsi/hpsa.c > +++ b/drivers/scsi/hpsa.c > @@ -8691,12 +8691,9 @@ static int hpsa_init_one(struct pci_dev *pdev, const > struct pci_device_id *ent) > /* Disable discovery polling.*/ > h->discovery_polling = 0; > > - > /* Turn the interrupts on so we can service requests */ > h->access.set_intr_mask(h, HPSA_INTR_ON); > > - hpsa_hba_inquiry(h); > - > h->lastlogicals = kzalloc(sizeof(*(h->lastlogicals)), GFP_KERNEL); > if (!h->lastlogicals) > dev_info(&h->pdev->dev, > @@ -8707,6 +8704,8 @@ static int hpsa_init_one(struct pci_dev *pdev, const > struct pci_device_id *ent) > if (rc) > goto clean7; /* perf, sg, cmd, irq, shost, pci, lu, aer/h */ > > + hpsa_hba_inquiry(h); > + > /* Monitor the controller for firmware lockups */ > h->heartbeat_sample_interval = HEARTBEAT_SAMPLE_INTERVAL; > INIT_DELAYED_WORK(&h->monitor_ctlr_work, hpsa_monitor_ctlr_worker); > -- > 2.9.5 Tested-by: Don Brace P441, P431, P830i, H240 Acked-by: Don Brace
RE: [PATCH V2 4/8] block: null_blk: introduce module parameter of 'g_global_tags'
> This patch introduces the parameter of 'g_global_tags' so that we can > test this feature by null_blk easiy. > > Not see obvious performance drop with global_tags when the whole hw > depth is kept as same: > > 1) no 'global_tags', each hw queue depth is 1, and 4 hw queues > modprobe null_blk queue_mode=2 nr_devices=4 shared_tags=1 global_tags=0 > submit_queues=4 hw_queue_depth=1 > > 2) 'global_tags', global hw queue depth is 4, and 4 hw queues > modprobe null_blk queue_mode=2 nr_devices=4 shared_tags=1 global_tags=1 > submit_queues=4 hw_queue_depth=4 > > 3) fio test done in above two settings: >fio --bs=4k --size=512G --rw=randread --norandommap --direct=1 -- > ioengine=libaio --iodepth=4 --runtime=$RUNTIME --group_reporting=1 -- > name=nullb0 --filename=/dev/nullb0 --name=nullb1 --filename=/dev/nullb1 -- > name=nullb2 --filename=/dev/nullb2 --name=nullb3 --filename=/dev/nullb3 > > 1M IOPS can be reached in both above tests which is done in one VM. > I am getting 1.1M IOPS for both cases. Tested-by: Don Brace
RE: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
> -Original Message-
> From: Ming Lei [mailto:ming@redhat.com]
> Sent: Tuesday, February 27, 2018 4:08 AM
> To: Jens Axboe ; linux-block@vger.kernel.org; Christoph
> Hellwig ; Mike Snitzer
> Cc: linux-s...@vger.kernel.org; Hannes Reinecke ; Arun Easi
> ; Omar Sandoval ; Martin K .
> Petersen ; James Bottomley
> ; Christoph Hellwig ;
> Don Brace ; Kashyap Desai
> ; Peter Rivera ;
> Laurence Oberman ; Ming Lei
> ; Meelis Roos
> Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
>
> EXTERNAL EMAIL
>
>
> From 84676c1f21 (genirq/affinity: assign vectors to all possible CPUs),
> one msix vector can be created without any online CPU mapped, then one
> command's completion may not be notified.
>
> This patch setups mapping between cpu and reply queue according to irq
> affinity info retrived by pci_irq_get_affinity(), and uses this mapping
> table to choose reply queue for queuing one command.
>
> Then the chosen reply queue has to be active, and fixes IO hang caused
> by using inactive reply queue which doesn't have any online CPU mapped.
>
> Cc: Hannes Reinecke
> Cc: Arun Easi
> Cc: "Martin K. Petersen" ,
> Cc: James Bottomley ,
> Cc: Christoph Hellwig ,
> Cc: Don Brace
> Cc: Kashyap Desai
> Cc: Peter Rivera
> Cc: Laurence Oberman
> Cc: Meelis Roos
> Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all possible CPUs")
> Signed-off-by: Ming Lei
I am getting some issues that need to be tracked down:
[ 1636.032984] hpsa :87:00.0: Acknowledging event: 0xc032 (HP SSD Smart
Path configuration change)
[ 1638.510656] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-Access HP
MO0400JDVEU PHYS DRV SSDSmartPathCap- En- Exp=0
[ 1653.967695] hpsa :87:00.0: Acknowledging event: 0x8020 (HP SSD Smart
Path configuration change)
[ 1656.770377] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-Access HP
MO0400JDVEU PHYS DRV SSDSmartPathCap- En- Exp=0
[ 2839.762267] hpsa :87:00.0: Acknowledging event: 0x8020 (HP SSD Smart
Path configuration change)
[ 2840.841290] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-Access HP
MO0400JDVEU PHYS DRV SSDSmartPathCap- En- Exp=0
[ 2917.582653] hpsa :87:00.0: Acknowledging event: 0xc020 (HP SSD Smart
Path configuration change)
[ 2919.087191] hpsa :87:00.0: scsi 3:1:0:1: updated Direct-Access HP
LOGICAL VOLUME RAID-5 SSDSmartPathCap+ En+ Exp=1
[ 2919.142527] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 2919.203915] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 2919.266921] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 2934.999629] hpsa :87:00.0: Acknowledging event: 0x4000 (HP SSD Smart
Path state change)
[ 2936.937333] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 2936.998707] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 2937.060101] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs: [3:1:0:2] A phys
disk component of LV is missing, turning off offload_enabled for LV.
[ 3619.711122] sd 3:1:0:3: [sde] tag#436 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
[ 3619.751150] sd 3:1:0:3: [sde] tag#436 Sense Key : Aborted Command [current]
[ 3619.784375] sd 3:1:0:3: [sde] tag#436 Add. Sense: Internal target failure
[ 3619.816530] sd 3:1:0:3: [sde] tag#436 CDB: Read(10) 28 00 01 1b ad af 00 00
01 00
[ 3619.852295] print_req_error: I/O error, dev sde, sector 18591151
[ 3619.880850] sd 3:1:0:3: [sde] tag#461 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
[ 3619.920981] sd 3:1:0:3: [sde] tag#461 Sense Key : Aborted Command [current]
[ 3619.955081] sd 3:1:0:3: [sde] tag#461 Add. Sense: Internal target failure
[ 3619.987054] sd 3:1:0:3: [sde] tag#461 CDB: Read(10) 28 00 02 15 31 40 00 00
01 00
[ 3620.022569] print_req_error: I/O error, dev sde, sector 34943296
[ 3620.050873] sd 3:1:0:3: [sde] tag#157 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
[ 3620.091124] sd 3:1:0:3: [sde] tag#157 Sense Key : Aborted Command [current]
[ 3620.124179] sd 3:1:0:3: [sde] tag#157 Add. Sense: Internal target failure
[ 3620.156203] sd 3:1:0:3: [sde] tag#157 CDB: Read(10) 28 00 03 65 9d 7e 00 00
01 00
[ 3620.191520] print_req_error: I/O error, dev sde, sector 56991102
[ 3620.220308] sd 3:1:0:3: [sde] tag#266 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
[ 3620.260273] sd 3:1:0:3: [sde] tag#266 Sense Key : Aborted Command [current]
[
RE: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
> -Original Message- > From: Laurence Oberman [mailto:lober...@redhat.com] > Sent: Friday, March 02, 2018 8:09 AM > To: Ming Lei > Cc: Don Brace ; Jens Axboe ; > linux-block@vger.kernel.org; Christoph Hellwig ; Mike > Snitzer ; linux-s...@vger.kernel.org; Hannes Reinecke > ; Arun Easi ; Omar Sandoval > ; Martin K . Petersen ; James > Bottomley ; Christoph Hellwig > ; Kashyap Desai ; Peter Rivera > ; Meelis Roos > Subject: Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue > > EXTERNAL EMAIL > > > On Fri, 2018-03-02 at 10:16 +0800, Ming Lei wrote: > > On Thu, Mar 01, 2018 at 04:19:34PM -0500, Laurence Oberman wrote: > > > On Thu, 2018-03-01 at 14:01 -0500, Laurence Oberman wrote: > > > > On Thu, 2018-03-01 at 16:18 +, Don Brace wrote: > > > > > > -Original Message- > > > > > > From: Ming Lei [mailto:ming@redhat.com] > > > > > > Sent: Tuesday, February 27, 2018 4:08 AM > > > > > > To: Jens Axboe ; linux-block@vger.kernel.org > > > > > > ; > > > > > > Christoph > > > > > > Hellwig ; Mike Snitzer > > > > > > > > > > > > Cc: linux-s...@vger.kernel.org; Hannes Reinecke > > > > > >; > > > > > > Arun Easi > > > > > > ; Omar Sandoval ; > > > > > > Martin K > > > > > > . > > > > > > Petersen ; James Bottomley > > > > > > ; Christoph Hellwig > > > > > ch@l > > > > > > st > > > > > > .de>; > > > > > > Don Brace ; Kashyap Desai > > > > > > ; Peter Rivera > > > > > dcom > > > > > > .c > > > > > > om>; > > > > > > Laurence Oberman ; Ming Lei > > > > > > ; Meelis Roos > > > > > > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply > > > > > > queue > > > > > > > > Seems Don run into IO failure without blk-mq, could you run your > > tests again > > in legacy mode? > > > > Thanks, > > Ming > > Hello Ming > I ran multiple passes on Legacy and still see no issues in my test bed > > BOOT_IMAGE=/vmlinuz-4.16.0-rc2.ming+ root=UUID=43f86d71-b1bf-4789- > a28e- > 21c6ddc90195 ro crashkernel=256M@64M log_buf_len=64M > console=ttyS1,115200n8 > > HEAD of the git kernel I am using > > 694e16f scsi: megaraid: improve scsi_mq performance via .host_tagset > 793686c scsi: hpsa: improve scsi_mq performance via .host_tagset > 60d5b36 block: null_blk: introduce module parameter of 'g_host_tags' > 8847067 scsi: Add template flag 'host_tagset' > a8fbdd6 blk-mq: introduce BLK_MQ_F_HOST_TAGS > 4710fab blk-mq: introduce 'start_tag' field to 'struct blk_mq_tags' > 09bb153 scsi: megaraid_sas: fix selection of reply queue > 52700d8 scsi: hpsa: fix selection of reply queue I checkout out Linus's tree (4.16.0-rc3+) and re-applied the above patches. I and have been running 24 hours with no issues. Evidently my forked copy was corrupted. So, my I/O testing has gone well. I'll run some performance numbers next. Thanks, Don
RE: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
> -Original Message- > From: Kashyap Desai [mailto:kashyap.de...@broadcom.com] > Sent: Monday, March 05, 2018 1:24 AM > To: Laurence Oberman ; Don Brace > ; Ming Lei > Cc: Jens Axboe ; linux-block@vger.kernel.org; Christoph > Hellwig ; Mike Snitzer ; linux- > s...@vger.kernel.org; Hannes Reinecke ; Arun Easi > ; Omar Sandoval ; Martin K . > Petersen ; James Bottomley > ; Christoph Hellwig ; > Peter Rivera ; Meelis Roos > Subject: RE: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue > > EXTERNAL EMAIL > > > > -Original Message- > > From: Laurence Oberman [mailto:lober...@redhat.com] > > Sent: Saturday, March 3, 2018 3:23 AM > > To: Don Brace; Ming Lei > > Cc: Jens Axboe; linux-block@vger.kernel.org; Christoph Hellwig; Mike > > Snitzer; > > linux-s...@vger.kernel.org; Hannes Reinecke; Arun Easi; Omar Sandoval; > > Martin K . Petersen; James Bottomley; Christoph Hellwig; Kashyap Desai; > > Peter > > Rivera; Meelis Roos > > Subject: Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue > > > > On Fri, 2018-03-02 at 15:03 +, Don Brace wrote: > > > > -Original Message- > > > > From: Laurence Oberman [mailto:lober...@redhat.com] > > > > Sent: Friday, March 02, 2018 8:09 AM > > > > To: Ming Lei > > > > Cc: Don Brace ; Jens Axboe > > > k>; > > > > linux-block@vger.kernel.org; Christoph Hellwig ; > > > > Mike Snitzer ; linux-s...@vger.kernel.org; > > > > Hannes Reinecke ; Arun Easi ; > > > > Omar Sandoval ; Martin K . Petersen > > > > ; James Bottomley > > > > ; Christoph Hellwig > > > > ; Kashyap Desai ; Peter > > > > Rivera ; Meelis Roos > > > > Subject: Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue > > > > > > > > EXTERNAL EMAIL > > > > > > > > > > > > On Fri, 2018-03-02 at 10:16 +0800, Ming Lei wrote: > > > > > On Thu, Mar 01, 2018 at 04:19:34PM -0500, Laurence Oberman wrote: > > > > > > On Thu, 2018-03-01 at 14:01 -0500, Laurence Oberman wrote: > > > > > > > On Thu, 2018-03-01 at 16:18 +, Don Brace wrote: > > > > > > > > > -Original Message- > > > > > > > > > From: Ming Lei [mailto:ming@redhat.com] > > > > > > > > > Sent: Tuesday, February 27, 2018 4:08 AM > > > > > > > > > To: Jens Axboe ; linux-block@vger.kernel > > > > > > > > > .org ; Christoph Hellwig ; Mike Snitzer > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cc: linux-s...@vger.kernel.org; Hannes Reinecke > > > > > > > > e.de > > > > > > > > > > ; > > > > > > > > > > > > > > > > > > Arun Easi > > > > > > > > > ; Omar Sandoval ; > > > > > > > > > Martin K . > > > > > > > > > Petersen ; James Bottomley > > > > > > > > > ; Christoph Hellwig > > > > > > > > > ; Don Brace ; > > > > > > > > > Kashyap Desai ; Peter Rivera > > > > > > > > > > > > > > > > > om>; > > > > > > > > > Laurence Oberman ; Ming Lei > > > > > > > > > ; Meelis Roos > > > > > > > > > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply > > > > > > > > > queue > > > > > > > > > > > > > > > > > > > Seems Don run into IO failure without blk-mq, could you run your > > > > > tests again in legacy mode? > > > > > > > > > > Thanks, > > > > > Ming > > > > > > > > Hello Ming > > > > I ran multiple passes on Legacy and still see no issues in my test > > > > bed > > > > Tests ran all weekend without issues. > > > > BOOT_IMAGE=/vmlinuz-4.16.0-rc2.ming+ root=UUID=43f86d71-b1bf- > 4789- > > > > a28e- > > > > 21c6ddc90195 ro crashkernel=256M@64M log_buf_len=64M > > > > console=ttyS1,115200n8 > > > > > > > > HEAD of the git kernel I am using > > > > > > > > 694e16f scsi: megaraid: improve scsi_mq performance via .host_tagset > >
Re: [PATCH V4 1/4] scsi: hpsa: fix selection of reply queue
From: Ming Lei
Sent: Thursday, March 8, 2018 7:32 PM
To: James Bottomley; Jens Axboe; Martin K . Petersen
Cc: Christoph Hellwig; linux-s...@vger.kernel.org; linux-block@vger.kernel.org;
Meelis Roos; Don Brace; Kashyap Desai; Laurence Oberman; Mike Snitzer; Ming
Lei; Hannes Reinecke; James Bottomley; Artem Bityutskiy
Subject: [PATCH V4 1/4] scsi: hpsa: fix selection of reply queue
EXTERNAL EMAIL
>From 84676c1f21 (genirq/affinity: assign vectors to all possible CPUs),
one msix vector can be created without any online CPU mapped, then one
command's completion may not be notified.
This patch setups mapping between cpu and reply queue according to irq
affinity info retrived by pci_irq_get_affinity(), and uses this mapping
table to choose reply queue for queuing one command.
Then the chosen reply queue has to be active, and fixes IO hang caused
by using inactive reply queue which doesn't have any online CPU mapped.
Cc: Hannes Reinecke
Cc: "Martin K. Petersen" ,
Cc: James Bottomley ,
Cc: Christoph Hellwig ,
Cc: Don Brace
Cc: Kashyap Desai
Cc: Laurence Oberman
Cc: Meelis Roos
Cc: Artem Bityutskiy
Cc: Mike Snitzer
Tested-by: Laurence Oberman
Tested-by: Don Brace
Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all possible CPUs")
Signed-off-by: Ming Lei
I got the following stack trace while testing: I need to pop off the patches
and re-test for a baseline.
[root@cyflym ~]# [18564.263896] XFS (dm-2): _xfs_buf_find: daddr 0x282084848
out of range, EOFS 0x7298000
[18564.301491] WARNING: CPU: 51 PID: 18275 at fs/xfs/xfs_buf.c:591
_xfs_buf_find+0x3f0/0x530 [xfs]
[18564.342614] Modules linked in: sg ip6t_rpfilter ip6t_REJECT nf_reject_ipv6
nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4
nf_defrag_ipv4 xt_conntrack nf_conntrack cfg80211 rfkill ebtable_nat
ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_mangle
ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_mangle
iptable_security iptable_raw iptable_filter ip_tables sb_edac
x86_pkg_temp_thermal coretemp kvm_intel kvm irqbypass crct10dif_pclmul
crc32_pclmul ghash_clmulni_intel pcbc iTCO_wdt aesni_intel iTCO_vendor_support
crypto_simd glue_helper cryptd pcspkr ipmi_si hpilo hpwdt lpc_ich ioatdma
pcc_cpufreq shpchp dca mfd_core wmi ipmi_msghandler acpi_power_meter uinput xfs
libcrc32c mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt
[18564.678017] sd_mod fb_sys_fops ttm drm crc32c_intel tg3 hpsa i2c_core
scsi_transport_sas usb_storage dm_mirror dm_region_hash dm_log dm_mod dax
[18564.739543] CPU: 51 PID: 18275 Comm: bash Not tainted 4.16.0-rc4+ #14
[18564.769923] Hardware name: HP ProLiant DL580 Gen8, BIOS P79 08/18/2016
[18564.80] RIP: 0010:_xfs_buf_find+0x3f0/0x530 [xfs]
[18564.825121] RSP: 0018:9f0aaabaf6b8 EFLAGS: 00010246
[18564.849811] RAX: RBX: 9f0aaabaf808 RCX:
[18564.883604] RDX: 9f0aaabaf5d8 RSI: 000a RDI: c046ad77
[18564.917315] RBP: 0001 R08: R09: 0021
[18564.951211] R10: R11: 000a R12: 8ade9c88dbc0
[18564.984925] R13: 8ade9c88dbc0 R14: 0001 R15: 9f0aaabaf808
[18565.018846] FS: 7f423c899740() GS:8aee9ef8()
knlGS:
[18565.057473] CS: 0010 DS: ES: CR0: 80050033
[18565.084562] CR2: 7ffc480f8070 CR3: 00105b8ce006 CR4: 001606e0
[18565.118377] Call Trace:
[18565.129851] ? _cond_resched+0x15/0x30
[18565.147590] xfs_buf_get_map+0x23/0x260 [xfs]
[18565.168557] xfs_buf_read_map+0x29/0x180 [xfs]
[18565.189845] xfs_trans_read_buf_map+0xec/0x300 [xfs]
[18565.213354] xfs_btree_read_buf_block.constprop.36+0x77/0xd0 [xfs]
[18565.242721] xfs_btree_lookup_get_block+0x82/0x170 [xfs]
[18565.268117] xfs_btree_lookup+0xce/0x3c0 [xfs]
[18565.289218] ? kmem_zone_alloc+0x95/0x100 [xfs]
[18565.310659] xfs_free_ag_extent+0x93/0x830 [xfs]
[18565.332491] xfs_free_extent+0xb6/0x150 [xfs]
[18565.353187] xfs_trans_free_extent+0x4f/0x110 [xfs]
[18565.376544] ? xfs_trans_add_item+0x50/0x80 [xfs]
[18565.399174] xfs_extent_free_finish_item+0x21/0x30 [xfs]
[18565.424638] xfs_defer_finish+0x13d/0x400 [xfs]
[18565.446007] xfs_itruncate_extents+0x11d/0x2d0 [xfs]
[18565.469501] xfs_setattr_size+0x275/0x300 [xfs]
[18565.490808] xfs_vn_setattr+0x40/0x60 [xfs]
[18565.510577] notify_change+0x269/0x440
[18565.529105] do_truncate+0x72/0xc0
[18565.545982] path_openat+0x5ed/0x1210
[18565.563916] ? xfs_iext_lookup_extent+0x60/0x140 [xfs]
[18565.588955] ? xfs_bmapi_read+0x158/0x330 [xfs]
[18565.611077] do_filp_open+0x91/0x100
[18565.628643] ? xfs_iunlock+0xb9/0x110 [xfs]
[18565.649355] do_sys_open+0x126/0x210
[18565.666880] do_syscall_64+0x6e/0x1a0
[18565.684795] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[18565.709158] RIP: 0033:0x7f423bf85a20
[18565.7
RE: [PATCH V4 1/4] scsi: hpsa: fix selection of reply queue
> -Original Message-
> From: Ming Lei [mailto:ming@redhat.com]
> Sent: Thursday, March 08, 2018 9:32 PM
> To: James Bottomley ; Jens Axboe
> ; Martin K . Petersen
> Cc: Christoph Hellwig ; linux-s...@vger.kernel.org; linux-
> bl...@vger.kernel.org; Meelis Roos ; Don Brace
> ; Kashyap Desai
> ; Laurence Oberman
> ; Mike Snitzer ; Ming Lei
> ; Hannes Reinecke ; James Bottomley
> ; Artem Bityutskiy
>
> Subject: [PATCH V4 1/4] scsi: hpsa: fix selection of reply queue
>
> EXTERNAL EMAIL
>
>
> From 84676c1f21 (genirq/affinity: assign vectors to all possible CPUs),
> one msix vector can be created without any online CPU mapped, then one
> command's completion may not be notified.
>
> This patch setups mapping between cpu and reply queue according to irq
> affinity info retrived by pci_irq_get_affinity(), and uses this mapping
> table to choose reply queue for queuing one command.
>
> Then the chosen reply queue has to be active, and fixes IO hang caused
> by using inactive reply queue which doesn't have any online CPU mapped.
>
> Cc: Hannes Reinecke
> Cc: "Martin K. Petersen" ,
> Cc: James Bottomley ,
> Cc: Christoph Hellwig ,
> Cc: Don Brace
> Cc: Kashyap Desai
> Cc: Laurence Oberman
> Cc: Meelis Roos
> Cc: Artem Bityutskiy
> Cc: Mike Snitzer
> Tested-by: Laurence Oberman
> Tested-by: Don Brace
> Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all possible CPUs")
> Signed-off-by: Ming Lei
> ---
Acked-by: Don Brace
Tested-by: Don Brace
* Rebuilt test rig: applied the following patches to Linus's tree
4.16.0-rc4+:
[PATCH V4 1_4] scsi: hpsa: fix selection of reply queue - Ming
Lei - 2018-03-08 2132.eml
[PATCH V4 3_4] scsi: introduce force_blk_mq - Ming Lei
- 2018-03-08 2132.eml
* fio tests on 6 LVs on P441 controller (fw 6.59) 5 days.
* fio tests on 10 HBA disks on P431 (fw 4.54) controller. 3 days. (
concurrent with P441 tests)
> drivers/scsi/hpsa.c | 73
> +++--
> drivers/scsi/hpsa.h | 1 +
> 2 files changed, 55 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
> index 5293e6827ce5..3a9eca163db8 100644
> --- a/drivers/scsi/hpsa.c
> +++ b/drivers/scsi/hpsa.c
> @@ -1045,11 +1045,7 @@ static void set_performant_mode(struct ctlr_info
> *h, struct CommandList *c,
> c->busaddr |= 1 | (h->blockFetchTable[c->Header.SGList] << 1);
> if (unlikely(!h->msix_vectors))
> return;
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - c->Header.ReplyQueue =
> - raw_smp_processor_id() % h->nreply_queues;
> - else
> - c->Header.ReplyQueue = reply_queue % h->nreply_queues;
> + c->Header.ReplyQueue = reply_queue;
> }
> }
>
> @@ -1063,10 +1059,7 @@ static void set_ioaccel1_performant_mode(struct
> ctlr_info *h,
> * Tell the controller to post the reply to the queue for this
> * processor. This seems to give the best I/O throughput.
> */
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - cp->ReplyQueue = smp_processor_id() % h->nreply_queues;
> - else
> - cp->ReplyQueue = reply_queue % h->nreply_queues;
> + cp->ReplyQueue = reply_queue;
> /*
> * Set the bits in the address sent down to include:
> * - performant mode bit (bit 0)
> @@ -1087,10 +1080,7 @@ static void
> set_ioaccel2_tmf_performant_mode(struct ctlr_info *h,
> /* Tell the controller to post the reply to the queue for this
> * processor. This seems to give the best I/O throughput.
> */
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - cp->reply_queue = smp_processor_id() % h->nreply_queues;
> - else
> - cp->reply_queue = reply_queue % h->nreply_queues;
> + cp->reply_queue = reply_queue;
> /* Set the bits in the address sent down to include:
> * - performant mode bit not used in ioaccel mode 2
> * - pull count (bits 0-3)
> @@ -1109,10 +1099,7 @@ static void set_ioaccel2_performant_mode(struct
> ctlr_info *h,
> * Tell the controller to post the reply to the queue for this
> * processor. This seems to give the best I/O throughput.
> */
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - cp->reply_queue = smp_processor_id() % h->nreply_queues;
> -
RE: [PATCH 1/3] blk-mq: Allow PCI vector offset for mapping queues
> -Original Message-
> From: linux-scsi-ow...@vger.kernel.org [mailto:linux-scsi-
> ow...@vger.kernel.org] On Behalf Of Keith Busch
> Sent: Tuesday, March 27, 2018 10:39 AM
> To: Linux NVMe ; Linux Block bl...@vger.kernel.org>
> Cc: Christoph Hellwig ; Sagi Grimberg ;
> Jianchao Wang ; Ming Lei
> ; Jens Axboe ; Keith Busch
> ; Don Brace ; qla2xxx-
> upstr...@qlogic.com; linux-s...@vger.kernel.org
> Subject: [PATCH 1/3] blk-mq: Allow PCI vector offset for mapping queues
>
> EXTERNAL EMAIL
>
>
> The PCI interrupt vectors intended to be associated with a queue may
> not start at 0; a driver may allocate pre_vectors for special use. This
> patch adds an offset parameter so blk-mq may find the intended affinity
> mask and updates all drivers using this API accordingly.
>
> Cc: Don Brace
> Cc:
> Cc:
> Signed-off-by: Keith Busch
> ---
> v1 -> v2:
>
> Update blk-mq API directly instead of chaining a default parameter to
> a new API, and update all drivers accordingly.
>
> block/blk-mq-pci.c| 6 --
> drivers/nvme/host/pci.c | 2 +-
> drivers/scsi/qla2xxx/qla_os.c | 2 +-
> drivers/scsi/smartpqi/smartpqi_init.c | 2 +-
> include/linux/blk-mq-pci.h| 3 ++-
> 5 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/scsi/smartpqi/smartpqi_init.c
> b/drivers/scsi/smartpqi/smartpqi_init.c
> index b2880c7709e6..10c94011c8a8 100644
> --- a/drivers/scsi/smartpqi/smartpqi_init.c
> +++ b/drivers/scsi/smartpqi/smartpqi_init.c
> @@ -5348,7 +5348,7 @@ static int pqi_map_queues(struct Scsi_Host *shost)
> {
> struct pqi_ctrl_info *ctrl_info = shost_to_hba(shost);
>
> - return blk_mq_pci_map_queues(&shost->tag_set, ctrl_info->pci_dev);
> + return blk_mq_pci_map_queues(&shost->tag_set, ctrl_info->pci_dev, 0);
> }
>
Acked-by: Don Brace
RE: device support in hpasa, was: Re: OOPS from cciss_ioctl in 4.12+git
> -Original Message- > From: Hannes Reinecke [mailto:h...@suse.de] > Sent: Friday, July 07, 2017 11:05 AM > To: Jens Axboe ; Christoph Hellwig > ; Meelis Roos > Cc: Linux Kernel list ; linux- > bl...@vger.kernel.org; Don Brace ; Scott > Benesh ; Scott Teel > ; Kevin Barnett > ; linux-s...@vger.kernel.org > Subject: Re: device support in hpasa, was: Re: OOPS from cciss_ioctl in > 4.12+git > > EXTERNAL EMAIL > > > On 07/07/2017 05:03 PM, Jens Axboe wrote: > > On 07/07/2017 09:00 AM, Christoph Hellwig wrote: > >> On Thu, Jul 06, 2017 at 12:55:04PM +0300, Meelis Roos wrote: > >>>> Also we're trying to move people away from the cciss driver, can you > >>>> check if the hpsa SCSI driver works for you as well? > >>> > >>> I have older adapter: > >>> > >>> Compaq Computer Corporation Smart Array 64xx [0e11:0046] (rev 01) > >>> > >>> That does not seem to be supported by hpsa AFAICS. > >> > >> Looks like. Although hpsa has support for various SA5 controllers > >> it seems like it decided to skip all Compaq branded controllers. > >> > >> As far as I can tell we could simply add support for those to > >> hpsa. Ccing hpsa folks to figure out if that's the case. > > > > Pretty sure Hannes had a patch he tested for that, he talked about > > that back at LSFMM earlier this year. Hannes? > > > Oh, I do. > hpsa is working happily on SLES for _all_ SmartArray controllers. > You need to enable 'hpsa_allow_any=1', though. > But I'm perfectly happy with making that the default and drop cciss > completely. > > Cheers, > > Hannes > -- > Dr. Hannes ReineckeTeamlead Storage & Networking > h...@suse.de +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton > HRB 21284 (AG Nürnberg) The 6400 controllers are quite old and are no longer tested. So, it would be nice to deprecate the cciss driver, but we would not support those formerly cciss devices with hpsa. I do not recall seeing a cciss_ioctl issue, what was the issue? Thanks, Don Brace ESC - Smart Storage Microsemi Corporation
RE: device support in hpsa, was: Re: OOPS from cciss_ioctl in 4.12+git
So, adding adding hpsa_allow_any=1 did not work...
When you added the 0x40800e11, did you add it to both tables?
/* define the PCI info for the cards we can control */
static const struct pci_device_id hpsa_pci_device_id[] = {
{PCI_VENDOR_ID_COMPAQ, PCI_DEVICE_ID_COMPAQ_CISSB, 0x0E11, 0x4080},
...
{0,}
};
/* board_id = Subsystem Device ID & Vendor ID
* product = Marketing Name for the board
* access = Address of the struct of function pointers
*/
static struct board_type products[] = {
{0x40800E11, "Smart Array 5i", &SA5B_access},
...
};
I added it at the very first entry to make it easier.
---
However, there is not a SA5B_access table in hpsa.h.
/*
* This card is the opposite of the other cards.
* 0 turns interrupts on...
* 0x04 turns them off...
*/
static void SA5B_intr_mask(ctlr_info_t *h, unsigned long val)
{
if (val)
{ /* Turn interrupts on */
h->interrupts_enabled = 1;
writel(0, h->vaddr + SA5_REPLY_INTR_MASK_OFFSET);
(void) readl(h->vaddr + SA5_REPLY_INTR_MASK_OFFSET);
} else /* Turn them off */
{
h->interrupts_enabled = 0;
writel( SA5B_INTR_OFF,
h->vaddr + SA5_REPLY_INTR_MASK_OFFSET);
(void) readl(h->vaddr + SA5_REPLY_INTR_MASK_OFFSET);
}
}
/*
* Returns true if an interrupt is pending..
*/
static bool SA5B_intr_pending(ctlr_info_t *h)
{
unsigned long register_value =
readl(h->vaddr + SA5_INTR_STATUS);
#ifdef CCISS_DEBUG
printk("cciss: intr_pending %lx\n", register_value);
#endif /* CCISS_DEBUG */
if( register_value & SA5B_INTR_PENDING)
return 1;
return 0 ;
}
static struct access_method SA5B_access = {
.submit_command = SA5_submit_command,
.set_intr_mask = SA5B_intr_mask,
.fifo_full = SA5_fifo_full,
.intr_pending = SA5B_intr_pending,
.command_completed = SA5_completed,
};
Can you try adding the two table entries and the SA5B definitions in hpsa.h?
> -Original Message-
> From: mr...@math.ut.ee [mailto:mr...@math.ut.ee] On Behalf Of Meelis
> Roos
> Sent: Monday, July 10, 2017 9:08 AM
> To: Christoph Hellwig
> Cc: Laurence Oberman ; Jens Axboe
> ; Linux Kernel list ; linux-
> bl...@vger.kernel.org; Don Brace ; Scott
> Benesh ; Scott Teel
> ; Kevin Barnett
> ; linux-s...@vger.kernel.org; Hannes
> Reinecke
> Subject: Re: device support in hpsa, was: Re: OOPS from cciss_ioctl in
> 4.12+git
>
> EXTERNAL EMAIL
>
>
> > On Fri, Jul 07, 2017 at 11:42:38AM -0400, Laurence Oberman wrote:
> > > What happens when hpsa_allow_any=1 with the Smart Array 64xx
> > > It should probe.
> >
> > But only if it has a HP vendor ID as far as I can tell. We'd
> > still need to add the compaq ids so that these controllers get
> > probed. But maybe it's time to add them and flip the hpsa_allow_any
> > default (maybe conditionally on a config option?) and mark cciss
> > deprecated.
>
> I added hpsa_allow_any=1, did not help.
>
> Added a wildcard Compaq entry with RAID class, like the one for HP,
> still no go:
>
> [5.199125] hpsa :00:04.0: unrecognized board ID: 0x40800e11, ignoring.
> [5.282517] hpsa :00:04.0: Board ID not found
>
> Added specific PCI ID and subdevice ID quad and I still get the same
> messages and the adapter is ignored.
>
> What am I doing wrong?
>
> --
> Meelis Roos (mr...@linux.ee)
