[PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-16 Thread Hannes Reinecke 
The device handler needs to check if a given queue belongs to
a scsi device; only then does it make sense to attach a device
handler.

Signed-off-by: Hannes Reinecke 
---
 drivers/scsi/scsi_dh.c | 22 --
 drivers/scsi/scsi_lib.c| 26 ++
 include/scsi/scsi_device.h |  1 +
 3 files changed, 31 insertions(+), 18 deletions(-)

diff --git a/drivers/scsi/scsi_dh.c b/drivers/scsi/scsi_dh.c
index b8d3b97..84addee 100644
--- a/drivers/scsi/scsi_dh.c
+++ b/drivers/scsi/scsi_dh.c
@@ -219,20 +219,6 @@ int scsi_unregister_device_handler(struct 
scsi_device_handler *scsi_dh)
 }
 EXPORT_SYMBOL_GPL(scsi_unregister_device_handler);
 
-static struct scsi_device *get_sdev_from_queue(struct request_queue *q)
-{
-   struct scsi_device *sdev;
-   unsigned long flags;
-
-   spin_lock_irqsave(q->queue_lock, flags);
-   sdev = q->queuedata;
-   if (!sdev || !get_device(&sdev->sdev_gendev))
-   sdev = NULL;
-   spin_unlock_irqrestore(q->queue_lock, flags);
-
-   return sdev;
-}
-
 /*
  * scsi_dh_activate - activate the path associated with the scsi_device
  *  corresponding to the given request queue.
@@ -251,7 +237,7 @@ int scsi_dh_activate(struct request_queue *q, 
activate_complete fn, void *data)
struct scsi_device *sdev;
int err = SCSI_DH_NOSYS;
 
-   sdev = get_sdev_from_queue(q);
+   sdev = scsi_device_from_queue(q);
if (!sdev) {
if (fn)
fn(data, err);
@@ -298,7 +284,7 @@ int scsi_dh_set_params(struct request_queue *q, const char 
*params)
struct scsi_device *sdev;
int err = -SCSI_DH_NOSYS;
 
-   sdev = get_sdev_from_queue(q);
+   sdev = scsi_device_from_queue(q);
if (!sdev)
return err;
 
@@ -321,7 +307,7 @@ int scsi_dh_attach(struct request_queue *q, const char 
*name)
struct scsi_device_handler *scsi_dh;
int err = 0;
 
-   sdev = get_sdev_from_queue(q);
+   sdev = scsi_device_from_queue(q);
if (!sdev)
return -ENODEV;
 
@@ -359,7 +345,7 @@ const char *scsi_dh_attached_handler_name(struct 
request_queue *q, gfp_t gfp)
struct scsi_device *sdev;
const char *handler_name = NULL;
 
-   sdev = get_sdev_from_queue(q);
+   sdev = scsi_device_from_queue(q);
if (!sdev)
return NULL;
 
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 9d6aed5..1a81ef9 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2145,6 +2145,32 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
blk_mq_free_tag_set(&shost->tag_set);
 }
 
+/**
+ * scsi_device_from_queue - return sdev associated with a request_queue
+ * @q: The request queue to return the sdev from
+ *
+ * Return the sdev associated with a request queue or NULL if the
+ * request_queue does not reference a SCSI device.
+ */
+struct scsi_device *scsi_device_from_queue(struct request_queue *q)
+{
+   struct scsi_device *sdev = NULL;
+   unsigned long flags;
+
+   spin_lock_irqsave(q->queue_lock, flags);
+   if (q->mq_ops) {
+   if (q->mq_ops == &scsi_mq_ops)
+   sdev = q->queuedata;
+   } else if (q->request_fn == scsi_request_fn)
+   sdev = q->queuedata;
+   if (!sdev || !get_device(&sdev->sdev_gendev))
+   sdev = NULL;
+   spin_unlock_irqrestore(q->queue_lock, flags);
+
+   return sdev;
+}
+EXPORT_SYMBOL_GPL(scsi_device_from_queue);
+
 /*
  * Function:scsi_block_requests()
  *
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index 8990e58..be41c76 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -315,6 +315,7 @@ extern int scsi_add_device(struct Scsi_Host *host, uint 
channel,
 extern int scsi_unregister_device_handler(struct scsi_device_handler *scsi_dh);
 void scsi_attach_vpd(struct scsi_device *sdev);
 
+extern struct scsi_device *scsi_device_from_queue(struct request_queue *q);
 extern int scsi_device_get(struct scsi_device *);
 extern void scsi_device_put(struct scsi_device *);
 extern struct scsi_device *scsi_device_lookup(struct Scsi_Host *,
-- 
1.8.5.6

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-16 Thread Bart Van Assche 
On Thu, 2017-02-16 at 16:12 +0100, Hannes Reinecke wrote:
> +struct scsi_device *scsi_device_from_queue(struct request_queue *q)
> +{
> + struct scsi_device *sdev = NULL;
> + unsigned long flags;
> +
> + spin_lock_irqsave(q->queue_lock, flags);
> + if (q->mq_ops) {
> + if (q->mq_ops == &scsi_mq_ops)
> + sdev = q->queuedata;
> + } else if (q->request_fn == scsi_request_fn)
> + sdev = q->queuedata;
> + if (!sdev || !get_device(&sdev->sdev_gendev))
> + sdev = NULL;
> + spin_unlock_irqrestore(q->queue_lock, flags);
> +
> + return sdev;
> +}

Hello Hannes,

Do we need to take the queue lock? Neither q->mq_ops nor q->request_fn are
modified after a block device has been created. q->queuedata is not modified
by any SCSI driver after it has been set. And since the caller of
scsi_device_from_queue() has to guarantee that the queue does not disappear
while this function is in progress, the queue lock does not have to be held
around the get_device() call either. Otherwise this patch looks fine to me.

Bart.

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-16 Thread Keith Busch 
On Thu, Feb 16, 2017 at 04:12:23PM +0100, Hannes Reinecke wrote:
> The device handler needs to check if a given queue belongs to
> a scsi device; only then does it make sense to attach a device
> handler.
> 
> Signed-off-by: Hannes Reinecke 

The thing I don't like is that this still has dm-mpath directly calling
into scsi. I don't think dm-mpath has any business calling protocol
specific APIs, and doesn't help other protocols with similar needs.

Can we solve this with an indirection layer instead? 

(untested; just checking if this direction is preferable)
---
diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
index 3570bcb..28310a2 100644
--- a/drivers/md/dm-mpath.c
+++ b/drivers/md/dm-mpath.c
@@ -847,7 +847,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
 
if (test_bit(MPATHF_RETAIN_ATTACHED_HW_HANDLER, &m->flags)) {
 retain:
-   attached_handler_name = scsi_dh_attached_handler_name(q, 
GFP_KERNEL);
+   attached_handler_name = blk_dh_attached_handler_name(q, 
GFP_KERNEL);
if (attached_handler_name) {
/*
 * Clear any hw_handler_params associated with a
@@ -870,7 +870,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
}
 
if (m->hw_handler_name) {
-   r = scsi_dh_attach(q, m->hw_handler_name);
+   r = blk_dh_attach(q, m->hw_handler_name);
if (r == -EBUSY) {
char b[BDEVNAME_SIZE];
 
@@ -885,7 +885,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
}
 
if (m->hw_handler_params) {
-   r = scsi_dh_set_params(q, m->hw_handler_params);
+   r = blk_dh_set_params(q, m->hw_handler_params);
if (r < 0) {
ti->error = "unable to set hardware "
"handler parameters";
@@ -1526,7 +1526,7 @@ static void activate_path(struct work_struct *work)
struct request_queue *q = bdev_get_queue(pgpath->path.dev->bdev);
 
if (pgpath->is_active && !blk_queue_dying(q))
-   scsi_dh_activate(q, pg_init_done, pgpath);
+   blk_dh_activate(q, pg_init_done, pgpath);
else
pg_init_done(pgpath, SCSI_DH_DEV_OFFLINED);
 }
diff --git a/drivers/pci/pcie/pcie-dpc.c b/drivers/pci/pcie/pcie-dpc.c
index 06d2ed4..49a061a 100644
--- a/drivers/pci/pcie/pcie-dpc.c
+++ b/drivers/pci/pcie/pcie-dpc.c
@@ -107,7 +107,7 @@ static int dpc_probe(struct pcie_device *dev)
int status;
u16 ctl, cap;
 
-   dpc = kzalloc(&dev->device, sizeof(*dpc), GFP_KERNEL);
+   dpc = kzalloc(sizeof(*dpc), GFP_KERNEL);
if (!dpc)
return -ENOMEM;
 
@@ -116,7 +116,7 @@ static int dpc_probe(struct pcie_device *dev)
INIT_WORK(&dpc->work, interrupt_event_handler);
set_service_data(dev, dpc);
 
-   status = request_irq(&dev->device, dev->irq, dpc_irq, IRQF_SHARED,
+   status = request_irq(dev->irq, dpc_irq, IRQF_SHARED,
  "pcie-dpc", dpc);
if (status) {
dev_warn(&dev->device, "request IRQ%d failed: %d\n", dev->irq,
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index e9e1e14..e23c7d5 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2028,6 +2028,13 @@ static u64 scsi_calculate_bounce_limit(struct Scsi_Host 
*shost)
return bounce_limit;
 }
 
+static struct dh_ops scsi_dh_ops = {
+   .dh_activate = scsi_dh_activate,
+   .dh_attach = scsi_dh_attach,
+   .dh_attached_handler_name = scsi_dh_attached_handler_name,
+   .dh_set_params = scsi_dh_set_params,
+};
+
 static void __scsi_init_queue(struct Scsi_Host *shost, struct request_queue *q)
 {
struct device *dev = shost->dma_dev;
@@ -2062,6 +2069,8 @@ static void __scsi_init_queue(struct Scsi_Host *shost, 
struct request_queue *q)
 * blk_queue_update_dma_alignment() later.
 */
blk_queue_dma_alignment(q, 0x03);
+
+   q->dh_ops = &scsi_dh_ops;
 }
 
 struct request_queue *__scsi_alloc_queue(struct Scsi_Host *shost,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 1ca8e8f..5899768 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -378,6 +378,14 @@ static inline int blkdev_reset_zones_ioctl(struct 
block_device *bdev,
 
 #endif /* CONFIG_BLK_DEV_ZONED */
 
+typedef void (*activate_complete)(void *, int);
+struct dh_ops {
+   int (*dh_activate)(struct request_queue *, activate_complete, void *);
+   int (*dh_attach)(struct request_queue *, const char *);
+   const char *(*dh_attached_handler_name)(struct request_queue *, gfp_t);
+   int (*dh_set_params)(struct request_queue *, const char *);
+};
+
 struct request_queue {
/*
 * Togethe

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-16 Thread Keith Busch 
On Thu, Feb 16, 2017 at 12:05:19PM -0500, Keith Busch wrote:
> On Thu, Feb 16, 2017 at 04:12:23PM +0100, Hannes Reinecke wrote:
> > The device handler needs to check if a given queue belongs to
> > a scsi device; only then does it make sense to attach a device
> > handler.
> > 
> > Signed-off-by: Hannes Reinecke 
> 
> The thing I don't like is that this still has dm-mpath directly calling
> into scsi. I don't think dm-mpath has any business calling protocol
> specific APIs, and doesn't help other protocols with similar needs.
> 
> Can we solve this with an indirection layer instead? 
> 
> (untested; just checking if this direction is preferable)

Eh osrry, I accidently included unrelated stuff in the previous. Here's
what I meant to send:

---
diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
index 3570bcb..28310a2 100644
--- a/drivers/md/dm-mpath.c
+++ b/drivers/md/dm-mpath.c
@@ -847,7 +847,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
 
if (test_bit(MPATHF_RETAIN_ATTACHED_HW_HANDLER, &m->flags)) {
 retain:
-   attached_handler_name = scsi_dh_attached_handler_name(q, 
GFP_KERNEL);
+   attached_handler_name = blk_dh_attached_handler_name(q, 
GFP_KERNEL);
if (attached_handler_name) {
/*
 * Clear any hw_handler_params associated with a
@@ -870,7 +870,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
}
 
if (m->hw_handler_name) {
-   r = scsi_dh_attach(q, m->hw_handler_name);
+   r = blk_dh_attach(q, m->hw_handler_name);
if (r == -EBUSY) {
char b[BDEVNAME_SIZE];
 
@@ -885,7 +885,7 @@ static struct pgpath *parse_path(struct dm_arg_set *as, 
struct path_selector *ps
}
 
if (m->hw_handler_params) {
-   r = scsi_dh_set_params(q, m->hw_handler_params);
+   r = blk_dh_set_params(q, m->hw_handler_params);
if (r < 0) {
ti->error = "unable to set hardware "
"handler parameters";
@@ -1526,7 +1526,7 @@ static void activate_path(struct work_struct *work)
struct request_queue *q = bdev_get_queue(pgpath->path.dev->bdev);
 
if (pgpath->is_active && !blk_queue_dying(q))
-   scsi_dh_activate(q, pg_init_done, pgpath);
+   blk_dh_activate(q, pg_init_done, pgpath);
else
pg_init_done(pgpath, SCSI_DH_DEV_OFFLINED);
 }
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index e9e1e14..e23c7d5 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2028,6 +2028,13 @@ static u64 scsi_calculate_bounce_limit(struct Scsi_Host 
*shost)
return bounce_limit;
 }
 
+static struct dh_ops scsi_dh_ops = {
+   .dh_activate = scsi_dh_activate,
+   .dh_attach = scsi_dh_attach,
+   .dh_attached_handler_name = scsi_dh_attached_handler_name,
+   .dh_set_params = scsi_dh_set_params,
+};
+
 static void __scsi_init_queue(struct Scsi_Host *shost, struct request_queue *q)
 {
struct device *dev = shost->dma_dev;
@@ -2062,6 +2069,8 @@ static void __scsi_init_queue(struct Scsi_Host *shost, 
struct request_queue *q)
 * blk_queue_update_dma_alignment() later.
 */
blk_queue_dma_alignment(q, 0x03);
+
+   q->dh_ops = &scsi_dh_ops;
 }
 
 struct request_queue *__scsi_alloc_queue(struct Scsi_Host *shost,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 1ca8e8f..5899768 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -378,6 +378,14 @@ static inline int blkdev_reset_zones_ioctl(struct 
block_device *bdev,
 
 #endif /* CONFIG_BLK_DEV_ZONED */
 
+typedef void (*activate_complete)(void *, int);
+struct dh_ops {
+   int (*dh_activate)(struct request_queue *, activate_complete, void *);
+   int (*dh_attach)(struct request_queue *, const char *);
+   const char *(*dh_attached_handler_name)(struct request_queue *, gfp_t);
+   int (*dh_set_params)(struct request_queue *, const char *);
+};
+
 struct request_queue {
/*
 * Together with queue_head for cacheline sharing
@@ -408,6 +416,7 @@ struct request_queue {
lld_busy_fn *lld_busy_fn;
 
struct blk_mq_ops   *mq_ops;
+   struct dh_ops   *dh_ops;
 
unsigned int*mq_map;
 
@@ -572,6 +581,35 @@ struct request_queue {
boolmq_sysfs_init_done;
 };
 
+static inline int blk_dh_activate(struct request_queue *q, activate_complete 
fn, void *data)
+{
+   if (q->dh_ops && q->dh_ops->dh_activate)
+   return q->dh_ops->dh_activate(q, fn, data);
+   fn(data, 0);
+   return 0;
+}
+
+static inline int blk_dh_attach(struct request_queue *q, const char *name)
+{
+   if (q->dh

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-16 Thread Hannes Reinecke 
On 02/16/2017 05:09 PM, Bart Van Assche wrote:
> On Thu, 2017-02-16 at 16:12 +0100, Hannes Reinecke wrote:
>> +struct scsi_device *scsi_device_from_queue(struct request_queue *q)
>> +{
>> +struct scsi_device *sdev = NULL;
>> +unsigned long flags;
>> +
>> +spin_lock_irqsave(q->queue_lock, flags);
>> +if (q->mq_ops) {
>> +if (q->mq_ops == &scsi_mq_ops)
>> +sdev = q->queuedata;
>> +} else if (q->request_fn == scsi_request_fn)
>> +sdev = q->queuedata;
>> +if (!sdev || !get_device(&sdev->sdev_gendev))
>> +sdev = NULL;
>> +spin_unlock_irqrestore(q->queue_lock, flags);
>> +
>> +return sdev;
>> +}
> 
> Hello Hannes,
> 
> Do we need to take the queue lock? Neither q->mq_ops nor q->request_fn are
> modified after a block device has been created. q->queuedata is not modified
> by any SCSI driver after it has been set. And since the caller of
> scsi_device_from_queue() has to guarantee that the queue does not disappear
> while this function is in progress, the queue lock does not have to be held
> around the get_device() call either. Otherwise this patch looks fine to me.
> 
Well, we did for the original code path, so I assumed that it'd be
prudent to do so.
And it's hardly time-critical anyway.
But yes, I guess we could do without the lock.

Cheers,

Hannes
-- 
Dr. Hannes ReineckeTeamlead Storage & Networking
h...@suse.de   +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-17 Thread Hannes Reinecke 
On 02/16/2017 06:05 PM, Keith Busch wrote:
> On Thu, Feb 16, 2017 at 04:12:23PM +0100, Hannes Reinecke wrote:
>> The device handler needs to check if a given queue belongs to
>> a scsi device; only then does it make sense to attach a device
>> handler.
>>
>> Signed-off-by: Hannes Reinecke 
> 
> The thing I don't like is that this still has dm-mpath directly calling
> into scsi. I don't think dm-mpath has any business calling protocol
> specific APIs, and doesn't help other protocols with similar needs.
> 
> Can we solve this with an indirection layer instead? 
> 
> (untested; just checking if this direction is preferable)

We could, but why?
ATM we're only having SCSI devices able to use device handler; adding
another layer of indirection doesn't solve anything here.
Moving the infrastructure one level up will only make sense if we're
getting non-SCSI device handler (ANA?), but until then I'd think it's
just overengineering.

Cheers,

Hannes
-- 
Dr. Hannes ReineckeTeamlead Storage & Networking
h...@suse.de   +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-17 Thread Christoph Hellwig 
On Fri, Feb 17, 2017 at 09:06:14AM +0100, Hannes Reinecke wrote:
> We could, but why?
> ATM we're only having SCSI devices able to use device handler; adding
> another layer of indirection doesn't solve anything here.
> Moving the infrastructure one level up will only make sense if we're
> getting non-SCSI device handler (ANA?), but until then I'd think it's
> just overengineering.

Agreed.  Independent of what does the balancing between queues hardware
handler should be attached by the low-level driver for any future
transport without any control from DM.

Re: [PATCHv2] scsi: use 'scsi_device_from_queue()' for scsi_dh

2017-02-18 Thread Mike Snitzer 
On Fri, Feb 17, 2017 at 3:27 AM, Christoph Hellwig  wrote:
>
> On Fri, Feb 17, 2017 at 09:06:14AM +0100, Hannes Reinecke wrote:
> > We could, but why?
> > ATM we're only having SCSI devices able to use device handler; adding
> > another layer of indirection doesn't solve anything here.
> > Moving the infrastructure one level up will only make sense if we're
> > getting non-SCSI device handler (ANA?), but until then I'd think it's
> > just overengineering.
>
> Agreed.  Independent of what does the balancing between queues hardware
> handler should be attached by the low-level driver for any future
> transport without any control from DM.

But doesn't Keith's abstraction makes a lot of sense given that you're
providing a device handler interface for NVMe?

The most important part of scsi_dh that DM uses is its calls to
scsi_dh_activate.  Attachment isn't interesting or the issue (DM's
call to scsi_dh_attach is purely legacy now that SCSI attaches the
proper scsi_dh during SCSI's device scan).