Re: [PATCH v2] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-05 Thread Michal Hocko
On Thu 06-04-17 12:23:51, NeilBrown wrote: [...] > diff --git a/drivers/block/loop.c b/drivers/block/loop.c > index 0ecb6461ed81..95679d988725 100644 > --- a/drivers/block/loop.c > +++ b/drivers/block/loop.c > @@ -847,10 +847,12 @@ static void loop_unprepare_queue(struct loop_device *lo) > static

Re: [PATCH 4/4] mtd: nand: nandsim: convert to memalloc_noreclaim_*()

2017-04-05 Thread Adrian Hunter
On 05/04/17 14:39, Vlastimil Babka wrote: > On 04/05/2017 01:36 PM, Richard Weinberger wrote: >> Michal, >> >> Am 05.04.2017 um 13:31 schrieb Michal Hocko: >>> On Wed 05-04-17 09:47:00, Vlastimil Babka wrote: Nandsim has own functions set_memalloc() and clear_memalloc() for robust setting

Re: [Nbd] [PATCH 3/4] treewide: convert PF_MEMALLOC manipulations to new helpers

2017-04-05 Thread Wouter Verhelst
On Wed, Apr 05, 2017 at 01:30:31PM +0200, Michal Hocko wrote: > On Wed 05-04-17 09:46:59, Vlastimil Babka wrote: > > We now have memalloc_noreclaim_{save,restore} helpers for robust setting and > > clearing of PF_MEMALLOC. Let's convert the code which was using the generic > > tsk_restore_flags().

Re: [PATCH 1/2] block: Implement global tagset

2017-04-05 Thread Arun Easi
Hi Hannes, Thanks for taking a crack at the issue. My comments below.. On Tue, 4 Apr 2017, 5:07am, Hannes Reinecke wrote: > Most legacy HBAs have a tagset per HBA, not per queue. To map > these devices onto block-mq this patch implements a new tagset > flag BLK_MQ_F_GLOBAL_TAGS, which will cause

Re: [PATCH 27/27] scsi: sd: Remove LBPRZ dependency for discards

2017-04-05 Thread Hannes Reinecke
On 04/05/2017 07:21 PM, Christoph Hellwig wrote: > From: "Martin K. Petersen" > > Separating discards and zeroout operations allows us to remove the LBPRZ > block zeroing constraints from discards and honor the device preferences > for UNMAP commands. > > If supported by the device, we'll also c

Re: [PATCH 26/27] scsi: sd: Separate zeroout and discard command choices

2017-04-05 Thread Hannes Reinecke
On 04/05/2017 07:21 PM, Christoph Hellwig wrote: > From: "Martin K. Petersen" > > Now that zeroout and discards are distinct operations we need to > separate the policy of choosing the appropriate command. Create a > zeroing_mode which can be one of: > > write:Zeroout ass

RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread Long Li
> -Original Message- > From: KY Srinivasan > Sent: Wednesday, April 5, 2017 9:21 PM > To: Bart Van Assche ; linux- > ker...@vger.kernel.org; linux-block@vger.kernel.org; Long Li > ; ax...@kernel.dk > Cc: Stephen Hemminger > Subject: RE: [PATCH] block-mq: set both block queue and hardware

Re: [PATCH v3 1/8] blk-mq: use the right hctx when getting a driver tag fails

2017-04-05 Thread Ming Lei
On Wed, Apr 05, 2017 at 12:01:29PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > While dispatching requests, if we fail to get a driver tag, we mark the > hardware queue as waiting for a tag and put the requests on a > hctx->dispatch list to be run later when a driver tag is freed. Howeve

RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread KY Srinivasan
> -Original Message- > From: Bart Van Assche [mailto:bart.vanass...@sandisk.com] > Sent: Wednesday, April 5, 2017 8:46 PM > To: linux-ker...@vger.kernel.org; linux-block@vger.kernel.org; Long Li > ; ax...@kernel.dk > Cc: Stephen Hemminger ; KY Srinivasan > > Subject: Re: [PATCH] block-mq

Re: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread Bart Van Assche
On Thu, 2017-04-06 at 03:38 +, Long Li wrote: > > -Original Message- > > From: Bart Van Assche [mailto:bart.vanass...@sandisk.com] > > > > Please drop this patch. I'm working on a better solution. > > Thank you. Looking forward to your patch. Hello Long, It would help if you could s

RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread Long Li
> -Original Message- > From: Bart Van Assche [mailto:bart.vanass...@sandisk.com] > Sent: Wednesday, April 5, 2017 5:32 PM > To: linux-ker...@vger.kernel.org; linux-block@vger.kernel.org; Long Li > ; ax...@kernel.dk > Cc: Stephen Hemminger ; KY Srinivasan > ; Long Li > Subject: Re: [PATCH

Re: [PATCH v2] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-05 Thread NeilBrown
On Wed, Apr 05 2017, Michal Hocko wrote: > On Wed 05-04-17 09:19:27, Michal Hocko wrote: >> On Wed 05-04-17 14:33:50, NeilBrown wrote: > [...] >> > diff --git a/drivers/block/loop.c b/drivers/block/loop.c >> > index 0ecb6461ed81..44b3506fd086 100644 >> > --- a/drivers/block/loop.c >> > +++ b/drive

Re: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread Bart Van Assche
On Wed, 2017-04-05 at 17:16 -0700, Long Li wrote: > Under heavy I/O, one hardware queue may be unable to dispatch any I/O to the > device layer. This poses a problem with restarting this hardware queue on I/O > finish in blk_mq_sched_restart_queues(), becaue there is nothing pending that > will fi

[PATCH] block-mq: set both block queue and hardware queue restart bit for restart

2017-04-05 Thread Long Li
From: Long Li Under heavy I/O, one hardware queue may be unable to dispatch any I/O to the device layer. This poses a problem with restarting this hardware queue on I/O finish in blk_mq_sched_restart_queues(), becaue there is nothing pending that will finish in future on this hardware qeueu. Thi

Re: [PATCH v2 3/5] blk-mq: Introduce blk_mq_ops.restart_hctx

2017-04-05 Thread Omar Sandoval
On Wed, Apr 05, 2017 at 08:51:51PM +, Bart Van Assche wrote: > On Wed, 2017-04-05 at 13:41 -0700, Omar Sandoval wrote: > > On Mon, Apr 03, 2017 at 04:22:26PM -0700, Bart Van Assche wrote: > > > If a tag set is shared among multiple hardware queues, leave > > > it to the block driver to rerun ha

Re: [PATCH v2 3/5] blk-mq: Introduce blk_mq_ops.restart_hctx

2017-04-05 Thread Bart Van Assche
On Wed, 2017-04-05 at 13:41 -0700, Omar Sandoval wrote: > On Mon, Apr 03, 2017 at 04:22:26PM -0700, Bart Van Assche wrote: > > If a tag set is shared among multiple hardware queues, leave > > it to the block driver to rerun hardware queues. Hence remove > > QUEUE_FLAG_RESTART and introduce blk_mq_o

Re: [PATCH v2 3/5] blk-mq: Introduce blk_mq_ops.restart_hctx

2017-04-05 Thread Omar Sandoval
On Mon, Apr 03, 2017 at 04:22:26PM -0700, Bart Van Assche wrote: > If a tag set is shared among multiple hardware queues, leave > it to the block driver to rerun hardware queues. Hence remove > QUEUE_FLAG_RESTART and introduce blk_mq_ops.restart_hctx. > Remove blk_mq_sched_mark_restart_queue() beca

[PATCH v3 7/8] blk-mq: make driver tag failure path easier to follow

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Minor cleanup that makes it easier to figure out what's going on in the driver tag allocation failure path of blk_mq_dispatch_rq_list(). Signed-off-by: Omar Sandoval --- block/blk-mq.c | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/

[PATCH v3 8/8] blk-mq: use true instead of 1 for blk_mq_queue_data.last

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Trivial cleanup. Signed-off-by: Omar Sandoval --- block/blk-mq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index de57e7727c52..c82fcb923ff8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1435,7 +1435,7 @@ stati

[PATCH v3 6/8] blk-mq-sched: provide hooks for initializing hardware queue data

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Schedulers need to be informed when a hardware queue is added or removed at runtime so they can allocate/free per-hardware queue data. So, replace the blk_mq_sched_init_hctx_data() helper, which only makes sense at init time, with .init_hctx() and .exit_hctx() hooks. Signed-o

[PATCH v3 2/8] blk-mq-sched: refactor scheduler initialization

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Preparation cleanup for the next couple of fixes, push blk_mq_sched_setup() and e->ops.mq.init_sched() into a helper. Signed-off-by: Omar Sandoval --- block/blk-mq-sched.c | 82 block/blk-mq-sched.h | 2 +- block/elevato

[PATCH v3 4/8] blk-mq-sched: fix crash in switch error path

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval In elevator_switch(), if blk_mq_init_sched() fails, we attempt to fall back to the original scheduler. However, at this point, we've already torn down the original scheduler's tags, so this causes a crash. Doing the fallback like the legacy elevator path is much harder for mq,

[PATCH v3 5/8] blk-mq: remap queues when adding/removing hardware queues

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval blk_mq_update_nr_hw_queues() used to remap hardware queues, which is the behavior that drivers expect. However, commit 4e68a011428a changed blk_mq_queue_reinit() to not remap queues for the case of CPU hotplugging, inadvertently making blk_mq_update_nr_hw_queues() not remap qu

[PATCH v3 1/8] blk-mq: use the right hctx when getting a driver tag fails

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval While dispatching requests, if we fail to get a driver tag, we mark the hardware queue as waiting for a tag and put the requests on a hctx->dispatch list to be run later when a driver tag is freed. However, blk_mq_dispatch_rq_list() may dispatch requests from multiple hardware

[PATCH v3 3/8] blk-mq-sched: set up scheduler tags when bringing up new queues

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval If a new hardware queue is added at runtime, we don't allocate scheduler tags for it, leading to a crash. This hooks up the scheduler framework to blk_mq_{init,exit}_hctx() to make sure everything gets properly initialized/freed. Signed-off-by: Omar Sandoval --- block/blk-m

[PATCH v3 0/8] blk-mq: various fixes and cleanups

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Rebase of v2 [1] onto block-next. As with v2, - Patch 1 is the new fix for a hang that Josef reported after trying v1. - Patches 2-6 are the original series. Patch 5 now has Christoph's and Sagi's Reviewed-by. - Patches 7 and 8 are trivial cleanups. Patches 1-5 should prob

Re: [PATCH v2 1/8] blk-mq: use the right hctx when getting a driver tag fails

2017-04-05 Thread Omar Sandoval
On Wed, Apr 05, 2017 at 06:33:14PM +, Bart Van Assche wrote: > On Wed, 2017-04-05 at 11:28 -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > While dispatching requests, if we fail to get a driver tag, we mark the > > hardware queue as waiting for a tag and put the requests on a > >

Re: [PATCH v2 8/8] blk-mq: clean up direct issue blk_mq_queue_data initialization

2017-04-05 Thread Omar Sandoval
On Wed, Apr 05, 2017 at 06:35:58PM +, Bart Van Assche wrote: > On Wed, 2017-04-05 at 11:28 -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Trivial cleanup. > > > > Signed-off-by: Omar Sandoval > > --- > > block/blk-mq.c | 3 +-- > > 1 file changed, 1 insertion(+), 2 deletions(-

Re: [PATCH v2 8/8] blk-mq: clean up direct issue blk_mq_queue_data initialization

2017-04-05 Thread Bart Van Assche
On Wed, 2017-04-05 at 11:28 -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Trivial cleanup. > > Signed-off-by: Omar Sandoval > --- > block/blk-mq.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/block/blk-mq.c b/block/blk-mq.c > index 71dc8608f3a8..779249a59

Re: [PATCH v2 1/8] blk-mq: use the right hctx when getting a driver tag fails

2017-04-05 Thread Bart Van Assche
On Wed, 2017-04-05 at 11:28 -0700, Omar Sandoval wrote: > From: Omar Sandoval > > While dispatching requests, if we fail to get a driver tag, we mark the > hardware queue as waiting for a tag and put the requests on a > hctx->dispatch list to be run later when a driver tag is freed. However, > bl

[PATCH v2 6/8] blk-mq-sched: provide hooks for initializing hardware queue data

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Schedulers need to be informed when a hardware queue is added or removed at runtime so they can allocate/free per-hardware queue data. So, replace the blk_mq_sched_init_hctx_data() helper, which only makes sense at init time, with .init_hctx() and .exit_hctx() hooks. Signed-o

[PATCH v2 2/8] blk-mq-sched: refactor scheduler initialization

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Preparation cleanup for the next couple of fixes, push blk_mq_sched_setup() and e->ops.mq.init_sched() into a helper. Signed-off-by: Omar Sandoval --- block/blk-mq-sched.c | 82 block/blk-mq-sched.h | 2 +- block/elevato

[PATCH v2 5/8] blk-mq: remap queues when adding/removing hardware queues

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval blk_mq_update_nr_hw_queues() used to remap hardware queues, which is the behavior that drivers expect. However, commit 4e68a011428a changed blk_mq_queue_reinit() to not remap queues for the case of CPU hotplugging, inadvertently making blk_mq_update_nr_hw_queues() not remap qu

[PATCH v2 7/8] blk-mq: make driver tag failure path easier to follow

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Minor cleanup that makes it easier to figure out what's going on in the driver tag allocation failure path of blk_mq_dispatch_rq_list(). Signed-off-by: Omar Sandoval --- block/blk-mq.c | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/

[PATCH v2 1/8] blk-mq: use the right hctx when getting a driver tag fails

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval While dispatching requests, if we fail to get a driver tag, we mark the hardware queue as waiting for a tag and put the requests on a hctx->dispatch list to be run later when a driver tag is freed. However, blk_mq_dispatch_rq_list() may dispatch requests from multiple hardware

[PATCH v2 0/8] blk-mq: various fixes and cleanups

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval This v2 of my series from a couple of days ago [1] with one extra fix and two extra cleanups. - Patch 1 is the new fix for a hang that Josef reported after trying v1. - Patches 2-6 are the original series. Patch 5 now has Christoph's and Sagi's Reviewed-by. - Patches 7 and

[PATCH v2 4/8] blk-mq-sched: fix crash in switch error path

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval In elevator_switch(), if blk_mq_init_sched() fails, we attempt to fall back to the original scheduler. However, at this point, we've already torn down the original scheduler's tags, so this causes a crash. Doing the fallback like the legacy elevator path is much harder for mq,

[PATCH v2 8/8] blk-mq: clean up direct issue blk_mq_queue_data initialization

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval Trivial cleanup. Signed-off-by: Omar Sandoval --- block/blk-mq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 71dc8608f3a8..779249a5999b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1451,8 +1451,7 @@ sta

[PATCH v2 3/8] blk-mq-sched: set up scheduler tags when bringing up new queues

2017-04-05 Thread Omar Sandoval
From: Omar Sandoval If a new hardware queue is added at runtime, we don't allocate scheduler tags for it, leading to a crash. This hooks up the scheduler framework to blk_mq_{init,exit}_hctx() to make sure everything gets properly initialized/freed. Signed-off-by: Omar Sandoval --- block/blk-m

Re: ->retries fixups V2

2017-04-05 Thread Christoph Hellwig
On Wed, Apr 05, 2017 at 12:06:53PM -0600, Jens Axboe wrote: > On Wed, Apr 05 2017, Christoph Hellwig wrote: > > This series fixes a few lose bits in terms of how nvme uses ->retries, > > including fixing it for non-PCIe transports. While at it I noticed that > > nvme and scsi use the field in enti

Re: ->retries fixups V2

2017-04-05 Thread Jens Axboe
On 04/05/2017 12:16 PM, Christoph Hellwig wrote: > On Wed, Apr 05, 2017 at 12:06:53PM -0600, Jens Axboe wrote: >> On Wed, Apr 05 2017, Christoph Hellwig wrote: >>> This series fixes a few lose bits in terms of how nvme uses ->retries, >>> including fixing it for non-PCIe transports. While at it I

Re: [PATCH v2 2/2] blk-mq: Add a polling specific stats function

2017-04-05 Thread Jens Axboe
On 04/05/2017 11:39 AM, sba...@raithlin.com wrote: > @@ -42,6 +42,33 @@ static LIST_HEAD(all_q_list); > static void blk_mq_poll_stats_start(struct request_queue *q); > static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); > > +static int blk_mq_poll_stats_bkt(const struct request *rq)

Re: ->retries fixups V2

2017-04-05 Thread Jens Axboe
On Wed, Apr 05 2017, Christoph Hellwig wrote: > This series fixes a few lose bits in terms of how nvme uses ->retries, > including fixing it for non-PCIe transports. While at it I noticed that > nvme and scsi use the field in entirely different ways, and no other > driver uses it at all. So I dec

[PATCH v2 2/2] blk-mq: Add a polling specific stats function

2017-04-05 Thread sbates
From: Stephen Bates Rather than bucketing IO statisics based on direction only we also bucket based on the IO size. This leads to improved polling performance. Update the bucket callback function and use it in the polling latency estimation. Signed-off-by: Stephen Bates --- block/blk-mq.c | 53

[PATCH v2 0/2] blk-stat: Add ability to not bucket IO; improve IO polling.

2017-04-05 Thread sbates
From: Stephen Bates Omar recently developed some patches for block layer stats that use callbacks to determine which bucket an IO should be considered for. At the same time there was discussion at LSF/MM that we might not want to consider all IO when generating stats for certain algorithms (e.g.

[PATCH v2 1/2] blk-stat: convert blk-stat bucket callback to signed

2017-04-05 Thread sbates
From: Stephen Bates In order to allow for filtering of IO based on some other properties of the request than direction we allow the bucket function to return an int. If the bucket callback returns a negative do no count it in the stats accumulation. Signed-off-by: Stephen Bates --- block/blk-

Re: [PATCH v2 4/5] scsi: Add scsi_restart_hctx()

2017-04-05 Thread Bart Van Assche
On 04/04/2017 11:20 PM, Christoph Hellwig wrote: > On Tue, Apr 04, 2017 at 03:56:34PM +, Bart Van Assche wrote: >>> This looks like generic block layer code, why is it in SCSI? >> >> That's an excellent question. I assume that you are fine with moving >> this code to the block layer? > > Yes.

Re: [PATCH 2/5] nvme: cleanup nvme_req_needs_retry

2017-04-05 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 3/5] nvme: mark nvme_max_retries static

2017-04-05 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 4/5] nvme: move the retries count to struct nvme_request

2017-04-05 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 1/5] nvme: move ->retries setup to nvme_setup_cmd

2017-04-05 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 5/5] block, scsi: move the retries field to struct scsi_request

2017-04-05 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

[PATCH 06/27] dm io: discards don't take a payload

2017-04-05 Thread Christoph Hellwig
Fix up do_region to not allocate a bio_vec for discards. We've got rid of the discard payload allocated by the caller years ago. Obviously this wasn't actually harmful given how long it's been there, but it's still good to avoid the pointless allocation. Signed-off-by: Christoph Hellwig Reviewe

[PATCH 05/27] md: support REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
Copy & paste from the REQ_OP_WRITE_SAME code. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/md/linear.c| 1 + drivers/md/md.h| 7 +++ drivers/md/multipath.c | 1 + drivers/md/raid0.c | 2 ++ drivers/md/raid1.c | 4 +++- drivers/md/raid10.c

[PATCH 04/27] sd: implement REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- drivers/scsi/sd.c | 31 ++- drivers/scsi/sd_zbc.c | 1 + 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index

[PATCH 01/27] sd: split sd_setup_discard_cmnd

2017-04-05 Thread Christoph Hellwig
Split sd_setup_discard_cmnd into one function per provisioning type. While this creates some very slight duplication of boilerplate code it keeps the code modular for additions of new provisioning types, and for reusing the write same functions for the upcoming scsi implementation of the Write Zer

always use REQ_OP_WRITE_ZEROES for zeroing offload V2

2017-04-05 Thread Christoph Hellwig
This series makes REQ_OP_WRITE_ZEROES the only zeroing offload supported by the block layer, and switches existing implementations of REQ_OP_DISCARD that correctly set discard_zeroes_data to it, removes incorrect discard_zeroes_data, and also switches WRITE SAME based zeroing in SCSI to this new me

[PATCH 03/27] block: implement splitting of REQ_OP_WRITE_ZEROES bios

2017-04-05 Thread Christoph Hellwig
Copy and past the REQ_OP_WRITE_SAME code to prepare to implementations that limit the write zeroes size. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- block/blk-merge.c | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-)

[PATCH 02/27] block: renumber REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
Make life easy for implementations that needs to send a data buffer to the device (e.g. SCSI) by numbering it as a data out command. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- include/linux/blk_types.h | 2 +- 1 file changed, 1 insertion(+

[PATCH 10/27] block: add a flags argument to (__)blkdev_issue_zeroout

2017-04-05 Thread Christoph Hellwig
Turn the existing discard flag into a new BLKDEV_ZERO_UNMAP flag with similar semantics, but without referring to diѕcard. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- block/blk-lib.c| 31 ++-

[PATCH 24/27] drbd: implement REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
It seems like DRBD assumes its on the wire TRIM request always zeroes data. Use that fact to implement REQ_OP_WRITE_ZEROES. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/drbd/drbd_main.c | 3 ++- drivers/block/drbd/drbd_nl.c | 2 ++ drivers/block/drbd

[PATCH 07/27] dm: support REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
Copy & paste from the REQ_OP_WRITE_SAME code. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/md/dm-core.h | 1 + drivers/md/dm-io.c| 8 ++-- drivers/md/dm-linear.c| 1 + drivers/md/dm-mpath.c | 1 + drivers/md/dm-rq.c

[PATCH 19/27] rbd: remove the discard_zeroes_data flag

2017-04-05 Thread Christoph Hellwig
rbd only supports discarding on large alignments, so the zeroing code would always fall back to explicit writings of zeroes. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/rbd.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/blo

[PATCH 18/27] brd: remove discard support

2017-04-05 Thread Christoph Hellwig
It's just a in-driver reimplementation of writing zeroes to the pages, which fails if the discards aren't page aligned. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/brd.c | 54 - 1 file changed, 54 deletions(

[PATCH 17/27] loop: implement REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
It's identical to discard as hole punches will always leave us with zeroes on reads. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/loop.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index cc981f34e017..3bb

[PATCH 14/27] sd: implement unmapping Write Zeroes

2017-04-05 Thread Christoph Hellwig
Try to use a write same with unmap bit variant if the device supports it and the caller allows for it. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- drivers/scsi/sd.c | 9 + 1 file changed, 9 insertions(+) diff --git a/drivers/scsi/s

[PATCH 23/27] drbd: make intelligent use of blkdev_issue_zeroout

2017-04-05 Thread Christoph Hellwig
drbd always wants its discard wire operations to zero the blocks, so use blkdev_issue_zeroout with the BLKDEV_ZERO_UNMAP flag instead of reinventing it poorly. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/drbd/drbd_debugfs.c | 3 -- drivers/block/drbd/drbd_

[PATCH 15/27] nvme: implement REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
But now for the real NVMe Write Zeroes yet, just to get rid of the discard abuse for zeroing. Also rename the quirk flag to be a bit more self-explanatory. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- drivers/nvme/host/core.c | 10 +

[PATCH 16/27] zram: implement REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
Just the same as discard if the block size equals the system page size. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/zram/zram_drv.c | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block

[PATCH 09/27] block: stop using blkdev_issue_write_same for zeroing

2017-04-05 Thread Christoph Hellwig
We'll always use the WRITE ZEROES code for zeroing now. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- block/blk-lib.c | 4 1 file changed, 4 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index e5b853f2b8a2..2a8d638544a7 10

[PATCH 20/27] rsxx: remove the discard_zeroes_data flag

2017-04-05 Thread Christoph Hellwig
rsxx only supports discarding on large alignments, so the zeroing code would always fall back to explicit writings of zeroes. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/block/rsxx/dev.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/block/rsxx/dev.c b/

[PATCH 11/27] block: add a REQ_NOUNMAP flag for REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
If this flag is set logical provisioning capable device should release space for the zeroed blocks if possible, if it is not set devices should keep the blocks anchored. Also remove an out of sync kerneldoc comment for a static function that would have become even more out of data with this change

[PATCH 21/27] mmc: remove the discard_zeroes_data flag

2017-04-05 Thread Christoph Hellwig
mmc only supports discarding on large alignments, so the zeroing code would always fall back to explicit writings of zeroes. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/mmc/core/queue.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/mmc/core/queue.c b

[PATCH 13/27] block_dev: use blkdev_issue_zerout for hole punches

2017-04-05 Thread Christoph Hellwig
This gets us support for non-discard efficient write of zeroes (e.g. NVMe) and prepares for removing the discard_zeroes_data flag. Also remove a pointless discard support check, which is done in blkdev_issue_discard already. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Revi

[PATCH 08/27] dm kcopyd: switch to use REQ_OP_WRITE_ZEROES

2017-04-05 Thread Christoph Hellwig
It seems like the code currently passes whatever it was using for writes to WRITE SAME. Just switch it to WRITE ZEROES, although that doesn't need any payload. Untested, and confused by the code, maybe someone who understands it better than me can help.. Signed-off-by: Christoph Hellwig Reviewe

[PATCH 12/27] block: add a new BLKDEV_ZERO_NOFALLBACK flag

2017-04-05 Thread Christoph Hellwig
This avoids fallbacks to explicit zeroing in (__)blkdev_issue_zeroout if the caller doesn't want them. Also clean up the convoluted check for the return condition that this new flag is added to. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke ---

[PATCH 22/27] block: stop using discards for zeroing

2017-04-05 Thread Christoph Hellwig
Now that we have REQ_OP_WRITE_ZEROES implemented for all devices that support efficient zeroing, we can remove the call to blkdev_issue_discard. This means we only have two ways of zeroing left and can simplify the code. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-b

[PATCH 27/27] scsi: sd: Remove LBPRZ dependency for discards

2017-04-05 Thread Christoph Hellwig
From: "Martin K. Petersen" Separating discards and zeroout operations allows us to remove the LBPRZ block zeroing constraints from discards and honor the device preferences for UNMAP commands. If supported by the device, we'll also choose UNMAP over one of the WRITE SAME variants for discards.

[PATCH 25/27] block: remove the discard_zeroes_data flag

2017-04-05 Thread Christoph Hellwig
Now that we use the proper REQ_OP_WRITE_ZEROES operation everywhere we can kill this hack. Signed-off-by: Christoph Hellwig Reviewed-by: Martin K. Petersen Reviewed-by: Hannes Reinecke --- Documentation/ABI/testing/sysfs-block | 10 ++- Documentation/block/queue-sysfs.txt | 5 bloc

[PATCH 26/27] scsi: sd: Separate zeroout and discard command choices

2017-04-05 Thread Christoph Hellwig
From: "Martin K. Petersen" Now that zeroout and discards are distinct operations we need to separate the policy of choosing the appropriate command. Create a zeroing_mode which can be one of: write: Zeroout assist not present, use regular WRITE writesame: Allow WRIT

[PATCH 2/5] nvme: cleanup nvme_req_needs_retry

2017-04-05 Thread Christoph Hellwig
Don't pass the status explicitly but derive it from the requeust, and unwind the complex condition to be more readable. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- drivers/nvme/host/core.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --g

[PATCH 3/5] nvme: mark nvme_max_retries static

2017-04-05 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- drivers/nvme/host/core.c | 3 +-- drivers/nvme/host/nvme.h | 2 -- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index b225aacf4b89..933e67c60e33 100644 --- a/

[PATCH 4/5] nvme: move the retries count to struct nvme_request

2017-04-05 Thread Christoph Hellwig
The way NVMe uses this field is entirely different from the older SCSI/BLOCK_PC usage, so move it into struct nvme_request. Also reduce the size of the file to a unsigned char so that we leave space for additional smaller fields that will appear soon. Signed-off-by: Christoph Hellwig Reviewed-by

[PATCH 5/5] block, scsi: move the retries field to struct scsi_request

2017-04-05 Thread Christoph Hellwig
Instead of bloating the generic struct request with it. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/scsi_ioctl.c | 8 drivers/scsi/osd/osd_initiator.c | 2 +- drivers/scsi/osst.c| 2 +- drivers/scsi/scsi_error.c

->retries fixups V2

2017-04-05 Thread Christoph Hellwig
This series fixes a few lose bits in terms of how nvme uses ->retries, including fixing it for non-PCIe transports. While at it I noticed that nvme and scsi use the field in entirely different ways, and no other driver uses it at all. So I decided to move it into the nvme_request and scsi_request

[PATCH 1/5] nvme: move ->retries setup to nvme_setup_cmd

2017-04-05 Thread Christoph Hellwig
->retries is counting the number of times a command is resubmitted, and be cleared on the first time we see the command. We currently don't do that for non-PCIe command, which is easily fixed by moving the setup to common code. Signed-off-by: Christoph Hellwig --- drivers/nvme/host/core.c | 5 +

Re: [PATCH] remove the obsolete hd driver

2017-04-05 Thread Jens Axboe
On 04/05/2017 07:59 AM, Christoph Hellwig wrote: > This driver is for pre-IDE hardisk that are only found in PC from the > stoneage of personal computing, and which we don't support elsewhere > in the kernel these days. > > It's also been marked broken forever. Applied for 4.12, who'd trust their

Re: [PATCH] remove the obsolete hd driver

2017-04-05 Thread Martin K. Petersen
Christoph Hellwig writes: > This driver is for pre-IDE hardisk that are only found in PC from the > stoneage of personal computing, and which we don't support elsewhere > in the kernel these days. > > It's also been marked broken forever. Reviewed-by: Martin K. Petersen -- Martin K. Petersen

[PATCH, RESEND] blk-mq: Remove blk_mq_queue_data.list

2017-04-05 Thread Bart Van Assche
The block layer core sets blk_mq_queue_data.list but no block drivers read that member. Hence remove it and also the code that is used to set this member. Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig Reviewed-by: Sagi Grimberg --- block/blk-mq.c | 17 -

Re: [PATCH, RESEND] blk-mq: Remove blk_mq_queue_data.list

2017-04-05 Thread Jens Axboe
On 04/05/2017 09:39 AM, Bart Van Assche wrote: > The block layer core sets blk_mq_queue_data.list but no block > drivers read that member. Hence remove it and also the code that > is used to set this member. Thanks, this came through fine. Applied for 4.12. -- Jens Axboe

Re: [PATCH] blk-mq: Remove blk_mq_queue_data.list

2017-04-05 Thread Bart Van Assche
On Wed, 2017-04-05 at 08:07 -0600, Jens Axboe wrote: > On Mon, Apr 03 2017, Bart Van Assche wrote: > > The block layer core sets blk_mq_queue_data.list but no block > > drivers read that member. Hence remove it and also the code that > > is used to set this member. > > Looks fine to me, might as w

Re: [PATCH 4/5] nvme: move the retries count to struct nvme_request

2017-04-05 Thread Christoph Hellwig
On Wed, Apr 05, 2017 at 11:14:30AM -0400, Keith Busch wrote: > On Wed, Apr 05, 2017 at 04:18:55PM +0200, Christoph Hellwig wrote: > > The way NVMe uses this field is entirely different from the older > > SCSI/BLOCK_PC usage, so move it into struct nvme_request. > > > > Also reduce the size of the

Re: [PATCH 4/5] nvme: move the retries count to struct nvme_request

2017-04-05 Thread Keith Busch
On Wed, Apr 05, 2017 at 04:18:55PM +0200, Christoph Hellwig wrote: > The way NVMe uses this field is entirely different from the older > SCSI/BLOCK_PC usage, so move it into struct nvme_request. > > Also reduce the size of the file to a unsigned char so that we leave space > for additional smaller

Re: [PATCH 1/5] nvme: move ->retries setup to nvme_setup_cmd

2017-04-05 Thread Christoph Hellwig
On Wed, Apr 05, 2017 at 04:43:34PM +0200, Johannes Thumshirn wrote: > On Wed, Apr 05, 2017 at 04:18:52PM +0200, Christoph Hellwig wrote: > > This way we get the behavior right for the non-PCIe transports. > > Could you please share a bit of your minds inner workings for us mere mortals? It's init

Re: always use REQ_OP_WRITE_ZEROES for zeroing offload V2

2017-04-05 Thread Christoph Hellwig
Looks like the mail server crapped out under the load.. I'll resend tomorrow morning, in the meantime the git url below works. On Wed, Apr 05, 2017 at 04:21:38PM +0200, Christoph Hellwig wrote: > This series makes REQ_OP_WRITE_ZEROES the only zeroing offload > supported by the block layer, and sw

Re: [PATCH 5/5] block, scsi: move the retries field to struct scsi_request

2017-04-05 Thread Johannes Thumshirn
On Wed, Apr 05, 2017 at 04:18:56PM +0200, Christoph Hellwig wrote: > Instead of bloating the generic struct request with it. > > Signed-off-by: Christoph Hellwig > --- Reviewed-by: Johannes Thumshirn -- Johannes Thumshirn Storage jthumsh...@suse.de

Re: [PATCH 4/5] nvme: move the retries count to struct nvme_request

2017-04-05 Thread Johannes Thumshirn
On Wed, Apr 05, 2017 at 04:18:55PM +0200, Christoph Hellwig wrote: > The way NVMe uses this field is entirely different from the older > SCSI/BLOCK_PC usage, so move it into struct nvme_request. > > Also reduce the size of the file to a unsigned char so that we leave space > for additional smaller

Re: [PATCH 3/5] nvme: mark nvme_max_retries static

2017-04-05 Thread Johannes Thumshirn
On Wed, Apr 05, 2017 at 04:18:54PM +0200, Christoph Hellwig wrote: > Signed-off-by: Christoph Hellwig > --- Reviewed-by: Johannes Thumshirn -- Johannes Thumshirn Storage jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH,

Re: [PATCH 2/5] nvme: cleanup nvme_req_needs_retry

2017-04-05 Thread Johannes Thumshirn
On Wed, Apr 05, 2017 at 04:18:53PM +0200, Christoph Hellwig wrote: > Don't pass the status explicitly but derive it from the requeust, > and unwind the complex condition to be more readable. > > Signed-off-by: Christoph Hellwig > --- Looks good, Reviewed-by: Johannes Thumshirn -- Johannes Thu

Re: [PATCH 1/5] nvme: move ->retries setup to nvme_setup_cmd

2017-04-05 Thread Johannes Thumshirn
On Wed, Apr 05, 2017 at 04:18:52PM +0200, Christoph Hellwig wrote: > This way we get the behavior right for the non-PCIe transports. Could you please share a bit of your minds inner workings for us mere mortals? Thanks, Johannes -- Johannes Thumshirn

  1   2   >