Re: [PATCH 7/8] blk-wbt: add general throttling mechanism

2016-11-09 Thread Jan Kara
On Tue 08-11-16 08:41:09, Jens Axboe wrote: > On Tue, Nov 08 2016, Jan Kara wrote: > > On Tue 01-11-16 15:08:50, Jens Axboe wrote: > > > We can hook this up to the block layer, to help throttle buffered > > > writes. > > > > > > wbt registers a few trace points that can be used to track what is >

Re: [PATCH 6/8] block: add scalable completion tracking of requests

2016-11-09 Thread Jan Kara
On Tue 08-11-16 08:25:52, Jens Axboe wrote: > On 11/08/2016 06:30 AM, Jan Kara wrote: > >On Tue 01-11-16 15:08:49, Jens Axboe wrote: > >>For legacy block, we simply track them in the request queue. For > >>blk-mq, we track them on a per-sw queue basis, which we can then > >>sum up through the hardw

Re: support for partial irq affinity assignment V3

2016-11-09 Thread Christoph Hellwig
On Wed, Nov 09, 2016 at 08:51:35AM +0100, Thomas Gleixner wrote: > It's available from > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block > > for you to pull into the block tree so you can apply the block changes. I don't actually need them in the block tree, but in th

Re: support for partial irq affinity assignment V3

2016-11-09 Thread Thomas Gleixner
On Wed, 9 Nov 2016, Christoph Hellwig wrote: > On Wed, Nov 09, 2016 at 08:51:35AM +0100, Thomas Gleixner wrote: > > It's available from > > > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block > > > > for you to pull into the block tree so you can apply the block changes.

Re: [PATCH 7/8] blk-wbt: add general throttling mechanism

2016-11-09 Thread Jens Axboe
On 11/09/2016 01:40 AM, Jan Kara wrote: So for devices with write cache, you will completely drain the device before waking anybody waiting to issue new requests. Isn't it too strict? In particular may_queue() will allow new writers to issue new writes once we drop below the limit so it can happe

Re: [PATCH 6/8] block: add scalable completion tracking of requests

2016-11-09 Thread Jens Axboe
On 11/09/2016 02:01 AM, Jan Kara wrote: On Tue 08-11-16 08:25:52, Jens Axboe wrote: On 11/08/2016 06:30 AM, Jan Kara wrote: On Tue 01-11-16 15:08:49, Jens Axboe wrote: For legacy block, we simply track them in the request queue. For blk-mq, we track them on a per-sw queue basis, which we can t

Re: BUG: Hung task timeouts in for-4.10/dio

2016-11-09 Thread Logan Gunthorpe
Hey, I just tested with the latest for-4.10/block branch and it looks like it fixed our problem. Thanks! Logan On 08/11/16 07:55 PM, Damien Le Moal wrote: > > Jens, > > On 11/9/16 11:45, Jens Axboe wrote: >> I just committed the work-around. But yes, let's have a logical revert >> and require

Re: [PATCH] Unittest framework based on nvme-cli.

2016-11-09 Thread Stephen Bates
Hi Chaitanya Thanks for doing this work. A simple test framework that builds on top of nvme-cli is a very good idea. My comments are below. Cheers Stephen >From 38de47d65b855f4dc361147f673a9b3721768e2d Mon Sep 17 00:00:00 2001 From: Chaitanya Kulkarni Date: Wed, 26 Oct 2016 12:55:21 -0700 Sub

[PATCH] block: clear all of bi_opf in bio_set_op_attrs

2016-11-09 Thread Christoph Hellwig
Since commit 87374179 ("block: add a proper block layer data direction encoding") we only OR the new op and flags into bi_opf in bio_set_op_attrs instead of clearing the old value. I've not seen any breakage with the new behavior, but it seems dangerous. Signed-off-by: Christoph Hellwig --- inc

[patch] nd_blk,nd_pmem,nd_btt: add endio blktrace events

2016-11-09 Thread Jeff Moyer
Right now, any of the above three drivers will report Q events in blktrace but no corresponding C events. Fix it. Signed-off-by: Jeff Moyer diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c index 9faaa96..90676f3 100644 --- a/drivers/nvdimm/blk.c +++ b/drivers/nvdimm/blk.c @@ -19,6 +19,7

Re: [patch] nd_blk,nd_pmem,nd_btt: add endio blktrace events

2016-11-09 Thread Christoph Hellwig
On Wed, Nov 09, 2016 at 02:08:33PM -0500, Jeff Moyer wrote: > Right now, any of the above three drivers will report Q events in > blktrace but no corresponding C events. Fix it. It seems like that trace point should simply go into bio_endio and be removed from any existing callsite. -- To unsubsc

Re: [patch] nd_blk,nd_pmem,nd_btt: add endio blktrace events

2016-11-09 Thread Jeff Moyer
Christoph Hellwig writes: > On Wed, Nov 09, 2016 at 02:08:33PM -0500, Jeff Moyer wrote: >> Right now, any of the above three drivers will report Q events in >> blktrace but no corresponding C events. Fix it. > > It seems like that trace point should simply go into bio_endio > and be removed from

Re: [patch] nd_blk,nd_pmem,nd_btt: add endio blktrace events

2016-11-09 Thread Christoph Hellwig
On Wed, Nov 09, 2016 at 02:31:30PM -0500, Jeff Moyer wrote: > bio_endio is still called for request_fn drivers, so you'd see two > completion events for those drivers if we did that, no? We'd see the bio_endio trace in addition to the request one, but they are at different granularities. Similar

Re: [patch] nd_blk,nd_pmem,nd_btt: add endio blktrace events

2016-11-09 Thread Jeff Moyer
Christoph Hellwig writes: > On Wed, Nov 09, 2016 at 02:31:30PM -0500, Jeff Moyer wrote: >> bio_endio is still called for request_fn drivers, so you'd see two >> completion events for those drivers if we did that, no? > > We'd see the bio_endio trace in addition to the request one, but > they are

Re: [PATCH 6/8] block: add scalable completion tracking of requests

2016-11-09 Thread Jens Axboe
On 11/09/2016 09:09 AM, Jens Axboe wrote: On 11/09/2016 02:01 AM, Jan Kara wrote: On Tue 08-11-16 08:25:52, Jens Axboe wrote: On 11/08/2016 06:30 AM, Jan Kara wrote: On Tue 01-11-16 15:08:49, Jens Axboe wrote: For legacy block, we simply track them in the request queue. For blk-mq, we track t

Re: [PATCH 7/8] blk-wbt: add general throttling mechanism

2016-11-09 Thread Jens Axboe
On 11/09/2016 09:07 AM, Jens Axboe wrote: On 11/09/2016 01:40 AM, Jan Kara wrote: So for devices with write cache, you will completely drain the device before waking anybody waiting to issue new requests. Isn't it too strict? In particular may_queue() will allow new writers to issue new writes o

Re: [PATCH 7/8] blk-wbt: add general throttling mechanism

2016-11-09 Thread Dave Chinner
On Wed, Nov 09, 2016 at 09:07:08AM -0700, Jens Axboe wrote: > On 11/09/2016 01:40 AM, Jan Kara wrote: > >Also I'm not sure why such logic for devices with writeback cache is > >needed. Sure the disk is fast to accept writes but if that causes long > >read latencies, we should scale down the writeba