Re: [PATCH 1/3] blk-mq: unify hctx delayed_run_work and run_work

2017-04-14 Thread Bart Van Assche
On Fri, 2017-04-14 at 14:02 -0600, Jens Axboe wrote: > I was waiting for further comments on patch 3/3. Hello Jens, Patch 3/3 is probably fine but I hope that you understand that the introduction of a new race condition does not make me enthusiast. Should your explanation of why that race is harm

Re: [PATCH block-tree] net: off by one in inet6_pton()

2017-04-14 Thread Jens Axboe
On 04/13/2017 01:42 PM, Dan Carpenter wrote: > If "scope_len" is sizeof(scope_id) then we would put the NUL terminator > one space beyond the end of the buffer. Added, thanks Dan. -- Jens Axboe

Re: [PATCH v4 0/5] blk-mq: Kyber multiqueue I/O scheduler

2017-04-14 Thread Jens Axboe
On 04/14/2017 01:59 AM, Omar Sandoval wrote: > From: Omar Sandoval > > This is v4 of Kyber, an I/O scheduler for multiqueue devices combining > several techniques: the scalable bitmap library, the new blk-stats API, > and queue depth throttling similar to blk-wbt. v1 was here [1], v2 was > here [

Re: [PATCH 1/3] blk-mq: unify hctx delayed_run_work and run_work

2017-04-14 Thread Jens Axboe
On 04/11/2017 12:00 PM, Bart Van Assche wrote: > On Mon, 2017-04-10 at 09:54 -0600, Jens Axboe wrote: >> void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx) >> { >> -cancel_work(&hctx->run_work); >> +cancel_delayed_work(&hctx->run_work); >> cancel_delayed_work(&hctx->delay_work); >

Re: [PATCH] remove the mg_disk driver

2017-04-14 Thread Jens Axboe
On 04/06/2017 05:28 AM, Christoph Hellwig wrote: > This drivers was added in 2008, but as far as a I can tell we never had a > single platform that actually registered resources for the platform driver. > > It's also been unmaintained for a long time and apparently has a ATA mode > that can be dri

Re: [PATCH] block: fix bio_will_gap()

2017-04-14 Thread Jens Axboe
On 04/14/2017 05:26 AM, Johannes Thumshirn wrote: > On Thu, Apr 13, 2017 at 6:06 PM, Ming Lei wrote: >> Now commit 729204ef49ec("block: relax check on sg gap") >> allows us to merge bios if both are physically contiguous, >> this change can merge huge of small bios in use case of mkfs, >> for exam

RE: Outstanding MQ questions from MMC

2017-04-14 Thread Avri Altman
> > 2. Turn RPMB and other ioctl() MMC operations into mmc_queue_req >things and funnel them into the block scheduler >using REQ_OP_DRV_IN/OUT requests. > Accessing the RPMB is done via a strange protocol, in which each access is comprised of several requests. For example, writing to

Re: [PATCH 1/6] blk-mq: Do not invoke queue operations on a dead queue

2017-04-14 Thread Bart Van Assche
On Fri, 2017-04-14 at 10:13 -0700, Omar Sandoval wrote: > Thanks, Bart. In this case, the absence of the "mq" directory should > tell you that the queue is dead. Will you move the cleanup in v2 or > should I submit a separate patch? Hello Omar, I will include a patch in v2 of this patch series th

Re: [PATCH 1/6] blk-mq: Do not invoke queue operations on a dead queue

2017-04-14 Thread Omar Sandoval
On Fri, Apr 14, 2017 at 04:12:01PM +, Bart Van Assche wrote: > On Fri, 2017-04-14 at 00:40 -0700, Omar Sandoval wrote: > > On Thu, Apr 13, 2017 at 11:05:32PM +, Bart Van Assche wrote: > > > On Thu, 2017-04-13 at 16:01 -0700, Omar Sandoval wrote: > > > > Looking at this, I think we have simi

Re: [PATCH v4 6/6] dm rq: Avoid that request processing stalls sporadically

2017-04-14 Thread Bart Van Assche
On Fri, 2017-04-14 at 09:13 +0800, Ming Lei wrote: > On Thu, Apr 13, 2017 at 09:59:57AM -0700, Bart Van Assche wrote: > > On 04/12/17 19:20, Ming Lei wrote: > > > On Wed, Apr 12, 2017 at 06:38:07PM +, Bart Van Assche wrote: > > > > If the blk-mq core would always rerun a hardware queue if a blo

Re: [PATCH 1/6] blk-mq: Do not invoke queue operations on a dead queue

2017-04-14 Thread Bart Van Assche
On Fri, 2017-04-14 at 00:40 -0700, Omar Sandoval wrote: > On Thu, Apr 13, 2017 at 11:05:32PM +, Bart Van Assche wrote: > > On Thu, 2017-04-13 at 16:01 -0700, Omar Sandoval wrote: > > > Looking at this, I think we have similar issues with most of the other > > > debugfs files. Should we move the

Re: [PATCH 5/6] blk-mq: Add blk_mq_ops.show_rq()

2017-04-14 Thread Bart Van Assche
On Thu, 2017-04-13 at 16:21 -0700, Omar Sandoval wrote: > How about passing the seq_file to the callback instead of this > arbitrarily-sized on-stack buffer? Hello Omar, That sounds like a good idea to me. I will make that change. Bart.

Re: [PATCH] block: Make writeback throttling defaults consistent for SQ devices

2017-04-14 Thread Bart Van Assche
On Wed, 2017-04-12 at 10:23 +0200, Jan Kara wrote: > +#ifndef CONFIG_BLK_WBT_MQ > + if (q->mq_ops) > + return; > +#endif > +#ifndef CONFIG_BLK_WBT_SQ > + if (q->request_fn) > + return; > +#endif > + > + /* > + * If this fails, we don't get throttling > +

[PATCH 6/8] nowait aio: ext4

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail for direct I/O: + i_rwsem is lockable + Writing beyond end of file (will trigger allocation) + Blocks are not allocated at the write location Signed-off-by: Goldwyn Rodrigues --- fs/ext4/file.c | 20 +++

[PATCH 7/8] nowait aio: xfs

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues If IOCB_NOWAIT is set, bail if the i_rwsem is not lockable immediately. IF IOMAP_NOWAIT is set, return EAGAIN in xfs_file_iomap_begin if it needs allocation either due to file extension, writing to a hole, or COW or waiting for other DIOs to finish. Signed-off-by: Goldwy

[PATCH 3/8] nowait aio: return if direct write will trigger writeback

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Find out if the write will trigger a wait due to writeback. If yes, return -EAGAIN. This introduces a new function filemap_range_has_page() which returns true if the file's mapping has a page within the range mentioned. Return -EINVAL for buffered AIO: there are multiple

[PATCH 5/8] nowait aio: return on congested block device

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues A new bio operation flag REQ_NOWAIT is introduced to identify bio's orignating from iocb with IOCB_NOWAIT. This flag indicates to return immediately if a request cannot be made instead of retrying. To facilitate this, QUEUE_FLAG_NOWAIT is set to devices which support this

[PATCH 8/8] nowait aio: btrfs

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail + i_rwsem is not lockable + NODATACOW or PREALLOC is not set + Cannot nocow at the desired location + Writing beyond end of file which is not allocated Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/file.c | 25 +

[PATCH 2/8] nowait aio: Introduce RWF_NOWAIT

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues This flag informs kernel to bail out if an AIO request will block for reasons such as file allocations, or a writeback triggered, or would block while allocating requests while performing direct I/O. Unfortunately, aio_flags is not checked for validity, which would break

[PATCH 0/8 v6] No wait AIO

2017-04-14 Thread Goldwyn Rodrigues
Formerly known as non-blocking AIO. This series adds nonblocking feature to asynchronous I/O writes. io_submit() can be delayed because of a number of reason: - Block allocation for files - Data writebacks for direct I/O - Sleeping because of waiting to acquire i_rwsem - Congested block device

[PATCH 4/8] nowait-aio: Introduce IOMAP_NOWAIT

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues IOCB_NOWAIT translates to IOMAP_NOWAIT for iomaps. This is used by XFS in the XFS patch. --- fs/iomap.c| 2 ++ include/linux/iomap.h | 1 + 2 files changed, 3 insertions(+) diff --git a/fs/iomap.c b/fs/iomap.c index 141c3cd55a8b..d1c81753d411 100644 --- a/fs/

[PATCH 1/8] Use RWF_* flags for AIO operations

2017-04-14 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues RWF_* flags is used for preadv2/pwritev2 calls. Port to use it for aio operations as well. For this, aio_rw_flags is introduced in struct iocb (using aio_reserved1) which will carry these flags. This is a precursor to the nowait AIO calls. Note, the only place RWF_HIPRI

Re: [PATCH] block: fix bio_will_gap()

2017-04-14 Thread Johannes Thumshirn
On Thu, Apr 13, 2017 at 6:06 PM, Ming Lei wrote: > Now commit 729204ef49ec("block: relax check on sg gap") > allows us to merge bios if both are physically contiguous, > this change can merge huge of small bios in use case of mkfs, > for example, mkfs.ntfs running time can be decreased to ~1/10. >

Re: [PATCH 02/25] block: remove the blk_execute_rq return value

2017-04-14 Thread h...@lst.de
On Thu, Apr 13, 2017 at 08:03:22PM +, Bart Van Assche wrote: > That blk_execute_rq() call can only be reached if a few lines above 0 was > assigned to the "error" variable. Since nfsd4_scsi_identify_device() returns > the value of the "error" variable I think -EIO should be assigned to that > v

Re: [PATCH 01/25] remove the mg_disk driver

2017-04-14 Thread h...@lst.de
On Thu, Apr 13, 2017 at 07:58:13PM +, Bart Van Assche wrote: > Should the person who submitted this driver be CC-ed for this patch (unsik > Kim )? Yes, he should. And in fact he was when I sent this patch out separately a little earlier, I just included it in this series for reference.

[PATCH v4 5/5] blk-mq: introduce Kyber multiqueue I/O scheduler

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval The Kyber I/O scheduler is an I/O scheduler for fast devices designed to scale to multiple queues. Users configure only two knobs, the target read and synchronous write latencies, and the scheduler tunes itself to achieve that latency goal. The implementation is based on "tok

[PATCH v4 2/5] blk-mq: add shallow depth option for blk_mq_get_tag()

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval Wire up the sbitmap_get_shallow() operation to the tag code so that a caller can limit the number of tags available to it. Signed-off-by: Omar Sandoval --- block/blk-mq-tag.c | 5 - block/blk-mq.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/

[PATCH v4 1/5] sbitmap: add sbitmap_get_shallow() operation

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval This operation supports the use case of limiting the number of bits that can be allocated for a given operation. Rather than setting aside some bits at the end of the bitmap, we can set aside bits in each word of the bitmap. This means we can keep the allocation hints spread o

[PATCH v4 4/5] blk-mq-sched: make completed_request() callback more useful

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval Currently, this callback is called right after put_request() and has no distinguishable purpose. Instead, let's call it before put_request() as soon as I/O has completed on the request, before we account it in blk-stat. With this, Kyber can enable stats when it sees a latency

[PATCH v4 3/5] blk-mq: export helpers

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval blk_mq_finish_request() is required for schedulers that define their own put_request(). blk_mq_run_hw_queue() is required for schedulers that hold back requests to be run later. Signed-off-by: Omar Sandoval --- block/blk-mq.c | 2 ++ include/linux/blk-mq.h | 1 + 2

[PATCH v4 0/5] blk-mq: Kyber multiqueue I/O scheduler

2017-04-14 Thread Omar Sandoval
From: Omar Sandoval This is v4 of Kyber, an I/O scheduler for multiqueue devices combining several techniques: the scalable bitmap library, the new blk-stats API, and queue depth throttling similar to blk-wbt. v1 was here [1], v2 was here [2], v3 was here [3]. v4 fixes a hang in v3 caused by a r

Re: [PATCH 1/6] blk-mq: Do not invoke queue operations on a dead queue

2017-04-14 Thread Omar Sandoval
On Thu, Apr 13, 2017 at 11:05:32PM +, Bart Van Assche wrote: > On Thu, 2017-04-13 at 16:01 -0700, Omar Sandoval wrote: > > On Tue, Apr 11, 2017 at 01:58:37PM -0700, Bart Van Assche wrote: > > > The blk-mq debugfs attributes are removed after blk_cleanup_queue() > > > has finished. Since running