Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

2018-01-21 Thread Goldwyn Rodrigues
On 01/20/2018 09:07 PM, Jens Axboe wrote: > On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: >> >> >> On 01/20/2018 08:11 PM, Andi Kleen wrote: >>> Goldwyn Rodrigues writes: >>> From: Goldwyn Rodrigues In case direct I/O encounters an error midway, it returns the error. Instead

blk-mq hangs easily with LLVM+clang test suite

2018-01-21 Thread David Zarzycki
Hello, I think I’ve narrowed down a kernel hang to the multi-queue block layer when running the LLVM+clang test suite. Here is what I’ve gathered: 1) The hang does *not* reproduce on tmpfs or a SATA SSD drive. 2) The hang does reproduce on blk-mq devices (NVMe or loopback to a file in tmpfs). 3

Re: blk-mq hangs easily with LLVM+clang test suite

2018-01-21 Thread Bart Van Assche
On Sun, 2018-01-21 at 11:19 -0500, David Zarzycki wrote: > How can I help debug this further? Please apply patch [1] on top of Jens' for-next branch [2], reproduce the hang and provide the output of the following command: (cd /sys/kernel/debug/block && find . -type f -exec grep -aH . {} \;) Than

Re: blk-mq hangs easily with LLVM+clang test suite

2018-01-21 Thread David Zarzycki
Hi Bart, I can do [1] and [2] but I won’t be able to provide the command output because the hang is almost total. No new commands can run because the scheduler is hung. For example, see this backtrace: http://znu.io/IMG_0362.jpg Is there another approach I can use? Dave > On Jan 21, 2018, at

[PATCH RESEND 1/3] blk-throttle: fix NULL pointer dereference risk in throtl_select_dispatch

2018-01-21 Thread Joseph Qi
tg in throtl_select_dispatch is used first and then do check. Since tg may be NULL, it has potential NULL pointer dereference risk. So fix it. Signed-off-by: Joseph Qi Reviewed-by: Jiufei Xue --- block/blk-throttle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/bl

[PATCH RESEND 2/3] blk-throttle: support only configure leaf nodes

2018-01-21 Thread Joseph Qi
Currently if we only configure leaf nodes, downgrade check won't function properly as parent's last low overflow time is meaningless and always returns now, thus it will be treated always run above low limit and doesn't have to do downgrade. In this case, since parent doesn't have low limit, ignore

[PATCH RESEND 3/3] blk-throttle: do downgrade/upgrade check when issuing io to lower layer

2018-01-21 Thread Joseph Qi
Currently downgrade/upgrade check is done when io firstly comes to block throttle layer. In case of writeback, a large number of ios will firstly be throttled in throttle queue and then dispatched when timer is kicked, which won't be checked because REQ_THROTTLED is set. This will lead to low limit

(v2. forward to qemu )-Panic with ext4,nbd,qemu-img,block

2018-01-21 Thread Hongzhi, Song
Hello, I create a virtual disk-image using qemu-img. And then I use /dev/nbd to map the image. I mount the /dev/nbd to a local dir with ext4-format Finally, I have some trouble about ext4-filesystem and block device, with using demand of rsync or dd to write the image. Reproduce :     qemu

Re: blk-mq hangs easily with LLVM+clang test suite

2018-01-21 Thread David Zarzycki
Hi Bart, Looks like I got lucky. Here is the output with the branch and patch you suggested: [root@dell block]# find . -type f -exec grep -aH . {} \; ./nvme0n1/hctx6/cpu319/completed:0 0 ./nvme0n1/hctx6/cpu319/merged:0 ./nvme0n1/hctx6/cpu319/dispatched:0 0 ./nvme0n1/hctx6/cpu317/completed:0 0 ./

Re: blk-mq: introduce BLK_STS_DEV_RESOURCE

2018-01-21 Thread Ming Lei
On Sat, Jan 20, 2018 at 10:52:01PM -0500, Mike Snitzer wrote: > On Sat, Jan 20 2018 at 7:57pm -0500, > Ming Lei wrote: > > > On Sat, Jan 20, 2018 at 12:30:02PM -0500, Mike Snitzer wrote: > > > On Sat, Jan 20 2018 at 8:48am -0500, > > > Ming Lei wrote: > > > > > > > This status is returned fro

[PATCH 1/5] blk-mq: introduce BLK_STS_DEV_RESOURCE

2018-01-21 Thread Ming Lei
This status is returned from driver to block layer if device related resource is run out of, but driver can guarantee that IO dispatch is triggered in future when the resource is available. This patch converts some drivers to use this return value. Meantime if driver returns BLK_STS_RESOURCE and S

[PATCH 0/5] blk-mq & dm: fix IO hang and deal with one performance issue

2018-01-21 Thread Ming Lei
Hi Jens, This 1st patch introduces BLK_STS_DEV_RESOURCE for dealing with IO hang when non-device resource is run out of and queue is idle, and this approach is suggested by you. The 2nd and 3rd patches cleans dm-rq a bit and prepares for applying blk_get_request_notify(). The 4th patch introduce

[PATCH 2/5] dm-rq: handle dispatch exception in dm_dispatch_clone_request()

2018-01-21 Thread Ming Lei
So we can clean up map_request() a bit, no functional change. Cc: Mike Snitzer Cc: Laurence Oberman Cc: Bart Van Assche Signed-off-by: Ming Lei --- drivers/md/dm-rq.c | 23 --- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/md/dm-rq.c b/drivers/md/d

[PATCH 3/5] dm-rq: return BLK_STS_* from map_request()

2018-01-21 Thread Ming Lei
Except for DM_MAPIO_REQUEUE, map_request() handles other dispatch exception already, so return BLK_STS_* from map_request() directly. Another change is that if dm_dispatch_clone_request() returns BLK_STS_DEV_RESOURCE from underlying queue, this status is returned to blk-mq too, since underlying qu

[PATCH 4/5] blk-mq: introduce blk_get_request_notify

2018-01-21 Thread Ming Lei
DM-MPATH need to allocate request from underlying queue, but when the allocation fails, there is no way to make underlying queue's RESTART to restart DM's queue. This patch introduces blk_get_request_notify() for this purpose, and caller need to pass 'wait_queue_entry_t' to this function, and make

[PATCH 5/5] dm-mpath: use blk_mq_alloc_request_notify for allocating blk-mq req

2018-01-21 Thread Ming Lei
When blk_get_request() fails to allocate one request, there is no way to make underlying queue's RESTART to restart DM's queue, this patch applies the new introduced blk_mq_alloc_request_notify() to deal with this issue. The following issues can be addressed: 1) When the blk_get_request() fails,

Re: [PATCH 3/5] dm-rq: return BLK_STS_* from map_request()

2018-01-21 Thread Ming Lei
On Mon, Jan 22, 2018 at 11:35:48AM +0800, Ming Lei wrote: > Except for DM_MAPIO_REQUEUE, map_request() handles other dispatch exception > already, so return BLK_STS_* from map_request() directly. > > Another change is that if dm_dispatch_clone_request() returns > BLK_STS_DEV_RESOURCE from underlyi

[PATCH TRIVIAL] bsg: use pr_debug instead of hand crafted macros

2018-01-21 Thread Johannes Thumshirn
Use pr_debug instead of hand crafted macros. This way it is not needed to re-compile the kernel to enable bsg debug outputs and it's possible to selectively enable specific prints. Signed-off-by: Johannes Thumshirn --- block/bsg.c | 35 +-- 1 file changed, 13 inse