Re: [PATCH blktests 00/15] Add SRP initiator driver tests

2018-06-22 Thread Ming Lei
On Sat, Jun 23, 2018 at 11:14 AM, Ming Lei wrote: > On Sat, Jun 23, 2018 at 6:19 AM, Bart Van Assche > wrote: >> Hello Omar, >> >> As promised during LSF/MM, I have converted the srp-tests software to the >> blktests framework. This patch series included all blktests patches I postd >> a few

Re: [PATCH blktests 00/15] Add SRP initiator driver tests

2018-06-22 Thread Ming Lei
On Sat, Jun 23, 2018 at 6:19 AM, Bart Van Assche wrote: > Hello Omar, > > As promised during LSF/MM, I have converted the srp-tests software to the > blktests framework. This patch series included all blktests patches I postd > a few days ago. Please consider this patch series for inclusion in

Re: [PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Ming Lei
On Fri, Jun 22, 2018 at 05:13:31PM -0600, Jens Axboe wrote: > On 6/22/18 5:11 PM, Ming Lei wrote: > > On Fri, Jun 22, 2018 at 04:51:26PM -0600, Jens Axboe wrote: > >> On 6/22/18 4:43 PM, Ming Lei wrote: > >>> On Fri, Jun 22, 2018 at 01:26:10PM -0600, Jens Axboe wrote: > blk-wbt adds waiters

Re: [PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Jens Axboe
On 6/22/18 5:11 PM, Ming Lei wrote: > On Fri, Jun 22, 2018 at 04:51:26PM -0600, Jens Axboe wrote: >> On 6/22/18 4:43 PM, Ming Lei wrote: >>> On Fri, Jun 22, 2018 at 01:26:10PM -0600, Jens Axboe wrote: blk-wbt adds waiters to the tail of the waitqueue, and factors in the task placement in

Re: [PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Ming Lei
On Fri, Jun 22, 2018 at 04:51:26PM -0600, Jens Axboe wrote: > On 6/22/18 4:43 PM, Ming Lei wrote: > > On Fri, Jun 22, 2018 at 01:26:10PM -0600, Jens Axboe wrote: > >> blk-wbt adds waiters to the tail of the waitqueue, and factors in the > >> task placement in its decision making on whether or not

Re: [PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Jens Axboe
On 6/22/18 4:43 PM, Ming Lei wrote: > On Fri, Jun 22, 2018 at 01:26:10PM -0600, Jens Axboe wrote: >> blk-wbt adds waiters to the tail of the waitqueue, and factors in the >> task placement in its decision making on whether or not the current task >> can proceed. This can cause issues for the

Re: [PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Ming Lei
On Fri, Jun 22, 2018 at 01:26:10PM -0600, Jens Axboe wrote: > blk-wbt adds waiters to the tail of the waitqueue, and factors in the > task placement in its decision making on whether or not the current task > can proceed. This can cause issues for the lowest class of writers, > since they can get

[PATCH blktests 12/15] check: Make TMPDIR absolute

2018-06-22 Thread Bart Van Assche
This avoids that ${TMPDIR} gets misinterpreted if a test script modifies the working directory. Signed-off-by: Bart Van Assche --- check | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/check b/check index cf2090f546a5..a7dd825ad03a 100755 --- a/check +++ b/check @@ -323,7

[PATCH blktests 14/15] Add the discontiguous-io test program

2018-06-22 Thread Bart Van Assche
This test program is used by one of the srp tests. Signed-off-by: Bart Van Assche --- src/.gitignore | 1 + src/Makefile | 4 + src/discontiguous-io.cpp | 340 +++ 3 files changed, 345 insertions(+) create mode 100644

[PATCH blktests 13/15] src/Makefile: Fix pattern rule

2018-06-22 Thread Bart Van Assche
According to the GNU make documentation, the syntax for pattern rules is as follows: +: Signed-off-by: Bart Van Assche --- src/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Makefile b/src/Makefile index 612282d14af8..d7ae832fc4fd 100644 ---

[PATCH blktests 08/15] check: Suppress a shellcheck warning about the DMESG_FILTER initialization

2018-06-22 Thread Bart Van Assche
Avoid that shellcheck reports the following: check:396:2: warning: Use var=$(command) to assign output (or quote to assign string). [SC2209] Signed-off-by: Bart Van Assche --- check | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/check b/check index

[PATCH blktests 00/15] Add SRP initiator driver tests

2018-06-22 Thread Bart Van Assche
Hello Omar, As promised during LSF/MM, I have converted the srp-tests software to the blktests framework. This patch series included all blktests patches I postd a few days ago. Please consider this patch series for inclusion in the official blktests repository. Note: this patch series is also

[PATCH blktests 09/15] Makefile: Do not suppress useful shellcheck warnings

2018-06-22 Thread Bart Van Assche
All the shellcheck warnings that are currently suppressed are useful. Additionally, it is easy to avoid false positives for the currently suppressed categories of shellcheck warnings. Hence stop suppressing shellcheck warnings. See also commit 17a59e0dc212 ("Fix all shellcheck warnings").

[PATCH blktests 10/15] tests/meta/group: Fix shellcheck complaints

2018-06-22 Thread Bart Van Assche
Avoid that shellcheck complains about unused global variables. Avoid that shellcheck complains about using cat instead of redirection. Signed-off-by: Bart Van Assche --- tests/meta/group | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/meta/group

[PATCH blktests 03/15] check, tests/meta/012: Use array["index"] instead of array[index]

2018-06-22 Thread Bart Van Assche
This causes shellcheck to stop complaining about these array element accesses. Signed-off-by: Bart Van Assche --- check | 44 ++-- tests/meta/012 | 4 ++-- 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/check b/check index

[PATCH blktests 06/15] Multiple tests: remove unused and undefined variables

2018-06-22 Thread Bart Van Assche
Signed-off-by: Bart Van Assche Acked-by: Chaitanya Kulkarni --- tests/loop/003 | 3 --- tests/nvme/010 | 3 +-- tests/nvme/011 | 3 +-- tests/nvme/012 | 1 - tests/nvme/013 | 1 - 5 files changed, 2 insertions(+), 9 deletions(-) diff --git a/tests/loop/003 b/tests/loop/003 index

[PATCH blktests 11/15] make check: Also check tests/*/group files

2018-06-22 Thread Bart Van Assche
Signed-off-by: Bart Van Assche --- Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index c4bffa2d3f32..8031d2ed0eca 100644 --- a/Makefile +++ b/Makefile @@ -5,7 +5,8 @@ clean: $(MAKE) -C src clean check: - shellcheck -x -f gcc

[PATCH blktests 01/15] common/rc: Fix _have_tracepoint()

2018-06-22 Thread Bart Van Assche
Make sure that _have_tracepoint() uses the argument passed to that function instead of using an undefined variable. Signed-off-by: Bart Van Assche --- common/rc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/common/rc b/common/rc index 7f1728025364..7592400dda82 100644 --- a/common/rc

[PATCH blktests 04/15] Suppress shellcheck complaints about global variables

2018-06-22 Thread Bart Van Assche
Add a new file common/shellcheck and include it in all test scripts that use global variables. The file common/shellcheck contains a statement that stops shellcheck to complain about global variables: echo "$CHECK_DMESG ${CPUS_ONLINE_SAVED[*]} $DESCRIPTION $DMESG_FILTER $FIO_PERF_FIELDS

[PATCH blktests 15/15] Add tests for the SRP initiator and target drivers

2018-06-22 Thread Bart Van Assche
This patch adds the following tests: 001: Create and remove LUNs 002: File I/O on top of multipath concurrently with logout and login (mq) 003: File I/O on top of multipath concurrently with logout and login (sq) 004: File I/O on top of multipath concurrently with logout and login (sq-on-mq) 005:

[PATCH blktests 02/15] Annotate include statements in shell scripts where the source file is a variable

2018-06-22 Thread Bart Van Assche
This causes shellcheck to stop complaining about these include statements. Signed-off-by: Bart Van Assche --- check | 3 +++ 1 file changed, 3 insertions(+) diff --git a/check b/check index 4baa8dde2436..5e1ba7bc8401 100755 --- a/check +++ b/check @@ -23,6 +23,7 @@ _found_test() {

[PATCH blktests 05/15] check: Avoid that shellcheck complains that $FULL appears unused

2018-06-22 Thread Bart Van Assche
$FULL is a global variable. Avoid that shellcheck complains about it. Signed-off-by: Bart Van Assche --- check | 2 ++ 1 file changed, 2 insertions(+) diff --git a/check b/check index 1774be4fe1a7..d9ffea6ec1f6 100755 --- a/check +++ b/check @@ -301,6 +301,8 @@ _call_test() { local

[PATCH blktests 07/15] Avoid passing tests/block/002 arguments to _init_scsi_debug

2018-06-22 Thread Bart Van Assche
This patch avoids that shellcheck reports the following: tests/block/002:34:7: note: Use _init_scsi_debug "$@" if function's $1 should mean script's $1. [SC2119] Signed-off-by: Bart Van Assche --- common/scsi_debug | 3 +++ tests/block/002 | 2 +- 2 files changed, 4 insertions(+), 1

Re: [PATCH] blk-mq: Fix timeout handling in case the timeout handler returns BLK_EH_DONE

2018-06-22 Thread Jens Axboe
On 6/22/18 2:18 PM, Bart Van Assche wrote: > Make sure that RQF_TIMED_OUT is cleared when a request is reused > after a block driver timeout handler has returned BLK_EH_DONE. Thanks Bart, I think this looks good. I'm going to try and take some time to go over the timeout parts, to ensure that

[PATCH] blk-mq: Fix timeout handling in case the timeout handler returns BLK_EH_DONE

2018-06-22 Thread Bart Van Assche
Make sure that RQF_TIMED_OUT is cleared when a request is reused after a block driver timeout handler has returned BLK_EH_DONE. Fixes: da6612673988 ("blk-mq: don't time out requests again that are in the timeout handler") Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang

[PATCH] blk-wbt: fix indefinite background writeback sleep

2018-06-22 Thread Jens Axboe
blk-wbt adds waiters to the tail of the waitqueue, and factors in the task placement in its decision making on whether or not the current task can proceed. This can cause issues for the lowest class of writers, since they can get woken up, denied access, and then put back to sleep at the end of

Re: [RESEND PATCH] lightnvm: pblk: add asynchronous partial read

2018-06-22 Thread Jens Axboe
On 6/22/18 12:17 PM, Matias Bjørling wrote: >> +/* Re-use allocated memory for intermediate lbas */ >> +lba_list_mem = (((void *)rqd->ppa_list) + pblk_dma_ppa_size); >> + >> +new_bio = bio_alloc(GFP_KERNEL, nr_holes); > > > new_bio can return NULL. It can't, not if __GFP_WAIT is set

Re: [RESEND PATCH] lightnvm: pblk: add asynchronous partial read

2018-06-22 Thread Matias Bjørling
On 06/18/2018 07:56 PM, Heiner Litz wrote: In the read path, partial reads are currently performed synchronously which affects performance for workloads that generate many partial reads. This patch adds an asynchronous partial read path as well as the required partial read ctx. Signed-off-by:

Re: [PATCH] bdi: Fix another oops in wb_workfn()

2018-06-22 Thread Jens Axboe
On 6/22/18 2:52 AM, Jan Kara wrote: > On Mon 18-06-18 10:40:14, Tejun Heo wrote: >> On Mon, Jun 18, 2018 at 03:46:58PM +0200, Jan Kara wrote: >>> syzbot is reporting NULL pointer dereference at wb_workfn() [1] due to >>> wb->bdi->dev being NULL. And Dmitry confirmed that wb->state was >>>

Re: [PATCH] lightnvm: pblk: Add read memory barrier when reading from rb

2018-06-22 Thread Matias Bjørling
On 06/21/2018 12:54 AM, Heiner Litz wrote: READ_ONCE does not imply a read memory barrier in the presence of control dependencies between two separate memory locations (flags and data). On x86 TSO, reading from the data page might be reordered before the flags read. See chapter CONTROL

Re: [GIT PULL] nvme fixes for 4.18

2018-06-22 Thread Jens Axboe
On 6/22/18 7:00 AM, Christoph Hellwig wrote: > Various relatively small fixes, mostly to fix error handling of various > sorts. > > > The following changes since commit 9c24c10a2c1e1bb478b6bb70612d9e885aee044f: > > Revert "block: Add warning for bi_next not NULL in bio_endio()" (2018-06-19 >

[GIT PULL] nvme fixes for 4.18

2018-06-22 Thread Christoph Hellwig
Various relatively small fixes, mostly to fix error handling of various sorts. The following changes since commit 9c24c10a2c1e1bb478b6bb70612d9e885aee044f: Revert "block: Add warning for bi_next not NULL in bio_endio()" (2018-06-19 11:59:47 -0600) are available in the Git repository at:

Re: [PATCH] bdi: Fix another oops in wb_workfn()

2018-06-22 Thread Jan Kara
On Mon 18-06-18 10:40:14, Tejun Heo wrote: > On Mon, Jun 18, 2018 at 03:46:58PM +0200, Jan Kara wrote: > > syzbot is reporting NULL pointer dereference at wb_workfn() [1] due to > > wb->bdi->dev being NULL. And Dmitry confirmed that wb->state was > > WB_shutting_down after wb->bdi->dev became