Re: [PATCH blktests 1/9] blktests: add hepler functions for new md tests

Hi Johannes & Jens, Thank you for your information and positive feedbacks on this patch. I'm also inspired by xfstests, too. Most conventional filesystems have their test cases on it. I believe block layer can also have one. I felt sorry. It's first time I realized there is a test suite under

Re: [GIT PULL] Block changes for 4.18-rc

On 6/4/18 6:56 PM, Kent Overstreet wrote: > On Mon, Jun 04, 2018 at 05:42:04PM -0700, Linus Torvalds wrote: >> On Mon, Jun 4, 2018 at 12:04 PM Kent Overstreet >> wrote: >>> >>> However, that's not correct as is because mddev_delayed_put() calls >>> kobject_put(), and the kobject isn't initialized

[PATCH v3 00/25] InfiniBand Transport (IBTRS) and Network Block Device (IBNBD)

Hi all, here is v3 of IBNBD/IBTRS patches, which have minor changes: sparse fixes, comments, sysfs API changes, etc. Changelog - v3: o Sparse fixes: - le32 -> le16 conversion - pcpu and RCU wrong declaration - sysfs: dynamically alloc array of sockaddr

[PATCH v3 02/25] ibtrs: public interface header to establish RDMA connections

Introduce public header which provides set of API functions to establish RDMA connections from client to server machine using IBTRS protocol, which manages RDMA connections for each session, does multipathing and load balancing. Main functions for client (active) side: ibtrs_clt_open() -

[PATCH v3 23/25] ibnbd: include client and server modules into kernel compilation

Add IBNBD Makefile, Kconfig and also corresponding lines into upper block layer files. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/Kconfig| 2 ++ drivers/block/Makefile | 1 + drivers/block/ibnbd/Kconfig | 22 ++

[PATCH v3 18/25] ibnbd: client: sysfs interface functions

This is the sysfs interface to IBNBD block devices on client side: /sys/devices/virtual/ibnbd-client/ctl/ |- map_device | *** maps remote device | |- devices/ *** all mapped devices /sys/block/ibnbd/ibnbd_client/ |- unmap_device | *** unmaps device |

[PATCH v3 20/25] ibnbd: server: main functionality

This is main functionality of ibnbd-server module, which handles IBTRS events and IBNBD protocol requests, like map (open) or unmap (close) device. Also server side is responsible for processing incoming IBTRS IO requests and forward them to local mapped devices. Signed-off-by: Roman Pen

[PATCH v3 24/25] ibnbd: a bit of documentation

README with description of major sysfs entries. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/README | 299 + 1 file changed, 299 insertions(+) create mode 100644 drivers/block/ibnbd/README diff --git

[PATCH v3 19/25] ibnbd: server: private header with server structs and functions

This header describes main structs and functions used by ibnbd-server module, namely structs for managing sessions from different clients and mapped (opened) devices. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/ibnbd-srv.h | 100

[PATCH v3 03/25] ibtrs: private headers with IBTRS protocol structs and helpers

These are common private headers with IBTRS protocol structures, logging, sysfs and other helper functions, which are used on both client and server sides. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/ulp/ibtrs/ibtrs-log.h | 91 ++

[PATCH v3 21/25] ibnbd: server: functionality for IO submission to file or block dev

This provides helper functions for IO submission to file or block dev. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/ibnbd-srv-dev.c | 413 drivers/block/ibnbd/ibnbd-srv-dev.h | 149 + 2 files

[PATCH v3 22/25] ibnbd: server: sysfs interface functions

This is the sysfs interface to IBNBD mapped devices on server side: /sys/devices/virtual/ibnbd-server/ctl/devices// |- block_dev | *** link pointing to the corresponding block device sysfs entry | |- sessions// | *** sessions directory | |- read_only |

[PATCH v3 15/25] ibnbd: private headers with IBNBD protocol structs and helpers

These are common private headers with IBNBD protocol structures, logging, sysfs and other helper functions, which are used on both client and server sides. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/ibnbd-log.h | 71

[PATCH v3 25/25] MAINTAINERS: Add maintainer for IBNBD/IBTRS modules

Signed-off-by: Roman Pen Cc: Danil Kipnis Cc: Jack Wang --- MAINTAINERS | 14 ++ 1 file changed, 14 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index ca4afd68530c..201c6c8e039e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6782,6 +6782,20 @@ IBM ServeRAID RAID DRIVER S:

[PATCH v3 10/25] ibtrs: server: main functionality

This is main functionality of ibtrs-server module, which accepts set of RDMA connections (so called IBTRS session), creates/destroys sysfs entries associated with IBTRS session and notifies upper layer (user of IBTRS API) about RDMA requests or link events. Signed-off-by: Roman Pen

[PATCH v3 09/25] ibtrs: server: private header with server structs and functions

This header describes main structs and functions used by ibtrs-server module, mainly for accepting IBTRS sessions, creating/destroying sysfs entries, accounting statistics on server side. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang ---

[PATCH v3 16/25] ibnbd: client: private header with client structs and functions

This header describes main structs and functions used by ibnbd-client module, mainly for managing IBNBD sessions and mapped block devices, creating and destroying sysfs entries. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/ibnbd-clt.h | 172

[PATCH v3 17/25] ibnbd: client: main functionality

This is main functionality of ibnbd-client module, which provides interface to map remote device as local block device /dev/ibnbd and feeds IBTRS with IO requests. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/block/ibnbd/ibnbd-clt.c | 1817

[PATCH v3 13/25] ibtrs: include client and server modules into kernel compilation

Add IBTRS Makefile, Kconfig and also corresponding lines into upper layer infiniband/ulp files. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/Kconfig| 1 + drivers/infiniband/ulp/Makefile | 1 +

[PATCH v3 05/25] ibtrs: client: private header with client structs and functions

This header describes main structs and functions used by ibtrs-client module, mainly for managing IBTRS sessions, creating/destroying sysfs entries, accounting statistics on client side. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang ---

[PATCH v3 11/25] ibtrs: server: statistics functions

This introduces set of functions used on server side to account statistics of RDMA data sent/received. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/ulp/ibtrs/ibtrs-srv-stats.c | 110 + 1 file changed, 110 insertions(+)

[PATCH v3 12/25] ibtrs: server: sysfs interface functions

This is the sysfs interface to IBTRS sessions on server side: /sys/devices/virtual/ibtrs-server// *** IBTRS session accepted from a client peer | |- paths// *** established paths from a client in a session | |- disconnect | *** disconnect path |

[PATCH v3 14/25] ibtrs: a bit of documentation

README with description of major sysfs entries. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/ulp/ibtrs/README | 390 1 file changed, 390 insertions(+) create mode 100644 drivers/infiniband/ulp/ibtrs/README

[PATCH v3 07/25] ibtrs: client: statistics functions

This introduces set of functions used on client side to account statistics of RDMA data sent/received, amount of IOs inflight, latency, cpu migrations, etc. Almost all statistics is collected using percpu variables. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang ---

[PATCH v3 06/25] ibtrs: client: main functionality

This is main functionality of ibtrs-client module, which manages set of RDMA connections for each IBTRS session, does multipathing, load balancing and failover of RDMA requests. Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/ulp/ibtrs/ibtrs-clt.c |

[PATCH v3 08/25] ibtrs: client: sysfs interface functions

This is the sysfs interface to IBTRS sessions on client side: /sys/devices/virtual/ibtrs-client// *** IBTRS session created by ibtrs_clt_open() API call | |- max_reconnect_attempts | *** number of reconnect attempts for session | |- add_path | *** adds another

[PATCH v3 04/25] ibtrs: core: lib functions shared between client and server modules

This is a set of library functions existing as a ibtrs-core module, used by client and server modules. Mainly these functions wrap IB and RDMA calls and provide a bit higher abstraction for implementing of IBTRS protocol on client or server sides. Signed-off-by: Roman Pen Signed-off-by: Danil

Re: [PATCH] block: always set partition number to '0' in blk_partition_remap()

On Wed, 6 Jun 2018 08:26:56 -0600 Jens Axboe wrote: > On 6/6/18 8:22 AM, Hannes Reinecke wrote: > > blk_partition_remap() will only clear bi_partno if an actual > > remapping has happened. But flush request et al don't have an > > actual size, so the remapping doesn't happen and bi_partno is

Re: [PATCH] block: pass failfast and driver-specific flags to flush requests

On Wed, Jun 06, 2018 at 04:21:40PM +0200, Hannes Reinecke wrote: > If flush requests are being sent to the device we need to inherit the > failfast and driver-specific flags, too, otherwise I/O will fail. Looks fine, Reviewed-by: Christoph Hellwig

Re: [PATCH] block: always set partition number to '0' in blk_partition_remap()

On Wed, Jun 06, 2018 at 08:26:56AM -0600, Jens Axboe wrote: > On 6/6/18 8:22 AM, Hannes Reinecke wrote: > > blk_partition_remap() will only clear bi_partno if an actual remapping > > has happened. But flush request et al don't have an actual size, so > > the remapping doesn't happen and bi_partno

Re: [PATCH blktests 1/9] blktests: add hepler functions for new md tests

On Wed, Jun 06, 2018 at 08:29:25AM -0600, Jens Axboe wrote: > Hopefully this can be the start of migrating over those tests! Yes this would be great. I just wanted to connect the submitter and the md developers and make them aware of possibly duplicated efforts ;-). -- Johannes Thumshirn

Re: [PATCH blktests 1/9] blktests: add hepler functions for new md tests

On 6/6/18 2:44 AM, Johannes Thumshirn wrote: > On Wed, Jun 06, 2018 at 04:06:40PM +0800, bingjingc wrote: >> We'd like to leverage this test framework for testing linux raid >> software. There are several resync tasks in md/raid. For this commit, >> we are trying to add creation resync and basic

Re: [PATCH] block: pass failfast and driver-specific flags to flush requests

On 6/6/18 8:21 AM, Hannes Reinecke wrote: > If flush requests are being sent to the device we need to inherit the > failfast and driver-specific flags, too, otherwise I/O will fail. Looks good to me. -- Jens Axboe

Re: [PATCH] block: always set partition number to '0' in blk_partition_remap()

On 6/6/18 8:22 AM, Hannes Reinecke wrote: > blk_partition_remap() will only clear bi_partno if an actual remapping > has happened. But flush request et al don't have an actual size, so > the remapping doesn't happen and bi_partno is never cleared. > So for stacked devices blk_partition_remap()

Re: blktests block/019 lead system hang

On Wed, Jun 06, 2018 at 01:42:15PM +0800, Yi Zhang wrote: > Here is the output, and I can see "HotPlug+ Surprise+" on SltCap Thanks. That looks like a perfectly capable port. I even have the same switch in one of my machines, but the test doesn't trigger fatal firmware-first errors. Might need

[PATCH] block: always set partition number to '0' in blk_partition_remap()

blk_partition_remap() will only clear bi_partno if an actual remapping has happened. But flush request et al don't have an actual size, so the remapping doesn't happen and bi_partno is never cleared. So for stacked devices blk_partition_remap() will be called on each level. If (as is the case for

[PATCH] block: pass failfast and driver-specific flags to flush requests

If flush requests are being sent to the device we need to inherit the failfast and driver-specific flags, too, otherwise I/O will fail. Signed-off-by: Hannes Reinecke --- block/blk-flush.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/block/blk-flush.c

Re: [PATCH blktests 1/9] blktests: add hepler functions for new md tests

On Wed, Jun 06, 2018 at 04:06:40PM +0800, bingjingc wrote: > We'd like to leverage this test framework for testing linux raid > software. There are several resync tasks in md/raid. For this commit, > we are trying to add creation resync and basic recovery tests for > every raid type. > > RAID is

[PATCH blktests 9/9] blktests: add a regression test for raid6 recovery

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/008 | 69 tests/md/008.out | 4 2 files changed, 73 insertions(+) create mode 100755 tests/md/008 create mode 100644 tests/md/008.out diff --git

[PATCH blktests 8/9] blktests: add a regression test for raid5 recovery

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/007 | 72 tests/md/007.out | 4 2 files changed, 76 insertions(+) create mode 100755 tests/md/007 create mode 100644 tests/md/007.out diff --git

[PATCH blktests 7/9] blktests: add a regression test for raid10 recovery

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/006 | 80 tests/md/006.out | 4 +++ 2 files changed, 84 insertions(+) create mode 100755 tests/md/006 create mode 100644 tests/md/006.out diff --git

[PATCH blktests 6/9] blktests: add a regression test for raid1 recovery

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/005 | 72 tests/md/005.out | 4 2 files changed, 76 insertions(+) create mode 100755 tests/md/005 create mode 100644 tests/md/005.out diff --git

[PATCH blktests 5/9] blktests: add a regression test for raid6 creation resync

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/004 | 60 tests/md/004.out | 4 2 files changed, 64 insertions(+) create mode 100755 tests/md/004 create mode 100644 tests/md/004.out diff --git

[PATCH blktests 4/9] blktests: add a regression test for raid5 creation resync

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/003 | 60 tests/md/003.out | 4 2 files changed, 64 insertions(+) create mode 100755 tests/md/003 create mode 100644 tests/md/003.out diff --git

[PATCH blktests 3/9] blktests: add a regression test for raid10 creation resync

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/002 | 68 tests/md/002.out | 4 2 files changed, 72 insertions(+) create mode 100755 tests/md/002 create mode 100644 tests/md/002.out diff --git

[PATCH blktests 2/9] blktests: add a regression test for raid1 creation resync

Reviewed-by: Chung-Chiang Cheng Signed-off-by: BingJing Chang --- tests/md/001 | 60 tests/md/001.out | 4 2 files changed, 64 insertions(+) create mode 100755 tests/md/001 create mode 100644 tests/md/001.out diff --git

[PATCH blktests 1/9] blktests: add hepler functions for new md tests

We'd like to leverage this test framework for testing linux raid software. There are several resync tasks in md/raid. For this commit, we are trying to add creation resync and basic recovery tests for every raid type. RAID is different from other block devices. It requires several raid devices