Re: [dm-devel] [PATCH 0/4] multipath-tools unit test fixups

2020-04-28 Thread Martin Wilck
Hello Christophe, On Tue, 2020-03-03 at 00:38 +0100, Christophe Varoqui wrote: > Ben, > > take the time you need for review, > I'll wait for this patchset before tagging the next version. did you notice that Ben acked this set on Mar 13th? https://www.redhat.com/archives/dm-devel/2020-March/msg

Re: [dm-devel] [RFC PATCH v2 0/3] multipath: new path validation library

2020-04-28 Thread Martin Wilck
On Fri, 2020-04-03 at 01:50 -0500, Benjamin Marzinski wrote: > This patchset is for a new library that programs can use to determine > if a device belongs to multipath. The primary user that this is > intended for is SID, the Storage Instantiation Daemon > > https://github.com/sid-project > > Ri

Re: [dm-devel] [RFC PATCH v2 3/3] multipath: add libmpathvalid library

2020-04-28 Thread Martin Wilck
On Fri, 2020-04-03 at 01:50 -0500, Benjamin Marzinski wrote: > This library allows other programs to check if a path should be > claimed > by multipath. Currently, it only includes two functions. > mpath_get_mode() get the configured find_multipaths mode. > mpath_is_path() returns whether the devi

Re: [dm-devel] [RFC PATCH v2 2/3] libmultipath: make sysfs_is_multipathed able to return wwid

2020-04-28 Thread Martin Wilck
On Fri, 2020-04-03 at 01:50 -0500, Benjamin Marzinski wrote: > sysfs_is_multipathed reads the wwid of the dm device holding a path > to > check if its a multipath device. Add code to optinally set pp->wwid > to > that wwid. This will be used by a future patch. > > Signed-off-by: Benjamin Marzins

Re: [dm-devel] [PATCH 3/3] block: bypass ->make_request_fn for blk-mq drivers

2020-04-28 Thread Mike Snitzer
On Sat, Apr 25 2020 at 3:53am -0400, Christoph Hellwig wrote: > Call blk_mq_make_request when no ->make_request_fn is set. This is > safe now that blk_alloc_queue always sets up the pointer for make_request > based drivers. This avoids an indirect call in the blk-mq driver I/O > fast path, whi

Re: [dm-devel] [PATCH 2/3] dm: remove the make_request_fn check in device_area_is_invalid

2020-04-28 Thread Mike Snitzer
On Sat, Apr 25 2020 at 3:53am -0400, Christoph Hellwig wrote: > Signed-off-by: Christoph Hellwig Think it'd be useful to add a commit message like you did for revert commit f01b411f41f91fc3196eae4317cf8b4d872830a6 , e.g.: We can't have queues without a make_request_fn any more (and the loop d

Re: [dm-devel] [PATCH 13/13] dm-zoned: metadata version 2

2020-04-28 Thread Mike Snitzer
On Tue, Apr 28 2020 at 6:54am -0400, Damien Le Moal wrote: > With this, everything works fine for single and dual device case. Cool. Hannes, pleasee fold Damien's changes in for v3, thanks! > But I only did very light testing (formating witth ext4, mounting, > running simple fio, unmount). I

Re: [dm-devel] dm: multipath: Use updated MPATHF_QUEUE_IO on mapping for BIO-based mpath

2020-04-28 Thread Mike Snitzer
On Mon, Apr 27 2020 at 8:39pm -0400, Gabriel Krisman Bertazi wrote: > When adding devices that don't have a scsi_dh on a BIO based multipath, > I was able to consistently hit the warning below and lock-up the system. > > The problem is that __map_bio reads the flag before it potentially being >

Re: [dm-devel] [PATCH v2 2/3] md: multipath: Pass io_start_time to the path selector

2020-04-28 Thread Mike Snitzer
On Mon, Apr 27 2020 at 8:51pm -0400, Gabriel Krisman Bertazi wrote: > HST need to know the IO start time in order to predict path > performance. For request-based multipath use the block layer > io_start_time, while for BIO use the dm_io start_time. > > The dm_start_time_ns_from_clone function

Re: [dm-devel] [PATCH v2 1/3] md: multipath: Encapsulate parameters passed to selectors

2020-04-28 Thread Mike Snitzer
On Mon, Apr 27 2020 at 8:51pm -0400, Gabriel Krisman Bertazi wrote: > Different selector will use different parameters, which means .io_start > and .io_end will get their signatures modified to include more and more > parameters. This encapsulates the data in a structure so we can > simplify th

[dm-devel] [PATCH -next] md: dm-ebs-target: fix build errors & Kconfig entry

2020-04-28 Thread Randy Dunlap
at.com --- drivers/md/Kconfig | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) --- linux-next-20200428.orig/drivers/md/Kconfig +++ linux-next-20200428/drivers/md/Kconfig @@ -338,12 +338,12 @@ config DM_WRITECACHE to be cached in standard RAM. config DM_EB

Re: [dm-devel] [PATCH 13/13] dm-zoned: metadata version 2

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:09, Hannes Reinecke wrote: > Implement handling for metadata version 2. The new metadata adds > a label and UUID for the device mapper device, and additional UUID > for the underlying block devices. > It also allows for an additional regular drive to be used for > emulating random

[dm-devel] [PATCH v2 2/3] md: multipath: Pass io_start_time to the path selector

2020-04-28 Thread Gabriel Krisman Bertazi
HST need to know the IO start time in order to predict path performance. For request-based multipath use the block layer io_start_time, while for BIO use the dm_io start_time. The dm_start_time_ns_from_clone function was suggested and implemented by Mike Snitzer . Cc: Mike Snitzer Cc: Khazhismel

[dm-devel] [PATCH] dm: multipath: Use updated MPATHF_QUEUE_IO on mapping for BIO-based mpath

2020-04-28 Thread Gabriel Krisman Bertazi
When adding devices that don't have a scsi_dh on a BIO based multipath, I was able to consistently hit the warning below and lock-up the system. The problem is that __map_bio reads the flag before it potentially being modified by choose_pgpath, and ends up using the older value. The WARN_ON below

[dm-devel] [PATCH v2 3/3] md: Add Historical Service Time Path Selector

2020-04-28 Thread Gabriel Krisman Bertazi
From: Khazhismel Kumykov This new selector keeps an exponential moving average of the service time for each path (losely defined as delta between start_io and end_io), and uses this along with the number of inflight requests to estimate future service time for a path. Since we don't have a probe

[dm-devel] [PATCH v2 0/3] Historical Service Time Path Selector

2020-04-28 Thread Gabriel Krisman Bertazi
Hi Mike, Please find an updated version of HST integrating the change you requested to also support BIO based multipath. I hope you don't mind me folding the function you implemented into patch 2. If you prefer, I can integrate a patch you provide into the series. One interesting data point is

[dm-devel] [PATCH v2 1/3] md: multipath: Encapsulate parameters passed to selectors

2020-04-28 Thread Gabriel Krisman Bertazi
Different selector will use different parameters, which means .io_start and .io_end will get their signatures modified to include more and more parameters. This encapsulates the data in a structure so we can simplify the interface for future users. For now it only passes nr_bytes, but HST will re

Re: [dm-devel] [PATCH 09/13] dm-zoned: replace 'target' pointer in the bio context

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:08, Hannes Reinecke wrote: > Replace the 'target' pointer in the bio context with the > device pointer as this is what's actually used. > > Signed-off-by: Hannes Reinecke > Reviewed-by: Bob Liu > --- > drivers/md/dm-zoned-target.c | 26 -- > 1 file chang

Re: [dm-devel] [PATCH 08/13] dm-zoned: remove 'dev' argument from reclaim

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:09, Hannes Reinecke wrote: > Use the dmz_zone_to_dev() mapping function to remove the > 'dev' argument from reclaim. > > Signed-off-by: Hannes Reinecke > Reviewed-by: Bob Liu > --- > drivers/md/dm-zoned-reclaim.c | 58 > +++ > drivers/md

Re: [dm-devel] [PATCH 07/13] dm-zoned: Introduce dmz_dev_is_dying() and dmz_check_dev()

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:08, Hannes Reinecke wrote: > Introduce accessors dmz_dev_is_dying() and dmz_check_dev() to > avoid having to reference the devices directly. > > Signed-off-by: Hannes Reinecke > Reviewed-by: Bob Liu > --- > drivers/md/dm-zoned-metadata.c | 14 -- > drivers/md/dm-zon

Re: [dm-devel] [PATCH 02/13] dm-zoned: store zone id within the zone structure and kill dmz_id()

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:08, Hannes Reinecke wrote: > Instead of calculating the zone index by the offset within the > zone array store the index within the structure itself. With that > the helper dmz_id() is pointless and can be replaced with accessing > the ->id value directly. > > Signed-off-by: Hanne

Re: [dm-devel] [PATCH 01/13] dm-zoned: add 'status' and 'message' callbacks

2020-04-28 Thread Damien Le Moal
On 2020/04/20 19:09, Hannes Reinecke wrote: > Add callbacks to supply information for 'dmsetup status' > and 'dmsetup info', and implement the message 'reclaim' > to start the reclaim worker. > > Signed-off-by: Hannes Reinecke > Reviewed-by: Bob Liu > --- > drivers/md/dm-zoned-metadata.c | 15 +