Re: [dm-devel] [PATCH 03/12] dm-zoned: use on-stack superblock for tertiary devices

2020-05-26 Thread Hannes Reinecke
On 5/25/20 4:09 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: Checking the teriary superblock just consists of validating UUIDs, s/teriary/tertiary crcs, and the generation number; it doesn't have contents which would be required during the actual operation. So we shou

Re: [dm-devel] [PATCH 03/12] dm-zoned: use on-stack superblock for tertiary devices

2020-05-26 Thread Damien Le Moal
On 2020/05/26 17:25, Hannes Reinecke wrote: > On 5/25/20 4:09 AM, Damien Le Moal wrote: >> On 2020/05/23 0:39, Hannes Reinecke wrote: >>> Checking the teriary superblock just consists of validating UUIDs, >> >> s/teriary/tertiary >> >>> crcs, and the generation number; it doesn't have contents whic

Re: [dm-devel] [RFC PATCH v3 00/12] Integrity Policy Enforcement LSM (IPE)

2020-05-26 Thread Jaskaran Singh Khurana
Hello Mickael, Could you please share your thoughts for the below proposal. Regards, JK On Sat, 16 May 2020, Jaskaran Singh Khurana wrote: Hello Mickael, On Thu, 14 May 2020, Mickaël Salaün wrote: On 12/05/2020 22:46, Deven Bowers wrote: On 5/11/2020 11:03 AM, Deven Bowers wrote:

Re: [dm-devel] [PATCH 01/16] block: add disk/bio-based accounting helpers

2020-05-26 Thread Christoph Hellwig
On Mon, May 25, 2020 at 03:28:07PM +0300, Konstantin Khlebnikov wrote: > I think it would be better to leave this jiffies legacy nonsense in > callers and pass here request duration in nanoseconds. jiffies is what the existing interfaces uses. But now that they come from the start helper fixing t

[dm-devel] [PATCH 01/16] block: add disk/bio-based accounting helpers

2020-05-26 Thread Christoph Hellwig
Add two new helpers to simplify I/O accounting for bio based drivers. Currently these drivers use the generic_start_io_acct and generic_end_io_acct helpers which have very cumbersome calling conventions, don't actually return the time they started accounting, and try to deal with accounting for par

[dm-devel] [PATCH 12/16] block: account merge of two requests

2020-05-26 Thread Christoph Hellwig
From: Konstantin Khlebnikov Also rename blk_account_io_merge() into blk_account_io_merge_request() to distinguish it from merging request and bio. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- block/blk-merge.c | 12 +--- 1 file changed, 5 in

[dm-devel] [PATCH 08/16] zram: nvdimm: use bio_{start, end}_io_acct and disk_{start, end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch zram to use the nicer bio accounting helpers, and as part of that ensure each bio is counted as a single I/O request. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/zram/zram_drv.c | 24 ++-- 1 file changed, 10 insertions(+), 14

[dm-devel] [PATCH 02/16] drbd: use bio_{start,end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch drbd to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/drbd/drbd_req.c | 27 --- 1 file changed, 4 insertions(+), 23 deletions(-) diff --git a/drivers/block/drbd/drbd_req.c b/drivers/blo

[dm-devel] [PATCH 04/16] lightnvm/pblk: use bio_{start, end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch rsxx to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/lightnvm/pblk-cache.c | 8 +++- drivers/lightnvm/pblk-read.c | 11 --- 2 files changed, 7 insertions(+), 12 deletions(-) diff --git a/drivers/light

[dm-devel] [PATCH 15/16] block: use __this_cpu_add() instead of access by smp_processor_id()

2020-05-26 Thread Christoph Hellwig
From: Konstantin Khlebnikov Most architectures have fast path to access percpu for current cpu. The required preempt_disable() is provided by part_stat_lock(). Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- include/linux/part_stat.h | 2 +- 1 file cha

[dm-devel] block I/O accounting improvements v2

2020-05-26 Thread Christoph Hellwig
Hi Jens, they series contains various improvement for block I/O accounting. The first bunch of patches switch the bio based drivers to better accounting helpers compared to the current mess. The end contains a fix and various performanc improvements. Most of this comes from a series Konstantin

[dm-devel] [PATCH 09/16] block: remove generic_{start, end}_io_acct

2020-05-26 Thread Christoph Hellwig
Remove these now unused functions. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/bio.c | 39 --- include/linux/bio.h | 6 -- 2 files changed, 45 deletions(-) diff --git a/block/bio.c b/block/bio.c index 9c101a0572

[dm-devel] [PATCH 07/16] nvdimm: use bio_{start,end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/nvdimm/blk.c | 6 -- drivers/nvdimm/btt.c | 6 -- drivers/nvdimm/nd.h | 19 --- drivers/nvdimm/pmem.c | 6 -- 4 files changed, 12

[dm-devel] [PATCH 06/16] dm: use bio_{start,end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/md/dm.c | 9 ++--- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index f215b86664484..3f39fa1ac756e 100644 ---

[dm-devel] [PATCH 10/16] block: move update_io_ticks to blk-core.c

2020-05-26 Thread Christoph Hellwig
All callers are in blk-core.c, so move update_io_ticks over. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/bio.c | 16 block/blk-core.c | 15 +++ block/blk.h | 1 - 3 files changed, 15 insertions(+), 17 deletions(-) diff -

[dm-devel] [PATCH 16/16] block: reduce part_stat_lock() scope

2020-05-26 Thread Christoph Hellwig
We only need the stats lock (aka preempt_disable()) for updating the states, not for looking up or dropping the hd_struct reference. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/blk-core.c | 5 +++-- block/blk-merge.c | 3 ++- 2 files changed, 5 insertions(+),

[dm-devel] [PATCH 05/16] bcache: use bio_{start,end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch bcache to use the nicer bio accounting helpers, and call the routines where we also sample the start time to give coherent accounting results. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov Acked-by: Coly Li --- drivers/md/bcache/request.c | 18 -- 1

[dm-devel] [PATCH 03/16] rsxx: use bio_{start,end}_io_acct

2020-05-26 Thread Christoph Hellwig
Switch rsxx to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/rsxx/dev.c | 19 ++- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/drivers/block/rsxx/dev.c b/drivers/block/rsxx/dev.c inde

[dm-devel] [PATCH 13/16] block: add a blk_account_io_merge_bio helper

2020-05-26 Thread Christoph Hellwig
From: Konstantin Khlebnikov Move the non-"new_io" branch of blk_account_io_start() into separate function. Fix merge accounting for discards (they were counted as write merges). The new blk_account_io_merge_bio() doesn't call update_io_ticks() unlike blk_account_io_start(), as there is no reaso

[dm-devel] [PATCH 14/16] block: remove rcu_read_lock() from part_stat_lock()

2020-05-26 Thread Christoph Hellwig
From: Konstantin Khlebnikov The RCU lock is required only in disk_map_sector_rcu() to lookup the partition. After that request holds reference to related hd_struct. Replace get_cpu() with preempt_disable() - returned cpu index is unused. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Sig

[dm-devel] [PATCH 11/16] block: always use a percpu variable for disk stats

2020-05-26 Thread Christoph Hellwig
percpu variables have a perfectly fine working stub implementation for UP kernels, so use that. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/blk.h | 2 +- block/genhd.c | 12 +++-- block/partitions/core.c | 5 ++-- include/linux

[dm-devel] [PATCH 04/15] dm-zoned: add a 'reserved' zone flag

2020-05-26 Thread Hannes Reinecke
Instead of counting the number of reserved zones in dmz_free_zone() we should mark the zone as 'reserved' during allocation and simplify dmz_free_zone(). Signed-off-by: Hannes Reinecke Reviewed-by: Damien Le Moal --- drivers/md/dm-zoned-metadata.c | 4 ++-- drivers/md/dm-zoned.h | 2 ++

[dm-devel] [PATCHv2 00/15] dm-zoned: multi-device support

2020-05-26 Thread Hannes Reinecke
Hi all, here's the second version of my patchset to support multiple zoned drives with dm-zoned. This patchset: - Converts the zone array to using xarray for better scalability - Separates out shared structures into per-device structure - Enforce drive-locality for allocating and reclaiming zones

[dm-devel] [PATCH 10/15] dm-zoned: per-device reclaim

2020-05-26 Thread Hannes Reinecke
Instead of having one reclaim workqueue for the entire set we should be allocating a reclaim workqueue per device; that will reduce contention and should boost performance for a multi-device setup. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-reclaim.c | 66 +++-

[dm-devel] [PATCH 09/15] dm-zoned: allocate dm devices dynamically

2020-05-26 Thread Hannes Reinecke
Allocate dm devices dynamically to allow for expansion to several devices. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-target.c | 29 +++-- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/md/dm-zoned-target.c b/drivers/md/dm-zoned-target

[dm-devel] [PATCH 14/15] dm-zoned: select reclaim zone based on device index

2020-05-26 Thread Hannes Reinecke
per-device reclaim should select zones on that device only. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 50 +- drivers/md/dm-zoned-reclaim.c | 3 ++- drivers/md/dm-zoned-target.c | 1 + drivers/md/dm-zoned.h | 5 -

[dm-devel] [PATCH 01/15] dm-zoned: add debugging message for reading superblocks

2020-05-26 Thread Hannes Reinecke
Signed-off-by: Hannes Reinecke Reviewed-by: Damien Le Moal --- drivers/md/dm-zoned-metadata.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/md/dm-zoned-metadata.c b/drivers/md/dm-zoned-metadata.c index 4a2e351365c5..ef1524d5928a 100644 --- a/drivers/md/dm-zoned-metadata.c +++ b

[dm-devel] [PATCH 05/15] dm-zoned: convert to xarray

2020-05-26 Thread Hannes Reinecke
The zones array is getting really large, and large arrays tend to wreak havoc with the CPU caches. So convert it to xarray to become more cache friendly. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 120 ++--- 1 file changed, 88 insertio

[dm-devel] [PATCH 03/15] dm-zoned: improve logging messages for reclaim

2020-05-26 Thread Hannes Reinecke
Instead of just reporting the errno this patch adds some more verbose debugging message in the reclaim path. Signed-off-by: Hannes Reinecke Reviewed-by: Damien Le Moal --- drivers/md/dm-zoned-reclaim.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/md

[dm-devel] [PATCH 12/15] dm-zoned: support arbitrary number of devices

2020-05-26 Thread Hannes Reinecke
Remove the hard-coded limit of two devices and support an unlimited number of additional zoned devices. With that we need to increase the device-mapper version number to 3.0.0 as we've modified the interface. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 15 +++- driver

[dm-devel] [PATCH 06/15] dm-zoned: temporary superblock for tertiary devices

2020-05-26 Thread Hannes Reinecke
Checking the tertiary superblock just consists of validating UUIDs, crcs, and the generation number; it doesn't have contents which would be required during the actual operation. So we should be allocating a temporary superblock when checking tertiary devices and avoid having to store it together w

[dm-devel] [PATCH 08/15] dm-zoned: add metadata pointer to struct dmz_dev

2020-05-26 Thread Hannes Reinecke
Add a metadata pointer to struct dmz_dev and use it as argument for blkdev_report_zones() instead of the metadata itself. Signed-off-by: Hannes Reinecke Reviewed-by: Damien Le Moal --- drivers/md/dm-zoned-metadata.c | 14 +- drivers/md/dm-zoned.h | 7 --- 2 files chang

[dm-devel] [PATCH 13/15] dm-zoned: allocate zone by device index

2020-05-26 Thread Hannes Reinecke
When allocating a zone we should pass in an indicator on which device the zone should be allocated; this increases performance for a multi-device setup as then reclaim can allocate zones on the device for which reclaim is running. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c

[dm-devel] [PATCH 02/15] dm-zoned: secondary superblock must reside on the same devices than primary superblock

2020-05-26 Thread Hannes Reinecke
The secondary superblock must reside on the same device than the primary superblock, so there's no need to re-calculate the device. Signed-off-by: Hannes Reinecke Reviewed-by: Damien Le Moal --- drivers/md/dm-zoned-metadata.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --gi

[dm-devel] [PATCH 15/15] dm-zoned: prefer full zones for reclaim

2020-05-26 Thread Hannes Reinecke
Prefer full zones when selecting the next zone for reclaim. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm-zoned-metadata.c b/drivers/md/dm-zoned-metadata.c index b89b3d3b9ec9..f161ef4e3d

[dm-devel] [PATCH 11/15] dm-zoned: move random and sequential zones into struct dmz_dev

2020-05-26 Thread Hannes Reinecke
Random and sequential zones should be part of the respective device structure to make arbitration between devices possible. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 139 +++-- drivers/md/dm-zoned-reclaim.c | 15 +++-- drivers/md/dm

[dm-devel] [PATCH 07/15] dm-zoned: add device pointer to struct dm_zone

2020-05-26 Thread Hannes Reinecke
Add a pointer to the containing device to struct dm_zone and kill dmz_zone_to_dev(). Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 39 ++- drivers/md/dm-zoned-reclaim.c | 13 + drivers/md/dm-zoned-target.c | 2 +- drivers/