Re: [dm-devel] [PATCH] udev-md-raid-assembly.rules: skip if DM_UDEV_DISABLE_OTHER_RULES_FLAG

2022-02-16 Thread NeilBrown
> flag does not necessarily imply SYSTEMD_READY=0 and must therefore > be tested separately. I really don't like this - looks like a hack. A Kludge. Can you provide a reference to a detailed discussion that explains why SYSTEMD_READY=0 cannot be used? Thanks, NeilBrown >

Re: [dm-devel] [PATCH] udev-md-raid-assembly.rules: skip if DM_UDEV_DISABLE_OTHER_RULES_FLAG

2022-02-21 Thread NeilBrown
ts own devices on top of DM > ones. If this is an established API that DM uses, then presumably it is documented somewhere. If a link to that documentation were provided, it would look a a whole lot less like a hack. Thanks, NeilBrown -- dm-devel mailing list dm-devel@redhat.com https://listman.

Re: [dm-devel] [PATCH] udev-md-raid-assembly.rules: skip if DM_UDEV_DISABLE_OTHER_RULES_FLAG

2022-02-22 Thread NeilBrown
On Wed, 23 Feb 2022, Martin Wilck wrote: > Neil, > > On Tue, 2022-02-22 at 10:36 +1100, NeilBrown wrote: > > > > > The flags that DM use for udev were introduced before systemd > > > project > > > even existed. We needed to introduce the > > &g

Re: [dm-devel] [PATCH] md: fix a crash in mempool_free

2022-11-03 Thread NeilBrown
if (atomic_dec_and_test(&mddev->pending_writes)) > wake_up(&mddev->sb_wait); > rdev_dec_pending(rdev, mddev); > - bio_put(bio); > } Thanks. I think this is a clear improvement. I think it would be a little better if the rdev_dec_pending were also move up. Then both code fragments would be: bio_put ; rdev_dec_pending ; atomic_dec_and_test Thanks, NeilBrown -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH] md: fix a crash in mempool_free

2022-11-05 Thread NeilBrown
On Sat, 05 Nov 2022, Mikulas Patocka wrote: > > On Fri, 4 Nov 2022, NeilBrown wrote: > > > > --- > > > drivers/md/md.c |6 -- > > > 1 file changed, 4 insertions(+), 2 deletions(-) > >

Re: [dm-devel] [PATCH v2 1/1] block: fix blk_queue_split() resource exhaustion

2017-01-03 Thread NeilBrown
> Thanks for the bump. >> >> Lars >> > Hi folks, > > As Michael mentioned, we hit a bug this patch is trying to fix. > Neil suggested another way to fix it. I attached below. > I personal prefer Neil's version as it's less code change, and str

Re: [dm-devel] [PATCH v2 1/1] block: fix blk_queue_split() resource exhaustion

2017-01-06 Thread NeilBrown
On Sat, Jan 07 2017, Mike Snitzer wrote: > On Fri, Jan 06 2017 at 12:34pm -0500, > Mikulas Patocka wrote: > >> >> >> On Fri, 6 Jan 2017, Mikulas Patocka wrote: >> >> > >> > >> > On Wed, 4 Jan 2017, Mike Snitzer wrote: >>

[dm-devel] [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread NeilBrown
() recovery thread for many block devices, and eventually it may be possible to remove it completely. Ref: http://www.spinics.net/lists/raid/msg54680.html Tested-by: Jinpu Wang Inspired-by: Lars Ellenberg Signed-off-by: NeilBrown --- block/blk-core.c | 25 + 1 file changed

Re: [dm-devel] [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
it from some easy-to-fix places. The series summary is below. NeilBrown NeilBrown (5): blk: improve order of bio handling in generic_make_request() blk: remove bio_set arg from blk_queue_split() blk: make the bioset rescue_workqueue optional. blk: use non-rescuing bios

[dm-devel] [PATCH 1/5 v3] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
ray of 2 lists, and relevant code examines both of them. Ref: http://www.spinics.net/lists/raid/msg54680.html Tested-by: Jinpu Wang Inspired-by: Lars Ellenberg Signed-off-by: NeilBrown --- block/bio.c | 11 --- block/blk-core.c|

[dm-devel] [PATCH 2/5] blk: remove bio_set arg from blk_queue_split()

2017-03-09 Thread NeilBrown
ys use q->bio_split inside blk_queue_split() Signed-off-by: NeilBrown --- block/blk-core.c |2 +- block/blk-merge.c |7 +++ block/blk-mq.c|4 ++-- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/block/ps3vr

[dm-devel] [PATCH 3/5] blk: make the bioset rescue_workqueue optional.

2017-03-09 Thread NeilBrown
bioset_create_nobvec() are converted to the _rescued() version, so that not change in behaviour is experienced. It is hoped that most, if not all, bioset can end up being the non-rescued version. Signed-off-by: NeilBrown --- block/bio.c | 30

[dm-devel] [PATCH 4/5] blk: use non-rescuing bioset for q->bio_split.

2017-03-09 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be and other bios allocated fro q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown --- block/blk-core.c |2 +- 1 file changed, 1 insertion(+), 1 delet

[dm-devel] [PATCH 5/5] block_dev: make blkdev_dio_pool a non-rescuing bioset

2017-03-09 Thread NeilBrown
Allocations from blkdev_dio_pool are never made under generic_make_request, so this bioset does not need a rescuer thread. Signed-off-by: NeilBrown --- fs/block_dev.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index c0ca5f0d0369

Re: [dm-devel] [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
On Thu, Mar 09 2017, Jens Axboe wrote: > On 03/09/2017 09:32 PM, NeilBrown wrote: >> >> I started looking further at the improvements we can make once >> generic_make_request is fixed, and realised that I had missed an >> important detail in this patch. >> Seve

[dm-devel] [PATCH 5/5] block_dev: make blkdev_dio_pool a non-rescuing bioset

2017-03-09 Thread NeilBrown
Allocations from blkdev_dio_pool are never made under generic_make_request, so this bioset does not need a rescuer thread. Signed-off-by: NeilBrown --- fs/block_dev.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index c0ca5f0d0369

[dm-devel] [PATCH 0/5] Updates following recent generic_make_request improvement

2017-03-09 Thread NeilBrown
possible that the changes to btrfs and xfs can just be dropped as a subsequent patch will be needed to revert them anyway. They are there only to be able to say that "blk: make the bioset rescue_workqueue optional." doesn't change any functionality at all. Thanks, NeilBrown --

[dm-devel] [PATCH 4/5] blk: use non-rescuing bioset for q->bio_split.

2017-03-09 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown --- block/blk-core.c |2 +- 1 file changed, 1 insertion(+), 1 delet

[dm-devel] [PATCH 1/5] blk: Ensure users for current->bio_list can see the full list.

2017-03-09 Thread NeilBrown
ueue selected bios, and others that check if the list is empty. These are no longer correct. So redefine current->bio_list to point to an array of two lists, which contain all queued bios, and adjust various code to test or walk both lists. Signed-off-by: NeilBrown --- block/bio.

[dm-devel] [PATCH 2/5] blk: remove bio_set arg from blk_queue_split()

2017-03-09 Thread NeilBrown
ys use q->bio_split inside blk_queue_split() Signed-off-by: NeilBrown --- block/blk-core.c |2 +- block/blk-merge.c |7 +++ block/blk-mq.c|4 ++-- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/block/ps3vr

[dm-devel] [PATCH 3/5] blk: make the bioset rescue_workqueue optional.

2017-03-09 Thread NeilBrown
bioset_create_nobvec() are converted to the _rescued() version, so that no change in behaviour is experienced. It is hoped that most, if not all, biosets can end up being the non-rescued version. Signed-off-by: NeilBrown --- block/bio.c | 30

Re: [dm-devel] [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread NeilBrown
nk though the consequences again and I'm now more confident. I actually now think that change probably wasn't necessary. It is safer though. It ensures that current functionality isn't removed without a clear justification. Thanks, NeilBrown signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH 0/5] Updates following recent generic_make_request improvement

2017-03-12 Thread NeilBrown
On Sat, Mar 11 2017, Jens Axboe wrote: > On 03/09/2017 11:00 PM, NeilBrown wrote: >> This is a rebase of the series I sent earlier, based on the >> very latest from Linus, which included my first patch. >> >> The first fixes a problem that patch introduced, and so sho

[dm-devel] [PATCH] block: trace completion of all bios.

2017-03-21 Thread NeilBrown
trace_block_bio_complete() pairs with trace_block_bio_queue(). Any bio for which a 'queue' event is traced, will subsequently generate a 'complete' event. Signed-off-by: NeilBrown --- block/bio.c| 3 +++ drivers/md/dm.c| 1 - drivers/md/raid5.c | 8 3 files changed

Re: [dm-devel] [PATCH] block: trace completion of all bios.

2017-03-22 Thread NeilBrown
On Wed, Mar 22 2017, Christoph Hellwig wrote: > On Wed, Mar 22, 2017 at 01:38:09PM +1100, NeilBrown wrote: >> >> Currently only dm and md/raid5 bios trigger trace_block_bio_complete(). >> Now that we have bio_chain(), it is not possible, in general, for a >> driver to

[dm-devel] [PATCH v2] block: trace completion of all bios.

2017-03-22 Thread NeilBrown
that. To handle these, we provide bio_endio_notrace(). This patch only adds uses of this in core code. Separate patches will be needed to update the filesystems to avoid tracing. Signed-off-by: NeilBrown --- block/bio-integrity.c | 4 ++-- block/bio.c | 46 ++

[dm-devel] [PATCH v3] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
ce completion events, each for their own range. So if generic_make_request() is called (which generates a QUEUED event), then bi_endio() will create a single COMPLETE event for each range that the bio is split into, unless the driver has explicitly requested it not to. Signed-off-by: Nei

Re: [dm-devel] [PATCH v2] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
On Thu, Mar 23 2017, Ming Lei wrote: > On Thu, Mar 23, 2017 at 05:29:02PM +1100, NeilBrown wrote: >> >> /** >> + * bio_endio_notrace - end I/O on a bio without tracing >> + * @bio:bio >> + * >> + * Description: >> + * bio_

Re: [dm-devel] [PATCH v3] block: trace completion of all bios.

2017-03-26 Thread NeilBrown
On Fri, Mar 24 2017, Ming Lei wrote: > On Fri, Mar 24, 2017 at 8:07 AM, NeilBrown wrote: ... >> @@ -102,6 +102,8 @@ struct bio { >> #define BIO_REFFED 8 /* bio has elevated ->bi_cnt */ >> #define BIO_THROTTLED 9 /* This bio has a

Re: [dm-devel] [PATCH v3] block: trace completion of all bios.

2017-03-27 Thread NeilBrown
when when blk_queue_split() (and similar code) calls it. Thanks, NeilBrown signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH v3] block: trace completion of all bios.

2017-03-27 Thread NeilBrown
On Mon, Mar 27 2017, Christoph Hellwig wrote: > On Mon, Mar 27, 2017 at 08:49:57PM +1100, NeilBrown wrote: >> On Mon, Mar 27 2017, Christoph Hellwig wrote: >> >> > I don't really like the flag at all. I'd much prefer a __bio_endio >> > with a '

[dm-devel] [PATCH 0/2] Trace completion of all bios

2017-04-06 Thread NeilBrown
Hi Jens, I think all objections to this patch have been answered so I'm resending. I've added a small cleanup patch first which makes some small enhancements to the documentation and #defines for the bio->flags field. Thanks, NeilBrown --- NeilBrown (2): block: simple

[dm-devel] [PATCH 2/2] block: trace completion of all bios.

2017-04-06 Thread NeilBrown
ce completion events, each for their own range. So if generic_make_request() is called (which generates a QUEUED event), then bi_endio() will create a single COMPLETE event for each range that the bio is split into, unless the driver has explicitly requested it not to. Signed-off-by: NeilB

[dm-devel] [PATCH 1/2] block: simple improvements for bio->flags

2017-04-06 Thread NeilBrown
iss. BVEC_POOL_BITS is larger than needed. The BVEC_POOL_IDX() ranges from 0 to 6, so 3 bits are sufficient. This patch make improvements in each of these areas. Signed-off-by: NeilBrown --- include/linux/blk_types.h | 22 +- 1 file changed, 13 insertions(+), 9 delet

[dm-devel] [PATCH] dm-region-hash: fix strange usage of mempool_alloc.

2017-04-09 Thread NeilBrown
mempool_alloc, and allow it to sleep. If no memory is available it will wait for something to be returned to the pool, and will retry a normal allocation regularly. Signed-off-by: NeilBrown --- drivers/md/dm-region-hash.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a

[dm-devel] [PATCH] dm-verity-fec: don't check for failure from mempool_alloc(GFP_NOIO)

2017-04-09 Thread NeilBrown
xtra_pool and ->prealloc_pool are repeated before releasing the previous allocation. This can deadlock if the code is servicing a write under high memory pressure. To avoid deadlocks, change these to use GFP_NOWAIT and leave the error handling in place. Signed-off-by: NeilBrown --- drivers/md/dm-v

Re: [dm-devel] [PATCH] dm-region-hash: fix strange usage of mempool_alloc.

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Mikulas Patocka wrote: > On Mon, 10 Apr 2017, NeilBrown wrote: > >> mempool_alloc() should only be called with GFP_ATOMIC when >> it is not safe to wait. Passing __GFP_NOFAIL to kmalloc() >> says that it is safe to wait indefinitely. So this

Re: [dm-devel] Autoload of DM kernel targets?

2017-04-23 Thread NeilBrown
MODULE_PARM_DESC(no_space_timeout, "Out of data space queue IO timeout in second MODULE_DESCRIPTION(DM_NAME " thin provisioning target"); MODULE_AUTHOR("Joe Thornber "); MODULE_LICENSE("GPL"); +MODULE_ALIAS("dm-thin-pool"); NeilBrown signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [mdadm PATCH 4/4] Create: tell udev device is not ready when first created.

2017-04-27 Thread NeilBrown
On Wed, Apr 26 2017, Peter Rajnoha wrote: > On 04/20/2017 11:35 PM, NeilBrown wrote: >> On Thu, Apr 20 2017, Jes Sorensen wrote: > ... >>> Second, isn't this going to be racey if you have multiple arrays >>> running? I am wondering if we cannot find a solutio

[dm-devel] [mdadm PATCH] Create: tell udev md device is not ready when first created.

2017-04-27 Thread NeilBrown
Y}="0". This is fairly uniformly used to suppress actions based on the contents of the device. Signed-off-by: NeilBrown --- Assemble.c | 2 +- Build.c | 2 +- Create.c| 9 +++- Incremental.c | 4

Re: [dm-devel] [mdadm PATCH 4/4] Create: tell udev device is not ready when first created.

2017-04-30 Thread NeilBrown
On Fri, Apr 28 2017, Peter Rajnoha wrote: > On 04/28/2017 05:55 AM, NeilBrown wrote: >> On Wed, Apr 26 2017, Peter Rajnoha wrote: >> >>> On 04/20/2017 11:35 PM, NeilBrown wrote: >>>> If we wanted an more permanent udev rule, we would need to record the >&g

Re: [dm-devel] [mdadm PATCH 4/4] Create: tell udev device is not ready when first created.

2017-05-02 Thread NeilBrown
simple" any day. NeilBrown On Tue, May 02 2017, Peter Rajnoha wrote: > On 05/01/2017 06:35 AM, NeilBrown wrote: >> On Fri, Apr 28 2017, Peter Rajnoha wrote: >> >>> On 04/28/2017 05:55 AM, NeilBrown wrote: >>>> On Wed, Apr 26 2017, Peter Rajnoha wrote: >>

Re: [dm-devel] [PATCH] dm-region-hash: fix strange usage of mempool_alloc.

2017-05-03 Thread NeilBrown
On Wed, May 03 2017, Mikulas Patocka wrote: > On Mon, 24 Apr 2017, NeilBrown wrote: >> >> I had a look at how the allocation 'dm_region' objects are used, >> and it would take a bit of work to make it really safe. >> My guess is __rh_find() should be allowe

[dm-devel] [mdadm PATCH] Fix typo in new udev rule.

2017-05-04 Thread NeilBrown
As pointed out by Peter Rajnoha, the correct usage in udev is TEST=="file", not TEST="file". Also improve a related comment which was a bit informal. Reported-by: Peter Rajnoha Fixes: cd6cbb08c458 ("Create: tell udev md device is not ready when first created.&q

[dm-devel] [PATCH] md: initialise ->writes_pending in personality modules.

2017-06-06 Thread NeilBrown
arily initialized (requiring memory allocation) when the personality doesn't use writes_pending. Reported-by: Heinz Mauelshagen Fixes: 4ad23a976413 ("MD: use per-cpu counter for writes_pending") Signed-off-by: NeilBrown --- drivers/md/md.c | 16 drivers/md/md.h

[dm-devel] [PATCH 2/2] dm-io: remove BIOSET_NEED_RESCUER flag from bios bioset

2017-08-29 Thread NeilBrown
is not needed. Signed-off-by: NeilBrown --- drivers/md/dm-io.c |3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index 25039607f3cb..bafd9f79aa3b 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -58,8 +58,7 @@ struct

[dm-devel] [PATCH 0/2] Remove BIOSET_NEED_RESCUER from some DM biosets

2017-08-29 Thread NeilBrown
eeded there before the flag can be removed. Thanks, NeilBrown --- NeilBrown (2): dm-crypt: remove BIOSET_NEED_RESCUER flag. dm-io: remove BIOSET_NEED_RESCUER flag from bios bioset drivers/md/dm-crypt.c |3 +-- drivers/md/dm-io.c|3 +-- 2 files changed, 2 insertions(

[dm-devel] [PATCH 1/2] dm-crypt: remove BIOSET_NEED_RESCUER flag.

2017-08-29 Thread NeilBrown
will not block. So BIOSET_NEED_RESCUER is not needed. Signed-off-by: NeilBrown --- drivers/md/dm-crypt.c |3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index cdf6b1e12460..e7dc1a6b5a9d 100644 --- a/drivers/md/dm-crypt.c

[dm-devel] [PATCH/rfc] dm: revise 'rescue' strategy for md->bs allocations

2017-08-30 Thread NeilBrown
*all* bios on the list as there is nothing to be gained by being more selective. This allows us to remove BIOSET_NEED_RESCUER when allocating this bioset. It also keeps all the mechanics of rescuing inside dm, so dm can be in full control. Signed-off-by: NeilBrown --- Hi, I have only tested t

Re: [dm-devel] [PATCH/rfc] dm: revise 'rescue' strategy for md->bs allocations

2017-08-31 Thread NeilBrown
k the deadlock would be reintroduced. Can you see something that I am missing? Thanks, NeilBrown signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH/rfc] dm: revise 'rescue' strategy for md->bs allocations

2017-09-03 Thread NeilBrown
On Fri, Sep 01 2017, Mikulas Patocka wrote: > On Fri, 1 Sep 2017, NeilBrown wrote: > >> On Thu, Aug 31 2017, Mikulas Patocka wrote: >> >> >> >> >> Note that only current->bio_list[0] is offloaded. current->bio_list[1] >> >> contains b

Re: [dm-devel] [PATCH/rfc] dm: revise 'rescue' strategy for md->bs allocations

2017-09-05 Thread NeilBrown
On Mon, Sep 04 2017, Mikulas Patocka wrote: > On Mon, 4 Sep 2017, NeilBrown wrote: > >> On Fri, Sep 01 2017, Mikulas Patocka wrote: >> >> > On Fri, 1 Sep 2017, NeilBrown wrote: >> > >> >> On Thu, Aug 31 2017, Mikulas Patocka wrote: >> >>

Re: [dm-devel] [PATCH/rfc] dm: revise 'rescue' strategy for md->bs allocations

2017-09-06 Thread NeilBrown
On Wed, Sep 06 2017, Mikulas Patocka wrote: > On Wed, 6 Sep 2017, NeilBrown wrote: > >> Thanks for the extra explanation. I've thought some more about this I can >> see a way forward that I am comfortable with. Please let me know what >> you think. >> I h

[dm-devel] [PATCH] dm: fix to_sector() for 32bit

2019-01-07 Thread NeilBrown
more correct. Kernels as early as 4.2 can have other problems due to to_sector() being used on the size of a device. Fixes: 0cf4503174c1 ("dm raid: add support for the MD RAID0 personality") cc: sta...@vger.kernel.org (v4.2+) Reported-and-tested-by: Guillaume Perréal Signed-off-by: N

Re: [dm-devel] [PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-20 Thread NeilBrown
t the flag and do the trace in bio_split(). Do you see any harm with doing it that way (in the next merge-window, I don't suggest you change this patch). Thanks, NeilBrown > -- > 2.15.0 > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [PATCH] md/raid0: Fail BIOs if their underlying block device is gone

2019-07-29 Thread NeilBrown
ter a machine reboot, e2fsck shows some severe fs errors > that demand fixing. This patch was tested in both ext4 and xfs > filesystems. > > Cc: NeilBrown > Cc: Song Liu > Signed-off-by: Guilherme G. Piccoli > --- > > After an attempt to change the way md/raid0 fails

Re: [dm-devel] [PATCH 1/2] md/raid0: Introduce new array state 'broken' for raid0

2019-07-29 Thread NeilBrown
eing useful only to distinguish if such array has some member > missing. Also, we show a rate-limited warning in kernel log in such case. > > This patch has no proper functional change other than adding a 'clean'-like > state; it was tested with ext4 and xfs filesystems. It req

Re: [dm-devel] [PATCH 1/2] md/raid0: Introduce new array state 'broken' for raid0

2019-07-30 Thread NeilBrown
On Tue, Jul 30 2019, Bob Liu wrote: > > > Curious why only raid0 has this issue? Actually, it isn't only raid0. 'linear' has the same issue. Probably the fix for raid0 should be applied to linear too. NeilBrown signature.asc Description: PGP signature -- dm-dev

Re: [dm-devel] [PATCH v3 1/2] md raid0/linear: Mark array as 'broken' and fail BIOs if a member is gone

2019-08-30 Thread NeilBrown
er some I/O errors, ext4 for instance aborts the journal > and prevents corruption. Without this change, we're able to keep writing > in the disk and after a machine reboot, e2fsck shows some severe fs errors > that demand fixing. This patch was tested in ext4 and xfs filesystems, an

Re: [PATCH v3 2/2] mdadm: Introduce new array state 'broken' for raid0/linear

2019-08-30 Thread NeilBrown
t > doesn't require the 'broken' state in order to work. > Also, this patch changes the way the array state is showed in the 'detail' > command (for raid0/linear only) - now it takes the 'array_state' sysfs > attribute into account instead of only r

Re: [dm-devel] WANTED new maintainer for Linux/md (and mdadm)

2016-01-26 Thread NeilBrown
tches, but several can collaborate at development and support and bug fixing and testing and I suggest you find ways to co-ordinate with them. This: https://bugzilla.kernel.org/show_bug.cgi?id=108741 is currently most important in my mind, but there are other things to do. Thanks

Re: [dm-devel] pvmove/mirror target mirroring data between two devices with different sector size

2016-03-30 Thread NeilBrown
t you could easily answer: 1/ what filesystem. What error when mounting? 2/ Is the data exactly the same after the move. e.g does sh1sum of the old and new device report the same content? If not, what about sha1sum of the first 10Gig? The first Gig? 100K? 16K? 1K? ... i.e. if there is a di

Re: [dm-devel] [RFC] block: fix blk_queue_split() resource exhaustion

2016-07-06 Thread NeilBrown
;requests for devices below the current level". I also really *don't* like the idea of punting to a separate thread - it seems to be just delaying the problem. Can you try move the bio_list_init(->recursion) call to just before the ->make_request_fn() call, and adding bio_list_merge_head(->remainder, ->recursion) just after? (or something like that) and confirm it makes sense, and works? Thanks! NeilBrown signature.asc Description: PGP signature -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [RFC] block: fix blk_queue_split() resource exhaustion

2016-07-07 Thread NeilBrown
On Thu, Jul 07 2016, Lars Ellenberg wrote: > On Thu, Jul 07, 2016 at 03:35:48PM +1000, NeilBrown wrote: >> On Wed, Jun 22 2016, Lars Ellenberg wrote: >> >> > For a long time, generic_make_request() converts recursion into >> > iteration by queuing recursi

Re: [dm-devel] block: fix blk_queue_split() resource exhaustion

2016-07-07 Thread NeilBrown
On Thu, Jul 07 2016, Mike Snitzer wrote: > On Thu, Jul 07 2016 at 1:35am -0400, > NeilBrown wrote: > >> On Wed, Jun 22 2016, Lars Ellenberg wrote: >> >> > For a long time, generic_make_request() converts recursion into >> > iteration by queuing r

Re: [dm-devel] [RFC] block: fix blk_queue_split() resource exhaustion

2016-07-08 Thread NeilBrown
On Fri, Jul 08 2016, Lars Ellenberg wrote: > On Fri, Jul 08, 2016 at 08:07:52AM +1000, NeilBrown wrote: >> Before I introduced the recursion limiting, requests were handled as an >> in-order tree walk. The code I wrote tried to preserve that but didn't >> for several re

Re: [dm-devel] [PATCH v2 1/1] block: fix blk_queue_split() resource exhaustion

2016-07-11 Thread NeilBrown
" list, we can make the recursion-to-iteration > logic in generic_make_request() process deepest level bios first, > and "sibling" bios of the same level in "natural" order. > > Signed-off-by: Lars Ellenberg > Signed-off-by: Roland Kammerer Reviewed-by: NeilBrown T

Re: [dm-devel] [RFC PATCH 1/2] mempool: do not consume memory reserves from the reclaim path

2016-07-21 Thread NeilBrown
> - /* If oom killed, memory reserves are essential to prevent livelock */ > - VM_WARN_ON_ONCE(gfp_mask & __GFP_NOMEMALLOC); > - /* No element size to zero on allocation */ > VM_WARN_ON_ONCE(gfp_mask & __GFP_ZERO); > - > might_sleep_if(gfp_mask

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-22 Thread NeilBrown
pted version in the page cache. But if dm-crypt is just allocating a transient page (which I think it is), then a mempool should be sufficient (and we should make sure it is sufficient) and access to an extra 10% (or whatever) of the page cache isn't justified. Thanks, NeilBrown If there

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-22 Thread NeilBrown
On Fri, Jul 22 2016, NeilBrown wrote: > > Looking at the current code, __GFP_DIRECT_RECLAIM is disabled the first > time through, but if the pool is empty, direct-reclaim is allowed on the > next attempt. Presumably this is where the throttling comes in ?? I > suspect that it r

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-22 Thread NeilBrown
ould a PF_MEMPOOL be any better? Why a PF rather than a GFP flag? NFSD uses a PF because there is no GFP interface for filesystem write. But mempool can pass down a GFP flag, so I think it should. The meaning of the flag is, in my opinion, that a 'transient' allocation is being requested

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-26 Thread NeilBrown
On Mon, Jul 25 2016, Michal Hocko wrote: > On Sat 23-07-16 10:12:24, NeilBrown wrote: >> Maybe that is impractical, but having firm rules like that would go a >> long way to make it possible to actually understand and reason about how >> MM works. As it is, there seems to

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-26 Thread NeilBrown
On Tue, Jul 26 2016, Mikulas Patocka wrote: > On Sat, 23 Jul 2016, NeilBrown wrote: > >> "dirtying ... from the reclaim context" ??? What does that mean? >> According to >> Commit: 26eecbf3543b ("[PATCH] vm: pageout throttling") >> From the hist

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-27 Thread NeilBrown
On Thu, Jul 28 2016, Mikulas Patocka wrote: > On Wed, 27 Jul 2016, NeilBrown wrote: > >> On Tue, Jul 26 2016, Mikulas Patocka wrote: >> >> > On Sat, 23 Jul 2016, NeilBrown wrote: >> > >> >> "dirtying ... from the reclaim context"

Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

2016-07-27 Thread NeilBrown
On Thu, Jul 28 2016, Michal Hocko wrote: > On Wed 27-07-16 13:43:35, NeilBrown wrote: >> On Mon, Jul 25 2016, Michal Hocko wrote: >> >> > On Sat 23-07-16 10:12:24, NeilBrown wrote: > [...] >> So should there be a limit on dirty >> pages in the swap cache ju

Re: [dm-devel] [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-11-20 Thread NeilBrown
On Mon, Nov 20 2017, Mike Snitzer wrote: > On Sun, Jun 18, 2017 at 5:36 PM, NeilBrown wrote: >> On Sun, Jun 18 2017, Jens Axboe wrote: >> >>> On Sun, Jun 18 2017, NeilBrown wrote: >>>> This is a resend of my series of patches working >>>> towards

Re: [dm-devel] [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-11-21 Thread NeilBrown
On Tue, Nov 21 2017, Mike Snitzer wrote: > On Mon, Nov 20 2017 at 8:35pm -0500, > Mike Snitzer wrote: > >> On Mon, Nov 20 2017 at 7:34pm -0500, >> NeilBrown wrote: >> >> > On Mon, Nov 20 2017, Mike Snitzer wrote: >> > >> > > >&

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER [was: Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.]

2017-11-21 Thread NeilBrown
t think it should...). > > The processing of md->rescued is also wrong - bios for different devices > must be offloaded to different helper threads, so that processing a bio > for a lower device doesn't depend on processing a bio for a higher device. > If you offload all the bios on curren

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER

2017-11-21 Thread NeilBrown
urrent dm_offload() approach. One possibility would be to copy the approach taken by crypt_alloc_buffer() which needs to allocate multiple entries from a mempool. It first tries the with GFP_NOWAIT. If that fails it take a mutex and tries with GFP_NOIO. This mean only one thread will try to a

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER

2017-11-22 Thread NeilBrown
On Wed, Nov 22 2017, Mikulas Patocka wrote: > On Wed, 22 Nov 2017, NeilBrown wrote: > >> On Tue, Nov 21 2017, Mikulas Patocka wrote: >> >> > On Tue, 21 Nov 2017, Mike Snitzer wrote: >> > >> >> On Tue, Nov 21 2017 at 4:23pm -0500, >> >>

[dm-devel] [PATCH] dm: use cloned bio as head, not remainder, in __split_and_process_bio()

2017-11-23 Thread NeilBrown
ready calling __map_bio(). This bio will only be dereferenced again after dec_pending() has found io->io_count to be zero, and this cannot happen before the dec_pending() call at the end of __split_and_process_bio(). Reported-by: Mikulas Patocka Signed-off-by: NeilBrown --- Hi, I think this sh

Re: [dm-devel] dm: use cloned bio as head, not remainder, in __split_and_process_bio()

2017-11-28 Thread NeilBrown
On Mon, Nov 27 2017, Mike Snitzer wrote: > On Thu, Nov 23 2017 at 5:52pm -0500, > NeilBrown wrote: > >> >> When we use bio_clone_bioset() to split off the front part of a bio >> and chain the two together and submit the remainder to >> generic_make_request(

Re: [dm-devel] [LSF/MM TOPIC] block: extend generic biosets to allow per-device frontpad

2018-02-02 Thread NeilBrown
mmunicate changes to this number. 5/ gather statistics on what percentage of bios have a too-small frontpad. Then start modifying places that allocate bios to use the hint, and when benchmarks show the percentage is high - use it to encourage other people to allocate better bios. NeilBrown

Re: [dm-devel] [LSF/MM TOPIC] block, dm: restack queue_limits

2018-02-02 Thread NeilBrown
e this as being two-stage. 1/ I'm going to change the device size to X - are you all OK with that? 2/ Device size is now X. That allows md and dm to check that filesystems aren't going to get mad when devices are made smaller, and can adjust (if they want to) when devices get bigger. Thank

Re: [dm-devel] DM Regression in 4.16-rc1 - read() returns data when it shouldn't

2018-02-14 Thread NeilBrown
On Wed, Feb 14 2018, Milan Broz wrote: > Hi, > > the commit (found by bisect) > > commit 18a25da84354c6bb655320de6072c00eda6eb602 > Author: NeilBrown > Date: Wed Sep 6 09:43:28 2017 +1000 > > dm: ensure bio submission follows a depth-first tree walk > &g

Re: [dm-devel] DM Regression in 4.16-rc1 - read() returns data when it shouldn't

2018-02-14 Thread NeilBrown
On Wed, Feb 14 2018, Mike Snitzer wrote: > On Wed, Feb 14 2018 at 3:39pm -0500, > NeilBrown wrote: > >> On Wed, Feb 14 2018, Milan Broz wrote: >> >> > Hi, >> > >> > the commit (found by bisect) >> > >> > commit 18a25da84354

Re: [dm-devel] DM Regression in 4.16-rc1 - read() returns data when it shouldn't

2018-02-15 Thread NeilBrown
On Thu, Feb 15 2018, Milan Broz wrote: > On 02/15/2018 01:07 AM, NeilBrown wrote: >> >> And looking again at the code, it doesn't seem so bad. I was worried >> about reassigning ci.io->orig_bio after calling >> __split_and_process_non_flush(), but the importa

[dm-devel] [RFC PATCH] dm: don't assign zero to ->bi_status of an active bio.

2018-02-15 Thread NeilBrown
to any other error status as for most purposes, one error is as good as another. This patch is untested and RFC only. It identifies a number of possible problem areas in dm and often suggests fixes. Please don't apply without careful review. Reviewed-by: Nobody yet. Signed-off-by: N

[dm-devel] [PATCH] dm: correctly handle chained bios in dec_pending()

2018-02-15 Thread NeilBrown
-and-tested-by: Milan Broz Cc: sta...@vger.kernel.org (v3.14+) Signed-off-by: NeilBrown --- drivers/md/dm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index d6de00f367ef..68136806d365 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c

[dm-devel] [PATCH] block: be more careful about status in __bio_chain_endio

2018-02-15 Thread NeilBrown
was found in dm.c Signed-off-by: NeilBrown --- block/bio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index e1708db48258..ad77140edc6f 100644 --- a/block/bio.c +++ b/block/bio.c @@ -312,7 +312,7 @@ static struct bio *__bio_chain_endi

Re: [dm-devel] DM Regression in 4.16-rc1 - read() returns data when it shouldn't

2018-02-26 Thread NeilBrown
back, which that commit just made more problematic. That is why I added the Cc: stable with the earliest version that needed fixing. Unfortunately, reality isn't always neat and tidy :-( when it is, I do use Fixes: Thanks, NeilBrown > > to the commit? For details see > Doc

Re: [dm-devel] why does __split_and_process_bio use bio_clone_bioset?

2018-06-14 Thread NeilBrown
ding the existing code. > > Neil, please still feel free to have a look at this to see if you can > recall why you used bio_clone_bioset(). > > If in the end you agree that the following patch is fine please let me > know and I'll get a proper fix staged. I agree with the

Re: [dm-devel] [RFC] [PATCH 0/1] Introduce emergency raid0 stop for mounted arrays

2018-08-01 Thread NeilBrown
. I'm not sure if this should cause reads to fail, but it seems quite reasonable that it would cause all writes to fail. I would only change the kernel to recognise the flag and refuse any writes after any error has been seen. I would use udev/mdadm to detect a device removal and to mark the

Re: [dm-devel] [RFC] [PATCH 0/1] Introduce emergency raid0 stop for mounted arrays

2018-08-05 Thread NeilBrown
On Thu, Aug 02 2018, Guilherme G. Piccoli wrote: > On 01/08/2018 22:51, NeilBrown wrote: >>> [...] >> If you have hard drive and some sectors or track stop working, I think >> you would still expect IO to the other sectors or tracks to keep >> working. >> >