Re: Too large badblocks sysfs file (was: [PATCH v3 0/7] badblocks improvement for multiple bad block ranges)

2021-09-23 Thread NeilBrown
the on-disk data the can be and should be read directly... Except that mdadm does. That was a mistake. check_for_cleared_bb() is wrong. I wonder why it was added. The commit message doesn't give any justification. NeilBrown > > Recently I work on the bad blocks API (block/badblocks.

Re: [PATCH] md/raid0: avoid RAID0 data corruption due to layout confusion.

2019-09-11 Thread NeilBrown
On Wed, Sep 11 2019, Song Liu wrote: > On Wed, Sep 11, 2019 at 12:10 AM NeilBrown wrote: >> >> On Tue, Sep 10 2019, Guoqing Jiang wrote: >> >> > On 9/10/19 5:45 PM, Song Liu wrote: >> >> >> >> >> >>> On Sep 10, 2019, at 12

Re: [PATCH] md/raid0: avoid RAID0 data corruption due to layout confusion.

2019-09-10 Thread NeilBrown
On Tue, Sep 10 2019, Guoqing Jiang wrote: > On 9/10/19 5:45 PM, Song Liu wrote: >> >> >>> On Sep 10, 2019, at 12:33 AM, NeilBrown wrote: >>> >>> On Mon, Sep 09 2019, Song Liu wrote: >>> >>>> Hi Neil, >>>> >>>&

Re: [PATCH] md/raid0: avoid RAID0 data corruption due to layout confusion.

2019-09-09 Thread NeilBrown
On Mon, Sep 09 2019, Guoqing Jiang wrote: > On 9/9/19 8:57 AM, NeilBrown wrote: >> >> If the drives in a RAID0 are not all the same size, the array is >> divided into zones. >> The first zone covers all drives, to the size of the smallest. >> The second zone

Re: [PATCH] md/raid0: avoid RAID0 data corruption due to layout confusion.

2019-09-09 Thread NeilBrown
On Mon, Sep 09 2019, Song Liu wrote: > Hi Neil, > >> On Sep 9, 2019, at 7:57 AM, NeilBrown wrote: >> >> >> If the drives in a RAID0 are not all the same size, the array is >> divided into zones. >> The first zone covers all drives, to the size of t

Re: [PATCH 2/2] md: add feature flag MD_FEATURE_RAID0_LAYOUT

2019-09-09 Thread NeilBrown
On Mon, Sep 09 2019, Guoqing Jiang wrote: > Hi Neil, > > On 9/9/19 8:58 AM, NeilBrown wrote: >> >> Due to a bug introduced in Linux 3.14 we cannot determine the >> correctly layout for a multi-zone RAID0 array - there are two >> possibiities. > > possibili

[PATCH 2/2] md: add feature flag MD_FEATURE_RAID0_LAYOUT

2019-09-08 Thread NeilBrown
metadata. So add a feature flag for this purpose. If it is set, then the 'layout' field of the superblock is used to determine which layout to use. If this flag is not set, then mddev->layout gets set to -1, which causes the module parameter to be required. Signed-off-by: NeilBrown

[PATCH] md/raid0: avoid RAID0 data corruption due to layout confusion.

2019-09-08 Thread NeilBrown
written by. So we add a module parameter to allow the old (0) or new (1) layout to be specified, and refused to assemble an affected array if that parameter is not set. Fixes: 20d0189b1012 ("block: Introduce new bio_split()") cc: sta...@vger.kernel.org (3.14+) Signed-off-by: NeilBrown --- Th

Re: Issues about the merge_bvec_fn callback in 3.10 series

2019-08-22 Thread NeilBrown
ix for it. > > And maybe it would be fixed until > 8ae126660fddbeebb9251a174e6fa45b6ad8f932 > block: kill merge_bvec_fn() completely > > Would anyone please give some suggestion on this ? One option would be to make sure that ->bi_rw is set before bio_add_page is called

Re: [RFC] How to handle an ugly md raid0 sector map bug ?

2019-08-22 Thread NeilBrown
en need to work out how to make it all usable via mdadm. mdadm needs to set the relevant flags and 'layout' values when creating an array, and it needs to be able to set them correctly on an existing array that doesn't have them set. We also need to decide what happens when a new mdadm is run on an old kernel. If we create an array with a new feature bit, the old kernel won't be able to assemble it. But if we don't set the feature bit, then the array will not assemble after an upgrade. It might be easier to think about this when we actually have concrete code, but if anyone does have useful input, I'd be very happy to read it. NeilBrown signature.asc Description: PGP signature

Re: [PATCH 1/2] md/raid0: Introduce new array state 'broken' for raid0

2019-07-30 Thread NeilBrown
On Tue, Jul 30 2019, Bob Liu wrote: > > > Curious why only raid0 has this issue? Actually, it isn't only raid0. 'linear' has the same issue. Probably the fix for raid0 should be applied to linear too. NeilBrown signature.asc Description: PGP signature

Re: [PATCH 1/2] md/raid0: Introduce new array state 'broken' for raid0

2019-07-29 Thread NeilBrown
eing useful only to distinguish if such array has some member > missing. Also, we show a rate-limited warning in kernel log in such case. > > This patch has no proper functional change other than adding a 'clean'-like > state; it was tested with ext4 and xfs filesystems. It req

Re: [PATCH] md/raid0: Fail BIOs if their underlying block device is gone

2019-07-29 Thread NeilBrown
ter a machine reboot, e2fsck shows some severe fs errors > that demand fixing. This patch was tested in both ext4 and xfs > filesystems. > > Cc: NeilBrown > Cc: Song Liu > Signed-off-by: Guilherme G. Piccoli > --- > > After an attempt to change the way md/raid0 fails

Re: [dm-devel] [PATCH 3/4] dm: fix missing bio_split() pattern code in __split_and_process_bio()

2019-01-21 Thread NeilBrown
t the flag and do the trace in bio_split(). Do you see any harm with doing it that way (in the next merge-window, I don't suggest you change this patch). Thanks, NeilBrown > -- > 2.15.0 > > -- > dm-devel mailing list > dm-de...@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel signature.asc Description: PGP signature

Re: [PATCH v5 1/2] lib: add crc64 calculation routines

2018-12-28 Thread NeilBrown
Ross N. Williams >> (see http://www.ross.net/crc/download/crc_v3.txt) and other public domain >> implementations. Hi Coly and Andrew, This patch breaks compilation if linux headers files are not installed (reported at https://github.com/gnubee-git/GnuBee_Docs/issues/75). Followin

Re: [GIT PULL] Block changes for 4.18-rc

2018-06-04 Thread NeilBrown
On Mon, Jun 04 2018, Kent Overstreet wrote: > On Tue, Jun 05, 2018 at 07:16:51AM +1000, NeilBrown wrote: >> I really should get back to BIOSET_NEED_RESCUER and see if I can discard >> it completely. >> Kent - the only remaining user is bcache, and the main reason I haven'

Re: [GIT PULL] Block changes for 4.18-rc

2018-06-04 Thread NeilBrown
) will take a mutex. This must have been in destory_workqueue() when destroying bs->rescue_workqueue. md biosets don't have a work queue (BIOSET_NEED_RESCUER isn't set), so these calls will no longer take a mutex. So this commit can now be reverted. I think that will clean up this code

Re: [PATCH 00/13] convert block layer to bioset_init()/mempool_init()

2018-05-20 Thread NeilBrown
vert to bioset_init()/mempool_init() > bcache: convert to bioset_init()/mempool_init() > md: convert to bioset_init()/mempool_init() Hi Kent, this conversion looks really good, thanks for Ccing me on it. However as Shaohua Li is now the maintainer of md, it probably should have go

[PATCH] block: be more careful about status in __bio_chain_endio

2018-02-15 Thread NeilBrown
was found in dm.c Signed-off-by: NeilBrown --- block/bio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index e1708db48258..ad77140edc6f 100644 --- a/block/bio.c +++ b/block/bio.c @@ -312,7 +312,7 @@ static struct bio *__bio_chain_endi

Re: [dm-devel] [LSF/MM TOPIC] block: extend generic biosets to allow per-device frontpad

2018-02-02 Thread NeilBrown
mmunicate changes to this number. 5/ gather statistics on what percentage of bios have a too-small frontpad. Then start modifying places that allocate bios to use the hint, and when benchmarks show the percentage is high - use it to encourage other people to allocate better bios. NeilBrown signature.asc Description: PGP signature

Re: [dm-devel] [LSF/MM TOPIC] block, dm: restack queue_limits

2018-02-02 Thread NeilBrown
e this as being two-stage. 1/ I'm going to change the device size to X - are you all OK with that? 2/ Device size is now X. That allows md and dm to check that filesystems aren't going to get mad when devices are made smaller, and can adjust (if they want to) when devices get bigger. Thank

Re: [LSF/MM TOPIC] De-clustered RAID with MD

2018-01-30 Thread NeilBrown
On Tue, Jan 30 2018, Wols Lists wrote: > On 29/01/18 21:50, NeilBrown wrote: >> By doing declustered parity you can sanely do raid6 on 100 drives, using >> a logical stripe size that is much smaller than 100. >> When recovering a single drive, the 10-groups-of-10 would p

Re: [LSF/MM TOPIC] De-clustered RAID with MD

2018-01-29 Thread NeilBrown
do raid6 on 100 drives, using a logical stripe size that is much smaller than 100. When recovering a single drive, the 10-groups-of-10 would put heavy load on 9 other drives, while the decluster approach puts light load on 99 other drives. No matter how clever md is at throttling recovery, I wo

Re: confusion about bio_split

2018-01-22 Thread NeilBrown
f months ago. Jens - can you apply that? Do you need it resent? https://www.mail-archive.com/linux-block@vger.kernel.org/msg16039.html Thanks, NeilBrown > > > Regards, > shenghui signature.asc Description: PGP signature

Re: [dm-devel] dm: use cloned bio as head, not remainder, in __split_and_process_bio()

2017-11-28 Thread NeilBrown
On Mon, Nov 27 2017, Mike Snitzer wrote: > On Thu, Nov 23 2017 at 5:52pm -0500, > NeilBrown wrote: > >> >> When we use bio_clone_bioset() to split off the front part of a bio >> and chain the two together and submit the remainder to >> generic_make_request(

[PATCH] dm: use cloned bio as head, not remainder, in __split_and_process_bio()

2017-11-23 Thread NeilBrown
ready calling __map_bio(). This bio will only be dereferenced again after dec_pending() has found io->io_count to be zero, and this cannot happen before the dec_pending() call at the end of __split_and_process_bio(). Reported-by: Mikulas Patocka Signed-off-by: NeilBrown --- Hi, I think this sh

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER

2017-11-22 Thread NeilBrown
On Wed, Nov 22 2017, Mikulas Patocka wrote: > On Wed, 22 Nov 2017, NeilBrown wrote: > >> On Tue, Nov 21 2017, Mikulas Patocka wrote: >> >> > On Tue, 21 Nov 2017, Mike Snitzer wrote: >> > >> >> On Tue, Nov 21 2017 at 4:23pm -0500, >> >>

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER

2017-11-21 Thread NeilBrown
urrent dm_offload() approach. One possibility would be to copy the approach taken by crypt_alloc_buffer() which needs to allocate multiple entries from a mempool. It first tries the with GFP_NOWAIT. If that fails it take a mutex and tries with GFP_NOIO. This mean only one thread will try to a

Re: [dm-devel] new patchset to eliminate DM's use of BIOSET_NEED_RESCUER [was: Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.]

2017-11-21 Thread NeilBrown
t think it should...). > > The processing of md->rescued is also wrong - bios for different devices > must be offloaded to different helper threads, so that processing a bio > for a lower device doesn't depend on processing a bio for a higher device. > If you offload all the bios on curren

Re: [dm-devel] [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-11-21 Thread NeilBrown
On Tue, Nov 21 2017, Mike Snitzer wrote: > On Mon, Nov 20 2017 at 8:35pm -0500, > Mike Snitzer wrote: > >> On Mon, Nov 20 2017 at 7:34pm -0500, >> NeilBrown wrote: >> >> > On Mon, Nov 20 2017, Mike Snitzer wrote: >> > >> > > >&

Re: [dm-devel] [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-11-20 Thread NeilBrown
On Mon, Nov 20 2017, Mike Snitzer wrote: > On Sun, Jun 18, 2017 at 5:36 PM, NeilBrown wrote: >> On Sun, Jun 18 2017, Jens Axboe wrote: >> >>> On Sun, Jun 18 2017, NeilBrown wrote: >>>> This is a resend of my series of patches working >>>> towards

Re: Enable skip_copy can cause data integrity issue in some storage stack

2017-09-06 Thread NeilBrown
use the STABLE_WRITES requirement doesn't affect the sort of bio that is passed down, but instead affect the way it is waited for. I have thought a few times that it would be useful if the "bd_claim" functionality included a callback so the claimed device could notify the claimer that things had changed. Then raid5 could notice that it has been claimed, and call the callback when it changes BDI_CAP_STABLE_WRITES. If the claimer is a stacked device, it can re-run blk_set_stacking_limits and call its own claimer. There are probably some interesting locking issues around this but, to me, it seems like the most likely path to a robust solution. NeilBrown signature.asc Description: PGP signature

Re: [PATCH v2 3/3] md: raid1-10: move raid1/raid10 common code into raid1-10.c

2017-07-16 Thread NeilBrown
On Fri, Jul 14 2017, Ming Lei wrote: > No function change, just move 'struct resync_pages' and related > helpers into raid1-10.c > > Signed-off-by: Ming Lei Thanks for doing this! I'm quite happy with this approach - and with all patches in this series. Thanks, Neil

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-13 Thread NeilBrown
On Thu, Jul 13 2017, Shaohua Li wrote: > On Thu, Jul 13, 2017 at 05:20:52PM +0800, Ming Lei wrote: >> On Thu, Jul 13, 2017 at 01:09:28PM +1000, NeilBrown wrote: >> > On Thu, Jul 13 2017, Ming Lei wrote: >> > >> > > On Thu, Jul 13, 2017 at 10:01:33AM +1000,

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-12 Thread NeilBrown
On Thu, Jul 13 2017, Ming Lei wrote: > On Thu, Jul 13, 2017 at 10:01:33AM +1000, NeilBrown wrote: >> On Wed, Jul 12 2017, Ming Lei wrote: >> >> > We will support multipage bvec soon, so initialize bvec >> > table using the standardy way instead of writing the

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-12 Thread NeilBrown
emove code duplication, then work on moving all raid1 functionality into raid10.c, then discard raid1.c Or at the very least, have a separate "raid1-10.c" file for the common code. NeilBrown > + > #ifndef MODULE > > /* > diff --git a/drivers/md/md.h b/drivers/md/md.

Re: [PATCH 1/2] md: remove 'idx' from 'struct resync_pages'

2017-07-12 Thread NeilBrown
rong since mempool is only > responsible for allocation, we can't use that for initialization. > > Suggested-by: NeilBrown > Reported-by: NeilBrown > Fixes: f0250618361d(md: raid10: don't use bio's vec table to manage resync > pages) > Fixes: 98d30c5812c3(md:

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-10 Thread NeilBrown
On Mon, Jul 10 2017, Shaohua Li wrote: > On Mon, Jul 10, 2017 at 03:25:41PM +0800, Ming Lei wrote: >> On Mon, Jul 10, 2017 at 02:38:19PM +1000, NeilBrown wrote: >> > On Mon, Jul 10 2017, Ming Lei wrote: >> > >> > > On Mon, Jul 10, 2017 at 11:35:12AM +0800,

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-09 Thread NeilBrown
On Mon, Jul 10 2017, Ming Lei wrote: > On Mon, Jul 10, 2017 at 11:35:12AM +0800, Ming Lei wrote: >> On Mon, Jul 10, 2017 at 7:09 AM, NeilBrown wrote: ... >> >> + >> >> + rp->idx = 0; >> > >> > This is the only place the ->id

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-09 Thread NeilBrown
rp->idx = 0; This is the only place the ->idx is initialized, in r1buf_pool_alloc(). The mempool alloc function is suppose to allocate memory, not initialize it. If the mempool_alloc() call cannot allocate memory it will use memory from the pool. If this memory has already been used, then it will no longer have the initialized value. In short: you need to initialise memory *after* calling mempool_alloc(), unless you ensure it is reset to the init values before calling mempool_free(). https://bugzilla.kernel.org/show_bug.cgi?id=196307 NeilBrown signature.asc Description: PGP signature

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-18 Thread NeilBrown
On Sun, Jun 18 2017, Jens Axboe wrote: > On Sun, Jun 18 2017, NeilBrown wrote: >> This is a resend of my series of patches working >> towards removing the bioset work queues. >> >> This set is based on for-4.13/block. >> >> It incorporates the revise

[PATCH 03/13] blk: make the bioset rescue_workqueue optional.

2017-06-17 Thread NeilBrown
: Ming Lei Signed-off-by: NeilBrown --- block/bio.c| 13 +++-- block/blk-core.c |3 ++- drivers/block/drbd/drbd_main.c |4 +++- drivers/md/bcache/super.c |8 ++-- drivers/md/dm-crypt.c |3 ++- drivers/md/dm-io.c

[PATCH 01/13] blk: remove bio_set arg from blk_queue_split()

2017-06-17 Thread NeilBrown
ys use q->bio_split inside blk_queue_split() Reviewed-by: Christoph Hellwig Reviewed-by: Ming Lei Credit-to: Javier González (Noticed that lightnvm was missed) Reviewed-by: Javier González Tested-by: Javier González Signed-off-by: NeilBrown --- block/blk-core.c |2 +- block/b

[PATCH 04/13] blk: use non-rescuing bioset for q->bio_split.

2017-06-17 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Reviewed-by: Christoph Hellwig Reviewed-by: Ming Lei Signed-off-by: NeilBrown --- block/blk-co

[PATCH 05/13] block: Improvements to bounce-buffer handling

2017-06-17 Thread NeilBrown
ch_segment loop) Reviewed-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- block/blk-merge.c | 14 -- block/bounce.c| 32 ++-- 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/block/blk-merge.c b/block/bl

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/16/2017 01:34 AM, Christoph Hellwig wrote: >> On Fri, Jun 16, 2017 at 05:30:50PM +1000, NeilBrown wrote: >>> I've pushed the new version to the same place. Do you actually want >>> me to re-post all the patches? >

[PATCH 06/13] rbd: use bio_clone_fast() instead of bio_clone()

2017-06-17 Thread NeilBrown
clone_fast(). Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/rbd.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 5420bc40c544..b008b6a98098 100644 --- a/drivers/block/rbd.c +++ b/dri

[PATCH 13/13] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-06-17 Thread NeilBrown
bio_clone_fast() doesn't clone the bi_io_vec. The main user of of bio_clone_bioset() at this level is bounce.c, and bouncing now happens before blk_bio_segment_split(), so that is not of concern. So remove the big helpful comment and the code. Reviewed-by: Christoph Hellwig Signed-off-by: Neil

[PATCH 08/13] pktcdvd: use bio_clone_fast() instead of bio_clone()

2017-06-17 Thread NeilBrown
o_set, pkt_bio_set, can also be use for the bio_split() call as the two allocations (bio_clone_fast, and bio_split) are independent, neither can block a bio allocated by the other. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/pktcdvd.c | 12 ++-- 1 file change

[PATCH 11/13] bcache: use kmalloc to allocate bio in bch_data_verify()

2017-06-17 Thread NeilBrown
is used either for the bio or the pages. Reviewed-by: Christoph Hellwig Acked-by: Kent Overstreet Reviewed-by : Ming Lei Signed-off-by: NeilBrown --- drivers/md/bcache/debug.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/16/2017 01:34 AM, Christoph Hellwig wrote: >> On Fri, Jun 16, 2017 at 05:30:50PM +1000, NeilBrown wrote: >>> I've pushed the new version to the same place. Do you actually want >>> me to re-post all the patches? >

[PATCH 09/13] lightnvm/pblk-read: use bio_clone_fast()

2017-06-17 Thread NeilBrown
stoph Hellwig Reviewed-by: Javier González Tested-by: Javier González Signed-off-by: NeilBrown --- drivers/lightnvm/pblk-init.c | 12 +++- drivers/lightnvm/pblk-read.c |2 +- drivers/lightnvm/pblk.h |1 + 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/dr

[PATCH 12/13] block: remove bio_clone() and all references.

2017-06-17 Thread NeilBrown
: Christoph Hellwig Reviewed-by: Ming Lei Signed-off-by: NeilBrown --- Documentation/block/biodoc.txt |2 +- block/bio.c|2 +- block/blk-merge.c |6 +++--- drivers/md/md.c|2 +- include/linux/bio.h|5 - 5 files

[PATCH 07/13] drbd: use bio_clone_fast() instead of bio_clone()

2017-06-17 Thread NeilBrown
fail if the GFP flags allow for sleeping. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/drbd/drbd_int.h |3 +++ drivers/block/drbd/drbd_main.c |9 + drivers/block/drbd/drbd_req.h |2 +- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git

[PATCH 10/13] xen-blkfront: remove bio splitting.

2017-06-17 Thread NeilBrown
bios that are re-submitted will pass through blk_queue_split() when blk_queue_bio() is called, and this will split the bio if necessary. There is no longer any need to do this splitting in xen-blkfront. Acked-by: Roger Pau Monné Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown

[PATCH 02/13] blk: replace bioset_create_nobvec() with a flags arg to bioset_create()

2017-06-17 Thread NeilBrown
gs' argument to bioset_create() and discard bioset_create_nobvec(). Note that the bio_split allocations in drivers/md/raid* do not need the bvec mempool - they should have used bioset_create_nobvec(). Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Reviewed-by: Ming Lei Sign

[PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
, and simple changes to compensate for a couple of changes to the block tree since the last posting. I hope to eventually get rid of the new BIOSET_NEED_RESCUER flag, but that needs work in dm and probably bcache first. Thanks, NeilBrown --- NeilBrown (13): blk: remove bio_set arg from

Re: [PATCH 0/2] Two fixes for loop devices

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/15/2017 11:02 PM, NeilBrown wrote: >> Hi Jens, >> one of these is a resend of a patch I sent a while back. >> The other is new - loop closes files differently from close() >> and in a way that can confuse NFS. > >

Re: [PATCH 1/2] loop: use filp_close() rather than fput()

2017-06-17 Thread NeilBrown
On Sat, Jun 17 2017, Al Viro wrote: > On Fri, Jun 16, 2017 at 03:02:09PM +1000, NeilBrown wrote: >> When a loop device is being shutdown the backing file is >> closed with fput(). This is different from how close(2) >> closes files - it uses filp_close(). >> >>

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-16 Thread NeilBrown
On Thu, Jun 15 2017, Christoph Hellwig wrote: > On Fri, Jun 16, 2017 at 03:54:30PM +1000, NeilBrown wrote: >> Hi Jens, >> I didn't hear back ... have you had a chance to look? >> In case it helps, you can pull the full set, based on a recent Linus tree, >> fr

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-15 Thread NeilBrown
On Thu, May 11 2017, NeilBrown wrote: > On Tue, May 02 2017, NeilBrown wrote: > >> This is a revision of my series of patches working >> towards removing the bioset work queues. > > Hi Jens, > could I get some feed-back about your thoughts on this series? > Will y

[PATCH 2/2] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-06-15 Thread NeilBrown
Hocko Signed-off-by: NeilBrown --- drivers/block/loop.c |8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 9c457ca6c55e..6ed7c4506951 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -843,10 +843,16 @@ static

[PATCH 1/2] loop: use filp_close() rather than fput()

2017-06-15 Thread NeilBrown
ast close, and gets confused otherwise. Signed-off-by: NeilBrown --- drivers/block/loop.c |6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index ebbd0c3fe0ed..9c457ca6c55e 100644 --- a/drivers/block/loop.c +++ b/drivers/bl

[PATCH 0/2] Two fixes for loop devices

2017-06-15 Thread NeilBrown
Hi Jens, one of these is a resend of a patch I sent a while back. The other is new - loop closes files differently from close() and in a way that can confuse NFS. Thanks, NeilBrown --- NeilBrown (2): loop: use filp_close() rather than fput() loop: Add PF_LESS_THROTTLE to block

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-05-10 Thread NeilBrown
On Tue, May 02 2017, NeilBrown wrote: > This is a revision of my series of patches working > towards removing the bioset work queues. Hi Jens, could I get some feed-back about your thoughts on this series? Will you apply it? When? Do I need to resend anything? Would you like a gi

Re: [PATCH v4 13/27] lib: add errseq_t type and infrastructure for handling it

2017-05-09 Thread NeilBrown
f Layton I like that this is a separate lib/*.c - nicely structured too. Reviewed-by: NeilBrown Thanks, NeilBrown > --- > include/linux/errseq.h | 19 + > lib/Makefile | 2 +- > lib/errseq.c | 199 >

[PATCH 04/13 V2] blk: use non-rescuing bioset for q->bio_split.

2017-05-02 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Reviewed-by: Christoph Hellwig Reviewed-by: Ming Lei Signed-off-by: NeilBrown --- This

[PATCH 07/13 V2] drbd: use bio_clone_fast() instead of bio_clone()

2017-05-02 Thread NeilBrown
fail if the GFP flags allow for sleeping. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- This patch needed to be refreshed after long lines were wrapped in an earlier patch. Also added the BIOSET_NEED_RESCUER flag for new bioset as it is not trivially obvious that this isn'

Re: [PATCH 13/13] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-05-02 Thread NeilBrown
On Tue, May 02 2017, Ming Lei wrote: > On Tue, May 02, 2017 at 01:42:26PM +1000, NeilBrown wrote: >> blk_bio_segment_split() makes sure bios have no more than >> BIO_MAX_PAGES entries in the bi_io_vec. >> This was done because bio_clone_bioset() (when given a >> mempo

[PATCH 03/13 V2] blk: make the bioset rescue_workqueue optional.

2017-05-02 Thread NeilBrown
From 09017acf74ec4df674b78ca66f0924187f10d8a4 Mon Sep 17 00:00:00 2001 From: NeilBrown Date: Fri, 10 Mar 2017 13:59:50 +1100 Subject: [PATCH] blk: make the bioset rescue_workqueue optional. This patch converts bioset_create() to not create a workqueue by default, so alloctions will never trigger

Re: [PATCH 03/13] blk: make the bioset rescue_workqueue optional.

2017-05-02 Thread NeilBrown
On Tue, May 02 2017, Ming Lei wrote: > On Tue, May 02, 2017 at 01:42:24PM +1000, NeilBrown wrote: >> This patch converts bioset_create() and >> bioset_create_nobvec() to not create a workqueue so >> alloctions will never trigger punt_bios_to_rescuer(). It >> also intro

Re: [PATCH 09/13] lightnvm/pblk-read: use bio_clone_fast()

2017-05-02 Thread NeilBrown
On Tue, May 02 2017, Javier González wrote: >> On 2 May 2017, at 05.42, NeilBrown wrote: >> >> pblk_submit_read() uses bio_clone_bioset() but doesn't change the >> io_vec, so bio_clone_fast() is a better choice. >> >> It also uses fs_bio_set which is in

Re: [PATCH 02/13] blk: replace bioset_create_nobvec() with a flags arg to bioset_create()

2017-05-02 Thread NeilBrown
27;t fight for it. > > Why not just > > BIOSET_NEED_BVECS = (1 << 0), > > which is a lot more intuitive. > > Otherwise looks fine to me: > > Reviewed-by: Christoph Hellwig Thanks, NeilBrown signature.asc Description: PGP signature

[PATCH 03/13] blk: make the bioset rescue_workqueue optional.

2017-05-01 Thread NeilBrown
: NeilBrown --- block/bio.c | 12 ++-- block/blk-core.c |2 +- drivers/md/bcache/super.c |6 -- drivers/md/dm-crypt.c |2 +- drivers/md/dm-io.c|2 +- drivers/md/dm.c |5 +++-- include/linux/bio.h |1 + 7 files

[PATCH 01/13] blk: remove bio_set arg from blk_queue_split()

2017-05-01 Thread NeilBrown
ys use q->bio_split inside blk_queue_split() Reviewed-by: Christoph Hellwig Reviewed-by: Ming Lei Credit-to: Javier González (Noticed that lightnvm was missed) Signed-off-by: NeilBrown --- block/blk-core.c |2 +- block/blk-merge.c |9 - block/blk-mq.c

[PATCH 04/13] blk: use non-rescuing bioset for q->bio_split.

2017-05-01 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- block/blk-core.c |2 +-

[PATCH 06/13] rbd: use bio_clone_fast() instead of bio_clone()

2017-05-01 Thread NeilBrown
clone_fast(). Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/rbd.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 089ac4179919..52120ed9385f 100644 --- a/drivers/block/rbd.c +++ b/dri

[PATCH 08/13] pktcdvd: use bio_clone_fast() instead of bio_clone()

2017-05-01 Thread NeilBrown
o_set, pkt_bio_set, can also be use for the bio_split() call as the two allocations (bio_clone_fast, and bio_split) are independent, neither can block a bio allocated by the other. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/pktcdvd.c | 12 ++-- 1 file change

[PATCH 11/13] bcache: use kmalloc to allocate bio in bch_data_verify()

2017-05-01 Thread NeilBrown
is used either for the bio or the pages. Reviewed-by: Christoph Hellwig Acked-by: Kent Overstreet Reviewed-by : Ming Lei Signed-off-by: NeilBrown --- drivers/md/bcache/debug.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache

[PATCH 12/13] block: remove bio_clone() and all references.

2017-05-01 Thread NeilBrown
: Christoph Hellwig Reviewed-by: Ming Lei Signed-off-by: NeilBrown --- Documentation/block/biodoc.txt |2 +- block/bio.c|2 +- block/blk-merge.c |6 +++--- drivers/md/md.c|2 +- include/linux/bio.h|5 - 5 files

[PATCH 10/13] xen-blkfront: remove bio splitting.

2017-05-01 Thread NeilBrown
bios that are re-submitted will pass through blk_queue_split() when blk_queue_bio() is called, and this will split the bio if necessary. There is no longer any need to do this splitting in xen-blkfront. Acked-by: Roger Pau Monné Signed-off-by: NeilBrown --- drivers/block/xen-blkfront.c | 54

[PATCH 13/13] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-05-01 Thread NeilBrown
bio_clone_fast() doesn't clone the bi_io_vec. The main user of of bio_clone_bioset() at this level is bounce.c, and bouncing now happens before blk_bio_segment_split(), so that is not of concern. So remove the big helpful comment and the code. Signed-off-by: NeilBrown --- block/blk-merge.c |

[PATCH 09/13] lightnvm/pblk-read: use bio_clone_fast()

2017-05-01 Thread NeilBrown
pblk_submit_read() uses bio_clone_bioset() but doesn't change the io_vec, so bio_clone_fast() is a better choice. It also uses fs_bio_set which is intended for filesystems. Using it in a device driver can deadlock. So allocate a new bioset, and and use bio_clone_fast(). Signed-off-by: Neil

[PATCH 05/13] block: Improvements to bounce-buffer handling

2017-05-01 Thread NeilBrown
ch_segment loop) Signed-off-by: NeilBrown --- block/blk-merge.c | 14 -- block/bounce.c| 32 ++-- 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index d59074556703..51c84540d3bb 100644 --- a/block/bl

[PATCH 07/13] drbd: use bio_clone_fast() instead of bio_clone()

2017-05-01 Thread NeilBrown
fail if the GFP flags allow for sleeping. Reviewed-by: Christoph Hellwig Signed-off-by: NeilBrown --- drivers/block/drbd/drbd_int.h |3 +++ drivers/block/drbd/drbd_main.c |9 + drivers/block/drbd/drbd_req.h |2 +- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git

[PATCH 02/13] blk: replace bioset_create_nobvec() with a flags arg to bioset_create()

2017-05-01 Thread NeilBrown
gs' argument to bioset_create() and discard bioset_create_nobvec(). Note that the bio_split allocations in drivers/md/raid* do not need the bvec mempool - they should have used bioset_create_nobvec(). Suggested-by: Christoph Hellwig Signed-off-by: NeilBrown --- block/bio.c

[PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-05-01 Thread NeilBrown
ate_nobvec() in favor of a flag arg to bioset_create(). There are also minor fixes and a little code clean-up. I hope to eventually get rid of the new BIOSET_NEED_RESCUER flag, but that needs work ing dm and probably bcache first. Thanks, NeilBrown --- NeilBrown (13): blk: remove bio_se

Re: [PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-05-01 Thread NeilBrown
On Mon, May 01 2017, Jens Axboe wrote: > On 04/30/2017 11:00 PM, NeilBrown wrote: >> On Mon, Apr 24 2017, Christoph Hellwig wrote: >> >>> On Mon, Apr 24, 2017 at 11:51:01AM +1000, NeilBrown wrote: >>>> >>>> I was following the existing p

Re: [PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-04-30 Thread NeilBrown
On Mon, Apr 24 2017, Christoph Hellwig wrote: > On Mon, Apr 24, 2017 at 11:51:01AM +1000, NeilBrown wrote: >> >> I was following the existing practice exemplified by >> bioset_create_nobvec(). > > Which is pretty ugly to start with.. That is a matter of personal tast

Re: [PATCH 11/11] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Ming Lei wrote: > On Fri, Apr 21, 2017 at 7:34 PM, Christoph Hellwig wrote: >> On Thu, Apr 20, 2017 at 03:38:50PM +1000, NeilBrown wrote: >>> blk_bio_segment_split() makes sure bios have no more than >>> BIO_MAX_PAGES entries in the bi_io_vec

Re: [PATCH 11/11] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Christoph Hellwig wrote: > On Thu, Apr 20, 2017 at 03:38:50PM +1000, NeilBrown wrote: >> blk_bio_segment_split() makes sure bios have no more than >> BIO_MAX_PAGES entries in the bi_io_vec. >> This was done because bio_clone_bioset() (when given a >>

Re: [PATCH 01/11] blk: remove bio_set arg from blk_queue_split()

2017-04-23 Thread NeilBrown
On Sat, Apr 22 2017, Javier González wrote: >> On 20 Apr 2017, at 07.38, NeilBrown wrote: >> >> blk_queue_split() is always called with the last arg being q->bio_split, >> where 'q' is the first arg. >> >> Also blk_queue_split() sometime

Re: [PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Christoph Hellwig wrote: > On Thu, Apr 20, 2017 at 03:38:48PM +1000, NeilBrown wrote: >> This patch converts bioset_create() and >> bioset_create_nobvec() to not create a workqueue so >> alloctions will never trigger punt_bios_to_rescuer().

[PATCH 08/11] xen-blkfront: remove bio splitting.

2017-04-19 Thread NeilBrown
bios that are re-submitted will pass through blk_queue_split() when blk_queue_bio() is called, and this will split the bio if necessary. There is no longer any need to do this splitting in xen-blkfront. Signed-off-by: NeilBrown --- drivers/block/xen-blkfront.c | 54

[PATCH 00/11] block: assorted cleanup for bio splitting and cloning.

2017-04-19 Thread NeilBrown
, bio_clone_fast() does. Finally we remove bio_clone() (which is no longer used) and stop blk_bio_segment_split() from splitting after BIO_MAX_PAGES, as that is no longer necessary. --- NeilBrown (11): blk: remove bio_set arg from blk_queue_split() blk: make the bioset rescue_workqueue optional

[PATCH 01/11] blk: remove bio_set arg from blk_queue_split()

2017-04-19 Thread NeilBrown
ys use q->bio_split inside blk_queue_split() Signed-off-by: NeilBrown --- block/blk-core.c |2 +- block/blk-merge.c |9 - block/blk-mq.c|2 +- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/block/ps3vr

[PATCH 03/11] blk: use non-rescuing bioset for q->bio_split.

2017-04-19 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown --- block/blk-core.c |2 +- 1 file changed, 1 insertion(+), 1 delet

[PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-04-19 Thread NeilBrown
: NeilBrown --- block/bio.c | 28 block/blk-core.c |2 +- drivers/md/bcache/super.c |4 ++-- drivers/md/dm-crypt.c |2 +- drivers/md/dm-io.c|2 +- drivers/md/dm.c |5 +++-- include/linux/bio.h

[PATCH 04/11] block: Improvements to bounce-buffer handling

2017-04-19 Thread NeilBrown
could lead to a deadlock. So: - allocate 2 private biosets for blk_queue_bounce, one for splitting enormous bios and one for cloning bios. - add code to split a bio that exceeds BIO_MAX_PAGES. - Fix up the comments in blk_queue_split() Signed-off-by: NeilBrown --- block/blk-merge.c | 14 --

[PATCH 05/11] rbd: use bio_clone_fast() instead of bio_clone()

2017-04-19 Thread NeilBrown
clone_fast(). Signed-off-by: NeilBrown --- drivers/block/rbd.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 089ac4179919..48eecffc612e 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -441,6 +441,8

  1   2   >