Re: [PATCH 13/14] btrfs: raid56: don't lock stripe cache table when freeing

On Fri, Jun 29, 2018 at 10:57:07AM +0200, David Sterba wrote: > This is called either at the end of the mount or if the mount fails. > In both cases, there's nothing running that can use the table so the > lock is pointless. And then lockdep says no. The umount path frees the table but there's

[PATCH 13/14] btrfs: raid56: don't lock stripe cache table when freeing

This is called either at the end of the mount or if the mount fails. In both cases, there's nothing running that can use the table so the lock is pointless. Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs

[PATCH 11/14] btrfs: raid56: use new helper for async_scrub_parity

Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 14 +++--- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index f9b349171d61..339cce0878d1 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -170,7 +170,7 @@ static int

[PATCH 09/14] btrfs: raid56: use new helper for async_rmw_stripe

Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 11 ++- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index f30d847baf07..96a7d3445623 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -162,7 +162,6 @@ static int

[PATCH 12/14] btrfs: raid56: merge rbio_is_full helpers

There's only one call site of the unlocked helper so it can be folded into the caller. Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 23 ++- 1 file changed, 6 insertions(+), 17 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 339cce0878d1

[PATCH 14/14] btrfs: raid56: catch errors from full_stripe_write

Add fall-back code to catch failure of full_stripe_write. Proper error handling from inside run_plug would need more code restructuring as it's called at arbitrary points by io scheduler. Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 5 - 1 file changed, 4 insertions(+), 1 deletion

[PATCH 08/14] btrfs: raid56: add new helper for starting async work

Add helper that schedules a given function to run on the rmw workqueue. This will replace several standalone helpers. Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 1a1b7d6c44cb

[PATCH 10/14] btrfs: raid56: use new helper for async_read_rebuild

Signed-off-by: David Sterba --- fs/btrfs/raid56.c | 15 +++ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 96a7d3445623..f9b349171d61 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -162,7 +162,6 @@ static int

Re: [PATCH] Btrfs: raid56: remove redundant async_missing_raid56

On Fri, Mar 02, 2018 at 04:10:39PM -0700, Liu Bo wrote: > async_missing_raid56() is identical to async_read_rebuild(). > > Signed-off-by: Liu Bo Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

[PATCH] Btrfs: raid56: remove redundant async_missing_raid56

async_missing_raid56() is identical to async_read_rebuild(). Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 18 +- 1 file changed, 1 insertion(+), 17 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index bb8a3c5..efb42dc 100644 --

Re: [PATCH] Btrfs: raid56: iterate raid56 internal bio with bio_for_each_segment_all

On Fri, Jan 12, 2018 at 06:07:01PM -0700, Liu Bo wrote: > Bio iterated by set_bio_pages_uptodate() is raid56 internal one, so it > will never be a BIO_CLONED bio, and since this is called by end_io > functions, bio->bi_iter.bi_size is zero, we mustn't use > bio_for_each_segment() as that is a

[PATCH] Btrfs: raid56: iterate raid56 internal bio with bio_for_each_segment_all

xes: 6592e58c6b68e61f003a01ba29a3716e7e2e9484 ("Btrfs: fix write corruption due to bio cloning on raid5/6") Cc: <sta...@vger.kernel.org> # v4.12-rc6+ Signed-off-by: Liu Bo <bo.li....@oracle.com> --- fs/btrfs/raid56.c | 11 +-- 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/rai

[PATCH v2] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

Update commit log and comments of code to explain the new idea. - This has been tested against btrfs/011 for 50 times. fs/btrfs/raid56.c | 37 + 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 77473

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

On Fri, Dec 08, 2017 at 04:02:35PM -0700, Liu Bo wrote: > We're not allowed to take any new bios to rbio->bio_list in > rbio_orig_end_io(), otherwise we may get merged with more bios and > rbio->bio_list is not empty. > > This should only happens in error-out cases, the normal path of > recover

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

s not empty. > > > > This should only happens in error-out cases, the normal path of > > recover and full stripe write have already set RBIO_RMW_LOCKED_BIT to > > disable merge before doing IO. > > > > Reported-by: Jérôme Carretero <cj...@zougloub.eu> >

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

recover and full stripe write have already set RBIO_RMW_LOCKED_BIT to > disable merge before doing IO. > > Reported-by: Jérôme Carretero <cj...@zougloub.eu> > Signed-off-by: Liu Bo <bo.li@oracle.com> > --- > fs/btrfs/raid56.c | 13 - > 1 file changed, 1

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

only happens in error-out cases, the normal path of > recover and full stripe write have already set RBIO_RMW_LOCKED_BIT to > disable merge before doing IO. > > Reported-by: Jérôme Carretero <cj...@zougloub.eu> > Signed-off-by: Liu Bo <bo.li@oracle.com> > --

[PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

isable merge before doing IO. Reported-by: Jérôme Carretero <cj...@zougloub.eu> Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 5aa9d

Re: WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __free_raid_bio+0x8e/0xa0

Hi, On Wed, 22 Nov 2017 15:35:35 -0800 Liu Bo <bo.li@oracle.com> wrote: > On Mon, Nov 20, 2017 at 02:00:07AM -0500, Jérôme Carretero wrote: > > [ cut here ] [633254.461294] WARNING: CPU: > > 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 >

Re: WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __free_raid_bio+0x8e/0xa0

/I8U2-4 sector 1373688) > [633251.034990] BTRFS info (device dm-18): read error corrected: ino 1376 off > 230985728 (dev /dev/mapper/I8U2-4 sector 1373688) > [633254.456570] [ cut here ] > [633254.461294] WARNING: CPU: 3 PID: 20953 at > /usr/src/linux/fs/btrfs/raid56.c:84

Re: WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __free_raid_bio+0x8e/0xa0

On Mon, 20 Nov 2017 02:00:07 -0500 Jérôme Carretero wrote: > [ cut here ] It should be noted that the filesystem doesn't want to be unmounted now. Regards, -- Jérôme -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __free_raid_bio+0x8e/0xa0

8U2-4 sector 1373688) [633251.034990] BTRFS info (device dm-18): read error corrected: ino 1376 off 230985728 (dev /dev/mapper/I8U2-4 sector 1373688) [633254.456570] [ cut here ] [633254.461294] WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __fre

Re: [PATCH 01/14] Btrfs: raid56: add raid56 log via add_dev v2 ioctl

--- > fs/btrfs/ctree.h| 6 ++ > fs/btrfs/ioctl.c| 48 > ++++++++- > fs/btrfs/raid56.c | 42 ++++++++ > fs/btrfs/raid56.h | 1 + > fs/btrfs/volumes.c

[PATCH 12/14] Btrfs: raid56: fix error handling while adding a log device

, and step #2 is to assign the pointer in %fs_info. And by running step #1 ahead of starting transaction, we can gracefully bail out on errors now. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 48 +--- fs/btrfs/raid56.h | 5 +++

[PATCH 10/14] Btrfs: raid56: use the readahead helper to get page

This updates recovery code to use the readahead helper. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 24f7cbb..8f47e56 100644 --- a/fs/btrfs/raid56.c ++

[PATCH 06/14] Btrfs: raid56: add reclaim support

and raid5/6 array. Also we need to take care of the case when IOs get reordered. A list is used to keep the order right. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/ctree.h | 10 +++- fs/btrfs/raid56.c | 63 -- fs

[PATCH 01/14] Btrfs: raid56: add raid56 log via add_dev v2 ioctl

| 48 - fs/btrfs/raid56.c | 42 ++++ fs/btrfs/raid56.h | 1 + fs/btrfs/volumes.c | 26 -- fs/btrfs/volumes.h | 3 ++- include/uapi/linux/btrfs.h

[PATCH 09/14] Btrfs: raid56: add readahead for recovery

While doing recovery, blocks are read from the raid5/6 disk one by one, so this is adding readahead so that we can read at most 256 contiguous blocks in one read IO. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 114 +++

[PATCH 14/14] Btrfs: raid56: maintain IO order on raid5/6 log

int offset where the highest successful writeback is, we cannot allow IO to be reordered. This is adding a list in which IO order is maintained properly. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 42 ++ fs/btrfs/raid56.h | 5

[PATCH 07/14] Btrfs: raid56: load r5log

if anything needs to be replayed, otherwise it creates a new empty block at the beginning of the disk and new writes will append to it. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/disk-io.c | 16 +++ fs/btrfs/raid56.c

[PATCH 05/14] Btrfs: raid56: add stripe log for raid5/6

with data/parity in a single bio(could be two bios, doesn't support more than two bios). Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 512 +++--- fs/btrfs/raid56.h | 65 +++ 2 files changed, 513 insertions(+), 64 del

[PATCH 08/14] Btrfs: raid56: log recovery

/parity. If a power loss happens, the new recovery starts again from block A, and since block B is now valid, it may replay block C as well which has become stale. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 151 ++

[PATCH 04/14] Btrfs: raid56: add verbose debug

Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 2 ++ fs/btrfs/volumes.c | 7 ++- fs/btrfs/volumes.h | 4 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 2b91b95..c75766f 100644 --- a/fs/btrfs/raid56.c

[PATCH 13/14] Btrfs: raid56: initialize raid5/6 log after adding it

We need to initialize the raid5/6 log after adding it, but we don't want to race with concurrent writes. So we initialize it before assigning the log pointer in %fs_info. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/disk-io.c | 2 +- fs/btrfs/raid56.c | 18

[PATCH 03/14] Btrfs: raid56: detect raid56 log on mount

We've put the flag BTRFS_DEV_RAID56_LOG in device->type, so we can recognize the journal device of raid56 while reading the chunk tree. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 12 1 file changed, 12 insertions(+) diff --git a/fs/btrfs/volumes.c

[PATCH 02/14] Btrfs: raid56: do not allocate chunk on raid56 log

The journal device (aka raid56 log) is not for chunk allocation, lets skip it. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index dafc541..5c50df7 100644 ---

[PATCH 11/14] Btrfs: raid56: add csum support

block, we just skip this this meta/data/parity pair and move onto the next one. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 235 -- fs/btrfs/raid56.h | 4 + 2 files changed, 197 insertions(+), 42 deletions(-)

Re: [PATCH v2 5/5] btrfs: raid56: Use bio_counter to protect scrub

of calling btrfs_map_sblock(), until rbio endio. Liu Bo <bo.li@oracle.com> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 2 ++ fs/btrfs/scrub.c | 5 + 2 files changed, 7 insertions(+) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index

Re: [PATCH v2 5/5] btrfs: raid56: Use bio_counter to protect scrub

sblock(), until rbio endio. > > Liu Bo <bo.li@oracle.com> > Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> > --- > fs/btrfs/raid56.c | 2 ++ > fs/btrfs/scrub.c | 5 + > 2 files changed, 7 insertions(+) > > diff --git a/fs/btrfs/raid56.c b/fs/bt

[PATCH v2 5/5] btrfs: raid56: Use bio_counter to protect scrub

Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 2 ++ fs/btrfs/scrub.c | 5 + 2 files changed, 7 insertions(+) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 1571bf26dc07..3a083165400f 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -2642,6 +2642,7 @@

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

t; > > > > > > > > > > > > > At 03/17/2017 12:44 PM, Liu Bo wrote: > > > > > > > > On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: > > > > > > > > > Before this patch, btrfs raid56 will keep raid5

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

wrote: At 03/18/2017 10:03 AM, Liu Bo wrote: On Fri, Mar 17, 2017 at 01:28:45PM +0800, Qu Wenruo wrote: At 03/17/2017 12:44 PM, Liu Bo wrote: On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

wrote: At 03/17/2017 12:44 PM, Liu Bo wrote: On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save some time allocating rbio, but it can cause deadly use-after-free bug, for the following case

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

Fri, Mar 17, 2017 at 01:28:45PM +0800, Qu Wenruo wrote: > > > > > > > > > > > > > > > At 03/17/2017 12:44 PM, Liu Bo wrote: > > > > > > On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: > > > > > > > Before

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

wrote: Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save some time allocating rbio, but it can cause deadly use-after-free bug, for the following case: Original fs: 4 devices RAID5 Process A | Process B

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: > > > > > Before this patch, btrfs raid56 will keep raid56 rbio even all its IO > > > > > is > > > > > done. > > > > > This may save some time allocating rbio, but it can ca

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

At 03/18/2017 10:03 AM, Liu Bo wrote: On Fri, Mar 17, 2017 at 01:28:45PM +0800, Qu Wenruo wrote: At 03/17/2017 12:44 PM, Liu Bo wrote: On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save

Re: [PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

2 new members, bad_ondisk_a/b, to struct btrfs_raid_bio, to info scrub code to use correct data pages to re-calculate parity. Reported-by: Goffredo Baroncelli <kreij...@inwind.it> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 62 +

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

On Fri, Mar 17, 2017 at 01:28:45PM +0800, Qu Wenruo wrote: > > > At 03/17/2017 12:44 PM, Liu Bo wrote: > > On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: > > > Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is > > > do

Re: [PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

> > > > -liubo > > > > >|- async_scrub_parity() > > > |- scrub_parity_work() (delayed_call to scrub_parity_work) > > > > > > scrub_parity_work() > > > |- raid56_parity_scrub_stripe() > > >|- validate_rbio_for_parity_scru

Re: [PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

: Goffredo Baroncelli <kreij...@inwind.it> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 62 +++ 1 file changed, 58 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index d

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

At 03/17/2017 12:44 PM, Liu Bo wrote: On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save some time allocating rbio, but it can cause deadly use-after-free bug, for the following case

Re: [PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

On Fri, Feb 03, 2017 at 04:20:22PM +0800, Qu Wenruo wrote: > Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is > done. > This may save some time allocating rbio, but it can cause deadly > use-after-free bug, for the following case: > > Original f

Re: [PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

re-calculate parity. Reported-by: Goffredo Baroncelli <kreij...@inwind.it> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 62 +++ 1 file changed, 58 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/raid56.c b/f

Re: [PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

new members, bad_ondisk_a/b, to struct > btrfs_raid_bio, to info scrub code to use correct data pages to > re-calculate parity. > > Reported-by: Goffredo Baroncelli <kreij...@inwind.it> > Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> > --- > fs/btrfs/raid56.c |

[PATCH 3/5] btrfs: raid56: Use correct stolen pages to calculate P/Q

rs, bad_ondisk_a/b, to struct btrfs_raid_bio, to info scrub code to use correct data pages to re-calculate parity. Reported-by: Goffredo Baroncelli <kreij...@inwind.it> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 62 ++

[PATCH 4/5] btrfs: raid56: Don't keep rbio for later steal

Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save some time allocating rbio, but it can cause deadly use-after-free bug, for the following case: Original fs: 4 devices RAID5 Process A | Process B

Re: [PATCH] btrfs: raid56: Remove unused variant in lock_stripe_add

At 01/21/2017 02:24 AM, David Sterba wrote: On Mon, Jan 16, 2017 at 10:23:06AM +0800, Qu Wenruo wrote: Variant 'walk' in lock_stripe_add() is never used. Remove it. Signed-off-by: Qu Wenruo Added to 4.11 queue, thaks. Changelog and subject edited. Thanks for

Re: [PATCH] btrfs: raid56: Remove unused variant in lock_stripe_add

On Mon, Jan 16, 2017 at 10:23:06AM +0800, Qu Wenruo wrote: > Variant 'walk' in lock_stripe_add() is never used. > Remove it. > > Signed-off-by: Qu Wenruo Added to 4.11 queue, thaks. Changelog and subject edited. -- To unsubscribe from this list: send the line

[PATCH 1/2] btrfs: raid56: Don't keep rbio for later steal

Before this patch, btrfs raid56 will keep raid56 rbio even all its IO is done. This may save some time allocating rbio, but it can cause deadly use-after-free bug, for the following case: Original fs: 4 devices RAID5 Process A | Process B

[PATCH] btrfs: raid56: Remove unused variant in lock_stripe_add

Variant 'walk' in lock_stripe_add() is never used. Remove it. Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 453eefdcb591..b8ffd9ea7499 100644 --- a/fs/btrfs/raid56.c

[PATCH 3/3] btrfs: raid56: Use correct stolen pages to calculate P/Q

rs, bad_ondisk_a/b, to struct btrfs_raid_bio, to info scrub code to use correct data pages to re-calculate parity. Reported-by: Goffredo Baroncelli <kreij...@inwind.it> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> --- fs/btrfs/raid56.c | 62 ++

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Tue, 2016-11-29 at 08:35 +0100, Adam Borowski wrote: > I administer no real storage at this time, and got only 16 disks > (plus a few > disk-likes) to my name right now.  Yet in a ~2 months span I've seen > three > cases of silent data corruption I didn't meant to say we'd have no silent data

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Tue, Nov 29, 2016 at 02:52:47AM +0100, Christoph Anton Mitterer wrote: > On Mon, 2016-11-28 at 16:48 -0500, Zygo Blaxell wrote: > > If a drive's embedded controller RAM fails, you get corruption on the > > majority of reads from a single disk, and most writes will be corrupted > > (even if they

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Tue, Nov 29, 2016 at 02:52:47AM +0100, Christoph Anton Mitterer wrote: > On Mon, 2016-11-28 at 16:48 -0500, Zygo Blaxell wrote: > > If a drive's > > embedded controller RAM fails, you get corruption on the majority of > > reads from a single disk, and most writes will be corrupted (even if > >

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Mon, 2016-11-28 at 16:48 -0500, Zygo Blaxell wrote: > If a drive's > embedded controller RAM fails, you get corruption on the majority of > reads from a single disk, and most writes will be corrupted (even if > they > were not before). Administrating a multi-PiB Tier-2 for the LHC Computing

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Mon, Nov 28, 2016 at 07:32:38PM +0100, Goffredo Baroncelli wrote: > On 2016-11-28 04:37, Christoph Anton Mitterer wrote: > > I think for safety it's best to repair as early as possible (and thus > > on read when a damage is detected), as further blocks/devices may fail > > till eventually a

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 2016-11-28 14:01, Christoph Anton Mitterer wrote: On Mon, 2016-11-28 at 19:45 +0100, Goffredo Baroncelli wrote: I am understanding that the status of RAID5/6 code is so badly Just some random thought: If the code for raid56 is really as bad as it's often claimed (I haven't read it, to be

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Mon, 2016-11-28 at 19:32 +0100, Goffredo Baroncelli wrote: > I am assuming that a corruption is a quite rare event. So > occasionally it could happens that a page is corrupted and the system > corrects it. This shouldn't  have an impact on the workloads. Probably, but it still make sense to

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Mon, 2016-11-28 at 19:45 +0100, Goffredo Baroncelli wrote: > I am understanding that the status of RAID5/6 code is so badly Just some random thought: If the code for raid56 is really as bad as it's often claimed (I haven't read it, to be honest) could it perhaps make sense to consider to

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 2016-11-28 01:40, Qu Wenruo wrote: > > At 11/27/2016 07:16 AM, Goffredo Baroncelli wrote: >> On 2016-11-26 19:54, Zygo Blaxell wrote: >>> On Sat, Nov 26, 2016 at 02:12:56PM +0100, Goffredo Baroncelli wrote: On 2016-11-25 05:31, Zygo Blaxell wrote: >> [...] BTW Btrfs in RAID1

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 2016-11-28 04:37, Christoph Anton Mitterer wrote: > I think for safety it's best to repair as early as possible (and thus > on read when a damage is detected), as further blocks/devices may fail > till eventually a scrub(with repair) would be run manually. > > However, there may some

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Mon, 2016-11-28 at 06:53 +0300, Andrei Borzenkov wrote: > If you allow any write to filesystem before resuming from hibernation > you risk corrupted filesystem. I strongly believe that "ro" must be > really read-only You're aware that "ro" already doesn't mean "no changes to the block device"

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

28.11.2016 06:37, Christoph Anton Mitterer пишет: > On Sat, 2016-11-26 at 14:12 +0100, Goffredo Baroncelli wrote: >> I cant agree. If the filesystem is mounted read-only this behavior >> may be correct; bur in others cases I don't see any reason to not >> correct wrong data even in the read case.

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Sat, 2016-11-26 at 14:12 +0100, Goffredo Baroncelli wrote: > I cant agree. If the filesystem is mounted read-only this behavior > may be correct; bur in others cases I don't see any reason to not > correct wrong data even in the read case. If your ram is unreliable > you have big problem

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

At 11/27/2016 07:16 AM, Goffredo Baroncelli wrote: On 2016-11-26 19:54, Zygo Blaxell wrote: On Sat, Nov 26, 2016 at 02:12:56PM +0100, Goffredo Baroncelli wrote: On 2016-11-25 05:31, Zygo Blaxell wrote: [...] BTW Btrfs in RAID1 mode corrects the data even in the read case. So Have you

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Sun, Nov 27, 2016 at 12:16:34AM +0100, Goffredo Baroncelli wrote: > On 2016-11-26 19:54, Zygo Blaxell wrote: > > On Sat, Nov 26, 2016 at 02:12:56PM +0100, Goffredo Baroncelli wrote: > >> On 2016-11-25 05:31, Zygo Blaxell wrote: > [...] > >> > >> BTW Btrfs in RAID1 mode corrects the data even in

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 2016-11-26 19:54, Zygo Blaxell wrote: > On Sat, Nov 26, 2016 at 02:12:56PM +0100, Goffredo Baroncelli wrote: >> On 2016-11-25 05:31, Zygo Blaxell wrote: [...] >> >> BTW Btrfs in RAID1 mode corrects the data even in the read case. So > > Have you tested this? I think you'll find that it

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Sat, Nov 26, 2016 at 02:12:56PM +0100, Goffredo Baroncelli wrote: > On 2016-11-25 05:31, Zygo Blaxell wrote: > >>> Do you mean, read the corrupted data won't repair it? > >>> > >>> IIRC that's the designed behavior. > >> :O > >> > >> You are right... I was unaware of that > > This is

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 11/22/2016 07:26 PM, Qu Wenruo wrote: We're changing which pages we kmap() but not which ones we kunmap(). Can you please update the kunmap loop to use this pointers array? Also it looks like this kmap is never unmapped. Oh I forget that. I'll update it soon. Thanks! This reminds

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 2016-11-25 05:31, Zygo Blaxell wrote: >>> Do you mean, read the corrupted data won't repair it? >>> >>> IIRC that's the designed behavior. >> :O >> >> You are right... I was unaware of that > This is correct. > > Ordinary reads shouldn't touch corrupt data, they should only read > around

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Fri, Nov 25, 2016 at 03:40:36PM +1100, Gareth Pye wrote: > On Fri, Nov 25, 2016 at 3:31 PM, Zygo Blaxell > wrote: > > > > This risk mitigation measure does rely on admins taking a machine in this > > state down immediately, and also somehow knowing not to start

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On Fri, Nov 25, 2016 at 3:31 PM, Zygo Blaxell wrote: > > This risk mitigation measure does rely on admins taking a machine in this > state down immediately, and also somehow knowing not to start a scrub > while their RAM is failing...which is kind of an annoying

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

e-calculate parity. > >>> > >>> Reported-by: Goffredo Baroncelli <kreij...@inwind.it> > >>> Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> > >>> --- > >>> Thanks to the above hell of delayed function all and damn stup

[PATCH v3] btrfs: raid56: Use correct stolen pages to calculate P/Q

ris. v3: Fix a bug using un-initialized pages, which leads to parity corruption again. --- fs/btrfs/raid56.c | 62 +++ 1 file changed, 58 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index d016d4a..08ba9cb

[v2 PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

he scrub race I fixed in previous patches. No one is able to found it before, because scrub doesn't even work correctly before. At this pace, RAID56 will never be stable, what we can do is to fix one bug and expose another bug hidden by previous bug. --- fs/btrfs/raid56.c | 42 +

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

At 11/23/2016 02:58 AM, Chris Mason wrote: On 11/21/2016 03:50 AM, Qu Wenruo wrote: In the following situation, scrub will calculate wrong parity to overwrite correct one: RAID5 full stripe: Before | Dev 1 | Dev 2 | Dev 3 | | Data stripe 1 | Data stripe 2 |

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

On 11/21/2016 03:50 AM, Qu Wenruo wrote: In the following situation, scrub will calculate wrong parity to overwrite correct one: RAID5 full stripe: Before | Dev 1 | Dev 2 | Dev 3 | | Data stripe 1 | Data stripe 2 | Parity Stripe |

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

function all and damn stupid code >>> logical, such bug is quite hard to trace. >>> >>> The damn kernel scrub is already multi-thread, why do such meaningless >>> delayed function call again and again? >>> >>> What's wrong with singl

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

nlock_full_stripe() We really need to re-work the whole damn scrub code. Also, we need to enhance btrfs-progs to detect scrub problem(my submitted offline scrub is good enough for such usage), and tools to corrupt extents reliably to put it into xfstests test cases. RAID56 scrub code is

Re: [PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

submit_and_wait_bios() > > out: > unlock_full_stripe() > > We really need to re-work the whole damn scrub code. > > Also, we need to enhance btrfs-progs to detect scrub problem(my > submitted offline scrub is good enough for such usage), and tools to > corrup

[PATCH] btrfs: raid56: Use correct stolen pages to calculate P/Q

is good enough for such usage), and tools to corrupt extents reliably to put it into xfstests test cases. RAID56 scrub code is neither tested nor well-designed. --- fs/btrfs/raid56.c | 29 - 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b

Re: btrfs [raid56] stability

eady for use. > > Is this someone's priority? Is it planned for the next one,two or three > years coming? [I took the liberty of updating the title, since you're not really asking about btrfs stability in general, but about btrfs raid56 mode stability...] You ask a very good question..

[PATCH v3 2/3] Btrfs: raid56: simplify the parameter of nr_parity_stripes().

type. Then we can use it more easily. Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- fs/btrfs/raid56.h | 8 fs/btrfs/volumes.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/raid56

[PATCH v2 2/3] Btrfs: raid56: simplify the parameter of nr_parity_stripes().

type. Then we can use it more easily. Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- fs/btrfs/raid56.h | 8 fs/btrfs/volumes.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/raid56

Re: [PATCH v2 2/3] Btrfs: raid56: simplify the parameter of nr_parity_stripes().

...@jp.fujitsu.com Thanks, Satoru --- fs/btrfs/raid56.h | 8 fs/btrfs/volumes.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/raid56.h b/fs/btrfs/raid56.h index ea5d73b..68ac9d3 100644 --- a/fs/btrfs/raid56.h +++ b/fs/btrfs/raid56.h @@ -19,11

[PATCH v2 2/3] Btrfs: raid56: simplify the parameter of nr_parity_stripes().

type. Then we can use it more easily. Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com --- fs/btrfs/raid56.h | 8 fs/btrfs/volumes.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/raid56.h b/fs/btrfs/raid56.h index ea5d73b..68ac9d3 100644 --- a/fs

[PATCH v4 03/10] Btrfs, raid56: don't change bbio and raid_map

...@cn.fujitsu.com --- Changelog v1 - v4: - None. --- fs/btrfs/raid56.c | 42 +++--- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 66944b9..cb31cc6 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c

[PATCH v4 05/10] Btrfs, raid56: use a variant to record the operation type

a variant to record the operation type. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- Changelog v1 - v4: - None. --- fs/btrfs/raid56.c | 31 +-- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index c954537..4924388

[PATCH v4 09/10] Btrfs, raid56: fix use-after-free problem in the final device replace procedure on raid56

-replace.c | 4 ++-- fs/btrfs/raid56.c | 41 - fs/btrfs/raid56.h | 4 ++-- fs/btrfs/scrub.c | 2 +- fs/btrfs/volumes.c | 7 ++- 6 files changed, 45 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

  1   2   >