Re: [PATCH v2] btrfs: not a disk error if the bio_add_page fails

On Wed, Jan 03, 2018 at 09:52:43PM +0800, Anand Jain wrote: > bio_add_page() can fail for logical reasons as from the bio_add_page() > comments:- 'This will only fail if either > bio->bi_vcnt == bio->bi_max_vecs or it's a cloned bio.' Don't inc the > write error statistics for this. And set

Re: [PATCH] fstests: btrfs/158: reproduce a scrub bug on raid6 corruption

On Thu, Jan 04, 2018 at 05:05:34PM +0800, Eryu Guan wrote: > On Tue, Jan 02, 2018 at 01:35:00PM -0700, Liu Bo wrote: > > This is to reproduce a bug of scrub, with which scrub is unable to > > repair raid6 corruption as expected. > > > > The kernel side fixes are >

[PATCH 2/2] Btrfs: fix scrub to repair raid6 corruption

to rebuild raid6 correctly with more retries in Patch "Btrfs: make raid6 rebuild retry more"[1], this is to fix scrub to do the exactly same rebuild process. [1]: https://patchwork.kernel.org/patch/10091755/ Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/bt

[PATCH 1/2 RESEND] Btrfs: make raid6 rebuild retry more

, if there is one more failure besides the failure on which we're recovering, this can always work. The worst case is to retry as many times as the number of raid6 disks, but given the fact that such a scenario is really rare in practice, it's still acceptable. Signed-off-by: Liu Bo <bo.li@oracle.

[PATCH] fstests: btrfs/158: reproduce a scrub bug on raid6 corruption

This is to reproduce a bug of scrub, with which scrub is unable to repair raid6 corruption as expected. The kernel side fixes are Btrfs: make raid6 rebuild retry more Btrfs: fix scrub to repair raid6 corruption Signed-off-by: Liu Bo <bo.li@oracle.com> --- tests/btrfs/158

Re: Btrfs allow compression on NoDataCow files? (AFAIK Not, but it does)

On Wed, Dec 20, 2017 at 11:59:20PM +0300, Timofey Titovets wrote: > How reproduce: > touch test_file > chattr +C test_file > dd if=/dev/zero of=test_file bs=1M count=1 > btrfs fi def -vrczlib test_file > filefrag -v test_file > > test_file > Filesystem type is: 9123683e > File size of test_file

Re: [PATCH V3] Btrfs: enchanse raid1/10 balance heuristic

On Sat, Dec 30, 2017 at 11:32:04PM +0300, Timofey Titovets wrote: > Currently btrfs raid1/10 balancer bаlance requests to mirrors, > based on pid % num of mirrors. > > Make logic understood: > - if one of underline devices are non rotational > - Queue leght to underline devices > > By default

Re: [PATCH 09/10] Btrfs: add tracepoint for em's EEXIST case

On Sat, Dec 23, 2017 at 09:39:00AM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 21:07, Liu Bo wrote: > > On Fri, Dec 22, 2017 at 10:56:31AM +0200, Nikolay Borisov wrote: > >> > >> > >> On 22.12.2017 00:42, Liu Bo wrote: > >>> This is add

[PATCH] Btrfs: remove unused wait in btrfs_stripe_hash

In fact nobody is waiting on @wait's waitqueue, it can be safely removed. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/ctree.h | 1 - fs/btrfs/raid56.c | 10 -- 2 files changed, 11 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 13c260b..b2e09fe

Re: [PATCH] btrfs: fix refcount_t usage when deleting btrfs_delayed_nodes

counts before radix tree deletion. > btrfs_get_delayed_node() was the only path that was allowing refcounts > to go from zero to one. > Reviewed-by: Liu Bo <bo.li@oracle.com> -liubo > Signed-off-by: Chris Mason <c...@fb.com> > Fixes: 6de5f18e7b0da > cc: <st

Re: [PATCH 07/10] Btrfs: fix unexpected EEXIST from btrfs_get_extent

On Fri, Dec 22, 2017 at 02:10:45PM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 00:42, Liu Bo wrote: > > This fixes a corner case that is caused by a race of dio write vs dio > > read/write. > > > > dio write: > > [0, 32k) -> [0, 8k) + [8k, 32

Re: [PATCH 09/10] Btrfs: add tracepoint for em's EEXIST case

On Fri, Dec 22, 2017 at 10:56:31AM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 00:42, Liu Bo wrote: > > This is adding a tracepoint 'btrfs_handle_em_exist' to help debug the > > subtle bugs around merge_extent_mapping. > > In the next patch you are already maki

Re: [PATCH 04/10] Btrfs: extent map selftest: buffered write vs dio read

On Fri, Dec 22, 2017 at 09:51:24AM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 00:42, Liu Bo wrote: > > This test case simulates the racy situation of buffered write vs dio > > read, and see if btrfs_get_extent() would return -EEXIST. > > Isn't mixing dio/b

Re: [PATCH 03/10] Btrfs: add extent map selftests

On Fri, Dec 22, 2017 at 09:42:17AM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 00:42, Liu Bo wrote: > > We've observed that btrfs_get_extent() and merge_extent_mapping() could > > return -EEXIST in several cases, and they are caused by some racy > > condition

Re: [PATCH 01/10] Btrfs: add helper for em merge logic

On Fri, Dec 22, 2017 at 09:23:40AM +0200, Nikolay Borisov wrote: > > > On 22.12.2017 00:42, Liu Bo wrote: > > This is a prepare work for the following extent map selftest, which > > runs tests against em merge logic. > > > > Signed-off-by: Liu Bo <bo.li..

Re: [PATCH] btrfs: fix refcount_t usage when deleting btrfs_delayed_nodes

On Sat, Dec 16, 2017 at 08:42:51AM +0200, Nikolay Borisov wrote: > > > On 15.12.2017 21:58, Chris Mason wrote: > > refcounts have a generic implementation and an asm optimized one. The > > generic version has extra debugging to make sure that once a refcount > > goes to zero, refcount_inc won't

[PATCH 02/10] Btrfs: move extent map specific code to extent_map.c

These helpers are extent map specific, this moves them to extent_map.c. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/ctree.h | 2 - fs/btrfs/extent_map.c | 118 ++ fs/btrfs/extent_map.h | 2 + fs/btrfs/inode.c

[PATCH 09/10] Btrfs: add tracepoint for em's EEXIST case

This is adding a tracepoint 'btrfs_handle_em_exist' to help debug the subtle bugs around merge_extent_mapping. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/extent_map.c| 1 + include/trace/events/btrfs.h | 35 +++ 2 files chang

[PATCH 10/10] Btrfs: noinline merge_extent_mapping

In order to debug subtle bugs around merge_extent_mapping(), perf probe can be used to check the arguments, but sometimes merge_extent_mapping() got inlined by compiler and couldn't be probed. This is adding noinline attribute to merge_extent_mapping(). Signed-off-by: Liu Bo <bo

[PATCH 04/10] Btrfs: extent map selftest: buffered write vs dio read

This test case simulates the racy situation of buffered write vs dio read, and see if btrfs_get_extent() would return -EEXIST. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/tests/extent-map-tests.c | 73 +++ 1 file changed, 73 insertions(+)

[PATCH 08/10] Btrfs: add WARN_ONCE to detect unexpected error from merge_extent_mapping

This is a subtle case, so in order to understand the problem, it'd be good to know the content of existing and em when any error occurs. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/extent_map.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs

[PATCH 03/10] Btrfs: add extent map selftests

situations. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/Makefile | 2 +- fs/btrfs/tests/btrfs-tests.c | 3 + fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 202 ++ 4 files changed, 207 inse

[PATCH 06/10] Btrfs: fix incorrect block_len in merge_extent_mapping

gly, while for regular extent, em->block_len always equals to em->len, hence this sets em->block_len with em->len. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/extent_map.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent_map.c b/

[PATCH 01/10] Btrfs: add helper for em merge logic

This is a prepare work for the following extent map selftest, which runs tests against em merge logic. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/inode.c | 101 ++- 2 files changed, 58 insertions(

[PATCH 00/10] bugfixes and regression tests of btrfs_get_extent

ch 1-2 are some preparatory work. Patch 3-5 are regression tests for handling EEXIST from adding extent map. Patch 6-7 are fixing two bugs which can be reproduced by the above test cases. Patch 8-10 are debugging wise, so that we can know what happened easily. Liu Bo (10): Btrfs: add helper

[PATCH 05/10] Btrfs: extent map selftest: dio write vs dio read

This test case simulates the racy situation of dio write vs dio read, and see if btrfs_get_extent() would return -EEXIST. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/tests/extent-map-tests.c | 88 +++ 1 file changed, 88 insertions(+) diff

[PATCH 07/10] Btrfs: fix unexpected EEXIST from btrfs_get_extent

the existing em should be returned, otherwise, we try our best to merge candidate em with sibling ems to form a larger em. Reported-by: David Vallender <david.vallen...@landmark.co.uk> Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/extent_map.c | 25 ++--

Re: [PATCH] btrfs: Fix bio leak

now > is that the flush bio has a refcount of 2 and we only ever bio_put it > once, leaving it to hang indefinitely. Fix this by removing the extra > bio_get in __alloc_device. > Reviewed-by: Liu Bo <bo.li@oracle.com> Thanks, -liubo > Fixes: e0ae99941423 ("btrfs: preall

Re: [PATCH] btrfs: Remove pair of bio_get/put in btrfs_schedule_bio

On Tue, Dec 12, 2017 at 07:52:38PM +0100, David Sterba wrote: > On Mon, Dec 11, 2017 at 02:57:56PM -0800, Liu Bo wrote: > > On Mon, Dec 11, 2017 at 04:38:48PM +0200, Nikolay Borisov wrote: > > > This code was added in 492bb6deee34 ("Btrfs: Hold a reference on bios > >

Re: [PATCH] btrfs: Remove pair of bio_get/put in btrfs_schedule_bio

; after the submit_bio returns and the bio is completed. In this > particular instance this is not the case so there is no need to hold > an extra reference since we directly return. > Looks good. If possible, please also drop other unnecessary bio_get() in btrfs. Reviewed-by: Liu Bo

[PATCH v2] Btrfs: do not merge rbios if their fail stripe index are not identical

Since fail stripe index in rbio would be used to decide which algorithm reconstruction would be run, we cannot merge rbios if their's fail striped index are different, otherwise, one of the two reconstructions would fail. Signed-off-by: Liu Bo <bo.li@oracle.com> --- v2: Change to use i

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

On Sat, Dec 09, 2017 at 03:32:18PM +0200, Nikolay Borisov wrote: > > > On 9.12.2017 01:02, Liu Bo wrote: > > We're not allowed to take any new bios to rbio->bio_list in > > rbio_orig_end_io(), otherwise we may get merged with more bios and > > rbio->bio_list i

Re: [PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

(Add Jérôme Carretero.) Thanks, -liubo On Fri, Dec 08, 2017 at 04:02:35PM -0700, Liu Bo wrote: > We're not allowed to take any new bios to rbio->bio_list in > rbio_orig_end_io(), otherwise we may get merged with more bios and > rbio->bio_list is not empty. > > This should

[PATCH] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

isable merge before doing IO. Reported-by: Jérôme Carretero <cj...@zougloub.eu> Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 5aa9d

Re: [PATCH 3/3] Btrfs: make raid6 rebuild retry more

On Wed, Dec 06, 2017 at 08:11:30AM +0800, Qu Wenruo wrote: > > > On 2017年12月06日 06:55, Liu Bo wrote: > > On Tue, Dec 05, 2017 at 07:09:25PM +0100, David Sterba wrote: > >> On Tue, Dec 05, 2017 at 04:07:35PM +0800, Qu Wenruo wrote: > >>>> @@ -2166,11 +2166

Re: [PATCH] fstests: btrfs: reproduce a read failure on raid6 setup

On Wed, Dec 06, 2017 at 04:56:11PM +0800, Eryu Guan wrote: > On Mon, Dec 04, 2017 at 03:33:23PM -0700, Liu Bo wrote: > > This test case is to reproduce a bug of raid6 reconstruction process. > > > > The kernel fix are > > Btrfs: do not merge rbios if their fail str

Re: [PATCH 3/3] Btrfs: make raid6 rebuild retry more

On Tue, Dec 05, 2017 at 07:09:25PM +0100, David Sterba wrote: > On Tue, Dec 05, 2017 at 04:07:35PM +0800, Qu Wenruo wrote: > > > @@ -2166,11 +2166,21 @@ int raid56_parity_recover(struct btrfs_fs_info > > > *fs_info, struct bio *bio, > > > } > > > > > > /* > > > - * reconstruct from the q

Re: [PATCH 3/3] Btrfs: make raid6 rebuild retry more

On Tue, Dec 05, 2017 at 04:07:35PM +0800, Qu Wenruo wrote: > > > On 2017年12月05日 06:40, Liu Bo wrote: > > There is a scenario that can end up with rebuild process failing to > > return good content, i.e. > > suppose that all disks can be read without problems and if th

[PATCH] Btrfs: remove useless wait in lock_stripe_add

The defined wait is not used anywhere. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 24a6222..62bcda2 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -670,7

[PATCH 2/3] Btrfs: do not merge rbios if their fail stripe index are not identical

Since fail stripe index in rbio would be used to decide which algorithm reconstruction would be run, we cannot merge rbios if their's fail striped index are different, otherwise, one of the two reconstructions would fail. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56

[PATCH 3/3] Btrfs: make raid6 rebuild retry more

, if there is one more failure besides the failure on which we're recovering, this can always work. The worst case is to retry as many times as the number of raid6 disks, but given the fact that such a scenario is really rare in practice, it's still acceptable. Signed-off-by: Liu Bo <bo.li@oracle.

[PATCH 1/3] Btrfs: remove redundant check in rbio_can_merge

Given the above ' if (last->operation != cur->operation) return 0; ', it's guaranteed that two operations are same. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/raid56.

[PATCH 0/3] Btrfs: loop retry on raid6 read failures

Patch 1 is a simple cleanup. Patch 2 fixes a bug in raid56 rbio merging code. Patch 3 fixes a bug in raid6 reconstruction process which can end up read failure when it can rebuild up good data. Liu Bo (3): Btrfs: remove redundant check in rbio_can_merge Btrfs: do not merge rbios if their fail

[PATCH] fstests: btrfs: reproduce a read failure on raid6 setup

This test case is to reproduce a bug of raid6 reconstruction process. The kernel fix are Btrfs: do not merge rbios if their fail stripe index are not identical Btrfs: make raid6 rebuild retry more Signed-off-by: Liu Bo <bo.li@oracle.com> --- tests/btrfs/155

[PATCH] Btrfs: use struct completion in scrub_submit_raid56_bio_wait

underneath us. So it can be removed safely. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/scrub.c | 21 - 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index e3f6c49..0cdf359 100644 --- a/fs/btrfs/scrub.c +++ b/fs

Re: [PATCH 0/7] retry write on error

On Wed, Nov 29, 2017 at 05:47:08PM +0100, David Sterba wrote: > On Wed, Nov 29, 2017 at 12:09:29PM +0800, Anand Jain wrote: > > On 11/29/2017 07:41 AM, p...@btrfs.list.sabi.co.uk wrote: > > If the underlying protocal doesn't support retry and there > > are some transient errors happening

Re: [PATCH 1/2] btrfs: submit a normal bio for the mirror read

On Wed, Nov 29, 2017 at 03:11:14PM +0800, Anand Jain wrote: > When the fist mirror failed to read we submit bio again to read from the > 2nd mirror, however during this, we also set the flag REQ_FAILFAST_DEV > which indicates to the underlying block device driver not to perform the > default IO

Re: [PATCH 0/7] retry write on error

On Tue, Nov 28, 2017 at 08:22:36PM +0100, David Sterba wrote: > On Tue, Nov 21, 2017 at 05:35:51PM -0700, Liu Bo wrote: > > If the underlying protocal doesn't support retry and there are some > > transient errors happening somewhere in our IO stack, we'd like to > > give a

Re: [PATCH 3/7] Btrfs: retry write for non-raid56

On Wed, Nov 22, 2017 at 04:41:10PM +0200, Nikolay Borisov wrote: > > > On 22.11.2017 02:35, Liu Bo wrote: > > If the underlying protocal doesn't support retry and there are some > > transient errors happening somewhere in our IO stack, we'd like to > >

Re: [PATCH] btrfs: handle dynamically reappearing missing device

On Fri, Nov 17, 2017 at 07:53:29PM +0800, Anand Jain wrote: > > > On 11/17/2017 07:28 AM, Liu Bo wrote: > > On Sun, Nov 12, 2017 at 06:56:50PM +0800, Anand Jain wrote: > > > If the device is not present at the time of (-o degrade) mount > > > the mount context w

Re: WARNING: CPU: 3 PID: 20953 at /usr/src/linux/fs/btrfs/raid56.c:848 __free_raid_bio+0x8e/0xa0

On Mon, Nov 20, 2017 at 02:00:07AM -0500, Jérôme Carretero wrote: > Hi, > > > > This was while doing a "userspace scrub" with "tar c": > > [633250.707455] btrfs_print_data_csum_error: 14608 callbacks suppressed > [633250.707459] BTRFS warning (device dm-18): csum failed root 5 ino 1376 off >

[PATCH 3/7] Btrfs: retry write for non-raid56

on errors _sector by sector_ in order to find out which sector is really bad so that we can mark it somewhere. And since endio is running in interrupt context, the real retry work is scheduled in system_unbound_wq in order to get a normal context. Signed-off-by: Liu Bo <bo.li@oracle.com> -

[PATCH 4/7] Btrfs: get rbio inside fail_bio_stripe

Because rbio will be bio->private in any case, we can remove @rbio in arguments. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 24a6222

[PATCH 2/7] Btrfs: add helper __btrfs_end_bio

parts need to be split. This add a helper __btrfs_end_bio() to do the second part. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/volumes.c | 60 +++--- 1 file changed, 34 insertions(+), 26 deletions(-) diff --git a/fs/btrfs/volumes.

[PATCH 0/7] retry write on error

changes it back to retry the whole bio. (I didn't fold patch 7 to patch 3 in the hope of just reverting patch 7 once badblocks support is done, but I'm open to it.) Liu Bo (7): Btrfs: keep a copy of bi_iter in btrfs_io_bio Btrfs: add helper __btrfs_end_bio Btrfs: retry write for non-raid56

[PATCH 7/7] Btrfs: retry the whole bio on write error

Since we haven't included badblocks support in our retry path, instead of retry sector by sector only retry the whole bio to give it an extra chance. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/volumes.c | 72 ++ 1 file c

[PATCH 6/7] Btrfs: retry write for raid56

Retry writes on raid56's final full stripe writes in order to get over some transient errors in our IO stack. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 42 -- 1 file changed, 40 insertions(+), 2 deletions(-) diff --gi

[PATCH 5/7] Btrfs: add helper __raid_write_end_io

We'd like to retry write on errors, so this splits raid_write_end_io() to two parts, a) checking errors and b) the rest in a helper __raid_write_end_io(). Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 23 ++- 1 file changed, 14 insertions

[PATCH 1/7] Btrfs: keep a copy of bi_iter in btrfs_io_bio

and normal bio in the same way. This keeps a copy of bi_iter in btrfs_io_bio before we submit bio so that in endio we can iter the bio with bio_for_each_segment() only. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/extent_io.c | 2 -- fs/btrfs/volumes.c | 3 ++- 2 files chan

[PATCH v2] Btrfs: fix list_add corruption and soft lockups in fsync

esting against his fsync error patch set[1]. [1]: https://www.spinics.net/lists/linux-btrfs/msg65308.html "btrfs list corruption and soft lockups while testing writeback error handling" Fixes: 8407f553268a4611f254 ("Btrfs: fix data corruption after fast fsync and writeback error&qu

Re: [PATCH] Btrfs: fix list_add corruption and soft lockups in fsync

On Mon, Nov 20, 2017 at 06:34:34PM +0100, David Sterba wrote: > On Mon, Nov 13, 2017 at 10:57:24AM -0700, Liu Bo wrote: > > Xfstests btrfs/146 revealed this corruption, > > > > [ 58.138831] Buffer I/O error on dev dm-0, logical block 2621424, async > > page read >

Re: [PATCH v2] iomap: report collisions between directio and buffered writes to userspace

l be reflected to userspace > during the next fsync. Replace the WARN_ON with a ratelimited pr_crit > so that the developers have /some/ kind of breadcrumb to track down the > offending program(s) and file(s) involved. > Looks good to me, thanks for addressing the warning. Reviewed-by

Re: 4.13.12: kernel BUG at fs/btrfs/ctree.h:1802!

On Thu, Nov 16, 2017 at 01:45:51PM -0800, Marc MERLIN wrote: > On Thu, Nov 16, 2017 at 06:27:44PM +0100, Holger Hoffstätte wrote: > > On 11/16/17 18:07, Marc MERLIN wrote: > > > Sorry, was missing the kernel number in the subject, just fixed that. > > > > > > On Thu, Nov 16, 2017 at 09:04:45AM

Re: [PATCH 10/10] btrfs: rework end io for extent buffer reads

On Tue, Nov 14, 2017 at 04:56:56PM -0500, Josef Bacik wrote: > From: Josef Bacik > > Now that the only thing that keeps eb's alive is io_pages and it's > refcount we need to hold the eb ref for the entire end io call so we > don't get it removed out from underneath us. Also the

Re: [PATCH 09/10] Btrfs: kill the btree_inode

On Tue, Nov 14, 2017 at 04:56:55PM -0500, Josef Bacik wrote: > From: Josef Bacik > > In order to more efficiently support sub-page blocksizes we need to stop > allocating pages from pagecache for our metadata. Instead switch to using the > account_metadata* counters for making

Re: [PATCH 02/10] writeback: convert WB_WRITTEN/WB_DIRITED counters to bytes

en > the two of them, so convert them to count bytes written/dirtied, and allow the > metadata accounting stuff to change the counters as well. > Reviewed-by: Liu Bo <bo.li@oracle.com> Thanks, -liubo > Signed-off-by: Josef Bacik <jba...@fb.com> > Acked-by:

Re: [PATCH 01/10] remove mapping from balance_dirty_pages*()

back is enabled on an inode > + * @inode: inode of interest > + * > + * Does the inode have cgroup writeback support. > + */ > +static inline bool inode_cgwb_enabled(struct inode *inode) > +{ > + return bdi_cgwb_enabled(inode_to_bdi(inode), inode->i_sb); > } > > /** &g

Re: [PATCH 6/6] btrfs: reoder btrfs_transaction members for better packing

On Wed, Nov 15, 2017 at 06:42:06PM +0100, David Sterba wrote: > There are now 20 bytes of holes, we can reduce that to 4 by minor > changes. Moving 'aborted' to the status and flags is also more logical, > similar for num_dirty_bgs. The size goes from 432 to 416. > Reviewed-by: L

Re: [PATCH] btrfs: handle dynamically reappearing missing device

On Sun, Nov 12, 2017 at 06:56:50PM +0800, Anand Jain wrote: > If the device is not present at the time of (-o degrade) mount > the mount context will create a dummy missing struct btrfs_device. > Later this device may reappear after the FS is mounted. This commit log doesn't explain what would

Re: [PATCH v2 1/3] btrfs: tree-checker: Fix false panic for sanity test

and > fallback to old btrfs_check_leaf() behavior. > > So we can still get early warning if we screw up item pointers, and > avoid false panic. > Reviewed-by: Liu Bo <bo.li@oracle.com> Thanks, -liubo > Cc: Filipe Manana <fdman...@gmail.com> >

Re: [PATCH v2 2/3] btrfs: tree-checker: Add checker for dir item

On Wed, Nov 08, 2017 at 09:59:39AM +0200, Nikolay Borisov wrote: > > > On 8.11.2017 02:54, Qu Wenruo wrote: > > Add checker for dir item, for key types DIR_ITEM, DIR_INDEX and > > XATTR_ITEM. > > > > This checker does comprehensive check for: > > 1) dir_item header and its data size > >

[PATCH v2] fstests: btrfs/143: make test case more reliable

ori...@suse.com> Signed-off-by: Liu Bo <bo.li@oracle.com> --- v2: Change 'fadvise -d' to _scratch_cycle_mount. To Nikolay, I didn't add your tested-by, but could you please verify if this also works on your test box? tests/btrfs/143 | 20 ++-- 1 file changed, 10 inser

[PATCH v3] Btrfs: avoid losing data raid profile when deleting a device

chunk. Metadata/System chunk are supposed to have non-zero bytes all the time so their raid profile is preserved. Reported-by: James Alandt <james.ala...@wdc.com> Signed-off-by: Liu Bo <bo.li@oracle.com> --- v3: More details about how losing data profile happens. v2: - return

[PATCH v2] Btrfs: set plug for fsync

disk so that we can save several disk access. Moreover, fsync issues synchronous writes, so plug can really take effect. Signed-off-by: Liu Bo <bo.li@oracle.com> --- v2: Explain why setting plug makes sense. fs/btrfs/file.c | 10 ++ 1 file changed, 10 insertions(+) diff --gi

Re: [PATCH] btrfs: fix deadlock when writing out space cache

On Wed, Nov 15, 2017 at 04:20:52PM -0500, Josef Bacik wrote: > From: Josef Bacik <jba...@fb.com> > > If we fail to prepare our pages for whatever reason (out of memory in > our case) we need to make sure to drop the block_group->data_rwsem, > otherwise hilarity ensues

Re: btrfs list corruption and soft lockups while testing writeback error handling

= > > > > > > It may be that they are a contributing factor, but this smells more like > > > a bug down in btrfs. Let me know if you need other info: > > [ btrfs inode logging ] > > > (cc'ing Liu Bo since we were discussing this earlier this week) > > &

[PATCH] Btrfs: fix list_add corruption and soft lockups in fsync

esting against his fsync error patch set[1]. [1]: https://www.spinics.net/lists/linux-btrfs/msg65308.html "btrfs list corruption and soft lockups while testing writeback error handling" Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/file.c | 1 + fs/btrfs/tree-log.c | 2

[PATCH] Btrfs: set plug for fsync

disk so that we can save several disk access. Moreover, fsync issues synchronous writes, so plug can really take effect. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/file.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index e43da6c..0

Re: [PATCH 2/4] Btrfs: fix data corruption in raid6

On Thu, Nov 09, 2017 at 05:29:25PM +0800, Anand Jain wrote: > > > On 11/09/2017 03:53 AM, Liu Bo wrote: > >On Tue, Nov 07, 2017 at 04:32:55PM +0800, Anand Jain wrote: > >> > >> > >>On 11/02/2017 08:54 AM, Liu Bo wrote: > >>>Wit

Re: [PATCH 2/4] Btrfs: fix data corruption in raid6

On Tue, Nov 07, 2017 at 04:32:55PM +0800, Anand Jain wrote: > > > On 11/02/2017 08:54 AM, Liu Bo wrote: > >With raid6 profile, btrfs can end up with data corruption by the > >following steps. > > > >Say we have a 5 disks that are set up with raid6 profile, >

Re: [PATCH 1/4] Btrfs: introduce device flags

On Mon, Nov 06, 2017 at 05:40:25PM +0100, David Sterba wrote: > On Wed, Nov 01, 2017 at 06:54:02PM -0600, Liu Bo wrote: > > Here we have defined two flags, > > - Fautly > > - In_sync > > > > Currently only In_sync is in use, it only matters when device ser

[PATCH] Btrfs: add __init macro to btrfs init function

Adding __init macro gives kernel a hint that this function is only used during the initialization phase and its memory resources can be freed up after. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/compression.h | 2 +- fs/btrfs/ctree.h | 6 +++--- fs/btrfs/delayed-ref

Re: [PATCH RFC] Btrfs: remove max_active and thresh

On Tue, Oct 31, 2017 at 07:53:47AM +0800, Qu Wenruo wrote: > > > On 2017年10月31日 01:14, Liu Bo wrote: > > First and foremost, here are the problems we have right now, > > > > a) %thread_pool is configurable via mount option, however for those > >'work

Re: [PATCH] fstests: btrfs: add test case for raid6 reconstruction bug

On Thu, Nov 02, 2017 at 11:49:52PM +0800, Eryu Guan wrote: > On Wed, Nov 01, 2017 at 06:01:23PM -0600, Liu Bo wrote: > > This is to reproduce a raid6 reconstruction bug after two drives > > getting offline and online via hotplug. > > > > Signed-off-by: James A

[PATCH 1/4] Btrfs: introduce device flags

valid generation can be retained through several reboot, btrfs is able to detect the out-of-sync device. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/disk-io.c | 21 - fs/btrfs/volumes.c | 47 --- fs/btrfs/volumes.

[PATCH 4/4] Btrfs: change how we set In_sync

. This changes it to set In_sync flag after iterating all devices, at this point we have the latest device so we can report out-of-sync device precisely. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/volumes.c | 38 ++ 1 file changed, 18 insertions(

[PATCH 2/4] Btrfs: fix data corruption in raid6

might be stale if 'In_sync' has not been set. With this, we can build the correct data2 from parity P, Q and data1. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/raid56.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c

[PATCH 0/4] Fix raid6 reconstruction bug

://patchwork.kernel.org/patch/10037769/ Liu Bo (4): Btrfs: introduce device flags Btrfs: fix data corruption in raid6 Btrfs: make raid1 and raid10 be aware of device flag In_sync Btrfs: change how we set In_sync fs/btrfs/disk-io.c | 21 - fs/btrfs/raid56.c | 3 ++- fs/btrfs/volumes.c

[PATCH 3/4] Btrfs: make raid1 and raid10 be aware of device flag In_sync

If a device is not in_sync, avoid reading data from it as data on it might be stale. Although checksum can detect stale data so we won't return stale data to users , this helps us read the good copy directly. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/volumes.

[PATCH] fstests: btrfs: add test case for raid6 reconstruction bug

This is to reproduce a raid6 reconstruction bug after two drives getting offline and online via hotplug. Signed-off-by: James Alandt <james.ala...@wdc.com> Signed-off-by: Liu Bo <bo.li@oracle.com> --- tests/btrfs/152 | 121 ++

[PATCH] Btrfs: document rules about bio async submit

These rules have been hidden in several if-else and are not straightforward to follow, for example, dio submit hook's nocsum case has a bug , i.e. doing async submit instead of sync submit, which has been fixed recently. This is documenting the rules for reference. Signed-off-by: Liu Bo <bo

Re: [PATCH] Btrfs: bail out gracefully rather than BUG_ON

On Tue, Oct 31, 2017 at 10:23:30PM +0200, Nikolay Borisov wrote: > > > On 30.10.2017 19:14, Liu Bo wrote: > > If a file's DIR_ITEM key is invalid (due to memory errors) and gets > > written to disk, a future lookup_path can end up with kernel panic due > > to BUG_O

Re: [PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

On Mon, Oct 16, 2017 at 11:53:08AM +0300, Nikolay Borisov wrote: > > > On 13.10.2017 23:51, Liu Bo wrote: > > On Wed, Oct 11, 2017 at 10:38:50AM +0300, Nikolay Borisov wrote: > >> > >> > >> On 10.10.2017 20:53, Liu Bo wrote: > >>> We'

Re: [PATCH] Btrfs: bail out gracefully rather than BUG_ON

On Mon, Oct 30, 2017 at 11:14:38AM -0600, Liu Bo wrote: > If a file's DIR_ITEM key is invalid (due to memory errors) and gets > written to disk, a future lookup_path can end up with kernel panic due > to BUG_ON(). > > This gets rid of the BUG_ON(), meanwhile output the corrupted k

[PATCH RFC] Btrfs: remove max_active and thresh

And all fields for limiting max_active are removed, including current_active, limit_active, thresh etc. Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/async-thread.c | 155 +++ fs/btrfs/async-thread.h | 27 +++-- fs/btrfs/delayed-i

[PATCH] Btrfs: bail out gracefully rather than BUG_ON

If a file's DIR_ITEM key is invalid (due to memory errors) and gets written to disk, a future lookup_path can end up with kernel panic due to BUG_ON(). This gets rid of the BUG_ON(), meanwhile output the corrupted key and return ENOENT if it's invalid. Signed-off-by: Liu Bo <bo.li@oracle.

[PATCH] Btrfs: kill threshold for submit_workers

will be min(num_devices, thread_pool_size). [1]: raid1 example, primary bio /\ bio1 bio2 | | dev1 dev2 | | endio1 endio2 \/ endio Signed-off-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/disk-io.c | 7 +-- 1 file changed, 1 inser

Re: brtfs BUG and erroneous IO

On Thu, Oct 26, 2017 at 09:01:33PM +0200, Guillaume Bouchard wrote: > Hi. > > I have a btrfs filesystem on linux 4.13.9 mounted with this options: > > /dev/sda3 on / type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/) > /dev/sda3 on /nix/store type btrfs >

Re: What is the purpose of EXTENT_PAGE_MAPPED

On Tue, Oct 24, 2017 at 05:47:11AM -0500, Goldwyn Rodrigues wrote: > > EXTENT_PAGE_MAPPED gets set in set_page_extent_mapped(), but I don't see > it being cross checked anytime. What is the purpose of setting it? Please check commit d1310b2e0cd98eb1348553e69b73827b436dca7b, it was used to

Re: Kernel Oops / btrfs problems

On Wed, Oct 25, 2017 at 02:05:48PM +0200, andreas.bt...@diezwickers.de wrote: > Hi, > > I've had problems with a btrfs filesystem on a usb disk. I made a > successfull backup of all data and created the filesystem from scratch. > I'm not able to restore all backuped data because of a kernel oops.

<    1   2   3   4   5   6   7   8   9   10   >