Re: [PATCH 0/2] btrfs: btrfs_dec_test_*_ordered_extent() refactor

2020-12-18 Thread Qu Wenruo
On 2020/12/18 下午11:57, David Sterba wrote: On Fri, Dec 18, 2020 at 01:16:59PM +0800, Qu Wenruo wrote: This small patchset is btrfs_dec_test_*_ordered_extent() refactor during subpage RW support development. This is mostly to make btrfs_dev_test_* functions more human readable and prepare it

Re: [PATCH v2 06/18] btrfs: extent_io: make attach_extent_buffer_page() to handle subpage case

2020-12-18 Thread Qu Wenruo
On 2020/12/18 下午11:41, Josef Bacik wrote: On 12/17/20 7:44 PM, Qu Wenruo wrote: On 2020/12/18 上午12:00, Josef Bacik wrote: On 12/10/20 1:38 AM, Qu Wenruo wrote: For subpage case, we need to allocate new memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to

[PATCH 1/2] btrfs: make btrfs_dio_private::bytes to be u32

2020-12-17 Thread Qu Wenruo
btrfs_dio_private::bytes is only assigned from bio::bi_iter::bi_size, which is no larger than U32. Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs_inode.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index d9bf53d9ff90

[PATCH 2/2] btrfs: refactor btrfs_dec_test_* functions for ordered extents

2020-12-17 Thread Qu Wenruo
w the return value is calculated The most anti-human part of the return value is: if (...) ret = 1; ... return ret == 0; This means, when we set ret to 1, the function returns 0. Change the local variable name to @finished, and directly return the value of it. Signed-off-b

[PATCH 0/2] btrfs: btrfs_dec_test_*_ordered_extent() refactor

2020-12-17 Thread Qu Wenruo
have one or more ordered extents for one bvec. Qu Wenruo (2): btrfs: make btrfs_dio_private::bytes to be u32 btrfs: refactor btrfs_dec_test_* functions for ordered extents fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/inode.c| 7 ++- fs/btrfs/ordered-data.c | 98

Re: [PATCH v2 07/18] btrfs: extent_io: make grab_extent_buffer_from_page() to handle subpage case

2020-12-17 Thread Qu Wenruo
On 2020/12/18 上午12:02, Josef Bacik wrote: On 12/10/20 1:38 AM, Qu Wenruo wrote: For subpage case, grab_extent_buffer_from_page() can't really get an extent buffer just from btrfs_subpage. Although we have btrfs_subpage::tree_block_bitmap, which can be used to grab the bytenr of an exi

Re: [PATCH v2 06/18] btrfs: extent_io: make attach_extent_buffer_page() to handle subpage case

2020-12-17 Thread Qu Wenruo
On 2020/12/18 上午12:00, Josef Bacik wrote: On 12/10/20 1:38 AM, Qu Wenruo wrote: For subpage case, we need to allocate new memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to return int    To indicate allocation failure - Prealloc page->private

Re: [PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

2020-12-17 Thread Qu Wenruo
On 2020/12/17 下午10:51, Josef Bacik wrote: On 12/16/20 11:57 PM, Qu Wenruo wrote: [BUG] With current subpage RW patchset, the following script can lead to filesystem hang:    # mkfs.btrfs -f -s 4k $dev    # mount $dev -o nospace_cache $mnt    # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

Re: [PATCH v2 07/18] btrfs: extent_io: make grab_extent_buffer_from_page() to handle subpage case

2020-12-16 Thread Qu Wenruo
On 2020/12/10 下午11:39, Nikolay Borisov wrote: On 10.12.20 г. 8:38 ч., Qu Wenruo wrote: For subpage case, grab_extent_buffer_from_page() can't really get an extent buffer just from btrfs_subpage. Although we have btrfs_subpage::tree_block_bitmap, which can be used to grab the bytenr

Re: [PATCH v2 06/18] btrfs: extent_io: make attach_extent_buffer_page() to handle subpage case

2020-12-16 Thread Qu Wenruo
On 2020/12/10 下午11:30, Nikolay Borisov wrote: On 10.12.20 г. 8:38 ч., Qu Wenruo wrote: For subpage case, we need to allocate new memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to return int To indicate allocation failure - Prealloc page->private

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

2020-12-16 Thread Qu Wenruo
On 2020/12/17 下午1:59, Nikolay Borisov wrote: On 17.12.20 г. 7:55 ч., Nikolay Borisov wrote: On 17.12.20 г. 6:57 ч., Qu Wenruo wrote: In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Remove such variable shadowing which can be very confusing. You

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

2020-12-16 Thread Qu Wenruo
On 2020/12/17 下午1:38, Su Yue wrote: On Thu 17 Dec 2020 at 12:57, Qu Wenruo wrote: In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Remove such variable shadowing which can be very confusing. Signed-off-by: Qu Wenruo ---  fs/btrfs/inode.c | 9

[PATCH 1/4] btrfs: inode: use min() to replace open-code in btrfs_invalidatepage()

2020-12-16 Thread Qu Wenruo
In btrfs_invalidatepage() we introduce a temporary variable, new_len, to update ordered->truncated_len. But we can use min() to replace it completely and no need for the variable. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) d

[PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

2020-12-16 Thread Qu Wenruo
make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 23 --- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index eb493fbb65f9..

[PATCH 3/4] btrfs: inode: move the timing of TestClearPagePrivate() in btrfs_invalidatepage()

2020-12-16 Thread Qu Wenruo
skipped, and cause never-end ordered io. This patch will move the TestClearPagePrivate2 before the loop, and save the result, so that we can finish all ordered extents. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/btrfs

[PATCH 0/4] btrfs: inode: btrfs_invalidatepage() related refactor and fix for subpage

2020-12-16 Thread Qu Wenruo
This small patchset contains 3 refactors which are subpage independent. And the last patch is RFC where I'm not certain about the existing code, but it solves the problem for subpage during test. Thus I'm here asking for help on the btrfs_invalidatepage() behavior. Qu Wenruo (4): bt

[PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

2020-12-16 Thread Qu Wenruo
In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Remove such variable shadowing which can be very confusing. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs

Re: csum errors on negative root id?

2020-12-16 Thread Qu Wenruo
On 2020/12/17 上午3:28, Ross Vandegrift wrote: Hello, I recently saw these in my dmesg, on 4.19.160: BTRFS warning (device dm-20): csum failed root -9 ino 1303 off 0 csum 0x47abffd5 expected csum 0xd5f0d335 mirror 1 BTRFS warning (device dm-20): csum failed root -9 ino 1303 off 0 csum 0

[PATCH v3 1/2] btrfs: volumes: Use more straightforward way to calculate map length

2019-10-23 Thread Qu Wenruo
length = min_t(u64, em->start + em->len - logical, length); Use above code to make the length calculate independent from other variable, thus slightly increase the readability. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) d

[PATCH v3 2/2] btrfs: extent-tree: Ensure we trim ranges across block group boundary

2019-10-23 Thread Qu Wenruo
sue correct trim for each range. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 41 +++-- fs/btrfs/volumes.c | 6 -- 2 files changed, 35 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c in

[PATCH v3 0/2] btrfs: trim: Fix a bug certain range may not be trimmed properly

2019-10-23 Thread Qu Wenruo
eadability improvement patch into its own patch v3: - Reset @num_bytes for btrfs_map_block() to limit the trim range inside the original range. Or we can trim into existing extent to corrupt data. Qu Wenruo (2): btrfs: volumes: Use more straightforward way to calculate map length btrfs: extent-tree:

Re: [PATCH v2 2/2] btrfs: extent-tree: Ensure we trim ranges across block group boundary

2019-10-23 Thread Qu Wenruo
On 2019/10/23 下午9:37, Nikolay Borisov wrote: > > > On 23.10.19 г. 15:56 ч., Qu Wenruo wrote: >> [BUG] >> When deleting large files (which cross block group boundary) with discard >> mount option, we find some btrfs_discard_extent() calls only trimmed part >>

[PATCH v2 2/2] btrfs: extent-tree: Ensure we trim ranges across block group boundary

2019-10-23 Thread Qu Wenruo
nd issue correct trim for each range. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 40 ++-- fs/btrfs/volumes.c | 6 -- 2 files changed, 34 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c in

[PATCH v2 0/2] btrfs: trim: Fix a bug certain range may not be trimmed properly

2019-10-23 Thread Qu Wenruo
eadability improvement patch into its own patch Qu Wenruo (2): btrfs: volumes: Use more straightforward way to calculate map length btrfs: extent-tree: Ensure we trim ranges across block group boundary fs/btrfs/extent-tree.c | 40 ++-- fs/btrfs/volumes.c | 8 +

[PATCH v2 1/2] btrfs: volumes: Use more straightforward way to calculate map length

2019-10-23 Thread Qu Wenruo
length = min_t(u64, em->start + em->len - logical, length); Use above code to make the length calculate independent from other variable, thus slightly increase the readability. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) d

Re: [PATCH 1/2] btrfs: volumes: Return the mapped length for discard

2019-10-23 Thread Qu Wenruo
On 2019/10/23 下午5:47, Filipe Manana wrote: > On Wed, Oct 23, 2019 at 9:53 AM Qu Wenruo wrote: >> > > Hi Qu, > >> For btrfs_map_block(), if we pass @op == BTRFS_MAP_DISCARD, the @length >> parameter will not be updated to reflect the real mapped length. >>

[PATCH 2/2] btrfs: extent-tree: Ensure we trim ranges across block group boundary

2019-10-23 Thread Qu Wenruo
ng trim needs to delete large extents at block group boundary without empting involved block groups. [FIX] Fix this bug by calling btrfs_map_block() in a loop, until we reach the end location. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 40 ++-- 1

[PATCH 1/2] btrfs: volumes: Return the mapped length for discard

2019-10-23 Thread Qu Wenruo
th. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index cdd7af424033..f66bd0d03f44 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5578,12 +5578,13 @@ void btr

[PATCH 0/2] btrfs: trim: Fix a bug certain range may not be trimmed properly

2019-10-23 Thread Qu Wenruo
. This patchset will fix it by ensuring btrfs_discard_extent() will iterate the full range before exiting. Meanwhile I'm still looking into how to craft such test case for btrfs, so the test case may be late for several days. Qu Wenruo (2): btrfs: volumes: Return the mapped length for di

[PATCH] btrfs: Remove btrfs_bio::flags member

2019-10-23 Thread Qu Wenruo
ember. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.h | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 7f6aa1816409..5ab2920b0c1f 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -331,7 +331,6 @@ struct btrfs_bio { u64 map_type; /* get

Re: first it froze, now the (btrfs) root fs won't mount ...

2019-10-22 Thread Qu Wenruo
On 2019/10/23 上午6:56, Christian Pernegger wrote: > [Please CC me, I'm not on the list.] > > Am Mo., 21. Okt. 2019 um 15:34 Uhr schrieb Qu Wenruo : >> [...] just fstrim wiped some old tree blocks. But maybe it's some >> unfortunate race, that fstrim trimme

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-22 Thread Qu Wenruo
On 2019/10/22 下午8:23, David Sterba wrote: > On Tue, Oct 22, 2019 at 02:30:22PM +0800, Qu Wenruo wrote: >> BTW, there is one important compatibility problem related to all the BGI >> related features. >> >> Although I'm putting the BG_TREE feature as incompatib

Re: Effect of punching holes

2019-10-22 Thread Qu Wenruo
On 2019/10/22 下午5:47, Tobias Reinhard wrote: > Hi, > > > I noticed that if you punch a hole in the middle of a file the available > filesystem space seems not to increase. > > Kernel is 5.2.11 > > To reproduce: > > ->mkfs.btrfs /dev/loop1 -f > > btrfs-progs v4.15.1 > See http://btrfs.wiki.k

Re: [PATCH 2/2] fstests: btrfs: dm-logwrites test for fstrim and fsstress workload

2019-10-22 Thread Qu Wenruo
On 2019/10/22 下午5:09, Amir Goldstein wrote: > On Tue, Oct 22, 2019 at 11:00 AM Qu Wenruo wrote: >> >> There is a fs corruption report of a tree block in use get trimmed, and >> cause fs corruption. >> >> Although I haven't found the cause from the source c

[PATCH 1/2] fstests: log-writes: Add new discard check point

2019-10-22 Thread Qu Wenruo
Despite the existing |fua|flush checkpoint, add a new discard check point to make sure discard is not screwing up things. Signed-off-by: Qu Wenruo --- src/log-writes/replay-log.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/log-writes/replay-log.c b/src/log

[PATCH 0/2] btrfs: dm-log-writes test for fstrim with fsstress

2019-10-22 Thread Qu Wenruo
workaround to avoid false alert. So far I haven't triggered a bug. Qu Wenruo (2): fstests: log-writes: Add new discard check point fstests: btrfs: dm-logwrites test for fstrim and fsstress workload src/log-writes/replay-log.c | 10 ++- tests/bt

[PATCH 2/2] fstests: btrfs: dm-logwrites test for fstrim and fsstress workload

2019-10-22 Thread Qu Wenruo
r fs can't go with the replay-log --check|--fsck hack as their fsck will report dirty journal as error. Signed-off-by: Qu Wenruo --- Due to recent change in btrfs side, already trimmed tree blocks won't get trimmed again until new data is written. This makes things safer, and I&#x

Re: [PATCH v2 1/2] btrfs-progs: warn users about the possible dangers of check --repair

2019-10-22 Thread Qu Wenruo
On 2019/10/22 下午3:33, Johannes Thumshirn wrote: > On 21/10/2019 17:22, David Sterba wrote: >> --force was added for a different reason, to allow check on a mounted >> filesystem. I don't think that combining --repair and --force just to >> allow repair is a good idea. There's a 'dangerous repair

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-21 Thread Qu Wenruo
tible. Now my question is, should we put this feature still as incompatible feature? Thanks, Qu On 2019/10/22 上午8:49, Qu Wenruo wrote: > > > On 2019/10/21 下午11:44, David Sterba wrote: >> On Sat, Oct 19, 2019 at 08:04:51AM +0800, Qu Wenruo wrote: >>> That's

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-21 Thread Qu Wenruo
On 2019/10/21 下午11:44, David Sterba wrote: > On Sat, Oct 19, 2019 at 08:04:51AM +0800, Qu Wenruo wrote: >> That's wonderful. >> Although I guess my patchset should provide the hint of where to modify >> the code, since there are only a limited number of places we

Re: [PATCH v3] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-21 Thread Qu Wenruo
On 2019/10/21 下午10:24, Steven Rostedt wrote: > On Mon, 21 Oct 2019 22:03:21 +0800 > Qu Wenruo wrote: > >> On 2019/10/21 下午9:56, Steven Rostedt wrote: >>> On Mon, 21 Oct 2019 17:47:30 +0800 >>> Qu Wenruo wrote: >>> >>>> +static void pr

Re: [PATCH v3] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-21 Thread Qu Wenruo
On 2019/10/21 下午9:56, Steven Rostedt wrote: > On Mon, 21 Oct 2019 17:47:30 +0800 > Qu Wenruo wrote: > >> +static void print_uuid_arg(struct trace_seq *s, void *data, int size, >> + struct tep_event *event, struct tep_print_arg *arg) >> +{

Re: first it froze, now the (btrfs) root fs won't mount ...

2019-10-21 Thread Qu Wenruo
On 2019/10/21 下午9:02, Christian Pernegger wrote: > [Please CC me, I'm not on the list.] > > Am Mo., 21. Okt. 2019 um 13:47 Uhr schrieb Austin S. Hemmelgarn > : >> I've [worked with fs clones] like this dozens of times on single-device >> volumes with exactly zero issues. > > Thank you, I have

Re: first it froze, now the (btrfs) root fs won't mount ...

2019-10-21 Thread Qu Wenruo
On 2019/10/21 下午6:47, Christian Pernegger wrote: > [Please CC me, I'm not on the list.] > > Am So., 20. Okt. 2019 um 12:28 Uhr schrieb Qu Wenruo : >>> Question: Can I work with the mounted backup image on the machine that >>> also contains the original disc?

[PATCH v3] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-21 Thread Qu Wenruo
e can at least print fsid correctly: 0.000 xfs_io/79619 btrfs:qgroup_meta_reserve:23ad1511-dd83-47d4-a79c-e96625a15a6e refroot=5(FS_TREE) type=0x0 diff=2 Signed-off-by: Qu Wenruo --- Changelog: v2: - Use more comment explaining the finetunings we skipped for %pU* - Extra check for the field bef

[PATCH 3/3] btrfs-progs: check: Introduce optional argument for -b|--backup

2019-10-21 Thread Qu Wenruo
() interface without introducing new parameters. - New arg_strol() helper to get negative int - New btrfs_fs_info::backup_gen_diff member Now btrfs_setup_all_roots() uses that member to search backup roots. Signed-off-by: Qu Wenruo --- Documentation/btrfs-check.asciidoc | 6 -- check

[PATCH 2/3] btrfs-progs: disk-io: Handle backup root more correctly

2019-10-21 Thread Qu Wenruo
t() can now return -1 to indicates error (no valid backup found), so change callers to co-operate. Signed-off-by: Qu Wenruo --- disk-io.c | 34 ++ 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/disk-io.c b/disk-io.c index be44eead5cef..36db1be264cd 1

[PATCH 0/3] btrfs-progs: check: Introduce optional argument for -b|--backup

2019-10-21 Thread Qu Wenruo
will introduce optional argument for -b|--backup, so user can specify which backup to use by providing the generation difference (-3, -2, -1). If the optional argument is not provided, the default value is -1, and the behavior should be pretty much the same. Qu Wenruo (3): btrfs-progs: utils-lib:

[PATCH 1/3] btrfs-progs: utils-lib: Use error() to replace fprintf(stderr, "ERROR: ")

2019-10-21 Thread Qu Wenruo
This saves several lines of code. Signed-off-by: Qu Wenruo --- utils-lib.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/utils-lib.c b/utils-lib.c index c2b6097f5df9..0202dd7677f0 100644 --- a/utils-lib.c +++ b/utils-lib.c @@ -23,8 +23,7 @@ u64 arg_strtou64(const

Re: [PATCH 0/3] btrfs-progs: Add check and repair for invalid inode generation

2019-10-20 Thread Qu Wenruo
On 2019/10/20 下午9:29, Ferry Toth wrote: > Op 20-10-2019 om 15:15 schreef Qu WenRuo: >> >> >> On 2019/10/20 下午9:04, Ferry Toth wrote: >>> Op 20-10-2019 om 02:51 schreef Qu Wenruo: >>>> >>>> >>>> On 2019/10/20 上午8:26, Qu Wenru

Re: [PATCH 0/3] btrfs-progs: Add check and repair for invalid inode generation

2019-10-20 Thread Qu WenRuo
On 2019/10/20 下午9:04, Ferry Toth wrote: > Op 20-10-2019 om 02:51 schreef Qu Wenruo: >> >> >> On 2019/10/20 上午8:26, Qu Wenruo wrote: >>> >>> >>> On 2019/10/20 上午12:24, Ferry Toth wrote: >>>> Hi, >>>> >>>> Op 19-10

Re: first it froze, now the (btrfs) root fs won't mount ...

2019-10-20 Thread Qu Wenruo
gle device fs, it should be mostly OK. Thanks, Qu > > Cheers, > Christian > > > Am So., 20. Okt. 2019 um 09:41 Uhr schrieb Qu Wenruo : >> >> >> >> On 2019/10/20 下午3:01, Christian Pernegger wrote: >>> [Please CC me, I'm not on the list

Re: [PATCH 0/3] btrfs-progs: Add check and repair for invalid inode generation

2019-10-19 Thread Qu Wenruo
On 2019/10/20 上午8:26, Qu Wenruo wrote: > > > On 2019/10/20 上午12:24, Ferry Toth wrote: >> Hi, >> >> Op 19-10-2019 om 01:50 schreef Qu WenRuo: >>> >>> >>> On 2019/10/19 上午4:32, Ferry Toth wrote: >>>> Op 24-09-2019 om 10:11 sc

Re: first it froze, now the (btrfs) root fs won't mount ...

2019-10-19 Thread Qu Wenruo
On 2019/10/20 上午6:34, Christian Pernegger wrote: > [Please CC me, I'm not on the list.] > > Hello, > > I'm afraid I could use some help. > > The affected machine froze during a game, was entirely unresponsive > locally, though ssh still worked. For completeness' sake, dmesg had: > [110592.1285

Re: [PATCH 0/3] btrfs-progs: Add check and repair for invalid inode generation

2019-10-19 Thread Qu Wenruo
On 2019/10/20 上午12:24, Ferry Toth wrote: > Hi, > > Op 19-10-2019 om 01:50 schreef Qu WenRuo: >> >> >> On 2019/10/19 上午4:32, Ferry Toth wrote: >>> Op 24-09-2019 om 10:11 schreef Qu Wenruo: >>>> We have at least two user reports about bad

Re: tree-checker: corrupt leaf: invalid mode: has 00 expect valid S_IF* bit(s)

2019-10-18 Thread Qu Wenruo
On 2019/10/19 上午1:18, ‍小太 wrote: > Hi, I recently got the following error in dmesg while taking an > incremental backup of my data: > > [26876.491167] BTRFS critical (device dm-0): corrupt leaf: root=258 > block=739531833344 slot=0 ino=19366566, invalid mode: has 00 expect > valid S_IF* bit(s)

Re: [PATCH 0/3] btrfs-progs: Add check and repair for invalid inode generation

2019-10-18 Thread Qu WenRuo
On 2019/10/19 上午4:32, Ferry Toth wrote: > Op 24-09-2019 om 10:11 schreef Qu Wenruo: >> We have at least two user reports about bad inode generation makes >> kernel reject the fs. > > May I add my report? I just upgraded Ubuntu from 19.04 -> 19.10 so > kernel went

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-18 Thread Qu Wenruo
On 2019/10/19 上午1:27, David Sterba wrote: > On Wed, Oct 16, 2019 at 11:19:54AM +0000, Qu WenRuo wrote: >>>> The most important aspect to me is, to allow real world user of super >>>> large fs to try this feature, to prove the usefulness of this design, >>

Re: [PATCH 1/2] btrfs-progs: warn users about the possible dangers of check --repair

2019-10-18 Thread Qu Wenruo
mshirn Looks good to me. Although I really hope I can soon remove this confirm soon enough. :) Reviewed-by: Qu wenruo Thanks, Qu > > --- > Changes to v1: > - Fix grammar mistakes in warning message > - Skip delay with --force > - Adjust tests to cope with btrfsck --repai

Re: [PATCH 5/5] btrfs: document extent buffer locking

2019-10-17 Thread Qu Wenruo
On 2019/10/18 上午3:39, David Sterba wrote: > Signed-off-by: David Sterba Great document. Some questions inlined below. > --- > fs/btrfs/locking.c | 110 +++-- > 1 file changed, 96 insertions(+), 14 deletions(-) > > diff --git a/fs/btrfs/locking.c b/fs/b

[PATCH] btrfs: qgroup: Automatically remove qgroup item when dropping a subvolume

2019-10-17 Thread Qu Wenruo
ction for later operation - Call btrfs_remove_qgroup() So that qgroup can be automatically removed when the subvolume get fully dropped. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 25 + 1 file changed, 25 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/

Re: [PATCH v2 2/7] btrfs-progs: Refactor btrfs_read_block_groups()

2019-10-16 Thread Qu Wenruo
On 2019/10/17 上午11:23, Anand Jain wrote: > On 10/8/19 12:49 PM, Qu Wenruo wrote: >> This patch does the following refactor: >> - Refactor parameter from @root to @fs_info >> >> - Refactor the large loop body into another function >>    Now we have a helper fun

Re: [PATCH v2 7/7] btrfs-progs: btrfstune: Allow to enable bg-tree feature offline

2019-10-16 Thread Qu Wenruo
kernel handle of migration >  of the bg items from the extent-tree to the bg-tree as and when the >  bg-items are written. > > Thanks, Anand > > > On 10/8/19 12:49 PM, Qu Wenruo wrote: >> Add a new option '-b' for btrfstune, to enable bg-tree feature for a >

Re: [PATCH v2] btrfs: qgroup: Fix wrong parameter order for trace events

2019-10-16 Thread Qu WenRuo
Please discard this patch. It's sent out by accident. The correct one is only sent to btrfs mail list. Thanks, Qu On 2019/10/17 上午10:23, Qu Wenruo wrote: > [BUG] > For btrfs:qgroup_meta_reserve event, the trace event can output garbage: > qgroup_meta_reserve: 9c7f6acc-

[PATCH v2 2/2] btrfs: qgroup: Fix bad entry members of qgroup trace events

2019-10-16 Thread Qu Wenruo
entry member Member entry::type is completely useless in trace_qgroup_meta_convert() [FIX] Fix these stupid bugs. Fixes: 4ee0d8832c2e ("btrfs: qgroup: Update trace events for metadata reservation") Signed-off-by: Qu Wenruo --- include/trace/events/btrfs.h | 3 ++- 1 file

[PATCH v2 1/2] btrfs: qgroup: Fix wrong parameter order for trace events

2019-10-16 Thread Qu Wenruo
r metadata reservation") Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index c4bb69941c77..3ad151655eb8 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -3629,7

[PATCH v2 0/2] btrfs: qgroup: Bug fixes for trace events

2019-10-16 Thread Qu Wenruo
Several stupid bugs exposed for qgroup related trace events. Fix them. Changelog: v2: - Split patches Qu Wenruo (2): btrfs: qgroup: Fix wrong parameter order for trace events btrfs: qgroup: Fix bad entry members of qgroup trace events fs/btrfs/qgroup.c| 4 ++-- include/trace

[PATCH v2] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-16 Thread Qu Wenruo
e can at least print fsid correctly: 0.000 xfs_io/79619 btrfs:qgroup_meta_reserve:23ad1511-dd83-47d4-a79c-e96625a15a6e refroot=5(FS_TREE) type=0x0 diff=2 Signed-off-by: Qu Wenruo --- Changelog: v2: - Use more comment explaining the finetunings we skipped for %pU* - Use more elegant way t

[PATCH v2] btrfs: qgroup: Fix wrong parameter order for trace events

2019-10-16 Thread Qu Wenruo
00844d46ba: refroot=5(FS_TREE) type=META_PREALLOC diff=-16384 Fixes: 4ee0d8832c2e ("btrfs: qgroup: Update trace events for metadata reservation") Signed-off-by: Qu Wenruo --- Changelog: v2: - Use more accurate comment about the fintuning options - Use more elegant method to output uuid --

Re: [PATCH] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-16 Thread Qu Wenruo
On 2019/10/16 下午10:54, Steven Rostedt wrote: > On Wed, 16 Oct 2019 14:39:20 +0800 > Qu Wenruo wrote: > >> +static void print_uuid_arg(struct trace_seq *s, void *data, int size, >> + struct tep_event *event, struct tep_print_arg *arg) >>

Re: [PATCH] btrfs-progs: warn users about the possible dangers of check --repair

2019-10-16 Thread Qu Wenruo
On 2019/10/16 下午10:05, Johannes Thumshirn wrote: > The manual page of btrfsck clearly states 'btrfs check --repair' is a > dangerous operation. > > Although this warning is in place users do not read the manual page and/or > are used to the behaviour of fsck utilities which repair the filesystem

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-16 Thread Qu WenRuo
On 2019/10/16 下午7:16, David Sterba wrote: > On Tue, Oct 15, 2019 at 08:32:30AM +0800, Qu Wenruo wrote: >>> Have we settled the argument whether to use a new tree or key tricks for >>> the blocgroup data? I think we have not and will read the previous >>> discussi

Re: kernel 5.2 read time tree block corruption

2019-10-16 Thread Qu Wenruo
On 2019/10/16 下午4:12, Roman Mamedov wrote: > On Wed, 16 Oct 2019 15:45:54 +0800 > Qu Wenruo wrote: > >> dirty fixes (a special branch for the user >> to do, never intended to upstream). > > Still it would be nice to get a btrfs check mode which would include trying &

Re: [PATCH] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-16 Thread Qu WenRuo
On 2019/10/16 下午3:39, Nikolay Borisov wrote: > > > On 16.10.19 г. 9:39 ч., Qu Wenruo wrote: >> [BUG] >> For btrfs related events, there is a field for fsid, but perf never >> parse it correctly. >> >> # perf trace -e btrfS:qgroup_meta_convert xfs_io

Re: kernel 5.2 read time tree block corruption

2019-10-16 Thread Qu Wenruo
On 2019/10/16 下午3:42, Nikolay Borisov wrote: > > > On 16.10.19 г. 3:43 ч., Qu Wenruo wrote: >> >> >> On 2019/10/16 上午1:55, José Luis wrote: >>> I also noticed the craziness of the previous dump. I cannot remember >>> the kernel running by thi

[PATCH] btrfs: qgroup: Fix wrong parameter order for trace events

2019-10-16 Thread Qu Wenruo
00844d46ba: refroot=5(FS_TREE) type=META_PREALLOC diff=-16384 Fixes: 4ee0d8832c2e ("btrfs: qgroup: Update trace events for metadata reservation") Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c| 4 ++-- include/trace/events/btrfs.h | 1 + 2 files changed, 3 insertions(+), 2 dele

[PATCH] tools/lib/traceevent, perf tools: Handle %pU format correctly

2019-10-15 Thread Qu Wenruo
e can at least print fsid correctly: 0.000 xfs_io/79619 btrfs:qgroup_meta_reserve:23ad1511-dd83-47d4-a79c-e96625a15a6e refroot=5(FS_TREE) type=0x0 diff=2 Signed-off-by: Qu Wenruo --- Please note in above case, the @type and @diff are not properly showed. That's another problem, will be a

Re: bad tree block start, want #### have 0

2019-10-15 Thread Qu Wenruo
On 2019/10/16 上午3:03, DrYak wrote: > Hello > > I'm having trouble on a BTRFS file system that I use on a Raspberry Pi. > I can still mount and access (nearly all) my data. > > The trouble origin itself is probably hardware (flacky USB3-to-mSATA > adapter and/or power stability), not necessarily

Re: kernel 5.2 read time tree block corruption

2019-10-15 Thread Qu Wenruo
block to delete them in a batch. Thanks, Qu > > Thanks Qu, > José Luis > > El mar., 15 oct. 2019 a las 15:25, Qu Wenruo > () escribió: >> >> >> >> On 2019/10/15 下午11:03, José Luis wrote: >>> Thanks for fast response Qu. >>> >>> I

Re: kernel 5.2 read time tree block corruption

2019-10-15 Thread Qu Wenruo
rameter to the grep redirection for: "btrfs ins > dump-tree -t 5 /dev/sdb2 | grep -A 7" My bad, the parameter is "(431" It will output all info about inode 431, so we can make sure what's going wrong. Thanks, Qu > > El mar., 15 oct. 2019 a las 14:38, Qu We

Re: kernel 5.2 read time tree block corruption

2019-10-15 Thread Qu Wenruo
On 2019/10/15 下午8:24, Qu Wenruo wrote: > > > On 2019/10/15 下午6:15, José Luis wrote: >> Dear devs, >> >> I cannot use kernel >= 5.2, They cannot mount sdb2 nor sb3 both btrfs >> filesystems. I can work as intended on 4.19 which is an LTS version, >> p

Re: kernel 5.2 read time tree block corruption

2019-10-15 Thread Qu Wenruo
On 2019/10/15 下午6:15, José Luis wrote: > Dear devs, > > I cannot use kernel >= 5.2, They cannot mount sdb2 nor sb3 both btrfs > filesystems. I can work as intended on 4.19 which is an LTS version, > previously using 5.1 but Manjaro removed it from their repositories. > > More info: > · dmesg: >

Re: [PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-14 Thread Qu Wenruo
On 2019/10/14 下午11:17, David Sterba wrote: > On Tue, Oct 08, 2019 at 12:49:29PM +0800, Qu Wenruo wrote: >> This patchset can be fetched from github: >> https://github.com/adam900710/btrfs-progs/tree/bg_tree >> Which is based on v5.2.2 tag. >> >> This patchs

[PATCH] btrfs: qgroup: Always free PREALLOC META reserve in btrfs_delalloc_release_extents()

2019-10-13 Thread Qu Wenruo
lease_extents(), and always pass true to btrfs_inode_rsv_release(). Reported-by: Filipe Manana Fixes: 43b18595d660 ("btrfs: qgroup: Use separate meta reservation type for delalloc") Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 3 +-- fs/btrfs/delalloc-space.c | 6 ++ fs/btrfs/f

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-10-13 Thread Qu Wenruo
On 2019/10/14 上午10:07, Adam Bahe wrote: >> Until the fix gets merged to 5.2 kernels (and 5.3), I don't really recommend >> running 5.2 or 5.3. > > I know fixes went in to distro specific kernels. But wanted to verify > if the fix went into the vanilla kernel.org kernel? If so, what > version sh

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Qu Wenruo
On 2019/10/10 下午11:06, Nikolay Borisov wrote: > The code responsible for reading and initilizing tree roots is > scattered in open_ctree among 2 labels, emulating a loop. This is rather > confusing to reason about. Instead, factor the code in a new function, > init_tree_roots which implements th

Re: [PULL REQUEST] btrfs-progs: For next merge window

2019-10-10 Thread Qu Wenruo
On 2019/10/11 上午12:17, David Sterba wrote: > On Thu, Oct 10, 2019 at 04:59:32PM +0800, Qu Wenruo wrote: >> This patchset can be fetched from github: >> https://github.com/adam900710/btrfs-progs/tree/for_next >> Which is based on devel branch, with the following HEAD: >

[PULL REQUEST] btrfs-progs: For next merge window

2019-10-10 Thread Qu Wenruo
al (3): btrfs-progs: receive: remove commented out transid checks btrfs-progs: receive: don't lookup clone root for received subvolume btrfs-progs: tests: add test for receiving clone from duplicate subvolume Qu Wenruo (15): btrfs-progs: image: Output error message for chunk tree

Re: [PATCH v2 0/4] btrfs-progs: fix clone from wrong subvolume

2019-10-10 Thread Qu Wenruo
On 2019/9/5 上午4:35, Omar Sandoval wrote: > On Mon, Jul 22, 2019 at 12:15:01PM -0700, Omar Sandoval wrote: >> From: Omar Sandoval >> >> This is v2 of [1]. Changes from v1: >> >> - Split out removing commented-out code to new patch 1 and removed a >> related comment block. >> - Made subvol_path

[PATCH v3 2/7] btrfs-progs: Refactor btrfs_read_block_groups()

2019-10-09 Thread Qu Wenruo
have the code handling ret > 0 from find_first_block_group(), it never works, as when there is no more block group, find_first_block_group() just return -ENOENT other than 1. This is super confusing, it's almost a mircle it even works. Signed-off-by: Qu Wenruo --- ctree.h

[PATCH v3 7/7] btrfs-progs: btrfstune: Allow to enable bg-tree feature offline

2019-10-09 Thread Qu Wenruo
Add a new option '-b' for btrfstune, to enable bg-tree feature for a unmounted fs. This feature will convert all BLOCK_GROUP_ITEMs in extent tree to bg tree, by reusing the existing btrfs_convert_to_bg_tree() function. Signed-off-by: Qu Wenruo --- Documentation/btrfstune.asc

[PATCH v3 3/7] btrfs-progs: Enable read-write ability for 'bg_tree' feature

2019-10-09 Thread Qu Wenruo
feature. This modification needs extra handling for converting case, where block group items can be either in extent tree or bg tree. 4) free_block_group_item() Just delete the block group item in extent tree. Signed-off-by: Qu Wenruo --- ctree.h | 11 +++- disk-io.c |

[PATCH v3 4/7] btrfs-progs: mkfs: Introduce -O bg-tree

2019-10-09 Thread Qu Wenruo
n mkfs after the basic fs is created. Signed-off-by: Qu Wenruo --- common/fsfeatures.c | 6 ++ ctree.h | 1 + extent-tree.c | 39 +++ mkfs/common.c | 6 -- mkfs/main.c | 25 + transaction.c

[PATCH v3 1/7] btrfs-progs: Refactor excluded extent functions to use fs_info

2019-10-09 Thread Qu Wenruo
The following functions are just using @root to reach fs_info: - exclude_super_stripes - free_excluded_extents - add_excluded_extent Refactor them to use fs_info directly. Signed-off-by: Qu Wenruo Reviewed-by: Johannes Thumshirn --- check/main.c | 4 ++-- ctree.h | 4 ++-- extent

[PATCH v3 5/7] btrfs-progs: dump-tree/dump-super: Introduce support for bg tree

2019-10-09 Thread Qu Wenruo
Just a new tree called BLOCK_GROUP_TREE. Signed-off-by: Qu Wenruo --- cmds/inspect-dump-super.c | 3 ++- cmds/inspect-dump-tree.c | 5 + print-tree.c | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/cmds/inspect-dump-super.c b/cmds/inspect-dump-super.c

[PATCH v3 6/7] btrfs-progs: check: Introduce support for bg-tree feature

2019-10-09 Thread Qu Wenruo
Just some minor modification. - original mode: * Block group item can occur in extent tree and bg tree. - lowmem mode: * search block group items in bg tree if BG_TREE feature is set. Signed-off-by: Qu Wenruo --- check/main.c| 3 ++- check/mode-lowmem.c | 9 +++-- 2 files

[PATCH v3 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-09 Thread Qu Wenruo
buffer get leaked This is caused by newly created bg tree not added to dirty list. Qu Wenruo (7): btrfs-progs: Refactor excluded extent functions to use fs_info btrfs-progs: Refactor btrfs_read_block_groups() btrfs-progs: Enable read-write ability for 'bg_tree' feature btrfs-progs: m

[PATCH] fstests: btrfs/15[78]: Detect non-raid6 data chunks before doing the test

2019-10-09 Thread Qu Wenruo
hunk and only 1 raid6 data chunk. Signed-off-by: Qu Wenruo --- Currently it will skip the test if the assumption is not met, but in fact the original failure detects a bug in my modification of mkfs.btrfs. so I'm not completely sure if I should _fail or _notrun. --- tests/btrfs/157 | 14 +

Re: [PATCH v3 0/3] btrfs: Introduce new incompat feature BG_TREE to hugely reduce mount time

2019-10-09 Thread Qu WenRuo
On 2019/10/10 上午10:39, Qu Wenruo wrote: > This patchset can be fetched from: > https://github.com/adam900710/linux/tree/bg_tree > Which is based on v5.4-rc1 tag. > > This patchset will hugely reduce mount time of large fs by putting all > block group items into its own

Re: [PATCH] btrfs: use bool argument in free_root_pointers()

2019-10-09 Thread Qu Wenruo
On 2019/10/10 上午10:39, Anand Jain wrote: > We don't need int argument bool shall do in free_root_pointers(). > And rename the argument as it confused two people. Victim here. :) Although it's mostly caused by my stupidness. :( > > Signed-off-by: Anand Jain Reviewed-by:

<    1   2   3   4   5   6   7   8   9   10   >