[PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root overwritting existing tree blocks

2018-07-05 Thread Qu Wenruo
We have function btrfs_fsck_reinit_root() to reinit csum or extent tree. However this function allows us to let it overwrite existing tree blocks using @overwrite parameter. Such behavior is pretty dangerous while no caller is using this feature explicitly. So just remove @overwrite parameter

[PATCH 2/4] btrfs-progs: transaction: Error out other than panic when committing transaction

2018-07-05 Thread Qu Wenruo
There are cases that btrfs_commit_transaction() itself can fail, mostly due to ENOSPC when allocating space. Don't panic out in this case. Signed-off-by: Qu Wenruo --- transaction.c | 19 +-- 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/transaction.c

[PATCH 0/4] Some random fuzz test fixes

2018-07-05 Thread Qu Wenruo
Can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/fuzzed_enhance Some random bugs exposed by the ill-fated fuzz-test (003 never seems to pass). It will be a big work to make fuzz/003 to pass, but at least there are some easy fixes. Qu Wenruo (4): btrfs-progs: check:

[PATCH 3/4] btrfs-progs: check/original: Avoid infinite loop when failed to repair inode

2018-07-05 Thread Qu Wenruo
Exposed by fuzz-tests/003-multi-check-unmounted/ on fuzzed image bko-161811.raw.xz. It's caused by the fact when check_fs_roots() finds tree root is modified, it re-search tree root by goto again: tag. However again: tag will also reset root objectid to 0. If we failed to repair one fs root but

RE: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root overwritting existing tree blocks

2018-07-05 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo > Sent: Thursday, July 05, 2018 3:37 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root >

[PATCH 1/3] btrfs: fsck/original: Don't panic out when unexpected root item is referring to one extent

2018-07-05 Thread Qu Wenruo
With crafted image, expected root item can refer to certain extent, and original mode uses BUG_ON() to handle such case. Fix it by gracefully return error. Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403 Signed-off-by: Qu Wenruo --- check/main.c | 7 ++- 1 file changed, 6

[PATCH 2/3] btrfs-progs: tests/fuzz: Add fuzzed test image for btrfs check BUG_ON

2018-07-05 Thread Qu Wenruo
This fuzzed image will not only cause kernel BUG_ON(), but also btrfs check BUG_ON() for original mode. Checking filesystem on /home/adam/btrfs/crafted_images/runtime/0.img UUID: 3381d111-94a3-4ac7-8f39-611bbbdab7e6 checking extents check/main.c:3677: check_owner_ref: BUG_ON `rec->is_root`

Re: [PATCH 1/2] btrfs-progs: fix nanosecs in task_period_start

2018-07-05 Thread Su Yue
On 07/05/2018 03:20 AM, Stéphane Lesimple wrote: This is a single-line fix on the preexisting task_period_start function. Signed-off-by: Stéphane Lesimple Reviewed-by: Su Yue --- task-utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task-utils.c

RE: [PATCH 2/4] btrfs-progs: transaction: Error out other than panic when committing transaction

2018-07-05 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo > Sent: Thursday, July 05, 2018 3:37 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH 2/4] btrfs-progs: transaction: Error out other than panic >

Re: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root overwritting existing tree blocks

2018-07-05 Thread Qu Wenruo
On 2018年07月05日 16:50, Gu, Jinxiang wrote: > > >> -Original Message- >> From: linux-btrfs-ow...@vger.kernel.org >> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo >> Sent: Thursday, July 05, 2018 3:37 PM >> To: linux-btrfs@vger.kernel.org >> Subject: [PATCH 1/4]

[PATCH v2] btrfs: qgroups: Move transaction management inside btrfs_quota_enable/disable

2018-07-05 Thread Nikolay Borisov
Commit 5d23515be669 ("btrfs: Move qgroup rescan on quota enable to btrfs_quota_enable") not only resulted in an easier to follow code but it also introduced a subtle bug. It changed the timing when the initial transaction rescan was happening - before the commit it would happen after transaction

Re: [PATCH v2] btrfs: qgroups: Move transaction management inside btrfs_quota_enable/disable

2018-07-05 Thread David Sterba
On Thu, Jul 05, 2018 at 02:50:48PM +0300, Nikolay Borisov wrote: > Commit 5d23515be669 ("btrfs: Move qgroup rescan on quota enable to > btrfs_quota_enable") not only resulted in an easier to follow code but > it also introduced a subtle bug. It changed the timing when the initial > transaction

Re: [PATCH v2] btrfs: qgroups: Move transaction management inside btrfs_quota_enable/disable

2018-07-05 Thread Nikolay Borisov
On 5.07.2018 14:50, Nikolay Borisov wrote: > Commit 5d23515be669 ("btrfs: Move qgroup rescan on quota enable to > btrfs_quota_enable") not only resulted in an easier to follow code but > it also introduced a subtle bug. It changed the timing when the initial > transaction rescan was happening

Re: [PATCH v2] btrfs: qgroups: Move transaction management inside btrfs_quota_enable/disable

2018-07-05 Thread David Sterba
On Thu, Jul 05, 2018 at 02:50:48PM +0300, Nikolay Borisov wrote: > @@ -921,8 +938,10 @@ int btrfs_quota_enable(struct btrfs_trans_handle *trans, > > ret = btrfs_insert_empty_item(trans, quota_root, path, , > sizeof(*ptr)); > - if (ret) > + if

Re: [PATCH v3] btrfs: Add chunk type check in read a chunk

2018-07-05 Thread David Sterba
On Wed, Jul 04, 2018 at 06:16:39PM +0800, Gu Jinxiang wrote: > Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199839, > which has a invalid chunk, not return error opportunlly. > > Add chunk type check in btrfs_check_chunk_valid, > to make error be returned in advance. > > Reported-by:

Re: [PATCH v3] btrfs: Add chunk type check in read a chunk

2018-07-05 Thread David Sterba
On Wed, Jul 04, 2018 at 03:42:30PM +0300, Nikolay Borisov wrote: > On 4.07.2018 13:16, Gu Jinxiang wrote: > > Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199839, > > which has a invalid chunk, not return error opportunlly. > > > > Add chunk type check in btrfs_check_chunk_valid, > >

Re: [PATCH] Btrfs: make sure there is always room for generation number

2018-07-05 Thread Zhihui Zhang
code inspection. If I understand it correctly, we are off by 4 bytes because CRC is 32 bits and generation is 64 bits. But this bug might never hit in practice. thanks, -Zhihui On Wed, Jul 4, 2018 at 11:21 AM, David Sterba wrote: > On Mon, Jul 02, 2018 at 08:00:54PM -0400, Zhihui Zhang wrote:

Re: [PATCH v2 2/2] btrfs: fix missing superblock update in the device delete commit transaction

2018-07-05 Thread David Sterba
On Tue, Jul 03, 2018 at 05:07:24PM +0800, Anand Jain wrote: > When a device is deleted, the btrfs_super_block::number_devices is > reduced by 1, but we do that after the commit transaction, so this > change did not made it to the disk and waits for the next commit > transaction whenever it

Re: [PATCH] btrfs: remove warnings superseded by refcount_t usage

2018-07-05 Thread David Sterba
On Fri, Jun 29, 2018 at 04:05:40PM +0200, David Sterba wrote: > On Tue, Jun 26, 2018 at 05:00:46PM +0300, Nikolay Borisov wrote: > > On 25.06.2018 19:38, David Sterba wrote: > > > There are several WARN_ON calls that catch incrorrect reference counter > > > updates, but this is what the refcount_t

Re: [PATCH] btrfs-progs: check: Fix wrong error message in case of corrupted extent

2018-07-05 Thread David Sterba
On Tue, Jun 05, 2018 at 02:39:10PM +0300, Nikolay Borisov wrote: > When btrfs check detects a freespace tree extent which ends beyond the > blockgroup containing it a misleading error messages is printed. For > example if we have the following extent in the freespace tree: > > item 5 key

Re: [PATCH] btrfs-progs: print bytenr of tree block in print_tree_block_error()

2018-07-05 Thread David Sterba
On Thu, Jun 14, 2018 at 10:40:39AM +0800, Su Yue wrote: > For easier debug, let print_tree_block_error() print bytenr of > tree block. > > Signed-off-by: Su Yue Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: [PATCH 5/5] btrfs: Verify every chunk has corresponding block group at mount time

2018-07-05 Thread David Sterba
On Tue, Jul 03, 2018 at 05:10:09PM +0800, Qu Wenruo wrote: > If a crafted btrfs has missing block group items, it could cause > unexpected behavior and breaks our expectation on 1:1 > chunk<->block group mapping. > > Although we added block group -> chunk mapping check, we still need > chunk ->

Re: [PATCH 0/5] Enhancement for block group/chunk verification

2018-07-05 Thread David Sterba
On Thu, Jul 05, 2018 at 09:36:47AM +0800, Qu Wenruo wrote: > > > On 2018年07月04日 21:36, David Sterba wrote: > > On Tue, Jul 03, 2018 at 05:10:04PM +0800, Qu Wenruo wrote: > >> Can be fetched from github, which is based on v4.18-rc1 tag: > >>

Re: [PATCH 5/5] btrfs: Verify every chunk has corresponding block group at mount time

2018-07-05 Thread Qu Wenruo
On 2018年07月05日 23:18, David Sterba wrote: > On Tue, Jul 03, 2018 at 05:10:09PM +0800, Qu Wenruo wrote: >> If a crafted btrfs has missing block group items, it could cause >> unexpected behavior and breaks our expectation on 1:1 >> chunk<->block group mapping. >> >> Although we added block group

Re: [PATCH 4/5] btrfs: Check each block group has corresponding chunk at mount time

2018-07-05 Thread Qu Wenruo
On 2018年07月04日 13:45, Gu, Jinxiang wrote: > > >> -Original Message- >> From: linux-btrfs-ow...@vger.kernel.org >> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo >> Sent: Tuesday, July 03, 2018 5:10 PM >> To: linux-btrfs@vger.kernel.org >> Subject: [PATCH 4/5] btrfs:

Re: [PATCH 5/5] btrfs: Verify every chunk has corresponding block group at mount time

2018-07-05 Thread Qu Wenruo
On 2018年07月04日 17:46, Qu Wenruo wrote: > > > On 2018年07月04日 15:08, Nikolay Borisov wrote: >> >> >> On 3.07.2018 12:10, Qu Wenruo wrote: >>> If a crafted btrfs has missing block group items, it could cause >>> unexpected behavior and breaks our expectation on 1:1 >>> chunk<->block group

RE: [PATCH 3/4] btrfs-progs: check/original: Avoid infinite loop when failed to repair inode

2018-07-05 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo > Sent: Thursday, July 05, 2018 3:38 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH 3/4] btrfs-progs: check/original: Avoid infinite loop when >

Re: [PATCH] fstests: btrfs/168 verify device ready after device delete

2018-07-05 Thread Eryu Guan
On Tue, Jul 03, 2018 at 04:47:53PM +0800, Anand Jain wrote: > This test case verifies if the device ready return success after the > device delete. > > Signed-off-by: Anand Jain Looks fine to me overall, but I may need some helps from btrfs folks :) > --- > tests/btrfs/168 | 68 >

RE: [PATCH 4/4] btrfs-progs: check/original: Don't overwrite return value when we failed to repair

2018-07-05 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo > Sent: Thursday, July 05, 2018 3:38 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH 4/4] btrfs-progs: check/original: Don't overwrite return >

[PATCH v2 2/2] btrfs: Verify every chunk has corresponding block group at mount time

2018-07-05 Thread Qu Wenruo
If a crafted btrfs has missing block group items, it could cause unexpected behavior and breaks our expectation on 1:1 chunk<->block group mapping. Although we added block group -> chunk mapping check, we still need chunk -> block group mapping check. This patch will do extra check to ensure

[PATCH v2 1/2] btrfs: Check each block group has corresponding chunk at mount time

2018-07-05 Thread Qu Wenruo
A crafted btrfs with incorrect chunk<->block group mapping, it could leads to a lot of unexpected behavior. Although the crafted image can be catched by block group item checker added in "[PATCH] btrfs: tree-checker: Verify block_group_item", if one crafted a valid enough block group item which

RE: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root overwritting existing tree blocks

2018-07-05 Thread Gu, Jinxiang
> -Original Message- > From: Qu Wenruo [mailto:quwenruo.bt...@gmx.com] > Sent: Thursday, July 05, 2018 5:41 PM > To: Gu, Jinxiang/顾 金香 ; Qu Wenruo ; > linux-btrfs@vger.kernel.org > Subject: Re: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild > root overwritting existing

Re: how to best segment a big block device in resizeable btrfs filesystems?

2018-07-05 Thread Andrei Borzenkov
03.07.2018 10:15, Duncan пишет: > Andrei Borzenkov posted on Tue, 03 Jul 2018 07:25:14 +0300 as excerpted: > >> 02.07.2018 21:35, Austin S. Hemmelgarn пишет: >>> them (trimming blocks on BTRFS gets rid of old root trees, so it's a >>> bit dangerous to do it while writes are happening). >> >>

Re: [PATCH] fstests: btrfs/168 verify device ready after device delete

2018-07-05 Thread Anand Jain
On 07/06/2018 12:40 PM, Eryu Guan wrote: On Tue, Jul 03, 2018 at 04:47:53PM +0800, Anand Jain wrote: This test case verifies if the device ready return success after the device delete. Signed-off-by: Anand Jain Looks fine to me overall, but I may need some helps from btrfs folks :) ---

[PATCH v2] fstests: btrfs/168 verify device ready after device delete

2018-07-05 Thread Anand Jain
This test case verifies if the device ready return success after the device delete. Signed-off-by: Anand Jain --- v1->v2: use _run_btrfs_util_prog instead of open coding it. tests/btrfs/168 | 68 + tests/btrfs/168.out | 2 ++