[PATCH v2] btrfs: Always check nocow for quota enabled case to make sure we won't reserve unnecessary data space

2018-08-20 Thread Qu Wenruo
Commit c6887cd11149 ("Btrfs: don't do nocow check unless we have to") makes nocow check less frequent to improve performance. However for quota enabled case, such optimization could lead to extra unnecessary data reservation, which results failure for test case like btrfs/153 in fstests. Fix it b

Re: [PATCH] btrfs: Always check nocow for quota enabled case to make sure we won't reserve unnecessary data space

2018-08-20 Thread Qu Wenruo
On 2018/8/21 下午12:21, Misono Tomohiro wrote: > On 2018/08/15 15:13, Qu Wenruo wrote: >> Commit c6887cd11149 ("Btrfs: don't do nocow check unless we have to") >> makes nocow check less frequent to improve performance. >> >> However for quota enabled case, such optimization could lead to extra >> u

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:17:18PM -0500, Eric Sandeen wrote: > > > On 8/20/18 7:49 PM, Dave Chinner wrote: > > Upon successful completion of this ioctl, the number of > > bytes successfully deduplicated is returned in bytes_deduped > > and a status code for the deduplication operatio

Re: [PATCH] btrfs: Always check nocow for quota enabled case to make sure we won't reserve unnecessary data space

2018-08-20 Thread Misono Tomohiro
On 2018/08/15 15:13, Qu Wenruo wrote: > Commit c6887cd11149 ("Btrfs: don't do nocow check unless we have to") > makes nocow check less frequent to improve performance. > > However for quota enabled case, such optimization could lead to extra > unnecessary data reservation, which results failure fo

[PATCH v4] btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock

2018-08-20 Thread Qu Wenruo
[BUG] For certains crafted image, whose csum root leaf has missing backref, if we try to trigger write with data csum, it could cause deadlock with the following kernel WARN_ON(): -- WARNING: CPU: 1 PID: 41 at fs/btrfs/locking.c:230 btrfs_tree_lock+0x3e2/0x400 CPU: 1 PID: 41 Comm: kworker/u4:1

[PATCH v2.1] btrfs: Handle owner mismatch gracefully when walking up tree

2018-08-20 Thread Qu Wenruo
[BUG] When mounting certain crafted image, btrfs will trigger kernel BUG_ON() when try to recover balance: -- [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:8956! invalid opcode: [#1] PREEMPT SMP NOPTI CPU: 1 PID: 662 Comm: mount Not tainted 4.18.0-rc1-custom+ #1

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Eric Sandeen
On 8/20/18 7:49 PM, Dave Chinner wrote: > Upon successful completion of this ioctl, the number of > bytes successfully deduplicated is returned in bytes_deduped > and a status code for the deduplication operation is > returned in status. If even a single byte in the rang

[PATCH v2] btrfs: Handle owner mismatch gracefully when walking up tree

2018-08-20 Thread Qu Wenruo
[BUG] When mounting certain crafted image, btrfs will trigger kernel BUG_ON() when try to recover balance: -- [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:8956! invalid opcode: [#1] PREEMPT SMP NOPTI CPU: 1 PID: 662 Comm: mount Not tainted 4.18.0-rc1-custom+ #1

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > So why was this dedupe request even accepted by the kernel? Well, > > I think there's a bug in the check just above this: > > > > /* If we're linking to

Re: [PATCH RFC] btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock

2018-08-20 Thread Qu Wenruo
On 2018/8/21 上午12:38, David Sterba wrote: > On Tue, Aug 14, 2018 at 01:51:21PM +0800, Qu Wenruo wrote: >> [BUG] >> For certains crafted image, whose csum root leaf has missing backref, if >> we try to trigger write with data csum, it could cause deadlock with the >> following kernel WARN_ON(): >>

Re: [PATCH RFC] btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock

2018-08-20 Thread David Sterba
On Tue, Aug 14, 2018 at 01:51:21PM +0800, Qu Wenruo wrote: > [BUG] > For certains crafted image, whose csum root leaf has missing backref, if > we try to trigger write with data csum, it could cause deadlock with the > following kernel WARN_ON(): > -- > WARNING: CPU: 1 PID: 41 at fs/btrfs/locki

[btrfs-progs] btrfs receive complains about not being able to open its temporary file(s) (e.g. o123802-148688-0)

2018-08-20 Thread Michal Soltys
I have small btrfs filesystem with the following structure (created and used during 4.x kernels): 17:35 # btrfs subvolume list /src -qu ID 263 gen 141492 top level 5 parent_uuid - uuid e3a77929-0e4d-6744-9aed-d4d6a4de5f78 path xenial ID 340 gen 134847 top level

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Darrick J. Wong
On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > [cc linux-fsdevel now, too] > > On Mon, Aug 20, 2018 at 09:11:26AM +1000, Dave Chinner wrote: > > [cc linux-...@vger.kernel.org] > > > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana

Re: [PATCH RFC] btrfs: extent-tree: refactor find_free_extent()

2018-08-20 Thread Qu Wenruo
On 2018/8/20 下午10:17, David Sterba wrote: > On Wed, Jul 25, 2018 at 04:12:42PM +0800, Qu Wenruo wrote: >> extent-tree.c::find_free_extent() could be one of the most >> ill-structured function, it has at least 4 non-exit tags and jumps >> between them. >> >> To make it a little easier to understan

Re: [PATCH RFC] btrfs: extent-tree: refactor find_free_extent()

2018-08-20 Thread David Sterba
On Wed, Jul 25, 2018 at 04:12:42PM +0800, Qu Wenruo wrote: > extent-tree.c::find_free_extent() could be one of the most > ill-structured function, it has at least 4 non-exit tags and jumps > between them. > > To make it a little easier to understand, extract that big functions > into 4 parts: > >

Re: [PATCH 1/1] btrfs: Handle owner mismatch gracefully when walking up tree

2018-08-20 Thread David Sterba
On Wed, Aug 01, 2018 at 04:08:01PM +0800, Qu Wenruo wrote: > @@ -8949,17 +8949,26 @@ static noinline int walk_up_proc(struct > btrfs_trans_handle *trans, > } > > if (eb == root->node) { > - if (wc->flags[level] & BTRFS_BLOCK_FLAG_FULL_BACKREF) > + if (wc->flag

Re: [PATCH 2/2] btrfstune: Rename change_header_uuid to change_buffer_header_uuid

2018-08-20 Thread Nikolay Borisov
On 20.08.2018 16:34, Qu Wenruo wrote: > > > On 2018/8/20 下午9:28, Nikolay Borisov wrote: >> Rename the function so its name falls in line with the rest of btrfs >> nomenclature. So when we change the uuid of the header it's in fact >> of the buffer header. > > We have setget functions like btr

Re: [PATCH] btrfs: use after free in btrfs_quota_enable()

2018-08-20 Thread David Sterba
On Mon, Aug 20, 2018 at 11:25:33AM +0300, Dan Carpenter wrote: > The issue here is that btrfs_commit_transaction() frees "trans" on both > the error and the success path. So the problem would be if > btrfs_commit_transaction() succeeds, and then qgroup_rescan_init() > fails. That means that "ret"

Re: [PATCH v2 0/2] btrfs-progs: completion: Small fixes to make debug simpler

2018-08-20 Thread David Sterba
On Mon, Aug 13, 2018 at 02:02:42PM +0800, Qu Wenruo wrote: > For developer, it's pretty common to use "btrfs check" or "btrfs ins > dump-tree" on raw dumps. > Even for end user, they may hit case where they need to run "btrfs check" > on raw images. > > However "btrfs check" can only complete real

Re: [PATCH 2/2] btrfstune: Rename change_header_uuid to change_buffer_header_uuid

2018-08-20 Thread Qu Wenruo
On 2018/8/20 下午9:28, Nikolay Borisov wrote: > Rename the function so its name falls in line with the rest of btrfs > nomenclature. So when we change the uuid of the header it's in fact > of the buffer header. We have setget functions like btrfs_header_nritems() or btrfs_header_bytenr(), IMHO bo

Re: [PATCH 1/2] btrfstune: Remove fs_info arg from change_device_uuid

2018-08-20 Thread Qu Wenruo
On 2018/8/20 下午9:28, Nikolay Borisov wrote: > This function already takes an extent buffer that contains a reference > to the fs_info. Use that and reduce argument count. No functional > changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- > btrfstune.c | 6

[PATCH 0/2] btrfstune fs_info cleanups

2018-08-20 Thread Nikolay Borisov
While inspecting btrfstune source code I came across 2 functions which needlessly take fs_info argument. This small series rectifies this. No functional changes. Nikolay Borisov (2): btrfstune: Remove fs_info arg from change_device_uuid btrfstune: Rename change_header_uuid to change_buffer_h

[PATCH 1/2] btrfstune: Remove fs_info arg from change_device_uuid

2018-08-20 Thread Nikolay Borisov
This function already takes an extent buffer that contains a reference to the fs_info. Use that and reduce argument count. No functional changes. Signed-off-by: Nikolay Borisov --- btrfstune.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/btrfstune.c b/btrfstune.c ind

[PATCH 2/2] btrfstune: Rename change_header_uuid to change_buffer_header_uuid

2018-08-20 Thread Nikolay Borisov
Rename the function so its name falls in line with the rest of btrfs nomenclature. So when we change the uuid of the header it's in fact of the buffer header. Also remove the root argument since it's used solely to take a reference to fs_info which can be done via the mandatory eb argument. No func

Re: Are the btrfs mount options inconsistent?

2018-08-20 Thread Qu Wenruo
On 2018/8/20 下午8:24, David Howells wrote: > David Sterba wrote: > >> No it's not. Compression needs the checksums so nodatasum should disable >> compression, which is missing as you found out. > > Thanks. > > Btw, do fs_info->mount_opt end up inscribed on disk as is? I don't see > anywhere t

Re: Are the btrfs mount options inconsistent?

2018-08-20 Thread David Howells
David Howells wrote: > Can I get rid of BTRFS_MOUNT_NOSSD as it would appear to be superfluous with > BTRFS_MOUNT_SSD? Ah... I guess it's not quite superfluous: if (!btrfs_test_opt(fs_info, NOSSD) && !fs_info->fs_devices->rotating) { btrfs_set_and_info(fs_in

Re: Are the btrfs mount options inconsistent?

2018-08-20 Thread David Howells
David Sterba wrote: > No it's not. Compression needs the checksums so nodatasum should disable > compression, which is missing as you found out. Thanks. Btw, do fs_info->mount_opt end up inscribed on disk as is? I don't see anywhere this obviously happens. Can I get rid of BTRFS_MOUNT_NOSSD a

Re: lazytime mount option—no support in Btrfs

2018-08-20 Thread Austin S. Hemmelgarn
On 2018-08-19 06:25, Andrei Borzenkov wrote: Отправлено с iPhone 19 авг. 2018 г., в 11:37, Martin Steigerwald написал(а): waxhead - 18.08.18, 22:45: Adam Hunt wrote: Back in 2014 Ted Tso introduced the lazytime mount option for ext4 and shortly thereafter a more generic VFS implementation

Re: [PATCH] btrfs: use after free in btrfs_quota_enable()

2018-08-20 Thread Nikolay Borisov
On 20.08.2018 11:25, Dan Carpenter wrote: > The issue here is that btrfs_commit_transaction() frees "trans" on both > the error and the success path. So the problem would be if > btrfs_commit_transaction() succeeds, and then qgroup_rescan_init() > fails. That means that "ret" is non-zero and "

[PATCH] btrfs: use after free in btrfs_quota_enable()

2018-08-20 Thread Dan Carpenter
The issue here is that btrfs_commit_transaction() frees "trans" on both the error and the success path. So the problem would be if btrfs_commit_transaction() succeeds, and then qgroup_rescan_init() fails. That means that "ret" is non-zero and "trans" is non-NULL and it leads to a use after free i

Re: [PATCH 1/2] Btrfs: kill btrfs_clear_path_blocking

2018-08-20 Thread Nikolay Borisov
On 20.08.2018 09:07, Liu Bo wrote: > On Fri, Aug 17, 2018 at 10:24:58AM +0300, Nikolay Borisov wrote: >> >> >> On 17.08.2018 00:07, Liu Bo wrote: >>> Btrfs's btree locking has two modes, spinning mode and blocking mode, >>> while searching btree, locking is always acquired in spinning mode and >