Re: [PATCH v2] fstests: btrfs/149 make it sectorsize independent

2018-09-10 Thread Nikolay Borisov
On 10.09.2018 06:25, Anand Jain wrote: > Originally this test case was designed to work with only 4K sectorsize. > Now enhance it to work with any sector sizes and makes the following > changes: > Output file not to contain any traces of sector size. > Use max_inline=0 mount option so that it me

Re: [PATCH v2] fstests: btrfs/149 make it sectorsize independent

2018-09-10 Thread anand . jain
On 09/10/2018 03:59 PM, Nikolay Borisov wrote: On 10.09.2018 06:25, Anand Jain wrote: Originally this test case was designed to work with only 4K sectorsize. Now enhance it to work with any sector sizes and makes the following changes: Output file not to contain any traces of sector size. U

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-10 Thread Dave Chinner
On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > > > On Thu, Aug 23, 2018 at 08:58:4

Re: Does ssd auto detected work for microSD cards?

2018-09-10 Thread David Sterba
On Mon, Sep 03, 2018 at 10:22:42PM -0600, Chris Murphy wrote: > On Mon, Sep 3, 2018 at 7:53 PM, GWB wrote: > > Curious instance here, but perhaps this is the expected behaviour: > > > > mount | grep btrfs > > /dev/sdb3 on / type btrfs (rw,ssd,subvol=@) > > /dev/sdb3 on /home type btrfs (rw,ssd,sub

Re: nbdkit as a flexible alternative to loopback mounts

2018-09-10 Thread David Sterba
On Wed, Sep 05, 2018 at 03:31:48PM +0100, Richard W.M. Jones wrote: > On Tue, Sep 04, 2018 at 07:55:00PM -0600, Chris Murphy wrote: > > https://rwmj.wordpress.com/2018/09/04/nbdkit-as-a-flexible-alternative-to-loopback-mounts/ > > > > This is a pretty cool writeup. I can vouch Btrfs will format mo

Re: [RFC PATCH linux-next] btrfs: btrfs_run_delayed_refs_for_head() can be static

2018-09-10 Thread Nikolay Borisov
On 10.09.2018 19:48, David Sterba wrote: > On Mon, Sep 10, 2018 at 11:00:29PM +0800, kbuild test robot wrote: >> >> Fixes: ac75a14eb672 ("btrfs: Factor out loop processing all refs of a head") >> Signed-off-by: kbuild test robot >> --- >> extent-tree.c |6 +++--- >> 1 file changed, 3 inser

[PATCH 0/4] Self-test cleanups

2018-09-10 Thread David Sterba
I've noticed a new use of the STATIC macro in Nikolay's patches cleaning up delayed refs. My series removes the macro and does a other minor reorganization to reduce ifdefs used in the test code. David Sterba (4): btrfs: tests: add separate stub for find_lock_delalloc_range btrfs: tests: move

[PATCH 2/4] btrfs: tests: move testing members of struct btrfs_root to the end

2018-09-10 Thread David Sterba
The data used only for tests are better placed at the end of the structure so that they don't change the structure layout. All new members of btrfs_root should be placed before. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff

[PATCH 1/4] btrfs: tests: add separate stub for find_lock_delalloc_range

2018-09-10 Thread David Sterba
The helper find_lock_delalloc_range is now conditionally built static, dpending on whether the self-tests are enabled or not. There's a macro that is suppsed to hide the export, used only once. To discourage further use, drop it an add a public stub for the helper required by tests. Signed-off-by:

[PATCH 4/4] btrfs: tests: polish ifdefs around testing helper

2018-09-10 Thread David Sterba
Avoid the inline ifdefs and use two sections for self-tests enabled and disabled. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 32d2fce4ac53..8dafc7bb6ad8 100644 --- a/fs/btr

[PATCH 3/4] btrfs: tests: group declarations of self-test helpers

2018-09-10 Thread David Sterba
Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index ee05857cc8ac..32d2fce4ac53 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3194,9 +3194,6 @@ int btrfs_prealloc_file_range_tra

Re: [RFC PATCH linux-next] btrfs: btrfs_run_delayed_refs_for_head() can be static

2018-09-10 Thread David Sterba
On Mon, Sep 10, 2018 at 08:15:18PM +0300, Nikolay Borisov wrote: > > > On 10.09.2018 19:48, David Sterba wrote: > > On Mon, Sep 10, 2018 at 11:00:29PM +0800, kbuild test robot wrote: > >> > >> Fixes: ac75a14eb672 ("btrfs: Factor out loop processing all refs of a > >> head") > >> Signed-off-by: k

Re: [PATCH] btrfs: fix error handling in btrfs_dev_replace_start

2018-09-10 Thread David Sterba
On Thu, Sep 06, 2018 at 03:52:17PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > When we fail to start a transaction in btrfs_dev_replace_start, > we leave dev_replace->replace_start set to STARTED but clear > ->srcdev and ->tgtdev. Later, that can result in an Oops in > btrfs_dev_replac

Re: [PATCH] btrfs-progs: tests: Add test for missing device delete error value

2018-09-10 Thread David Sterba
On Mon, Sep 03, 2018 at 01:02:57PM +0300, Nikolay Borisov wrote: > Add a test which ensures the kernel returns the correct error value > when missing device removal is requested. This test verifies that kernel > refactoring didn't broken the return value. > > Signed-off-by: Nikolay Borisov > ---

Re: [PATCH 1/3] btrfs: Make btrfs_find_device_by_path return struct btrfs_device

2018-09-10 Thread David Sterba
On Mon, Sep 03, 2018 at 12:46:12PM +0300, Nikolay Borisov wrote: > Currently this function returns an error code as well as uses one of > its arguments as a return value for struct btrfs_device. Change the > function so that it returns btrfs_device directly and use the usual > "encode error in poin

Re: [PATCH 0/3] cleanup couple of device-related functions' retval

2018-09-10 Thread David Sterba
On Mon, Sep 03, 2018 at 12:46:11PM +0300, Nikolay Borisov wrote: > Currently btrfs_find_device_by_path, btrfs_find_device_missing_or_by_path and > btrfs_find_device_by_devspec are called in a chain and they all return an > integer value to signal error and at the same time use one of their parame

Re: [PATCH v2] Btrfs: use correct args for kcalloc in btrfsic_read_block

2018-09-10 Thread David Sterba
On Sat, Sep 08, 2018 at 04:41:40AM +0800, Liu Bo wrote: > kcalloc is defined as > kcalloc(size_t n, size_t size, gfp_t flags) > > Although this won't cause problems in practice, btrfsic_read_block() > has switched n with size. > > This updates btrfsic_read_block() to be correct in using kcalloc.

Re: [PATCH v3 1/2] btrfs: Enhance btrfs_trim_fs function to handle error better

2018-09-10 Thread David Sterba
On Wed, Aug 29, 2018 at 05:40:12PM +0300, Nikolay Borisov wrote: > > > On 29.08.2018 16:53, Qu Wenruo wrote: > > > > > > On 2018/8/29 下午9:43, Nikolay Borisov wrote: > >> > >> > >> On 29.08.2018 08:15, Qu Wenruo wrote: > >>> Function btrfs_trim_fs() doesn't handle errors in a consistent way, if

[PATCH v6 0/2] vfs: fix dedupe permission check

2018-09-10 Thread Mark Fasheh
Hi Andrew/Al, Could I please have these patches put in a tree for more public testing? They've hit fsdevel a few times now, I have links to the discussions in the change log below. The following patches fix a couple of issues with the permission check we do in vfs_dedupe_file_range(). The first

[PATCH 2/2] vfs: dedupe should return EPERM if permission is not granted

2018-09-10 Thread Mark Fasheh
Right now we return EINVAL if a process does not have permission to dedupe a file. This was an oversight on my part. EPERM gives a true description of the nature of our error, and EINVAL is already used for the case that the filesystem does not support dedupe. Signed-off-by: Mark Fasheh Reviewed-

[PATCH 1/2] vfs: allow dedupe of user owned read-only files

2018-09-10 Thread Mark Fasheh
The permission check in vfs_dedupe_file_range_one() is too coarse - We only allow dedupe of the destination file if the user is root, or they have the file open for write. This effectively limits a non-root user from deduping their own read-only files. In addition, the write file descriptor that t

Re: [PATCH 2/4] btrfs: tests: move testing members of struct btrfs_root to the end

2018-09-10 Thread Omar Sandoval
On Mon, Sep 10, 2018 at 07:22:26PM +0200, David Sterba wrote: > The data used only for tests are better placed at the end of the > structure so that they don't change the structure layout. All new > members of btrfs_root should be placed before. Reviewed-by: Omar Sandoval > Signed-off-by: David

Re: [PATCH 1/4] btrfs: tests: add separate stub for find_lock_delalloc_range

2018-09-10 Thread Omar Sandoval
On Mon, Sep 10, 2018 at 07:22:24PM +0200, David Sterba wrote: > The helper find_lock_delalloc_range is now conditionally built static, > dpending on whether the self-tests are enabled or not. There's a macro > that is suppsed to hide the export, used only once. To discourage > further use, drop it

Re: [PATCH 3/4] btrfs: tests: group declarations of self-test helpers

2018-09-10 Thread Omar Sandoval
On Mon, Sep 10, 2018 at 07:22:29PM +0200, David Sterba wrote: Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 4 +--- > 1 file changed, 1 insertion(+), 3 deletions(-) > > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h > index ee05857cc8ac..32d2fce4ac53 10

Re: [PATCH 4/4] btrfs: tests: polish ifdefs around testing helper

2018-09-10 Thread Omar Sandoval
On Mon, Sep 10, 2018 at 07:22:31PM +0200, David Sterba wrote: > Avoid the inline ifdefs and use two sections for self-tests enabled and > disabled. > > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 9 ++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/fs/btrfs

[PATCH] btrfs: delayed-ref: Introduce new parameter for btrfs_add_delayed_tree_ref() to reduce unnecessary qgroup tracing

2018-09-10 Thread Qu Wenruo
For btrfs_add_delayed_tree_ref(), its ref_root parameter can be different from its real root. This is pretty common for reloc tree, in that case @ref_root is passed as the original tree owner (source file tree). However btrfs_add_delayed_tree_ref() uses @ref_root to determine whether we should do

Re: [PATCH v2 0/5] btrfs: qgroup: Skip unrelated tree blocks for balance

2018-09-10 Thread Qu Wenruo
On 2018/9/7 下午5:32, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/linux/tree/qgroup_balance_skip_trees > The base commit is v4.19-rc1 tag. > > There are a lot of reports of system hang for balance on quota enabled > fs. > It's most obvious for large

[PATCH v3 0/7] btrfs: qgroup: Reduce dirty extents for metadata

2018-09-10 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/linux/tree/qgroup_balance_skip_trees The base commit is v4.19-rc1 tag. There are a lot of reports of system hang for balance on quota enabled fs. It's most obvious for large fs. The hang is caused by tons of unmodified extent

[PATCH v3 1/7] btrfs: qgroup: Introduce trace event to analyse the number of dirty extents accounted

2018-09-10 Thread Qu Wenruo
Number of qgroup dirty extents is directly linked to the performance overhead, so add a new trace event, trace_qgroup_num_dirty_extents(), to record how many dirty extents is processed in btrfs_qgroup_account_extents(). This will be pretty handy to analyse later balance performance improvement. S

[PATCH v3 2/7] btrfs: qgroup: Introduce function to trace two swaped extents

2018-09-10 Thread Qu Wenruo
Introduce a new function, qgroup_trace_extent_swap(), which will be used later for balance qgroup speedup. The basis idea of balance is swapping tree blocks between reloc tree and the real file tree. The swap will happen in highest tree block, but there may be a lot of tree blocks involved. For

[PATCH v3 5/7] btrfs: qgroup: Don't trace subtree if we're dropping reloc tree

2018-09-10 Thread Qu Wenruo
Reloc tree doesn't contribute to qgroup numbers, as we have accounted them at balance time (check replace_path()). Skip such unneeded subtree trace should reduce some performance overhead. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 8 +++- 1 file changed, 7 insertions(+), 1 delet

[PATCH v3 3/7] btrfs: qgroup: Introduce function to find all new tree blocks of reloc tree

2018-09-10 Thread Qu Wenruo
Introduce new function, qgroup_trace_new_subtree_blocks(), to iterate all new tree blocks in a reloc tree. So that qgroup could skip unrelated tree blocks during balance, which should hugely speedup balance speed when quota is enabled. The function qgroup_trace_new_subtree_blocks() itself only car

[PATCH v3 4/7] btrfs: qgroup: Use generation aware subtree swap to mark dirty extents

2018-09-10 Thread Qu Wenruo
Before this patch, for quota enabled balance, btrfs needs to mark the whole subtree dirty for quota. E.g. OO = Old tree blocks (from file tree) NN = New tree blocks (from reloc tree) File tree (src) Reloc tree (dst) OO (a) NN (a)

[PATCH v3 6/7] btrfs: delayed-ref: Introduce new parameter for btrfs_add_delayed_tree_ref() to reduce unnecessary qgroup tracing

2018-09-10 Thread Qu Wenruo
For btrfs_add_delayed_tree_ref(), its ref_root parameter can be different from its real root. This is pretty common for reloc tree, in that case @ref_root is passed as the original tree owner (source file tree). However btrfs_add_delayed_tree_ref() uses @ref_root to determine whether we should do

[PATCH v3 7/7] btrfs: qgroup: Only trace data extents in leaves if we're relocating data block group

2018-09-10 Thread Qu Wenruo
For qgroup_trace_extent_swap(), if we find one leaf needs to be traced, btrfs will also iterate all file extents and trace them. This is OK if we're relocating data block groups, but if we're relocating metadata block groups, balance code itself has ensured that both subtree of file tree and reloc