Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: > The generic/513 tell that cloning into a file did not strip security > privileges (suid, capabilities) like a regular write would. > > Signed-off-by: Lu Fengqi > --- > The xfs and ocfs2 call generic_remap_file_range_prep to drop file > privileges,

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: > On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: >> >> >> On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: >>> The generic/513 tell that cloning into a file did not strip security >>> privileges (suid, capabilities) like a regular write

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Christoph Hellwig
On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: > > > On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: > > The generic/513 tell that cloning into a file did not strip security > > privileges (suid, capabilities) like a regular write would. > > > > Signed-off-by: Lu Fengqi > > --- > >

[PATCH 0/3] Misc cosmetic changes for map_private_extent_buffer

2018-11-27 Thread Johannes Thumshirn
While trying to understand the checksum code I came across some oddities regarding map_private_extent_buffer() and it's interaction with csum_tree_block(). These patches address them but are either purely cosmetic or only add a comment documenting behaviour. Johannes Thumshirn (3): btrfs:

[PATCH 2/3] btrfs: use offset_in_page for start_offset in map_private_extent_buffer()

2018-11-27 Thread Johannes Thumshirn
In map_private_extent_buffer() use offset_in_page() to initialize 'start_offset' instead of open-coding it. Signed-off-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index

[PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Johannes Thumshirn
Document why map_private_extent_buffer() cannot return '1' (i.e. the map spans two pages) for the csum_tree_block() case. The current algorithm for detecting a page boundary crossing in map_private_extent_buffer() will return a '1' *IFF* the product of the extent buffer's offset in the page + the

[PATCH 1/3] btrfs: don't initialize 'offset' in map_private_extent_buffer()

2018-11-27 Thread Johannes Thumshirn
In map_private_extent_buffer() the 'offset' variable is initialized to a page aligned version of the 'start' parameter. But later on it is overwritten with either the offset from the extent buffer's start or 0. So get rid of the initial initialization. Signed-off-by: Johannes Thumshirn ---

Re: [PATCH 1/3] btrfs: don't initialize 'offset' in map_private_extent_buffer()

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > In map_private_extent_buffer() the 'offset' variable is initialized to a > page aligned version of the 'start' parameter. > > But later on it is overwritten with either the offset from the extent > buffer's start or 0. > > So get rid of the

Re: [PATCH 5/6] btrfs: introduce delayed_refs_rsv

2018-11-27 Thread David Sterba
On Mon, Nov 26, 2018 at 11:14:12AM +0200, Nikolay Borisov wrote: > > - if (global_rsv->space_info->full) { > > - num_dirty_bgs_bytes <<= 1; > > - num_bytes <<= 1; > > - } > > + struct btrfs_block_rsv *global_rsv = _info->global_block_rsv; > > + struct btrfs_block_rsv

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-27 Thread Josef Bacik
On Thu, Nov 22, 2018 at 12:09:34PM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > > From: Josef Bacik > > > > The cleanup_extent_op function actually would run the extent_op if it > > needed running, which made the name sort of a misnomer. Change it to > >

Re: [PATCH 2/3] btrfs: use offset_in_page for start_offset in map_private_extent_buffer()

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > In map_private_extent_buffer() use offset_in_page() to initialize > 'start_offset' instead of open-coding it. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent_io.c | 2 +- > 1 file changed, 1

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > Document why map_private_extent_buffer() cannot return '1' (i.e. the map > spans two pages) for the csum_tree_block() case. > > The current algorithm for detecting a page boundary crossing in > map_private_extent_buffer() will return a '1'

[PATCH] btrfs: Refactor btrfs_merge_bio_hook

2018-11-27 Thread Nikolay Borisov
This function really checks whether adding more data to the bio will straddle a stripe/chunk. So first let's give it a more appropraite name - btrfs_bio_fits_in_stripe. Secondly, the offset parameter was never used to just remove it. Thirdly, pages are submitted to either btree or data inodes so

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 21:08 ч., Noah Massey wrote: > On Tue, Nov 27, 2018 at 11:43 AM Nikolay Borisov wrote: >> >> On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: >>> Document why map_private_extent_buffer() cannot return '1' (i.e. the map >>> spans two pages) for the csum_tree_block() case. >>>

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Noah Massey
On Tue, Nov 27, 2018 at 11:43 AM Nikolay Borisov wrote: > > On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > > Document why map_private_extent_buffer() cannot return '1' (i.e. the map > > spans two pages) for the csum_tree_block() case. > > > > The current algorithm for detecting a page

Re: [PATCH 5/6] btrfs: introduce delayed_refs_rsv

2018-11-27 Thread Josef Bacik
On Mon, Nov 26, 2018 at 11:14:12AM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > > From: Josef Bacik > > > > Traditionally we've had voodoo in btrfs to account for the space that > > delayed refs may take up by having a global_block_rsv. This works most > >

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-27 Thread Josef Bacik
On Fri, Nov 23, 2018 at 04:59:32PM +, Filipe Manana wrote: > On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > > > I noticed in a giant dbench run that we spent a lot of time on lock > > contention while running transaction commit. This is because dbench > > results in a lot of

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-27 Thread Filipe Manana
On Tue, Nov 27, 2018 at 7:22 PM Josef Bacik wrote: > > On Fri, Nov 23, 2018 at 04:59:32PM +, Filipe Manana wrote: > > On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > > > > > I noticed in a giant dbench run that we spent a lot of time on lock > > > contention while running transaction

Re: [PATCH 5/8] btrfs: don't enospc all tickets on flush failure

2018-11-27 Thread Josef Bacik
On Mon, Nov 26, 2018 at 02:25:52PM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 21:03 ч., Josef Bacik wrote: > > With the introduction of the per-inode block_rsv it became possible to > > have really really large reservation requests made because of data > > fragmentation. Since the

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 07:43:39PM +, Filipe Manana wrote: > On Tue, Nov 27, 2018 at 7:22 PM Josef Bacik wrote: > > > > On Fri, Nov 23, 2018 at 04:59:32PM +, Filipe Manana wrote: > > > On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > > > > > > > I noticed in a giant dbench run

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 10:29:57AM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > > The throttle path doesn't take cleaner_delayed_iput_mutex, which means > > Which one is the throttle path? btrfs_end_transaction_throttle is only > called during snapshot drop

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 07:59:42PM +, Chris Mason wrote: > On 27 Nov 2018, at 14:54, Josef Bacik wrote: > > > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: > >> > >> > >> On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > >>> The cleaner thread usually takes care of delayed

[PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: ordered extent flags. Signed-off-by: David Sterba --- fs/btrfs/ordered-data.h | 45 +++-- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ordered-data.h

[PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: tree lock types. Signed-off-by: David Sterba --- fs/btrfs/locking.h | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/locking.h b/fs/btrfs/locking.h index 29135def468e..684d0ef4faa4

[PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: root tree flags. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 33 + 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: internal filesystem states. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 63 1 file changed, 31 insertions(+), 32 deletions(-) diff --git a/fs/btrfs/ctree.h

[PATCH 0/9] Switch defines to enums

2018-11-27 Thread David Sterba
This is motivated by a merging mistake that happened a few releases ago. Two patches updated BTRFS_FS_* flags independently to the same value, git did not see any direct merge conflict. The two values got mixed at runtime and caused crash. Update all #define sequential values, the above merging

[PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: global filesystem states. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 25 +++-- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: block reserve types. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

Balance: invalid convert data profile raid10

2018-11-27 Thread Mikko Merikivi
I seem unable to convert an existing btrfs device array to RAID 10. Since it's pretty much RAID 0 and 1 combined, and 5 and 6 are unstable, it's what I would like to use. After I did tried this with 4.19.2-arch1-1-ARCH and btrfs-progs v4.19, I updated my system and tried btrfs balance again with

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > > The cleaner thread usually takes care of delayed iputs, with the > > exception of the btrfs_end_transaction_throttle path. The cleaner > > thread only gets woken up every 30

[PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: extent map flags. Signed-off-by: David Sterba --- fs/btrfs/extent_map.h | 21 ++--- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/extent_map.h b/fs/btrfs/extent_map.h index

[PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: extent buffer flags; Signed-off-by: David Sterba --- fs/btrfs/extent_io.h | 28 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index

[PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread David Sterba
The first auto-assigned value to enum is 0, we can use that and not initialize all members where the auto-increment does the same. This is used for values that are not part of on-disk format. Signed-off-by: David Sterba --- fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/ctree.h | 28

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Chris Mason
On 27 Nov 2018, at 14:54, Josef Bacik wrote: > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: >> >> >> On 21.11.18 г. 21:09 ч., Josef Bacik wrote: >>> The cleaner thread usually takes care of delayed iputs, with the >>> exception of the btrfs_end_transaction_throttle path. The

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Noah Massey
On Tue, Nov 27, 2018 at 2:32 PM Nikolay Borisov wrote: > > On 27.11.18 г. 21:08 ч., Noah Massey wrote: > > On Tue, Nov 27, 2018 at 11:43 AM Nikolay Borisov wrote: > >> > >> On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > >>> Document why map_private_extent_buffer() cannot return '1' (i.e.

Re: [PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:48PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > root tree flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 33 + > 1 file changed,

Re: [PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:50PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent buffer flags; This one has a "swtich" typo in the subject. Otherwise, Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- >

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:41PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > global filesystem states. Reviewed-by: Omar Sandoval Some typos/wording suggestions below. > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 25

Re: [PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:45PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > internal filesystem states. Hah, looks like we never had a bit 0 ;) Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 63

Re: [PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:43PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > block reserve types. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 19 --- > 1 file changed, 12

Re: [PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:52PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent map flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/extent_map.h | 21 ++--- > 1 file changed, 14

Re: Balance: invalid convert data profile raid10

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午5:16, Mikko Merikivi wrote: > I seem unable to convert an existing btrfs device array to RAID 10. > Since it's pretty much RAID 0 and 1 combined, and 5 and 6 are > unstable, it's what I would like to use. > > After I did tried this with 4.19.2-arch1-1-ARCH and btrfs-progs v4.19,

Re: [PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > internal filesystem states. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 63

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > tree lock types. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/locking.h | 10 ++ > 1 file changed, 6 insertions(+), 4

Re: [PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent map flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/extent_map.h | 21 ++--- > 1 file changed, 14

Re: [PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent buffer flags; > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/extent_io.h | 28 > 1 file changed, 16

Re: [PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > The first auto-assigned value to enum is 0, we can use that and not > initialize all members where the auto-increment does the same. This is > used for values that are not part of on-disk format. > > Signed-off-by: David Sterba Reviewed-by: Qu

[PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-27 Thread Lu Fengqi
The @found is always false when it comes to the if branch. Besides, the bool type is more suitable for @found. Signed-off-by: Lu Fengqi --- fs/btrfs/extent_io.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index

[PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-27 Thread Lu Fengqi
The generic_write_checks will check the combination of IOCB_NOWAIT and !IOCB_DIRECT. Signed-off-by: Lu Fengqi --- fs/btrfs/file.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 3835bb8c146d..190db9a685a2 100644 --- a/fs/btrfs/file.c +++

[PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-27 Thread Lu Fengqi
When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer after aborting a transaction"), it's useless. Signed-off-by: Lu Fengqi --- fs/btrfs/transaction.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:55PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > tree lock types. > > Signed-off-by: David Sterba > --- > fs/btrfs/locking.h | 10 ++ > 1 file changed, 6 insertions(+), 4 deletions(-) > > diff --git

Re: [PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > block reserve types. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo However more comment will always be a good thing. Thanks, Qu > --- > fs/btrfs/ctree.h | 19

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > global filesystem states. > > Signed-off-by: David Sterba Good comment. Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 25 +++-- > 1 file

RE: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges of "subvolume list/show"

2018-11-27 Thread misono.tomoh...@fujitsu.com
Hi, > -Original Message- > From: Martin Steigerwald [mailto:mar...@lichtvoll.de] > Sent: Tuesday, November 27, 2018 6:48 PM > To: Misono, Tomohiro > Cc: linux-btrfs@vger.kernel.org > Subject: Re: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges > of "subvolume list/show" > >

Re: [PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > ordered extent flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ordered-data.h | 45 +++-- > 1

[RFC PATCH 12/17] btrfs: priority alloc: introduce find_free_extent_search()

2018-11-27 Thread Su Yue
In origin, find_free_extent() just searches block groups in space_info one by one. In priority aware allocator, we first search block groups in higher priority tree than in lower priority tree. This helper unify above two ways for further use. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c

[RFC PATCH 13/17] btrfs: priority alloc: modify find_free_extent() to fit priority allocator

2018-11-27 Thread Su Yue
Add member priority_tree to find_free_extent_ctl to represents the tree using. Modify find_free_extent to use find_free_extent_search, so it can work in default mount option and priorit aware allocator. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 114

[RFC PATCH 06/17] btrfs: priority alloc: introduce three macros to mark block group status

2018-11-27 Thread Su Yue
In origin design, the order of space_info::block_groups is changed only when add/remove block group(s). In design of priority aware allocator, block groups may be moved from one priority tree to another one. What the operation is usually that 1) lock block_group down_write first_tree

[RFC PATCH 03/17] btrfs: priority alloc: introduce compute_block_group_priority/usage

2018-11-27 Thread Su Yue
Introduce compute_block_group_usage() and compute_block_group_usage(). And call the latter in btrfs_make_block_group() and btrfs_read_block_groups(). compute_priority_level use ilog2(free) to compute priority level. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60

[RFC PATCH 11/17] btrfs: priority alloc: remove block group from priority tree while removing block group

2018-11-27 Thread Su Yue
Export btrfs_remove_block_group_priority() to header ctree.h. Call btrfs_remove_block_group_priority() while deleting transaction->deleted_bgs, btrfs_free_block_groups() and btrfs_remove_block_group(). Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 3 +++

[RFC PATCH 02/17] btrfs: add mount definition BTRFS_MOUNT_PRIORITY_USAGE

2018-11-27 Thread Su Yue
Now implementation of priority allocator only support usage option. Add BTRFS_MOUNT_PRIORITY_USAGE for further commits. Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5c4651d8a524..4c56baf9f7cf 100644 ---

[RFC PATCH 16/17] btrfs: priority alloc: write bg->priority_tree->groups_sem to avoid race in btrfs_delete_unused_bgs()

2018-11-27 Thread Su Yue
If use priority aware allocator, bg->priority_tree->groups_sem should be written instead of space_info->groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60 +++--- 1 file changed, 44 insertions(+), 16 deletions(-) diff --git

[RFC PATCH 08/17] btrfs: priority alloc: add btrfs_update_block_group_priority()

2018-11-27 Thread Su Yue
Introduce btrfs_update_block_group_priority() to update block_groups::priority. It will move block group from old tree to new tree if need. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76 ++ 1 file changed, 76 insertions(+) diff --git

[RFC PATCH 04/17] btrfs: priority alloc: add functions to create/remove priority trees

2018-11-27 Thread Su Yue
Introduce create_priority_trees() to create priority trees in space_info. Introduce remove_priority_trees() to remove priority trees in space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 94 ++ 1 file changed, 94 insertions(+) diff --git

[RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-11-27 Thread Su Yue
This patchset can be fetched from repo: https://github.com/Damenly/btrfs-devel/commits/priority_aware_allocator. Since patchset 'btrfs: Refactor find_free_extent()' does a nice work to simplify find_free_extent(). This patchset dependents on the refactor. The base is the commit in kdave/misc-next:

[RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-27 Thread Su Yue
To implement priority aware allocator, this patch: Introduces struct btrfs_priority_tree which contains block groups in same level. Adds member priority to struct btrfs_block_group_cache and pointer points to the priority tree it's located. Adds member priority_trees to struct btrfs_space_info to

[RFC PATCH 10/17] btrfs: priority alloc: call add_block_group_priority while reading or making block group

2018-11-27 Thread Su Yue
Add block group to priority tree in btrfs_read_block_groups() and btrfs_make_block_groups(). Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 2dec02782df1..fc40901b4772 100644 ---

[RFC PATCH 14/17] btrfs: priority alloc: introduce btrfs_set_bg_updating and call btrfs_update_block_group_prioriy

2018-11-27 Thread Su Yue
For usage as priority, the varaiables in block groups we concered are reserved, bytes_super and btrfs_block_group_used(>item). This patch calls btrfs_set_bg_updating() in locations where above three varaiables changed to mark block groups needs to be updated, then calls btrfs_update_block_group()

[RFC PATCH 05/17] btrfs: priority alloc: introduce functions to add block group to priority tree

2018-11-27 Thread Su Yue
Introduce compute_priority_level() to compute priority level according priority, now just divides PRIORITY_USAGE_FACOTR. Introduce add_block_group_priority() to add block groups to priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76 ++

[RFC PATCH 07/17] btrfs: priority alloc: add functions to remove block group from priority tree

2018-11-27 Thread Su Yue
Introduce btrfs_remove_block_group_priority() to remove block group from priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 37 + 1 file changed, 37 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index

[RFC PATCH 09/17] btrfs: priority alloc: call create/remove_priority_trees in space_info

2018-11-27 Thread Su Yue
Call create_priority_trees() in create_space_info(). Call remove_priority_trees() before free of space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[RFC PATCH 17/17] btrfs: add mount option "priority_alloc=%s"

2018-11-27 Thread Su Yue
Add mount option "priority_alloc=%s", now %s only supports "usage" and "off". The latter is used for remount. "priority_alloc=usage" will active priority aware allocator. This mount option changes the way of find_free_extent to search block groups and may cost more time. Signed-off-by: Su Yue

[RFC PATCH 15/17] btrfs: priority alloc: write bg->priority_groups_sem while waiting reservation

2018-11-27 Thread Su Yue
Since if use priority alloc, we should down/up_write() bg->priority_groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f530a4344368..6627bbe56ad5 100644

Re: [PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:57PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > ordered extent flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ordered-data.h | 45

Re: [PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:59PM +0100, David Sterba wrote: > The first auto-assigned value to enum is 0, we can use that and not > initialize all members where the auto-increment does the same. This is > used for values that are not part of on-disk format. Reviewed-by: Omar Sandoval >

Re: [PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > root tree flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 33 + > 1 file changed, 17

Re: [PATCH 0/9] Switch defines to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > This is motivated by a merging mistake that happened a few releases ago. > Two patches updated BTRFS_FS_* flags independently to the same value, > git did not see any direct merge conflict. The two values got mixed at > runtime and caused crash. > >

[RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Lu Fengqi
The generic/513 tell that cloning into a file did not strip security privileges (suid, capabilities) like a regular write would. Signed-off-by: Lu Fengqi --- The xfs and ocfs2 call generic_remap_file_range_prep to drop file privileges, I'm not sure whether btrfs should do the same thing. Any

[PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Su Yue
In function do_trimming(), block_group->lock should be unlocked first. Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") Signed-off-by: Su Yue --- fs/btrfs/free-space-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/free-space-cache.c

Re: [RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午11:11, Su Yue wrote: > This patchset can be fetched from repo: > https://github.com/Damenly/btrfs-devel/commits/priority_aware_allocator. > Since patchset 'btrfs: Refactor find_free_extent()' does a nice work > to simplify find_free_extent(). This patchset dependents on the

[PATCH] btrfs: qgroup: Introduce more accurate early reloc tree detection

2018-11-27 Thread Qu Wenruo
The biggest challenge for qgroup to skip reloc tree extents is to detect correct owner of reloc tree blocks owner. Unlike most data operations, the root of tree reloc tree can't be easily detected. For example, for relocation we call btrfs_copy_root to init reloc tree: btrfs_copy_root(root=257,

Re: [PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Lu Fengqi wrote: > The @found is always false when it comes to the if branch. Besides, the > bool type is more suitable for @found. Well if you are ranging the type of found variable it also makes sense to change the return value of the function to bool as well. > >

Re: [PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:22 ч., Lu Fengqi wrote: > When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer > after aborting a transaction"), it's useless. > > Signed-off-by: Lu Fengqi Reviewed-by: Nikolay Borisov > --- > fs/btrfs/transaction.c | 1 - > 1 file changed, 1 deletion(-)

Re: [PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:23 ч., Lu Fengqi wrote: > The generic_write_checks will check the combination of IOCB_NOWAIT and > !IOCB_DIRECT. True, however btrfs will return ENOSUPP whereas the generic code returns EINVAL. I guess this is not a big deal and it's likely generic code is correct, so:

Re: [PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Su Yue wrote: > In function do_trimming(), block_group->lock should be unlocked first. > > Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") > Signed-off-by: Su Yue Reviewed-by: Nikolay Borisov > --- > fs/btrfs/free-space-cache.c | 2 +- > 1 file

Re: Balance: invalid convert data profile raid10

2018-11-27 Thread Mikko Merikivi
Well, excuse me for thinking it wouldn't since in md-raid it worked. https://wiki.archlinux.org/index.php/RAID#RAID_level_comparison Anyway, the error message is truly confusing for someone who doesn't know about btrfs's implementation. I suppose in md-raid the near layout is actually RAID 1 and

[PATCH v2 5/5] btrfs-progs: misc-tests/021: Do extra btrfs check before mounting

2018-11-27 Thread Qu Wenruo
Test case misc/021 is testing if we could mount a single disk btrfs image recovered from multi disk fs. The problem is, current kernel has extra check for block group, chunk and dev extent. This means any image can't pass btrfs check for chunk tree will not mount. So do extra btrfs check before

[PATCH v2 4/5] btrfs-progs: image: Remove all existing dev extents for later rebuild

2018-11-27 Thread Qu Wenruo
For multi-disk btrfs image recovered to single disk, the dev tree would look like: item 2 key (1 DEV_EXTENT 22020096) dev extent chunk_tree 3 chunk_objectid 256 chunk_offset 22020096 length 8388608 item 3 key (1 DEV_EXTENT 30408704)

[PATCH v2 0/5] btrfs-progs: image: Fix error when restoring multi-disk image to single disk

2018-11-27 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/image_recover The base commit is as usual, the latest stable tag, v4.19. Test case misc/021 will fail if using latest upstream kernel. This is due to the enhanced code in kernel to check block group <->

[PATCH v2 3/5] btrfs-progs: volumes: Refactor btrfs_alloc_dev_extent() into two functions

2018-11-27 Thread Qu Wenruo
We have btrfs_alloc_dev_extent() accepting @convert flag to toggle special handling for convert. However that @convert flag only determine whether we call find_free_dev_extent(), and we may later need to insert dev extents without searching dev tree. So refactor btrfs_alloc_dev_extent() into 2

[PATCH v2 2/5] btrfs-progs: image: Fix block group item flags when restoring multi-device image to single device

2018-11-27 Thread Qu Wenruo
Since btrfs-image is just restoring tree blocks without really check if that tree block contents makes sense, for multi-device image, block group items will keep that incorrect block group flags. For example, for a metadata RAID1 data RAID0 btrfs recovered to a single disk, its chunk tree will

[PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-27 Thread Qu Wenruo
Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM size. Later we need to do more fixup works, so change the name to fixup_chunks_and_devices() and refactor the original device size fixup to fixup_device_size(). Signed-off-by: Qu Wenruo --- image/main.c | 52

Re: unable to fixup (regular) error

2018-11-27 Thread Alexander Fieroch
Actually the data on raid0 is not my data but of my users and they knew and accepted the risk for raid0. So in my case it should be ok - I don't know the importance of the data files which are affected. I just wanted to help finding a possible bug and experiment with a broken btrfs filesystem

Re: [PATCH v2 3/5] btrfs-progs: volumes: Refactor btrfs_alloc_dev_extent() into two functions

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 10:38 ч., Qu Wenruo wrote: > We have btrfs_alloc_dev_extent() accepting @convert flag to toggle > special handling for convert. > > However that @convert flag only determine whether we call > find_free_dev_extent(), and we may later need to insert dev extents > without searching

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Nikolay Borisov
On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > The cleaner thread usually takes care of delayed iputs, with the > exception of the btrfs_end_transaction_throttle path. The cleaner > thread only gets woken up every 30 seconds, so instead wake it up to do > it's work so that we can free up that

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-27 Thread Qu Wenruo
On 2018/11/27 下午4:46, Nikolay Borisov wrote: > > > On 27.11.18 г. 10:38 ч., Qu Wenruo wrote: >> Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM >> size. >> Later we need to do more fixup works, so change the name to >> fixup_chunks_and_devices() and refactor the original

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-11-27 Thread Nikolay Borisov
On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > The throttle path doesn't take cleaner_delayed_iput_mutex, which means Which one is the throttle path? btrfs_end_transaction_throttle is only called during snapshot drop and relocation? What scenario triggered your observation and prompted this

Re: Btrfs and fanotify filesystem watches

2018-11-27 Thread Jan Kara
On Fri 23-11-18 19:53:11, Amir Goldstein wrote: > On Fri, Nov 23, 2018 at 3:34 PM Amir Goldstein wrote: > > > So open_by_handle() should work fine even if we get mount_fd of /subvol1 > > > and handle for inode on /subvol2. mount_fd in open_by_handle() is really > > > only used to get the

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 10:38 ч., Qu Wenruo wrote: > Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM > size. > Later we need to do more fixup works, so change the name to > fixup_chunks_and_devices() and refactor the original device size fixup > to fixup_device_size(). > >

  1   2   >