Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-27 Thread Marek Szyprowski
Hi Ming, On 2019-02-28 00:29, Ming Lei wrote: > On Wed, Feb 27, 2019 at 08:47:09PM +, Jon Hunter wrote: >> On 21/02/2019 08:42, Marek Szyprowski wrote: >>> On 2019-02-15 12:13, Ming Lei wrote: This patch pulls the trigger for multi-page bvecs. Reviewed-by: Omar Sandoval Si

[PATCH v4] btrfs-progs: dump-tree: add noscan option

2019-02-27 Thread Anand Jain
From: Anand Jain There is no way to dump tree from a disk in the degraded mode. Such as, when you specify a device for the cli 'btrfs inspect dump-tree /dev/sda' it would invariably scan for its partner devices to built and print the tree. This approach at times defeats the purpose. Suppose in RA

[PATCH v2] btrfs-progs: gitignore: Ignore hidden files

2019-02-27 Thread Qu Wenruo
A lot of editor/IDE related config files are dotfiles, like .vimrc or .clang_complete. Instead of adding gitignore entry for each editor/IDE, just ignore all dotfiles. Signed-off-by: Qu Wenruo --- changelog v2: - Change "hidden files" to "dotfiles" in the commit message. - No longer mention the

Re: [PATCH v2] btrfs: extent_io: Always return error pointer for extent buffer allocation failure

2019-02-27 Thread Qu Wenruo
[snip] >> ret = radix_tree_preload(GFP_NOFS); >> -if (ret) >> -goto free_eb; >> +if (ret) { >> +btrfs_release_extent_buffer(eb); >> +return ERR_PTR(ret); >> +} >> spin_lock(&fs_info->buffer_lock); >> ret = radix_tree_insert(&fs_info->bu

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-27 Thread Ming Lei
On Wed, Feb 27, 2019 at 08:47:09PM +, Jon Hunter wrote: > > On 21/02/2019 08:42, Marek Szyprowski wrote: > > Dear All, > > > > On 2019-02-15 12:13, Ming Lei wrote: > >> This patch pulls the trigger for multi-page bvecs. > >> > >> Reviewed-by: Omar Sandoval > >> Signed-off-by: Ming Lei > >

Re: [PATCH v2] btrfs: zstd ensure reclaim timer is properly cleaned up

2019-02-27 Thread Dennis Zhou
On Wed, Feb 27, 2019 at 07:36:50PM +0100, David Sterba wrote: > On Wed, Feb 27, 2019 at 01:29:16PM -0500, Dennis Zhou wrote: > > > I've noticed while reading the code, why do you use the indirect call > > > here? The wsm.ops points to btrfs_zstd_compress so free_workspace is > > > always zstd_free_

[PATCH] btrfs: remove indirect function calls from zstd

2019-02-27 Thread Dennis Zhou
While calling functions inside zstd, we don't need to use the indirection provided by the workspace_manager. Forward declarations are added to maintain the function order of btrfs_compress_op. Signed-off-by: Dennis Zhou --- fs/btrfs/zstd.c | 11 +++ 1 file changed, 7 insertions(+), 4 del

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-27 Thread Jon Hunter
On 21/02/2019 08:42, Marek Szyprowski wrote: > Dear All, > > On 2019-02-15 12:13, Ming Lei wrote: >> This patch pulls the trigger for multi-page bvecs. >> >> Reviewed-by: Omar Sandoval >> Signed-off-by: Ming Lei > > Since Linux next-20190218 I've observed problems with block layer on one > of

Re: [PATCH v2] Btrfs: fix file corruption after snapshotting due to mix of buffered/DIO writes

2019-02-27 Thread Filipe Manana
On Wed, Feb 27, 2019 at 6:30 PM David Sterba wrote: > > On Wed, Feb 27, 2019 at 05:32:55PM +, Filipe Manana wrote: > > On Wed, Feb 27, 2019 at 5:25 PM David Sterba wrote: > > > > > > On Wed, Feb 27, 2019 at 01:42:30PM +, fdman...@kernel.org wrote: > > > > @@ -1897,15 +1899,45 @@ static in

Re: [PATCH v2] btrfs: zstd ensure reclaim timer is properly cleaned up

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 01:29:16PM -0500, Dennis Zhou wrote: > > I've noticed while reading the code, why do you use the indirect call > > here? The wsm.ops points to btrfs_zstd_compress so free_workspace is > > always zstd_free_workspace. > > > > The compiler is usually smart to replace such thin

Re: [PATCH v2] Btrfs: fix file corruption after snapshotting due to mix of buffered/DIO writes

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 05:32:55PM +, Filipe Manana wrote: > On Wed, Feb 27, 2019 at 5:25 PM David Sterba wrote: > > > > On Wed, Feb 27, 2019 at 01:42:30PM +, fdman...@kernel.org wrote: > > > @@ -1897,15 +1899,45 @@ static inline int > > > btrfs_start_delalloc_flush(struct btrfs_fs_info *

Re: [PATCH v2] btrfs: zstd ensure reclaim timer is properly cleaned up

2019-02-27 Thread Dennis Zhou
On Wed, Feb 27, 2019 at 05:44:41PM +0100, David Sterba wrote: > On Fri, Feb 22, 2019 at 02:53:48PM -0500, Dennis Zhou wrote: > > The timer function, zstd_reclaim_timer_fn(), reschedules itself under > > certain conditions. When cleaning up, take the lock and remove all > > workspaces. This prevents

Re: Btrfs progs release 4.20.2

2019-02-27 Thread David Sterba
On Tue, Feb 26, 2019 at 01:06:48PM +0800, Anand Jain wrote: > > > https://patchwork.kernel.org/patch/10732689/ > > Would like to bring to your notice. Above patch is not yet integrated > for a long time. Any comments? I'm too busy with kernel patches to don't at progs very often. For unknown

Re: [PATCH v2] Btrfs: fix file corruption after snapshotting due to mix of buffered/DIO writes

2019-02-27 Thread Filipe Manana
On Wed, Feb 27, 2019 at 5:25 PM David Sterba wrote: > > On Wed, Feb 27, 2019 at 01:42:30PM +, fdman...@kernel.org wrote: > > @@ -1897,15 +1899,45 @@ static inline int btrfs_start_delalloc_flush(struct > > btrfs_fs_info *fs_info) > >* from already being in a transaction and our join_tr

Re: [PATCH v2] Btrfs: fix file corruption after snapshotting due to mix of buffered/DIO writes

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 01:42:30PM +, fdman...@kernel.org wrote: > @@ -1897,15 +1899,45 @@ static inline int btrfs_start_delalloc_flush(struct > btrfs_fs_info *fs_info) >* from already being in a transaction and our join_transaction doesn't >* have to re-take the fs freeze lock

Re: [PATCH] Btrfs: fix file corruption after snapshotting

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 01:42:31PM +, Filipe Manana wrote: > > > > So, how about calling writeback_inodes_sb in that case as a fallback? > > > > > > Thought about it, but the reason I didn't do it is that if you > > > fallback to writeback_unodes_sb, you'll never have the error reported > > > t

Re: [PATCH v2] btrfs: extent_io: Always return error pointer for extent buffer allocation failure

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 12:38:01PM +0800, Qu Wenruo wrote: > There are 5 extent buffer allocation functions in btrfs: > __alloc_extent_buffer(); > alloc_extent_buffer(); > __alloc_dummy_extent_buffer(); > alloc_dummy_extent_buffer(); > alloc_test_extent_buffer(); > > However they return different

Re: [PATCH 1/3] btrfs: Initialize btrfs_io_ctl instead of memsetting it

2019-02-27 Thread Goldwyn Rodrigues
On 16:47 27/02, David Sterba wrote: > On Mon, Feb 25, 2019 at 01:07:42PM -0600, Goldwyn Rodrigues wrote: > > From: Goldwyn Rodrigues > > > > io_ctl_init() memsets it to zero anyways. However, I presume the > > memset was added to avoid the WARN_ON in io_ctl_init(). > > I don't see any WARN_ON in

Re: [PATCH v2] btrfs: zstd ensure reclaim timer is properly cleaned up

2019-02-27 Thread David Sterba
On Fri, Feb 22, 2019 at 02:53:48PM -0500, Dennis Zhou wrote: > The timer function, zstd_reclaim_timer_fn(), reschedules itself under > certain conditions. When cleaning up, take the lock and remove all > workspaces. This prevents the timer from rearming itself. Lastly, switch > to del_timer_sync()

Re: [PATCH] btrfs-progs: balance: Sync the fs before balancing metadata chunks

2019-02-27 Thread David Sterba
On Tue, Feb 26, 2019 at 01:55:06PM +0800, Qu Wenruo wrote: > > > On 2019/2/26 上午1:21, David Sterba wrote: > > On Tue, Jan 29, 2019 at 02:57:39PM +0800, Qu Wenruo wrote: > >> The most straightforward is to sync the fs before balancing metadata > >> chunks. > >> > >> We could enhance the kernel byt

Re: [PATCH v4 10/10] btrfs: kill btrfs_setxattr

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 01:39:52AM +0800, Anand Jain wrote: > Now btrfs_setxattr() is a very small function with just check for > readonly FS and redirect the call to do_setxattr(). So instead > move that checks to the parent functions. > > Signed-off-by: Anand Jain > --- > v4: born > fs/btrfs/a

Re: [PATCH v4 09/10] btrfs: btrfs_setxattr argument trans is never NULL

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 01:39:51AM +0800, Anand Jain wrote: > The following patches >btrfs: cleanup btrfs_set_acl >btrfs: start transaction in btrfs_xattr_handler_set >btrfs: create transaction in btrfs_set_prop_notrans > > made the btrfs_setxattr() argument trans to be never NULL, so

Re: [PATCH v4 04/10] btrfs: rename fs_info argument to fs_private

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 01:39:46AM +0800, Anand Jain wrote: > fs_info is commonly used to represent struct fs_info *, rename > to fs_private to avoid confusion. Yeah, I had the same thoughts. The parameter name in interface function security_inode_init_security is fs_data, I think we can use that

Re: [PATCH v4 06/10] btrfs: start transaction in btrfs_set_prop_trans

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 01:39:48AM +0800, Anand Jain wrote: > In preparation to make trans argument of btrfs_setxattr() a mandatory, > start the transaction in btrfs_set_prop_trans. > > Signed-off-by: Anand Jain > --- > v4: born > fs/btrfs/props.c | 21 - > 1 file changed, 20

Re: [PATCH v4 03/10] btrfs: trivial, fix c coding style

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 01:39:45AM +0800, Anand Jain wrote: > Maintain the lines extented upto 80 char where possible, and indent the > argument. > > Signed-off-by: Anand Jain > Reviewed-by: Nikolay Borisov I don't think such patches are necessary, there's a lot of strange formatting from the p

Re: [PATCH 1/3] btrfs: Initialize btrfs_io_ctl instead of memsetting it

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 01:07:42PM -0600, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > io_ctl_init() memsets it to zero anyways. However, I presume the > memset was added to avoid the WARN_ON in io_ctl_init(). I don't see any WARN_ON in io_ctl_init, you probably mean __btrfs_write_out_

Re: [PATCH 2/3] btrfs: Initialize inode->i_op once in btrfs_symlink

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 01:07:43PM -0600, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > inode->i_op is initialized multiple times. Perform it once. > > Signed-off-by: Goldwyn Rodrigues Leftover from 4779cc04248deff6 Reviewed-by: David Sterba

Re: [PATCH 3/3] btrfs: Perform locking/unlocking in btrfs_remap_file_range()

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 03:52:42PM +0100, David Sterba wrote: > > > > - struct btrfs_root *root_out = BTRFS_I(inode_out)->root; > > > > - > > > > - if (btrfs_root_readonly(root_out))xfs_reflink_remap_prep > > > > - return -EROFS; > > > > - > > > > -

Re: [PATCH 3/3] btrfs: Perform locking/unlocking in btrfs_remap_file_range()

2019-02-27 Thread David Sterba
On Tue, Feb 26, 2019 at 06:57:17AM -0600, Goldwyn Rodrigues wrote: > On 12:08 26/02, Filipe Manana wrote: > > On Mon, Feb 25, 2019 at 7:08 PM Goldwyn Rodrigues wrote: > > > > > > From: Goldwyn Rodrigues > > > > > > Moves code to make it more readable, so as locking and unlocking is > > > done in

Re: [PATCH v2] btrfs: Don't panic when we can't find a root key

2019-02-27 Thread David Sterba
On Tue, Feb 26, 2019 at 04:33:56PM +0800, Qu Wenruo wrote: > When we failed to find a root key in btrfs_update_root(), we just panic. > > That's definitely not cool, fix it by outputting an unique error > message, aborting current transaction and return -EUCLEAN. > > Signed-off-by: Qu Wenruo Re

Re: [PATCH] Btrfs: fix corruption reading shared and compressed extents after hole punching

2019-02-27 Thread David Sterba
On Thu, Feb 14, 2019 at 03:17:20PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > In the past we had data corruption when reading compressed extents that > are shared within the same file and they are consecutive, this got fixed > by commit 005efedf2c7d0 ("Btrfs: fix read corruption

Re: [PATCH 0/5] Unify the return value of alloc/clone_extent_buffer()

2019-02-27 Thread David Sterba
On Fri, Feb 22, 2019 at 09:31:29PM +0800, Qu Wenruo wrote: > >> Nope. > >> > >> for (i = 0; i < num_pages; i++) { > >> eb->pages[i] = alloc_page(GFP_NOFS); > >> if (!eb->pages[i]) > >> goto err; <<< Page alloc failure here > >>

Re: [PATCH] btrfs-progs: gitignore: Ignore hidden files

2019-02-27 Thread Mike Fleetwood
On Wed, 27 Feb 2019 at 11:44, Stefan Traby wrote: > > On Wed, Feb 27, 2019 at 07:29:13PM +0800, Qu Wenruo wrote: > > On 2019/2/27 下午7:22, Mike Fleetwood wrote: > > > On Wed, 27 Feb 2019 at 05:16, Qu Wenruo wrote: > > >> > > >> A lot of editor/IDE related config files are hidden files, like .vimrc

Re: [PATCH 1/5] btrfs: extent_io: Add comment about the return value of alloc_extent_buffer()

2019-02-27 Thread David Sterba
On Wed, Feb 27, 2019 at 09:41:17PM +0800, Qu Wenruo wrote: > > > On 2019/2/27 下午9:36, David Sterba wrote: > > On Fri, Feb 22, 2019 at 06:16:40PM +0800, Qu Wenruo wrote: > >> To inform later developers how to check the return value of it. > >> > >> Signed-off-by: Qu Wenruo > >> --- > >> fs/btrfs

Re: [PATCH] Btrfs: fix file corruption after snapshotting

2019-02-27 Thread Filipe Manana
On Wed, Feb 27, 2019 at 1:04 PM David Sterba wrote: > > On Mon, Feb 18, 2019 at 05:27:54PM +, Filipe Manana wrote: > > On Mon, Feb 18, 2019 at 5:09 PM David Sterba wrote: > > > > > > On Mon, Feb 04, 2019 at 02:28:10PM +, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > >

[PATCH v2] Btrfs: fix file corruption after snapshotting due to mix of buffered/DIO writes

2019-02-27 Thread fdmanana
From: Filipe Manana When we are mixing buffered writes with direct IO writes against the same file and snapshotting is happening concurrently, we can end up with a corrupt file content in the snapshot. Example: 1) Inode/file is empty. 2) Snapshotting starts. 2) Buffered write at offset 0 lengt

Re: [PATCH 1/5] btrfs: extent_io: Add comment about the return value of alloc_extent_buffer()

2019-02-27 Thread Qu Wenruo
On 2019/2/27 下午9:36, David Sterba wrote: > On Fri, Feb 22, 2019 at 06:16:40PM +0800, Qu Wenruo wrote: >> To inform later developers how to check the return value of it. >> >> Signed-off-by: Qu Wenruo >> --- >> fs/btrfs/extent_io.c | 7 +++ >> 1 file changed, 7 insertions(+) >> >> diff --git

Re: [PATCH v5.1 00/12] btrfs: Enhancement to tree block validation

2019-02-27 Thread Qu Wenruo
On 2019/2/27 下午8:22, David Sterba wrote: > On Sat, Feb 23, 2019 at 08:47:09AM +0800, Qu Wenruo wrote: >> >> >> On 2019/2/22 下午11:18, David Sterba wrote: >>> On Mon, Feb 18, 2019 at 01:27:41PM +0800, Qu Wenruo wrote: v5.1: - Add "block=%llu " output for write/read time error line. -

Re: [PATCH 1/5] btrfs: extent_io: Add comment about the return value of alloc_extent_buffer()

2019-02-27 Thread David Sterba
On Fri, Feb 22, 2019 at 06:16:40PM +0800, Qu Wenruo wrote: > To inform later developers how to check the return value of it. > > Signed-off-by: Qu Wenruo > --- > fs/btrfs/extent_io.c | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c > in

Re: [PATCH 2/2] btrfs: warn if extent buffer mapping crosses a page boundary in csum_tree_block

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 02:24:16PM +0100, Johannes Thumshirn wrote: > Since commit d2e174d5d3ee ("btrfs: document extent mapping assumptions in > checksum") we have a comment in place why map_private_extent_buffer() > can't return 1 in the csum_tree_block() case. > > Make this a bit more explicit

Re: [PATCH v4 1/2] btrfs: factor our read/write stage off csum_tree_block() into its callers

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 02:24:15PM +0100, Johannes Thumshirn wrote: > Currently csum_tree_block() does two things, first it as it's name > suggests it calculates the checksum for a tree-block. But it also writes > this checksum to disk or reads an extent_buffer from disk and compares the > checksum

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-27 Thread Matthew Wilcox
On Fri, Feb 22, 2019 at 09:45:05AM -0700, Keith Busch wrote: > On Thu, Feb 21, 2019 at 09:51:12PM -0500, Martin K. Petersen wrote: > > > > Keith, > > > > > With respect to fs block sizes, one thing making discards suck is that > > > many high capacity SSDs' physical page sizes are larger than the

Re: [PATCH 0/2] Fix missing reference aborts when resuming snapshot delete

2019-02-27 Thread David Sterba
On Wed, Feb 06, 2019 at 03:46:13PM -0500, Josef Bacik wrote: > With my delayed refs rsv patches in place we started hitting issues in our > build > servers that do a lot of snapshot deletions. Turns out there was a bug in > btrfs_end_transaction_throttle() that caused it to basically always commi

Re: [PATCH] Btrfs: fix file corruption after snapshotting

2019-02-27 Thread David Sterba
On Mon, Feb 18, 2019 at 05:27:54PM +, Filipe Manana wrote: > On Mon, Feb 18, 2019 at 5:09 PM David Sterba wrote: > > > > On Mon, Feb 04, 2019 at 02:28:10PM +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When we are mixing buffered writes with direct IO writes against

Re: [PATCH v2] btrfs: move ulist allocation out of transaction in quota enable

2019-02-27 Thread Filipe Manana
On Wed, Feb 27, 2019 at 12:12 PM David Sterba wrote: > > The allocation happens with GFP_KERNEL after a transaction has been > started, this can potentially cause deadlock if reclaim tries to get the > memory by flushing filesystem data. > > The fs_info::qgroup_ulist is not used during transaction

Re: [PATCH v5.1 00/12] btrfs: Enhancement to tree block validation

2019-02-27 Thread David Sterba
On Sat, Feb 23, 2019 at 08:47:09AM +0800, Qu Wenruo wrote: > > > On 2019/2/22 下午11:18, David Sterba wrote: > > On Mon, Feb 18, 2019 at 01:27:41PM +0800, Qu Wenruo wrote: > >> v5.1: > >> - Add "block=%llu " output for write/read time error line. > >> - Also output read time error message for fsid/

[PATCH v2] btrfs: move ulist allocation out of transaction in quota enable

2019-02-27 Thread David Sterba
The allocation happens with GFP_KERNEL after a transaction has been started, this can potentially cause deadlock if reclaim tries to get the memory by flushing filesystem data. The fs_info::qgroup_ulist is not used during transaction start when quotas are not enabled. The status bit BTRFS_FS_QUOTA

Re: [PATCH] btrfs: move ulist allocation out of transaction in quota enable

2019-02-27 Thread David Sterba
On Mon, Feb 25, 2019 at 07:09:02PM +, Filipe Manana wrote: > On Mon, Feb 25, 2019 at 6:58 PM David Sterba wrote: > > > > The allocation happens with GFP_KERNEL after a transaction has been > > started, this can potentially cause deadlock if reclaim tries to get the > > memory by flushing files

Re: [PATCH] btrfs-progs: gitignore: Ignore hidden files

2019-02-27 Thread Stefan Traby
On Wed, Feb 27, 2019 at 07:29:13PM +0800, Qu Wenruo wrote: > On 2019/2/27 下午7:22, Mike Fleetwood wrote: > > On Wed, 27 Feb 2019 at 05:16, Qu Wenruo wrote: > >> > >> A lot of editor/IDE related config files are hidden files, like .vimrc > >> or .clang_complete. > >> > >> Instead of adding gitignore

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-27 Thread Ric Wheeler
On 2/22/19 11:45 AM, Keith Busch wrote: On Thu, Feb 21, 2019 at 09:51:12PM -0500, Martin K. Petersen wrote: Keith, With respect to fs block sizes, one thing making discards suck is that many high capacity SSDs' physical page sizes are larger than the fs block size, and a sub-page discard is wo

Re: [PATCH] btrfs-progs: gitignore: Ignore hidden files

2019-02-27 Thread Qu Wenruo
On 2019/2/27 下午7:22, Mike Fleetwood wrote: > On Wed, 27 Feb 2019 at 05:16, Qu Wenruo wrote: >> >> A lot of editor/IDE related config files are hidden files, like .vimrc >> or .clang_complete. >> >> Instead of adding gitignore entry for each editor/IDE, just ignore all >> hidden files like what k

Re: [PATCH] btrfs-progs: gitignore: Ignore hidden files

2019-02-27 Thread Mike Fleetwood
On Wed, 27 Feb 2019 at 05:16, Qu Wenruo wrote: > > A lot of editor/IDE related config files are hidden files, like .vimrc > or .clang_complete. > > Instead of adding gitignore entry for each editor/IDE, just ignore all > hidden files like what kernel does. I think you mean: files starting with "."