Re: [PATCH] fs: use READ_ONCE/WRITE_ONCE with the i_size helpers

2019-10-11 Thread Rik van Riel
On Fri, 2019-10-11 at 16:20 -0400, Josef Bacik wrote: > > and this works out properly, we only read the value once and so we > won't > trip over this problem again. > > Signed-off-by: Josef Bacik Reviewed-by: Rik van Riel -- All Rights Reversed. signature.asc Description: This is a digitall

[PATCH] fs: use READ_ONCE/WRITE_ONCE with the i_size helpers

2019-10-11 Thread Josef Bacik
I spent the last few weeks running down a weird regression in btrfs we were seeing in production. It turned out to be introduced by 62b37622718c, which took the following loff_t isize = i_size_read(inode); actual_end = min_t(u64, isize, end + 1); and turned it into actual_end = min_t(u64, i_si

Re: [PATCH v3 1/3] btrfs: block-group: Fix a memory leak due to missing btrfs_put_block_group()

2019-10-11 Thread David Sterba
On Thu, Oct 10, 2019 at 10:39:26AM +0800, Qu Wenruo wrote: > In btrfs_read_block_groups(), if we have an invalid block group which > has mixed type (DATA|METADATA) while the fs doesn't have MIX_BGS > feature, we error out without freeing the block group cache. > > This patch will add the missing b

Re: [PATCH 2/5] btrfs: get bdev from latest_dev for dio bh_result

2019-10-11 Thread David Sterba
On Fri, Oct 11, 2019 at 08:26:42PM +0200, David Sterba wrote: > You're right, and actually I got crashes in clean_bdev_aliases when I > supplied a NULL bdev, so I'll add it to the changelog. Thanks. Unless there are further comments, I won't resend the whole patchset. The changelog in this patch w

Re: [PATCH] Btrfs: fix metadata space leak on fixup worker failure to set range as delalloc

2019-10-11 Thread David Sterba
On Wed, Oct 09, 2019 at 05:43:59PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > In the fixup worker, if we fail to mark the range as delalloc in the io > tree, we must release the previously reserved metadata, as well as update > the outstanding extents counter for the inode, other

Re: [PATCH 2/5] btrfs: get bdev from latest_dev for dio bh_result

2019-10-11 Thread David Sterba
On Wed, Oct 09, 2019 at 01:42:00PM +0300, Nikolay Borisov wrote: > > > On 7.10.19 г. 22:37 ч., David Sterba wrote: > > To remove use of extent_map::bdev we need to find a replacement, and the > > latest_bdev is the only one we can use here, because inode::i_bdev and > > superblock::s_bdev are NUL

Re: [PATCH v2] Btrfs: fix negative subv_writers counter and data space leak after buffered write

2019-10-11 Thread David Sterba
On Fri, Oct 11, 2019 at 04:41:20PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When doing a buffered write it's possible to leave the subv_writers > counter of the root, used for synchronization between buffered nocow > writers and snapshotting. This happens in an exceptional case

Re: rsync -ax and subvolumes

2019-10-11 Thread Andrei Borzenkov
11.10.2019 9:15, Ulli Horlacher пишет: > On Thu 2019-10-10 (20:47), Kai Krakow wrote: > >> Actually, you could also just bind-mount into /mnt/btrfs, bind-mounts >> won't inherit other mounts but will still see pure subvolumes. > > Next problem: > > root@ptm1:~# sysinfo > System:Linux pt

Re: [PATCH 10/19] btrfs: calculate discard delay based on number of extents

2019-10-11 Thread Dennis Zhou
On Thu, Oct 10, 2019 at 11:41:33AM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 04:17:41PM -0400, Dennis Zhou wrote: > > Use the number of discardable extents to help guide our discard delay > > interval. This value is reevaluated every transaction commit. > > > > Signed-off-by: Dennis Zhou

Re: [PATCH] Btrfs: add missing extents release on file extent cluster relocation error

2019-10-11 Thread David Sterba
On Wed, Oct 09, 2019 at 05:43:45PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > If we error out when finding a page at relocate_file_extent_cluster(), we > need to release the outstanding extents counter on the relocation inode, > set by the previous call to btrfs_delalloc_reserve_

Re: [PATCH] btrfs: ioctl: Try to use btrfs_fs_info instead of *file

2019-10-11 Thread David Sterba
On Thu, Oct 10, 2019 at 09:23:11PM -0300, Marcos Paulo de Souza wrote: > Some functions are doing some bikeshedding to reach the btrfs_fs_info > struct. Change these functions to receive a btrfs_fs_info struct instead > of a *file. Makes sense, thanks. Added to msic-next.

Re: [PATCH] btrfs: save i_size in compress_file_range

2019-10-11 Thread Josef Bacik
On Fri, Oct 11, 2019 at 04:28:28PM +0100, Filipe Manana wrote: > On Fri, Oct 11, 2019 at 2:05 PM Josef Bacik wrote: > > > > We hit a regression while rolling out 5.2 internally where we were > > hitting the following panic > > > > kernel BUG at mm/page-writeback.c:2659! > > RIP: 0010:clear_page_di

Re: [PATCH v2] Btrfs: fix negative subv_writers counter and data space leak after buffered write

2019-10-11 Thread Josef Bacik
On Fri, Oct 11, 2019 at 04:41:20PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When doing a buffered write it's possible to leave the subv_writers > counter of the root, used for synchronization between buffered nocow > writers and snapshotting. This happens in an exceptional case

Re: [PATCH 06/19] btrfs: handle empty block_group removal

2019-10-11 Thread Dennis Zhou
On Thu, Oct 10, 2019 at 11:00:42AM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 04:17:37PM -0400, Dennis Zhou wrote: > > block_group removal is a little tricky. It can race with the extent > > allocator, the cleaner thread, and balancing. The current path is for a > > block_group to be added

Re: [PATCH 04/19] btrfs: keep track of cleanliness of the bitmap

2019-10-11 Thread Dennis Zhou
On Thu, Oct 10, 2019 at 10:16:30AM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 04:17:35PM -0400, Dennis Zhou wrote: > > There is a cap in btrfs in the amount of free extents that a block group > > can have. When it surpasses that threshold, future extents are placed > > into bitmaps. Instea

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-11 Thread Dennis Zhou
On Thu, Oct 10, 2019 at 09:40:37AM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 06:38:10PM -0400, Dennis Zhou wrote: > > On Mon, Oct 07, 2019 at 04:37:28PM -0400, Josef Bacik wrote: > > > On Mon, Oct 07, 2019 at 04:17:34PM -0400, Dennis Zhou wrote: > > > > Async discard will use the free spa

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-11 Thread Dennis Zhou
On Tue, Oct 08, 2019 at 03:46:18PM +0300, Nikolay Borisov wrote: > > > On 7.10.19 г. 23:17 ч., Dennis Zhou wrote: > > Async discard will use the free space cache as backing knowledge for > > which extents to discard. This patch plumbs knowledge about which > > extents need to be discarded into th

[PATCH v2] Btrfs: fix negative subv_writers counter and data space leak after buffered write

2019-10-11 Thread fdmanana
From: Filipe Manana When doing a buffered write it's possible to leave the subv_writers counter of the root, used for synchronization between buffered nocow writers and snapshotting. This happens in an exceptional case like the following: 1) We fail to allocate data space for the write, since th

Re: [PATCH] Btrfs: fix negative subv_writers counter and data space leak after buffered write

2019-10-11 Thread Filipe Manana
On Fri, Oct 11, 2019 at 2:27 PM Josef Bacik wrote: > > On Wed, Oct 09, 2019 at 05:44:22PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When doing a buffered write it's possible to leave the subv_writers > > counter of the root, used for synchronization between buffered nocow

Re: [PATCH] btrfs: save i_size in compress_file_range

2019-10-11 Thread Filipe Manana
On Fri, Oct 11, 2019 at 2:05 PM Josef Bacik wrote: > > We hit a regression while rolling out 5.2 internally where we were > hitting the following panic > > kernel BUG at mm/page-writeback.c:2659! > RIP: 0010:clear_page_dirty_for_io+0xe6/0x1f0 > Call Trace: > __process_pages_contig+0x25a/0x350 >

Re: [PATCH] Btrfs: fix negative subv_writers counter and data space leak after buffered write

2019-10-11 Thread Josef Bacik
On Wed, Oct 09, 2019 at 05:44:22PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When doing a buffered write it's possible to leave the subv_writers > counter of the root, used for synchronization between buffered nocow > writers and snapshotting. This happens in an exceptional case

Re: [PATCH v3 3/3] btrfs: Introduce new incompat feature, BG_TREE, to speed up mount time

2019-10-11 Thread Josef Bacik
On Thu, Oct 10, 2019 at 10:39:28AM +0800, Qu Wenruo wrote: > The overall idea of the new BG_TREE is pretty simple: > Put BLOCK_GROUP_ITEMS into a separate tree. > > This brings one obvious enhancement: > - Reduce mount time of large fs > > Although it could be possible to accept BLOCK_GROUP_ITEMS

[PATCH] btrfs: save i_size in compress_file_range

2019-10-11 Thread Josef Bacik
We hit a regression while rolling out 5.2 internally where we were hitting the following panic kernel BUG at mm/page-writeback.c:2659! RIP: 0010:clear_page_dirty_for_io+0xe6/0x1f0 Call Trace: __process_pages_contig+0x25a/0x350 ? extent_clear_unlock_delalloc+0x43/0x70 submit_compressed_extents+0

Re: rsync -ax and subvolumes

2019-10-11 Thread Austin S. Hemmelgarn
On 2019-10-10 17:21, Ulli Horlacher wrote: On Thu 2019-10-10 (20:47), Kai Krakow wrote: I run into the problem that "rsync -ax" sees btrfs subvolumes as "other filesystems" and ignores them. I worked around it by mounting the btrfs-pool at a special directory: mount -o subvolid=0 /dev/disk/b

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Nikolay Borisov
On 11.10.19 г. 12:31 ч., Qu Wenruo wrote: > > > On 2019/10/10 下午11:06, Nikolay Borisov wrote: >> The code responsible for reading and initilizing tree roots is >> scattered in open_ctree among 2 labels, emulating a loop. This is rather >> confusing to reason about. Instead, factor the code in

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Qu Wenruo
On 2019/10/10 下午11:06, Nikolay Borisov wrote: > The code responsible for reading and initilizing tree roots is > scattered in open_ctree among 2 labels, emulating a loop. This is rather > confusing to reason about. Instead, factor the code in a new function, > init_tree_roots which implements th

Re: [PATCH 02/19] btrfs: rename DISCARD opt to DISCARD_SYNC

2019-10-11 Thread Nikolay Borisov
On 7.10.19 г. 23:17 ч., Dennis Zhou wrote: > This series introduces async discard which will use the flag > DISCARD_ASYNC, so rename the original flag to DISCARD_SYNC as it is > synchronously done in transaction commit. > > Signed-off-by: Dennis Zhou > --- > fs/btrfs/block-group.c | 2 +- > f

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Johannes Thumshirn
On 11/10/2019 10:21, Nikolay Borisov wrote: > It went to the top: > + bool should_retry = btrfs_test_opt(fs_info, USEBACKUPROOT); > Args, /me is blind Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Nikolay Borisov
On 11.10.19 г. 10:50 ч., Johannes Thumshirn wrote: > On 10/10/2019 17:06, Nikolay Borisov wrote: >> -recovery_tree_root: >> -if (!btrfs_test_opt(fs_info, USEBACKUPROOT)) >> -goto fail_tree_roots; > > What happened to this test in your refactoring? It went to the top: + bo

Re: [PATCH 1/3] btrfs: Factor out tree roots initialization during mount

2019-10-11 Thread Johannes Thumshirn
On 10/10/2019 17:06, Nikolay Borisov wrote: > -recovery_tree_root: > - if (!btrfs_test_opt(fs_info, USEBACKUPROOT)) > - goto fail_tree_roots; What happened to this test in your refactoring? -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de

Re: [RFC PATCH 00/19] btrfs: async discard support

2019-10-11 Thread Nikolay Borisov
On 7.10.19 г. 23:17 ч., Dennis Zhou wrote: > Hello, > > > With async discard, we try to emphasize discarding larger regions > and reusing the lba (implicit discard). The first is done by using the > free space cache to maintain discard state and thus allows us to get > coalescing for fairly

Re: [PATCH] Btrfs: fix metadata space leak on fixup worker failure to set range as delalloc

2019-10-11 Thread Nikolay Borisov
On 9.10.19 г. 19:43 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > In the fixup worker, if we fail to mark the range as delalloc in the io > tree, we must release the previously reserved metadata, as well as update > the outstanding extents counter for the inode, otherwise we leak me