Re: [PATCH 17/19] btrfs: add async discard header

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:48PM -0400, Dennis Zhou wrote: > Give a brief overview for how async discard is implemented. > > Signed-off-by: Dennis Zhou > --- Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 16/19] btrfs: keep track of discard reuse stats

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:47PM -0400, Dennis Zhou wrote: > Keep track of how much we are discarding and how often we are reusing > with async discard. > > Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 15/19] btrfs: load block_groups into discard_list on mount

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:46PM -0400, Dennis Zhou wrote: > Async discard doesn't remember the discard state of a block_group when > unmounting or when we crash. So, any block_group that is not fully used > may have undiscarded regions. However, free space caches are read in on > demand. Let the

Re: [PATCH 14/19] btrfs: only keep track of data extents for async discard

2019-10-10 Thread Josef Bacik
; explicitly discarding metadata blocks, just leave them be and let the > latter implicit discarding be done for them. > > Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 13/19] btrfs: have multiple discard lists

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:44PM -0400, Dennis Zhou wrote: > Non-block group destruction discarding currently only had a single list > with no minimum discard length. This can lead to caravaning more > meaningful discards behind a heavily fragmented block group. > > This adds support for multipl

Re: [PATCH 12/19] btrfs: limit max discard size for async discard

2019-10-10 Thread Josef Bacik
s well. I assume at some point in the far, far future SSD's will stop being shitty and it would be nice to be able to easily adjust and test. Also this only applies to async, so BTRFS_ASYNC_DISCARD_MAX_SIZE. You can add a follow up patch for the sysfs stuff, just adjust the name and you can add Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 11/19] btrfs: add bps discard rate limit

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:42PM -0400, Dennis Zhou wrote: > Provide an ability to rate limit based on mbps in addition to the iops > delay calculated from number of discardable extents. > > Signed-off-by: Dennis Zhou > --- > fs/btrfs/ctree.h | 2 ++ > fs/btrfs/discard.c | 11 +++ >

Re: [PATCH 10/19] btrfs: calculate discard delay based on number of extents

2019-10-10 Thread Josef Bacik
ecs_to_jiffies(cache->discard_delay - now); delay = max_t(u64, delay, atomic_read(&discard_ctl->delay); Looks a little cleaner. Otherwise Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 09/19] btrfs: keep track of discardable_bytes

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:40PM -0400, Dennis Zhou wrote: > Keep track of this metric so that we can understand how ahead or behind > we are in discarding rate. > > Signed-off-by: Dennis Zhou Same comment as the discard_extents patch. Thanks, Josef

Re: [PATCH 08/19] btrfs: track discardable extents for asnyc discard

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:39PM -0400, Dennis Zhou wrote: > The number of discardable extents will serve as the rate limiting metric > for how often we should discard. This keeps track of discardable extents > in the free space caches by maintaining deltas and propagating them to > the global co

Re: [PATCH 07/19] btrfs: discard one region at a time in async discard

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:38PM -0400, Dennis Zhou wrote: > The prior two patches added discarding via a background workqueue. This > just piggybacked off of the fstrim code to trim the whole block at once. > Well inevitably this is worse performance wise and will aggressively > overtrim. But it

Re: [PATCH 06/19] btrfs: handle empty block_group removal

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:37PM -0400, Dennis Zhou wrote: > block_group removal is a little tricky. It can race with the extent > allocator, the cleaner thread, and balancing. The current path is for a > block_group to be added to the unused_bgs list. Then, when the cleaner > thread comes around

Re: [PATCH 05/19] btrfs: add the beginning of async discard, discard workqueue

2019-10-10 Thread Josef Bacik
, our failure recovery mode will be to consider extents > untrimmed. This lets us handle failure and unmounting as one in the > same. > > Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 04/19] btrfs: keep track of cleanliness of the bitmap

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:35PM -0400, Dennis Zhou wrote: > There is a cap in btrfs in the amount of free extents that a block group > can have. When it surpasses that threshold, future extents are placed > into bitmaps. Instead of keeping track of if a certain bit is trimmed or > not in a secon

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-10 Thread Josef Bacik
On Mon, Oct 07, 2019 at 06:38:10PM -0400, Dennis Zhou wrote: > On Mon, Oct 07, 2019 at 04:37:28PM -0400, Josef Bacik wrote: > > On Mon, Oct 07, 2019 at 04:17:34PM -0400, Dennis Zhou wrote: > > > Async discard will use the free space cache as backing knowledge for > > &g

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-07 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:34PM -0400, Dennis Zhou wrote: > Async discard will use the free space cache as backing knowledge for > which extents to discard. This patch plumbs knowledge about which > extents need to be discarded into the free space cache from > unpin_extent_range(). > > An untri

Re: [PATCH 02/19] btrfs: rename DISCARD opt to DISCARD_SYNC

2019-10-07 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:33PM -0400, Dennis Zhou wrote: > This series introduces async discard which will use the flag > DISCARD_ASYNC, so rename the original flag to DISCARD_SYNC as it is > synchronously done in transaction commit. > > Signed-off-by: Dennis Zhou Reviewed-

Re: [PATCH 01/19] bitmap: genericize percpu bitmap region iterators

2019-10-07 Thread Josef Bacik
*re = find_next_zero_bit(bitmap, end, *rs + 1); > +} > + > +/* > + * Bitmap region iterators. Iterates over the bitmap between [@start, @end). Gonna be that guy here, should be '[@start, @end]' Reviewed-by: Josef Bacik Thanks, Josef

[PATCH][v3] btrfs/194: add a test for multi-subvolume fsyncing

2019-10-07 Thread Josef Bacik
y the file system is still mountable and the log is replayable. This test is to verify the fix btrfs: fix incorrect updating of log root tree actually fixed the problem. Reviewed-by: Filipe Manana Signed-off-by: Josef Bacik --- v2->v3: - cleaned up various bits that were leftover from the or

[PATCH][v2] btrfs/194: add a test for multi-subvolume fsyncing

2019-10-02 Thread Josef Bacik
y the file system is still mountable and the log is replayable. This test is to verify the fix btrfs: fix incorrect updating of log root tree actually fixed the problem. Signed-off-by: Josef Bacik --- v1->v2: - added the patchname related to this test in the comments and changelog. - running fio

[PATCH] btrfs: fix uninitialized ret in ref-verify

2019-10-02 Thread Josef Bacik
-by: Colin Ian King Signed-off-by: Josef Bacik --- fs/btrfs/ref-verify.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/ref-verify.c b/fs/btrfs/ref-verify.c index e87cbdad02a3..b57f3618e58e 100644 --- a/fs/btrfs/ref-verify.c +++ b/fs/btrfs/ref-verify.c @@ -500,7

Re: Btrfs: add a extent ref verify tool (static analysis bug report)

2019-10-02 Thread Josef Bacik
2fdbf15868c4 > Author: Josef Bacik > Date: Fri Sep 29 15:43:50 2017 -0400 > > Btrfs: add a extent ref verify tool > > The potential issue is when on the unlikely event when all the items > contain unknown key.types and so ret is not assigned a value. Since ret > is not

Re: [PATCH] btrfs/194: add a test for multi-subvolume fsyncing

2019-10-02 Thread Josef Bacik
On Wed, Oct 02, 2019 at 11:22:40AM +0100, Filipe Manana wrote: > On Wed, Oct 2, 2019 at 5:29 AM Josef Bacik wrote: > > > > From: Josef Bacik > > > > I discovered a problem in btrfs where we'd end up pointing at a block we > > hadn't written out

[PATCH] btrfs/194: add a test for multi-subvolume fsyncing

2019-10-01 Thread Josef Bacik
From: Josef Bacik I discovered a problem in btrfs where we'd end up pointing at a block we hadn't written out yet. This is triggered by a race when two different files on two different subvolumes fsync. This test exercises this path with dm-log-writes, and then replays the log at ev

Re: [PATCH][v3] btrfs: add a force_chunk_alloc to space_info's sysfs

2019-10-01 Thread Josef Bacik
On Tue, Oct 01, 2019 at 08:23:51PM +0200, David Sterba wrote: > On Mon, Aug 05, 2019 at 02:31:53PM -0400, Josef Bacik wrote: > > In testing various things such as the btrfsck patch to detect over > > allocation of chunks, empty block group deletion, and balance I've had >

[PATCH] btrfs: fix incorrect updating of log root tree

2019-09-30 Thread Josef Bacik
_tree. This way we do not modify the log_root_tree while we're committing it, fixing the problem. cc: sta...@vger.kernel.org Signed-off-by: Josef Bacik Reviewed-by: Chris Mason --- fs/btrfs/tree-log.c | 36 +++- 1 file changed, 27 insertions(+), 9 deleti

Re: [PATCH] Btrfs: fix memory leak due to concurrent append writes with fiemap

2019-09-30 Thread Josef Bacik
red, sporadically, by test case generic/561 from the > fstests suite, which makes duperemove run concurrently with fsstress, and > duperemove does plenty of calls to fiemap. When CONFIG_BTRFS_DEBUG is set > the leak is reported in dmesg/syslog when removing the btrfs module with > a message like the following: > > [77100.039461] BTRFS: state leak: start 6574080 end 6582271 state 16402 in > tree 0 refs 1 > > Otherwise (CONFIG_BTRFS_DEBUG not set) detectable with kmemleak. > > CC: sta...@vger.kernel.org # 4.16+ > Signed-off-by: Filipe Manana Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH v2 1/3] btrfs: Speed up btrfs_file_llseek

2019-09-27 Thread Josef Bacik
On Fri, Sep 27, 2019 at 01:23:16PM +0300, Nikolay Borisov wrote: > Modifying the file position is done on a per-file basis. This renders > holding the inode lock for writing useless and makes the performance of > concurrent llseek's abysmal. > > Fix this by holding the inode for read. This provide

Re: [PATCH RFC] btrfs: relocation: Hunt down BUG_ON() in merge_reloc_roots()

2019-09-26 Thread Josef Bacik
On Thu, Sep 26, 2019 at 02:35:45PM +0800, Qu Wenruo wrote: > [BUG] > There is one BUG_ON() report where a transaction is aborted during > balance, then kernel BUG_ON() in merge_reloc_roots(): > > void merge_reloc_roots(struct reloc_control *rc) > { > ... > BUG_ON(!RB_EMPTY_ROOT(&rc

Re: [PATCH 3/3] btrfs: Return offset from find_desired_extent

2019-09-26 Thread Josef Bacik
t' variable in > btrfs_llseek_file can be removed. Additional (subjective) benefit is > that btrfs' llseek function now resemebles those of the other major > filesystems. > > Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 1/3] btrfs: Speed up btrfs_file_llseek

2019-09-26 Thread Josef Bacik
> real 0m41.412s > user 0m28.777s > sys 2m16.510s > > With patch: > > num workprocesses = 16 > num fseek/fread = 800 > step = 256 > fork 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 > > real 0m11.479s > user 0m27.629s > sys 0m18.187s > > Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 2/3] btrfs: Simplify btrfs_file_llseek

2019-09-26 Thread Josef Bacik
changesl. > > Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik Thanks, Josef

[PATCH][v3] btrfs: use refcount_inc_not_zero in kill_all_nodes

2019-09-26 Thread Josef Bacik
that have a refcount of 0. Fix this by zero'ing out that element in our temporary array so we don't try to kill it again. Signed-off-by: Josef Bacik --- v2->v3: - always git show after a revision kids, to make sure your new code still makes sense with the original code, not

[PATCH][v2] btrfs: use refcount_inc_not_zero in kill_all_nodes

2019-09-26 Thread Josef Bacik
that have a refcount of 0. Fix this by zero'ing out that element in our temporary array so we don't try to kill it again. Signed-off-by: Josef Bacik --- v1->v2: - I'm an idiot. fs/btrfs/delayed-inode.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) dif

Re: [PATCH] btrfs: use refcount_inc_not_zero in kill_all_nodes

2019-09-26 Thread Josef Bacik
On Thu, Sep 26, 2019 at 02:13:44PM +0300, Nikolay Borisov wrote: > > > On 26.09.19 г. 13:54 ч., Josef Bacik wrote: > > We hit the following warning while running down a different problem > > > > [ 6197.175850] [ cut here ] > > [ 6197.1

Re: [PATCH 2/3] btrfs: move ref finding machinery out of build_backref_tree()

2019-09-26 Thread Josef Bacik
On Tue, Sep 24, 2019 at 04:49:20PM +0200, David Sterba wrote: > On Wed, Sep 11, 2019 at 12:09:29PM -0400, Josef Bacik wrote: > > On Fri, Sep 06, 2019 at 10:15:32AM -0700, Mark Fasheh wrote: > > > From: Mark Fasheh > > > > > > build_backref_tree() is walking

[PATCH] btrfs: use refcount_inc_not_zero in kill_all_nodes

2019-09-26 Thread Josef Bacik
d use refcount_inc_not_zero() and only process the list for the nodes we get a refcount on. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-inode.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c index 1f7f39b

Re: [PATCH 0/4] The remaining extent_io.c split code

2019-09-25 Thread Josef Bacik
On Wed, Sep 25, 2019 at 03:47:47PM +0200, David Sterba wrote: > On Tue, Sep 24, 2019 at 04:32:48PM -0400, Josef Bacik wrote: > > Hopefully all of it makes it this time, if you want you can pull from > > > > git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.gi

Re: [PATCH 0/4] Minor cleanups in locking helpers

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 07:33:16PM +0200, David Sterba wrote: > Reorganizing code wrt locking helpers, minor text and stack savings. > Reviewed-by: Josef Bacik Thanks, Josef

[PATCH 2/2] btrfs: use btrfs_block_group_cache_done in update_block_group

2019-09-24 Thread Josef Bacik
load the space cache unconditionally here, which will result in the caller waiting for the fast caching to complete and keep us from truncating the free space inode. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/block-grou

[PATCH 1/2] btrfs: check page->mapping when loading free space cache

2019-09-24 Thread Josef Bacik
t in a followup patch. However we still shouldn't allow those sort of mistakes to result in messing with pages that do not belong to us. So add the page->mapping check to verify that we still own this page after dropping and re-acquiring the page lock. Signed-off-by: Josef Bacik --- fs/b

[PATCH 2/4] btrfs: separate out the extent buffer init code

2019-09-24 Thread Josef Bacik
We want to move this into it's own file, so separate out the init/exit code for setting up the extent_buffer cache. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 36 +++- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 - 3 files ch

[PATCH 4/4] btrfs: move the extent-buffer code

2019-09-24 Thread Josef Bacik
This moves the extent buffer code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/Makefile|3 +- fs/btrfs/extent-buffer.c | 1268 ++ fs/btrfs/extent_io.c | 1256 - 3 files changed, 1270 insertions

[PATCH 3/4] btrfs: migrate the extent_buffer code out of extent-io.h

2019-09-24 Thread Josef Bacik
There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +

[PATCH 0/4] The remaining extent_io.c split code

2019-09-24 Thread Josef Bacik
Hopefully all of it makes it this time, if you want you can pull from git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git \ extent-io-rearranging I've rebased on top of misc-next and fixed up all the conflicts. Thanks, Josef

Re: [PATCH] Re: Possible FS race condition between iterate_dir and d_alloc_parallel

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 04:11:07PM +0100, Al Viro wrote: > On Tue, Sep 24, 2019 at 11:01:45AM -0400, Josef Bacik wrote: > > > Sorry I mis-read the code a little bit. This is purely for the subvolume > > link > > directories. We haven't wandered down int

Re: [PATCH] Re: Possible FS race condition between iterate_dir and d_alloc_parallel

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 03:51:04PM +0100, Al Viro wrote: > On Tue, Sep 24, 2019 at 09:30:26AM -0400, Josef Bacik wrote: > > > > We pass next->d_name.name to dir_emit() (i.e. potentially to > > > copy_to_user()). And we have no warranty that it's not a long >

Re: WITH regression patch, still btrfs-transacti blocked for... (forever)

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 03:23:06PM +0100, Filipe Manana wrote: > On Tue, Sep 24, 2019 at 3:19 PM Josef Bacik wrote: > > > > On Tue, Sep 24, 2019 at 03:16:56PM +0100, Filipe Manana wrote: > > > On Tue, Sep 24, 2019 at 2:21 PM Josef Bacik wrote: > > > > > &

Re: WITH regression patch, still btrfs-transacti blocked for... (forever)

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 03:16:56PM +0100, Filipe Manana wrote: > On Tue, Sep 24, 2019 at 2:21 PM Josef Bacik wrote: > > > > On Tue, Sep 24, 2019 at 07:07:41AM -0400, James Harvey wrote: > > > On Tue, Sep 24, 2019 at 5:58 AM Filipe Manana wrote: > > > > >

Re: [PATCH] Btrfs: fix race setting up and completing qgroup rescan workers

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 03:12:56PM +0100, Filipe Manana wrote: > On Tue, Sep 24, 2019 at 2:53 PM Josef Bacik wrote: > > > > On Tue, Sep 24, 2019 at 10:49:54AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > There is a race bet

Re: [PATCH] Btrfs: fix race setting up and completing qgroup rescan workers

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 10:49:54AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > There is a race between setting up a qgroup rescan worker and completing > a qgroup rescan worker that can lead to callers of the qgroup rescan wait > ioctl to either not wait for the rescan worker to c

Re: [PATCH] Re: Possible FS race condition between iterate_dir and d_alloc_parallel

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 03:52:15AM +0100, Al Viro wrote: > [btrfs and cifs folks Cc'd] > > On Sat, Sep 21, 2019 at 03:07:31PM +0100, Al Viro wrote: > > > No "take cursors out of the list" parts yet. > > Argh... The things turned interesting. The tricky part is > where do we handle switching cu

Re: Balance ENOSPC during balance despite additional storage added

2019-09-24 Thread Josef Bacik
On Fri, Sep 20, 2019 at 03:29:32PM +0100, Pete wrote: > I have a btrfs that is on top of an lvm logical volume on top of dm-crypt on > a single nvme drive (Samsung 870 Pro 512GB). > > I added a second logical volume to give more space to get rid of ENOSPC > errors during balance, but to no avail.

Re: WITH regression patch, still btrfs-transacti blocked for... (forever)

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 07:07:41AM -0400, James Harvey wrote: > On Tue, Sep 24, 2019 at 5:58 AM Filipe Manana wrote: > > > > On Sun, Sep 15, 2019 at 2:55 PM Filipe Manana wrote: > > > > > > On Sun, Sep 15, 2019 at 1:46 PM James Harvey > > > wrote: > > > > ... > > > > You'll see they're differen

Re: [PATCH 0/9][V3] btrfs: break up extent_io.c a little bit

2019-09-24 Thread Josef Bacik
On Tue, Sep 24, 2019 at 09:46:15AM +0200, David Sterba wrote: > On Mon, Sep 23, 2019 at 02:21:04PM -0400, Josef Bacik wrote: > > > I got some strange merge conflicts, it turns out patch 6/9 did not make > > > it to the mailinglist (nor patchwork where I could pick it eventu

Re: [PATCH 0/9][V3] btrfs: break up extent_io.c a little bit

2019-09-23 Thread Josef Bacik
On Mon, Sep 23, 2019 at 07:01:01PM +0200, David Sterba wrote: > On Mon, Sep 23, 2019 at 10:05:16AM -0400, Josef Bacik wrote: > > v1->v2: > > - renamed find_delalloc_range to btrfs_find_delalloc_range for now. > > > > -- Original email -- > > > > Curren

Re: [PATCH 0/9][V3] btrfs: break up extent_io.c a little bit

2019-09-23 Thread Josef Bacik
On Mon, Sep 23, 2019 at 04:32:19PM +0200, David Sterba wrote: > On Mon, Sep 23, 2019 at 10:05:16AM -0400, Josef Bacik wrote: > > v1->v2: > > - renamed find_delalloc_range to btrfs_find_delalloc_range for now. > > The series is V3, what changed V2->V3? Wtf, I must not

[PATCH 7/9] btrfs: separate out the extent buffer init code

2019-09-23 Thread Josef Bacik
We want to move this into it's own file, so separate out the init/exit code for setting up the extent_buffer cache. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 36 +++- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 - 3 files ch

[PATCH 5/9] btrfs: move the failrec tree stuff into extent-io-tree.h

2019-09-23 Thread Josef Bacik
point. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 18 ++ fs/btrfs/extent_io.c | 8 fs/btrfs/extent_io.h | 11 --- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h

[PATCH 9/9] btrfs: move the extent-buffer code

2019-09-23 Thread Josef Bacik
This moves the extent buffer code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/Makefile|3 +- fs/btrfs/extent-buffer.c | 1268 ++ fs/btrfs/extent_io.c | 1257 - 3 files changed, 1270 insertions

[PATCH 1/9] btrfs: separate out the extent leak code

2019-09-23 Thread Josef Bacik
We check both extent buffer and extent state leaks in the same function, separate these two functions out so we can move them around. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 31 +++ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/fs

[PATCH 8/9] btrfs: migrate the extent_buffer code out of extent-io.h

2019-09-23 Thread Josef Bacik
There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +

[PATCH 3/9] btrfs: move extent_io_tree defs to their own header

2019-09-23 Thread Josef Bacik
extent_io.c/h are huge, encompassing a bunch of different things. The extent_io_tree code can live on its own, so separate this out. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-io-tree.h | 227 ++ fs/btrfs/extent_io.c

[PATCH 4/9] btrfs: export find_delalloc_range

2019-09-23 Thread Josef Bacik
This utilizes internal stuff to the extent_io_tree, so we need to export it before we move it. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 3 +++ fs/btrfs/extent_io.c | 10 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b

[PATCH 0/9][V3] btrfs: break up extent_io.c a little bit

2019-09-23 Thread Josef Bacik
v1->v2: - renamed find_delalloc_range to btrfs_find_delalloc_range for now. -- Original email -- Currently extent_io.c includes all of the extent-io-tree code, the extent buffer code, the code to do IO on extent buffers and data extents, as well as a bunch of other random stuff. The random stuff

[PATCH 2/9] btrfs: separate out the extent io init function

2019-09-23 Thread Josef Bacik
We are moving extent_io_tree into it's on file, so separate out the extent_state init stuff from extent_io_tree_init(). Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 18 +++--- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 - 3 files changed, 21 inser

Re: [PATCH v2] btrfs: relocation: Fix KASAN report about use-after-free due to dead reloc tree cleanup race

2019-09-23 Thread Josef Bacik
llowing modifications: > - Test if the root has dead reloc tree before accessing root->reloc_root > If the root has BTRFS_ROOT_DEAD_RELOC_TREE, then we don't need to > create or update root->reloc_tree > > - Clear the BTRFS_ROOT_DEAD_RELOC_TREE flag until we have fully dropped > reloc tree > To co-operate with above modification, so as long as > BTRFS_ROOT_DEAD_RELOC_TREE is still set, we won't try to re-create > reloc tree at record_root_in_trans(). > > Reported-by: Cebtenzzre > Fixes: d2311e698578 ("btrfs: relocation: Delay reloc tree deletion after > merge_reloc_roots") > Signed-off-by: Qu Wenruo > --- Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] btrfs: relocation: Fix KASAN report about use-after-free due to dead reloc tree cleanup race

2019-09-20 Thread Josef Bacik
On Fri, Sep 20, 2019 at 10:41:50AM +0800, Qu Wenruo wrote: > [BUG] > One user reported a reproduciable KASAN report about use-after-free: > BTRFS info (device sdi1): balance: start > -dvrange=1256811659264..1256811659265 > BTRFS info (device sdi1): relocating block group 1256811659264 flags >

[PATCH 4/9] btrfs: export find_delalloc_range

2019-09-17 Thread Josef Bacik
This utilizes internal stuff to the extent_io_tree, so we need to export it before we move it. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 3 +++ fs/btrfs/extent_io.c | 10 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b

[PATCH 3/9] btrfs: move extent_io_tree defs to their own header

2019-09-17 Thread Josef Bacik
extent_io.c/h are huge, encompassing a bunch of different things. The extent_io_tree code can live on its own, so separate this out. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-io-tree.h | 227 ++ fs/btrfs/extent_io.c

[PATCH 2/9] btrfs: separate out the extent io init function

2019-09-17 Thread Josef Bacik
We are moving extent_io_tree into it's on file, so separate out the extent_state init stuff from extent_io_tree_init(). Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 18 +++--- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 - 3 files changed, 21 inser

[PATCH 1/9] btrfs: separate out the extent leak code

2019-09-17 Thread Josef Bacik
We check both extent buffer and extent state leaks in the same function, separate these two functions out so we can move them around. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 28 +--- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/fs/btrfs

[PATCH 0/9][V2] btrfs: break up extent_io.c a little bit

2019-09-17 Thread Josef Bacik
v1->v2: - renamed find_delalloc_range to btrfs_find_delalloc_range for now. -- Original email -- Currently extent_io.c includes all of the extent-io-tree code, the extent buffer code, the code to do IO on extent buffers and data extents, as well as a bunch of other random stuff. The random stuf

[PATCH 5/9] btrfs: move the failrec tree stuff into extent-io-tree.h

2019-09-17 Thread Josef Bacik
point. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 18 ++ fs/btrfs/extent_io.c | 8 fs/btrfs/extent_io.h | 11 --- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h

[PATCH 9/9] btrfs: move the extent-buffer code

2019-09-17 Thread Josef Bacik
This moves the extent buffer code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/Makefile|3 +- fs/btrfs/extent-buffer.c | 1266 ++ fs/btrfs/extent_io.c | 1255 - 3 files changed, 1268 insertions

[PATCH 7/9] btrfs: separate out the extent buffer init code

2019-09-17 Thread Josef Bacik
We want to move this into it's own file, so separate out the init/exit code for setting up the extent_buffer cache. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 36 +++- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 - 3 files ch

[PATCH 8/9] btrfs: migrate the extent_buffer code out of extent-io.h

2019-09-17 Thread Josef Bacik
There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +

Re: [PATCH] btrfs: fix balance convert to single on 32-bit host CPUs

2019-09-13 Thread Josef Bacik
On Thu, Sep 12, 2019 at 07:55:01PM -0400, Zygo Blaxell wrote: > Currently, the command: > > btrfs balance start -dconvert=single,soft . > > on a Raspberry Pi produces the following kernel message: > > BTRFS error (device mmcblk0p2): balance: invalid convert data profile > single >

Re: [PATCH] fstests: btrfs: Verify falloc on multiple holes won't cause qgroup reserved data space leak

2019-09-13 Thread Josef Bacik
led: > "btrfs: qgroup: Fix the wrong target io_tree when freeing > reserved data space" > > Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] btrfs: Add assert to catch nested transaction commit

2019-09-13 Thread Josef Bacik
ch such cases add an assert in btrfs_commit_transaction ensuring > btrfs_trans_handle::use_count is always 1. > > Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] Btrfs: fix assertion failure during fsync and use of stale transaction

2019-09-12 Thread Josef Bacik
On Thu, Sep 12, 2019 at 02:19:55PM +0100, Filipe Manana wrote: > On Thu, Sep 12, 2019 at 1:18 PM Josef Bacik wrote: > > > > On Tue, Sep 10, 2019 at 03:26:49PM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > Sometimes when fsync&#

Re: [PATCH] Btrfs: fix assertion failure during fsync and use of stale transaction

2019-09-12 Thread Josef Bacik
On Tue, Sep 10, 2019 at 03:26:49PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Sometimes when fsync'ing a file we need to log that other inodes exist and > when we need to do that we acquire a reference on the inodes and then drop > that reference using iput() after logging them.

Re: [PATCH] Btrfs: fix assertion failure during fsync and use of stale transaction

2019-09-12 Thread Josef Bacik
On Tue, Sep 10, 2019 at 03:26:49PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Sometimes when fsync'ing a file we need to log that other inodes exist and > when we need to do that we acquire a reference on the inodes and then drop > that reference using iput() after logging them.

Re: [PATCH RFC v2 0/2] readmirror feature

2019-09-12 Thread Josef Bacik
On Thu, Sep 12, 2019 at 06:10:08PM +0800, Anand Jain wrote: > > > > On 12 Sep 2019, at 6:03 PM, Josef Bacik wrote: > > > > On Thu, Sep 12, 2019 at 06:00:21PM +0800, Anand Jain wrote: > >> > >> > >>> On 12 Sep 2019, at 5:50 PM, Josef Ba

Re: [PATCH RFC v2 0/2] readmirror feature

2019-09-12 Thread Josef Bacik
On Thu, Sep 12, 2019 at 06:00:21PM +0800, Anand Jain wrote: > > > > On 12 Sep 2019, at 5:50 PM, Josef Bacik wrote: > > > > On Thu, Sep 12, 2019 at 03:41:42PM +0800, Anand Jain wrote: > >> > >> > >> Thanks for the comments. More belo

Re: [PATCH RFC] btrfs: Introduce btrfs child tree block verification system

2019-09-12 Thread Josef Bacik
On Thu, Sep 12, 2019 at 07:38:14AM +0800, Qu Wenruo wrote: > > > On 2019/9/12 上午12:02, Josef Bacik wrote: > > On Wed, Sep 11, 2019 at 03:46:24PM +0800, Qu Wenruo wrote: > >> Although we have btrfs_verify_level_key() function to check the first > >> key and leve

Re: [PATCH RFC v2 0/2] readmirror feature

2019-09-12 Thread Josef Bacik
On Thu, Sep 12, 2019 at 03:41:42PM +0800, Anand Jain wrote: > > > Thanks for the comments. More below. > > On 12/9/19 3:16 AM, Josef Bacik wrote: > > On Wed, Sep 11, 2019 at 03:13:21PM -0400, Eli V wrote: > > > On Wed, Sep 11, 2019 at 2:46 PM Josef Bacik wrote: &

Re: [PATCH RFC v2 0/2] readmirror feature

2019-09-11 Thread Josef Bacik
On Wed, Sep 11, 2019 at 03:13:21PM -0400, Eli V wrote: > On Wed, Sep 11, 2019 at 2:46 PM Josef Bacik wrote: > > > > On Mon, Aug 26, 2019 at 05:04:36PM +0800, Anand Jain wrote: > > > Function call chain __btrfs_map_block()->find_live_mirror() uses > > > thr

Re: [PATCH] Btrfs: make btrfs_wait_extents() static

2019-09-11 Thread Josef Bacik
On Wed, Sep 11, 2019 at 05:42:38PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > It's not used ouside of transaction.c > > Signed-off-by: Filipe Manana Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] Btrfs: fix missing error return if writeback for extent buffer never started

2019-09-11 Thread Josef Bacik
esult in the endio callback end_bio_extent_buffer_writepage() > being invoked, which sets the BTRFS_FS_*_ERR bits appropriately, so that > there's no risk a transaction or log commit doesn't catch a writeback > failure. > > Signed-off-by: Filipe Manana Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH RFC v2 0/2] readmirror feature

2019-09-11 Thread Josef Bacik
On Mon, Aug 26, 2019 at 05:04:36PM +0800, Anand Jain wrote: > Function call chain __btrfs_map_block()->find_live_mirror() uses > thread pid to determine the %mirror_num when the mirror_num=0. > > This patch introduces a framework so that we can add policies to determine > the %mirror_num. And als

Re: [PATCH 3/3] btrfs: move useless node processing out of build_backref_cache

2019-09-11 Thread Josef Bacik
On Fri, Sep 06, 2019 at 10:15:33AM -0700, Mark Fasheh wrote: > From: Mark Fasheh > > The backref cache has to clean up nodes referring to tree leaves. It is > trivial to pull that code into it's own function, which is what I do here. > > Signed-off-by: Mark Fasheh &g

Re: [PATCH 2/3] btrfs: move ref finding machinery out of build_backref_tree()

2019-09-11 Thread Josef Bacik
On Fri, Sep 06, 2019 at 10:15:32AM -0700, Mark Fasheh wrote: > From: Mark Fasheh > > build_backref_tree() is walking extent refs in what is an otherwise self > contained chunk of code. We can shrink the total number of lines in > build_backref_tree() *and* make it more readable by moving that wa

Re: [PATCH 1/3] btrfs: Move backref cache code out of relocation.c

2019-09-11 Thread Josef Bacik
e.h. > > Signed-off-by: Mark Fasheh Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] Btrfs: fix assertion failure during fsync and use of stale transaction

2019-09-11 Thread Josef Bacik
s_add_delayed_iput(), so that the logging path of fsync never drops > the last reference on an inode, that step is offloaded to a safe context > (usually the cleaner kthread). > > The assertion failure issue was sporadically triggered by the test case > generic/475 from fstests, which loads the dm error target while fsstress > is running, which lead to fsync failing while logging inodes with -EIO > errors and then trying later to commit the transaction, triggering the > assertion failure. > > CC: sta...@vger.kernel.org # 4.4+ > Signed-off-by: Filipe Manana Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH RFC] btrfs: Introduce btrfs child tree block verification system

2019-09-11 Thread Josef Bacik
On Wed, Sep 11, 2019 at 03:46:24PM +0800, Qu Wenruo wrote: > Although we have btrfs_verify_level_key() function to check the first > key and level at tree block read time, it has its limitation due to tree > lock context, it's not reliable handling new tree blocks. > How is it not reliable with n

[PATCH 5/9] btrfs: move the failrec tree stuff into extent-io-tree.h

2019-09-11 Thread Josef Bacik
point. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 18 ++ fs/btrfs/extent_io.c | 8 fs/btrfs/extent_io.h | 11 --- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h

[PATCH 1/9] btrfs: separate out the extent leak code

2019-09-11 Thread Josef Bacik
We check both extent buffer and extent state leaks in the same function, separate these two functions out so we can move them around. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 28 +--- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/fs/btrfs

[PATCH 8/9] btrfs: migrate the extent_buffer code out of extent-io.h

2019-09-11 Thread Josef Bacik
There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +

[PATCH 0/9] btrfs: break up extent_io.c a little bit

2019-09-11 Thread Josef Bacik
Currently extent_io.c includes all of the extent-io-tree code, the extent buffer code, the code to do IO on extent buffers and data extents, as well as a bunch of other random stuff. The random stuff just needs to be cleaned up, and is probably too invasive for this point in the development cycle.

<    1   2   3   4   5   6   7   8   9   10   >