Re: trouble with btrfs lockup

2019-01-18 Thread Liu Bo
On Fri, Jan 18, 2019 at 10:58 AM Krishna Mannem wrote: > > Hi, > > I work on Concourse-CI (https://concourse-ci.org/). It's a container > based CI system where we create volumes using Btrfs. Due to the nature > of Concourse, btrfs subvolumes are short lived ( sometimes a few > seconds if it's a sm

Re: [PATCH v2] btrfs: raid56: data corruption on a device removal

2018-12-25 Thread Liu Bo
ntained non zero bio->bi_iter.bi_done field despite > of EIO bi_status. bi_sector field was also increased from original one > by that 'bi_done' value. Looks like this is a quite rare condition. > Subsequently, in the raid_rmw_end_io handler that failed bio can be > translated to

Re: [PATCH] Btrfs: remove no longer used stuff for tracking pending ordered extents

2018-10-26 Thread Liu Bo
the stuff to track for pending > ordered extents, which was not completely removed in the mentioned commit. > So remove the remaining of the pending ordered extents infrastructure. > Reviewed-by: Liu Bo thanks, liubo > Signed-off-by: Filipe Manana > --- > fs/btrfs/ordered-da

Re: [GIT PULL] Btrfs updates for 4.20, part 1

2018-10-22 Thread Liu Bo
ces that don't support it > btrfs: keep trim from interfering with transaction commits > > Josef Bacik (7): > btrfs: wait on caching when putting the bg cache > btrfs: release metadata before running delayed refs > btrfs: protect space cache inode all

Re: Btrfs resize seems to deadlock

2018-10-22 Thread Liu Bo
On Sat, Oct 20, 2018 at 1:34 PM Filipe Manana wrote: > > On Sat, Oct 20, 2018 at 9:27 PM Liu Bo wrote: > > > > On Fri, Oct 19, 2018 at 7:09 PM Andrew Nelson > > wrote: > > > > > > I am having an issue with btrfs resize in Fedora 28. I am attempting

Re: Btrfs resize seems to deadlock

2018-10-20 Thread Liu Bo
On Fri, Oct 19, 2018 at 7:09 PM Andrew Nelson wrote: > > I am having an issue with btrfs resize in Fedora 28. I am attempting > to enlarge my Btrfs partition. Every time I run "btrfs filesystem > resize max $MOUNT", the command runs for a few minutes and then hangs > forcing the system to be reset

Re: [PATCH v2] Btrfs: fix null pointer dereference on compressed write path error

2018-10-12 Thread Liu Bo
uccess > creating an inline extent) or an error (like -ENOMEM for example) we jump > to the "free_pages_out" label and then access "pages[i]" leading to a NULL > pointer dereference, since "nr_pages" has a value greater than zero at > that point. > > Fix

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-20 Thread Liu Bo
On Thu, Sep 20, 2018 at 06:49:59PM +0200, David Sterba wrote: > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > Several structs in btrfs are using rb_first() in a while loop, it'd be > > more efficient to do this with rb_first_cached() which has the O(1) > >

Re: btrfs panic problem

2018-09-19 Thread Liu Bo
On Mon, Sep 17, 2018 at 5:28 PM, sunny.s.zhang wrote: > Hi All, > > My OS(4.1.12) panic in kmem_cache_alloc, which is called by > btrfs_get_or_create_delayed_node. > > I found that the freelist of the slub is wrong. > > crash> struct kmem_cache_cpu 887e7d7a24b0 > > struct kmem_cache_cpu { >

[PATCH v2] Btrfs: remove wait_ordered_rane in btrfs_evict_inode

2018-09-14 Thread Liu Bo
his step), wait_ordered_range() is just a noop. Reviewed-by: David Sterba Signed-off-by: Liu Bo --- v2: More details in the description. fs/btrfs/inode.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ff1d2ed2dc94..d3febc3a6bc0 100644 --- a/fs/bt

Re: [PATCH] Btrfs: remove wait_ordered_range in btrfs_evict_inode

2018-09-14 Thread Liu Bo
On Fri, Sep 14, 2018 at 03:14:33PM +0200, David Sterba wrote: > On Wed, Sep 12, 2018 at 06:06:22AM +0800, Liu Bo wrote: > > As VFS has called ->invalidatepage() to get all ordered extents done > > and truncated all page cache pages, wait_ordered_range() is just a > > n

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-14 Thread Liu Bo
On Fri, Sep 14, 2018 at 05:11:02PM +0200, David Sterba wrote: > On Tue, Sep 11, 2018 at 11:31:49AM -0700, Liu Bo wrote: > > On Tue, Sep 11, 2018 at 05:34:03PM +0200, David Sterba wrote: > > > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > > > Seve

[PATCH v2] Btrfs: remove level==0 check in balance_level

2018-09-13 Thread Liu Bo
btrfs_search_slot() if (level != 0) setup_nodes_for_search() balance_level() It is just impossible to have level=0 in balance_level. Reviewed-by: Nikolay Borisov Signed-off-by: Liu Bo --- v2: add assertion for level just in case someone breaks the rule in the future. fs

[PATCH v2] Btrfs: assert page dirty bit

2018-09-13 Thread Liu Bo
Just in case that someone breaks the rule that pages are dirty as long as eb is dirty. Signed-off-by: Liu Bo --- v2: fix typo of CONFIG_BTRFS_DEBUG fs/btrfs/extent_io.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index fb2bf50134a1

[PATCH v2] Btrfs: skip set_page_dirty if eb is dirty

2018-09-13 Thread Liu Bo
As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages are dirty, so no need to set pages dirty again. Ftrace showed that the loop took 10us on my dev box, so removing this can save us at least 10us if eb is already dirty. Signed-off-by: Liu Bo --- v2: Update the patch&#

Re: [PATCH] Btrfs: skip set_page_dirty if eb is dirty

2018-09-13 Thread Liu Bo
On Thu, Sep 13, 2018 at 01:29:59PM +0200, David Sterba wrote: > On Wed, Sep 12, 2018 at 12:27:46PM -0700, Liu Bo wrote: > > On Wed, Sep 12, 2018 at 09:37:20AM +0300, Nikolay Borisov wrote: > > > > > > > > > On 12.09.2018 01:06, Liu Bo wrote: &

Re: [PATCH] Btrfs: remove level==0 check in balance_level

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 02:52:38PM +0200, David Sterba wrote: > On Wed, Sep 12, 2018 at 06:06:23AM +0800, Liu Bo wrote: > > btrfs_search_slot() > >if (level != 0) > > setup_nodes_for_search() > > balance_level() > > > > It is just impo

Re: [PATCH] Btrfs: assert page dirty bit

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 09:38:49AM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 01:06, Liu Bo wrote: > > Just in case that someone breaks the rule that pages are dirty as long > > as eb is dirty. > > > > Signed-off-by: Liu Bo > > --- > > fs/bt

Re: [PATCH] Btrfs: skip set_page_dirty if eb is dirty

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 09:37:20AM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 01:06, Liu Bo wrote: > > As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages > > are dirty, so no need to set pages dirty again. > > > > Signed-off-by: Liu Bo

[PATCH] Btrfs: assert page dirty bit

2018-09-11 Thread Liu Bo
Just in case that someone breaks the rule that pages are dirty as long as eb is dirty. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index fb2bf50134a1..99895f196ecb 100644 --- a/fs/btrfs

[PATCH] Btrfs: remove level==0 check in balance_level

2018-09-11 Thread Liu Bo
btrfs_search_slot() if (level != 0) setup_nodes_for_search() balance_level() It is just impossible to have level=0 in balance_level. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index

[PATCH] Btrfs: unify error handling of btrfs_lookup_dir_item

2018-09-11 Thread Liu Bo
Unify the error handling part with IS_ERR_OR_NULL. No functional changes. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 21 + fs/btrfs/send.c | 8 ++-- 2 files changed, 7 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index

[PATCH] Btrfs: remove wait_ordered_range in btrfs_evict_inode

2018-09-11 Thread Liu Bo
As VFS has called ->invalidatepage() to get all ordered extents done and truncated all page cache pages, wait_ordered_range() is just a noop. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ff1d2ed2d

[PATCH] Btrfs: remove unused code in __btrfs_unlink_inode

2018-09-11 Thread Liu Bo
It might get @leaf and @key in order to do some sanity checks on key's fields, but since we don't check it any more, it's fine to remove the code. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrf

[PATCH] Btrfs: do not wait after queue async work for delaye refs

2018-09-11 Thread Liu Bo
help. In fact we don't have to wait for asynchronous delayed refs processing as delayed refs are not blocking the subsequent operations(unless within committing transaction we need to make sure filesystem is consistent). Signed-off-by: Liu Bo --- fs/btrfs/ctree.h | 2 +- fs/bt

[PATCH] Btrfs: skip set_page_dirty if eb is dirty

2018-09-11 Thread Liu Bo
As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages are dirty, so no need to set pages dirty again. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 11 +++ fs/btrfs/extent_io.h | 2 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/extent_io.c

[PATCH] Btrfs: skip setting path to blocking mode if balance is not needed

2018-09-11 Thread Liu Bo
igned-off-by: Liu Bo --- fs/btrfs/ctree.c | 41 ++--- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 858085490e23..ba267a069ca1 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -1758,6 +17

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-11 Thread Liu Bo
On Tue, Sep 11, 2018 at 05:34:03PM +0200, David Sterba wrote: > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > Several structs in btrfs are using rb_first() in a while loop, it'd be > > more efficient to do this with rb_first_cached() which has the O(1) > >

[PATCH v2] Btrfs: use correct args for kcalloc in btrfsic_read_block

2018-09-07 Thread Liu Bo
kcalloc is defined as kcalloc(size_t n, size_t size, gfp_t flags) Although this won't cause problems in practice, btrfsic_read_block() has switched n with size. This updates btrfsic_read_block() to be correct in using kcalloc. Reviewed-by: Omar Sandoval Signed-off-by: Liu Bo --- v2

Re: [PATCH] Btrfs: use correct args for kcalloc in btrfsic_read_block

2018-09-07 Thread Liu Bo
On Fri, Sep 7, 2018 at 11:21 AM, Omar Sandoval wrote: > On Sat, Sep 08, 2018 at 01:59:50AM +0800, Liu Bo wrote: >> kcalloc is defined as >> kcalloc(size_t n, size_t size, gfp_t flags) >> >> Although this won't cause problems in practice, btrfsic_read_blo

[PATCH] Btrfs: use correct args for kcalloc in btrfsic_read_block

2018-09-07 Thread Liu Bo
kcalloc is defined as kcalloc(size_t n, size_t size, gfp_t flags) Although this won't cause problems in practice, btrfsic_read_block() has switched n with size. This updates btrfsic_read_block() to be correct in using kcalloc. Signed-off-by: Liu Bo --- fs/btrfs/check-integrity.c | 6 +++-

Re: [PATCH] Btrfs: remove redundant btrfs_trans_release_metadata"

2018-09-06 Thread Liu Bo
On Thu, Sep 6, 2018 at 11:50 AM, Nikolay Borisov wrote: > > > On 6.09.2018 09:47, Liu Bo wrote: >> On Wed, Sep 5, 2018 at 10:45 PM, Liu Bo wrote: >>> Somehow this ends up with crash in btrfs/124, I'm trying to figure out >>> what went wrong. >>

Re: [PATCH] Btrfs: remove redundant btrfs_trans_release_metadata"

2018-09-05 Thread Liu Bo
On Wed, Sep 5, 2018 at 10:45 PM, Liu Bo wrote: > Somehow this ends up with crash in btrfs/124, I'm trying to figure out > what went wrong. > It revealed the problem addressed in Josef's patch[1], so with it, this patch works just fine. [1] btrfs: make sure we create all new

Re: [PATCH 22/35] btrfs: make sure we create all new bgs

2018-09-05 Thread Liu Bo
serve_extent => btrfs_alloc_tree_block => __btrfs_cow_block => btrfs_cow_block => btrfs_search_slot => btrfs_update_device => btrfs_finish_chunk_alloc => btrfs_create_pending_block_groups ... Reviewed-by: Liu Bo thanks, liubo

Re: [PATCH] Btrfs: remove redundant btrfs_trans_release_metadata"

2018-09-05 Thread Liu Bo
Somehow this ends up with crash in btrfs/124, I'm trying to figure out what went wrong. thanks, liubo On Tue, Sep 4, 2018 at 6:14 PM, Liu Bo wrote: > __btrfs_end_transaction() has done the metadata release twice, > probably because it used to process delayed refs in between, but no

[PATCH v2] Btrfs: remove confusing tracepoint in btrfs_add_reserved_bytes

2018-09-04 Thread Liu Bo
Here we're not releasing any space, but transferring bytes from ->bytes_may_use to ->bytes_reserved. Signed-off-by: Liu Bo --- v2: Add missing commit log. fs/btrfs/extent-tree.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-t

Re: [PATCH 20/35] btrfs: reset max_extent_size on clear in a bitmap

2018-09-04 Thread Liu Bo
On Thu, Aug 30, 2018 at 10:42 AM, Josef Bacik wrote: > From: Josef Bacik > > We need to clear the max_extent_size when we clear bits from a bitmap > since it could have been from the range that contains the > max_extent_size. > Looks OK. Reviewed-by: Liu Bo thanks, liub

Re: [PATCH 08/35] btrfs: release metadata before running delayed refs

2018-09-04 Thread Liu Bo
On Thu, Aug 30, 2018 at 10:41 AM, Josef Bacik wrote: > We want to release the unused reservation we have since it refills the > delayed refs reserve, which will make everything go smoother when > running the delayed refs if we're short on our reservation. > Looks good. Re

[PATCH] Btrfs: remove confusing tracepoint in btrfs_add_reserved_bytes

2018-09-04 Thread Liu Bo
Signed-off-by: Liu Bo --- fs/btrfs/extent-tree.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 41a02cbb5a4a..76ee5ebef2b9 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -6401,10 +6401,6 @@ static int

[PATCH] Btrfs: remove redundant btrfs_trans_release_metadata"

2018-09-04 Thread Liu Bo
__btrfs_end_transaction() has done the metadata release twice, probably because it used to process delayed refs in between, but now that we don't process delayed refs any more, the 2nd release is always a noop. Signed-off-by: Liu Bo --- fs/btrfs/transaction.c | 6 -- 1 file chang

Re: [PATCH 02/35] btrfs: add cleanup_ref_head_accounting helper

2018-09-04 Thread Liu Bo
nsure that > we don't screw up accounting in the future for other things that we add. > Reviewed-by: Liu Bo thanks, liubo > Signed-off-by: Josef Bacik > --- > fs/btrfs/extent-tree.c | 67 > +- > 1 file changed, 39 inse

[PATCH] Btrfs: fix alignment in declaration and prototype of btrfs_get_extent

2018-08-24 Thread Liu Bo
This fixes btrfs_get_extent to be consistent with our existing declaration style. Signed-off-by: Liu Bo --- fs/btrfs/ctree.h | 4 ++-- fs/btrfs/inode.c | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 1aeed3c0e949..00e506de70ba

[PATCH v3] Btrfs: set leave_spinning in btrfs_get_extent

2018-08-24 Thread Liu Bo
ath in spinning mode typically has less impact (faster) on waiters rather than that in blocking mode. Also fixes the misalignment of the prototype, which is too trivial for a single patch. Signed-off-by: Liu Bo --- v3: separate alignment fix out of this patch. fs/btrfs/inode.c | 8 1

Re: [PATCH v2] Btrfs: set leave_spinning in btrfs_get_extent

2018-08-24 Thread Liu Bo
On Fri, Aug 24, 2018 at 04:59:49PM +0200, David Sterba wrote: > On Thu, Aug 23, 2018 at 07:41:05AM +0800, Liu Bo wrote: > > Unless it's going to read inline extents from btree leaf to page, > > btrfs_get_extent won't sleep during the period of holding path lock. > >

[PATCH v2] Btrfs: set leave_spinning in btrfs_get_extent

2018-08-22 Thread Liu Bo
ath in spinning mode typically has less impact (faster) on waiters rather than that in blocking mode. Also fixes the misalignment of the prototype, which is too trivial for a single patch. Signed-off-by: Liu Bo --- v2: send out the correct patch. fs/btrfs/ctree.h | 4 ++-- fs/btr

[PATCH] Btrfs: btrfs_get_extent: put btrfs_free_path ahead

2018-08-22 Thread Liu Bo
trace_btrfs_get_extent() has nothing to do with path, place btrfs_free_path ahead so that we can unlock path on error. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 31d43355f052

[PATCH] Btrfs: set leave_spinning in btrfs_get_extent

2018-08-22 Thread Liu Bo
ath in spinning mode typically has less impact (faster) on waiters rather than that in blocking mode. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 10e86f7b5398..4219be9c21f9 100644

Re: [PATCH 0/2] address lock contention of btree root

2018-08-22 Thread Liu Bo
On Tue, Aug 21, 2018 at 02:48:47PM -0400, Chris Mason wrote: > On 21 Aug 2018, at 14:15, Liu Bo wrote: > > > On Tue, Aug 21, 2018 at 01:54:11PM -0400, Chris Mason wrote: > > > On 16 Aug 2018, at 17:07, Liu Bo wrote: > > > > > > > The lock contention on

[PATCH 4/5] Btrfs: extent_map: use rb_first_cached

2018-08-22 Thread Liu Bo
rb_first_cached() trades an extra pointer "leftmost" for doing the same job as rb_first() but in O(1). As evict_inode_truncate_pages() removes all extent mapping by always looking for the first rb entry, it's helpful to use rb_first_cached instead. Signed-off-by: Liu Bo

[PATCH 2/5] Btrfs: href->ref_tree: use rb_first_cached

2018-08-22 Thread Liu Bo
rb_first_cached() trades an extra pointer "leftmost" for doing the same job as rb_first() but in O(1). Functions manipulating href->ref_tree need to get the first entry, this converts href->ref_tree to use rb_first_cached(). Signed-off-by: Liu Bo --- fs/btrfs/backref.c |

[PATCH 0/5] rb_first to rb_first_cached conversion

2018-08-22 Thread Liu Bo
Several structs in btrfs are using rb_first() in a while loop, it'd be more efficient to do this with rb_first_cached() which has the O(1) complexity. This patch set updates five structs which may have a large rb tree in practice Liu Bo (5): Btrfs: href_root: use rb_first_cached Btrfs:

[PATCH 1/5] Btrfs: href_root: use rb_first_cached

2018-08-22 Thread Liu Bo
rb_first_cached() trades an extra pointer "leftmost" for doing the same job as rb_first() but in O(1). Functions manipulating href_root need to get the first entry, this converts href_root to use rb_first_cached(). Signed-off-by: Liu Bo --- fs/btrfs/delayed-

[PATCH 3/5] Btrfs: delayed_items: use rb_first_cached

2018-08-22 Thread Liu Bo
rb_first_cached() trades an extra pointer "leftmost" for doing the same job as rb_first() but in O(1). Functions manipulating delayed_item need to get the first entry, this converts it to use rb_first_cached(). Signed-off-by: Liu Bo --- fs/btrfs/delayed-in

[PATCH 5/5] Btrfs: preftree: use rb_first_cached

2018-08-22 Thread Liu Bo
rb_first_cached() trades an extra pointer "leftmost" for doing the same job as rb_first() but in O(1). While resolving indirect refs and missing refs, it always looks for the first rb entry in a while loop, it's helpful to use rb_first_cached instead. Signed-off-by: Liu Bo

[PATCH] Btrfs: use next_state in find_first_extent_bit

2018-08-22 Thread Liu Bo
As next_state() is already defined to get the next state, update us to use the helper. No functional changes. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 628f1aef34b0

Re: [PATCH v2] Btrfs: kill btrfs_clear_path_blocking

2018-08-22 Thread Liu Bo
On Wed, Aug 22, 2018 at 03:13:28PM +0200, David Sterba wrote: > On Wed, Aug 22, 2018 at 05:54:37AM +0800, Liu Bo wrote: > > w/o patch: > > real2m27.307s > > user0m12.839s > > sys 13m42.831s > > > > w/ patch: > > real1m2.273s > >

Re: [PATCH 0/2] address lock contention of btree root

2018-08-21 Thread Liu Bo
On Tue, Aug 21, 2018 at 02:48:47PM -0400, Chris Mason wrote: > On 21 Aug 2018, at 14:15, Liu Bo wrote: > > > On Tue, Aug 21, 2018 at 01:54:11PM -0400, Chris Mason wrote: > > > On 16 Aug 2018, at 17:07, Liu Bo wrote: > > > > > > > The lock contention on

Re: [PATCH 2/2] Btrfs: kill leave_spinning

2018-08-21 Thread Liu Bo
Please kindly ignore this patch as it did introduce soft lockup when btrfs_search_slot() returns a spinning path and its callers goes to sleep before releasing the path. thanks, liubo On Thu, Aug 16, 2018 at 2:07 PM, Liu Bo wrote: > As btrfs_clear_path_blocking() turns out to be a major sou

Re: [PATCH 0/2] address lock contention of btree root

2018-08-21 Thread Liu Bo
On Tue, Aug 21, 2018 at 08:35:30PM +0200, Holger Hoffstätte wrote: > On 08/21/18 20:15, Liu Bo wrote: > > I just realize that patch 2 can result in softlockup as > > btrfs_search_slot() may return a path with all nodes being in spinning > > lock, and if the callers want to sl

[PATCH v2] Btrfs: kill btrfs_clear_path_blocking

2018-08-21 Thread Liu Bo
ready to not sleep before releasing the path, but it won't cause problems if they don't want to sleep in blocking mode. [1]: https://github.com/iovisor/bcc/blob/master/tools/funclatency.py Signed-off-by: Liu Bo --- v2: Add latency histogram for btrfs_tree*

Re: [PATCH 0/2] address lock contention of btree root

2018-08-21 Thread Liu Bo
On Tue, Aug 21, 2018 at 01:54:11PM -0400, Chris Mason wrote: > On 16 Aug 2018, at 17:07, Liu Bo wrote: > > > The lock contention on btree nodes (esp. root node) is apparently a > > bottleneck when there're multiple readers and writers concurrently > > trying to access

Re: [PATCH 1/2] Btrfs: kill btrfs_clear_path_blocking

2018-08-21 Thread Liu Bo
On Mon, Aug 20, 2018 at 10:31:49AM +0300, Nikolay Borisov wrote: > > > On 20.08.2018 09:07, Liu Bo wrote: > > On Fri, Aug 17, 2018 at 10:24:58AM +0300, Nikolay Borisov wrote: > >> > >> > >> On 17.08.2018 00:07, Liu Bo wrote: > >>> Btrfs

Re: [PATCH 1/2] Btrfs: kill btrfs_clear_path_blocking

2018-08-19 Thread Liu Bo
On Fri, Aug 17, 2018 at 10:24:58AM +0300, Nikolay Borisov wrote: > > > On 17.08.2018 00:07, Liu Bo wrote: > > Btrfs's btree locking has two modes, spinning mode and blocking mode, > > while searching btree, locking is always acquired in spinning mode and > > th

[PATCH 1/2] Btrfs: kill btrfs_clear_path_blocking

2018-08-16 Thread Liu Bo
_spinning. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 57 fs/btrfs/ctree.h | 2 -- fs/btrfs/delayed-inode.c | 3 --- 3 files changed, 4 insertions(+), 58 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index d436fb4c

[PATCH 2/2] Btrfs: kill leave_spinning

2018-08-16 Thread Liu Bo
As btrfs_clear_path_blocking() turns out to be a major source of lock contention, we've kill it and without it btrfs_search_slot() and btrfs_search_old_slot() are not able to return a path in spinning mode, lets kill leave_spinning, too. Signed-off-by: Liu Bo --- fs/btrfs/back

[PATCH 0/2] address lock contention of btree root

2018-08-16 Thread Liu Bo
and dbench. Patch 2 kills leave_spinning due to the behaviour change from patch 1. Liu Bo (2): Btrfs: kill btrfs_clear_path_blocking Btrfs: kill leave_spinning fs/btrfs/backref.c| 3 -- fs/btrfs/ctree.c | 73 +-- fs/btr

[PATCH] Btrfs: remove always true if branch in btrfs_get_extent

2018-08-16 Thread Liu Bo
@path is always NULL when it comes to the if branch. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 17 +++-- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 8b135a46835f..4b79916472fb 100644 --- a/fs/btrfs/inode.c +++ b/fs

Re: [PATCH] Btrfs: remove duplicated btrfs_release_path in btrfs_unlink_subvol

2018-08-15 Thread Liu Bo
On Wed, Aug 15, 2018 at 4:48 AM, David Sterba wrote: > On Wed, Aug 15, 2018 at 10:52:56AM +0800, Liu Bo wrote: >> On Tue, Aug 14, 2018 at 12:46:00PM +0200, David Sterba wrote: >> > On Tue, Aug 14, 2018 at 10:47:09AM +0800, Liu Bo wrote: >> > > The btrfs_release_pat

Re: [PATCH] Btrfs: remove duplicated btrfs_release_path in btrfs_unlink_subvol

2018-08-14 Thread Liu Bo
On Tue, Aug 14, 2018 at 12:46:00PM +0200, David Sterba wrote: > On Tue, Aug 14, 2018 at 10:47:09AM +0800, Liu Bo wrote: > > The btrfs_release_path() is just useless as path is only used in error > > handling. > > Where is it duplicated? And I don't think it's usel

[PATCH] Btrfs: remove duplicated btrfs_release_path in btrfs_unlink_subvol

2018-08-13 Thread Liu Bo
The btrfs_release_path() is just useless as path is only used in error handling. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index eba61bcb9bb3..5680e9c462a3 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs

[PATCH] Btrfs: do not pass write_lock_level when processing leaf

2018-08-13 Thread Liu Bo
As we're going to return, it doesn't make sense to get a new write_lock_level from unlock_up. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 41eb47488e75..f032b48094b4 100644 ---

Re: [Bug 199931] New: systemd/rtorrent file data corruption when using echo 3 >/proc/sys/vm/drop_caches

2018-06-06 Thread Liu Bo
On Wed, Jun 6, 2018 at 9:44 PM, Chris Mason wrote: > On 6 Jun 2018, at 9:38, Liu Bo wrote: > >> On Wed, Jun 6, 2018 at 8:18 AM, Chris Mason wrote: >>> >>> >>> >>> On 5 Jun 2018, at 16:03, Andrew Morton wrote: >>> >>>> (switched

Re: [Bug 199931] New: systemd/rtorrent file data corruption when using echo 3 >/proc/sys/vm/drop_caches

2018-06-06 Thread Liu Bo
On Wed, Jun 6, 2018 at 8:18 AM, Chris Mason wrote: > > > On 5 Jun 2018, at 16:03, Andrew Morton wrote: > >> (switched to email. Please respond via emailed reply-to-all, not via the >> bugzilla web interface). >> >> On Tue, 05 Jun 2018 18:01:36 + bugzilla-dae...@bugzilla.kernel.org >> wrote: >

[PATCH v3] Btrfs: remove unused check of skip_locking

2018-05-29 Thread Liu Bo
The check is superfluous since all of callers who set search_for_commit also have skip_locking set. ASSERT() is put in place to ensure skip_locking is set by callers. Reviewed-by: Qu Wenruo Signed-off-by: Liu Bo --- v3: Add assertion and comments to ensure skip_locking is not forgot by callers

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-29 Thread Liu Bo
On Mon, May 28, 2018 at 06:17:18PM +0200, David Sterba wrote: > On Mon, May 28, 2018 at 04:40:28PM +0200, David Sterba wrote: > > On Fri, May 18, 2018 at 11:00:18AM +0800, Liu Bo wrote: > > > Here are a collection of patches I did for btrfs_search_slot(). > > > > &g

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-23 Thread Liu Bo
On Wed, May 23, 2018 at 02:34:40PM +0200, David Sterba wrote: > On Wed, May 23, 2018 at 10:16:55AM +0800, Su Yue wrote: > > >>> [ 47.692084] kernel BUG at fs/btrfs/locking.c:286! > > >> > > >> I saw the crash too but did not investigate the root cause. So I'll > > >> remove the branch from for-ne

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-18 Thread Liu Bo
On Wed, May 16, 2018 at 04:12:21PM +0200, David Sterba wrote: > On Wed, May 16, 2018 at 11:00:14AM +0300, Nikolay Borisov wrote: > > > > > > On 15.05.2018 20:37, Liu Bo wrote: > > > If a btree block, aka. extent buffer, is not available in the extent > > &g

[PATCH v2 4/6] Btrfs: remove unused check of skip_locking

2018-05-17 Thread Liu Bo
The check is superfluous since all of callers who set search_for_commit also have skip_locking set. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index d12fc0474e21..8d3b09038f37 100644 --- a/fs/btrfs/ctree.c

[PATCH v2 2/6] Btrfs: use more straightforward extent_buffer_uptodate

2018-05-17 Thread Liu Bo
If parent_transid "0" is passed to btrfs_buffer_uptodate(), btrfs_buffer_uptodate() is equivalent to extent_buffer_uptodate(), but extent_buffer_uptodate() is preferred since we don't have to look into verify_parent_transid(). Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1

[PATCH v2 1/6] Btrfs: remove superfluous free_extent_buffer

2018-05-17 Thread Liu Bo
read_block_for_search() can be simplified as, tmp = find_extent_buffer(); if (tmp) return; free_extent_buffer(); read_tree_block(); Apparently, @tmp must be NULL at this point, free_extent_buffer() is not needed. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 1 - 1 file changed, 1 deletion

[PATCH v2 3/6] Btrfs: move get root of btrfs_search_slot to a helper

2018-05-17 Thread Liu Bo
It's good to have a helper instead of having all get-root details open-coded. The new helper locks (if necessary) and sets root node of the path. Also invert the checks to make the code flow easier to read. There is no functional change in this commit. Signed-off-by: Liu Bo --- fs/

[PATCH v2 5/6] Btrfs: grab write lock directly if write_lock_level is the max level

2018-05-17 Thread Liu Bo
e root node's write lock directly. In this particular case, the dance of acquiring read lock and then trading for write lock can be saved. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 29 - 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/ct

[PATCH v2 6/6] Btrfs: remove always true check in unlock_up

2018-05-17 Thread Liu Bo
As unlock_up() is written as for () { if (!path->locks[i]) break; ... if (... && path->locks[i]) { } } Apparently, @path->locks[i] is always true at this 'if'. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file changed, 1 insertion(+), 1 d

[PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-17 Thread Liu Bo
Here are a collection of patches I did for btrfs_search_slot(). v2: more explicit commit log for each patch. Liu Bo (6): Btrfs: remove superfluous free_extent_buffer Btrfs: use more straightforward extent_buffer_uptodate Btrfs: move get root of btrfs_search_slot to a helper Btrfs: remove

[PATCH v2] Btrfs: add parent_transid parameter to veirfy_level_key

2018-05-17 Thread Liu Bo
() reports something wrong, we're not going to know if it's caused by corrupted metadata or non-checkecd eb (e.g. stale eb). @parent_transid is able to tell whether the eb's generation has been verified by the caller. Signed-off-by: Liu Bo --- v2: - more explicit commit log. - adju

Re: [PATCH 4/6] Btrfs: remove unused check of skip_locking

2018-05-17 Thread Liu Bo
On Wed, May 16, 2018 at 3:03 PM, Nikolay Borisov wrote: > > > On 15.05.2018 20:52, Liu Bo wrote: >> The check is superfluous since all of callers who set search_for_commit >> also have skip_locking set. > > This is false. For example btrfs_qgroup_rescan_worker sets &

Re: [PATCH 2/6] Btrfs: use more straightforward extent_buffer_uptodate

2018-05-17 Thread Liu Bo
On Wed, May 16, 2018 at 2:43 PM, Nikolay Borisov wrote: > > > On 15.05.2018 20:52, Liu Bo wrote: >> In read_block_for_search(), it's straightforward to use >> extent_buffer_uptodate() instead since 0 is passed as parent transid to > > "instead of the more

[PATCH 2/6] Btrfs: use more straightforward extent_buffer_uptodate

2018-05-15 Thread Liu Bo
In read_block_for_search(), it's straightforward to use extent_buffer_uptodate() instead since 0 is passed as parent transid to btrfs_buffer_uptodate(), which means the check for parent transid is not needed. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file changed, 1 insertion(

[PATCH 6/6] Btrfs: remove always true check in unlock_up

2018-05-15 Thread Liu Bo
@path->lock[i] is always true at this point. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index f7c3f581f647..16d28a4ec54f 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -233

[PATCH 4/6] Btrfs: remove unused check of skip_locking

2018-05-15 Thread Liu Bo
The check is superfluous since all of callers who set search_for_commit also have skip_locking set. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 399839df7a8f..cf34eca41d4e 100644 --- a/fs/btrfs/ctree.c

[PATCH 3/6] Btrfs: move get root of btrfs_search_slot to a helper

2018-05-15 Thread Liu Bo
It's good to have a helper instead of having all get-root details open-coded. The new helper locks (if necessary) and sets root node of the path. There is no functional change in this commit. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c

[PATCH 1/6] Btrfs: remove superfluous free_extent_buffer

2018-05-15 Thread Liu Bo
@tmp must be NULL at this point, free_extent_buffer is not needed at all. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index b3f6f300e492..9fa3d77c98d4 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c

[PATCH 0/6] btrfs_search_slot cleanups

2018-05-15 Thread Liu Bo
These're the cleanups I made for btrfs_search_slot() and its callees. Liu Bo (6): Btrfs: remove superfluous free_extent_buffer Btrfs: use more straightforward extent_buffer_uptodate Btrfs: move get root of btrfs_search_slot to a helper Btrfs: remove unused check of skip_locking

[PATCH 5/6] Btrfs: grab write lock directly if write_lock_level is the max level

2018-05-15 Thread Liu Bo
In case of (cow && (p->keep_locks || p->lowest_level)), write_lock_level is the max level, and we should grab write lock of root node from the very beginning. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 29 - 1 file changed, 16 insertions(+), 13 dele

[PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-15 Thread Liu Bo
needs to be passed to read_tree_block(). In order to get a valid parent transid, we need to hold the parent's lock until finish reading child. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.

[PATCH] Btrfs: add parent_transid parameter to veirfy_level_key

2018-05-15 Thread Liu Bo
@parent_transid could tell whether the eb's generation has been verified by the caller. Signed-off-by: Liu Bo --- fs/btrfs/disk-io.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 60caa68c3618..b5d55b0ec19b 100644

Re: verify key failure

2018-05-15 Thread Liu Bo
On Tue, May 15, 2018 at 9:29 AM, Qu Wenruo wrote: > > > On 2018年05月14日 22:35, Liu Bo wrote: >> Hi, >> >> I got another warning of verify_level_key by running btrfs/124 in a loop, >> I'm testing against 4.17-rc3. >> >> Not sure if it's fa

Re: verify key failure

2018-05-15 Thread Liu Bo
On Tue, May 15, 2018 at 12:10 AM, Chris Mason wrote: > > > On 14 May 2018, at 10:35, Liu Bo wrote: > >> Hi, >> >> I got another warning of verify_level_key by running btrfs/124 in a loop, >> I'm testing against 4.17-rc3. >> >> Not sure if

verify key failure

2018-05-14 Thread Liu Bo
Hi, I got another warning of verify_level_key by running btrfs/124 in a loop, I'm testing against 4.17-rc3. Not sure if it's false positive. [101414.336691] WARNING: CPU: 3 PID: 30194 at fs/btrfs/disk-io.c:455 btree_read_extent_buffer_pages+0x183/0x220 [btrfs] [101414.340372] Modules linked in

Re: Problem with current->journal_info for BTRFS, Null pointer dereference in start_transaction

2018-05-07 Thread Liu Bo
On Mon, May 7, 2018 at 2:49 PM, robbieko wrote: > Hi, > > When send process requires memory allocation, shrinker may be triggered due > to insufficient memory. > Then evict_inode gets called when inode is dropped, and this function may > need to start transaction. > However, the journal_info is al

  1   2   3   4   5   6   7   8   9   10   >