This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/falloc_speedup
Which is based on v5.0-rc1 tag, with another enospc debug patch.
Btrfs falloc can be slower and slower when there are more and more block
groups.
One cause of this problem is find_free_dev_extent(),
[PROBLEM]
When allocating a 4G file, falloc() call is slower and slower if the fs
is more and more filled.
This is going to be super obvious when doing some crazy test.
E.g. fallocating a 1PiB file TiB by TiB.
The first 16T only takes 10 seconds to finish while the last 16TiB can
take over 15min t
verify_one_dev_extent() will call btrfs_find_device() for each dev
extent, this waste some CPU time just searching the devices list.
Move the search one level up, into the btrfs_verify_dev_extents(), so
for each device we only call btrfs_find_device() once.
Signed-off-by: Qu Wenruo
---
fs/btrfs
[PROBLEM]
When allocating a 4G file, falloc() call is slower and slower if the fs
is more and more filled.
This is going to be super obvious when doing some crazy test.
E.g. fallocating a 1PiB file TiB by TiB.
The first 16T only takes 10 seconds to finish while the last 16TiB can
take over 15min t
Circular locking dependency check reports warning[1], that's because
the btrfs_scrub_dev() calls the stack #0 below with, the
fs_info::scrub_lock held. The test case leading to this warning..
mkfs.btrfs -fq /dev/sdb && mount /dev/sdb /btrfs
btrfs scrub start -B /btrfs
In fact we have fs_info:
Fixes the circular locking dependency warning as in patch 1/3,
and patch 2/3 adds lockdep_assert_held() to scrub_workers_get().
Patch 3/3 converts scrub_workers_refcnt into refcount_t.
Anand Jain (3):
btrfs: scrub: fix circular locking dependency warning
btrfs: scrub: add scrub_lock lockdep ch
scrub_workers_refcnt is protected by scrub_lock, add lockdep_assert_held()
in scrub_workers_get().
Signed-off-by: Anand Jain
Suggested-by: Nikolay Borisov
---
v4: none
v3: none
v2: born
fs/btrfs/scrub.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
in
Use the refcount_t for fs_info::scrub_workers_refcnt instead of int.
Signed-off-by: Anand Jain
---
v4: born
fs/btrfs/ctree.h | 2 +-
fs/btrfs/disk-io.c | 2 +-
fs/btrfs/scrub.c | 6 +++---
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
inde
Since inc_block_group_ro() would return -ENOSPC, outputting debug info
for enospc_debug mount option would be helpful to debug some balance
false ENOSPC report.
Signed-off-by: Qu Wenruo
---
fs/btrfs/extent-tree.c | 15 +--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git
Going back to my original email, would the BTRFS wiki admins consider a
better more reflective update of the RAID56 status page?
It still states "multiple serious data-loss bugs" which as Qu Wenruo has
already clarified is not the case. The only "bug" left is the write
hole edge-case problem.
On Tue, Jan 29, 2019 at 4:15 PM Hans van Kranenburg
wrote:
>
> Hi,
>
> Thought experiment time...
>
> I have an HP z820 workstation here (with ECC memory, yay!) and 4x250G
> 10k SAS disks (and some spare disks). It's donated hardware, and I'm
> going to use it to replace the current server in the
Hi Nikolay,
On Tue, Jan 29, 2019 at 10:22:34AM +0200, Nikolay Borisov wrote:
>
>
> On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> > The previous patch added generic helpers for get_workspace() and
> > put_workspace(). Now, we can migrate ownership of the workspace_manager
> > to be in the compres
Hi,
Thought experiment time...
I have an HP z820 workstation here (with ECC memory, yay!) and 4x250G
10k SAS disks (and some spare disks). It's donated hardware, and I'm
going to use it to replace the current server in the office of a
non-profit organization (so it's not work stuff this time).
T
On Tue, Jan 29, 2019 at 09:25:54AM +0200, Nikolay Borisov wrote:
>
>
> On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> > Zstd compression requires different amounts of memory for each level of
> > compression. The prior patches implemented indirection to allow for each
> > compression type to manag
On Tue, Jan 29, 2019 at 09:53:33AM +0200, Nikolay Borisov wrote:
>
>
> On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> > While the heuristic workspaces aren't really compression workspaces,
> > they use the same interface for managing them. So rather than branching,
> > let's just handle them once
On Mon, Jan 28, 2019 at 04:24:27PM -0500, Dennis Zhou wrote:
> It is very easy to miss places that rely on a certain bitshifting for
> decyphering the type_level overloading. Make macros handle this instead.
>
> Signed-off-by: Dennis Zhou
> ---
> fs/btrfs/compression.c | 2 +-
> fs/btrfs/compres
> On Jan 29, 2019, at 9:18 AM, David Sterba wrote:
>
> On Mon, Jan 28, 2019 at 04:24:26PM -0500, Dennis Zhou wrote:
>> As mentioned above, a requirement that differs zstd from zlib is that
>> higher levels of compression require more memory. To manage this, each
>> compression level has its ow
On 29/01/2019 20.02, Chris Murphy wrote:
> On Mon, Jan 28, 2019 at 3:52 PM Remi Gauvin wrote:
>>
>> On 2019-01-28 5:07 p.m., DanglingPointer wrote:
>>
>>> From Qu's statement and perspective, there's no difference to other
>>> non-BTRFS software RAID56's out there that are marked as stable (except
On Mon, Jan 28, 2019 at 3:52 PM Remi Gauvin wrote:
>
> On 2019-01-28 5:07 p.m., DanglingPointer wrote:
>
> > From Qu's statement and perspective, there's no difference to other
> > non-BTRFS software RAID56's out there that are marked as stable (except
> > ZFS).
> > Also there are no "multiple ser
On Tue, Jan 29, 2019 at 01:17:17PM -0500, Josef Bacik wrote:
> On Mon, Jan 28, 2019 at 04:24:33PM -0500, Dennis Zhou wrote:
> > The previous patch added generic helpers for get_workspace() and
> > put_workspace(). Now, we can migrate ownership of the workspace_manager
> > to be in the compression t
On Mon, Jan 28, 2019 at 04:24:34PM -0500, Dennis Zhou wrote:
> Zlib compression supports multiple levels, but doesn't require changing
> in how a workspace itself is created and managed. Zstd introduces a
> different memory requirement such that higher levels of compression
> require more memory. T
On Mon, Jan 28, 2019 at 04:24:33PM -0500, Dennis Zhou wrote:
> The previous patch added generic helpers for get_workspace() and
> put_workspace(). Now, we can migrate ownership of the workspace_manager
> to be in the compression type code as the compression code itself
> doesn't care beyond being a
On Mon, Jan 28, 2019 at 04:24:32PM -0500, Dennis Zhou wrote:
> There are two levels of workspace management. First, alloc()/free()
> which are responsible for actually creating and destroy workspaces.
> Second, at a higher level, get()/put() which is the compression code
> asking for a workspace fr
On Mon, Jan 28, 2019 at 04:24:31PM -0500, Dennis Zhou wrote:
> Workspace manager init and cleanup code is open coded inside a for loop
> over the compression types. This forces each compression type to rely on
> the same workspace manager implementation. This patch creates helper
> methods that wil
On Mon, Jan 28, 2019 at 04:24:30PM -0500, Dennis Zhou wrote:
> Make the workspace_manager own the interface operations rather than
> managing index-paired arrays for the workspace_manager and compression
> operations.
>
> Signed-off-by: Dennis Zhou
Reviewed-by: Josef Bacik
Thanks,
Josef
On Mon, Jan 28, 2019 at 04:24:29PM -0500, Dennis Zhou wrote:
> While the heuristic workspaces aren't really compression workspaces,
> they use the same interface for managing them. So rather than branching,
> let's just handle them once again as the index 0 compression type.
>
> Signed-off-by: Den
On Mon, Jan 28, 2019 at 04:24:28PM -0500, Dennis Zhou wrote:
> This is in preparation for zstd compression levels. As each level will
> require different sized workspaces, workspaces_list is no longer a
> really fitting name.
>
> Signed-off-by: Dennis Zhou
> ---
Reviewed-by: Josef Bacik
Thanks
On Mon, Jan 28, 2019 at 04:24:27PM -0500, Dennis Zhou wrote:
> It is very easy to miss places that rely on a certain bitshifting for
> decyphering the type_level overloading. Make macros handle this instead.
>
> Signed-off-by: Dennis Zhou
Reviewed-by: Josef Bacik
Thanks,
Josef
On Mon, Jan 28, 2019 at 04:24:26PM -0500, Dennis Zhou wrote:
> As mentioned above, a requirement that differs zstd from zlib is that
> higher levels of compression require more memory. To manage this, each
> compression level has its own queue of workspaces. A global LRU is used
> to help with recl
On Tue, Jan 29, 2019 at 04:21:27PM +0100, David Sterba wrote:
> On Tue, Jan 29, 2019 at 02:01:46PM +0100, Anders Roxell wrote:
> > When CONFIG_BRTFS_ASSERT isn't enabled, variable root_objectid isn't used.
> >
> > fs/btrfs/relocation.c: In function ‘insert_dirty_subv’:
> > fs/btrfs/relocation.c:21
On Tue, Jan 29, 2019 at 02:01:46PM +0100, Anders Roxell wrote:
> When CONFIG_BRTFS_ASSERT isn't enabled, variable root_objectid isn't used.
>
> fs/btrfs/relocation.c: In function ‘insert_dirty_subv’:
> fs/btrfs/relocation.c:2138:6: warning: unused variable ‘root_objectid’
> [-Wunused-variable]
>
When CONFIG_BRTFS_ASSERT isn't enabled, variable root_objectid isn't used.
fs/btrfs/relocation.c: In function ‘insert_dirty_subv’:
fs/btrfs/relocation.c:2138:6: warning: unused variable ‘root_objectid’
[-Wunused-variable]
u64 root_objectid = root->root_key.objectid;
^
Reworke
This is a test case for a long existing bug, caused by over-estimated
metadata space_info::bytes_may_use.
There is one proposed patch for btrfs-progs to fix it, titled:
"btrfs-progs: balance: Sync the fs before balancing metadata chunks"
The test case itself is almost the same as btrfs/181, which
Commit a514d63882c3 ("btrfs: qgroup: Commit transaction in advance to reduce
early EDQUOT") is no longer forcing transaction commit to reclaim space,
and only commits transaction asynchronously in advance to address it.
However the criteria used in async transaction commit is not
comprehensive, th
Kernel commit 64403612b73a ("btrfs: rework
btrfs_check_space_for_delayed_refs") is introducing a regression for
btrfs balance performance.
Since that commit will cause btrfs to commit too many transactions for
nothing during balance/relocation, it will slow balance dramatically even
we only need t
On Mon, Jan 28, 2019 at 2:05 AM Qu Wenruo wrote:
>
> Commit a514d63882c3 ("btrfs: qgroup: Commit transaction in advance to reduce
> early EDQUOT") is no longer forcing transaction commit to reclaim space,
> and only commits transaction asynchronously in advance to address it.
>
> However the crite
On Tue, Jan 29, 2019 at 7:19 AM Qu Wenruo wrote:
>
> Kernel commit 64403612b73a ("btrfs: rework
> btrfs_check_space_for_delayed_refs") is introducing a regression for
> btrfs balance performance.
>
> Since that commit will cause btrfs to commit transaction for nothing
> during transaction,
to com
On Tue, Jan 29, 2019 at 7:28 AM Qu Wenruo wrote:
>
> This is a long existing bug, caused by over-estimated metadata
> space_info::bytes_may_use.
Nop, this is a test case :)
The sentence should be more like "This is a test case for a long
existing bug, ...".
>
> There is one purposed patch for bt
On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> The previous patch added generic helpers for get_workspace() and
> put_workspace(). Now, we can migrate ownership of the workspace_manager
> to be in the compression type code as the compression code itself
> doesn't care beyond being able to get a w
On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> Zstd currently only supports the default level of compression. This
> patch switches to using the level passed in for btrfs zstd
> configuration.
>
> Zstd workspaces now keep track of the requested level as this can differ
> from the size of the wor
On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> Currently, the only user of set_level() is zlib which sets an internal
> workspace parameter. As level is now plumbed into get_workspace(), this
> can be handled there rather than separately.
>
> This repurposes set_level() to bound the level passed
On 28.01.19 г. 23:24 ч., Dennis Zhou wrote:
> Zlib compression supports multiple levels, but doesn't require changing
> in how a workspace itself is created and managed. Zstd introduces a
> different memory requirement such that higher levels of compression
> require more memory. This requires c
42 matches
Mail list logo
