[PATCH] btrfs-progs: image: Rebuild dev extents for restore

2018-08-16 Thread Qu Wenruo
When restoring image from a dump of multiple device, we just restore device as is, even the restore destination is just a single device. In that case, due to dev extents mismatch, latest kernel will refuse to mount it as it detects such mismatch as good as btrfs check. Fix it by rebuilding the

Re: btrfsck out of memory for big fs

2018-08-16 Thread Qu Wenruo
On 2018/8/17 下午1:26, litaibaich...@gmail.com wrote: > thanks Qu. > I am runing 4.12. > /# btrfs --version > btrfs-progs v4.12 > > Do you think btrfsck can fix the fs issue ? Nope, transid is pretty tricky especially for super old corruption. >  Or we better backup data and re-do the fs ?

Re: BTRFS w/ quotas hangs on read-write mount using all available RAM

2018-08-16 Thread Qu Wenruo
On 2018/8/17 上午11:47, Loren M. Lang wrote: > Hello, > > I am unable to mount my btrfs in read-write mode after enabling quotas > and running a full balance on it. My service is running Ubuntu 17.10 > with Linux kernel 4.13.0-17-generic and btrfs-progs 4.12. I am trying to > recover with a

Re: btrfsck out of memory for big fs

2018-08-16 Thread Qu Wenruo
On 2018/8/17 上午10:44, litaibaich...@gmail.com wrote: > Hi Guys, > > I had a big btrfs on a md device,  it be mounted , but after a while it will > become ready only: > # btrfs fi df /data/ > Data, single: total=24.46TiB, used=24.46TiB > System, DUP: total=8.00MiB, used=2.59MiB > System,

BTRFS w/ quotas hangs on read-write mount using all available RAM

2018-08-16 Thread Loren M. Lang
Hello, I am unable to mount my btrfs in read-write mode after enabling quotas and running a full balance on it. My service is running Ubuntu 17.10 with Linux kernel 4.13.0-17-generic and btrfs-progs 4.12. I am trying to recover with a live CD of Ubuntu 18.04.1 running Linux 4.15.0-29-generic

btrfsck out of memory for big fs

2018-08-16 Thread litaibaich...@gmail.com
Hi Guys, I had a big btrfs on a md device,  it be mounted , but after a while it will become ready only: # btrfs fi df /data/ Data, single: total=24.46TiB, used=24.46TiB System, DUP: total=8.00MiB, used=2.59MiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=81.00GiB,

[PATCH 1/2] Btrfs: kill btrfs_clear_path_blocking

2018-08-16 Thread Liu Bo
Btrfs's btree locking has two modes, spinning mode and blocking mode, while searching btree, locking is always acquired in spinning mode and then converted to blocking mode if necessary, and in some hot paths we may switch the locking back to spinning mode by btrfs_clear_path_blocking(). When

[PATCH 2/2] Btrfs: kill leave_spinning

2018-08-16 Thread Liu Bo
As btrfs_clear_path_blocking() turns out to be a major source of lock contention, we've kill it and without it btrfs_search_slot() and btrfs_search_old_slot() are not able to return a path in spinning mode, lets kill leave_spinning, too. Signed-off-by: Liu Bo --- fs/btrfs/backref.c|

[PATCH 0/2] address lock contention of btree root

2018-08-16 Thread Liu Bo
The lock contention on btree nodes (esp. root node) is apparently a bottleneck when there're multiple readers and writers concurrently trying to access them. Unfortunately this is by design and it's not easy to fix it unless with some complex changes, however, there is still some room. With a

[PATCH] Btrfs: remove always true if branch in btrfs_get_extent

2018-08-16 Thread Liu Bo
@path is always NULL when it comes to the if branch. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 17 +++-- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 8b135a46835f..4b79916472fb 100644 --- a/fs/btrfs/inode.c +++

Re: List of known BTRFS Raid 5/6 Bugs?

2018-08-16 Thread erenthetitan
Could you show scrub status -d, then start a new scrub (all drives) and show scrub status -d again? This may help us diagnose the problem. Am 15-Aug-2018 09:27:40 +0200 schrieb men...@gmail.com: > I needed to resume scrub two times after an unclear shutdown (I was > cooking and using too much

[PATCH 3/8] btrfs-progs: Add delayed refs infrastructure

2018-08-16 Thread Nikolay Borisov
This commit pulls those portions of the kernel implementation of delayed refs which are necessary to have them working in user-space. I've done the following modifications: 1. Replaced all kmem_cache_alloc calls to kmalloc. 2. Removed all locking-related code, since we are single threaded in

[PATCH 1/8] btrfs-progs: Add __free_extent2 function

2018-08-16 Thread Nikolay Borisov
This is a simple adapter to convert the arguments delayed ref arguments to the existing arguments of __free_extent. Signed-off-by: Nikolay Borisov Signed-off-by: David Sterba --- extent-tree.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/extent-tree.c b/extent-tree.c index

[PATCH 8/8] btrfs-progs: Merge alloc_reserved_tree_block(2|)

2018-08-16 Thread Nikolay Borisov
Now that delayed refs have been wired let's merge the two function. In the process also remove one BUG_ON since alloc_reserved_tree_block's callers can handle errors. No functional changes. Signed-off-by: Nikolay Borisov --- extent-tree.c | 77

[PATCH 2/8] btrfs-progs: Add alloc_reserved_tree_block2 function

2018-08-16 Thread Nikolay Borisov
This is a simple adapter function to convert the delayed-refs structures to the current arguments of alloc_reserved_tree_block. Signed-off-by: Nikolay Borisov Signed-off-by: David Sterba --- extent-tree.c | 24 1 file changed, 24 insertions(+) diff --git

[PATCH 6/8] btrfs-progs: Remove old delayed refs infrastructure

2018-08-16 Thread Nikolay Borisov
Given that the new delayed refs infrastructure is implemented and wired up, there is no point in keeping the old code. So just remove it. Signed-off-by: Nikolay Borisov Signed-off-by: David Sterba --- ctree.h | 2 - disk-io.c | 2 - extent-tree.c | 137

[PATCH 7/8] btrfs-progs: Remove __free_extent2

2018-08-16 Thread Nikolay Borisov
Now that delayed refs have been all wired up clean up the __free_extent2 adapter function since it's no longer needed. No functional changes. Signed-off-by: Nikolay Borisov --- extent-tree.c | 15 +++ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/extent-tree.c

[PATCH 5/8] btrfs-progs: Wire up delayed refs

2018-08-16 Thread Nikolay Borisov
This commit enables the delayed refs infrastructures. This entails doing the following: 1. Replacing existing calls of btrfs_extent_post_op (which is the equivalent of delayed refs) with the proper btrfs_run_delayed_refs. As well as eliminating open-coded calls to finish_current_insert and

[PATCH 4/8] btrfs-progs: Make btrfs_write_dirty_block_groups take only trans argument

2018-08-16 Thread Nikolay Borisov
The root argument is used only to get a reference to the fs_info, this can be achieved with the transaction handle being passed so use that. This is in preparation for moving this function in the main transaction commit routine. No functional changes. Signed-off-by: Nikolay Borisov --- ctree.h

[PATCH 0/8 V2] Add delayed-refs support to btrfs-progs

2018-08-16 Thread Nikolay Borisov
Hello, Here is the second version of the delayed refs for progs support. The first version can be found here [1]. I've taken into account all the feedback from Misono and have verified the code is working and rebased it atop btrfs-progs 4.17.1. Changes since v1: * Removed num_entries

Re: Are the btrfs mount options inconsistent?

2018-08-16 Thread David Sterba
On Thu, Aug 16, 2018 at 12:01:25PM +0100, David Howells wrote: > I'm trying to convert btrfs to use the new mount API stuff and I'm finding it > hard to work out the relationships between some of the arguments, specifically > datacow, datasum and compress*. > > What I see is that enabling datasum

Are the btrfs mount options inconsistent?

2018-08-16 Thread David Howells
Hi Chris, I'm trying to convert btrfs to use the new mount API stuff and I'm finding it hard to work out the relationships between some of the arguments, specifically datacow, datasum and compress*. What I see is that enabling datasum implies enabling datacow and that disabling datacow implies

Re: Transaction aborted - 4.16.17 kernel

2018-08-16 Thread Nikolay Borisov
On 16.08.2018 11:07, Qu Wenruo wrote: > > > On 2018/8/16 下午3:51, David Goodwin wrote: >> I've just spotted this on one server. >> >> Running : umount /backups && mount /backups >> >> seems to allow it to become r/w again, but it does write : >> >> BTRFS error (device xvdj): cleaner

Re: Transaction aborted - 4.16.17 kernel

2018-08-16 Thread Qu Wenruo
On 2018/8/16 下午3:51, David Goodwin wrote: > I've just spotted this on one server. > > Running : umount /backups && mount /backups > > seems to allow it to become r/w again, but it does write : > > BTRFS error (device xvdj): cleaner transaction attach returned -30 > > to 'dmesg'. > > >

Transaction aborted - 4.16.17 kernel

2018-08-16 Thread David Goodwin
I've just spotted this on one server. Running : umount /backups && mount /backups seems to allow it to become r/w again, but it does write : BTRFS error (device xvdj): cleaner transaction attach returned -30 to 'dmesg'. David. [ cut here ] BTRFS: Transaction