Fw: Re: Migration to BTRFS
For some reason, this did not come through... -- Weitergeleitete Nachricht -- Von: "Hendrik Friedel" An: "linux-btrfs@vger.kernel.org" Gesendet: 25.05.2019 15:21:43 Betreff: Re: Migration to BTRFS Hello now after the filesystem worked fine as a single drive for a while, I'd like to add the second device. Status: btrfs fi show . Label: 'DataPool1' uuid: c4a6a2c9-5cf0-49b8-812a-0784953f9ba3 Total devices 1 FS bytes used 6.61TiB devid 1 size 7.28TiB used 6.89TiB path /dev/sdh1 I intend to move to BTRFS and of course I have some data already. I currently have several single 4TB drives and I would like to move the Data onto new drives (2*8TB). I need no raid, as I prefer a backup. Nevertheless, having raid nice for availability. So why not in the end. I currently use ~6TB, so it may work, but I would be able to remove the redundancy later. So, if I understand correctly, today I want -m raid1 -d raid1 whereas later, I want -m raid1 -d single What is very important to me is, that with one failing drive, I have no risk of losing the whole filesystem, but only losing the affected drive. Is that possible with both of these variants? So, now I'd like to go this step: -m raid1 -d raid1 Is it correct to: btrfs device add /dev/sdd /srv/DataPool btrfs balance start -dconvert=raid1 -mconvert=raid1 Or is there anything else, that I need to take care off? There is not so much space left. Is it sufficient for the balance? Regards, Hendrik
[PATCH] btrfs-progs: Output extent tree leaf if we failed to find a backref
There is a bug report of BUG_ON() which is caused by __free_extent()
failed to lookup a backref extent:
Failed to find [1429288337408, 168, 16384]
btrfs unable to find ref byte nr 1429288583168 parent 0 root 2 owner 0 offset 0
convert/source-ext2.c:834: ext2_copy_inodes: BUG_ON ret triggered, value -5
./btrfs-convert[0x410941]
./btrfs-convert(main+0x1fdc)[0x40d3b8]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f93bb7d2f33]
./btrfs-convert(_start+0x2e)[0x40a96e]
It's still uncertain how this bug can be triggered, but adding such
debug output will provide more info for us to debug.
Signed-off-by: Qu Wenruo
---
extent-tree.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/extent-tree.c b/extent-tree.c
index 1cff617baa97..db69c538bf2f 100644
--- a/extent-tree.c
+++ b/extent-tree.c
@@ -2177,6 +2177,8 @@ static int __free_extent(struct btrfs_trans_handle *trans,
(unsigned long long)root_objectid,
(unsigned long long)owner_objectid,
(unsigned long long)owner_offset);
+ printf("path->slots[0]: %u path->nodes[0]:\n", path->slots[0]);
+ btrfs_print_leaf(path->nodes[0]);
ret = -EIO;
goto fail;
}
--
2.21.0
[PATCH] btrfs-progs: Enable crc32 optimization probe for convert and mkfs
Although moderm hardware is fast enough and crc32 calculation is not a
hotspot, doing such optimization won't hurt anyway.
Signed-off-by: Qu Wenruo
---
convert/main.c | 2 ++
mkfs/main.c| 3 +++
2 files changed, 5 insertions(+)
diff --git a/convert/main.c b/convert/main.c
index 68f76f716be9..31e54a5c000e 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -101,6 +101,7 @@
#include "mkfs/common.h"
#include "convert/common.h"
#include "convert/source-fs.h"
+#include "kernel-lib/crc32c.h"
#include "fsfeatures.h"
extern const struct btrfs_convert_operations ext2_convert_ops;
@@ -1845,6 +1846,7 @@ int main(int argc, char *argv[])
return 1;
}
+ crc32c_optimization_init();
if (rollback) {
ret = do_rollback(file);
} else {
diff --git a/mkfs/main.c b/mkfs/main.c
index 1d03ec52ddd6..5727d61df2d7 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -42,6 +42,7 @@
#include "rbtree-utils.h"
#include "mkfs/common.h"
#include "mkfs/rootdir.h"
+#include "kernel-lib/crc32c.h"
#include "fsfeatures.h"
static int verbose = 1;
@@ -1121,6 +1122,8 @@ int main(int argc, char **argv)
dev_cnt--;
+ crc32c_optimization_init();
+
/*
* Open without O_EXCL so that the problem should not occur by the
* following operation in kernel:
--
2.21.0
[PATCH] btrfs-progs: convert: Workaround delayed ref bug by limiting the size of a transaction
In convert we use trans->block_reserved >= 4096 as a threshold to commit
transaction, where block_reserved is the number of new tree blocks
allocated inside a transaction.
The problem is, we still have a hidden bug in delayed ref implementation
in btrfs-progs, when we have a large enough transaction, delayed ref may
failed to find certain tree blocks in extent tree and cause transaction
abort.
This workaround will workaround it by committing transaction at a much
lower threshold.
The old 4096 means 4096 new tree blocks, when using default (16K)
nodesize, it's 64M, which can contain over 12k inlined data extent or
csum for around 60G, or over 800K file extents.
The new threshold will limit the size of new tree blocks to 2M, aligning
with the chunk preallocator threshold, and reducing the possibility to
hit that delayed ref bug.
Signed-off-by: Qu Wenruo
---
convert/source-ext2.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/convert/source-ext2.c b/convert/source-ext2.c
index a136e5652898..63cf71fe10d5 100644
--- a/convert/source-ext2.c
+++ b/convert/source-ext2.c
@@ -829,7 +829,18 @@ static int ext2_copy_inodes(struct btrfs_convert_context
*cctx,
pthread_mutex_unlock(&p->mutex);
if (ret)
return ret;
- if (trans->blocks_used >= 4096) {
+ /*
+* blocks_used is the number of new tree blocks allocated in
+* current transaction.
+* Use a small amount of it to workaround a bug where delayed
+* ref may fail to locate tree blocks in extent tree.
+*
+* 2M is the threshold to kick chunk preallocator into work,
+* For default (16K) nodesize it will be 128 tree blocks,
+* large enough to contain over 300 inlined files or
+* around 26k file extents. Which should be good enough.
+*/
+ if (trans->blocks_used >= SZ_2M / root->fs_info->nodesize) {
ret = btrfs_commit_transaction(trans, root);
BUG_ON(ret);
trans = btrfs_start_transaction(root, 1);
--
2.21.0
