Re: [PATCH] Btrfs: return failure if btrfs_dev_replace_finishing() failed

2014-10-10 Thread Eryu Guan
On Thu, Sep 25, 2014 at 06:28:14PM +0800, Eryu Guan wrote: device replace could fail due to another running scrub process, but this failure doesn't get returned to userspace. The following steps could reproduce this issue mkfs -t btrfs -f /dev/sdb1 /dev/sdb2 mount /dev/sdb1

Re: [PATCH v2] btrfs-progs: force overwrite should wipe stale SB

2014-10-10 Thread David Sterba
On Thu, Oct 02, 2014 at 07:22:09AM +0800, Anand Jain wrote: (I am unable to reproduce the issue, tried to go back with progs versions but still the same. So as of now this code remains untested, suggest to wait till we have a reproducible test case). Here is a test case which says it all..

Re: [PATCH v2] btrfs-progs: force overwrite should wipe stale SB

2014-10-10 Thread Anand Jain
Thanks. Anand On 10/10/2014 03:39 PM, David Sterba wrote: On Thu, Oct 02, 2014 at 07:22:09AM +0800, Anand Jain wrote: (I am unable to reproduce the issue, tried to go back with progs versions but still the same. So as of now this code remains untested, suggest to wait till we have a

[PATCH 2/2 v2] Btrfs: add helper btrfs_fdatawrite_range

2014-10-10 Thread Filipe Manana
To avoid duplicating this double filemap_fdatawrite_range() call for inodes with async extents (compressed writes) so often. Signed-off-by: Filipe Manana fdman...@suse.com --- V2: Pass right arguments to the new helper. Missed unstaged changes. fs/btrfs/ctree.h| 1 + fs/btrfs/file.c

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-10-10 Thread Filipe David Manana
On Fri, Oct 10, 2014 at 3:39 AM, Qu Wenruo quwen...@cn.fujitsu.com wrote: Original Message Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map From: Filipe David Manana fdman...@gmail.com To: Qu Wenruo quwen...@cn.fujitsu.com

Re: [PATCH 1/2] btrfs-progs: don't report internal dev replace result if ioctl failed

2014-10-10 Thread David Sterba
On Wed, Oct 08, 2014 at 05:42:28PM +0800, Eryu Guan wrote: If BTRFS_IOC_DEV_REPLACE ioctl failed, there's no result returned to fill args.result, it doesn't make sense to report this internal result to user. And the arg has been initialized with 0, the result is always 0, which is

Re: [PATCH] Btrfs: return failure if btrfs_dev_replace_finishing() failed

2014-10-10 Thread Miao Xie
On Fri, 10 Oct 2014 15:13:31 +0800, Eryu Guan wrote: On Thu, Sep 25, 2014 at 06:28:14PM +0800, Eryu Guan wrote: device replace could fail due to another running scrub process, but this failure doesn't get returned to userspace. The following steps could reproduce this issue mkfs -t

Re: [PATCH] Btrfs-progs: repair missing dir index

2014-10-10 Thread David Sterba
On Thu, Oct 02, 2014 at 02:50:39PM -0400, Josef Bacik wrote: +static int repair_inode_backrefs(struct btrfs_root *root, + struct inode_record *rec, + struct cache_tree *inode_cache) +{ + struct btrfs_trans_handle *trans; +

[PATCH 1/2 v2] Btrfs: report error after failure inlining extent in compressed write path

2014-10-10 Thread Filipe Manana
If cow_file_range_inline() failed, when called from compress_file_range(), we were tagging the locked page for writeback, end its writeback and unlock it, but not marking it with an error nor setting AS_EIO in inode's mapping flags. This made it impossible for a caller of filemap_fdatawrite_range

Re: [PATCH] Btrfs-progs: repair missing dir index

2014-10-10 Thread Josef Bacik
I'm redoing this patch a bit so don't take it yet. Thanks, Josef David Sterba dste...@suse.cz wrote: On Thu, Oct 02, 2014 at 02:50:39PM -0400, Josef Bacik wrote: +static int repair_inode_backrefs(struct btrfs_root *root, + struct inode_record *rec, +

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Bob Marley
On 10/10/2014 03:58, Chris Murphy wrote: * mount -o recovery Enable autorecovery attempts if a bad tree root is found at mount time. I'm confused why it's not the default yet. Maybe it's continuing to evolve at a pace that suggests something could sneak in that makes things worse?

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Roman Mamedov
On Fri, 10 Oct 2014 12:53:38 +0200 Bob Marley bobmar...@shiftmail.org wrote: On 10/10/2014 03:58, Chris Murphy wrote: * mount -o recovery Enable autorecovery attempts if a bad tree root is found at mount time. I'm confused why it's not the default yet. Maybe it's continuing to

Re: [PATCH 1/2] btrfs-progs: don't report internal dev replace result if ioctl failed

2014-10-10 Thread Eryu Guan
On Fri, Oct 10, 2014 at 10:20:23AM +0200, David Sterba wrote: On Wed, Oct 08, 2014 at 05:42:28PM +0800, Eryu Guan wrote: If BTRFS_IOC_DEV_REPLACE ioctl failed, there's no result returned to fill args.result, it doesn't make sense to report this internal result to user. And the arg has

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Bob Marley
On 10/10/2014 12:59, Roman Mamedov wrote: On Fri, 10 Oct 2014 12:53:38 +0200 Bob Marley bobmar...@shiftmail.org wrote: On 10/10/2014 03:58, Chris Murphy wrote: * mount -o recovery Enable autorecovery attempts if a bad tree root is found at mount time. I'm confused why it's not the

Re: Confusion with newly converted filesystem

2014-10-10 Thread Tim Cuthbertson
Thank you very much, Duncan and Chris. Especially to Duncan, for his detailed reply and further suggestions. I will try to follow your advice as best I can. I am old, but I'm still learning! Tim -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

[PATCH v2 1/2] btrfs-progs: only report internal dev replace result if there's a result

2014-10-10 Thread Eryu Guan
If BTRFS_IOC_DEV_REPLACE ioctl failed, args.result usually won't be updated by the ioctl. And the arg has been initialized with 0, the result is always 0, which is BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR, and the resulting error message looks confusing: ERROR: ioctl(DEV_REPLACE_START) failed on

Problems running Balance and checking integrity concurrently.

2014-10-10 Thread Wang Shilong
Hello, I have reproduced a problem with Btrfs integrity and balance using latest btrfs kernel,it is very easy to reproduce: With mount option “check_int”, and run xfstests/btrfs/ tests, below test could definitely reproduce problem: ./check tests/btrfs/014 5.122248] Written block @25800343552

[PATCH] Btrfs: fix allocationg memory failure for btrfsic_state structure

2014-10-10 Thread Wang Shilong
size of @btrfsic_state needs more than 2M, it is very likely to fail allocating memory using kzalloc(). see following mesage: [91428.902148] Call Trace: [816f6e0f] dump_stack+0x4d/0x66 [811b1c7f] warn_alloc_failed+0xff/0x170 [811b66e1] __alloc_pages_nodemask+0x951/0xc30

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Chris Murphy
On Oct 10, 2014, at 6:53 AM, Bob Marley bobmar...@shiftmail.org wrote: On 10/10/2014 03:58, Chris Murphy wrote: * mount -o recovery Enable autorecovery attempts if a bad tree root is found at mount time. I'm confused why it's not the default yet. Maybe it's continuing to evolve at

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread cwillu
If -o recovery is necessary, then you're either running into a btrfs bug, or your hardware is lying about when it has actually written things to disk. The first case isn't unheard of, although far less common than it used to be, and it should continue to improve with time. In the second case,

[PATCH] btrfs-progs: debug: print more info about inode

2014-10-10 Thread David Sterba
Add uid, gid, rdev and flags to btrfs_print_leaf. Signed-off-by: David Sterba dste...@suse.cz --- print-tree.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/print-tree.c b/print-tree.c index 7df5798e539c..70a7acc632f2 100644 --- a/print-tree.c +++ b/print-tree.c @@

[PATCH] btrfs-progs: document the limit balance filter

2014-10-10 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz --- Documentation/btrfs-balance.txt | 5 + 1 file changed, 5 insertions(+) diff --git a/Documentation/btrfs-balance.txt b/Documentation/btrfs-balance.txt index 89fd44901a9d..e1ad6511571c 100644 --- a/Documentation/btrfs-balance.txt +++

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Bob Marley
On 10/10/2014 16:37, Chris Murphy wrote: The fail safe behavior is to treat the known good tree root as the default tree root, and bypass the bad tree root if it cannot be repaired, so that the volume can be mounted with default mount options (i.e. the ones in fstab). Otherwise it's a

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Bardur Arantsson
On 2014-10-10 19:43, Bob Marley wrote: On 10/10/2014 16:37, Chris Murphy wrote: The fail safe behavior is to treat the known good tree root as the default tree root, and bypass the bad tree root if it cannot be repaired, so that the volume can be mounted with default mount options (i.e. the

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Austin S Hemmelgarn
On 2014-10-10 13:43, Bob Marley wrote: On 10/10/2014 16:37, Chris Murphy wrote: The fail safe behavior is to treat the known good tree root as the default tree root, and bypass the bad tree root if it cannot be repaired, so that the volume can be mounted with default mount options (i.e. the

[PATCH 01/12] Btrfs-progs: repair missing dir index

2014-10-10 Thread Josef Bacik
If we have an inode backref entry then we know enough to add back a missing dir index. When messing with the inode backrefs we need to do all of that first before we process the inode recs themselves as we may clear errors on the inode recs as we fix the directory indexes. This adds the

Btrfs-progs: fix horrible things

2014-10-10 Thread Josef Bacik
This is the culmination of two weeks worth of work with Jaap Pieroen and his broekn file system. This pulls back a few things from the kernel in order to support the dir item stuff, and the rbtree stuff to fix a weird bug we were seeing. There will be more test images coming, but this is what I

[PATCH 08/12] Btrfs-progs: delete bogus dir indexes

2014-10-10 Thread Josef Bacik
We may run across dir indexes that are corrupt in such a way that it makes them useless, such as having a bad location key or a bad name. In this case we can just delete dir indexes that don't show up properly and then re-create what we need. When we delete dir indexes however we need to restart

[PATCH 05/12] Btrfs-progs: lookup all roots that point to a corrupt block

2014-10-10 Thread Josef Bacik
If we have a corrupt block that multiple snapshots point to we will only fix the guy who originally pointed to the block, and then simply loop forever because we keep finding the same bad block. So instead lookup all roots that point to this block, and then search down to the block for each root

[PATCH 06/12] Btrfs-progs: reset chunk state if we restart check

2014-10-10 Thread Josef Bacik
If we hid a corrupt block that we fix and we restart the fsck loop you will get lots of noise about duplicate block groups and such. This is because we don't clear the block group and chunk cache when we do this restart. This patch fixes that, which is a little tricky since the structs are

[PATCH 09/12] Btrfs-progs: add a dummy backref if our location is wrong

2014-10-10 Thread Josef Bacik
If our location is bogus in our dir item we were just skipping the thing. However in this case we want to just delete the dir index, so create a dummy inode rec using BTRFS_MULTIPLE_OBJECTIDS and just add every backref we find to the list so we know to straight up delete all of these items.

[PATCH 07/12] Btrfs-progs: re-search tree root if it changes

2014-10-10 Thread Josef Bacik
If we change something while scanning fs-roots we need to redo our search so that we get valid root items and have valid root cache. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- cmds-check.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/cmds-check.c

[PATCH 11/12] Btrfs-progs: add ability to corrupt dir items

2014-10-10 Thread Josef Bacik
In order to test the dir index corruption fixing patches in fsck we need to add functionality to btrfs-corrupt-block to corrupt dir item fields. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- btrfs-corrupt-block.c | 103 +- 1 file changed,

[PATCH 10/12] Btrfs-progs: deal with mismatch index between dir index and inode ref

2014-10-10 Thread Josef Bacik
Sometimes we have a dir index and an inode ref that don't agree on the index. In this case just assume that the inode ref is the ultimate authority on the subject and delete the dir index. This means we have to not reset index if we find a mismatched inode ref to make sure we delete the right dir

[PATCH 02/12] Btrfs-progs: pull back backref.c and fix it up

2014-10-10 Thread Josef Bacik
This patch pulls back backref.c, adds a couple of helpers everywhere that it needs, and cleans up backref.c to fit in btrfs-progs. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- Makefile |2 +- backref.c| 1651 ++

[PATCH 04/12] Btrfs-progs: update rbtree libs

2014-10-10 Thread Josef Bacik
While debugging a broken fs we were seeing hangs in the rb_erase loops. The rbtree was simple and wasn't corrupted so it appeared to be a bug in our rbtree library. Updating to the kernels latest rbtree code made the infinite loop go away, so pull it back. Thanks, Signed-off-by: Josef Bacik

[PATCH 03/12] Btrfs-progs: break out rbtree util functions

2014-10-10 Thread Josef Bacik
These were added to deal with duplicated functionality within btrfs-progs, but we specifically copied rbtree.c from the kernel, so move these functions out into their own file. This will make it easier to keep rbtree.c in sync. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- Makefile

Re: What is the vision for btrfs fs repair?

2014-10-10 Thread Eric Sandeen
On 10/10/14 2:35 PM, Austin S Hemmelgarn wrote: On 2014-10-10 13:43, Bob Marley wrote: On 10/10/2014 16:37, Chris Murphy wrote: The fail safe behavior is to treat the known good tree root as the default tree root, and bypass the bad tree root if it cannot be repaired, so that the volume can

[GIT PULL] Btrfs

2014-10-10 Thread Chris Mason
Hi Linus, Please pull my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This is based on 3.17-rc5 because that's when I forked off for -next, but I've been testing it against linux-next or 3.17 for a while now. The largest set of changes here