Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Juan Orti Alcaine
El 2014-10-14 18:54, Robert White escribió: Howdy, So I run several gentoo systems and I upgraded two of them to kernel 3.17.0 One using BTRFS for root. One using ext3 for root (via the ext4 driver) _Both_ systems exhibited strange behavior (long pauses and then hangs requiring hard-power)

[PATCH 1/1] btrfs-progs: mkfs should be consistent in calling register device

2014-10-15 Thread Anand Jain
When we have one device we don't call register device. (in fact not mandatory, but to make it consistent) And when we have more than one we call register device. reproducer: Nothing in the kernel device list cat /proc/fs/btrfs/devlist | egrep fsid | wc -l 0 mkfs.btrfs will automatically call

[PATCH 1/1] btrfs-progs: introduce a proper structure on which cli will call register-device ioctl

2014-10-15 Thread Anand Jain
As of now commands mentioned below (with in [..]) are calling call register-device ioctl BTRFS_IOC_SCAN_DEV for all the devices in the system. Some issues with it: BTRFS_IOC_SCAN_DEV: ioctl is a write operation, we don't want command like btrfs-debug-tree threads to do that.. eg:

[PATCH 1/1] btrfs-progs: open RW to register device using btrfs-control

2014-10-15 Thread Anand Jain
We are passing device path to be registered with in kernel, so we need to open with RW Signed-off-by: Anand Jain anand.j...@oracle.com --- utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils.c b/utils.c index dd4ee52..78cb93f 100644 --- a/utils.c +++ b/utils.c @@

[PATCH 1/1] btrfs-progs: code optimize cmd_scan_dev() use btrfs_register_one_device()

2014-10-15 Thread Anand Jain
cmd_scan_dev() has it own code to register device (calling ioctl BTRFS_IOC_SCAN_DEV), apparently it could use btrfs_register_one_device(). Signed-off-by: Anand Jain anand.j...@oracle.com --- cmds-device.c | 33 ++--- utils.c | 8 +--- utils.h | 2 +-

[PATCH 2/2] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-15 Thread Anand Jain
btrfs_scan_lblikd is called by most the device related command functions. And btrfs_scan_lblkid is most expensive function and it becomes more expensive as number of devices in the system increase. This patch will: move out calling register_one_device with in btrfs_scan_lblkid() and so

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Duncan
Juan Orti Alcaine posted on Wed, 15 Oct 2014 09:08:14 +0200 as excerpted: I've also experienced Btrfs corruptions with 3.17.0 I do readonly snapshots every hour of all the subvolumes, so I have hundreds of snapshots. That's a known issue with read-only snapshots in 3.17.0. There's quite a

Re: Wishlist Item :: One Subvol in Multiple Places

2014-10-15 Thread Austin S Hemmelgarn
On 2014-10-14 18:25, Robert White wrote: I've got no idea if this is possible given the current storage layout, but it would be Really Nice™ if there were a way to have a single subvolume exist in more than one place in hirearchy. I know this can be faked via mount tricks (bind or use of

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Josef Bacik
On 10/15/2014 03:08 AM, Juan Orti Alcaine wrote: El 2014-10-14 18:54, Robert White escribió: Howdy, So I run several gentoo systems and I upgraded two of them to kernel 3.17.0 One using BTRFS for root. One using ext3 for root (via the ext4 driver) _Both_ systems exhibited strange behavior

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Juan Orti Alcaine
El 2014-10-15 15:46, Josef Bacik escribió: On 10/15/2014 03:08 AM, Juan Orti Alcaine wrote: I've also experienced Btrfs corruptions with 3.17.0 (Fedora 21 alpha). It has happened two times, each one after a clean reinstall and a wipe of the old fs. In less than a day, both installations got

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Josef Bacik
On 10/15/2014 10:05 AM, Juan Orti Alcaine wrote: El 2014-10-15 15:46, Josef Bacik escribió: On 10/15/2014 03:08 AM, Juan Orti Alcaine wrote: I've also experienced Btrfs corruptions with 3.17.0 (Fedora 21 alpha). It has happened two times, each one after a clean reinstall and a wipe of the old

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Juan Orti Alcaine
El 2014-10-15 16:30, Josef Bacik escribió: On 10/15/2014 10:05 AM, Juan Orti Alcaine wrote: El 2014-10-15 15:46, Josef Bacik escribió: On 10/15/2014 03:08 AM, Juan Orti Alcaine wrote: I've also experienced Btrfs corruptions with 3.17.0 (Fedora 21 alpha). It has happened two times, each one

RAID1 fails to recover chunk tree

2014-10-15 Thread Zack Coffey
Revisit of a previous issue. Summary, single drive btrfs has lots of data. Made a RAID1 with another drive of just the metadata. Was in that state for less than 12 hours-ish, removed the second drive and now cannot get to any data on the original drive. Single drive btrfs was made on Ubuntu with

Re: [RFC 1/1 linux-next] btrfs: don't opencode zero_user_segment

2014-10-15 Thread Zach Brown
On Tue, Oct 14, 2014 at 10:18:09PM +0200, Fabian Frederick wrote: On 14 October 2014 at 21:15 Zach Brown z...@zabbo.net wrote: On Tue, Oct 14, 2014 at 07:46:14PM +0200, Fabian Frederick wrote: use function defined in include/linux/highmem.h Note that this reverts 2 last function

[PATCH] Revert Btrfs: race free update of commit root for ro snapshots

2014-10-15 Thread Filipe Manana
This reverts commit 9c3b306e1c9e6be4be09e99a8fe2227d1005effc. Switching only one commit root during a transaction is wrong because it leads the fs into an inconsistent state. All commit roots should be switched at once, at transaction commit time, otherwise backref walking can often miss

[PATCH] fstests: btrfs: regression test for ro snapshot creation

2014-10-15 Thread Filipe Manana
Regression test for a btrfs issue where creation of readonly snapshots caused the filesystem to get into an inconsistent state. This regression was introduced in the 3.17 kernel and fixed by reverting the following linux kernel commit: Btrfs: race free update of commit root for ro snapshots

Re: [PATCH] fstests: btrfs: regression test for ro snapshot creation

2014-10-15 Thread Josef Bacik
On 10/15/2014 02:37 PM, Filipe Manana wrote: Regression test for a btrfs issue where creation of readonly snapshots caused the filesystem to get into an inconsistent state. This regression was introduced in the 3.17 kernel and fixed by reverting the following linux kernel commit: Btrfs:

Re: Wishlist Item :: One Subvol in Multiple Places

2014-10-15 Thread Robert White
On 10/15/2014 04:30 AM, Austin S Hemmelgarn wrote: On 2014-10-14 18:25, Robert White wrote: I've got no idea if this is possible given the current storage layout, but it would be Really Nice™ if there were a way to have a single subvolume exist in more than one place in hirearchy. I know this

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Rich Freeman
On Wed, Oct 15, 2014 at 10:30 AM, Josef Bacik jba...@fb.com wrote: We've found it, the Fedora guys are reverting the bad patch now, we'll get the fix sent back to stable shortly. Sorry about that. After reverting this commit, can the bad snapshots be deleted/repaired/etc without wiping and

Re: Random file system corruption in 3.17 (not BTRFS related...?)

2014-10-15 Thread Josef Bacik
On 10/15/2014 03:30 PM, Rich Freeman wrote: On Wed, Oct 15, 2014 at 10:30 AM, Josef Bacik jba...@fb.com wrote: We've found it, the Fedora guys are reverting the bad patch now, we'll get the fix sent back to stable shortly. Sorry about that. After reverting this commit, can the bad snapshots

[PATCH] Btrfs-progs: check, fix return value check of is_child_root()

2014-10-15 Thread Filipe Manana
The following commit: btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure f495a2ac66116f0a1b15e73380c8cbca6e0a4ca0 introduced a regression, detected through xfstests/btrfs/054, where previously a negative return value (-1) was used to mean a particular root didn't

Re: [PATCH] Btrfs-progs: check, fix return value check of is_child_root()

2014-10-15 Thread David Sterba
On Wed, Oct 15, 2014 at 10:58:40PM +0100, Filipe Manana wrote: This affects only the 3.17 release candidates (3.16 and older releases don't have this issue). Thank you, I'm adding this to the branch and tagging rc4. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the

RAID1 fails to recover chunk tree

2014-10-15 Thread Zack Coffey
Revisit of a previous issue. Summary, single drive btrfs has lots of data. Made a RAID1 with another drive of just the metadata. Was in that state for less than 12 hours-ish, removed the second drive and now cannot get to any data on the original drive. Single drive btrfs was made on Ubuntu with

Re: [PATCH] Revert Btrfs: race free update of commit root for ro snapshots

2014-10-15 Thread Chris Mason
On Wed, Oct 15, 2014 at 2:37 PM, Filipe Manana fdman...@gmail.com wrote: This reverts commit 9c3b306e1c9e6be4be09e99a8fe2227d1005effc. Switching only one commit root during a transaction is wrong because it leads the fs into an inconsistent state. All commit roots should be switched at

[PATCH] Btrfs: properly clean up btrfs_end_io_wq_cache

2014-10-15 Thread Josef Bacik
In one of Dave's cleanup commits he forgot to call btrfs_end_io_wq_exit on unload, which makes us unable to unload and then re-load the btrfs module. This fixes the problem. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/super.c | 1 + 1 file changed, 1 insertion(+) diff --git

Re: [PATCH] Revert Btrfs: race free update of commit root for ro snapshots

2014-10-15 Thread john terragon
Hi. I applied the patch to 3.17.1 but although I haven't seen any corrupted ro snapshot yet it's still impossible to do btrfs send. As soon as I start btrfs send I still get ERROR: send ioctl failed with -12: Cannot allocate memory even if I redirect btrfs send's output to a file (instead of

Re: [PATCH] Revert Btrfs: race free update of commit root for ro snapshots

2014-10-15 Thread Filipe David Manana
On Wed, Oct 15, 2014 at 11:42 PM, john terragon jterra...@gmail.com wrote: Hi. I applied the patch to 3.17.1 but although I haven't seen any corrupted ro snapshot yet it's still impossible to do btrfs send. As soon as I start btrfs send I still get ERROR: send ioctl failed with -12: Cannot

[PATCH 1/4] btrfs-progs: check sscanf return code

2014-10-15 Thread Zach Brown
coverity warned that the return code from sscanf() assigned to 'i' wasn't checked before being assigned again. Check it. Signed-off-by: Zach Brown z...@zabbo.net --- utils.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/utils.c b/utils.c index c2f30d4..51e55be 100644

[PATCH 0/4] some btrfs-progs coverity fixes

2014-10-15 Thread Zach Brown
Hi gang, Here's another set of coverity fixes for btrfs-progs against David's integration-20141007 branch. I got tired of adding error checking after a few so I moved on to the other warnings. Maybe we should subscribe linux-btrfs to the reports that coverity can send out? - z -- To

[PATCH 4/4] btrfs-progs: fix csum root copy-n-paste error

2014-10-15 Thread Zach Brown
btrfs_setup_all_roots() had some copy and pasted code for trying to setup a root and then creating a blank node if that failed. The copy for the csum_root created the blank node in the extent_root. So we create a function to use a consistent root. Signed-off-by: Zach Brown z...@zabbo.net ---

[PATCH 2/4] btrfs-progs: check read extent errors when mapping

2014-10-15 Thread Zach Brown
coverity barked out a warning that btrfs-map-logical was storing but ignoring errors from read_extent_from_disk(). So don't ignore 'em. I made extent reading errors fatal to match the fatal errors from mapping mirrors above. And while we're at it have read_extent_from_disk() return -errno

[PATCH 3/4] btrfs-progs: fix show super unknown flag output

2014-10-15 Thread Zach Brown
coverity pointed out that unknown flag printing in show super had some dead code. It turns out that first was reset when the first flag was tested, not when it was output. We only want to clear it if the first matching bit is output. If there are no matching bits then we'll want to output the

[PATCH v2] Btrfs-progs: check, fix return value check of is_child_root()

2014-10-15 Thread Filipe Manana
The following commit: btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure f495a2ac66116f0a1b15e73380c8cbca6e0a4ca0 introduced a regression, detected through xfstests/btrfs/054, where previously a negative return value (-1) was used to mean a particular root didn't

Re: [PATCH] Revert Btrfs: race free update of commit root for ro snapshots

2014-10-15 Thread john terragon
-It's not a brand new fs. It has been created four or five days ago with btrfs-progs 3.16.2 (in fact it was created because of the dead unremovable ro snapshots in the previous fs) -the snapshot in question has been created after applying the patch (and it has not become corrupted so far) -not

Re: [PATCH 1/4] btrfs: correct empty compression property behavior

2014-10-15 Thread Satoru Takeuchi
Hi David and Chris, (2014/09/29 18:36), David Sterba wrote: On Fri, Sep 19, 2014 at 05:45:49PM +0900, Satoru Takeuchi wrote: In the current implementation, compression property == has the two different meanings: one is with BTRFS_INODE_NOCOMPRESS, and the other is without this flag. So, even

[PATCH] btrfs: regression test for btrfs extent merge

2014-10-15 Thread Qu Wenruo
The following kernel commit introduced the bug: 51f395ad btrfs: Use right extent length when inserting overlap extent map. When btrfs commit race with btrfs_get_extent(), merge_extent_mapping() may build up a new extent which length overflows and cause extent map insert fail, causing the caller

[PATCH] btrfs: fix wrong pad for kernelspace arg of btrfs dev stat

2014-10-15 Thread Gui Hecheng
The @btrfs_ioctl_get_dev_stats fails to pad to 1k as descripted, actually it valuates to 1032 bytes. The corresponding userspace change follows this change. Signed-off-by: Gui Hecheng guihc.f...@cn.fujitsu.com --- include/uapi/linux/btrfs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

[PATCH] btrfs-progs: fix wrong pad for userspace arg of btrfs dev stat

2014-10-15 Thread Gui Hecheng
The @btrfs_ioctl_get_dev_stats fails to pad to 1k as descripted, actually it valuates to 1032 bytes. Correct it following the kernel patch. Signed-off-by: Gui Hecheng guihc.f...@cn.fujitsu.com --- ioctl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ioctl.h b/ioctl.h index

[PATCH] btrfs-progs: prevent silent damage when add dev to an invalid mntpnt

2014-10-15 Thread Gui Hecheng
Problem: # mkfs.btrfs -f /dev/sda1 # btrfs dev add /dev/sda1 /dir -f == dir is not a mntpnt btrfs dev add just report invalid ioctl but it has already made changes to /dev/sda1 with @btrfs_prepare_device(), so the fs on /dev/sda1 is damaged. We could check whether /dev/sda1 is