[PATCH] btrfs-progs: Fix the argument requirement for '--subvol-extents'

2014-10-30 Thread Qu Wenruo
The following commit changed the argument requirement for '--subvol-extents', which causes it to call arg_strtou64() on NULL, resulting a segfault. d34cbe76 btrfs-progs: check: do not require argument for --subvol-extents This patch revert the patch and change the help string and man page to make

Re: Btrfs raid1 array has issues with rtorrent usage pattern.

2014-10-30 Thread Koen Kooi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan Merillat schreef op 30-10-14 04:17: It's specifically BTRFS related, I was able to reproduce it on a bare drive (no lvm, no md, no bcache). It's not bad RAM, I was able to reproduce it on multiple machines running either 3.17 or late RCs.

[PATCH] btrfs: Fix a lockdep warning when running xfstest.

2014-10-30 Thread Qu Wenruo
The following lockdep warning is triggered during xfstests: [ 1702.980872] = [ 1702.981181] [ INFO: possible irq lock inversion dependency detected ] [ 1702.981482] 3.18.0-rc1 #27 Not tainted [ 1702.981781]

btrfs deduplication and linux cache management

2014-10-30 Thread luvar
Hi, I want to ask, if deduplicated file content will be cached in linux kernel just once for two deduplicated files. To explain in deep: - I use btrfs for whole system with few subvolumes with some compression on some subvolumes. - I have two directories with eclipse SDK with slightly

Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation.

2014-10-30 Thread Liu Bo
On Thu, Oct 30, 2014 at 11:46:18AM +0800, Qu Wenruo wrote: Original Message Subject: Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation. From: Qu Wenruo quwen...@cn.fujitsu.com To: bo.li@oracle.com

Re: v3.18-rc2 at a 32 bit KVM gives :INFO: trying to register non-static key.the code is fine but needs lockdep annotation.

2014-10-30 Thread Liu Bo
On Wed, Oct 29, 2014 at 05:56:33PM +0100, Toralf Förster wrote: This is new in my eyes, or ? : Also new to me, could you please turn on lock debug and try again? thanks, -liubo Oct 29 17:53:04 n22kvmclone kernel: INFO: trying to register non-static key. Oct 29 17:53:04 n22kvmclone kernel:

Re: btrfs deduplication and linux cache management

2014-10-30 Thread Austin S Hemmelgarn
On 2014-10-30 05:26, lu...@plaintext.sk wrote: Hi, I want to ask, if deduplicated file content will be cached in linux kernel just once for two deduplicated files. To explain in deep: - I use btrfs for whole system with few subvolumes with some compression on some subvolumes. - I have two

Re: RAID1 fails to recover chunk tree

2014-10-30 Thread Zack Coffey
Rob, That second drive was immediately put to use elsewhere. I figured having only the metadata on that drive, it wouldn't matter. The data stayed single and wasn't part of the second drive, only the metadata was. I must not be capable of understanding why that wouldn't work. I thought all I

Re: [PATCH 2/2 v2] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-30 Thread David Sterba
On Thu, Oct 23, 2014 at 02:30:08PM +0800, Anand Jain wrote: introduce a global variable scan_done, which is set when scan is done succssfully per thread. So that following calls to this function will just return success. The rest is good, I'm a bit concerned about the global variable. It

Re: Performance Issues

2014-10-30 Thread Rob Spanton
Hi Everyone, I wrote: I have a particularly uncomplicated setup (a desktop PC with a hard disk) and I'm seeing particularly slow performance from btrfs. A `git status` in the linux source tree takes about 46 seconds after dropping caches, whereas on other machines using ext4 this takes about

Re: [PATCH 1/2] btrfs-progs: introduce btrfs_register_all_device()

2014-10-30 Thread David Sterba
On Wed, Oct 15, 2014 at 08:51:09AM +0800, Anand Jain wrote: --- a/volumes.c +++ b/volumes.c @@ -30,6 +30,7 @@ #include print-tree.h #include volumes.h #include math.h +#include utils.h struct stripe { struct btrfs_device *dev; @@ -1533,6 +1534,30 @@

Re: [PATCH] btrfs-progs: Fix number of arguments check of 'btrfs fi df'

2014-10-30 Thread David Sterba
On Tue, Oct 21, 2014 at 03:42:13PM +0800, Qu Wenruo wrote: 'btrfs fi df' needs exactly one arguments as mount option, and due to the introduce of human readable options, some check is not valid now, the new optind can point to the ending NULL string. For example, you can run 'btrfs fi df'

Re: RAID1 fails to recover chunk tree

2014-10-30 Thread Zygo Blaxell
On Thu, Oct 30, 2014 at 09:30:46AM -0400, Zack Coffey wrote: Rob, That second drive was immediately put to use elsewhere. I figured having only the metadata on that drive, it wouldn't matter. The data stayed single and wasn't part of the second drive, only the metadata was. I must not be

Re: btrfs deduplication and linux cache management

2014-10-30 Thread Zygo Blaxell
On Thu, Oct 30, 2014 at 10:26:07AM +0100, lu...@plaintext.sk wrote: Hi, I want to ask, if deduplicated file content will be cached in linux kernel just once for two deduplicated files. To explain in deep: - I use btrfs for whole system with few subvolumes with some compression on some

Re: [PATCH] btrfs-progs: Fix the argument requirement for '--subvol-extents'

2014-10-30 Thread David Sterba
On Thu, Oct 30, 2014 at 02:34:06PM +0800, Qu Wenruo wrote: The following commit changed the argument requirement for '--subvol-extents', which causes it to call arg_strtou64() on NULL, resulting a segfault. d34cbe76 btrfs-progs: check: do not require argument for --subvol-extents That was me,

Re: read block failed check_tree_block / Couldn't read chunk tree

2014-10-30 Thread Rene Thomas
With commit patched out. All devices in Place uname -a Linux engelserver 3.18.0-031800rc1-generic #201410192135 SMP Mon Oct 20 01:37:04 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux root@engelserver:/# btrfs --version Btrfs v3.17-dirty root@engelserver:/# btrfs fi show Label: 'mythstorage' uuid:

[PATCH] btrfs-progs: fix linking with libbtrfs

2014-10-30 Thread David Sterba
Reported at https://github.com/openSUSE/snapper/issues/128 Commit cdb9e22e292275237c added another rbtree file that defines functions that libbtrfs uses. Signed-off-by: David Sterba dste...@suse.cz --- Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile

Re: RAID1 fails to recover chunk tree

2014-10-30 Thread Chris Murphy
On Oct 30, 2014, at 7:30 AM, Zack Coffey tech42.click...@gmail.com wrote: Rob, That second drive was immediately put to use elsewhere. I figured having only the metadata on that drive, it wouldn't matter. The data stayed single and wasn't part of the second drive, only the metadata was. I

Quota question

2014-10-30 Thread Cyril Scetbon
Hi, At https://btrfs.wiki.kernel.org/index.php/Quota_support it's said that Using btrfs subvolume delete will break qgroup unshared space usage. Does it mean that if the qgroup associated with the subvolume is attached to another qgroup, this other qgroup won't be able to correctly apply its

[PATCH] Btrfs: move read only block groups onto their own list

2014-10-30 Thread Josef Bacik
Our gluster boxes were spending lots of time in statfs because our fs'es are huge. The problem is statfs loops through all of the block groups looking for read only block groups, and when you have several terabytes worth of data that ends up being a lot of block groups. Move the read only block

Quota question

2014-10-30 Thread Cyril Scetbon
Hi, At https://btrfs.wiki.kernel.org/index.php/Quota_support it's said that Using btrfs subvolume delete will break qgroup unshared space usage. Does it mean that if the qgroup associated with the subvolume is attached to another qgroup, this other qgroup won't be able to correctly apply its

3.17.2 kernel patches for btrfs

2014-10-30 Thread Tim Cuthbertson
Does the new Linux 3.17.2 kernel have sufficient patches to safely use btrfs with subvolume snapshots and incremental send/receive? I ask because 3.17.1 was released just before the btrfs patches for these issues, so I have been waiting for 3.17.2. However, from reading the kernel changelog, it

Re: v3.18-rc2 at a 32 bit KVM gives :INFO: trying to register non-static key.the code is fine but needs lockdep annotation.

2014-10-30 Thread Toralf Förster
On 10/30/2014 11:15 AM, Liu Bo wrote: On Wed, Oct 29, 2014 at 05:56:33PM +0100, Toralf Förster wrote: This is new in my eyes, or ? : Also new to me, could you please turn on lock debug and try again? thanks, -liubo you mean CONFIG_PROVE_LOCKING=y right ? -- Toralf pgp key: 0076 E94E --

fstrim on BTRFS

2014-10-30 Thread Noah Massey
Hello, I am looking for some clarification on TRIM / SSD maintenance. The wiki [1] suggests periodic fstrim, but fstrim does not discard unallocated blocks[2]. Which makes sense, given that mkfs issues a device wide trim, so they shouldn't have data. But it seems like both a balance, and a

filesystem corruption

2014-10-30 Thread Tobias Holst
Hi I was using a btrfs RAID1 with two disks under Ubuntu 14.04, kernel 3.13 and btrfs-tools 3.14.1 for weeks without issues. Now I updated to kernel 3.17.1 and btrfs-tools 3.17. After a reboot everything looked fine and I started some tests. While running duperemover (just scanning, not doing

Re: filesystem corruption

2014-10-30 Thread Tobias Holst
Addition: I found some posts here about a general file system corruption in 3.17 and 3.17.1 - is this the cause? Additionally I am using ro-snapshots - maybe this is the cause, too? Anyway: Can I fix that or do I have to reinstall? Haven't touched the filesystem, just did a scrub (found 0

Re: RAID1 fails to recover chunk tree

2014-10-30 Thread Duncan
Chris Murphy posted on Thu, 30 Oct 2014 12:04:48 -0600 as excerpted: Rob, That second drive was immediately put to use elsewhere. I figured having only the metadata on that drive, it wouldn't matter. The data stayed single and wasn't part of the second drive, only the metadata was. I must not

Re: v3.18-rc2 at a 32 bit KVM gives :INFO: trying to register non-static key.the code is fine but needs lockdep annotation.

2014-10-30 Thread Liu Bo
On Fri, Oct 31, 2014 at 12:33:43AM +0100, Toralf Förster wrote: On 10/30/2014 11:15 AM, Liu Bo wrote: On Wed, Oct 29, 2014 at 05:56:33PM +0100, Toralf Förster wrote: This is new in my eyes, or ? : Also new to me, could you please turn on lock debug and try again? thanks, -liubo you

[PATCH 2/2 v3] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-30 Thread Anand Jain
btrfs_scan_lblikd() is called by most the device related command functions. And btrfs_scan_lblkid() is most expensive function and it becomes more expensive as number of devices in the system increase. Further some threads call this function more than once for absolutely no extra benefit and the

Re: [PATCH 2/2 v2] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-30 Thread Anand Jain
Hi David, Thanks for the review.. On 30/10/2014 22:18, David Sterba wrote: On Thu, Oct 23, 2014 at 02:30:08PM +0800, Anand Jain wrote: introduce a global variable scan_done, which is set when scan is done succssfully per thread. So that following calls to this function will just

Re: RAID1 fails to recover chunk tree

2014-10-30 Thread Chris Murphy
On Oct 30, 2014, at 7:27 PM, Duncan 1i5t5.dun...@cox.net wrote: Chris Murphy posted on Thu, 30 Oct 2014 12:04:48 -0600 as excerpted: Rob, That second drive was immediately put to use elsewhere. I figured having only the metadata on that drive, it wouldn't matter. The data stayed single and

[PATCH 2/2 v4] btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls

2014-10-30 Thread Anand Jain
btrfs_scan_lblikd() is called by most the device related command functions. And btrfs_scan_lblkid() is most expensive function and it becomes more expensive as number of devices in the system increase. Further some threads call this function more than once for absolutely no extra benefit and the

[PATCH 1/2 v4] btrfs-progs: introduce btrfs_register_all_device()

2014-10-30 Thread Anand Jain
This function is to register all devices found after scanning the system. Before we had this functionality with in the btrfs_scan_lblkid(), however scanning and registering are two different distinct operation its better keep them separate. Also we want to optimize btrfs_scan_lblkid and avoid

Re: [PATCH 1/2] btrfs-progs: introduce btrfs_register_all_device()

2014-10-30 Thread Anand Jain
On 30/10/2014 22:33, David Sterba wrote: On Wed, Oct 15, 2014 at 08:51:09AM +0800, Anand Jain wrote: --- a/volumes.c +++ b/volumes.c @@ -30,6 +30,7 @@ #include print-tree.h #include volumes.h #include math.h +#include utils.h struct stripe { struct btrfs_device *dev; @@

Re: read block failed check_tree_block / Couldn't read chunk tree

2014-10-30 Thread Anand Jain
There might be more than one issue. I was trying solve errors Check tree block failed, want=5845480062976, have=0 when 'btrfs fi show' was run. Thanks for confirming patchout helped to fix that. On 31/10/2014 00:47, Rene Thomas wrote: With commit patched out. All devices in Place uname

Re: filesystem corruption

2014-10-30 Thread Rich Freeman
On Thu, Oct 30, 2014 at 9:02 PM, Tobias Holst to...@tobby.eu wrote: Addition: I found some posts here about a general file system corruption in 3.17 and 3.17.1 - is this the cause? Additionally I am using ro-snapshots - maybe this is the cause, too? Anyway: Can I fix that or do I have to

Re: read block failed check_tree_block / Couldn't read chunk tree

2014-10-30 Thread Qu Wenruo
Original Message Subject: Re: read block failed check_tree_block / Couldn't read chunk tree From: Rene Thomas re.tho...@gmx.de To: Anand Jain anand.j...@oracle.com Date: 2014年10月31日 00:47 With commit patched out. All devices in Place uname -a Linux engelserver

Re: read block failed check_tree_block / Couldn't read chunk tree

2014-10-30 Thread Gui Hecheng
On Wed, 2014-10-29 at 11:45 +0800, Anand Jain wrote: this is (most likely) due to patch below, commit 915902c5002485fb13d27c4b699a73fb66cc0f09 btrfs-progs: fix device missing of btrfs fi show with seed devices Could you try to back out the patch