[PATCH] Btrfs-progs: btrfs-send: free used memory and close fds

2013-04-23 Thread Stefan Behrens
Not important at all since exit() is called afterwards and this is not part of the library. It just makes valgrind happy and thus allows to search for real flaws. Signed-off-by: Stefan Behrens --- cmds-send.c | 30 +++--- 1 file changed, 23 insertions(+), 7 deletions(-)

[PATCH] Btrfs-progs: in btrfs-debug-tree, print -1 in key for (u64)-1

2013-04-23 Thread Stefan Behrens
For the objectid and offset field of a key, print -1 instead of the decimal representation of 0x. At least for me it is more readable like this. Signed-off-by: Stefan Behrens --- print-tree.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/print-tree.c

[PATCH] Btrfs-progs: add function to map subvol ID to path

2013-04-23 Thread Stefan Behrens
Several tools like btrfs-send and btrfs-receive need to map a subvolume ID to a filesystem path. The so far existing methods in btrfs-list.c cause a horrible effort when performing this operation (and the effort is dependent on the number of existing subvolumes with quadratic effort). This commit a

[PATCH v3 3/3] Btrfs: automatic rescan after "quota enable" command

2013-04-23 Thread Jan Schmidt
When qgroup tracking is enabled, we do an automatic cycle of the new rescan mechanism. Signed-off-by: Jan Schmidt --- fs/btrfs/qgroup.c | 10 ++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 249dd64..b1ae0ab 100644 --- a/fs/

[PATCH v3 1/3] Btrfs: split btrfs_qgroup_account_ref into four functions

2013-04-23 Thread Jan Schmidt
The function is separated into a preparation part and the three accounting steps mentioned in the qgroups documentation. The goal is to make steps two and three usable by the rescan functionality. A side effect is that the function is restructured into readable subunits. Signed-off-by: Jan Schmidt

[PATCH v3 0/3] Btrfs: quota rescan for 3.10

2013-04-23 Thread Jan Schmidt
The kernel side for rescan, which is needed if you want to enable qgroup tracking on a non-empty volume. The first patch splits btrfs_qgroup_account_ref into readable ans reusable units. The second patch adds the rescan implementation (refer to its commit message for a description of the algorithm)

[PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Jan Schmidt
If qgroup tracking is out of sync, a rescan operation can be started. It iterates the complete extent tree and recalculates all qgroup tracking data. This is an expensive operation and should not be used unless required. A filesystem under rescan can still be umounted. The rescan continues on the

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Wang Shilong
Hello Jan, > If qgroup tracking is out of sync, a rescan operation can be started. It > iterates the complete extent tree and recalculates all qgroup tracking data. > This is an expensive operation and should not be used unless required. > > A filesystem under rescan can still be umounted. The re

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Wang Shilong
Hello Jan, [..snip..] > /* >* the delayed ref sequence number we pass depends on the direction of >* the operation. for add operations, we pass (node->seq - 1) to skip > @@ -1401,7 +1428,17 @@ int btrfs_qgroup_account_ref(struct btrfs_trans_handle > *trans, > if (re

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Jan Schmidt
On Tue, April 23, 2013 at 14:05 (+0200), Wang Shilong wrote: > Hello Jan, > > [..snip..] > > > >> /* >> * the delayed ref sequence number we pass depends on the direction of >> * the operation. for add operations, we pass (node->seq - 1) to skip >> @@ -1401,7 +1428,17 @@ int bt

[PATCH 0/3] speedup btrfs send/receive

2013-04-23 Thread Stefan Behrens
This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start fast and are independent on the number of subvolumes/snapshots that exist. Before this commit, mapping UUIDs to subvolu

[PATCH 2/3] Btrfs-progs: add uuid-tree lookup methods

2013-04-23 Thread Stefan Behrens
This commit adds uuid-tree lookup methods that make use of the search ioctl. The code is based on the kernel code. Signed-off-by: Stefan Behrens --- Makefile| 2 +- ctree.h | 5 ++ uuid-tree.c | 196 3 files changed, 202 in

[PATCH 3/3] Btrfs-progs: use UUID tree for send/receive

2013-04-23 Thread Stefan Behrens
This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start fast and are independent on the number of subvolules/snapshot that exist. Before this commit, mapping UUIDs to subvolum

[PATCH 1/3] Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

2013-04-23 Thread Stefan Behrens
Support printing these things. Signed-off-by: Stefan Behrens --- ctree.h | 29 +++ print-tree.c | 91 ++-- 2 files changed, 117 insertions(+), 3 deletions(-) diff --git a/ctree.h b/ctree.h index 4ea37ac..f25e2df 100644

Re: corrupted filesystem. mounts with -o recovery,ro but not -o recovery or -o ro

2013-04-23 Thread Jon Nelson
I need to know soon if there is going to be anything I can do to rescue this filesystem. I've tried 3.7.10, 3.8.[5,6,7,8] and btrfs-next as of ( bba653d1207646b17671c6cb9a0629736811848a ). btrfs-next - at least - merely failed the mount, all of the others failed but also ran into a BUG, requiring a

[PATCH] Btrfs: don't BUG_ON() in btrfs_num_copies

2013-04-23 Thread Josef Bacik
A user sent me a btrfs-image that was panicing because of some corruption. This is because we pass in a bogus value to btrfs_num_copies, and it panics. Instead just return 1. We only call btrfs_num_copies to see if there are other copies to try and read for things, so if we just return 1 it will

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Wang Shilong
Hello Jan, > > +static void btrfs_qgroup_rescan_worker(struct btrfs_work *work) > +{ > + struct qgroup_rescan *qscan = container_of(work, struct qgroup_rescan, > +work); > + struct btrfs_path *path; > + struct btrfs_trans_handle *trans

Re: corrupted filesystem. mounts with -o recovery,ro but not -o recovery or -o ro

2013-04-23 Thread Liu Bo
On Tue, Apr 23, 2013 at 09:20:45AM -0500, Jon Nelson wrote: > I need to know soon if there is going to be anything I can do to > rescue this filesystem. > I've tried 3.7.10, 3.8.[5,6,7,8] and btrfs-next as of ( > bba653d1207646b17671c6cb9a0629736811848a ). > btrfs-next - at least - merely failed th

[PATCH] Btrfs: don't try and free ebs twice in log replay

2013-04-23 Thread Josef Bacik
This work is done by btrfs_free_path() anyway so there's no need for this duplicate work. Thanks, Signed-off-by: Josef Bacik --- fs/btrfs/tree-log.c |7 --- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 52287ec..1fdc221 10

[PATCH] Btrfs: add tree block level sanity check

2013-04-23 Thread Josef Bacik
With a users corrupted fs I was getting weird behavior and panics and it turns out it was because one of his tree blocks had a bogus header level. So add this to the sanity checks in the endio handler for tree blocks. Thanks, Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c |6 ++ 1 f

Re: [PATCH v3 3/3] Btrfs: automatic rescan after "quota enable" command

2013-04-23 Thread David Sterba
On Tue, Apr 23, 2013 at 01:26:51PM +0200, Jan Schmidt wrote: > --- a/fs/btrfs/qgroup.c > +++ b/fs/btrfs/qgroup.c > @@ -1494,10 +1494,14 @@ int btrfs_run_qgroups(struct btrfs_trans_handle > *trans, > { > struct btrfs_root *quota_root = fs_info->quota_root; > int ret = 0; > + int st

Re: [PATCH v3 3/3] Btrfs: automatic rescan after "quota enable" command

2013-04-23 Thread David Sterba
On Tue, Apr 23, 2013 at 05:36:23PM +0200, David Sterba wrote: > The original question I've had is what sort of work does rescan do > because it's on the commit path and we don't want to add more work and > delay commit. It just wakes up the rescan worker, not a problem. -- To unsubscribe from this

Re: [PATCH 0/2] btrfs-progs: standardize utility filenames and add default rules to Makefile

2013-04-23 Thread David Sterba
On Sun, Apr 21, 2013 at 05:10:44PM -0500, Eric Sandeen wrote: > 2 patches here. I see only [PATCH 2/2] btrfs-progs: Add default rules to Makefile in my mbox that's threaded under the 0/2 cover mail. > I think this is a net win, with a bit less Makefile code > duplication, but if people don't li

Re: corrupted filesystem. mounts with -o recovery,ro but not -o recovery or -o ro

2013-04-23 Thread Jon Nelson
On Tue, Apr 23, 2013 at 10:03 AM, Liu Bo wrote: > > Can you please show us where it BUG_ON(or logs) when mounting with "-o > recovery"? > (the stack info below seems not to be a resulf of '-o recovery'?) I have this from 3.8.8: 2013-04-19T21:19:47.060937-05:00 turnip kernel: [ 100.608815] devi

Re: [PATCH 2/2 V2] btrfs-progs: update generation_v2 in btrfs_update_root

2013-04-23 Thread David Sterba
On Mon, Apr 22, 2013 at 12:16:41AM -0500, Eric Sandeen wrote: > This addresses the same issue as did: > > 2bd1169 btrfs-progs: root_item generation_v2 is out of sync after btrfsck > > but rather than optionally updating generation_v2 based > on the size of the existing item, increase the size of

Re: [PATCH 0/2] btrfs-progs: standardize utility filenames and add default rules to Makefile

2013-04-23 Thread Eric Sandeen
On Apr 23, 2013, at 11:09 AM, David Sterba wrote: > On Sun, Apr 21, 2013 at 05:10:44PM -0500, Eric Sandeen wrote: >> 2 patches here. > > I see only > > [PATCH 2/2] btrfs-progs: Add default rules to Makefile > > in my mbox that's threaded under the 0/2 cover mail. > I must have run into a size

[PATCH] Btrfs: only exclude supers in the range of our block group

2013-04-23 Thread Josef Bacik
If we fail to load block groups halfway through we can leave extent_state's on the excluded tree. This is because we just lookup the supers and add them to the excluded tree regardless of which block group we are looking at currently. This is a problem because we remove the excluded extents for th

Re: [PATCH V3] btrfs: move leak debug code to functions

2013-04-23 Thread David Sterba
On Mon, Apr 22, 2013 at 11:12:31AM -0500, Eric Sandeen wrote: > Clean up the leak debugging in extent_io.c by moving > the debug code into functions. This also removes the > list_heads used for debugging from the extent_buffer > and extent_state structures when debug is not enabled. > > Since we

Re: [PATCH] Btrfs-progs: add function to map subvol ID to path

2013-04-23 Thread David Sterba
On Tue, Apr 23, 2013 at 12:18:55PM +0200, Stefan Behrens wrote: > Several tools like btrfs-send and btrfs-receive need to map a > subvolume ID to a filesystem path. The so far existing methods > in btrfs-list.c cause a horrible effort when performing this > operation (and the effort is dependent on

Re: [PATCH v3 3/3] Btrfs: automatic rescan after "quota enable" command

2013-04-23 Thread Jan Schmidt
On Tue, April 23, 2013 at 17:36 (+0200), David Sterba wrote: > On Tue, Apr 23, 2013 at 01:26:51PM +0200, Jan Schmidt wrote: >> --- a/fs/btrfs/qgroup.c >> +++ b/fs/btrfs/qgroup.c >> @@ -1494,10 +1494,14 @@ int btrfs_run_qgroups(struct btrfs_trans_handle >> *trans, >> { >> struct btrfs_root *q

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-23 Thread Jan Schmidt
On Tue, April 23, 2013 at 16:54 (+0200), Wang Shilong wrote: > > Hello Jan, > >> >> +static void btrfs_qgroup_rescan_worker(struct btrfs_work *work) >> +{ >> +struct qgroup_rescan *qscan = container_of(work, struct qgroup_rescan, >> + work); >>

[PATCH] Btrfs: separate sequence numbers for delayed ref tracking and tree mod log

2013-04-23 Thread Jan Schmidt
Sequence numbers for delayed refs have been introduced in the first version of the qgroup patch set. To solve the problem of find_all_roots on a busy file system, the tree mod log was introduced. The sequence numbers for that were simply shared between those two users. However, at one point in qgr

[PATCH] Btrfs: fix all callers of read_tree_block

2013-04-23 Thread Josef Bacik
We kept leaking extent buffers when mounting a broken file system and it turns out it's because not everybody uses read_tree_block properly. You need to check and make sure the extent_buffer is uptodate before you use it. This patch fixes everybody who calls read_tree_block directly to make sure

[PATCH] Btrfs: only exclude supers in the range of our block group V2

2013-04-23 Thread Josef Bacik
If we fail to load block groups halfway through we can leave extent_state's on the excluded tree. This is because we just lookup the supers and add them to the excluded tree regardless of which block group we are looking at currently. This is a problem because we remove the excluded extents for th

Re: [PATCH V2 2/2] Btrfs: remove btrfs_sector_sum structure

2013-04-23 Thread Josef Bacik
On Wed, Apr 03, 2013 at 03:14:56AM -0600, Miao Xie wrote: > Using the structure btrfs_sector_sum to keep the checksum value is > unnecessary, because the extents that btrfs_sector_sum points to are > continuous, we can find out the expected checksums by btrfs_ordered_sum's > bytenr and the offset,

[PATCH 1/2 (resend)] btrfs-progs: standardize tool source filenames

2013-04-23 Thread Eric Sandeen
For any btrfs-$FOO executable, rename the main source file from $FOO.c to to btrfs-$FOO.c This makes it slightly more obvious what's building what, and allows us to write a default rule in the Makefile for these tools. (also add btrfs-calc-size to the list of objects to remove on make clean) Sig