Re: Announcing btrfs-dedupe

2016-11-13 Thread James Pharaoh
I've updated the BTRFS wiki here with all the new tools people have mentioned: https://btrfs.wiki.kernel.org/index.php/Deduplication#Other_tools Please let me know if anyone who does not have access to the wiki has any additions, updates or corrections to what I've written here. James On 08

Re: Btrfs progs release 4.8.3

2016-11-13 Thread Henk Slager
On Fri, Nov 11, 2016 at 4:38 PM, David Sterba wrote: > Hi, > > btrfs-progs version 4.8.3 have been released. Handful of fixes and lots of > cleanups. > > Changes: > * check: > * support for clearing space cache (v1) > * size reduction of inode backref structure > * send: > * fix ha

Re: how to understand "btrfs fi show" output? "No space left" issues

2016-11-13 Thread Tomasz Chmielewski
On 2016-09-21 11:51, Chris Murphy wrote: I'm coming into this late and realize most questions have been answered. But I take the position this is a bug, clearly there's enough space when df reports only 33% used, and therefore it's important to gather information about the file system in its cur

Re: when btrfs scrub reports errors and btrfs check --repair does not

2016-11-13 Thread Marc MERLIN
On Fri, Nov 11, 2016 at 07:17:08PM -0800, Marc MERLIN wrote: > On Fri, Nov 11, 2016 at 11:55:21AM +0800, Qu Wenruo wrote: > > It seems to be orphan inodes. > > Btrfs doesn't remove all the contents of an inode at rm time. > > It just unlink the inode and put it into a state called orphan inodes.(Ca

Re: when btrfs scrub reports errors and btrfs check --repair does not

2016-11-13 Thread Roman Mamedov
On Sun, 13 Nov 2016 07:06:30 -0800 Marc MERLIN wrote: > So first: > a) find -inum returns some inodes that don't match > b) but argh, multiple files (very different) have the same inode number, so > finding > files by inode number after scrub flagged an inode bad, isn't going to work :( I wonde

Re: when btrfs scrub reports errors and btrfs check --repair does not

2016-11-13 Thread Marc MERLIN
On Sun, Nov 13, 2016 at 08:13:29PM +0500, Roman Mamedov wrote: > On Sun, 13 Nov 2016 07:06:30 -0800 > Marc MERLIN wrote: > > > So first: > > a) find -inum returns some inodes that don't match > > b) but argh, multiple files (very different) have the same inode number, so > > finding > > files by

[PATCH 2/6] btrfs-progs: format FREE_SPACE_TREE{,_VALID} nicely in dump-super

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval Signed-off-by: Omar Sandoval --- cmds-inspect-dump-super.c | 24 1 file changed, 24 insertions(+) diff --git a/cmds-inspect-dump-super.c b/cmds-inspect-dump-super.c index d9f7bfb..ba0d708 100644 --- a/cmds-inspect-dump-super.c +++ b/cmds-inspect-dum

[PATCH 3/6] btrfs-progs: add OPEN_CTREE_INVALIDATE_FST flag

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval If this flag is passed to open_ctree(), we'll clear the FREE_SPACE_TREE_VALID compat_ro bit. The kernel will then reconstruct the free space tree the next time the filesystem is mounted. Signed-off-by: Omar Sandoval --- chunk-recover.c | 2 +- disk-io.c | 28

[PATCH 0/6] btrfs-progs: better space_cache=v2 support

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval Hi, This series implements some support for space_cache=v2 in btrfs-progs. In particular, this adds support for `btrfs check --clear-space-cache v2`, proper printing of the free space tree flags for `btrfs inspect-internal dump-super`, and better documentation. We'd previous

[PATCH 1/6] btrfs-progs: add the FREE_SPACE_TREE_VALID compat_ro bit definition

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval This is just the definition; we don't support it yet. Signed-off-by: Omar Sandoval --- ctree.h | 13 + 1 file changed, 13 insertions(+) diff --git a/ctree.h b/ctree.h index d47f0ae..d67b852 100644 --- a/ctree.h +++ b/ctree.h @@ -467,6 +467,14 @@ struct btrfs_su

[PATCH 4/6] btrfs-progs: add btrfs_clear_free_space_tree() from the kernel

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval Signed-off-by: Omar Sandoval --- ctree.h | 6 extent-tree.c | 10 +++ free-space-tree.c | 87 +++ free-space-tree.h | 1 + root-tree.c | 22 ++ 5 files changed, 126 insertions(+)

[PATCH 6/6] btrfs-progs: document space_cache=v2 more thoroughly

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval Signed-off-by: Omar Sandoval --- Documentation/btrfs-man5.asciidoc | 43 +++ 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/Documentation/btrfs-man5.asciidoc b/Documentation/btrfs-man5.asciidoc index d12b059..3b003d3 1006

[PATCH 5/6] btrfs-progs: implement btrfs check --clear-space-cache v2

2016-11-13 Thread Omar Sandoval
From: Omar Sandoval Signed-off-by: Omar Sandoval --- Documentation/btrfs-check.asciidoc | 14 +- cmds-check.c | 34 +- 2 files changed, 34 insertions(+), 14 deletions(-) diff --git a/Documentation/btrfs-check.asciidoc b/Documen

Re: [PATCH 3/6] btrfs-progs: add OPEN_CTREE_INVALIDATE_FST flag

2016-11-13 Thread Qu Wenruo
At 11/14/2016 03:35 AM, Omar Sandoval wrote: From: Omar Sandoval If this flag is passed to open_ctree(), we'll clear the FREE_SPACE_TREE_VALID compat_ro bit. The kernel will then reconstruct the free space tree the next time the filesystem is mounted. This feature is really helpful. Especi

Re: [PATCH 4/6] btrfs-progs: add btrfs_clear_free_space_tree() from the kernel

2016-11-13 Thread Qu Wenruo
At 11/14/2016 03:35 AM, Omar Sandoval wrote: From: Omar Sandoval Signed-off-by: Omar Sandoval Only small nits about the BUG_ON() and return value. Despite that, feel free to add my reviewed-by: Reviewed-by: Qu Wenruo --- ctree.h | 6 extent-tree.c | 10 +++ free

Re: [PATCH 5/6] btrfs-progs: implement btrfs check --clear-space-cache v2

2016-11-13 Thread Qu Wenruo
At 11/14/2016 03:35 AM, Omar Sandoval wrote: From: Omar Sandoval Signed-off-by: Omar Sandoval Reviewed-by: Qu Wenruo --- Documentation/btrfs-check.asciidoc | 14 +- cmds-check.c | 34 +- 2 files changed, 34 insertions(+),

Re: [RFC] btrfs: make max inline data can be equal to sectorsize

2016-11-13 Thread Qu Wenruo
At 11/12/2016 04:22 AM, Liu Bo wrote: On Tue, Oct 11, 2016 at 02:47:42PM +0800, Wang Xiaoguang wrote: If we use mount option "-o max_inline=sectorsize", say 4096, indeed even for a fresh fs, say nodesize is 16k, we can not make the first 4k data completely inline, I found this conditon causing

[PATCH v14.1 00/16] Btrfs In-band De-duplication

2016-11-13 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/linux.git wang_dedupe_latest This version is just another update mainly related to ENOSPC fix, rebased to Chris' for-linus-4.9 branch. The main part is the ENOSPC fix, which is first exposed in dedupe development, and thanks

[PATCH v14.1 04/16] btrfs: dedupe: Introduce dedupe framework and its header

2016-11-13 Thread Qu Wenruo
From: Wang Xiaoguang Introduce the header for btrfs in-band(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedupe methods and 1 dedupe hash. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h

[PATCH v14.1 03/16] btrfs: Introduce COMPRESS reserve type to fix false enospc for compression

2016-11-13 Thread Qu Wenruo
From: Wang Xiaoguang When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my test environment when compression is enabled. After some debuging work, we f

[PATCH v14.1 02/16] btrfs: introduce type based delalloc metadata reserve

2016-11-13 Thread Qu Wenruo
From: Wang Xiaoguang Introduce type based metadata reserve parameter for delalloc space reservation/freeing function. The problem we are going to solve is, btrfs use different max extent size for different mount options. For compression, the max extent size is 128K, while for non-compress write