Re: nocow 'C' flag ignored after balance

2013-05-09 Thread Liu Bo
On Thu, May 09, 2013 at 03:41:49PM -0500, Kyle Gates wrote: I'll preface that I'm running Ubuntu 13.04 with the standard 3.8 series kernel so please disregard if this has been fixed in higher versions. This is on a btrfs RAID1 with 3 then 4 disks. My use case is to set the nocow 'C' flag on

[PATCH] Btrfs: remove useless copy in quota_ctl

2013-05-13 Thread Liu Bo
We don't need to copy it back to user side as it remains unchanged. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ioctl.c |3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 0de4a2f..c189189 100644 --- a/fs/btrfs/ioctl.c

Re: [PATCH] Btrfs-progs: add dedup register

2013-05-13 Thread Liu Bo
On Mon, May 13, 2013 at 05:55:10PM +0200, David Sterba wrote: On Thu, May 02, 2013 at 12:27:39AM +0800, Liu Bo wrote: +static int cmd_dedup(int argc, char **argv) +{ + int fd, res, e; + char*path; + + if (check_argc_exact(argc, 2)) + usage(cmd_dedup_usage

[PATCH] Btrfs: return errno if possible when we fail to allocate memory

2013-05-13 Thread Liu Bo
We need to set return value explicitly, otherwise we'll lose the error value. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 1669c3b..99a9c25 100644 --- a/fs

Re: [PATCH v2 4/8] Btrfs: maintain subvolume items in the UUID tree

2013-05-14 Thread Liu Bo
On Tue, May 14, 2013 at 11:36:56AM +0200, Stefan Behrens wrote: When a new subvolume or snapshot is created, a new UUID item is added to the UUID tree. Such items are removed when the subvolume is deleted. The ioctl to set the received subvolume UUID is also touched and will now also add this

Re: [PATCH v2 3/8] Btrfs: create UUID tree if required

2013-05-14 Thread Liu Bo
On Tue, May 14, 2013 at 11:36:55AM +0200, Stefan Behrens wrote: This tree is not created by mkfs.btrfs. Therefore when a filesystem is mounted writable and the UUID tree does not exist, this tree is created if required. The tree is also added to the fs_info structure and initialized, but this

Re: [PATCH v2 0/8] Btrfs: introduce a tree for UUID to subvol ID mapping

2013-05-14 Thread Liu Bo
On Tue, May 14, 2013 at 11:36:52AM +0200, Stefan Behrens wrote: Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single

[PATCH] Btrfs-progs: add dedup subcommand

2013-05-14 Thread Liu Bo
This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem. Signed-off-by: Liu Bo bo.li@oracle.com --- Makefile |2 +- btrfs.c |1 + cmds-dedup.c | 101

[RFC PATCH V4 0/2] Online data deduplication

2013-05-14 Thread Liu Bo
) btrfs dedup unregister /mnt_fs_your_btrfs (NOTE: 'unregister' won't work unless you do step 1 FIRSTLY.) --- Liu Bo (2): Btrfs: skip merge part for delayed data refs Btrfs: online data deduplication fs/btrfs/ctree.h | 63

[RFC PATCH V4 1/2] Btrfs: skip merge part for delayed data refs

2013-05-14 Thread Liu Bo
-by: Liu Bo bo.li@oracle.com --- fs/btrfs/delayed-ref.c |7 +++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index c219463..be8d5cb 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -320,6 +320,13

[PATCH] Btrfs: remove unused code in btrfs_del_root

2013-05-14 Thread Liu Bo
'leaf' and 'ri' is not used somehow. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/root-tree.c |4 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c index 5bf1ed5..4a5abda 100644 --- a/fs/btrfs/root-tree.c +++ b/fs/btrfs

Re: [PATCH] xfstests btrfs/284: shorten duration, fix output

2013-05-14 Thread Liu Bo
commit changed it to 0. So accommodate that too. And remove a stray HAVE_DEFRAG=1 while we're at it. That variable is never used. Thanks for working on this, Eric! You can add Reviewed-by: Liu Bo bo.li@oracle.com thanks, liubo Signed-off-by: Eric Sandeen sand...@redhat.com

Re: I/O errors block the entire filesystem

2013-05-14 Thread Liu Bo
On Thu, Apr 04, 2013 at 01:10:27PM -0300, Alexandre Oliva wrote: I've been trying to figure out the btrfs I/O stack to try to understand why, sometimes (but not always), after a failure to read a (data non-replicated) block from the disk, the file being accessed becomes permanently locked, and

Re: Defragmentation of large files

2013-05-14 Thread Liu Bo
On Sat, May 11, 2013 at 05:57:11PM +0100, Tim Eggleston wrote: Yes. The command just triggers the defragmentation which takes place in the background. Try a sync afterwards :) Sorry Martin, I should have specified that I wondered if it was like the scrub operation in that respect, so I

Re: [PATCH v2 4/8] Btrfs: maintain subvolume items in the UUID tree

2013-05-15 Thread Liu Bo
On Wed, May 15, 2013 at 05:39:35PM +0200, Stefan Behrens wrote: On Tue, 14 May 2013 18:44:11 +0800, Liu Bo wrote: On Tue, May 14, 2013 at 11:36:56AM +0200, Stefan Behrens wrote: @@ -396,7 +403,7 @@ static noinline int create_subvol(struct inode *dir, * of create_snapshot

Re: [PATCH 10/17] Btrfs: just flush the delalloc inodes in the source tree before snapshot creation

2013-05-15 Thread Liu Bo
On Wed, May 15, 2013 at 03:48:24PM +0800, Miao Xie wrote: Before applying this patch, we need flush all the delalloc inodes in the fs when we want to create a snapshot, it wastes time, and make the transaction commit be blocked for a long time. It means some other user operation would also be

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-15 Thread Liu Bo
On Wed, May 15, 2013 at 03:48:20PM +0800, Miao Xie wrote: The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use reference counter instead of the srcu lock is to aovid blocking the memory reclaim task, which invokes

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-15 Thread Liu Bo
On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May 2013 11:36:46 +0800, Liu Bo wrote: On Wed, May 15, 2013 at 03:48:20PM +0800, Miao Xie wrote: The grab/put funtions will be used in the next patch, which need grab the root object and ensure it is not freed. We use

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 01:34:11PM +0800, Miao Xie wrote: On Thu, 16 May 2013 13:15:57 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May 2013 11:36:46 +0800, Liu Bo wrote: On Wed, May 15, 2013 at 03:48:20PM +0800, Miao Xie wrote: The grab/put

Re: [PATCH v2 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-05-16 Thread Liu Bo
On Tue, May 14, 2013 at 11:36:53AM +0200, Stefan Behrens wrote: Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 03:22:37PM +0800, Miao Xie wrote: On thu, 16 May 2013 14:15:52 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 01:34:11PM +0800, Miao Xie wrote: On Thu, 16 May 2013 13:15:57 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 10:34:55AM -0400, Chris Mason wrote: Quoting Liu Bo (2013-05-16 10:31:39) On Thu, May 16, 2013 at 07:54:17AM -0400, Chris Mason wrote: Quoting Miao Xie (2013-05-16 03:22:37) I must say that the patch itself looks harmless, the reason is not good enough

Re: nocow 'C' flag ignored after balance

2013-05-17 Thread Liu Bo
. Thanks for taking the time to read, Kyle P.S. I'm CCing David as I believe he wrote the patch to get the 'C' flag working on empty files and directories. Hi Kyle, Can you please apply this patch and see if it helps? thanks, liubo From: Liu Bo bo.li@oracle.com Subject: [PATCH] Btrfs: fix

[PATCH] Btrfs: fix memory leak in replace_path

2013-05-17 Thread Liu Bo
We need to unlock and free extent buffer before the return. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/relocation.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 704a1b8..5c5b8bb 100644 --- a/fs/btrfs

Re: [PATCH] Btrfs: fix memory leak in replace_path

2013-05-17 Thread Liu Bo
On Fri, May 17, 2013 at 10:52:16AM +0200, Stefan Behrens wrote: On Fri, 17 May 2013 16:45:44 +0800, Liu Bo wrote: We need to unlock and free extent buffer before the return. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/relocation.c |2 +- 1 files changed, 1 insertions

Re: [PATCH] Btrfs: fix memory leak in replace_path

2013-05-17 Thread Liu Bo
oops, please ignore this as there is already a same patch from Stefan. Sorry for the trouble. thanks, liubo On Fri, May 17, 2013 at 04:45:44PM +0800, Liu Bo wrote: We need to unlock and free extent buffer before the return. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs

[PATCH 1/2] Btrfs: dont do log_removal in insert_new_root

2013-05-22 Thread Liu Bo
that either. As above, insert_new_root can get rid of log_removal. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ctree.c | 10 +- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 02fae7f..c07b6ea 100644 --- a/fs/btrfs

[PATCH 2/2] Btrfs: check if leaf's parent exists before pushing items around

2013-05-22 Thread Liu Bo
During splitting a leaf, pushing items around to hopefully get some space only works when we have a parent, ie. we have at least one sibling leaf. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ctree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs

[PATCH 1/5] Btrfs: fix use-after-free bug during umount

2013-05-26 Thread Liu Bo
before actually getting to free block groups where caching thread needs valid extent_root-commit_root. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/disk-io.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index e7b3cb5

[PATCH 2/5] Btrfs: update new flags for tracepoint

2013-05-26 Thread Liu Bo
Adding new flags to keep tracepoints consistent with btrfs. Signed-off-by: Liu Bo bo.li@oracle.com --- include/trace/events/btrfs.h | 35 +++ 1 files changed, 23 insertions(+), 12 deletions(-) diff --git a/include/trace/events/btrfs.h b/include/trace/events

[PATCH 3/5] Btrfs: kill replicate code in replay_one_buffer

2013-05-26 Thread Liu Bo
EXTREF is treated same as REF, so we can make the code tidy. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/tree-log.c |9 ++--- 1 files changed, 2 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index c276ac9..3f30053 100644 --- a/fs/btrfs

[PATCH 4/5 RESEND] Btrfs: remove unused code in btrfs_del_root

2013-05-26 Thread Liu Bo
'leaf' and 'ri' is not used somehow. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/root-tree.c |4 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c index 5bf1ed5..4a5abda 100644 --- a/fs/btrfs/root-tree.c +++ b/fs/btrfs

[PATCH 5/5 RESEND] Btrfs: allow file data clone within a file

2013-05-26 Thread Liu Bo
Sterba dste...@suse.cz Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ioctl.c | 26 +++--- 1 files changed, 19 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 0f81d67..112153a 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c

[Question] How to restore btrfs raid0 image file?

2013-05-26 Thread Liu Bo
Hi, So the case is, now I've got a btrfs image file, which is created from a raid0 btrfs fs. And if I run 'btrfs-image -r image_file /dev/sdf', then I have to mount it with 'degraded' mode, and that still fails because raid0 requires two disks at least. So any ideas how to make it work?

Re: [PATCH 1/5] Btrfs: fix use-after-free bug during umount

2013-05-27 Thread Liu Bo
On Mon, May 27, 2013 at 02:07:00PM +0200, David Sterba wrote: On Sun, May 26, 2013 at 09:50:27PM +0800, Liu Bo wrote: Commit be283b2e674a09457d4563729015adb637ce7cc1 (Btrfs: use helper to cleanup tree roots) introduced the following bug, Well, it did not introduce the bug, but made

Re: nocow 'C' flag ignored after balance

2013-05-28 Thread Liu Bo
On Tue, May 28, 2013 at 09:22:11AM -0500, Kyle Gates wrote: From: Liu Bo bo.li@oracle.com Subject: [PATCH] Btrfs: fix broken nocow after a normal balance [...] Sorry for the long wait in replying. This patch was unsuccessful in fixing the problem (on my 3.8 Ubuntu Raring kernel). I

Re: [PATCH] Btrfs: init relocate extent_io_tree with a mapping

2013-05-31 Thread Liu Bo
the relocation's processed_blocks with the btree_inode-i_mapping. Thanks, That fits my suspection, hhh. Reviewed-by: Liu Bo bo.li@oracle.com Reported-by: David Sterba dste...@suse.cz Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/relocation.c |9 + 1 files changed

Re: Is there a way to flag specific directories nodatacow?

2013-06-02 Thread Liu Bo
On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them. My bandaid fix for this will be to mark the /var filesystem nodatacow at boot. But I am wondering if

Re: Is there a way to flag specific directories nodatacow?

2013-06-02 Thread Liu Bo
On Sun, Jun 02, 2013 at 07:19:50PM -0700, George Mitchell wrote: On 06/02/2013 06:28 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them

Re: Is there a way to flag specific directories nodatacow?

2013-06-02 Thread Liu Bo
On Sun, Jun 02, 2013 at 07:11:10PM -0700, George Mitchell wrote: On 06/02/2013 06:28 PM, Liu Bo wrote: On Sun, Jun 02, 2013 at 07:40:52AM -0700, George Mitchell wrote: I am seeing massive journal corruptions that seem to be unique to btrfs and I am suspecting that cow might be causing them

[PATCH] Btrfs: fix broken nocow after balance

2013-06-03 Thread Liu Bo
a delayed ref if the file extent is really shared by several parents, we can make nocow happy again without having to check confusing last_snapshot. Reported-by: Kyle Gates kylega...@hotmail.com Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c |4 fs/btrfs/inode.c

[PATCH] Btrfs-progs: elaborate error handling of mkfs

2013-06-08 Thread Liu Bo
$./mkfs.btrfs -f /dev/sdd -b 2M [...] mkfs.btrfs: volumes.c:845: btrfs_alloc_chunk: Assertion `!(ret)' failed. Aborted (core dumped). We should return error to userspace instead of the above. Signed-off-by: Liu Bo bo.li@oracle.com --- mkfs.c| 23 +++ volumes.c

[PATCH] Btrfs-progs: fix misuse of skinny metadata in btrfs-image

2013-06-13 Thread Liu Bo
As for skinny metadata, key.offset stores levels rather than extent length. Signed-off-by: Liu Bo bo.li@oracle.com --- btrfs-image.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-image.c b/btrfs-image.c index 739ae35..e5ff795 100644 --- a/btrfs-image.c +++ b

Re: converting lzo compression to zlib compression?

2013-06-16 Thread Liu Bo
On Mon, Jun 17, 2013 at 02:16:53AM +0900, Tomasz Chmielewski wrote: I have a btrfs filesystem mounted with compress=lzo now. I'd like to see how much space would I save by converting it to zlib. Would it be possible to convert lzo compression to zlib compression with the following: -

[PATCH 1/4] Btrfs-progs: fix misuse of skinny metadata in btrfs-image

2013-06-20 Thread Liu Bo
As for skinny metadata, key.offset stores levels rather than extent length. Signed-off-by: Liu Bo bo.li@oracle.com --- btrfs-image.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-image.c b/btrfs-image.c index 739ae35..e5ff795 100644 --- a/btrfs-image.c

[PATCH 2/4] Btrfs-progs: skip open devices which is missing

2013-06-20 Thread Liu Bo
A device can be added to the device list without getting a name, so we may access to illegal addresses while opening devices with their name. Signed-off-by: Liu Bo bo.li@oracle.com --- volumes.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/volumes.c b/volumes.c

[PATCH 0/4] multiple disks restore support of btrfs-image

2013-06-20 Thread Liu Bo
Patch 1-3 are bug fixes for several places. Patch 4 adds btrfs-image support of multiple disks restore. Liu Bo (4): Btrfs-progs: fix misuse of skinny metadata in btrfs-image Btrfs-progs: skip open devices which is missing Btrfs-progs: delete fs_devices itself from fs_uuid list before

[PATCH 4/4] Btrfs-progs: exhance btrfs-image to restore image onto multiple disks

2013-06-20 Thread Liu Bo
size of the original disks). Besides, this also works with raid5 and raid6 metadata image. Signed-off-by: Liu Bo bo.li@oracle.com --- btrfs-image.c | 294 ++--- ctree.h |1 + disk-io.c | 90 +- disk-io.h

[PATCH 3/4] Btrfs-progs: delete fs_devices itself from fs_uuid list before freeing

2013-06-20 Thread Liu Bo
Otherwise we will access illegal addresses while searching on fs_uuid list. Signed-off-by: Liu Bo bo.li@oracle.com --- disk-io.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/disk-io.c b/disk-io.c index 21b410d..2892300 100644 --- a/disk-io.c +++ b/disk-io.c

Re: [PATCH 4/4] Btrfs-progs: exhance btrfs-image to restore image onto multiple disks

2013-06-20 Thread Liu Bo
On Thu, Jun 20, 2013 at 08:39:19AM -0400, Josef Bacik wrote: On Thu, Jun 20, 2013 at 08:24:32AM -0400, Josef Bacik wrote: On Thu, Jun 20, 2013 at 08:05:30PM +0800, Liu Bo wrote: This adds a 'btrfs-image -m' option, which let us restore an image that is built from a btrfs of multiple disks

Re: [PATCH 4/4] Btrfs-progs: exhance btrfs-image to restore image onto multiple disks

2013-06-20 Thread Liu Bo
On Thu, Jun 20, 2013 at 08:39:19AM -0400, Josef Bacik wrote: On Thu, Jun 20, 2013 at 08:24:32AM -0400, Josef Bacik wrote: On Thu, Jun 20, 2013 at 08:05:30PM +0800, Liu Bo wrote: This adds a 'btrfs-image -m' option, which let us restore an image that is built from a btrfs of multiple disks

Re: [PATCH 4/4] Btrfs-progs: exhance btrfs-image to restore image onto multiple disks

2013-06-20 Thread Liu Bo
On Thu, Jun 20, 2013 at 09:10:24PM -0400, Chris Mason wrote: Quoting Liu Bo (2013-06-20 08:05:30) This adds a 'btrfs-image -m' option, which let us restore an image that is built from a btrfs of multiple disks onto several disks altogether. I'd like to pull this in, could you please rebase

Re: [PATCH 1/5 v2] Btrfs-progs: fix closing of devices

2013-06-21 Thread Liu Bo
0x00403ed4 in main (argc=2, argv=0x7fff9a583d28) at btrfs.c:295 Good catch! This addresses one of my problems. Reviewed-by: Liu Bo bo.li@oracle.com thanks, liubo Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- disk-io.c |4 ++-- volumes.c |5

Re: [PATCH 1/5 v2] Btrfs-progs: fix closing of devices

2013-06-21 Thread Liu Bo
On Tue, Jun 11, 2013 at 12:52:36AM +0100, Filipe David Borba Manana wrote: If a device could not be opened in volumes.c:read_one_dev(), a btrfs_device instance was allocated and added to the list of devices of the fs - however this device instance had its fd, name and label fields not

[PATCH 4/5] Btrfs-progs: cleanup btrfs-image usage

2013-06-21 Thread Liu Bo
A '\n' is missing. Signed-off-by: Liu Bo bo.li@oracle.com --- btrfs-image.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/btrfs-image.c b/btrfs-image.c index 30c1a9a..ebd283e 100644 --- a/btrfs-image.c +++ b/btrfs-image.c @@ -2272,7 +2272,7 @@ static void

[PATCH 2/5] Btrfs-progs: skip open devices which is missing

2013-06-21 Thread Liu Bo
A device can be added to the device list without getting a name, so we may access to illegal addresses while opening devices with their name. Signed-off-by: Liu Bo bo.li@oracle.com --- volumes.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/volumes.c b/volumes.c

[PATCH 3/5] Btrfs-progs: delete fs_devices itself from fs_uuid list before freeing

2013-06-21 Thread Liu Bo
Otherwise we will access illegal addresses while searching on fs_uuid list. Signed-off-by: Liu Bo bo.li@oracle.com --- disk-io.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/disk-io.c b/disk-io.c index 768adda..9cfd003 100644 --- a/disk-io.c +++ b/disk-io.c

[PATCH 1/5] Btrfs-progs: fix misuse of skinny metadata in btrfs-image

2013-06-21 Thread Liu Bo
As for skinny metadata, key.offset stores levels rather than extent length. Signed-off-by: Liu Bo bo.li@oracle.com --- btrfs-image.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-image.c b/btrfs-image.c index 22239fe..30c1a9a 100644 --- a/btrfs-image.c

Re: Filebench varmail can not run in btrfs of kernel 3.8.2

2013-06-22 Thread Liu Bo
On Sat, Jun 22, 2013 at 03:35:23PM +0800, ZHANG, Benlong wrote: Hello, When I run filebench varmail test under btrfs of Linux kernel 3.8.2, I got this error: [11496.552440] WARNING: at fs/btrfs/tree-log.c:3825 btrfs_log_inode_parent+0x28e/0x2f1 [btrfs]() [11496.552560] Hardware name:

Re: hang on 3.9, 3.10-rc5

2013-06-25 Thread Liu Bo
On Fri, Jun 21, 2013 at 08:42:55AM +0200, Clemens Eisserer wrote: Hi Jon, Is this what you are looking for? After this, the CPU gets stuck and I have to reboot. This issue has already been reported: https://bugzilla.kernel.org/show_bug.cgi?id=59451 Hi, Could you please try this patch?

[PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-25 Thread Liu Bo
while it's not OK for a rbtree as it uses pointers. So krealloc() will mess up our rbtree and it ends up with crash. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ulist.c | 13 - 1 files changed, 12 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/ulist.c b/fs/btrfs

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 08:38:21AM -0400, Josef Bacik wrote: On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 01:18:29PM -0700, Zach Brown wrote: On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 01:18:29PM -0700, Zach Brown wrote: On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 07:23:41PM -0700, Zach Brown wrote: But in the krealloc() case the rb_erase() will be trying to reference freed memmory because krealloc() frees the old pointer on success. Yeah, I realize that you're absolutely right, but my box didn't complain about the

[PATCH v2] Btrfs: fix crash regarding to ulist_add_merge

2013-06-27 Thread Liu Bo
while it's not OK for a rbtree as it uses pointers. So krealloc() will mess up our rbtree and it ends up with crash. Signed-off-by: Liu Bo bo.li@oracle.com --- v2: fix an use-after-free bug and a finger error(Thanks Zach and Josef). fs/btrfs/ulist.c | 17 + 1 files changed

Re: [PATCH v2] Btrfs: fix crash regarding to ulist_add_merge

2013-06-27 Thread Liu Bo
old data to new memory area, so it's OK for an array as it doesn't use pointers while it's not OK for a rbtree as it uses pointers. So krealloc() will mess up our rbtree and it ends up with crash. Signed-off-by: Liu Bo bo.li@oracle.com --- v2: fix an use-after-free bug

[PATCH v3] Btrfs: fix crash regarding to ulist_add_merge

2013-06-27 Thread Liu Bo
while it's not OK for a rbtree as it uses pointers. So krealloc() will mess up our rbtree and it ends up with crash. Reviewed-by: Wang Shilong wangsl-f...@cn.fujitsu.com Signed-off-by: Liu Bo bo.li@oracle.com --- v3: fix a return value problem(Thanks Wang Shilong). v2: fix an use-after-free bug

Re: [PATCH v3] Btrfs: fix crash regarding to ulist_add_merge

2013-06-29 Thread Liu Bo
On Fri, Jun 28, 2013 at 12:43:14PM -0700, Zach Brown wrote: On Fri, Jun 28, 2013 at 12:37:45PM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: [PATCH v2] Btrfs: fix crash regarding to ulist_add_merge

2013-06-29 Thread Liu Bo
On Fri, Jun 28, 2013 at 01:08:21PM -0400, Josef Bacik wrote: On Fri, Jun 28, 2013 at 10:25:39AM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: [PATCH] Btrfs: make backref walking code handle skinny metadata

2013-06-29 Thread Liu Bo
0. This is because the delayed refs will be found for stuff just fine, but it won't find the skinny metadata extent refs. With this patch I'm not seeing warnings anymore. Thanks, Reviewed-by: Liu Bo bo.li@oracle.com - liubo Signed-off-by: Josef Bacik jba...@fusionio.com --- fs

Re: [PATCH] Btrfs: hold the tree mod lock in __tree_mod_log_rewind

2013-07-01 Thread Liu Bo
On Sun, Jun 30, 2013 at 02:01:17PM -0400, Josef Bacik wrote: On Sun, Jun 30, 2013 at 11:02:10PM +0800, Liu Bo wrote: On Sun, Jun 30, 2013 at 07:22:00AM -0400, Josef Bacik wrote: On Sun, Jun 30, 2013 at 10:25:05AM +0200, Jan Schmidt wrote: On 30.06.2013 05:17, Josef Bacik wrote: We

[PATCH] Btrfs: fix a bug of snapshot-aware defrag to make it work on partial extents

2013-07-01 Thread Liu Bo
$ btrfs sub snap /mnt/sub /mnt/snap2 $ sync; btrfs filesystem defrag /mnt/sub/foo; $ umount /mnt $ btrfs-debug-tree sda (Here we can check whether the defrag operation is snapshot-awared. This addresses the above two problems. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c

[RFC PATCH] Btrfs: rework ulist with list operation

2013-07-01 Thread Liu Bo
. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ctree.h |1 + fs/btrfs/super.c |9 +- fs/btrfs/ulist.c | 85 ++--- fs/btrfs/ulist.h | 21 +++-- 4 files changed, 62 insertions(+), 54 deletions(-) diff --git a/fs/btrfs

Re: [PATCH] Btrfs: only do the tree_mod_log_free_eb if this is our last ref

2013-07-02 Thread Liu Bo
in __tree_mod_log_rewind. Thanks, This fixes my reproducer for __tree_mod_log_rewind() BUG_ON in line 1183. Tested-by: Liu Bo bo.li@oracle.com Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/ctree.c |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/ctree.c

[PATCH] Btrfs: fix extent buffer leak after backref walking

2013-07-03 Thread Liu Bo
commit 47fb091fb787420cd195e66f162737401cce023f(Btrfs: fix unlock after free on rewinded tree blocks) takes an extra increment on the reference of allocated dummy extent buffer, so now we cannot free this dummy one, and end up with extent buffer leak. Signed-off-by: Liu Bo bo.li@oracle.com

Re: WARNING: at fs/btrfs/backref.c:903 find_parent_nodes+0x616/0x815 [btrfs]()

2013-07-03 Thread Liu Bo
On Wed, Jul 03, 2013 at 11:37:09AM +0900, Tomasz Chmielewski wrote: I've upgraded to linux 3.10 and enabled extended inode refs and skinny metadata extent refs with these commands: btrfstune -r /dev/sdc1 btrfstune -x /dev/sdc1 Since then, I have WARNING: at fs/btrfs/backref.c:903

[PATCH] Xfstests/btrfs/308: add snapshot-aware defrag for partial extents testcases

2013-07-03 Thread Liu Bo
This is to test whether snapshot-aware defrag can work well on partial extents. Signed-off-by: Liu Bo bo.li@oracle.com --- tests/btrfs/308 | 93 +++ tests/btrfs/308.out |2 + tests/btrfs/group |1 + 3 files changed, 96

Re: [PATCH] Btrfs: fix a bug of snapshot-aware defrag to make it work on partial extents

2013-07-03 Thread Liu Bo
On Mon, Jul 01, 2013 at 04:22:06PM -0400, Josef Bacik wrote: On Mon, Jul 01, 2013 at 10:13:26PM +0800, Liu Bo wrote: For partial extents, snapshot-aware defrag does not work as expected, since a) we use the wrong logical offset to search for parents, which should be disk_bytenr

Re: [RFC PATCH 00/12] Btrfs-progs: introduce chunk recover function

2013-07-03 Thread Liu Bo
On Wed, Jul 03, 2013 at 04:36:44PM -0400, Chris Mason wrote: Quoting Miao Xie (2013-07-03 09:25:08) This patchset introduced chunk recover function, which was implemented by scanning the whoel disks in the filesystem. Now, we can recover Single, Dup, RAID1 chunks, and RAID0, RAID10, RAID5,

[PATCH] Btrfs: make free space caching faster with many non-inline extent references

2013-07-11 Thread Liu Bo
the tree with the next address to skip non-inline references. (This is found by dedup feature because dedup extents can end up with many non-inline extent refs.) Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c | 11 +++ 1 files changed, 11 insertions(+), 0 deletions

[PATCH] Btrfs/tracepoint: update delayed ref tracepoints

2013-07-16 Thread Liu Bo
This shows exactly how btrfs processes the delayed refs onto disks, which is very helpful on understanding delayed ref mechanism and debugging related bugs. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/delayed-ref.c |6 ++-- fs/btrfs/extent-tree.c |6 include

Re: [PATCH v3] Btrfs: fix crash regarding to ulist_add_merge

2013-07-29 Thread Liu Bo
On Mon, Jul 29, 2013 at 10:23:34AM +0200, Jan Schmidt wrote: On Fri, June 28, 2013 at 06:37 (+0200), Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist

Re: Online data deduplication

2013-07-29 Thread Liu Bo
On Mon, Jul 29, 2013 at 09:05:42PM +0530, Hemanth Kumar wrote: Hello, I am willing to perform QA on online data deduplication. From where can i download the patches? Hi Hemanth Kumar H C, I really appreciate this :) Right now I'm planning v5 version patch set, which will come out probably in

[PATCH RESEND] Btrfs: fix crash regarding to ulist_add_merge

2013-07-30 Thread Liu Bo
while it's not OK for a rbtree as it uses pointers. So krealloc() will mess up our rbtree and it ends up with crash. Cc: sta...@vger.kernel.org Reviewed-by: Wang Shilong wangsl-f...@cn.fujitsu.com Reviewed-by: Jan Schmidt list.bt...@jan-o-sch.net Signed-off-by: Liu Bo bo.li@oracle.com Signed

Re: [bug] balance BUG at fs/btrfs/relocation.c:886

2013-07-31 Thread Liu Bo
On Wed, Jul 31, 2013 at 05:53:29PM +0800, Anand Jain wrote: 3.11.0-rc3+ -- [ 225.586356] kernel BUG at fs/btrfs/relocation.c:886! [ 225.586362] invalid opcode: [#1] SMP :: [ 225.586431] Call Trace: [ 225.586440] [a0055cd1] ?

[RFC PATCH v5 3/5] Btrfs: introduce a head ref rbtree

2013-07-31 Thread Liu Bo
refs. When we have a great number of delayed refs pending to process, this'll cost time a lot. Here we introduce a head ref specific rbtree, it only has head refs, so troubles go away. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/delayed-ref.c | 124

[RFC PATCH v5 0/5] Online data deduplication

2013-07-31 Thread Liu Bo
for dedup tree because the disk format understands backref well. - fix a fsync hang with dedup enabled. - rebase onto the latest btrfs. Liu Bo (5): Btrfs: skip merge part for delayed data refs Btrfs: improve the delayed refs process in rm case Btrfs: introduce a head ref rbtree Btrfs

[RFC PATCH v5 1/5] Btrfs: skip merge part for delayed data refs

2013-07-31 Thread Liu Bo
it down for data refs. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/delayed-ref.c |7 +++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index f7be9f7..fc4ce8b 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed

[PATCH] Btrfs-progs: add dedup subcommand

2013-07-31 Thread Liu Bo
This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem. Signed-off-by: Liu Bo bo.li@oracle.com --- Makefile |2 +- btrfs.c |1 + cmds-dedup.c | 101

[RFC PATCH v5 2/5] Btrfs: improve the delayed refs process in rm case

2013-07-31 Thread Liu Bo
there is no such thing, and then start over again to find BTRFS_DROP_DELAYED_REF. This is really unnecessary, we can improve this by tracking how many BTRFS_ADD_DELAYED_REF refs we have and search by the right type. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/delayed-ref.c | 10 ++ fs

[RFC PATCH v5 4/5] Btrfs: disable qgroups accounting when quota is off

2013-07-31 Thread Liu Bo
So we don't need to do qgroups accounting trick without enabling quota. This reduces my tester's costing time from ~28s to ~23s. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c |6 ++ fs/btrfs/qgroup.c |6 ++ 2 files changed, 12 insertions(+), 0

Re: [RFC PATCH v5 5/5] Btrfs: online data deduplication

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 03:50:50PM -0700, Zach Brown wrote: +#define BTRFS_DEDUP_HASH_SIZE 32 /* 256bit = 32 * 8bit */ +#define BTRFS_DEDUP_HASH_LEN 4 + +struct btrfs_dedup_hash_item { + /* FIXME: put a hash type field here */ + + __le64 hash[BTRFS_DEDUP_HASH_LEN]; +}

Re: [RFC PATCH v5 0/5] Online data deduplication

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 05:20:27PM -0400, Josef Bacik wrote: On Wed, Jul 31, 2013 at 11:37:40PM +0800, Liu Bo wrote: Data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data.[1] This patch set is also related to Content based

Re: [PATCH] Btrfs-progs: add dedup subcommand

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 06:30:37PM +0200, Stefan Behrens wrote: On Wed, 31 Jul 2013 23:37:46 +0800, Liu Bo wrote: This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem

Re: filefrag and btrfs filesystem defragment and maybe snapshots

2013-08-01 Thread Liu Bo
On Thu, Aug 01, 2013 at 05:18:50PM -0400, Sandy McArthur wrote: While exploring some btrfs maintenance with respect to defragmenting I ran the following commands: # filefrag /path/to/34G.file /path/to/5.7G.file /path/to/34G.file: 2406 extents found /path/to/5.7G.file: 572 extents found

Re: [PATCH] Btrfs-progs: add dedup subcommand

2013-08-01 Thread Liu Bo
On Thu, Aug 01, 2013 at 03:01:37PM -0700, Mark Fasheh wrote: On Wed, Jul 31, 2013 at 11:37:46PM +0800, Liu Bo wrote: This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem

[PATCH] Btrfs: allow compressed extents to be merged during defragment

2013-08-02 Thread Liu Bo
to the same data. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 55dda87..a7aeecc 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2229,7 +2229,7

Re: [PATCH] Btrfs: allow compressed extents to be merged during defragment

2013-08-02 Thread Liu Bo
On Fri, Aug 02, 2013 at 04:11:16PM +0800, Miao Xie wrote: On fri, 2 Aug 2013 14:49:54 +0800, Liu Bo wrote: The rule originally comes from nocow writing, but snapshot-aware defrag is a different case, the extent has been writen and we're not going to change the extent but add a reference

[PATCH v2] Btrfs: allow compressed extents to be merged during defragment

2013-08-02 Thread Liu Bo
to the same data. Reviewed-by: Miao Xie mi...@cn.fujitsu.com Signed-off-by: Liu Bo bo.li@oracle.com --- v1-v2: - pass the whole struct instead to keep the code simple. fs/btrfs/inode.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/inode.c

  1   2   3   4   5   6   7   8   9   10   >