[PATCH 3/8] Btrfs: kill unused argument of update_block_group

2012-12-27 Thread Liu Bo
Argument 'trans' is not used any more. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c | 12 +--- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index b57c233..6f71a7a 100644 --- a/fs/btrfs/extent

[PATCH 4/8] Btrfs: kill unused argument of btrfs_pin_extent_for_log_replay

2012-12-27 Thread Liu Bo
Argument 'trans' is not used any more. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ctree.h |3 +-- fs/btrfs/extent-tree.c |3 +-- fs/btrfs/tree-log.c|3 +-- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH 5/8] Btrfs: use token to avoid times mapping extent buffer

2012-12-27 Thread Liu Bo
The API in tree log code has done sort of changes, and it proves that we can benifit from using token, so do the same thing here. function_graph tracer's timer shows that it costs nearly half time of before(39.788us - 22.391us). Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c

[PATCH 6/8] Btrfs: save us a read_lock

2012-12-27 Thread Liu Bo
This does not change the logic of code, but can save us a read_lock. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/locking.c |5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c index 2a1762c..e95df43 100644 --- a/fs/btrfs

[PATCH 7/8] Btrfs: record first logical byte in memory

2012-12-27 Thread Liu Bo
This'd save us a rbtree search which may become expensive in large filesystem. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ctree.h |1 + fs/btrfs/disk-io.c |1 + fs/btrfs/extent-tree.c | 19 ++- 3 files changed, 20 insertions(+), 1 deletions

[PATCH 8/8] Btrfs: let allocation start from the right raid type

2012-12-27 Thread Liu Bo
This'd avoid us empty looping. Say we have only one disk and the metadata raid type will be defaultly DUP, and we do not need to start from index=0(RAID10) and get over two empty loops to index=2(DUP). Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c |2 +- 1 files

[PATCH] Btrfs: reset path lock state to zero

2012-12-28 Thread Liu Bo
We forgot to reset the path lock state to zero after we unlock the path block, and this can lead to the ASSERT checker in tree unlock API. Reported-by: Slava Barinov raysl...@gmail.com Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c |2 ++ 1 files changed, 2 insertions

Re: BUG at fs/btrfs/locking.c:265

2012-12-28 Thread Liu Bo
Hi Slava, David, I've sent a patch to fix the bug(Cced you), could you please check if it works? :) thanks, liubo On Thu, Dec 27, 2012 at 02:24:37PM +0100, David Sterba wrote: On Thu, Dec 27, 2012 at 04:07:31PM +0800, Liu Bo wrote: On Thu, Dec 27, 2012 at 08:27:11AM +0400, Slava Barinov

Re: [PATCH] Btrfs: reset path lock state to zero

2012-12-29 Thread Liu Bo
On Fri, Dec 28, 2012 at 12:32:25PM -0600, Mitch Harder wrote: On Fri, Dec 28, 2012 at 3:33 AM, Liu Bo bo.li@oracle.com wrote: We forgot to reset the path lock state to zero after we unlock the path block, and this can lead to the ASSERT checker in tree unlock API. Reported

[PATCH] Btrfs: use right range to find checksum for compressed extents

2013-01-05 Thread Liu Bo
For compressed extents, the range of checksum is covered by disk length, and the disk length is different with ram length, so we need to use disk length instead to get us the right checksum. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/tree-log.c |5 + 1 files changed, 5

[PATCH] xfstests: fix 284 ENOENT error

2013-01-06 Thread Liu Bo
284 misses a 'mkdir' operation. Signed-off-by: Liu Bo liub.li...@gmail.com --- 284 |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/284 b/284 index c00e6f1..7f4a630 100644 --- a/284 +++ b/284 @@ -44,6 +44,7 @@ _create_file() if [ $1 -ne 2

[PATCH] Btrfs: fix off-by-one in lseek

2013-01-06 Thread Liu Bo
Lock end is inclusive. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 77061bf..1e16b6d 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -2241,6 +2241,7 @@ static

[PATCH] Btrfs: fix a bug when llseek for delalloc bytes behind prealloc extents

2013-01-07 Thread Liu Bo
xfstests case 285 complains. It it because btrfs did not try to find unwritten delalloc bytes(only dirty pages, not yet writeback) behind prealloc extents, it ends up finding nothing while we're with SEEK_DATA. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file.c |9 ++--- fs

Re: Open for contribution towards xfstests for btrfs

2013-01-07 Thread Liu Bo
On Mon, Jan 07, 2013 at 03:48:43PM +0530, Kiran Patil wrote: Hello, We have a team of 5 students who would like to contribute to btrfs filesystem testing using xfstests. Is there space for them to contribute? xfstests is lacking of testcases targetting for btrfs send/recieve, you may be

Re: [PATCH] Btrfs: fix off-by-one in lseek

2013-01-07 Thread Liu Bo
On Mon, Jan 07, 2013 at 05:20:50PM +0100, David Sterba wrote: On Mon, Jan 07, 2013 at 11:53:08AM +0800, Liu Bo wrote: Lock end is inclusive. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs

Re: chattr +C vs. btrfs subvolume snapshot

2013-01-07 Thread Liu Bo
On Tue, Jan 08, 2013 at 12:37:11AM +, Jun Lion wrote: What happens if you set an individual file inside a subvolume as nocow (chattr +C) and then take a snapshot of that subvolume and modify the file in both? Will btrfs now ignore the nocow attribute completely or will it do as few

[PATCH 1/2] Btrfs: add leak debug for extent map

2013-01-08 Thread Liu Bo
This is for detecting extent map leak. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent_map.c | 31 +++ fs/btrfs/extent_map.h |1 + 2 files changed, 32 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c index

[PATCH 2/2] Btrfs: fix memory leak on extent map after fsync

2013-01-08 Thread Liu Bo
, the memory leak has gone away. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent_map.c |5 +++-- fs/btrfs/extent_map.h |4 ++-- fs/btrfs/tree-log.c | 12 +--- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent_map.c b/fs/btrfs

Re: [PATCH] Btrfs: fix off-by-one in lseek

2013-01-08 Thread Liu Bo
On Tue, Jan 08, 2013 at 06:26:27PM +0100, David Sterba wrote: On Tue, Jan 08, 2013 at 09:30:54AM +0100, David Sterba wrote: On Tue, Jan 08, 2013 at 10:46:38AM +0800, Liu Bo wrote: Hmm, not always here, lockend = inode-i_size - 1, so lockend % 2 == 1 may not be true. Yeah, that's

Re: [PATCH 1/2] Btrfs: add leak debug for extent map

2013-01-09 Thread Liu Bo
On Tue, Jan 08, 2013 at 12:07:34PM -0800, Zach Brown wrote: This is for detecting extent map leak. Hmm, I guess it's cool to get the allocation-specific decoding which you don't get from the generic kernel leak tracking? Hi Zach, Thanks for the advice, but what allocation-specific decoding

Re: [PATCH] Btrfs: fix off-by-one in lseek

2013-01-09 Thread Liu Bo
On Wed, Jan 09, 2013 at 12:50:25PM +0100, David Sterba wrote: On Wed, Jan 09, 2013 at 12:34:45PM +0800, Liu Bo wrote: [20191.948060] D: __set_extent_bit isize = 0 odd range [774144,7384799041917984768) [20191.956581] D: clear_extent_bit isize = 0 odd range

Re: [PATCH 1/2] Btrfs: add leak debug for extent map

2013-01-10 Thread Liu Bo
On Thu, Jan 10, 2013 at 12:54:26PM +0100, David Sterba wrote: On Thu, Jan 10, 2013 at 10:05:39AM +0800, Liu Bo wrote: On Tue, Jan 08, 2013 at 12:07:34PM -0800, Zach Brown wrote: This is for detecting extent map leak. Hmm, I guess it's cool to get the allocation-specific decoding

Re: [PATCH] Btrfs: fix off-by-one in lseek

2013-01-10 Thread Liu Bo
On Thu, Jan 10, 2013 at 12:04:58PM +0100, David Sterba wrote: On Thu, Jan 10, 2013 at 10:17:49AM +0800, Liu Bo wrote: Thanks for the report, could you please show me what options you're using? Default mkfs, mount options 'space_cache,noatime', 40G test partition, 10G scratch partition (ie

Re: [PATCH 1/2] Btrfs: add leak debug for extent map

2013-01-11 Thread Liu Bo
On Thu, Jan 10, 2013 at 09:06:34AM -0800, Zach Brown wrote: Hmm, I guess it's cool to get the allocation-specific decoding which you don't get from the generic kernel leak tracking? I mean that by doing this in btrfs, instead of doing it generically in the allocator, you get specific

Re: [PATCH 1/2] Btrfs: add leak debug for extent map

2013-01-13 Thread Liu Bo
On Fri, Jan 11, 2013 at 12:54:32PM -0800, Zach Brown wrote: But after flipping slab code, I find that another callback will disable merging slabs when allocating a slab, so I'm not sure if it worth doing so... Do you mean the find_mergeable() stuff in SLUB? Yes, that's what I'm worried

Re: [PATCH] btrfs: update timestamps on truncate()

2013-01-13 Thread Liu Bo
On Fri, Jan 11, 2013 at 08:57:22PM -0600, Eric Sandeen wrote: truncate() vs. ftruncate() differ in the VFS; truncate() doesn't set (ATTR_CTIME | ATTR_MTIME), and it's up to the fs to do the timestamp updates if the size changes. Signed-off-by: Eric Sandeen sand...@redhat.com --- Hm, am I

[PATCH] mm/slab: add a leak decoder callback

2013-01-13 Thread Liu Bo
. So adding a callback for leak tracking can avoid this as well as runtime overhead. Signed-off-by: Liu Bo bo.li@oracle.com --- The idea is from Zach Brown z...@zabbo.net. fs/btrfs/extent_io.c | 24 fs/btrfs/extent_map.c| 12 include/linux

Re: [PATCH 10/11] Btrfs: use bit operation for -fs_state

2013-01-14 Thread Liu Bo
On Mon, Jan 14, 2013 at 03:50:31PM +0800, Miao Xie wrote: Onthu, 10 Jan 2013 18:57:35 +0100, David Sterba wrote: On Thu, Jan 10, 2013 at 08:51:59PM +0800, Miao Xie wrote: There is no lock to protect fs_info-fs_state, it will introduce some problems, such as the value may be covered

Re: [PATCH] mm/slab: add a leak decoder callback

2013-01-15 Thread Liu Bo
On Tue, Jan 15, 2013 at 04:30:52PM +, Christoph Lameter wrote: On Mon, 14 Jan 2013, Liu Bo wrote: This adds a leak decoder callback so that kmem_cache_destroy() can use to generate debugging output for the allocated objects. Interesting idea. @@ -3787,6 +3789,9 @@ static int

Re: [PATCH] mm/slab: add a leak decoder callback

2013-01-15 Thread Liu Bo
On Tue, Jan 15, 2013 at 09:01:05AM -0800, Zach Brown wrote: The merge processing occurs during kmem_cache_create and you are setting up the decoder field afterwards! Wont work. In the thread I suggested providing the callback at destruction:

Re: [PATCH] Btrfs: fix crash of starting balance

2013-01-15 Thread Liu Bo
On Tue, Jan 15, 2013 at 06:59:04PM +0200, Ilya Dryomov wrote: On Tue, Jan 15, 2013 at 10:47:57PM +0800, Liu Bo wrote: We will crash on BUG_ON(ret == -EEXIST) when we do not resume the existing balance but attempt to start a new one. The steps can be: 1. start balance 2. pause balance

[PATCH V2] mm/slab: add a leak decoder callback

2013-01-15 Thread Liu Bo
adding a callback for leak tracking can avoid this as well as runtime overhead. (The idea is from Zach Brown z...@zabbo.net.) Signed-off-by: Liu Bo bo.li@oracle.com --- v2: add a wrapper API for slab destruction to make decoder only work in particular path. fs/btrfs/extent_io.c | 26

Re: [PATCH 1/2] Btrfs: fix wrong max device number for single profile

2013-01-16 Thread Liu Bo
On Wed, Jan 16, 2013 at 07:27:17PM +0800, Miao Xie wrote: The max device number of single profile is 1, not 0 (0 means 'as many as possible'). Fix it. Reviewed-by: Liu Bo bo.li@oracle.com Cc: Liu Bo bo.li@oracle.com Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs

[PATCH V5] Btrfs: snapshot-aware defrag

2013-01-16 Thread Liu Bo
. Signed-off-by: Li Zefan l...@cn.fujitsu.com Signed-off-by: Liu Bo bo.li@oracle.com --- v4-v5: - Clarify the comments for duplicated refs. - Clear defrag flag after we're ready to defrag. - Fix a bug on HOLE extent. v3-v4: - Fix duplicated refs bugs detected by mounting

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-17 Thread Liu Bo
On Thu, Jan 17, 2013 at 08:42:46AM -0600, Mitch Harder wrote: On Wed, Jan 16, 2013 at 6:36 AM, Liu Bo bo.li@oracle.com wrote: This comes from one of btrfs's project ideas, As we defragment files, we break any sharing from other snapshots. The balancing code will preserve the sharing

Re: [PATCH V2] mm/slab: add a leak decoder callback

2013-01-17 Thread Liu Bo
On Thu, Jan 17, 2013 at 05:34:46PM +0900, Joonsoo Kim wrote: Hello, Liu Bo. On Wed, Jan 16, 2013 at 11:03:13AM +0800, Liu Bo wrote: This adds a leak decoder callback so that slab destruction can use to generate debugging output for the allocated objects. Callers like btrfs are using

Re: [PATCH V2] mm/slab: add a leak decoder callback

2013-01-17 Thread Liu Bo
On Wed, Jan 16, 2013 at 03:20:57PM +, Christoph Lameter wrote: On Wed, 16 Jan 2013, Liu Bo wrote: --- a/include/linux/slub_def.h +++ b/include/linux/slub_def.h @@ -93,6 +93,7 @@ struct kmem_cache { gfp_t allocflags; /* gfp flags to use on each alloc */ int refcount

Re: [PATCH V2] mm/slab: add a leak decoder callback

2013-01-17 Thread Liu Bo
On Wed, Jan 16, 2013 at 01:34:38PM +0800, Miao Xie wrote: On wed, 16 Jan 2013 11:03:13 +0800, Liu Bo wrote: This adds a leak decoder callback so that slab destruction can use to generate debugging output for the allocated objects. Callers like btrfs are using their own leak tracking

Re: A couple bugs with btrfs and 3.5.0 kernel

2013-01-20 Thread Liu Bo
On Sun, Jan 20, 2013 at 05:39:57PM -0800, Elladan wrote: I upgraded to Ubuntu 12.10 and thought, Hey, that 3.5 kernel is relatively recent. And they seem to finally have implemented restriping. Maybe it's time to try btrfs again! So, first off, I backed up all my data. Next, I decided I

Re: [PATCH 1/5] Btrfs: fix repeated delalloc work allocation

2013-01-22 Thread Liu Bo
On Tue, Jan 22, 2013 at 06:49:00PM +0800, Miao Xie wrote: btrfs_start_delalloc_inodes() locks the delalloc_inodes list, fetches the first inode, unlocks the list, triggers btrfs_alloc_delalloc_work/ btrfs_queue_worker for this inode, and then it locks the list, checks the head of the list

Re: [GIT PULL] Btrfs fixes

2013-01-22 Thread Liu Bo
into this pull. It isn't nailed down yet, but we were finally able to get a solid way to reproduce. The only good news is it isn't a recent regression. The most important batch of fixes in here come from Ilya. They address a regression Liu Bo found in the balance ioctls for pausing

Re: [PATCH 1/5] Btrfs: fix repeated delalloc work allocation

2013-01-22 Thread Liu Bo
On Wed, Jan 23, 2013 at 10:54:39AM +0800, Miao Xie wrote: On Tue, 22 Jan 2013 22:24:15 +0800, Liu Bo wrote: On Tue, Jan 22, 2013 at 06:49:00PM +0800, Miao Xie wrote: btrfs_start_delalloc_inodes() locks the delalloc_inodes list, fetches the first inode, unlocks the list, triggers

Re: [bug] csum mismatches and failed xfstests with 3.8-rc1 -rc4

2013-01-22 Thread Liu Bo
On Tue, Jan 22, 2013 at 09:39:02AM -0500, Chris Mason wrote: On Tue, Jan 22, 2013 at 07:26:15AM -0700, David Sterba wrote: On Fri, Jan 04, 2013 at 01:50:59PM +0100, David Sterba wrote: I've noticed a few csum mismatch messages, and a few failed xfstests: They're still there, we're on

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-22 Thread Liu Bo
On Tue, Jan 22, 2013 at 11:41:19AM -0600, Mitch Harder wrote: On Thu, Jan 17, 2013 at 8:42 AM, Mitch Harder mitch.har...@sabayonlinux.org wrote: On Wed, Jan 16, 2013 at 6:36 AM, Liu Bo bo.li@oracle.com wrote: This comes from one of btrfs's project ideas, As we defragment files, we

Re: [PATCH 1/5] Btrfs: fix repeated delalloc work allocation

2013-01-23 Thread Liu Bo
On Wed, Jan 23, 2013 at 02:33:27PM +0800, Miao Xie wrote: On wed, 23 Jan 2013 14:06:21 +0800, Liu Bo wrote: On Wed, Jan 23, 2013 at 12:44:49PM +0800, Miao Xie wrote: No, we can't. The other tasks which flush the delalloc data may remove the inode from the delalloc list/splice list. If we

Re: [PATCH 1/5] Btrfs: fix repeated delalloc work allocation

2013-01-23 Thread Liu Bo
On Wed, Jan 23, 2013 at 04:58:27PM +0800, Miao Xie wrote: On wed, 23 Jan 2013 16:17:53 +0800, Liu Bo wrote: On Wed, Jan 23, 2013 at 02:33:27PM +0800, Miao Xie wrote: On wed, 23 Jan 2013 14:06:21 +0800, Liu Bo wrote: On Wed, Jan 23, 2013 at 12:44:49PM +0800, Miao Xie wrote: No, we can't

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-23 Thread Liu Bo
On Wed, Jan 23, 2013 at 10:05:04AM -0600, Mitch Harder wrote: On Wed, Jan 23, 2013 at 1:51 AM, Liu Bo bo.li@oracle.com wrote: On Tue, Jan 22, 2013 at 11:41:19AM -0600, Mitch Harder wrote: On Thu, Jan 17, 2013 at 8:42 AM, Mitch Harder mitch.har...@sabayonlinux.org wrote: On Wed, Jan

Re: unmountable filesystem with kernel 3.7.3

2013-01-23 Thread Liu Bo
On Wed, Jan 23, 2013 at 08:14:44PM +0100, F. Frederich wrote: Hi, os: Gentoo kernel: 3.7.3 from kernel.org 2 harddisks with btrfs and Raid1, lzo compression trying to start a virtual machine (qemu-kvm) the system crashed giving following messages: kernel: [177417.378526] [

Re: [PATCH 2/2] Btrfs: fix memory leak on extent map after fsync

2013-01-24 Thread Liu Bo
On Thu, Jan 24, 2013 at 11:44:33AM -0500, Josef Bacik wrote: On Tue, Jan 08, 2013 at 07:49:21AM -0700, Liu Bo wrote: During fsync, we put the changed parts(i.e. extent map) into the log tree, and we ship these parts from a list of modified_extents to a local list to process, of course, we

Re: [PATCH 2/2] Btrfs: fix memory leak on extent map after fsync

2013-01-24 Thread Liu Bo
On Fri, Jan 25, 2013 at 09:38:06AM +0800, Liu Bo wrote: On Thu, Jan 24, 2013 at 11:44:33AM -0500, Josef Bacik wrote: On Tue, Jan 08, 2013 at 07:49:21AM -0700, Liu Bo wrote: During fsync, we put the changed parts(i.e. extent map) into the log tree, and we ship these parts from a list

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-25 Thread Liu Bo
On Fri, Jan 25, 2013 at 08:55:58AM -0600, Mitch Harder wrote: On Wed, Jan 23, 2013 at 6:52 PM, Liu Bo bo.li@oracle.com wrote: On Wed, Jan 23, 2013 at 10:05:04AM -0600, Mitch Harder wrote: On Wed, Jan 23, 2013 at 1:51 AM, Liu Bo bo.li@oracle.com wrote: On Tue, Jan 22, 2013 at 11:41

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-27 Thread Liu Bo
On Fri, Jan 25, 2013 at 12:16:29PM -0600, Mitch Harder wrote: [...] I've changed up my reproducer to try some things that may hit the issue quicker and more reliably. It gave me a slightly different set of warnings in dmesg, which seem to suggest issues in the dead_root list.

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-01-27 Thread Liu Bo
On Sun, Jan 27, 2013 at 11:20:41PM -0600, Mitch Harder wrote: On Sun, Jan 27, 2013 at 6:41 AM, Liu Bo bo.li@oracle.com wrote: Hi Mitch, Many thanks for testing it! Well, after some debugging, I finally figure out the whys: (1) btrfs_ioctl_snap_destroy() will free the inode

Re: [PATCH] Btrfs: fix EDQUOT handling in btrfs_delalloc_reserve_metadata

2013-01-28 Thread Liu Bo
, the error handling code didn't take into account that we dropped the inode lock which might require more cleanup. Luckily, all the cleanup code we need is already there and can be shared with reserve_metadata_bytes, which is exactly what this patch does. Reviewed-by: Liu Bo bo.li@oracle.com

[PATCH] Btrfs: fix race between snapshot deletion and getting inode

2013-01-28 Thread Liu Bo
. Reported-by: Mitch Harder mitch.har...@sabayonlinux.org Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file.c | 12 fs/btrfs/scrub.c | 25 - 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index

Re: [PATCH] Btrfs: fix race between snapshot deletion and getting inode

2013-01-28 Thread Liu Bo
On Mon, Jan 28, 2013 at 06:48:24PM -0600, Mitch Harder wrote: On Mon, Jan 28, 2013 at 5:04 AM, Liu Bo bo.li@oracle.com wrote: While running snapshot testscript created by Mitch and David, the race between autodefrag and snapshot deletion can lead to corruption of dead_root list so

Re: [PATCH] Btrfs: fix a deadlock on chunk mutex

2013-01-28 Thread Liu Bo
On Mon, Jan 28, 2013 at 04:23:31PM -0500, Josef Bacik wrote: On Thu, Jan 03, 2013 at 11:44:46AM -0700, Jim Schutt wrote: Hi Josef, Thanks for the patch - sorry for the long delay in testing... Jim, I've been trying to reason out how this happens, could you do a btrfs fi df on the

Re: [PATCH] Btrfs: do not merge logged extents if we've removed them from the tree

2013-01-28 Thread Liu Bo
it. The problem is we unconditionally try to merge it back into the em tree, but if we've removed it from the em tree that will cause problems. So fix this by only trying to merge it in if it is still a part of the tree. Thanks, Reviewed-by: Liu Bo bo.li@oracle.com Signed-off-by: Josef Bacik jba

Re: [PATCH] Btrfs: fix race between snapshot deletion and getting inode

2013-01-28 Thread Liu Bo
On Mon, Jan 28, 2013 at 11:21:27PM +0100, David Sterba wrote: On Mon, Jan 28, 2013 at 07:04:02PM +0800, Liu Bo wrote: While running snapshot testscript created by Mitch and David, the race between autodefrag and snapshot deletion can lead to corruption of dead_root list so that we can get

[PATCH v6] Btrfs: snapshot-aware defrag

2013-01-28 Thread Liu Bo
. Signed-off-by: Li Zefan l...@cn.fujitsu.com Signed-off-by: Liu Bo bo.li@oracle.com --- v5-v6: - Fix double list add of dead_roots crash reported by Mitch v4-v5: - Clarify the comments for duplicated refs. - Clear defrag flag after we're ready to defrag. - Fix a bug on HOLE

[PATCH v2] Btrfs: fix race between snapshot deletion and getting inode

2013-01-28 Thread Liu Bo
. Reported-by: Mitch Harder mitch.har...@sabayonlinux.org Signed-off-by: Liu Bo bo.li@oracle.com --- v1-v2: Address nice cleanup comments from David Sterba. fs/btrfs/file.c | 22 ++ fs/btrfs/scrub.c | 25 - 2 files changed, 38 insertions(+), 9

Re: [RFC][PATCH 2/2] Btrfs: implement unlocked dio write

2013-01-31 Thread Liu Bo
On Thu, Jan 31, 2013 at 05:39:03PM +0800, Miao Xie wrote: This idea is from ext4. By this patch, we can make the dio write parallel, and improve the performance. Interesting, AFAIK, ext4 can only do nolock dio write on some conditions(should be a overwrite, file size remains unchanged, no

[PATCH] Btrfs: extend the checksum item as much as possible

2013-02-04 Thread Liu Bo
in my test. The idea behind is that it can reduce the times we insert/extend so that it saves us precious reserved space. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file-item.c | 67 ++--- 1 files changed, 46 insertions(+), 21 deletions

Re: [PATCH] btrfs: add delayed_iput list head to btrfs inode

2013-02-05 Thread Liu Bo
On Tue, Feb 05, 2013 at 03:14:05PM -0800, Zach Brown wrote: + struct btrfs_inode *b_inode = BTRFS_I(inode); + struct btrfs_fs_info *fs_info = b_inode-root-fs_info; if (atomic_add_unless(inode-i_count, -1, 1)) return; - delayed = kmalloc(sizeof(*delayed),

Re: btrfs wastes disk space after snapshot deletetion.

2013-02-05 Thread Liu Bo
On Tue, Feb 05, 2013 at 05:27:45PM +0200, Moshe Melnikov wrote: Is it possible in step 1) to create few smaller extents instead of 1 gig data extent? DIO or O_SYNC can help to create extents whose size is your 'bs=xxx', but you know, this is not expected as fast as buffered write. thanks,

Re: btrfs wastes disk space after snapshot deletetion.

2013-02-05 Thread Liu Bo
On Mon, Feb 04, 2013 at 11:08:01AM +0200, Moshe wrote: Hello, If I write large sequential file on snapshot, then create another snapshot, overwrite file with small amount of data and delete first snapshot, second snapshot has very large data extent and only small part of it is used. For

Re: [PATCH] btrfs: add delayed_iput list head to btrfs inode

2013-02-06 Thread Liu Bo
On Wed, Feb 06, 2013 at 09:53:05AM -0600, Eric Sandeen wrote: On 2/5/13 8:08 PM, Liu Bo wrote: On Tue, Feb 05, 2013 at 03:14:05PM -0800, Zach Brown wrote: + struct btrfs_inode *b_inode = BTRFS_I(inode); + struct btrfs_fs_info *fs_info = b_inode-root-fs_info; if (atomic_add_unless

[PATCH] Btrfs: cleanup the left refs on the cluster list

2013-02-08 Thread Liu Bo
Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/extent-tree.c |6 ++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 61da9d0..7e7884f 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2500,6 +2500,12

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-02-15 Thread Liu Bo
On Mon, Jan 28, 2013 at 05:55:57PM +0100, Stefan Behrens wrote: [CC list reduced (my initial statement was that such dead_list corruptions happen without the snapshot-aware defrag patch, by now the contents is not related to the snapshot-aware defrag patch anymore)] [...] No, this did not

[PATCH] Btrfs-progs: fix segmentation fault of 'btrfs-debug-tree -e'

2013-02-18 Thread Liu Bo
Due to some historical reasons, we remove 'printing leaf' part, which'd lead to 'Segmentation fault' of btrfs-debug-tree -e, this patch adds it back. Signed-off-by: Liu Bo bo.li@oracle.com --- debug-tree.c |5 + 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/debug

Re: [PATCH V5] Btrfs: snapshot-aware defrag

2013-02-18 Thread Liu Bo
On Mon, Feb 18, 2013 at 05:53:50PM +0100, Stefan Behrens wrote: On Sat, 16 Feb 2013 14:47:45 +0800, Liu Bo wrote: What about this patch(UNTESTED)? thanks, liubo diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ca7ace7..dac9d4b 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs

Re: Snapshot Cleaner not Working with inode_cache

2013-02-19 Thread Liu Bo
On Tue, Feb 19, 2013 at 02:22:27PM -0600, Mitch Harder wrote: I've encountered an issue where the space from previously deleted snapshots is not being freed up by the cleaner thread. I'm only encountering this issue when I mount with the inode_cache option. I've reproduced this on a 3.7.9

[PATCH] Btrfs: fix cleaner thread not working with inode cache option

2013-02-20 Thread Liu Bo
mitch.har...@sabayonlinux.org Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c |3 ++- fs/btrfs/ioctl.c |6 ++ 2 files changed, 8 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ca7ace7..d9984fa 100644 --- a/fs/btrfs/inode.c +++ b/fs

Re: [RFC] Btrfs: Allow the compressed extent size limit to be modified v2

2013-02-20 Thread Liu Bo
On Thu, Feb 07, 2013 at 03:38:34PM -0600, Mitch Harder wrote: Provide for modification of the limit of compressed extent size utilizing mount-time configuration settings. The size of compressed extents was limited to 128K, which leads to fragmentation of the extents (although the extents

Re: [PATCH 1/3] Btrfs: fix the qgroup reserved space is released prematurely

2013-02-20 Thread Liu Bo
On Wed, Feb 20, 2013 at 05:13:32PM +0800, Miao Xie wrote: In start_transactio(), we will try to join the transaction again after the current transaction is committed, so we should not release the reserved space of the qgroup. Fix it. Why do we release the reserved space of the qgroup? join |

Re: LAST CALL FOR BTRFS-NEXT

2013-02-20 Thread Liu Bo
On Wed, Feb 20, 2013 at 09:26:12AM -0500, Josef Bacik wrote: Hello, If you have patches in btrfs-next, make sure they are the right ones so we don't have to rebase and repull and merge and all that crap for this merge window. If I'm missing stuff that's fine, I can throw that on top,

Re: Another defrag question

2013-02-21 Thread Liu Bo
On Thu, Feb 21, 2013 at 04:46:14PM +0100, Swâmi Petaramesh wrote: Hi again, Having numerous snapshots, I prefer to ask rather than take the risk of exploding my storage space, better safe than sorry ;-) man btrfs states : « NOTE: defragmenting with kernels up to 2.6.37 will unlink

Re: [PATCH] Btrfs: make sure NODATACOW also gets NODATASUM set

2013-02-21 Thread Liu Bo
nodatacow and then write into preallocated space, or chattr +C a directory and move a file into that directory. This patch fixes up those two cases and now we should uniformly have NODATACOW and NODATASUM set always. Thanks, Reviewed-by: Liu Bo bo.li@oracle.com Signed-off-by: Josef Bacik

[PATCH] Btrfs: use reserved space for creating a snapshot

2013-02-21 Thread Liu Bo
. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/transaction.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index fc03aa6..5878bb4 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1063,6 +1063,7

Re: bug: per file cow flag is lost when renaming

2013-02-21 Thread Liu Bo
On Mon, Jan 28, 2013 at 10:49:20AM -0500, Marios Titas wrote: Try this: touch test chattr +C test lsattr test mv test test2 lsattr test2 The original file (test) will have the C flag but when renamed the flag disappears. If the volume is unmounted and then mounted

Re: [PATCH] Btrfs: make sure NODATACOW also gets NODATASUM set

2013-02-21 Thread Liu Bo
On Fri, Feb 22, 2013 at 01:34:26AM -0500, Marios Titas wrote: A few weeks ago I reported a similar bug [1]. It has to do with file renaming. Do you think that it is related? This patch doesn't seem to help. Yes, they're related, and I sent you a patch in [1] thread, could you check it?

[PATCH] Btrfs: update inode flags when renaming

2013-02-22 Thread Liu Bo
flags, Reported-by: Marios Titas redneb8...@gmail.com Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c |7 +-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d9984fa..d2e3352 100644 --- a/fs/btrfs/inode.c +++ b/fs

Re: [PATCH] Btrfs: update inode flags when renaming

2013-02-22 Thread Liu Bo
on COW mode. So the failure of 'clone' does not equal to the file is NODATACOW. Feel free to correct me. thanks, liubo Thanks On Fri, Feb 22, 2013 at 3:40 AM, Liu Bo bo.li@oracle.com wrote: On Fri, Feb 22, 2013 at 03:32:50AM -0500, Marios Titas wrote: Sorry, but the bug persists

Re: Snapshot Cleaner not Working with inode_cache

2013-02-22 Thread Liu Bo
On Fri, Feb 22, 2013 at 11:16:22AM +0100, Norbert Scheibner wrote: Am 20.02.2013, 02:14 Uhr, schrieb Liu Bo bo.li@oracle.com: I think I know why inode_cache keeps us from freeing space, inode_cache adds a cache_inode in each btrfs root, and this cache_inode will be iput at the very

Re: [PATCH] Btrfs: update inode flags when renaming

2013-02-24 Thread Liu Bo
On Fri, Feb 22, 2013 at 11:04:40PM +0100, David Sterba wrote: On Fri, Feb 22, 2013 at 05:34:47PM +0800, Miao Xie wrote: On fri, 22 Feb 2013 16:40:35 +0800, Liu Bo wrote: On Fri, Feb 22, 2013 at 03:32:50AM -0500, Marios Titas wrote: Sorry, but the bug persists even with the above patch

[PATCH] Btrfs: do not change inode flags in rename

2013-02-24 Thread Liu Bo
...@gmail.com Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c | 25 - 1 files changed, 0 insertions(+), 25 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d9984fa..383a7d8 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7325,29 +7325,6

Re: [PATCH] Btrfs: update inode flags when renaming

2013-02-24 Thread Liu Bo
On Mon, Feb 25, 2013 at 12:23:03PM +0800, Miao Xie wrote: Onmon, 25 Feb 2013 11:50:01 +0800, Liu Bo wrote: On Fri, Feb 22, 2013 at 11:04:40PM +0100, David Sterba wrote: On Fri, Feb 22, 2013 at 05:34:47PM +0800, Miao Xie wrote: Onfri, 22 Feb 2013 16:40:35 +0800, Liu Bo wrote

Re: [PATCH] Btrfs: do not change inode flags in rename

2013-02-25 Thread Liu Bo
(add Miao to CC) On Mon, Feb 25, 2013 at 12:04:42PM +0800, Liu Bo wrote: Before we forced to change a file's NOCOW and COMPRESS flag due to the parent directory's, but this ends up a bad idea, because it confuses end users a lot about file's NOCOW status, eg. if someone change a file to NOCOW

Re: [PATCH] Btrfs: do not change inode flags in rename

2013-02-25 Thread Liu Bo
On Mon, Feb 25, 2013 at 01:56:47PM -0500, Josef Bacik wrote: On Sun, Feb 24, 2013 at 09:04:42PM -0700, Liu Bo wrote: Before we forced to change a file's NOCOW and COMPRESS flag due to the parent directory's, but this ends up a bad idea, because it confuses end users a lot about file's NOCOW

Re: WARNING: at fs/btrfs/inode.c:2165 btrfs_orphan_commit_root+0xcb/0xdf()

2013-02-25 Thread Liu Bo
On Sun, Feb 24, 2013 at 06:55:46PM -0800, Marc MERLIN wrote: Is this useful to anyone? Hi Marc, Thanks for the report, of course they're useful. Could you please also show us your workloads and it'd be better to know how to reproduce this? thanks, liubo Got this after a crash/reboot:

Re: WARNING: at fs/btrfs/inode.c:2165 btrfs_orphan_commit_root+0xcb/0xdf()

2013-02-26 Thread Liu Bo
On Tue, Feb 26, 2013 at 09:20:43AM -0500, Josef Bacik wrote: On Sun, Feb 24, 2013 at 07:55:46PM -0700, Marc MERLIN wrote: Is this useful to anyone? Got this after a crash/reboot: if (block_rsv) { WARN_ON(block_rsv-size 0); btrfs_free_block_rsv(root,

[PATCH 0/2] bug fixes for error handling

2013-02-27 Thread Liu Bo
These two fixes are for error handling after we abort a transaction. One is a memory leak fix, and the other one is a NULL pointer fix. Liu Bo (2): Btrfs: fix memory leak of log roots Btrfs: fix NULL pointer after aborting a transaction fs/btrfs/disk-io.c |5 + fs/btrfs

[PATCH 1/2] Btrfs: fix memory leak of log roots

2013-02-27 Thread Liu Bo
When we abort a transaction while fsyncing, we'll skip freeing log roots part of committing a transaction, which leads to memory leak. This adds a 'free log roots' in putting super when no more users hold references on log roots, so it's safe and clean. Signed-off-by: Liu Bo bo.li@oracle.com

[PATCH 2/2] Btrfs: fix NULL pointer after aborting a transaction

2013-02-27 Thread Liu Bo
transaction handle joining, then waits other existing handles to abort or finish so that we can safely set the above global pointer to NULL. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/transaction.c |8 +++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/fs/btrfs

Re: [GIT PULL] Btrfs

2013-03-02 Thread Liu Bo
orphan reservation if truncate fails (+2/-0) Btrfs: make sure NODATACOW also gets NODATASUM set (+2/-1) Btrfs: don't re-enter when allocating a chunk (+9/-0) Btrfs: remove unused extent io tree ops V2 (+11/-27) Btrfs: fix chunk allocation error handling (+22/-10) Liu Bo (14

Re: [PATCH 1/2] Btrfs: fix wrong handle at error path of create_snapshot() when the commit fails

2013-03-04 Thread Liu Bo
On Mon, Mar 04, 2013 at 05:44:29PM +0800, Miao Xie wrote: There are several bugs at error path of create_snapshot() when the transaction commitment failed. - access the freed transaction handler. At the end of the transaction commitment, the transaction handler was freed, so we should not

[PATCH 0/6] Several bugfixes

2013-03-04 Thread Liu Bo
Patch 1-5 can make balance process bail out gracefully after we get into readonly by aborting transaction. Patch 6 addresses a hang when we have two or more trans handle aborted and committed themselves. Liu Bo (6): Btrfs: check for NULL pointer in updating reloc roots Btrfs: build up error

[PATCH 1/6] Btrfs: check for NULL pointer in updating reloc roots

2013-03-04 Thread Liu Bo
Add a check for NULL pointer to avoid invalid reference. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/relocation.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 300e09a..d9be73b 100644 --- a/fs/btrfs

[PATCH 3/6] Btrfs: free all recorded tree blocks on error

2013-03-04 Thread Liu Bo
We've missed the 'free blocks' part on ENOMEM error. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/relocation.c |9 ++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 8608afb..8cae3d7 100644 --- a/fs/btrfs

[PATCH 4/6] Btrfs: do not BUG_ON in prepare_to_reloc

2013-03-04 Thread Liu Bo
We can bail out from here gracefully instead of a cold BUG_ON. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/relocation.c | 10 +- 1 files changed, 9 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 8cae3d7..c45c833 100644

<    5   6   7   8   9   10   11   12   13   14   >