[RFC PATCH 1/3] Btrfs: add the Probabilistic Skiplist

2012-01-05 Thread Liu Bo
is able to contain (2^32-1)/3 nodes at most. [1] http://www.csee.umbc.edu/courses/undergraduate/341/fall01/Lectures/SkipLists/skip_lists/skip_lists.html Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/Makefile |2 +- fs/btrfs/skiplist.c | 98 fs

[RFC PATCH 3/3] Btrfs: convert rwlock to RCU for extent_map

2012-01-05 Thread Liu Bo
extent_map reclaim after dropping the updater side lock. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/compression.c |8 +++--- fs/btrfs/disk-io.c | 15 ++ fs/btrfs/extent_io.c | 13 - fs/btrfs/extent_map.c | 39 +- fs/btrfs

[RFC PATCH 2/3] Btrfs: rebuild extent_map based on skiplist

2012-01-05 Thread Liu Bo
extent_map applies a read more senario, since we want to build a RCU-skiplist later, we build a new version extent_map based on skiplist firstly. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_map.c | 265 +++-- fs/btrfs

Re: [RFC PATCH 0/3] apply the Probabilistic Skiplist on btrfs

2012-01-06 Thread Liu Bo
On 01/06/2012 07:51 AM, David Sterba wrote: Hi, I've let it run through xfstests and ended at 091, patches applied on top of 3.2, mount options compress-force=lzo,discard,inode_cache,space_cache,autodefrag fresh mkfs with defaults. Hi David, Thanks a lot for your work! I also find this

[RFC PATCH v2 0/3] Btrfs: apply the Probabilistic Skiplist on btrfs

2012-01-09 Thread Liu Bo
, and I've tested them without my patches and found that they still fails on the upstream. --- Liu Bo (3): Btrfs: add the Probabilistic Skiplist Btrfs: rebuild extent_map based on skiplist Btrfs: convert rwlock to RCU for extent_map fs/btrfs/Makefile |2 +- fs/btrfs/compression.c |8

[RFC PATCH v2 3/3] Btrfs: convert rwlock to RCU for extent_map

2012-01-09 Thread Liu Bo
extent_map instead of rwlock. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- changes v2: - fix a bug reported by David Sterba d...@jikos.cz, thanks a lot! - use mutex lock to protect extent_map updater side, so that we can make the reclaim code much easier. --- fs/btrfs/compression.c |8

[RFC PATCH v2 2/3] Btrfs: rebuild extent_map based on skiplist

2012-01-09 Thread Liu Bo
extent_map applies a read more senario, since we want to build a RCU-skiplist later, we build a new version extent_map based on skiplist firstly. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_map.c | 258 - fs/btrfs

[PATCH] xfstests: fix 251's cp -axT problem

2012-01-10 Thread Liu Bo
When I ran xfstests, 251 got failed cause cp -axT did not work as wish: cp: cannot overwrite directory `/mnt/scratch/1' with non-directory With this patch, 251 has passed. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- 251 |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff

Re: [RFC PATCH v2 1/3] Btrfs: add the Probabilistic Skiplist

2012-01-10 Thread Liu Bo
On 01/11/2012 08:37 AM, David Sterba wrote: Hi, a few thoughts and comments below: On Tue, Jan 10, 2012 at 03:31:34PM +0800, Liu Bo wrote: c) The level limit may need to be adjusted. I know it is a magic number, but now for simplicity we just keep it at 16, and then each skiplist

Re: [PATCH] xfstests: fix 251's cp -axT problem

2012-01-11 Thread Liu Bo
On 01/11/2012 08:14 PM, Lukas Czerner wrote: On Wed, 11 Jan 2012, Christoph Hellwig wrote: On Tue, Jan 10, 2012 at 07:39:20PM +0800, Liu Bo wrote: When I ran xfstests, 251 got failed cause cp -axT did not work as wish: cp: cannot overwrite directory `/mnt/scratch/1' with non-directory

Re: [RFC PATCH v2 0/3] Btrfs: apply the Probabilistic Skiplist on btrfs

2012-01-12 Thread Liu Bo
On 01/13/2012 05:28 AM, Andi Kleen wrote: Liu Bo liubo2...@cn.fujitsu.com writes: Here we choose extent_map firstly, since it is a read mostly thing, and the change is quite direct, all we need to do is a) to replace rbtree with skiplist, b) to add rcu support. And more details are in patch

Re: Bug(?): btrfs carries on working if part of a device disappears

2012-01-13 Thread Liu Bo
On 01/06/2012 02:02 AM, Maik Zumstrull wrote: Hello list, I hit a funny BIOS bug the other day where the BIOS suddenly sets a HPA on a random hard disk, leaving only the first 33 MB accessible. That disk had one device of a multi-device btrfs on it in my case. (With dm-crypt/LUKS in

[PATCH] Btrfs: do not defrag a file partially

2012-01-16 Thread Liu Bo
xfstests 218 complains that btrfs defrags a file partially: After: 1 Write backwards sync, but contiguous - should defrag to 1 extent Before: 10 -After: 1 +After: 2 To fix this, we need to set max_to_defrag count properly. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c

[PATCH] Xfstest: make 218 support btrfs

2012-01-16 Thread Liu Bo
Btrfs progs has a defragment tool, so we can test 218 on btrfs now. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- 218 |2 +- common.defrag |3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/218 b/218 index 53d0b61..77782c6 100755 --- a/218 +++ b/218

[PATCH] Btrfs: hold enough space for global_rsv

2012-01-17 Thread Liu Bo
/btrfs/foobar: 66964 extents found and then btrfs fi defrag /mnt/btrfs/foobar sync would pop the warnings. I found that the global_rsv size is just not enough for defragment, and didn't find any space leak in using global_rsv, so double it and go ahead. Signed-off-by: Liu Bo liubo2

Re: [PATCH 3/3] xfstests: add btrfs online defragments QA test

2012-02-06 Thread Liu Bo
On 02/06/2012 08:11 PM, Christoph Hellwig wrote: On Mon, Feb 06, 2012 at 04:27:49PM +0800, Liu Bo wrote: +_runtest() +{ +$progs_dir/runtest.sh $defrag_args $SCRATCH_MNT $SCRATCH_DEV please put the actual tests into test cases themselves instead of calling out into shell scripts under

Re: Several unhappy btrfs's after RAID meltdown

2012-02-06 Thread Liu Bo
On 02/07/2012 11:39 AM, Ryan C. Underwood wrote: Does anyone have any idea how I should proceed with the below quoted situation? Unfortunately, I am going to have to give up on btrfs if it is really so fragile. I am using kernel 3.2.2 and btrfs-tools from November. On Sun, Feb 05, 2012 at

Re: btrfs support for efficient SSD operation (data blocks alignment)

2012-02-08 Thread Liu Bo
On 02/09/2012 03:24 AM, Martin wrote: My understanding is that for x86 architecture systems, btrfs only allows a sector size of 4kB for a HDD/SSD. That is fine for the present HDDs assuming the partitions are aligned to a 4kB boundary for that device. However for SSDs... I'm using for

Re: Premature ENOSPC only with zlib Compression

2012-02-08 Thread Liu Bo
On 02/09/2012 05:01 AM, Mitch Harder wrote: On Wed, Jan 18, 2012 at 10:13 AM, Mitch Harder mitch.har...@sabayonlinux.org wrote: I have a Btrfs partition that is reliably reproducing premature ENOSPC when restoring the disk from a tar file, but it is only happening with zlib compression (lzo

[PATCH 3/3 v2] xfstests: add btrfs online defragments QA test

2012-02-08 Thread Liu Bo
As the title shows, we port btrfs online defragments QA test into xfstests. v1-v2: - place the real tests inside testcases. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- 278 | 247 ++ 278.args | 18 + 278.out | 75

Re: Bulk discard doesn't work after add/delete of devices

2012-02-09 Thread Liu Bo
On 02/06/2012 04:37 AM, Lutz Euler wrote: ... maybe even the block group cache is nonfunctional then? I am using a btrfs file system, mirrored data and metadata on two SSDs, and recently tried for the first time to run fstrim on it. fstrim unexpectedly said 0 bytes were trimmed. strace -T

[PATCH] Btrfs: fix trim 0 bytes after a device delete

2012-02-09 Thread Liu Bo
the device, the block group may start from a non-zero place, which will confuse trim to discard nothing. Reported-by: Lutz Euler lutz.eu...@freenet.de Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git

Re: Bulk discard doesn't work after add/delete of devices

2012-02-09 Thread Liu Bo
On 02/09/2012 11:50 PM, Lutz Euler wrote: Hi Liu, thanks for looking into this! You wrote: Would you please test the following patch on your box? thanks, liubo diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 77ea23c..b6e2c92 100644 --- a/fs/btrfs/extent-tree.c

Re: Bulk discard doesn't work after add/delete of devices

2012-02-12 Thread Liu Bo
On 02/13/2012 01:01 AM, Lutz Euler wrote: Hi, tl;dr: btrfs_trim_fs, IMHO, needs more thorough surgery. Thanks for providing the new patch. I think it will work in the case that fstrim is called without specifying a range to trim (that is, defaulting to the whole filesystem), but I didn't

Re: Set nodatacow per file?

2012-02-13 Thread Liu Bo
and database storage? Of course cloning should still cause COW. IIRC this is already a feature in btrfs. I didn't catch the whole talk, but Chris mentioned something like this at Scale 10x. I also remember seeing a patch for it a while back (I think it was from liu bo) that does

Re: [PATCH 3/3 v2] xfstests: add btrfs online defragments QA test

2012-02-13 Thread Liu Bo
On 02/14/2012 01:53 AM, Christoph Hellwig wrote: This still needs a bit more work: +test_path=`pwd` +progs_dir=$test_path/src/btrfs_online_defragment/ this isn't actually used. +tmp=tmp/$$ +defrag_args=$test_path/${seq}.args Just hardcode the arguments, preferably without the args

[PATCH 3/3 v3] xfstests: add btrfs online defragments QA test

2012-02-14 Thread Liu Bo
As the title shows, we port btrfs online defragments QA test into xfstests. v3: - use xfstests wrappers. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- 278 | 171 +++ 278.out | 11 group |1 + 3 files changed, 183

[RFC PATCH] Btrfs: batch merge state at endio time

2012-02-15 Thread Liu Bo
. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 434 +++--- fs/btrfs/extent_io.h |5 +- 2 files changed, 382 insertions(+), 57 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 49f3c9d..0cced4d

[PATCH 2/4] Btrfs: skip states when they does not contain bits to clear

2012-02-16 Thread Liu Bo
Clearing a range's bits is different with setting them, since we don't need to touch them when states do not contain bits we want. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 15 ++- 1 files changed, 10 insertions(+), 5 deletions(-) diff --git a/fs

[PATCH 1/4] Btrfs: be less strict on finding next node in clear_extent_bit

2012-02-16 Thread Liu Bo
In clear_extent_bit, it is enough that next node is adjacent in tree level. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c |3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index fcf77e1..e941cc4 100644

[PATCH 3/4] Btrfs: kick out redundant stuff in convert_extent_bit

2012-02-16 Thread Liu Bo
clear_state_bit will do merge_state for us, so kick out the redundant one. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c |5 - 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index e791fef..039c5c6

Re: Balance RAID10 with odd device count

2012-02-20 Thread Liu Bo
On 02/21/2012 08:45 AM, Wes wrote: I've noticed similar behavior when even RAID0'ing an odd number of devices which should be even more trivial in practice. You would expect something like: sda A1 B1 sdb A2 B2 sdc A3 B3 or at least, if BTRFS can only handle block pairs, sda A1 B2 sdb

Re: [PATCH] Btrfs: clear the extent uptodate bits during parent transid failures

2012-02-22 Thread Liu Bo
the uptodate state bits, which is safe because this code is only called when the parent transid fails. Reviewed-by: Liu Bo liubo2...@cn.fujitsu.com or we can be safer: diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index fcf77e1..c1fe25d 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs

Re: [BUG] Kernel Bug at fs/btrfs/volumes.c:3638

2012-02-24 Thread Liu Bo
On 02/24/2012 06:41 PM, Nageswara R Sastry wrote: Hello, While working with 'fsfuzz - file system fuzzing tool' on 'btrfs' encountered the following kernel bug. Environment: Kernel Version: 3.3.0-rc4 Architecture: s390, x86 Providing the kernel trace from 's390' arch. Btrfs loaded

Re: [PATCH] Btrfs: hold enough space for global_rsv

2012-02-27 Thread Liu Bo
On 02/27/2012 09:29 PM, Johannes Hirte wrote: Am Tue, 17 Jan 2012 17:51:59 +0800 schrieb Liu Bo liubo2...@cn.fujitsu.com: I've kept hitting enospc warnings of global_rsv while running defragment on files: btrfs: block rsv returned -28 WARNING: at fs/btrfs/extent-tree.c:5984

Re: [PATCH] Btrfs: hold enough space for global_rsv

2012-03-08 Thread Liu Bo
On 03/09/2012 03:22 AM, Johannes Hirte wrote: Am Tue, 6 Mar 2012 14:50:32 +0100 schrieb Johannes Hirte johannes.hi...@fem.tu-ilmenau.de: I've backed up the filesystem, deleted the subvolumes, recreated them and copied the data back. Now everything seems to work again. I've also a full image

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-08 Thread Liu Bo
On 03/09/2012 03:35 AM, Jacek Luczak wrote: 2012/3/8 David Sterba d...@jikos.cz: On Thu, Mar 08, 2012 at 01:10:45PM +0100, Jacek Luczak wrote: kernel BUG at fs/btrfs/delayed-inode.c:1466! 1461 ret = btrfs_delayed_item_reserve_metadata(trans, root, item); 1462 /* 1463

[PATCH] Btrfs: fix deadlock during allocating chunks

2012-03-15 Thread Liu Bo
on chunk_mutex. So instead we need to allocate the system chunk first if we find we're in ENOSPC. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 50 1 files changed, 50 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent

[RFC PATCH 0/5] apply rwlock for extent state

2012-03-15 Thread Liu Bo
file with pagecache, $ dd if=/mnt/btrfs/foobar of=/dev/null bs=1M without this patchset, lock for read : lock for write ~= 2:1 with this patchset, lock for read : lock for write ~= 200:1 I've run through xfstests, and no bugs jump out by then. MORE TESTS ARE WELCOME! Liu Bo (5

[PATCH 2/5] Btrfs: merge adjacent states as much as possible

2012-03-15 Thread Liu Bo
In order to reduce write locks, we do merge_state as much as much as possible. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 47 +++ 1 files changed, 27 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c b

[PATCH 1/5] Btrfs: use radix tree for checksum

2012-03-15 Thread Liu Bo
We used to issue a checksum to an extent state of 4K range for read endio, but we want to use larger range for performance optimization, so instead we create a radix tree for checksum, where an item stands for checksum of 4K data. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs

[PATCH 5/5] Btrfs: avoid splitting state when truncating pagecache

2012-03-15 Thread Liu Bo
. And this way we can avoid splitting state and reduce the write locks of extent state tree. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/free-space-cache.c |5 + fs/btrfs/inode.c| 15 +++ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git

[PATCH 4/5] Btrfs: apply rwlock for extent state

2012-03-15 Thread Liu Bo
We used to protect both extent state tree and an individual state's state by tree-lock, but this can be an obstacle of lockless read. So we seperate them: tree-lock protects the tree while state-lock protects its state. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c

[PATCH 3/5] Btrfs: use large extent range for read and its endio

2012-03-15 Thread Liu Bo
we use larger extent state range for both readpages and read endio, so that we can lock or unlock less and avoid most of split ops, then we'll reduce write locks taken at endio time. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 201

Re: [RFC PATCH 0/5] apply rwlock for extent state

2012-03-20 Thread Liu Bo
Any comments? On 03/15/2012 05:12 PM, Liu Bo wrote: This patchset is against one of project ideas, RBtree lock contention: Btrfs uses a number of rbtrees to index in-memory data structures. Some of these are dominated by reads, and the lock contention from searching them is showing up

[PATCH RESEND] Xfstest: make 218 support btrfs

2012-03-20 Thread Liu Bo
Btrfs progs has a defragment tool, so we can test 218 on btrfs now. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- 218 |2 +- common.defrag |3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/218 b/218 index 53d0b61..77782c6 100755 --- a/218 +++ b/218

Re: [RFC PATCH 0/5] apply rwlock for extent state

2012-03-21 Thread Liu Bo
On 03/22/2012 09:04 AM, Chris Samuel wrote: On 21/03/12 23:34, Andrea Gelmini wrote: No comment, but I'm using this patches without problem since you published it (compressed /home with hourly snapshot delete/creation). Well worth sending a Tested-By: tag then, it's useful information.

Re: [patch 17/35] btrfs: split extent_state ops

2012-03-21 Thread Liu Bo
On 03/22/2012 09:11 AM, Jeff Mahoney wrote: set_extent_bit can do exclusive locking but only when called by lock_extent*, Drop the exclusive bits argument except when called by lock_extent. Hi Jeff, I have to say this is not a good cleanup, any benefits? thanks, liubo Signed-off-by:

Re: [patch 08/35] btrfs: Remove set bits return from clear_extent_bit

2012-03-21 Thread Liu Bo
On 03/22/2012 09:11 AM, Jeff Mahoney wrote: There is only one caller of clear_extent_bit that checks the return value and it only checks if it's negative. Since there are no users of the returned bits functionality of clear_extent_bit, stop returning it and avoid complicating error handling.

[PATCH] Btrfs: fix recursive defragment with autodefrag option

2012-03-22 Thread Liu Bo
for COW code, i.e. insert an inode when we're really going to make some defragments. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c |8 +--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 892b347..7f5018d 100644

Re: btrfs crash after disk reconnect

2012-03-25 Thread Liu Bo
On 03/23/2012 08:07 PM, Jan Engelhardt wrote: Observed on Linux 3.2.9 after the controller/disk flaked in-out. (The world still needs a SCSI error decoding tool to tell normal people what cmd and res are about.) I'm not that sure if your 3.2.9-jng4-default build contains this commit or not:

[PATCH 00/10] several fixes and cleanups

2012-03-27 Thread Liu Bo
This patchset consists of a bug fix from allocating chunk, six bug fixes from autodefrag, and other cleanups. I've tested it with xfstests plus autodefrag option. Liu Bo (10): Btrfs: show useful info in space reservation tracepoint Btrfs: fix deadlock during allocating chunks Btrfs: fix

[PATCH 01/10] Btrfs: show useful info in space reservation tracepoint

2012-03-27 Thread Liu Bo
o For space info, the type of space info is useful for debug. o For transaction handle, its transid is useful. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 29 ++--- fs/btrfs/inode-map.c |6 ++ fs/btrfs/transaction.c |3

[PATCH 02/10][RESEND] Btrfs: fix deadlock during allocating chunks

2012-03-27 Thread Liu Bo
on chunk_mutex. So instead we need to allocate the system chunk first if we find we're in ENOSPC. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 50 1 files changed, 50 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent

[PATCH 04/10] Btrfs: fix the mismatch of page-mapping

2012-03-27 Thread Liu Bo
i_mutex lock, inode size remains unchanged and we can drop isize overflow checks. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 35 +++ 1 files changed, 19 insertions(+), 16 deletions(-) diff

[PATCH 06/10] Btrfs: add a check to decide if we should defrag the range

2012-03-27 Thread Liu Bo
If our file's layout is as follows: | hole | data1 | hole | data2 | we do not need to defrag this file, because this file has holes and cannot be merged into one extent. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 36 +++- 1 files

[PATCH 05/10][RESEND] Btrfs: fix recursive defragment with autodefrag option

2012-03-27 Thread Liu Bo
for COW code, i.e. insert an inode when we're really going to make some defragments. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c |8 +--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 892b347..7f5018d 100644

[PATCH 08/10] Btrfs: update to the right index of defragment

2012-03-27 Thread Liu Bo
When we use autodefrag, we forget to update the index which indicates the last page we've dirty. And we'll set dirty flags on a same set of pages again and again. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions

[PATCH 10/10] Btrfs: drop cache with VACANCY em when we fail to start a transaction

2012-03-27 Thread Liu Bo
We need to clean a VACANCY em(if we have) when we fail to start a transaction. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index bacf441..2b2f0b6 100644

[PATCH 09/10] Btrfs: use PagePrivate2 to check ordered data

2012-03-27 Thread Liu Bo
If a page has PagePrivate2 flag, it still remains as ordered data, so we can check this flag directly instead of looking up an ordered extent. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c | 45 +++-- 1 files changed, 15 insertions

[PATCH 07/10] Btrfs: do not bother to defrag an extent if it is a big real extent

2012-03-27 Thread Liu Bo
, blocksize 4096) ext logical physical expected length flags 0 0 3082 10 eof /mnt/btrfs/foobar: 1 extent found So if we already find a big real extent, we're ok about that, just skip it. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c |9 +++-- 1

[PATCH 03/10] Btrfs: fix race between direct io and autodefrag

2012-03-27 Thread Liu Bo
. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/ioctl.c |6 +- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index d8b5471..0acc828 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs

Re: Create subvolume from a directory?

2012-03-27 Thread Liu Bo
On 03/28/2012 06:24 AM, Matthias G. Eckermann wrote: Hello Alex and all, On 2012-03-27 T 17:19 + Alex wrote: Just a quick question but can't find an obvious answer. Can I create/convert a existing (btrfs) directory into a subvolume? It would be very helpful when transferring

Re: Create subvolume from a directory?

2012-03-28 Thread Liu Bo
On 03/29/2012 12:54 AM, Goffredo Baroncelli wrote: Could you elaborate which would be the issue ? cp --reflink-ing a file is not different than snapshotting a file. In any case I could mount a snapshot and not the source subvolume. We already have a debate about this cross-link

Re: [PATCH 10/10] Btrfs: drop cache with VACANCY em when we fail to start a transaction

2012-03-29 Thread Liu Bo
On 03/29/2012 10:01 PM, Chris Mason wrote: On Tue, Mar 27, 2012 at 02:44:38PM +0800, Liu Bo wrote: We need to clean a VACANCY em(if we have) when we fail to start a transaction. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c |6 +++--- 1 files changed, 3

Re: [GIT PULL] Btrfs fixes and features

2012-04-02 Thread Liu Bo
large change (thanks Stefan Behrens). We also have an assortment of fixes and updates, especially to the balancing code (Ilya Dryomov), the back ref walker (Jan Schmidt) and the defragging code (Liu Bo). Jeff Mahoney (21) commits (+1982/-1051): btrfs: clean_tree_block should panic

[PATCH] Btrfs: do not mount when we have a sectorsize larger than PAGE_SIZE

2012-04-02 Thread Liu Bo
Our code is not ready to cope with a sectorsize that's larger than PAGE_SIZE. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/disk-io.c |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 20196f4..08e49d2

Re: [PATCH] Btrfs: do not mount when we have a sectorsize larger than PAGE_SIZE

2012-04-02 Thread Liu Bo
On 04/02/2012 08:17 PM, David Sterba wrote: On Mon, Apr 02, 2012 at 07:28:18PM +0800, Liu Bo wrote: --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2104,6 +2104,14 @@ int open_ctree(struct super_block *sb, err = -EINVAL; goto fail_alloc

[PATCH] Btrfs: do not mount when we have a sectorsize unequal to PAGE_SIZE

2012-04-02 Thread Liu Bo
Our code is not ready to cope with a sectorsize that's not equal to PAGE_SIZE. It will lead to hanging-on while writing something. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/disk-io.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/disk

Re: [PATCH] Btrfs: do not mount when we have a sectorsize unequal to PAGE_SIZE

2012-04-03 Thread Liu Bo
On 04/03/2012 04:06 PM, Karel Zak wrote: On Tue, Apr 03, 2012 at 09:56:53AM +0800, Liu Bo wrote: Our code is not ready to cope with a sectorsize that's not equal to PAGE_SIZE. It will lead to hanging-on while writing something. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs

Re: [PATCH 0/3] btrfs: extended inode refs

2012-04-05 Thread Liu Bo
On 04/06/2012 09:24 AM, Liu Bo wrote: On 04/06/2012 04:09 AM, Mark Fasheh wrote: Currently btrfs has a limitation on the maximum number of hard links an inode can have. Specifically, links are stored in an array of ref items: struct btrfs_inode_ref { __le64 index; __le16 name_len

Re: [3.4-rc1] attempt to access beyond end of device and livelock

2012-04-08 Thread Liu Bo
On 04/06/2012 07:36 PM, Daniel J Blueman wrote: Hi Josef, Chris, When testing BTRFS with RAID 0 metadata on linux-3.4-rc1, we see discard ranges exceeding the end of the block device [1], potentially causing dataloss; when this occurs, filesystem writeback becomes catatonic due to continual

[PATCH] Btrfs: fix eof while discarding extents

2012-04-08 Thread Liu Bo
We miscalculate the length of extents we're discarding, and it leads to an eof of device. Reported-by: Daniel Blueman dan...@quora.org Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/volumes.c | 20 ++-- 1 files changed, 18 insertions(+), 2 deletions(-) diff --git

Re: [PATCH] Revert Btrfs: increase the global block reserve estimates

2012-04-08 Thread Liu Bo
On 04/09/2012 07:40 AM, David Sterba wrote: This reverts commit 5500cdbe14d7435e04f66ff3cfb8ecd8b8e44ebf. We had numerous reports of premature ENOSPC that were bisected to this patch. Reverting will not break things but a warning in 'use_block_rsv' may show up in the syslog. There's no

Re: storing metadata on a dedicated device

2012-04-10 Thread Liu Bo
On 04/10/2012 02:57 PM, Jan Killius wrote: Hello, I just wanted to ask if storing metadata on a dedicated device is implemented at the moment ? It's listed under Project ideas and there is supposed to be a patch but I can't find it anywhere. AFAIK, not yet, but it is on plan :) -- liubo

[PATCH RFC] Btrfs: improve space count for files with fragments

2012-04-12 Thread Liu Bo
---| | - A - | this part, A, includes the _start_ of the extent. We know that our data checksum item is taking this _start_ as a index, so just leave A's space where it is, do not free it. NOTE: This has compatability issue, so please use this on a fresh-build btrfs. Signed-off-by: Liu Bo liubo2

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-12 Thread Liu Bo
On 04/12/2012 11:54 PM, Arne Jansen wrote: This patchset reimplements snapshot deletion with the help of the readahead framework. For this callbacks are added to the framework. The main idea is to traverse many snapshots at once at read many branches at once. This way readahead get many

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-13 Thread Liu Bo
On 04/13/2012 02:53 PM, Arne Jansen wrote: On 13.04.2012 05:40, Liu Bo wrote: On 04/12/2012 11:54 PM, Arne Jansen wrote: This patchset reimplements snapshot deletion with the help of the readahead framework. For this callbacks are added to the framework. The main idea

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-13 Thread Liu Bo
On 04/13/2012 03:10 PM, Liu Bo wrote: On 04/13/2012 02:53 PM, Arne Jansen wrote: On 13.04.2012 05:40, Liu Bo wrote: On 04/12/2012 11:54 PM, Arne Jansen wrote: This patchset reimplements snapshot deletion with the help of the readahead framework. For this callbacks are added

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-13 Thread Liu Bo
On 04/13/2012 03:19 PM, Arne Jansen wrote: might be out of memory. How much does this vm (?) have? Can you try to reduce the constants in disk-io.c:2003-2005? Thanks, Arne Seems not related to an OOM: # free -m total used free sharedbuffers cached

Re: [PATCH] btrfs: fix early abort in 'remount'

2012-04-15 Thread Liu Bo
On 04/16/2012 06:46 AM, Sergei Trofimovich wrote: From: Sergei Trofimovich sly...@gentoo.org Cc: Jeff Mahoney je...@suse.com Cc: Chris Mason chris.ma...@oracle.com Cc: Josef Bacik jo...@redhat.com Signed-off-by: Sergei Trofimovich sly...@gentoo.org --- fs/btrfs/super.c |4 1

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-17 Thread Liu Bo
On 04/17/2012 03:35 PM, Arne Jansen wrote: On 13.04.2012 09:43, Liu Bo wrote: On 04/13/2012 03:19 PM, Arne Jansen wrote: might be out of memory. How much does this vm (?) have? Can you try to reduce the constants in disk-io.c:2003-2005? Thanks, Arne Seems not related to an OOM: # free

Re: [PATCH] Btrfs: complete page writeback before doing ordered extents

2012-04-23 Thread Liu Bo
On 04/24/2012 01:33 AM, Josef Bacik wrote: We can deadlock waiting for pages to end writeback because we are doing an allocation while hold a tree lock since the ordered extent stuff will require tree locks. A quick easy way to fix this is to end page writeback before we do our ordered io

Re: [PATCH] Btrfs: complete page writeback before doing ordered extents

2012-04-25 Thread Liu Bo
On 04/24/2012 10:15 PM, Chris Mason wrote: On Tue, Apr 24, 2012 at 09:50:39AM +0800, Liu Bo wrote: On 04/24/2012 01:33 AM, Josef Bacik wrote: We can deadlock waiting for pages to end writeback because we are doing an allocation while hold a tree lock since the ordered extent stuff

[RFC PATCH v2] Btrfs: improve space count for files with fragments

2012-04-26 Thread Liu Bo
things. v1-v2: rewrite the whole patch with a new approach. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ctree.h |8 ++ fs/btrfs/delayed-ref.c | 56 ++-- fs/btrfs/delayed-ref.h |7 ++ fs/btrfs/extent-tree.c | 226

Re: [RFC PATCH v2] Btrfs: improve space count for files with fragments

2012-04-26 Thread Liu Bo
On 04/27/2012 01:14 AM, Chris Mason wrote: On Thu, Apr 26, 2012 at 02:39:23PM +0800, Liu Bo wrote: Here is a simple scenario: $ dd if=/dev/zero of=/mnt/btrfs/foobar bs=1k count=20;sync $ btrfs fi df /mnt/btrfs we get 20K used, but then $ dd if=/dev/zero of=/mnt/btrfs/foobar bs=1k

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-26 Thread Liu Bo
On 04/17/2012 03:35 PM, Arne Jansen wrote: On 13.04.2012 09:43, Liu Bo wrote: On 04/13/2012 03:19 PM, Arne Jansen wrote: might be out of memory. How much does this vm (?) have? Can you try to reduce the constants in disk-io.c:2003-2005? Thanks, Arne Seems not related

Re: [PATCH 0/5] btrfs: snapshot deletion via readahead

2012-04-27 Thread Liu Bo
On 04/27/2012 02:13 PM, Arne Jansen wrote: Hi Liu Bo, thanks for testing :) so one block reserve ran out of space. Did you use the same test as before? On SSD? Yes, the same test on SSD, but with different number of snapshot: 1000, 2000 respectively :) For the block reserve ENOSPC

Re: kernel 3.3.4 damages filesystem (?)

2012-05-07 Thread Liu Bo
On 05/07/2012 06:46 PM, Helmut Hullen wrote: btrfs: error reading free space cache BUG: unable to handle kernel NULL pointer dereference at 0001 IP: [c1295c36] io_ctl_drop_pages+0x26/0x50 *pdpt = 29712001 *pde = Oops: 0002 [#1] Could you please try this and

Re: 3.4-rc6: delayed alloc deadlock...

2012-05-08 Thread Liu Bo
On 05/08/2012 04:56 PM, Daniel J Blueman wrote: Delayed allocation ref mutexes are taken [1] inside btrfs_commit_transaction. A later call fails and jumps to the cleanup_transaction label (transaction.c:1501) with these mutexes still held causing deadlock [2] when they are reacquired.

Re: [RFC PATCH v2] Btrfs: improve space count for files with fragments

2012-05-09 Thread Liu Bo
On 05/10/2012 01:29 AM, David Sterba wrote: On Fri, Apr 27, 2012 at 09:44:13AM +0800, Liu Bo wrote: Let's take the above case: 0k 20k | --- extent --- | | - A - | 1k 19k And we assume that this extent starts from disk_bytenr on its FS logical offset

[PATCH 1/2] Btrfs: fix wrong error returned by adding a device

2012-05-10 Thread Liu Bo
-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/volumes.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 1411b99..48a06d1 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1633,7 +1633,7 @@ int btrfs_init_new_device

[PATCH 2/2] Btrfs: use fastpath in extent state ops as much as possible

2012-05-10 Thread Liu Bo
Fully utilize our extent state's new helper functions to use fastpath as much as possible. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 44 ++-- 1 files changed, 18 insertions(+), 26 deletions(-) diff --git a/fs/btrfs

[PATCH] Btrfs: do not do balance in readonly mode

2012-05-11 Thread Liu Bo
--- fail. $ btrfs dev add /dev/sdb8 /mnt/btrfs $ btrfs fi bal /mnt/btrfs --- works! It should not be designed as an exception, and we'd better add another check for mnt flags. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 12 +--- 1 files

Re: 3.4.0-rc6: WARNING: at fs/btrfs/super.c:219 __btrfs_abort_transaction+0xae/0xc0 [btrfs]()

2012-05-14 Thread Liu Bo
On 05/14/2012 09:18 PM, Arnd Hannemann wrote: Hi, I just got the following warning on a compressed btrfs filesystem, while writing on it until no remaining space was available. Looks like a bit verbose disk full message, is it expected behavior? Hi Arnd, Can you add another mount

Re: [PATCH v3 3/3] Btrfs: read device stats on mount, write modified ones during commit

2012-05-16 Thread Liu Bo
On 05/17/2012 12:50 AM, Stefan Behrens wrote: The device statistics are written into the device tree with each transaction commit. Only modified statistics are written. When a filesystem is mounted, the device statistics for each involved device are read from the device tree and used to

[PATCH 1/2] Btrfs: do not resize a seeding device

2012-05-17 Thread Liu Bo
Seeding devices are not supposed to change any more. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c |7 +++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index f056469..ec2245d 100644 --- a/fs/btrfs/ioctl.c +++ b

[PATCH 2/2] Btrfs: resize all devices when we dont assign a specific device id

2012-05-17 Thread Liu Bo
: new size for /dev/sdb7 is 980844544 $ btrfs fi resize 1:-100m /mnt/btrfs then we can get from dmesg: btrfs: resizing devid 2 btrfs: new size for /dev/sdb8 is 980844544 Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 101

[PATCH 1/2] Btrfs: avoid memory leak of extent state in error handling routine

2012-05-18 Thread Liu Bo
=4096, reserved=0, may_use=0, readonly=4194304 btrfs state leak: start 29364224 end 29376511 state 1 in tree 880075f20090 refs 1 ... Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/disk-io.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/disk

[PATCH] Btrfs: destroy the items of the delayed inodes in error handling routine

2012-05-18 Thread Liu Bo
From: Miao Xie mi...@cn.fujitsu.com the items of the delayed inodes were forgotten to be freed, this patch fix it. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/delayed-inode.c | 18 ++ fs/btrfs/delayed-inode.h |3 +++ fs/btrfs/disk-io.c |6 ++ 3

<    2   3   4   5   6   7   8   9   10   11   >