[PATCH] Btrfs: fix deadlock during allocating chunks

2012-03-15 Thread Liu Bo
This deadlock comes from xfstests 251. We'll hold the chunk_mutex throughout the whole of a chunk allocation. But if we find that we've used up system chunk space, we need to allocate a new system chunk, but this will lead to a recursion of chunk allocation and end up with a deadlock on

Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/10 Ted Ts'o ty...@mit.edu: Hey Jacek, I'm curious parameters of the set of directories on your production server.  On an ext4 file system, assuming you've copied the directories over, what are the result of this command pipeline when you are cd'ed into the top of the directory

[RFC PATCH 0/5] apply rwlock for extent state

2012-03-15 Thread Liu Bo
This patchset is against one of project ideas, RBtree lock contention: Btrfs uses a number of rbtrees to index in-memory data structures. Some of these are dominated by reads, and the lock contention from searching them is showing up in profiles. We need to look into an RCU and sequence counter

[PATCH 2/5] Btrfs: merge adjacent states as much as possible

2012-03-15 Thread Liu Bo
In order to reduce write locks, we do merge_state as much as much as possible. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 47 +++ 1 files changed, 27 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c

[PATCH 1/5] Btrfs: use radix tree for checksum

2012-03-15 Thread Liu Bo
We used to issue a checksum to an extent state of 4K range for read endio, but we want to use larger range for performance optimization, so instead we create a radix tree for checksum, where an item stands for checksum of 4K data. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com ---

[PATCH 5/5] Btrfs: avoid splitting state when truncating pagecache

2012-03-15 Thread Liu Bo
o While we're invalidating a page, we hold the page lock, so lock_extent is not needed. o When we're truncating pagecache in both evict and truncate_freespace, after waiting pages' writeback flag, it is ensured that no IO comes in, so we can safely clear the range's state. And

[PATCH 4/5] Btrfs: apply rwlock for extent state

2012-03-15 Thread Liu Bo
We used to protect both extent state tree and an individual state's state by tree-lock, but this can be an obstacle of lockless read. So we seperate them: tree-lock protects the tree while state-lock protects its state. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c |

[PATCH 3/5] Btrfs: use large extent range for read and its endio

2012-03-15 Thread Liu Bo
we use larger extent state range for both readpages and read endio, so that we can lock or unlock less and avoid most of split ops, then we'll reduce write locks taken at endio time. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 201

Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/11 Ted Ts'o ty...@mit.edu: Well, my goal in proposing this optimization is that helps for the medium size directories in the cold cache case.  The ext4 user who first kicked off this thread was using his file system for an SVN server, as I recall.  I could easily believe that he has

Re: [PATCH] btrfs: don't add both copies of DUP to reada extent tree

2012-03-15 Thread Andrea Gelmini
On Sat, Feb 25, 2012 at 09:09:47AM +0100, Arne Jansen wrote: Normally when there are 2 copies of a block, we add both to the reada extent tree and prefetch only the one that is easier to reach. This way we can better utilize multiple devices. In case of DUP this makes no sense as both copies

Re: [PATCH] btrfs/ulist: replace the magic number 128 with macro definition ULIST_SIZE_INCREMENT

2012-03-15 Thread Andrea Gelmini
On Thu, Mar 08, 2012 at 12:38:17PM +0800, Wang Sheng-Hui wrote: Remove the magic number 128 in btrfs/ulist.c. Introduce the macro definition ULIST_SIZE_INCREMENT in btrfs/ulist.h. I'm using this patch without problem since you published it (compressed /home with hourly snapshot

Re: [PATCH] btrfs: fixup module.h usage as required

2012-03-15 Thread Andrea Gelmini
On Sun, Feb 26, 2012 at 08:16:58PM -0500, Paul Gortmaker wrote: Delete the instances of module.h that aren't actually used or needed. Replace with export.h as required. I'm using this patch without problem since you published it (compressed /home with hourly snapshot delete/creation). Thanks

Re: [PATCH v2] Btrfs: fix deadlock during allocating chunks

2012-03-15 Thread Andrea Gelmini
On Wed, Mar 14, 2012 at 07:17:09PM +0800, Liu Bo wrote: This deadlock comes from xfstests 251. We'll hold the chunk_mutex throughout the whole of a chunk allocation. But if we find that we've used up system chunk space, we need to allocate a new system chunk, but this will lead to a

Re: 3.2-rc4: scrubbing locks up the kernel, then hung tasks on boot

2012-03-15 Thread Martin Steigerwald
Am Samstag, 25. Februar 2012 schrieb Arne Jansen: On 02/24/12 16:51, Martin Steigerwald wrote: Am Samstag, 21. Januar 2012 schrieb Martin Steigerwald: Am Samstag, 21. Januar 2012 schrieb Martin Steigerwald: I still have this with 3.2.0-1-pae - which is a debian kernel based on 3.2.1.

Re: 3.2-rc4: scrubbing locks up the kernel, then hung tasks on boot

2012-03-15 Thread Martin Steigerwald
Am Donnerstag, 15. März 2012 schrieb Martin Steigerwald: [20880.564225] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [20880.564234] btrfs D 12d2 0 3348 1 0x [20880.564251] c0b72080 0086 2cae1a17 12d2 ec3ae5a0 12d2

Re: 3.2-rc4: scrubbing locks up the kernel, then hung tasks on boot

2012-03-15 Thread Chris Mason
On Thu, Mar 15, 2012 at 06:32:19PM +0100, Martin Steigerwald wrote: Am Samstag, 25. Februar 2012 schrieb Arne Jansen: On 02/24/12 16:51, Martin Steigerwald wrote: Am Samstag, 21. Januar 2012 schrieb Martin Steigerwald: Am Samstag, 21. Januar 2012 schrieb Martin Steigerwald: I still

Re: 3.2-rc4: scrubbing locks up the kernel, then hung tasks on boot

2012-03-15 Thread Martin Steigerwald
Am Donnerstag, 15. März 2012 schrieb Chris Mason: On Thu, Mar 15, 2012 at 06:32:19PM +0100, Martin Steigerwald wrote: Am Samstag, 25. Februar 2012 schrieb Arne Jansen: On 02/24/12 16:51, Martin Steigerwald wrote: Am Samstag, 21. Januar 2012 schrieb Martin Steigerwald: Am Samstag, 21.

[PATCH] btrfs/async-thread.c: decrease workers-num_workers only on worker can be freed in btrfs_stop_workers

2012-03-15 Thread Wang Sheng-Hui
If the worker cannot be freed, decrease workers-num_workers can corrupt the statistics. The workers-num_workers should be decreased only when the worker can be stopped and freed in btrfs_stop_workers. Signed-off-by: Wang Sheng-Hui shh...@gmail.com --- fs/btrfs/async-thread.c |2 +- 1 files