[PATCH 6/6 v3][RFC] Btrfs: async helper for state merge

2012-07-24 Thread Liu Bo
This is the second part of parallel endios for read. Here we use an async helper thread to process batched merges, so we eventually get endio for read to avoid acquiring or holding any write locks of extent state tree. Signed-off-by: Liu Bo --- fs/btrfs/ctree.h |1 + fs/btrfs/disk-io.c

[PATCH 4/6 v3][RFC] Btrfs: apply rwlock for extent state

2012-07-24 Thread Liu Bo
We used to protect both extent state tree and an individual state's state by tree->lock, and now we want to reduce lock contention on this lock. So we adopt rwlock for tree->lock and seperate them here for reducing lock granularity: o tree->lock protects the tree o state->lock protects the sta

[PATCH 5/6 v3][RFC] Btrfs: batch merge state in readpage endio

2012-07-24 Thread Liu Bo
This is the first part of parallel endio for read. The main idea behind it is that in theory we can gain a lot of performance if we are able to reduce the write locks taken at endio time. Here we batch the merge state part in unlocking extent state and we don't need to touch the tree, which means

[PATCH 2/6 v3][RFC] Btrfs: add helper function to test if we can merge state

2012-07-24 Thread Liu Bo
This is a helper function to test if two states are adjacent. It is used for applying rwlock for extent state. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 27 +++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/exten

[PATCH 1/6 v3][RFC] Btrfs: merge adjacent states as much as possible

2012-07-24 Thread Liu Bo
In order to reduce write locks, we do merge_state as much as much as possible. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 47 +++ 1 files changed, 27 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index

[PATCH 3/6 v3][RFC] Btrfs: break clear_state_bit into two parts

2012-07-24 Thread Liu Bo
clear_state_bit() can be broken into two parts: 1) a part for clearing bits, 2) a part for freeing the state or merging it with adjacent ones. And the first one does not need to touch the tree, so holding read locks is enough, and this gives us the oppotunity to rework tree's lock with rwlock. Si

[PATCH 0/6 v3][RFC] rwlock for extent state

2012-07-24 Thread Liu Bo
This patchset is against one of project ideas, RBtree lock contention: "Btrfs uses a number of rbtrees to index in-memory data structures. Some of these are dominated by reads, and the lock contention from searching them is showing up in profiles. We need to look into an RCU and sequence counter c

Re: Upgrading from 2.6.38, how?

2012-07-24 Thread Fajar A. Nugraha
On Wed, Jul 25, 2012 at 11:39 AM, Gareth Pye wrote: > My proposed upgrade method is: > Boot from a live CD with the latest kernel I can find so I can do a few tests: > A - run the fsck in read only mode to confirm things look good > B - mount read only, confirm that I can read files well > C -

Upgrading from 2.6.38, how?

2012-07-24 Thread Gareth Pye
Firstly I know what I've been doing has been less than 100% safe, but I've been prepared to live with it. For about 2 years now (you know from around the time btrfs looked like RAID5/6 was just around the corner) I've had a server with a 5 disk RAID10 btrfs array. I realise there has been quite so

No/bad auto-detection of fs type for small volumes (related to mixed metadata/data?)

2012-07-24 Thread Marios Titas
When I create a btrfs volume of size strictly less than 256 MiB then if I do mount /dev/sdb1 /mnt/test the kernel tries unsuccessfully to do the mount with many other file systems before successfully trying with btrfs. For volumes of size larger than or equal to 256 MiB it just mounts the volum

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-24 Thread Alexander Block
On Thu, Jul 19, 2012 at 3:25 PM, Alex Lyakas wrote: > +static int process_link(const char *path, const char *lnk, void *user) > +{ > + int ret; > + struct btrfs_receive *r = user; > + char *full_path = path_cat(r->full_subvol_path, path); > + > + if (g_verbose >= 1) > +

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-24 Thread Alexander Block
On Wed, Jul 18, 2012 at 7:45 PM, Alex Lyakas wrote: > Hi Alexander, > I am testing different scenarios in order to better understand the > non-trivial magic of > get_cur_path()/will_overwrite_ref()/did_overwrite_ref()/did_overwrite_first_ref(). > I hit the following issue, when testing full-send:

Re: [PATCH v4] Btrfs: Check INCOMPAT flags on remount and add helper function

2012-07-24 Thread David Sterba
On Tue, Jul 24, 2012 at 12:58:43PM -0500, Mitch Harder wrote: > In support of the recently added capability to remount with lzo > compression, provide a helper function to check the compression > INCOMPAT flags when remounting with lzo compression, and set > the flags if necessary. > > Also, imple

Re: [PATCH] Xfstests: add btrfs snapshot function test

2012-07-24 Thread David Sterba
On Sat, Jul 21, 2012 at 11:46:00AM +0800, Liu Bo wrote: > From: Zhou Bo > > This patch adds btrfs snapshot function test to xfstests. > > Signed-off-by: Zhou Bo > --- > 285 | 365 > +++ > 285.out |2 + > group |1 + >

[PATCH v4] Btrfs: Check INCOMPAT flags on remount and add helper function

2012-07-24 Thread Mitch Harder
In support of the recently added capability to remount with lzo compression, provide a helper function to check the compression INCOMPAT flags when remounting with lzo compression, and set the flags if necessary. Also, implement the new helper function when defragmenting with explicit lzo compress

Re: [PATCH v3 1/1] Btrfs: Check INCOMPAT flags on remount and add helper function

2012-07-24 Thread David Sterba
We don't need a helper for every incompatibility bit, let's do it in a more generic way as suggested below [modulo syntax errors]: On Fri, Jul 20, 2012 at 05:16:41PM -0500, Mitch Harder wrote: > --- a/fs/btrfs/ctree.h > +++ b/fs/btrfs/ctree.h > @@ -3103,6 +3103,19 @@ void __btrfs_abort_transaction

[PATCH 2/2] Btrfs-progs: show generation in command btrfs subvol list

2012-07-24 Thread Liu Bo
This adds the ability to show root's generation when we use btrfs subvol list. Signed-off-by: Liu Bo --- btrfs-list.c | 61 - 1 files changed, 55 insertions(+), 6 deletions(-) diff --git a/btrfs-list.c b/btrfs-list.c index ac6507a..05360

[PATCH 1/2] Btrfs-progs: search subvolumes with proper objectid

2012-07-24 Thread Liu Bo
Btrfs's subvolume/snapshot is limited to [BTRFS_FIRST_FREE_OBJECTID, BTRFS_LAST_FREE_OBJECTID], so just apply the range. Signed-off-by: Liu Bo --- btrfs-list.c |6 -- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/btrfs-list.c b/btrfs-list.c index c53d016..ac6507a 100644

Re: How can btrfs take 23sec to stat 23K files from an SSD?

2012-07-24 Thread Martin Steigerwald
Am Montag, 23. Juli 2012 schrieb Marc MERLIN: > I just realized that the older thread got a bit confusing, so I'll keep > problems separate and make things simpler :) > > On an _unencrypted_ partition on the SSD, running du -sh on a directory > with 15K files, takes 23 seconds on unencrypted SSD a