[PATCH 0/6 v3][RFC] rwlock for extent state

2012-07-25 Thread Liu Bo
This patchset is against one of project ideas, RBtree lock contention: Btrfs uses a number of rbtrees to index in-memory data structures. Some of these are dominated by reads, and the lock contention from searching them is showing up in profiles. We need to look into an RCU and sequence counter

[PATCH 3/6 v3][RFC] Btrfs: break clear_state_bit into two parts

2012-07-25 Thread Liu Bo
clear_state_bit() can be broken into two parts: 1) a part for clearing bits, 2) a part for freeing the state or merging it with adjacent ones. And the first one does not need to touch the tree, so holding read locks is enough, and this gives us the oppotunity to rework tree's lock with rwlock.

[PATCH 1/6 v3][RFC] Btrfs: merge adjacent states as much as possible

2012-07-25 Thread Liu Bo
In order to reduce write locks, we do merge_state as much as much as possible. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 47 +++ 1 files changed, 27 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c

[PATCH 2/6 v3][RFC] Btrfs: add helper function to test if we can merge state

2012-07-25 Thread Liu Bo
This is a helper function to test if two states are adjacent. It is used for applying rwlock for extent state. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent_io.c | 27 +++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git

[PATCH 5/6 v3][RFC] Btrfs: batch merge state in readpage endio

2012-07-25 Thread Liu Bo
This is the first part of parallel endio for read. The main idea behind it is that in theory we can gain a lot of performance if we are able to reduce the write locks taken at endio time. Here we batch the merge state part in unlocking extent state and we don't need to touch the tree, which

[PATCH 4/6 v3][RFC] Btrfs: apply rwlock for extent state

2012-07-25 Thread Liu Bo
We used to protect both extent state tree and an individual state's state by tree-lock, and now we want to reduce lock contention on this lock. So we adopt rwlock for tree-lock and seperate them here for reducing lock granularity: o tree-lock protects the tree o state-lock protects the state.

[PATCH 6/6 v3][RFC] Btrfs: async helper for state merge

2012-07-25 Thread Liu Bo
This is the second part of parallel endios for read. Here we use an async helper thread to process batched merges, so we eventually get endio for read to avoid acquiring or holding any write locks of extent state tree. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ctree.h |

Re: No/bad auto-detection of fs type for small volumes (related to mixed metadata/data?)

2012-07-25 Thread Hugo Mills
On Tue, Jul 24, 2012 at 08:39:36PM -0400, Marios Titas wrote: When I create a btrfs volume of size strictly less than 256 MiB then if I do mount /dev/sdb1 /mnt/test the kernel tries unsuccessfully to do the mount with many other file systems before successfully trying with btrfs. For

Re: No/bad auto-detection of fs type for small volumes (related to mixed metadata/data?)

2012-07-25 Thread cwillu
On Tue, Jul 24, 2012 at 6:39 PM, Marios Titas redneb8...@gmail.com wrote: When I create a btrfs volume of size strictly less than 256 MiB then if I do mount /dev/sdb1 /mnt/test the kernel tries unsuccessfully to do the mount with many other file systems before successfully trying with

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-25 Thread Alex Lyakas
Thanks! So now: A_PATH - path - full_path - newpath A_PATH_LINK - lnk - full_link_path - oldpath while I viewed it the other way around. I guess it's not important what is left/right, old/new :) as long as it's consistent. Alex. On Tue, Jul 24, 2012 at 11:27 PM, Alexander Block

Re: No/bad auto-detection of fs type for small volumes (related to mixed metadata/data?)

2012-07-25 Thread Marios Titas
On Wed, Jul 25, 2012 at 5:09 AM, cwillu cwi...@cwillu.com wrote: On Tue, Jul 24, 2012 at 6:39 PM, Marios Titas redneb8...@gmail.com wrote: When I create a btrfs volume of size strictly less than 256 MiB then if I do mount /dev/sdb1 /mnt/test the kernel tries unsuccessfully to do the mount

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side of btrfs send/receive. You can apply them manually or use my git repo: git://github.com/ablock84/btrfs-progs.git (branch send) The

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-25 Thread Alexander Block
On Tue, Jul 24, 2012 at 7:55 AM, Arne Jansen sensi...@gmx.net wrote: On 23.07.2012 21:41, Alexander Block wrote: On Mon, Jul 16, 2012 at 4:56 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:38, Alexander Block wrote: + + ret = btrfs_update_root(trans, root-fs_info-tree_root, +

Question about btrfs metadata structure

2012-07-25 Thread Guenther Rasch
Hi! I'm writing my bsc thesis about btrfs and I'm analyzing the btrfs metadata structure at the moment. I'm using fedora 17 with kernel 3.4 and btrfs-prog from your git-repository. For an overview I use btrfs-debug-tree and here are my questions: The root tree contains some inode_items,

Btrfs-progs: segfault when re-creating filesystem with 3+ devices

2012-07-25 Thread Cyril B.
Hello, Running on Linux 3.5.0, with the latest btrfs-progs from today. I was doing some tests with my 7 devices, and mkfs.btrfs segfaults when executed twice, creating a filesystem with 3+ devices: # ./mkfs.btrfs /dev/sdc /dev/sdd /dev/sde WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL

mkfs devices ordering relevant with devices of different sizes?

2012-07-25 Thread Cyril B.
Hello, When creating a filesystem with devices of different sizes, the resulting filesystem total size depends on the device order specified to mkfs. When the smaller device is specified first, the second (larger) device is seen as the same size as the first. This doesn't occur when the order

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Hugo Mills
On Wed, Jul 25, 2012 at 12:41:56PM +0200, Alexander Block wrote: On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side of btrfs send/receive. You can apply them manually or use my git

[PATCH 07/16] btrfs: nuke write_super from comments

2012-07-25 Thread Artem Bityutskiy
From: Artem Bityutskiy artem.bityuts...@linux.intel.com The '-write_super' superblock method is gone, and this patch removes all the references to 'write_super' from btrfs. Cc: Chris Mason chris.ma...@fusionio.com Cc: linux-btrfs@vger.kernel.org Signed-off-by: Artem Bityutskiy

[PATCH 08/16] btrfs: nuke pdflush from comments

2012-07-25 Thread Artem Bityutskiy
From: Artem Bityutskiy artem.bityuts...@linux.intel.com The pdflush thread is long gone, so this patch removes references to pdflush from btrfs comments. Cc: Chris Mason chris.ma...@fusionio.com Cc: linux-btrfs@vger.kernel.org Signed-off-by: Artem Bityutskiy artem.bityuts...@linux.intel.com ---

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Chris Mason
On Wed, Jul 25, 2012 at 08:00:36AM -0600, Hugo Mills wrote: On Wed, Jul 25, 2012 at 12:41:56PM +0200, Alexander Block wrote: On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side of

Re: [PATCH 07/16] btrfs: nuke write_super from comments

2012-07-25 Thread cwillu
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index ecaad40..9f2416c 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1738,10 +1738,6 @@ int btrfs_init_new_device(struct btrfs_root *root, char *device_path) device-fs_devices = root-fs_info-fs_devices; -

Re: [PATCH 07/16] btrfs: nuke write_super from comments

2012-07-25 Thread Artem Bityutskiy
On Wed, 2012-07-25 at 09:46 -0600, cwillu wrote: mutex_lock(root-fs_info-fs_devices-device_list_mutex); list_add_rcu(device-dev_list, root-fs_info-fs_devices-devices); list_add(device-dev_alloc_list, Is the locking still required for approximately the same

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 4:00 PM, Hugo Mills h...@carfax.org.uk wrote: On Wed, Jul 25, 2012 at 12:41:56PM +0200, Alexander Block wrote: On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alex Lyakas
Alexander, can you pls let know like a day or two before you run out of time? I have compiled a list of questions, but also want to do more testing before I publish them all. Thanks for your work, Alex. On Wed, Jul 25, 2012 at 7:56 PM, Alexander Block abloc...@googlemail.com wrote: On Wed, Jul

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 7:10 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, can you pls let know like a day or two before you run out of time? I have compiled a list of questions, but also want to do more testing before I publish them all. My flight goes on 6. August...after

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-25 Thread Alex Lyakas
Alexander, Same is true for BTRFS_FILE_EXTENT_PREALLOC extents, I think. Those also don't contain real data. So something like: if (left_disknr == 0 || left_type == BTRFS_FILE_EXTENT_REG) { ret = 1; goto out; } Do you mean || left_type == BTRFS_FILE_EXTENT_PREALLOC? I see

Re: [RFC PATCH 6/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 1)

2012-07-25 Thread Alexander Block
Thanks for the review :) On 07/18/2012 08:59 AM, Arne Jansen wrote: On 04.07.2012 15:38, Alexander Block wrote: This patch introduces the BTRFS_IOC_SEND ioctl that is required for send. It allows btrfs-progs to implement full and incremental sends. Patches for btrfs-progs will follow. I had

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 7:20 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, Same is true for BTRFS_FILE_EXTENT_PREALLOC extents, I think. Those also don't contain real data. So something like: if (left_disknr == 0 || left_type == BTRFS_FILE_EXTENT_REG) { ret = 1;