Re: [RFC v2 05/10] vfs: introduce one hash table

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 5:54 PM, Ram Pai wrote: > On Sun, Sep 23, 2012 at 08:56:30PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> Adds a hash table structure which contains >> a lot of hash list and is used to efficiently >> look up the data temperature of a file or its >> rang

Re: [RFC v2 01/10] vfs: introduce private rb structures

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 6:20 PM, Ram Pai wrote: > On Sun, Sep 23, 2012 at 08:56:26PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> One root structure hot_info is defined, is hooked >> up in super_block, and will be used to hold rb trees >> root, hash list root and some other inf

Re: [PATCH v2 2/2] btrfs-progs: Fix up memory leakage

2012-09-25 Thread Zhi Yong Wu
On Wed, Sep 26, 2012 at 1:14 AM, Goffredo Baroncelli wrote: > On 09/25/2012 12:14 PM, David Sterba wrote: >> >> On Tue, Sep 25, 2012 at 10:02:16AM +0800, zwu.ker...@gmail.com wrote: >>> >>> From: Zhi Yong Wu >>> >>>Some code pathes forget to free memory on exit. >> >> >> Same as with the fd's,

Re: [RFC v2 03/10] vfs: add one new mount option '-o hottrack'

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 5:28 PM, Dave Chinner wrote: > On Sun, Sep 23, 2012 at 08:56:28PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> Introduce one new mount option '-o hottrack', >> and add its parsing support. >> Its usage looks like: >>mount -o hottrack >>mount -o

Re: [RFC v2 02/10] vfs: add support for updating access frequency

2012-09-25 Thread Zhi Yong Wu
thanks a lot for your review in my heart, Dave. It is very helpful to me. On Tue, Sep 25, 2012 at 5:17 PM, Dave Chinner wrote: > On Sun, Sep 23, 2012 at 08:56:27PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> Add some utils helpers to update access frequencies >> for one file

Re: [PATCH 2/2 v3] Btrfs: snapshot-aware defrag

2012-09-25 Thread Liu Bo
On 09/26/2012 01:39 AM, Mitch Harder wrote: > On Mon, Sep 17, 2012 at 4:58 AM, Liu Bo wrote: >> This comes from one of btrfs's project ideas, >> As we defragment files, we break any sharing from other snapshots. >> The balancing code will preserve the sharing, and defrag needs to grow this >> as w

Re: [PATCH v4 0/4] btrfs: extended inode refs

2012-09-25 Thread Chris Mason
On Mon, Aug 20, 2012 at 02:29:17PM -0600, Mark Fasheh wrote: > > Testing wise, the basic namespace operations work well (link, unlink, etc). > The rest has gotten less debugging (and I really don't have a great way of > testing the code in tree-log.c) Attached to this e-mail are btrfs-progs > patc

various processes blocked in btrfs_tree_read_lock for more than 120 secs

2012-09-25 Thread Jim Schutt
Hi, I'm hitting the following on a btrfs filesystem used as a Ceph OSD data store, under a heavy write load. My kernel is current Linus master (commit 56d27adcb536) merged with Josef Bacik's btrfs-next master (commit d5b04fb3bbb6). What can I do to help resolve this? [ 1558.754105] INFO: task

[PATCH] Btrfs: do not hold the file extent leaf locked when adding extent item

2012-09-25 Thread Josef Bacik
For some reason we unlock everything except the leaf we are on, set the path blocking and then add the extent item for the extent we just finished writing. I can't for the life of me figure out why we would want to do this, and the history doesn't really indicate that there was a real reason for i

[PATCH] Btrfs: do not async metadata csumming in certain situations

2012-09-25 Thread Josef Bacik
There are a coule scenarios where farming metadata csumming off to an async thread doesn't help. The first is if our processor supports crc32c, in which case the csumming will be fast and so the overhead of the async model is not worth the cost. The other case is for our tree log. We will be mak

Re: [PATCH 2/2 v3] Btrfs: snapshot-aware defrag

2012-09-25 Thread Mitch Harder
On Mon, Sep 17, 2012 at 4:58 AM, Liu Bo wrote: > This comes from one of btrfs's project ideas, > As we defragment files, we break any sharing from other snapshots. > The balancing code will preserve the sharing, and defrag needs to grow this > as well. > > Now we're able to fill the blank with thi

Re: [PATCH v2 2/2] btrfs-progs: Fix up memory leakage

2012-09-25 Thread Goffredo Baroncelli
On 09/25/2012 12:14 PM, David Sterba wrote: On Tue, Sep 25, 2012 at 10:02:16AM +0800, zwu.ker...@gmail.com wrote: From: Zhi Yong Wu Some code pathes forget to free memory on exit. Same as with the fd's, kernel will free all memory for us at exit(). I strongly disagree with this approach.

Re: ENOSPC design issues

2012-09-25 Thread Josef Bacik
On Tue, Sep 25, 2012 at 10:43:36AM -0600, David Sterba wrote: > On Thu, Sep 20, 2012 at 03:03:06PM -0400, Josef Bacik wrote: > > I'm going to look at fixing some of the performance issues that crop up > > because > > of our reservation system. Before I go and do a whole lot of work I want > > so

Re: ENOSPC design issues

2012-09-25 Thread David Sterba
On Thu, Sep 20, 2012 at 03:03:06PM -0400, Josef Bacik wrote: > I'm going to look at fixing some of the performance issues that crop up > because > of our reservation system. Before I go and do a whole lot of work I want some > feedback. I've done a brain dump here > https://btrfs.wiki.kernel.org

Re: [PATCH v2 2/2] btrfs-progs: Fix up memory leakage

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 6:14 PM, David Sterba wrote: > On Tue, Sep 25, 2012 at 10:02:16AM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> Some code pathes forget to free memory on exit. > > Same as with the fd's, kernel will free all memory for us at exit(). hi, can you let me kno

Re: [PATCH v2 1/2] btrfs-progs: Close file descriptor on exit

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 6:12 PM, David Sterba wrote: > On Tue, Sep 25, 2012 at 10:02:15AM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> Need to close fd on exit. > > Strictly you don't need to, kernel will do that at exit() time. I know, but it is not so nice. > > david --

Re: [PATCH] Btrfs: do not async metadata csums if we have hardware crc32c

2012-09-25 Thread David Sterba
On Tue, Sep 25, 2012 at 07:40:17PM +0800, ching wrote: > How about enhancing the "*thread_pool=/number" mount option instead? > > thread_pool=n enable threadpool/**/for compression and checksum, MAY > improve bandwidth/* > */thread_pool=0 disable threadpool for compression and checksum, M

Re: [PATCH] Btrfs: limit thread pool size when remounting

2012-09-25 Thread David Sterba
On Tue, Sep 25, 2012 at 08:07:00PM +0800, Liu Bo wrote: > On 09/25/2012 07:39 PM, David Sterba wrote: > > On Tue, Sep 25, 2012 at 02:48:33PM +0800, Liu Bo wrote: > >> --- a/fs/btrfs/super.c > >> +++ b/fs/btrfs/super.c > >> @@ -1158,17 +1158,20 @@ static void btrfs_resize_thread_pool(struct > >> bt

Re: [PATCH] Btrfs: limit thread pool size when remounting

2012-09-25 Thread Liu Bo
On 09/25/2012 07:39 PM, David Sterba wrote: > On Tue, Sep 25, 2012 at 02:48:33PM +0800, Liu Bo wrote: >> --- a/fs/btrfs/super.c >> +++ b/fs/btrfs/super.c >> @@ -1158,17 +1158,20 @@ static void btrfs_resize_thread_pool(struct >> btrfs_fs_info *fs_info, >> printk(KERN_INFO "btrfs: resize thread

Re: [PATCH] Btrfs: do not async metadata csums if we have hardware crc32c

2012-09-25 Thread ching
On 09/25/2012 06:51 PM, David Sterba wrote: > On Mon, Sep 24, 2012 at 11:03:49PM +0200, David Sterba wrote: >> Could you please put the check into a separate helper > Please note that checksum will become a variable per-filesystem > property, stored within the superblock, so the helper should be pa

Re: [PATCH] Btrfs: do not async metadata csums if we have hardware crc32c

2012-09-25 Thread ching
On 09/25/2012 06:51 PM, David Sterba wrote: > On Mon, Sep 24, 2012 at 11:03:49PM +0200, David Sterba wrote: >> Could you please put the check into a separate helper > Please note that checksum will become a variable per-filesystem > property, stored within the superblock, so the helper should be pa

Re: [PATCH] Btrfs: limit thread pool size when remounting

2012-09-25 Thread David Sterba
On Tue, Sep 25, 2012 at 02:48:33PM +0800, Liu Bo wrote: > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1158,17 +1158,20 @@ static void btrfs_resize_thread_pool(struct > btrfs_fs_info *fs_info, > printk(KERN_INFO "btrfs: resize thread pool %d -> %d\n", > old_pool_size, n

Re: [PATCH] Btrfs: do not async metadata csums if we have hardware crc32c

2012-09-25 Thread David Sterba
On Mon, Sep 24, 2012 at 11:03:49PM +0200, David Sterba wrote: > Could you please put the check into a separate helper Please note that checksum will become a variable per-filesystem property, stored within the superblock, so the helper should be passed a fs_info pointer. thanks, david -- To unsub

Re: [PATCH V3 1/2] Btrfs: cleanup duplicated division functions

2012-09-25 Thread Miao Xie
On mon, 24 Sep 2012 21:42:42 +0300, Ilya Dryomov wrote: > On Mon, Sep 24, 2012 at 06:47:42PM +0200, David Sterba wrote: >> On Mon, Sep 24, 2012 at 10:05:02AM +0800, Miao Xie wrote: >>> Generally, we should check the value when it is input. If not, we might >>> run our program with the wrong va

Re: [RFC v2 01/10] vfs: introduce private rb structures

2012-09-25 Thread Ram Pai
On Sun, Sep 23, 2012 at 08:56:26PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > One root structure hot_info is defined, is hooked > up in super_block, and will be used to hold rb trees > root, hash list root and some other information, etc. > Adds hot_inode_tree struct to keep t

Re: [PATCH v2 2/2] btrfs-progs: Fix up memory leakage

2012-09-25 Thread David Sterba
On Tue, Sep 25, 2012 at 10:02:16AM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > Some code pathes forget to free memory on exit. Same as with the fd's, kernel will free all memory for us at exit(). If there's lots of memory allocated, it may be even faster to leave the unallocati

Re: [PATCH v2 1/2] btrfs-progs: Close file descriptor on exit

2012-09-25 Thread David Sterba
On Tue, Sep 25, 2012 at 10:02:15AM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > Need to close fd on exit. Strictly you don't need to, kernel will do that at exit() time. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to m

Re: [RFC v2 05/10] vfs: introduce one hash table

2012-09-25 Thread Ram Pai
On Sun, Sep 23, 2012 at 08:56:30PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > Adds a hash table structure which contains > a lot of hash list and is used to efficiently > look up the data temperature of a file or its > ranges. > In each hash list of hash table, the hash node >

Re: [RFC v2 03/10] vfs: add one new mount option '-o hottrack'

2012-09-25 Thread Dave Chinner
On Sun, Sep 23, 2012 at 08:56:28PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > Introduce one new mount option '-o hottrack', > and add its parsing support. > Its usage looks like: >mount -o hottrack >mount -o nouser,hottrack >mount -o nouser,hottrack,loop >mount

Re: [RFC v2 02/10] vfs: add support for updating access frequency

2012-09-25 Thread Dave Chinner
On Sun, Sep 23, 2012 at 08:56:27PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > Add some utils helpers to update access frequencies > for one file or its range. > > Signed-off-by: Zhi Yong Wu > --- > fs/hot_tracking.c | 359 > ++

Re: [RFC v2 01/10] vfs: introduce private rb structures

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 3:37 PM, Dave Chinner wrote: > On Sun, Sep 23, 2012 at 08:56:26PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> One root structure hot_info is defined, is hooked >> up in super_block, and will be used to hold rb trees >> root, hash list root and some othe

Re: [RFC v2 01/10] vfs: introduce private rb structures

2012-09-25 Thread Zhi Yong Wu
On Tue, Sep 25, 2012 at 3:37 PM, Dave Chinner wrote: > On Sun, Sep 23, 2012 at 08:56:26PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> One root structure hot_info is defined, is hooked >> up in super_block, and will be used to hold rb trees >> root, hash list root and some othe

Re: [RFC v2 01/10] vfs: introduce private rb structures

2012-09-25 Thread Dave Chinner
On Sun, Sep 23, 2012 at 08:56:26PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > One root structure hot_info is defined, is hooked > up in super_block, and will be used to hold rb trees > root, hash list root and some other information, etc. > Adds hot_inode_tree struct to keep t