[PATCH 09/13] writeback: requeue_io() on redirtied inode

2008-01-15 Thread Fengguang Wu
igned-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs-writeback.c |2 +- 1 files changed, 1 insertion(+), 1 deletion(-) --- linux-mm.orig/fs/fs-writeback.c +++ linux-mm/fs/fs-writeback.c @@ -294,7 +294,7 @@ __sync_single_inode(struct inode *inode, * Someone r

[PATCH 07/13] writeback: defer writeback on locked inode

2008-01-15 Thread Fengguang Wu
Convert to requeue_io_wait() for case: inode is locked. Cc: Michael Rubin <[EMAIL PROTECTED]> Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs-writeback.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-)

[PATCH 11/13] writeback: queue_dirty() on memory-backed bdi

2008-01-15 Thread Fengguang Wu
Replace redirty_tail() with queue_dirty() on memory backed bdi. It makes no difference - only simpler. Cc: Michael Rubin <[EMAIL PROTECTED]> Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs-writeback.c |2 +- 1 files changed,

[PATCH 06/13] writeback: defer writeback on not-all-pages-written

2008-01-15 Thread Fengguang Wu
into s_more_io_wait will be served _after_ normal inodes, hence won't stand in the way of other inodes in the next run. Cc: Michael Rubin <[EMAIL PROTECTED]> Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- f

[PATCH 00/13] writeback bug fixes and simplifications take 2

2008-01-15 Thread Fengguang Wu
fs/fs-writeback.c | 121 +++--- fs/super.c |1 include/linux/fs.h |1 mm/page-writeback.c | 46 +++ 5 files changed, 72 insertions(+), 99 deletions(-) Regards, Fengguang Wu -- -- To unsubscribe from this list: send the line "u

[PATCH 10/13] writeback: introduce queue_dirty()

2008-01-15 Thread Fengguang Wu
Introduce queue_dirty() to enqueue a newly dirtied inode. It helps remove duplicate code. Cc: Michael Rubin <[EMAIL PROTECTED]> Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs-writeback.c | 21 + 1

Re: [PATCH 00/13] writeback bug fixes and simplifications take 2

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 10:33:01AM -0800, Michael Rubin wrote: > On Jan 15, 2008 4:36 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > Andrew, > > > > This patchset mainly polishes the writeback queuing policies. > > Anyone know which tree is this patched based out

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 09:53:42AM -0800, Michael Rubin wrote: > On Jan 15, 2008 12:46 AM, Peter Zijlstra <[EMAIL PROTECTED]> wrote: > > Just a quick question, how does this interact/depend-uppon etc.. with > > Fengguangs patches I still have in my mailbox? (Those from Dec 28th) > > They don't. Th

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 07:44:15PM -0800, Andrew Morton wrote: > On Wed, 16 Jan 2008 11:01:08 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > On Tue, Jan 15, 2008 at 09:53:42AM -0800, Michael Rubin wrote: > > > On Jan 15, 2008 12:46 AM, Peter Zijlstra <[EMAIL PRO

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 08:42:36PM -0800, Andrew Morton wrote: > On Wed, 16 Jan 2008 12:25:53 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > list_heads are OK if we use them for one and only function. > > Not really. They're inappropriate when you wish to re

Re: regression: 100% io-wait with 2.6.24-rcX

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 10:42:13PM +0100, Ingo Molnar wrote: > > * Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > On Mon, Jan 14, 2008 at 12:41:26PM +0100, Peter Zijlstra wrote: > > > > > > On Mon, 2008-01-14 at 12:30 +0100, Joerg Platte wrote: >

Re: regression: 100% io-wait with 2.6.24-rcX

2008-01-15 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 04:13:22PM -0500, Mike Snitzer wrote: > On Jan 14, 2008 7:50 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > On Mon, Jan 14, 2008 at 12:41:26PM +0100, Peter Zijlstra wrote: > > > > > > On Mon, 2008-01-14 at 12:30 +0100, Joerg Platte wrote: &

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-16 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 09:51:49PM -0800, Andrew Morton wrote: > On Wed, 16 Jan 2008 12:55:07 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > On Tue, Jan 15, 2008 at 08:42:36PM -0800, Andrew Morton wrote: > > > On Wed, 16 Jan 2008 12:25:53 +0800 Fengguang W

Re: regression: 100% io-wait with 2.6.24-rcX

2008-01-16 Thread Fengguang Wu
On Wed, Jan 16, 2008 at 01:26:41AM -0800, Martin Knoblauch wrote: > > For those interested in using your writeback improvements in > > production sooner rather than later (primarily with ext3); what > > recommendations do you have? Just heavily test our own 2.6.24 + your > > evolving "close, but n

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-16 Thread Fengguang Wu
On Wed, Jan 16, 2008 at 12:13:43AM -0800, Andrew Morton wrote: > On Wed, 16 Jan 2008 18:55:38 +1100 David Chinner <[EMAIL PROTECTED]> wrote: > > > On Tue, Jan 15, 2008 at 07:44:15PM -0800, Andrew Morton wrote: > > > On Wed, 16 Jan 2008 11:01:08 +0800 Fengguang W

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-16 Thread Fengguang Wu
On Thu, Jan 17, 2008 at 09:35:10AM +1100, David Chinner wrote: > On Wed, Jan 16, 2008 at 05:07:20PM +0800, Fengguang Wu wrote: > > On Tue, Jan 15, 2008 at 09:51:49PM -0800, Andrew Morton wrote: > > > > Then to do better ordering by adopting radix tree(or rbtree > > >

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-16 Thread Fengguang Wu
On Wed, Jan 16, 2008 at 10:55:28AM -0800, Michael Rubin wrote: > On Jan 15, 2008 7:01 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > Basically I think rbtree is an overkill to do time based ordering. > > Sorry, Michael. But s_dirty would be enough for that. Plus, s_more

Re: [PATCH 09/13] writeback: requeue_io() on redirtied inode

2008-01-16 Thread Fengguang Wu
On Wed, Jan 16, 2008 at 07:13:07PM +1100, David Chinner wrote: > On Tue, Jan 15, 2008 at 08:36:46PM +0800, Fengguang Wu wrote: > > Redirtied inodes could be seen in really fast writes. > > They should really be synced as soon as possible. > > > > redirty_tail() could de

[PATCH] writeback: speed up writeback of big dirty files

2008-01-17 Thread Fengguang Wu
t more_io on super_blocks we actually visited. This aviods the interaction between two pdflush deamons. Also in __sync_single_inode() we don't blindly keep requeuing the io if the filesystem cannot progress. Failing to do so may lead to 100% iowait. Tested-by: Mike Snitzer <[EMAIL

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Fengguang Wu
On Tue, Jan 15, 2008 at 12:09:21AM -0800, Michael Rubin wrote: > 1) Adding a datastructure to guarantee fairness when writing >inodes to disk and simplify the code base. > >When inodes are parked for writeback they are parked in the >flush_tree. The flush tree is a data structure based

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Fengguang Wu
On Thu, Jan 17, 2008 at 01:07:05PM -0800, Michael Rubin wrote: > On Jan 17, 2008 1:41 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > On Tue, Jan 15, 2008 at 12:09:21AM -0800, Michael Rubin wrote: > > The main benefit of rbtree is possibly better support of future policies. &g

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Fengguang Wu
> Fairness is a tradeoff between seeks and bandwidth. Ideally, we > want to spend 50% of the *disk* time servicing sequential writes and > 50% of the time servicing seeky writes - that way neither get > penalised unfairly by the other type of workload. > > Switching inodes during writeback implie

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-18 Thread Fengguang Wu
On Fri, Jan 18, 2008 at 06:41:09AM +0100, Andi Kleen wrote: > Fengguang Wu <[EMAIL PROTECTED]> writes: > > > > Suppose we want to grant longer expiration window for temp files, > > adding a new list named s_dirty_tmpfile would be a handy solution. > > How would

Re: [PATCH 00/13] writeback bug fixes and simplifications take 2

2008-01-18 Thread Fengguang Wu
On Thu, Jan 17, 2008 at 11:51:51PM -0800, Michael Rubin wrote: > On Jan 15, 2008 4:36 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > Andrew, > > > > This patchset mainly polishes the writeback queuing policies. > > The main goals are: > > > > (1) s

[PATCH 3/9] readahead: auto detection of sequential mmap reads

2007-12-16 Thread Fengguang Wu
nservative to require valid readahead state for async readahead, which means we don't do readahead for interleaved reads for now, but let's make it safe for this initial try. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- --- mm/filemap.c |6 -- 1 file changed, 4

[PATCH 0/9] mmap read-around and readahead

2007-12-16 Thread Fengguang Wu
Andrew, Here are the mmap read-around related patches initiated by Linus. They are for linux-2.6.24-rc4-mm1. The one major new feature - auto detection and early readahead for mmap sequential reads - runs as expected on my desktop :-) [PATCH 1/9] readahead: simplify readahead call scheme [PATCH

[PATCH 9/9] readahead: call max_sane_readahead() in ondemand_readahead()

2007-12-16 Thread Fengguang Wu
Apply the max_sane_readahead() limit in ondemand_readahead(). Just in case someone aggressively set a huge readahead size. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/readahead.c |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- linux-2.6.24-rc4-mm1.orig/mm/reada

[PATCH 5/9] readahead: make ra_submit() non-static

2007-12-16 Thread Fengguang Wu
Make ra_submit() non-static and callable from other files. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- --- include/linux/mm.h |3 +++ mm/readahead.c |2 +- 2 files changed, 4 insertions(+), 1 deletion(-) --- linux-2.6.24-rc4-mm1.orig/include/linux/mm.h +++ linux-2.6.

[PATCH 7/9] readahead: remove unused do_page_cache_readahead()

2007-12-16 Thread Fengguang Wu
executables. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/mm.h |2 -- mm/readahead.c | 16 2 files changed, 18 deletions(-) --- linux-2.6.24-rc4-mm1.orig/include/linux/mm.h +++ linux-2.6.24-rc4-mm1/include/linux/mm.h @@ -1084,8 +1084,6 @@ int write_on

[PATCH 4/9] readahead: quick startup on sequential mmap readahead

2007-12-16 Thread Fengguang Wu
ubled to the full readahead size anyway. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/filemap.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- linux-2.6.24-rc4-mm1.orig/mm/filemap.c +++ linux-2.6.24-rc4-mm1/mm/filemap.c @@ -1320,7 +1320,7 @@ static void do_sync_mma

[PATCH 1/9] readahead: simplify readahead call scheme

2007-12-16 Thread Fengguang Wu
)) page_cache_async_readahead(...); Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/readahead.c |8 1 file changed, 8 insertions(+) --- linux-2.6.24-rc4-mm1.orig/mm/readahead.c +++ linux-2.6.24-rc4-mm1/mm/readahead.c @@ -402,6 +

[PATCH 6/9] readahead: save mmap read-around states in file_ra_state

2007-12-16 Thread Fengguang Wu
Change mmap read-around to share the same code style and data structure with readahead code. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/filemap.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) --- linux-2.6.24-rc4-mm1.orig/mm/filemap.c +++ linux-2.6.

[PATCH 8/9] readahead: move max_sane_readahead() calls into force_page_cache_readahead()

2007-12-16 Thread Fengguang Wu
Simplify code by moving max_sane_readahead() calls into force_page_cache_readahead(). Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/fadvise.c |2 +- mm/filemap.c |3 +-- mm/madvise.c |3 +-- mm/readahead.c |1 + 4 files changed, 4 insertions(+), 5 del

[PATCH 2/9] readahead: clean up and simplify the code for filemap page fault readahead

2007-12-16 Thread Fengguang Wu
rather than inside some subtle loop and thus avoiding the now unnecessary extra state variables (ie "did_readaround" is gone). Cc: Nick Piggin <[EMAIL PROTECTED]> Cc: Andrew Morton <[EMAIL PROTECTED]> Cc: Fengguang Wu <[EMAIL PROTECTED]> Signed-off-by: Linus Torvalds

Re: [PATCH 0/9] mmap read-around and readahead

2007-12-18 Thread Fengguang Wu
On Sun, Dec 16, 2007 at 03:35:58PM -0800, Linus Torvalds wrote: > > > On Sun, 16 Dec 2007, Fengguang Wu wrote: > > > > Here are the mmap read-around related patches initiated by Linus. > > They are for linux-2.6.24-rc4-mm1. The one major new feature - > > auto

Re: [PATCH 2/9] readahead: clean up and simplify the code for filemap page fault readahead

2007-12-18 Thread Fengguang Wu
On Tue, Dec 18, 2007 at 09:19:07AM +0100, Nick Piggin wrote: > On Sun, Dec 16, 2007 at 07:59:30PM +0800, Fengguang Wu wrote: > > + read_lock_irq(&mapping->tree_lock); > > + page = radix_tree_lookup(&mapping->page_tree, offset); > > + if (likely(pa

Re: [PATCH 0/9] mmap read-around and readahead

2007-12-18 Thread Fengguang Wu
On Tue, Dec 18, 2007 at 07:46:09PM +0800, Fengguang Wu wrote: > No timings for now... but I wrote a debug patch(attached) and watched > it running for about a week. Here are some interesting numbers: Here are the (forgotten) readahead-debug.patch: --- include/linux/fs.h

Re: [PATCH 2/9] readahead: clean up and simplify the code for filemap page fault readahead

2007-12-18 Thread Fengguang Wu
On Wed, Dec 19, 2007 at 12:54:23AM +0100, Nick Piggin wrote: > On Tue, Dec 18, 2007 at 07:50:33PM +0800, Fengguang Wu wrote: > > On Tue, Dec 18, 2007 at 09:19:07AM +0100, Nick Piggin wrote: > > > On Sun, Dec 16, 2007 at 07:59:30PM +0800, Fengguang Wu wrote: > > >

Re: [PATCH 0/9] mmap read-around and readahead

2007-12-18 Thread Fengguang Wu
On Sun, Dec 16, 2007 at 03:35:58PM -0800, Linus Torvalds wrote: > > > On Sun, 16 Dec 2007, Fengguang Wu wrote: > > > > Here are the mmap read-around related patches initiated by Linus. > > They are for linux-2.6.24-rc4-mm1. The one major new feature - > > auto

[PATCH] writeback: speed up writeback of big dirty files

2008-01-27 Thread Fengguang Wu
m cannot progress. Failing to do so may lead to 100% iowait. Tested-by: Mike Snitzer <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs-writeback.c | 18 -- include/linux/writeback.h |1 + mm/page-writeback.c |

Re: Many processes stuck in blk_congestion_wait

2008-01-27 Thread Fengguang Wu
On Wed, Jan 23, 2008 at 05:19:14PM -0500, Sev Binello wrote: > We are experiencing numerous hangs/slow downs on our WS4 U5 nfs client > machines (not seen on WS3). > Many processes end up in a D state for long periods of time > and are in blk_congestion_wait(). > > Time in this state can vary, >

Re: pdflush stuck in D state with v2.6.24-rc1-192-gef49c32

2007-10-29 Thread Fengguang Wu
ountered_congestion) congestion_wait(WRITE, HZ/10); Subject: track redirty_tail() calls It helps a lot to know how redirty_tail() are called. Cc: Ken Chen <[EMAIL PROTECTED]> Cc: Andrew Morton <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/fs

Re: pdflush stuck in D state with v2.6.24-rc1-192-gef49c32

2007-10-30 Thread Fengguang Wu
On Tue, Oct 30, 2007 at 06:42:50AM -0500, Florin Iucha wrote: > On Tue, Oct 30, 2007 at 03:54:03PM +0800, Fengguang Wu wrote: > > It could be triggered by the more aggressive writeback behavior - the > > new code will keep on retrying as long as there are dirty inodes pending.

Re: pdflush stuck in D state with v2.6.24-rc1-192-gef49c32

2007-10-30 Thread Fengguang Wu
On Tue, Oct 30, 2007 at 10:52:45PM -0500, Florin Iucha wrote: > On Tue, Oct 30, 2007 at 07:02:42PM -0500, Florin Iucha wrote: > > I have added the patches and started a linux kernel compilation, and > > something really interesting happens. I run the build with the > > equivalent of "make -j3" and

Re: pdflush stuck in D state with v2.6.24-rc1-192-gef49c32

2007-10-31 Thread Fengguang Wu
On Wed, Oct 31, 2007 at 12:53:18PM -0500, Florin Iucha wrote: > On Wed, Oct 31, 2007 at 07:16:06AM -0500, Florin Iucha wrote: > > On Wed, Oct 31, 2007 at 02:53:25PM +0800, Fengguang Wu wrote: > > > On Tue, Oct 30, 2007 at 10:52:45PM -0500, Florin Iucha wrote: > > > >

Re: 100% iowait on one of cpus in current -git

2007-10-31 Thread Fengguang Wu
On Wed, Oct 31, 2007 at 04:22:10PM +0100, Torsten Kaiser wrote: > On 10/22/07, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: > > > Hmm, Maybe it's an reiserfs related issue. Do you have the full log f

Re: pdflush stuck in D state with v2.6.24-rc1-192-gef49c32

2007-11-01 Thread Fengguang Wu
On Thu, Nov 01, 2007 at 07:25:58AM -0500, Florin Iucha wrote: > On Thu, Nov 01, 2007 at 03:15:32PM +0800, Fengguang Wu wrote: > > On Wed, Oct 31, 2007 at 12:53:18PM -0500, Florin Iucha wrote: > > > This patch does not fix anything for me. Even such light use of the > >

[PATCH 1/6] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
now. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h | 10 +- mm/filemap.c |2 +- mm/readahead.c |5 +++-- 3 files changed, 9 insertions(+), 8 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linux/fs.h +++ linux-2.6.22-rc6-mm1/include/lin

[PATCH 3/6] remove several readahead macros

2007-07-20 Thread Fengguang Wu
Remove VM_MAX_CACHE_HIT, MAX_RA_PAGES and MIN_RA_PAGES. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/mm.h |2 -- mm/readahead.c | 10 +- 2 files changed, 1 insertion(+), 11 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linux/mm.h +++ linux-2.6.

[PATCH 5/6] introduce radix_tree_scan_hole()

2007-07-20 Thread Fengguang Wu
Introduce radix_tree_scan_hole(root, index, max_scan) to scan radix tree for the first hole. It will be used in interleaved readahead. The implementation is dumb and obviously correct. It can help debug the possible smart one in future. Signed-off-by: Fengguang Wu <[EMAIL PROTEC

[PATCH 4/6] remove the limit max_sectors_kb imposed on max_readahead_kb

2007-07-20 Thread Fengguang Wu
Remove the size limit max_sectors_kb imposed on max_readahead_kb. max_sectors_kb cannot grow larger than max_hw_sectors_kb, which can be rather small for some disk drives. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- block/ll_rw_blk.c |9 - 1 file changed, 9 del

[PATCH 0/6] readahead cleanups and interleaved readahead

2007-07-20 Thread Fengguang Wu
|2 - include/linux/radix-tree.h |2 + lib/radix-tree.c | 34 mm/filemap.c |6 ++-- mm/readahead.c | 48 +++ 7 files changed, 72 insertions(+), 42 deletions(-) Regards, Fengguang Wu

[PATCH 6/6] basic support of interleaved reads

2007-07-20 Thread Fengguang Wu
1006 131007 Here steam 1 will start a readahead at page 2, and stream 2 will start its first readahead at page 1003. From then on the two streams will be served right. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/readahead.c | 33 ++

[PATCH 2/6] mmap read-around simplification

2007-07-20 Thread Fengguang Wu
Fold file_ra_state.mmap_hit into file_ra_state.mmap_miss and make it an int. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h |3 +-- mm/filemap.c |4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linu

Re: [PATCH 1/6] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
On Fri, Jul 20, 2007 at 12:48:59PM +0200, Peter Zijlstra wrote: > On Fri, 2007-07-20 at 12:26 +0200, Peter Zijlstra wrote: > > On Fri, 2007-07-20 at 18:07 +0800, Fengguang Wu wrote: > > > plain text document attachment (short-rasize.patch) > > > Use 'unsigned int

Re: [PATCH 4/6] remove the limit max_sectors_kb imposed on max_readahead_kb

2007-07-20 Thread Fengguang Wu
On Fri, Jul 20, 2007 at 01:07:43PM +0200, Jens Axboe wrote: > On Fri, Jul 20 2007, Fengguang Wu wrote: > > Remove the size limit max_sectors_kb imposed on max_readahead_kb. > > > > max_sectors_kb cannot grow larger than max_hw_sectors_kb, which can be > > rather

Re: [PATCH 1/6] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
On Fri, Jul 20, 2007 at 01:34:42PM +0200, Andi Kleen wrote: > Fengguang Wu <[EMAIL PROTECTED]> writes: > > > Use 'unsigned int' instead of 'unsigned long' for the readahead > > indexes/sizes. > > > > This helps reduce memory con

[PATCH 0/8] readahead cleanups and interleaved readahead take 2

2007-07-20 Thread Fengguang Wu
gards, Fengguang Wu --- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/

[PATCH 8/8] basic support of interleaved reads

2007-07-20 Thread Fengguang Wu
1006 131007 Here steam 1 will start a readahead at page 2, and stream 2 will start its first readahead at page 1003. From then on the two streams will be served right. Cc: Nick Piggin <[EMAIL PROTECTED]> Cc: Rusty Russell <[EMAIL PROTECTED]> Sig

[PATCH 1/8] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
Use 'unsigned int' instead of 'unsigned long' for readahead sizes. This helps reduce memory consumption on 64bit CPU when a lot of files are opened. CC: Andi Kleen <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h

[PATCH 5/8] remove several readahead macros

2007-07-20 Thread Fengguang Wu
Remove VM_MAX_CACHE_HIT, MAX_RA_PAGES and MIN_RA_PAGES. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/mm.h |2 -- mm/readahead.c | 10 +- 2 files changed, 1 insertion(+), 11 deletions(-) --- linux-2.6.22-git15.orig/include/linux/mm.h +++ linux-2.6.22

[PATCH 6/8] remove the limit max_sectors_kb imposed on max_readahead_kb

2007-07-20 Thread Fengguang Wu
Remove the size limit max_sectors_kb imposed on max_readahead_kb. The size restriction is unreasonable. Especially when max_sectors_kb cannot grow larger than max_hw_sectors_kb, which can be rather small for some disk drives. Cc: Jens Axboe <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu &

[PATCH 7/8] introduce radix_tree_scan_hole()

2007-07-20 Thread Fengguang Wu
Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/radix-tree.h |2 ++ lib/radix-tree.c | 34 ++ 2 files changed, 36 insertions(+) --- linux-2.6.22-git15.orig/include/linux/radix-tree.h +++ linux-2.6.22-git15/include/linux/

[PATCH 3/8] combine file_ra_state.prev_index/prev_offset into prev_pos

2007-07-20 Thread Fengguang Wu
! Cc: Peter Zijlstra <[EMAIL PROTECTED]> Cc: Christoph Lameter <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/ext3/dir.c |2 +- fs/ext4/dir.c |2 +- fs/splice.c|2 +- include/linux/fs.h |3 +-- mm/filemap.c |

[PATCH 4/8] trivial filemap.c cleanups

2007-07-20 Thread Fengguang Wu
- remove unused local next_index in do_generic_mapping_read() - convert some 'unsigned long' to pgoff_t - wrap a long line Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/filemap.c | 16 +++- 1 file changed, 7 insertions(+), 9 deletions(-) --- linux-2.6.

[PATCH 2/8] mmap read-around simplification

2007-07-20 Thread Fengguang Wu
Fold file_ra_state.mmap_hit into file_ra_state.mmap_miss and make it an int. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h |3 +-- mm/filemap.c |4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) --- linux-2.6.22-git15.orig/include/linu

Re: [PATCH 1/8] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
Sorry, forgot to prefix the patch titles with [readahead]. Should I repost? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.

Re: [PATCH 1/8] compacting file_ra_state

2007-07-20 Thread Fengguang Wu
On Fri, Jul 20, 2007 at 09:27:01PM -0700, Linus Torvalds wrote: > > > On Sat, 21 Jul 2007, Fengguang Wu wrote: > > > > Sorry, forgot to prefix the patch titles with [readahead]. > > Should I repost? > > Not for me, but on the other hand, I'd prefer for

[PATCH 5/7] readahead: remove the limit max_sectors_kb imposed on max_readahead_kb

2007-07-20 Thread Fengguang Wu
Remove the size limit max_sectors_kb imposed on max_readahead_kb. The size restriction is unreasonable. Especially when max_sectors_kb cannot grow larger than max_hw_sectors_kb, which can be rather small for some disk drives. Cc: Jens Axboe <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu &

[PATCH 4/7] readahead: remove several readahead macros

2007-07-20 Thread Fengguang Wu
Remove VM_MAX_CACHE_HIT, MAX_RA_PAGES and MIN_RA_PAGES. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/mm.h |2 -- mm/readahead.c | 10 +- 2 files changed, 1 insertion(+), 11 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linux/mm.h +++ linux-2.6.

[PATCH 7/7] readahead: basic support of interleaved reads

2007-07-20 Thread Fengguang Wu
1006 131007 Here steam 1 will start a readahead at page 2, and stream 2 will start its first readahead at page 1003. From then on the two streams will be served right. Cc: Rusty Russell <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED

[PATCH 1/7] readahead: compacting file_ra_state

2007-07-20 Thread Fengguang Wu
Use 'unsigned int' instead of 'unsigned long' for readahead sizes. This helps reduce memory consumption on 64bit CPU when a lot of files are opened. CC: Andi Kleen <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs

[PATCH 6/7] radixtree: introduce radix_tree_scan_hole()

2007-07-20 Thread Fengguang Wu
Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/radix-tree.h |2 ++ lib/radix-tree.c | 34 ++ 2 files changed, 36 insertions(+) --- linux-2.6.22-rc6-mm1.orig/include/linux/radix-tree.h +++ linux-2.6.22-rc6-mm1/include/linux/

[PATCH 2/7] readahead: mmap read-around simplification

2007-07-20 Thread Fengguang Wu
Fold file_ra_state.mmap_hit into file_ra_state.mmap_miss and make it an int. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h |3 +-- mm/filemap.c |4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linu

[PATCH 3/7] readahead: combine file_ra_state.prev_index/prev_offset into prev_pos

2007-07-20 Thread Fengguang Wu
! Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/ext3/dir.c |2 +- fs/ext4/dir.c |2 +- fs/splice.c|2 +- include/linux/fs.h |3 +-- mm/filemap.c | 11 ++- mm/readahead.c | 15 ++

[PATCH 0/7] readahead cleanups and interleaved readahead take 3

2007-07-20 Thread Fengguang Wu
/mm.h |2 - include/linux/radix-tree.h |2 + lib/radix-tree.c | 34 mm/filemap.c | 17 +- mm/readahead.c | 58 +++ 10 files changed, 86 insertions(+), 56 deletions(-) Regards, Fenggua

Re: [PATCH 6/7] radixtree: introduce radix_tree_scan_hole()

2007-07-20 Thread Fengguang Wu
On Fri, Jul 20, 2007 at 10:48:59PM -0700, Andrew Morton wrote: > On Sat, 21 Jul 2007 12:43:06 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > Introduce radix_tree_scan_hole(root, index, max_scan) to scan radix tree > > for the first hole. It will be used in interl

Re: [PATCH 7/7] readahead: basic support of interleaved reads

2007-07-21 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 05:24:39PM +1000, Rusty Russell wrote: > This is really clever! Thank you :) > Only one slight complaint: I wonder if "radix_tree_scan_hole" could be > expressed as "radix_tree_extent_size" which return the number of In fact the full context based readahead re

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 11:00:05PM +0200, Peter Zijlstra wrote: > The various readahead bits I have lying about. > > Wu, would you agree with asking Andrew to stick these behind your latest > readahead series? They are generally good feature to have. Thank you! - default readahead size It makes

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 10:44:28PM -0400, Dave Jones wrote: > On Sun, Jul 22, 2007 at 10:39:23AM +0800, Fengguang Wu wrote: > > > It makes sense to raise it beyond 128K. 1M default readahead > > absolutely makes sense for sequential workloads. For the desktop, > > t

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sun, Jul 22, 2007 at 10:24:37AM +0200, Peter Zijlstra wrote: > On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > > - It will avoid large-file-reads-thrashing-my-desktop problem, > > so most desktop users should like it. But sure there will be counter > > c

Re: [PATCH 3/3] readahead: scale max readahead size depending on memory size

2007-07-22 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 11:00:08PM +0200, Peter Zijlstra wrote: > Scale the default max readahead size with the system memory size. > > Signed-off-by: Peter Zijlstra <[EMAIL PROTECTED]> > --- > block/ll_rw_blk.c |2 +- > include/linux/fs.h |1 + > mm/readahead.c | 32 ++

Re: [PATCH 3/3] readahead: scale max readahead size depending on memory size

2007-07-22 Thread Fengguang Wu
On Sun, Jul 22, 2007 at 10:59:11AM +0200, Peter Zijlstra wrote: > On Sun, 2007-07-22 at 16:45 +0800, Fengguang Wu wrote: > > How about the following rules? > > - limit it under 1MB: we have to consider latencies > > readahead is done async and we have these cond_resched() t

Re: [PATCH 6/7] radixtree: introduce radix_tree_scan_hole()

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 05:58:02PM +1000, Nick Piggin wrote: > Fengguang Wu wrote: > >Introduce radix_tree_scan_hole(root, index, max_scan) to scan radix tree > >for the first hole. It will be used in interleaved readahead. > > > >The implementation is dumb and obviou

Re: [PATCH 6/7] radixtree: introduce radix_tree_scan_hole()

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 04:04:05PM +0800, Fengguang Wu wrote: > On Mon, Jul 23, 2007 at 05:58:02PM +1000, Nick Piggin wrote: > > Fengguang Wu wrote: > > >Introduce radix_tree_scan_hole(root, index, max_scan) to scan radix tree > > >for the first hole. It will be us

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: > Rusty Russell wrote: > >On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > >>So I opt for it being made tunable, safe, and turned off by default. > > I hate tunables :) Unless we have worklo

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 12:40:09PM -0700, Andrew Morton wrote: > On Mon, 23 Jul 2007 22:24:57 +0800 > Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: > > > Rusty Russell wrote: > > > >On Sun, 200

[PATCH 00/10] readahead cleanups and interleaved readahead take 4

2007-07-23 Thread Fengguang Wu
--- mm/readahead.c | 58 +-- 11 files changed, 122 insertions(+), 93 deletions(-) Regards, Fengguang Wu -- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majo

[PATCH 04/10] radixtree: introduce radix_tree_scan_hole()

2007-07-23 Thread Fengguang Wu
Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/radix-tree.h |2 + lib/radix-tree.c | 36 +++ 2 files changed, 38 insertions(+) --- linux-2.6.22-rc6-mm1.orig/include/linux/radix-tree.h +++ linux-2.6.22-rc6-mm1/include/linux/

[PATCH 07/10] readahead: remove several readahead macros

2007-07-23 Thread Fengguang Wu
Remove VM_MAX_CACHE_HIT, MAX_RA_PAGES and MIN_RA_PAGES. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/mm.h |2 -- mm/readahead.c | 10 +- 2 files changed, 1 insertion(+), 11 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linux/mm.h +++ linux-2.6.

[PATCH 02/10] readahead: mmap read-around simplification

2007-07-23 Thread Fengguang Wu
Fold file_ra_state.mmap_hit into file_ra_state.mmap_miss and make it an int. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs.h |3 +-- mm/filemap.c |4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) --- linux-2.6.22-rc6-mm1.orig/include/linu

[PATCH 06/10] readahead: remove the local copy of ra in do_generic_mapping_read()

2007-07-23 Thread Fengguang Wu
in: Linux: Random File I/O Regressions In 2.6 <http://kerneltrap.org/node/3039> Cc: Nick Piggin <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/filemap.c | 20 +--- 1 file changed, 9 insertions(+), 11 deletions(-) --- linux-2.6.

[PATCH 01/10] readahead: compacting file_ra_state

2007-07-23 Thread Fengguang Wu
Use 'unsigned int' instead of 'unsigned long' for readahead sizes. This helps reduce memory consumption on 64bit CPU when a lot of files are opened. CC: Andi Kleen <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/fs

[PATCH 08/10] readahead: remove the limit max_sectors_kb imposed on max_readahead_kb

2007-07-23 Thread Fengguang Wu
Remove the size limit max_sectors_kb imposed on max_readahead_kb. The size restriction is unreasonable. Especially when max_sectors_kb cannot grow larger than max_hw_sectors_kb, which can be rather small for some disk drives. Cc: Jens Axboe <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu &

[PATCH 05/10] readahead: basic support of interleaved reads

2007-07-23 Thread Fengguang Wu
1006 131007 Here steam 1 will start a readahead at page 2, and stream 2 will start its first readahead at page 1003. From then on the two streams will be served right. Cc: Rusty Russell <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED

[PATCH 09/10] filemap: trivial code cleanups

2007-07-23 Thread Fengguang Wu
- remove unused local next_index in do_generic_mapping_read() - wrap a long line - remove a redudant page_cache_read() declaration Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- mm/filemap.c |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) --- linux-2.6.22-rc6-mm1.o

[PATCH 03/10] readahead: combine file_ra_state.prev_index/prev_offset into prev_pos

2007-07-23 Thread Fengguang Wu
! Cc: Peter Zijlstra <[EMAIL PROTECTED]> Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- fs/ext3/dir.c |2 +- fs/ext4/dir.c |2 +- fs/splice.c|2 +- include/linux/fs.h |3 +-- mm/filemap.c | 11 ++- mm/readahead.c | 15 ++

[PATCH 10/10] filemap: convert some unsigned long to pgoff_t

2007-07-23 Thread Fengguang Wu
Convert some 'unsigned long' to pgoff_t. Signed-off-by: Fengguang Wu <[EMAIL PROTECTED]> --- include/linux/pagemap.h | 23 --- mm/filemap.c| 32 2 files changed, 28 insertions(+), 27 deletions(-) --- linux-2.6

Re: [PATCH 03/10] readahead: combine file_ra_state.prev_index/prev_offset into prev_pos

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 08:52:41PM -0700, Andrew Morton wrote: > On Tue, 24 Jul 2007 10:00:12 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > - ra->prev_index = page->index; > > + ra->prev_pos = page->index << PAGE_CACHE_SHIFT; > > bug! Th

Re: [PATCH 03/10] readahead: combine file_ra_state.prev_index/prev_offset into prev_pos

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 08:55:35PM -0700, Andrew Morton wrote: > On Tue, 24 Jul 2007 10:00:12 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > @@ -342,11 +342,9 @@ ondemand_readahead(struct address_space > >bool hit_readahead_marker, pgoff_t offset, >

<    1   2   3   4   5   6   7   8   9   10   >