Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-29 Thread Fengguang Wu
On Wed, Aug 29, 2007 at 02:33:08AM +1000, David Chinner wrote: > On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: > > On Wed, 29 Aug 2007 00:55:30 +1000 > > David Chinner <[EMAIL PROTECTED]> wrote: > > > On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > > > > On Thu, Aug

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-29 Thread Fengguang Wu
On Wed, Aug 29, 2007 at 02:33:08AM +1000, David Chinner wrote: On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: On Wed, 29 Aug 2007 00:55:30 +1000 David Chinner [EMAIL PROTECTED] wrote: On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread Chris Mason
On Wed, 29 Aug 2007 02:33:08 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: > > > > > > > > I wonder if XFS can benefit any more from the general writeback > > > > clustering. How large would be a typical XFS cluster? > > > > > >

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: > On Wed, 29 Aug 2007 00:55:30 +1000 > David Chinner <[EMAIL PROTECTED]> wrote: > > On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > > > On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: > > > > On Wed, Aug

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread Chris Mason
On Wed, 29 Aug 2007 00:55:30 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > > On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: > > > On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > > > > On Tue, Aug

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: > > On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > > > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > > > Notes: > > > (1) I'm not sure

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: Notes: (1) I'm not sure inode number

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread Chris Mason
On Wed, 29 Aug 2007 00:55:30 +1000 David Chinner [EMAIL PROTECTED] wrote: On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread Chris Mason
On Wed, 29 Aug 2007 02:33:08 +1000 David Chinner [EMAIL PROTECTED] wrote: On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: I wonder if XFS can benefit any more from the general writeback clustering. How large would be a typical XFS cluster? Depends on inode size.

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: On Wed, 29 Aug 2007 00:55:30 +1000 David Chinner [EMAIL PROTECTED] wrote: On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Chris Mason
On Fri, 24 Aug 2007 21:24:58 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > > 2) s_dirty and s_io both become radix trees. s_dirty is indexed by > > a sequence number that corresponds to age. It is treated as a big > > circular indexed list that can wrap around over time. Radix tree > > tags

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: > On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > > Notes: > > (1) I'm not sure inode number is correlated to disk location in > > filesystems other

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: > > My vague idea is to > > - keep the s_io/s_more_io as a FIFO/cyclic writeback dispatching > > queue. > > - convert s_dirty to some radix-tree/rbtree based data structure. > > It would have dual functions: delayed-writeback and > >

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Thu, Aug 23, 2007 at 08:13:41AM -0400, Chris Mason wrote: > On Thu, 23 Aug 2007 12:47:23 +1000 > David Chinner <[EMAIL PROTECTED]> wrote: > > > On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: > > > I think we should assume a full scan of s_dirty is impossible in the > > > presence

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: My vague idea is to - keep the s_io/s_more_io as a FIFO/cyclic writeback dispatching queue. - convert s_dirty to some radix-tree/rbtree based data structure. It would have dual functions: delayed-writeback and

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Thu, Aug 23, 2007 at 08:13:41AM -0400, Chris Mason wrote: On Thu, 23 Aug 2007 12:47:23 +1000 David Chinner [EMAIL PROTECTED] wrote: On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: I think we should assume a full scan of s_dirty is impossible in the presence of

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Fengguang Wu
On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: Notes: (1) I'm not sure inode number is correlated to disk location in filesystems other than

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-24 Thread Chris Mason
On Fri, 24 Aug 2007 21:24:58 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: 2) s_dirty and s_io both become radix trees. s_dirty is indexed by a sequence number that corresponds to age. It is treated as a big circular indexed list that can wrap around over time. Radix tree tags are used

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-23 Thread Chris Mason
On Thu, 23 Aug 2007 12:47:23 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: > > I think we should assume a full scan of s_dirty is impossible in the > > presence of concurrent writers. We want to be able to pick a start > > time

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-23 Thread Chris Mason
On Thu, 23 Aug 2007 12:47:23 +1000 David Chinner [EMAIL PROTECTED] wrote: On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: I think we should assume a full scan of s_dirty is impossible in the presence of concurrent writers. We want to be able to pick a start time (right now)

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: > I think we should assume a full scan of s_dirty is impossible in the > presence of concurrent writers. We want to be able to pick a start > time (right now) and find all the inodes older than that start time. > New things will come in

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > Notes: > (1) I'm not sure inode number is correlated to disk location in > filesystems other than ext2/3/4. Or parent dir? The correspond to the exact location on

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread Chris Mason
On Wed, 22 Aug 2007 09:18:41 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > > On Sun, 12 Aug 2007 17:11:20 +0800 > > Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > > > Andrew and Ken, > > > > > > Here are some more experiments on

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread Chris Mason
On Wed, 22 Aug 2007 09:18:41 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: On Sun, 12 Aug 2007 17:11:20 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: Andrew and Ken, Here are some more experiments on the writeback stuff.

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: Notes: (1) I'm not sure inode number is correlated to disk location in filesystems other than ext2/3/4. Or parent dir? The correspond to the exact location on disk on

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: I think we should assume a full scan of s_dirty is impossible in the presence of concurrent writers. We want to be able to pick a start time (right now) and find all the inodes older than that start time. New things will come in

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-21 Thread Fengguang Wu
On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > On Sun, 12 Aug 2007 17:11:20 +0800 > Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > Andrew and Ken, > > > > Here are some more experiments on the writeback stuff. > > Comments are highly welcome~ > > I've been doing benchmarks

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-21 Thread Chris Mason
On Sun, 12 Aug 2007 17:11:20 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > Andrew and Ken, > > Here are some more experiments on the writeback stuff. > Comments are highly welcome~ I've been doing benchmarks lately to try and trigger fragmentation, and one of them is a simulation of make -j

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-21 Thread Chris Mason
On Sun, 12 Aug 2007 17:11:20 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: Andrew and Ken, Here are some more experiments on the writeback stuff. Comments are highly welcome~ I've been doing benchmarks lately to try and trigger fragmentation, and one of them is a simulation of make -j N. It

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-21 Thread Fengguang Wu
On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: On Sun, 12 Aug 2007 17:11:20 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: Andrew and Ken, Here are some more experiments on the writeback stuff. Comments are highly welcome~ I've been doing benchmarks lately to try and

[PATCH 0/6] writeback time order/delay fixes take 3

2007-08-12 Thread Fengguang Wu
Andrew and Ken, Here are some more experiments on the writeback stuff. Comments are highly welcome~ writeback fixes: [PATCH 1/6] writeback: fix time ordering of the per superblock inode lists 8 [PATCH 2/6] writeback: fix ntfs with sb_has_dirty_inodes() [PATCH 3/6]

[PATCH 0/6] writeback time order/delay fixes take 3

2007-08-12 Thread Fengguang Wu
Andrew and Ken, Here are some more experiments on the writeback stuff. Comments are highly welcome~ writeback fixes: [PATCH 1/6] writeback: fix time ordering of the per superblock inode lists 8 [PATCH 2/6] writeback: fix ntfs with sb_has_dirty_inodes() [PATCH 3/6]