Re: [RFC] basic delayed allocation in VFS

2007-07-30 Thread Andrew Morton
On Mon, 30 Jul 2007 10:49:14 -0700 Mingming Cao <[EMAIL PROTECTED]> wrote: > On Sun, 2007-07-29 at 20:24 +0100, Christoph Hellwig wrote: > > On Sun, Jul 29, 2007 at 11:30:36AM -0600, Andreas Dilger wrote: > > > Sigh, we HAVE a patch that was only adding delalloc to ext4, but it > > > was rejected

Re: [RFC] basic delayed allocation in VFS

2007-07-30 Thread Mingming Cao
On Sun, 2007-07-29 at 20:24 +0100, Christoph Hellwig wrote: > On Sun, Jul 29, 2007 at 11:30:36AM -0600, Andreas Dilger wrote: > > Sigh, we HAVE a patch that was only adding delalloc to ext4, but it > > was rejected because "that functionality should go into the VFS". > > Since the performance impro

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread David Chinner
On Sun, Jul 29, 2007 at 04:09:20PM +0400, Alex Tomas wrote: > David Chinner wrote: > >On Fri, Jul 27, 2007 at 11:51:56AM +0400, Alex Tomas wrote: > >But this is really irrelevant - the issue at hand is what we want > >for VFS level delalloc support. IMO, that mechanism needs to support > >both XFS

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Theodore Tso
On Sun, Jul 29, 2007 at 08:24:37PM +0100, Christoph Hellwig wrote: > I'm a big proponent of having proper common delalloc code, but the > one proposed here is not generic for the existing filesystem using > delalloc. It's still on my todo list to revamp the xfs code to get > rid of some of the exi

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Alex Tomas
I'm a bit worried about one thing ... it looks like XFS and ext4 use different techniques to order data and metadata referencing them. now I'm not that optimistic that we can separate ordering from delalloc itself clean and reasonable way. In general, I'd prefer common code in fs/ (mm/?) of course

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Christoph Hellwig
On Sun, Jul 29, 2007 at 11:30:36AM -0600, Andreas Dilger wrote: > Sigh, we HAVE a patch that was only adding delalloc to ext4, but it > was rejected because "that functionality should go into the VFS". > Since the performance improvement of delalloc is quite large, we'd > like to get this into the

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Christoph Hellwig
On Sun, Jul 29, 2007 at 09:48:10PM +0400, Alex Tomas wrote: > I think the latter one is better because it supports bs < pagesize > (though I'm not sure about data=ordered yet). I'm not against putting > most of the patch into fs/ext4/, but at least few bits to be changed > in fs/ - exports in fs/m

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Alex Tomas
Andreas Dilger wrote: Sigh, we HAVE a patch that was only adding delalloc to ext4, but it was rejected because "that functionality should go into the VFS". Since the performance improvement of delalloc is quite large, we'd like to get this into the kernel one way or another. Can we make a decisi

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Andreas Dilger
On Jul 28, 2007 20:51 +0100, Christoph Hellwig wrote: > That doesn't mean I want to arge against Alex's code although I'd of > course be more happy if we could actually shared code between multiple > filesystems. > > Of ourse the code in it's current form should not go into mpage.c but > rather i

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread Alex Tomas
David Chinner wrote: On Fri, Jul 27, 2007 at 11:51:56AM +0400, Alex Tomas wrote: But this is really irrelevant - the issue at hand is what we want for VFS level delalloc support. IMO, that mechanism needs to support both XFS and ext4, and I'd prefer if it doesn't perpetuate the bufferhead abuses

Re: [RFC] basic delayed allocation in VFS

2007-07-29 Thread David Chinner
On Fri, Jul 27, 2007 at 11:51:56AM +0400, Alex Tomas wrote: > David Chinner wrote: > >Using a new API for new functionality is a bad thing? > > if existing API can be used ... Sure, but using the existing APIs is no good if the only filesystem in the kernel that supports delalloc cannot use the n

Re: [RFC] basic delayed allocation in VFS

2007-07-28 Thread Christoph Hellwig
On Fri, Jul 27, 2007 at 04:38:44PM +0400, Alex Tomas wrote: > I just realized that you're talking about data=ordered mode in ext4, > where care is taken to prevent on-disk references to no-yet-written > blocks. Any reference to non-written blocks is a bug. - To unsubscribe from this list: send th

Re: [RFC] basic delayed allocation in VFS

2007-07-28 Thread Alex Tomas
Christoph Hellwig wrote: This is not based on my attempt to make the xfs writeout path generic. Alex's variant is a lot simpler and thus missed various bits required for high sustained writeout performance or xfs functionality. I'd very appreciate any details about high writeout performance.

Re: [RFC] basic delayed allocation in VFS

2007-07-28 Thread Christoph Hellwig
On Fri, Jul 27, 2007 at 11:51:56AM +0400, Alex Tomas wrote: > >Secondly, apart from delalloc, XFS cannot use the generic code paths > >for writeback because unwritten extent conversion also requires > >custom I/O completion handlers. Given that __mpage_writepage() only > >calls ->writepage when it

Re: [RFC] basic delayed allocation in VFS

2007-07-28 Thread Christoph Hellwig
On Fri, Jul 27, 2007 at 03:07:14PM +1000, David Chinner wrote: > > It duplicates fs/mpage.c in bio building and introduces new generic API > > (iomap, map_blocks_t, etc). > > Using a new API for new functionality is a bad thing? Depends on wht you do. This patch is just a quickhack to shoe-horn

Re: [RFC] basic delayed allocation in VFS

2007-07-28 Thread Christoph Hellwig
On Thu, Jul 26, 2007 at 06:32:56AM -0400, Jeff Garzik wrote: > Is this based on Christoph's work? > > Christoph, or some other XFS hacker, already did generic delalloc, > modeled on the XFS delalloc code. This is not based on my attempt to make the xfs writeout path generic. Alex's variant is a

Re: [RFC] basic delayed allocation in VFS

2007-07-27 Thread Alex Tomas
Jeff Garzik wrote: Alex Tomas wrote: So without the ability to attach specific I/O completions to bios or support for unwritten extents directly in __mpage_writepage, there is no way XFS can use this "generic" delayed allocation code. I didn't say "generic", see Subject: :) Well, it shouldn'

Re: [RFC] basic delayed allocation in VFS

2007-07-27 Thread Alex Tomas
David Chinner wrote: Firstly, XFS attaches a different I/O completion to delalloc writes to allow us to update the file size when the write is beyond the current on disk EOF. This code cannot do that as all it does is allocation and present "normal looking" buffers to the generic code path. how

Re: [RFC] basic delayed allocation in VFS

2007-07-27 Thread Jeff Garzik
Alex Tomas wrote: So without the ability to attach specific I/O completions to bios or support for unwritten extents directly in __mpage_writepage, there is no way XFS can use this "generic" delayed allocation code. I didn't say "generic", see Subject: :) Well, it shouldn't even be in the VFS

Re: [RFC] basic delayed allocation in VFS

2007-07-27 Thread Alex Tomas
David Chinner wrote: Using a new API for new functionality is a bad thing? if existing API can be used ... No, it doesn't provide the same functionality. Firstly, XFS attaches a different I/O completion to delalloc writes to allow us to update the file size when the write is beyond the curre

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread David Chinner
[please don't top post!] On Thu, Jul 26, 2007 at 05:33:08PM +0400, Alex Tomas wrote: > Jeff Garzik wrote: > >The XFS one is proven and the work was already completed. > > > >What were the specific technical issues that made it unsuitable for ext4? > > > >I would rather not reinvent the wheel, part

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread Alex Tomas
It duplicates fs/mpage.c in bio building and introduces new generic API (iomap, map_blocks_t, etc). In contrast, my trivial implementation re-use existing code in fs/mpage.c, doesn't introduce new API and I tend to think provides quite the same functionality. I can be wrong, of course ... thanks,

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread Jeff Garzik
Alex Tomas wrote: Jeff Garzik wrote: Is this based on Christoph's work? Christoph, or some other XFS hacker, already did generic delalloc, modeled on the XFS delalloc code. nope, this one is simple (something I'd prefer for ext4). The XFS one is proven and the work was already completed.

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread Aneesh Kumar K.V
Alex Tomas wrote: Good day, please review ... thanks, Alex basic delayed allocation in VFS: * block_prepare_write() can be passed special ->get_block() which doesn't allocate blocks, but reserve them and mark bh delayed * a filesystem can use mpage_da_writepages() with other ->get_blo

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread Alex Tomas
Jeff Garzik wrote: Is this based on Christoph's work? Christoph, or some other XFS hacker, already did generic delalloc, modeled on the XFS delalloc code. nope, this one is simple (something I'd prefer for ext4). thanks, Alex - To unsubscribe from this list: send the line "unsubscribe linu

Re: [RFC] basic delayed allocation in VFS

2007-07-26 Thread Jeff Garzik
Alex Tomas wrote: Good day, please review ... thanks, Alex basic delayed allocation in VFS: * block_prepare_write() can be passed special ->get_block() which doesn't allocate blocks, but reserve them and mark bh delayed * a filesystem can use mpage_da_writepages() with other ->get_block

[RFC] basic delayed allocation in VFS

2007-07-26 Thread Alex Tomas
Good day, please review ... thanks, Alex basic delayed allocation in VFS: * block_prepare_write() can be passed special ->get_block() which doesn't allocate blocks, but reserve them and mark bh delayed * a filesystem can use mpage_da_writepages() with other ->get_block() which doesn't