Re: 2.6.21-rc6 new aops patchset

2007-04-17 Thread Nick Piggin
On Tue, Apr 17, 2007 at 10:58:05AM +0100, Christoph Hellwig wrote: On Tue, Apr 17, 2007 at 11:47:11AM +0200, Nick Piggin wrote: I've tried to go through and convert most of the easier ones, and there are only a handful of remainders, many of which seem pretty straightforward and I'll

Re: 2.6.21-rc6 new aops patchset

2007-04-17 Thread Nick Piggin
On Tue, Apr 17, 2007 at 11:41:41AM -0700, Badari Pulavarty wrote: On Sat, 2007-04-14 at 02:52 +0200, Nick Piggin wrote: .. And also, you are missing few of the cleanups I sent + ext4 aop conversion. Do you want me to send it again ? If you could, that would be great. Convert

Re: 2.6.21-rc6 new aops patchset

2007-04-16 Thread Nick Piggin
On Mon, Apr 16, 2007 at 05:19:32PM -0700, Mark Fasheh wrote: On Thu, Apr 12, 2007 at 06:48:52AM +0200, Nick Piggin wrote: http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ 2.6.21-rc6-new-aops* New aops patchset against 2.6.21-rc6. Ok, here's an ocfs2 patch

Re: [AppArmor 37/41] AppArmor: Main Part

2007-04-13 Thread Nick Piggin
Andreas Gruenbacher wrote: On Thursday 12 April 2007 12:37, Alan Cox wrote: + if (PTR_ERR(sa-name) == -ENOENT (check AA_CHECK_FD)) + denied_mask = 0; Now there is an interesting question. Is PTR_ERR() safe for kernel pointers on all platforms or just for

Re: 2.6.21-rc6 new aops patchset

2007-04-13 Thread Nick Piggin
On Fri, Apr 13, 2007 at 09:42:04AM -0700, Badari Pulavarty wrote: On Fri, 2007-04-13 at 01:25 +0200, Nick Piggin wrote: On Thu, Apr 12, 2007 at 09:37:36AM -0700, Badari Pulavarty wrote: On Thu, 2007-04-12 at 06:48 +0200, Nick Piggin wrote: http://www.kernel.org/pub/linux/kernel/people

Re: 2.6.21-rc6 new aops patchset

2007-04-12 Thread Nick Piggin
On Thu, Apr 12, 2007 at 07:05:02PM +0200, Miklos Szeredi wrote: Did some performance testing of the fuse_perform_write implementation. Result with a passthrough filesystem onto a backing tmpfs directory is that bulk (1MB) writes are nearly 4 times faster (256MB/s vs 71MB/s), because FUSE

Re: 2.6.21-rc6 new aops patchset

2007-04-12 Thread Nick Piggin
On Thu, Apr 12, 2007 at 10:27:34AM -0700, Mark Fasheh wrote: On Thu, Apr 12, 2007 at 06:48:52AM +0200, Nick Piggin wrote: Need to think about how to merge this. Maybe a spin in -mm? That'll have to be minus fs-ocfs2-aops.patch, but I'm just working out the last few issues in a new one

2.6.21-rc6 new aops patchset

2007-04-11 Thread Nick Piggin
http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ 2.6.21-rc6-new-aops* New aops patchset against 2.6.21-rc6. Reworked the cont helpers to be better aligned with the old scheme. This unbroke reiserfs (hopefully the only showstopper), and made fat conversion simpler.

2.6.21-rc5-git12 new aops patchset

2007-04-05 Thread Nick Piggin
http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files are 2.6.21-rc5-git12* This contains an update to recent git kernels from Mark, ext3 merge, XFS fix from David Chinner, some FUSE fixes (now lightly tested and working). Documents perform_write a little better, and

Re: [ANNOUNCE] new new aops patchset

2007-04-05 Thread Nick Piggin
On Thu, Apr 05, 2007 at 04:18:09PM +1000, David Chinner wrote: [lots of interesting stuff] Ah, found it. page_cache_write_begin() returns zero on success. So this: status = pagecache_write_begin(NULL, mapping, pos, bytes,

Re: [ANNOUNCE] new new aops patchset

2007-04-05 Thread Nick Piggin
On Thu, Apr 05, 2007 at 08:45:29AM +0100, Christoph Hellwig wrote: On Thu, Apr 05, 2007 at 04:13:05AM +0200, Nick Piggin wrote: On a related note - what's the rules for a perform_write() implementation? I noticed that wasn't documented with write_begin and write_end and I don't see any

Re: [ANNOUNCE] new new aops patchset

2007-04-05 Thread Nick Piggin
On Thu, Apr 05, 2007 at 08:21:54AM -0700, Badari Pulavarty wrote: On Thu, 2007-04-05 at 04:08 +0200, Nick Piggin wrote: -readpage can still return AOP_TRUNCATED_PAGE. Were there any from prepare_write or commit_write still around? Not a big deal. But trying to understand it better

Re: 2.6.21-rc5-git12 new aops patchset

2007-04-05 Thread Nick Piggin
On Thu, Apr 05, 2007 at 09:22:35AM -0700, Badari Pulavarty wrote: On Thu, 2007-04-05 at 11:46 +0200, Nick Piggin wrote: http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files are 2.6.21-rc5-git12* ext3_ordered_write_end() did + ret

Re: [ANNOUNCE] new new aops patchset

2007-04-04 Thread Nick Piggin
On Wed, Apr 04, 2007 at 04:32:24PM -0700, Badari Pulavarty wrote: On Wed, 2007-04-04 at 16:17 -0700, Mark Fasheh wrote: On Wed, Apr 04, 2007 at 04:05:19PM -0700, Badari Pulavarty wrote: Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of

Re: [ANNOUNCE] new new aops patchset

2007-04-04 Thread Nick Piggin
On Wed, Apr 04, 2007 at 03:10:34PM -0700, Mark Fasheh wrote: On Mon, Apr 02, 2007 at 02:09:34PM +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Thanks again for pushing this stuff out Nick

Re: [ANNOUNCE] new new aops patchset

2007-04-04 Thread Nick Piggin
On Thu, Apr 05, 2007 at 12:43:50PM +1000, David Chinner wrote: On Thu, Apr 05, 2007 at 10:10:18AM +1000, David Chinner wrote: On Mon, Apr 02, 2007 at 02:09:34PM +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin

Re: [ANNOUNCE] new new aops patchset

2007-04-03 Thread Nick Piggin
On Tue, Apr 03, 2007 at 02:08:53AM +0200, Nick Piggin wrote: BTW, I will take a shot at ext4 tomorrow. Thanks, so long as you think ext3 is looking OK? BTW. is it a known issue that ext3 fails fsx-linux? (I tried 2.6.21-rc3 IIRC, and ordered and writeback both eventually failed I think

Re: [ANNOUNCE] new new aops patchset

2007-04-03 Thread Nick Piggin
On Tue, Apr 03, 2007 at 08:57:03AM -0700, Badari Pulavarty wrote: On Tue, 2007-04-03 at 01:49 +0200, Nick Piggin wrote: Ahh, just the person I wanted to ask! ;) How useful is it, out of curiosity? What sort of users use it, and what sort of improvements do they get? Well, at the time

Re: [ANNOUNCE] new new aops patchset

2007-04-03 Thread Nick Piggin
On Tue, Apr 03, 2007 at 08:59:57AM -0700, Badari Pulavarty wrote: On Tue, 2007-04-03 at 01:58 +0200, Nick Piggin wrote: On Mon, Apr 02, 2007 at 01:44:59PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5

Re: [ANNOUNCE] new new aops patchset

2007-04-03 Thread Nick Piggin
On Tue, Apr 03, 2007 at 10:35:19AM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops* Here

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 04:31:22PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Sorry to send you so many silly fixes, but I

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 04:14:59PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops* Baaah !! You

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 01:44:59PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops* Contains

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 04:56:11PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 05:00:50PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops* One more

Re: [ANNOUNCE] new new aops patchset

2007-04-02 Thread Nick Piggin
On Mon, Apr 02, 2007 at 01:18:13PM -0700, Badari Pulavarty wrote: On Mon, 2007-04-02 at 14:09 +0200, Nick Piggin wrote: Updated aops patchset against 2.6.21-rc5. http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/ Files/dirs are 2.6.21-rc5-new-aops* Contains

Re: Announce: new-aops-1 for 2.6.21-rc3

2007-03-31 Thread Nick Piggin
Hi, Progress is coming along: as well as the several fixes Mark has posted, we've got a GFS2 patch from Steve as well, which also enables us to get rid of the AOPS_TRUNCATED_PAGE handling in the legacy -prepare_write paths. And I've been auditing code and doing stress and code coverage tests for

Re: [PATCH 1 of 2] block_page_mkwrite() Implementation V2

2007-03-19 Thread Nick Piggin
David Chinner wrote: On Mon, Mar 19, 2007 at 05:37:03PM +1100, Nick Piggin wrote: David Chinner wrote: +block_page_mkwrite(struct vm_area_struct *vma, struct page *page, + get_block_t get_block) +{ + struct inode *inode = vma-vm_file-f_path.dentry-d_inode

Re: [PATCH 1 of 2] block_page_mkwrite() Implementation V2

2007-03-19 Thread Nick Piggin
Nick Piggin wrote: David Chinner wrote: On Mon, Mar 19, 2007 at 05:37:03PM +1100, Nick Piggin wrote: David Chinner wrote: +block_page_mkwrite(struct vm_area_struct *vma, struct page *page, + get_block_t get_block) +{ +struct inode *inode = vma-vm_file-f_path.dentry-d_inode

Re: [PATCH 1 of 2] block_page_mkwrite() Implementation V2

2007-03-19 Thread Nick Piggin
Christoph Hellwig wrote: On Mon, Mar 19, 2007 at 09:11:31PM +1100, Nick Piggin wrote: I've got the patches in -mm now. I hope they will get merged when the the next window opens. I didn't submit the -page_mkwrite conversion yet, because I didn't have any callers to look at. It is is slightly

Re: Announce: new-aops-1 for 2.6.21-rc3

2007-03-19 Thread Nick Piggin
On Thu, Mar 15, 2007 at 04:47:13PM -0700, Mark Fasheh wrote: On Thu, Mar 15, 2007 at 05:17:04PM +0100, Nick Piggin wrote: (excludes the OCFS2 patch that Mark sent, in anticipation of an update) Attached is said patch. I needed to export __grab_cache_page (ext2/ext3 also need this if they're

Re: [PATCH 1 of 2] block_page_mkwrite() Implementation V2

2007-03-18 Thread Nick Piggin
David Chinner wrote: Generic page_mkwrite functionality. Filesystems that make use of the VM -page_mkwrite() callout will generally use the same core code to implement it. There are several tricky truncate-related issues that we need to deal with here as we cannot take the i_mutex as we

Re: [patch 2/5] fs: introduce new aops and infrastructure

2007-03-15 Thread Nick Piggin
On Wed, Mar 14, 2007 at 11:23:05PM -0700, Joel Becker wrote: On Thu, Mar 15, 2007 at 05:36:42AM +0100, Nick Piggin wrote: On Wed, Mar 14, 2007 at 09:13:29PM -0700, Mark Fasheh wrote: Are we going to get rid of the file and intr arguments btw? I'm not sure intr is useful, and mapping

Re: [patch 2/5] fs: introduce new aops and infrastructure

2007-03-15 Thread Nick Piggin
Dmitriy Monakhov wrote: Nick Piggin [EMAIL PROTECTED] writes: Index: linux-2.6/fs/splice.c === --- linux-2.6.orig/fs/splice.c +++ linux-2.6/fs/splice.c @@ -559,7 +559,7 @@ static int pipe_to_file(struct pipe_inod struct

Announce: new-aops-1 for 2.6.21-rc3

2007-03-15 Thread Nick Piggin
OK, I've gone through and fixed several bugs until the thing actually survives fsx-linux for both ext2 and ext3 ordered and writeback (both when using the new aops, and the legacy prepare_write path). Actually ext3 sometimes breaks, but it does in unpatched kernels anyway. At 15 patches

Re: Announce: new-aops-1 for 2.6.21-rc3

2007-03-15 Thread Nick Piggin
On Thu, Mar 15, 2007 at 12:32:45PM -0700, Joel Becker wrote: On Thu, Mar 15, 2007 at 05:17:04PM +0100, Nick Piggin wrote: At 15 patches (including the initial buffered write deadlock fixes), it is too much to keep posting -- not much has fundamentally changed, so I'll just post occasionally

Re: Announce: new-aops-1 for 2.6.21-rc3

2007-03-15 Thread Nick Piggin
On Thu, Mar 15, 2007 at 12:53:51PM -0700, Mark Fasheh wrote: On Thu, Mar 15, 2007 at 05:17:04PM +0100, Nick Piggin wrote: OK, I've gone through and fixed several bugs until the thing actually survives fsx-linux for both ext2 and ext3 ordered and writeback (both when using the new aops

Re: [patch 2/3] fs: introduce perform_write aop

2007-03-14 Thread Nick Piggin
On Sat, Mar 10, 2007 at 09:25:41AM +, Christoph Hellwig wrote: On Fri, Mar 09, 2007 at 03:33:01PM -0800, Mark Fasheh wrote: -kernel_write() as opposed to genericizing -perform_write() would be fine with me. Just so long as we get rid of -prepare_write and -commit_write in that other

[patch 1/5] fs: add an iovec iterator

2007-03-14 Thread Nick Piggin
Add an iterator data structure to operate over an iovec. Add usercopy operators needed by generic_file_buffered_write, and convert that function over. include/linux/fs.h | 33 mm/filemap.c | 144 +++-- mm/filemap.h |

[patch 3/5] fs: convert some simple filesystems

2007-03-14 Thread Nick Piggin
Implement new aops for some of the simpler filesystems. fs/configfs/inode.c |4 ++-- fs/sysfs/inode.c|4 ++-- mm/shmem.c | 16 ++-- 3 files changed, 14 insertions(+), 10 deletions(-) Index: linux-2.6/mm/shmem.c

[patch 5/5] ext3: convert to new aops

2007-03-14 Thread Nick Piggin
Implement new aops for ext3. Probably has some bugs in interaction with journalling, and corner cases aren't tested/thought out fully, but it boots and runs. I don't see a fundamental reason why it can't work... fs/ext3/inode.c | 137 +++- 1

[patch 4/5] ext2: convert to new aops

2007-03-14 Thread Nick Piggin
Implement new aops for ext2. fs/ext2/inode.c | 12 1 file changed, 12 insertions(+) Index: linux-2.6/fs/ext2/inode.c === --- linux-2.6.orig/fs/ext2/inode.c +++ linux-2.6/fs/ext2/inode.c @@ -643,6 +643,16 @@

Re: [patch 1/5] fs: add an iovec iterator

2007-03-14 Thread Nick Piggin
On Wed, Mar 14, 2007 at 02:38:12PM +0100, Nick Piggin wrote: Add an iterator data structure to operate over an iovec. Add usercopy operators needed by generic_file_buffered_write, and convert that function over. Just a note to anyone looking at these -- they don't apply to any tree, and I'm

Re: [patch 2/5] fs: introduce new aops and infrastructure

2007-03-14 Thread Nick Piggin
On Thu, Mar 15, 2007 at 12:28:04AM +0300, Dmitriy Monakhov wrote: Nick Piggin [EMAIL PROTECTED] writes: + +int pagecache_write_end(struct file *file, struct address_space *mapping, + loff_t pos, unsigned len, unsigned copied, + struct

Re: [patch 2/5] fs: introduce new aops and infrastructure

2007-03-14 Thread Nick Piggin
On Wed, Mar 14, 2007 at 10:46:25PM +0100, Mariusz Kozlowski wrote: Hello, I guess no need to define 'ret' twice here. [...] Hi Mariusz, Thanks, I'll clean that up. Nick - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL

Re: [patch 2/5] fs: introduce new aops and infrastructure

2007-03-14 Thread Nick Piggin
On Wed, Mar 14, 2007 at 09:13:29PM -0700, Mark Fasheh wrote: Hi Nick, On Wed, Mar 14, 2007 at 02:38:22PM +0100, Nick Piggin wrote: Introduce write_begin, write_end, and perform_write aops. These are intended to replace prepare_write and commit_write with more flexible alternatives

Re: [patch 2/3] fs: introduce perform_write aop

2007-03-09 Thread Nick Piggin
for taking a look. On Thu, Feb 08, 2007 at 02:07:36PM +0100, Nick Piggin wrote: as a single call to copy a given amount of userdata at the given offset. This is more flexible, because the implementation can determine how to best handle errors, or multi-page ranges (eg. it may use a gang

[rfc][patch 7/6] mm: merge page_mkwrite

2007-03-07 Thread Nick Piggin
efficiency for everyone). Then, we introduce another fault flag to signal that the fault is an event notification for a page, rather than a request for a pgoff. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/include/linux/mm.h

[patch] fs: introduce some page/buffer invariants

2007-02-20 Thread Nick Piggin
, and enforce this ordering. Bring the order of operations for truncate into line with those of invalidate. This will prevent a page from being able to go !uptodate while we're holding the tree_lock, which is probably a good thing anyway. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/fs

Re: [rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-09 Thread Nick Piggin
On Fri, Feb 09, 2007 at 12:41:01AM -0800, Andrew Morton wrote: On Thu, 8 Feb 2007 14:07:15 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: So I have finally finished a first slightly-working draft of my new aops op (perform_write) proposal. I would be interested to hear comments about

Re: [rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-09 Thread Nick Piggin
On Fri, Feb 09, 2007 at 02:09:54AM -0800, Andrew Morton wrote: On Fri, 9 Feb 2007 10:54:05 +0100 Nick Piggin [EMAIL PROTECTED] wrote: That's still got a deadlock, It does? Yes, PG_lock vs mm-mmap_sem. and also it doesn't work if we want to lock the page when performing a minor

Re: [rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-09 Thread Nick Piggin
On Fri, Feb 09, 2007 at 02:52:49AM -0800, Andrew Morton wrote: On Fri, 9 Feb 2007 11:32:58 +0100 Nick Piggin [EMAIL PROTECTED] wrote: On Fri, Feb 09, 2007 at 02:09:54AM -0800, Andrew Morton wrote: On Fri, 9 Feb 2007 10:54:05 +0100 Nick Piggin [EMAIL PROTECTED] wrote: That's

Re: [rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-09 Thread Nick Piggin
On Fri, Feb 09, 2007 at 03:46:44AM -0800, Andrew Morton wrote: On Fri, 9 Feb 2007 12:31:16 +0100 Nick Piggin [EMAIL PROTECTED] wrote: We'll never, ever, ever update and test all filesytems. What you're calling legacy code will be there for all time. I didn't say all; I still

Re: [patch 3/3] ext2: use perform_write aop

2007-02-09 Thread Nick Piggin
On Fri, Feb 09, 2007 at 11:45:39AM -0800, Andrew Morton wrote: On Fri, 9 Feb 2007 11:14:55 -0800 Andrew Morton [EMAIL PROTECTED] wrote: If so, that might be preventable by leaving the buffer nonuptodate. oh, OK, it was buffer_new(), so zeroes are the right thing for a reader to see.

[rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-08 Thread Nick Piggin
In my last set of numbers for my buffered-write deadlock fix using 2 copies per page, I realised there is no real performance hit for !uptodate pages as opposed to uptodate ones. This is unexpected because the uptodate pages only require a single copy... The problem turns out to be operator

[patch 1/3] fs: add an iovec iterator

2007-02-08 Thread Nick Piggin
Add an iterator data structure to operate over an iovec. Add usercopy operators needed by generic_file_buffered_write, and convert that function over. include/linux/fs.h | 32 mm/filemap.c | 132 ++--- mm/filemap.h |

[patch 2/3] fs: introduce perform_write aop

2007-02-08 Thread Nick Piggin
Add a new perform_write aop, which replaces prepare_write and commit_write as a single call to copy a given amount of userdata at the given offset. This is more flexible, because the implementation can determine how to best handle errors, or multi-page ranges (eg. it may use a gang lookup), and

[patch 3/3] ext2: use perform_write aop

2007-02-08 Thread Nick Piggin
Convert ext2 to use -perform_write. This uses the main loop out of generic_perform_write, but when encountering a short usercopy, it zeroes out new uninitialised blocks, and passes in a short-length commit to __block_commit_write, which does the right thing (in terms of not setting things

Re: [patch 1/3] fs: add an iovec iterator

2007-02-08 Thread Nick Piggin
On Thu, Feb 08, 2007 at 07:49:53PM +, Christoph Hellwig wrote: On Thu, Feb 08, 2007 at 02:07:24PM +0100, Nick Piggin wrote: Add an iterator data structure to operate over an iovec. Add usercopy operators needed by generic_file_buffered_write, and convert that function over

Re: [rfc][patch 0/3] a faster buffered write deadlock fix?

2007-02-08 Thread Nick Piggin
On Thu, Feb 08, 2007 at 04:38:01PM -0800, Mark Fasheh wrote: On Thu, Feb 08, 2007 at 02:07:15PM +0100, Nick Piggin wrote: The problem is that the existing aops interface is crap. correct, fast, compatible -- choose any 2 Agreed. There's lots of problems with the interface (imho), but my

Re: [patch 1/3] fs: add an iovec iterator

2007-02-08 Thread Nick Piggin
On Thu, Feb 08, 2007 at 06:03:50PM -0800, Nate Diller wrote: On 2/8/07, Nick Piggin [EMAIL PROTECTED] wrote: On Thu, Feb 08, 2007 at 07:49:53PM +, Christoph Hellwig wrote: On Thu, Feb 08, 2007 at 02:07:24PM +0100, Nick Piggin wrote: Add an iterator data structure to operate over

Re: [PATCH 1 of 2] Implement generic block_page_mkwrite() functionality

2007-02-07 Thread Nick Piggin
Chris Mason wrote: On Thu, Feb 08, 2007 at 01:44:15AM +1100, David Chinner wrote: So, do I need to grab the i_mutex here? Is that safe to do that in the middle of a page fault? If we do race with a truncate and the page is now beyond EOF, what am I supposed to return? Should it check to

[patch 0/3] 2.6.20 fix for PageUptodate memorder problem

2007-02-06 Thread Nick Piggin
Still no independent confirmation as to whether this is a problem or not. I think it is, so I'll propose this patchset to fix it. Patch 1/3 has a reasonable description of the problem. Thanks, Nick -- SuSE Labs - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body

[patch 1/3] mm: fix PageUptodate memorder

2007-02-06 Thread Nick Piggin
follows). Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/include/linux/highmem.h === --- linux-2.6.orig/include/linux/highmem.h +++ linux-2.6/include/linux/highmem.h @@ -57,8 +57,6 @@ static inline void clear_user_highpage(s

[patch 2/3] fs: buffer don't PageUptodate without page locked

2007-02-06 Thread Nick Piggin
such a thing anyway). Instead just leave it to the read side to bring the page uptodate when it notices that all buffers are uptodate. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/fs/buffer.c === --- linux-2.6.orig/fs

[patch 3/3] mm: make read_cache_page synchronous

2007-02-06 Thread Nick Piggin
in ecryptfs, 1 in jffs2, and a possible cleared data overwritten with readpage in block2mtd. All depending on whether the filler is async and/or can return with a !uptodate page. Also, a memory leak in sys_swapon(). Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/fs/afs/dir.c

Re: [patch 2/3] fs: buffer don't PageUptodate without page locked

2007-02-06 Thread Nick Piggin
On Tue, Feb 06, 2007 at 12:21:40AM -0800, Andrew Morton wrote: On Tue, 6 Feb 2007 09:02:23 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: __block_write_full_page is calling SetPageUptodate without the page locked. This is unusual, but not incorrect, as PG_writeback is still set

Re: [patch 3/3] mm: make read_cache_page synchronous

2007-02-06 Thread Nick Piggin
Andrew Morton wrote: On Tue, 6 Feb 2007 09:02:33 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: Ensure pages are uptodate after returning from read_cache_page, which allows us to cut out most of the filesystem-internal PageUptodate_NoLock calls. Normally it's good to rename functions

Re: [patch 1/3] mm: fix PageUptodate memorder

2007-02-06 Thread Nick Piggin
Andrew Morton wrote: On Tue, 6 Feb 2007 09:02:11 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: +static inline void __SetPageUptodate(struct page *page) +{ +#ifdef CONFIG_S390 if (!test_and_set_bit(PG_uptodate, page-flags)) page_test_and_clear_dirty(page); -} #else

Re: [patch 0/3] 2.6.20 fix for PageUptodate memorder problem

2007-02-06 Thread Nick Piggin
On Wed, Feb 07, 2007 at 09:58:57AM +1100, David Chinner wrote: On Tue, Feb 06, 2007 at 09:02:01AM +0100, Nick Piggin wrote: Still no independent confirmation as to whether this is a problem or not. I think it is, so I'll propose this patchset to fix it. Patch 1/3 has a reasonable

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-05 Thread Nick Piggin
On Sun, Feb 04, 2007 at 05:40:35PM +, Anton Altaparmakov wrote: On Sun, 4 Feb 2007, Andrew Morton wrote: truncate's OK: we're holding i_mutex. How about excluding readpage() (in addition to truncate if Nick is right and some cases of truncate do not hold i_mutex) with an extra page

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-05 Thread Nick Piggin
On Tue, Feb 06, 2007 at 03:25:49AM +0100, Nick Piggin wrote: On Sun, Feb 04, 2007 at 10:36:20AM -0800, Andrew Morton wrote: On Sun, 4 Feb 2007 16:10:51 +0100 Nick Piggin [EMAIL PROTECTED] wrote: They're not likely to hit the deadlocks, either. Probability gets more likely after my

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-05 Thread Nick Piggin
On Mon, Feb 05, 2007 at 09:30:06PM -0800, Andrew Morton wrote: On Tue, 6 Feb 2007 05:41:46 +0100 Nick Piggin [EMAIL PROTECTED] wrote: Not necessarily -- they could read from one part of a page and write to another. I see this as the biggest data corruption problem. The kernel gets

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-05 Thread Nick Piggin
On Tue, Feb 06, 2007 at 06:49:05AM +0100, Nick Piggin wrote: - If the get_user() doesn't fault, and if we're copying from and to the same page, we know that we've locked it, so nobody will be able to unmap it while we're copying from it. Close, but no cigar! This is still

[patch 2/9] mm: revert generic_file_buffered_write(): handle zero length iovec segments

2007-02-04 Thread Nick Piggin
From: Andrew Morton [EMAIL PROTECTED] Revert 81b0c8713385ce1b1b9058e916edcf9561ad76d6. This was a bugfix against 6527c2bdf1f833cc18e8f42bd97973d583e4aa83, which we also revert. Signed-off-by: Andrew Morton [EMAIL PROTECTED] Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm

[patch 5/9] mm: debug write deadlocks

2007-02-04 Thread Nick Piggin
-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.c === --- linux-2.6.orig/mm/filemap.c +++ linux-2.6/mm/filemap.c @@ -2103,6 +2103,7 @@ generic_file_buffered_write(struct kiocb if (maxlen bytes

[patch 6/9] mm: be sure to trim blocks

2007-02-04 Thread Nick Piggin
If prepare_write fails with AOP_TRUNCATED_PAGE, or if commit_write fails, then we may have failed the write operation despite prepare_write having instantiated blocks past i_size. Fix this, and consolidate the trimming into one place. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-04 Thread Nick Piggin
On Sun, Feb 04, 2007 at 01:44:45AM -0800, Andrew Morton wrote: On Sun, 4 Feb 2007 09:51:07 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: 2. If we find the destination page is non uptodate, unlock it (this could be made slightly more optimal), then find and pin the source page

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-04 Thread Nick Piggin
On Sun, Feb 04, 2007 at 11:46:09AM +0100, Nick Piggin wrote: It's better than taking mmap_sem and walking pagetables... I'm not convinced. Though I am more convinced that looking at mm *at all* (either to take the mmap_sem and find the vma, or to take the mmap_sem and run get_user_pages

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-04 Thread Nick Piggin
On Sun, Feb 04, 2007 at 03:15:49AM -0800, Andrew Morton wrote: On Sun, 4 Feb 2007 12:03:17 +0100 Nick Piggin [EMAIL PROTECTED] wrote: On Sun, Feb 04, 2007 at 02:56:02AM -0800, Andrew Morton wrote: On Sun, 4 Feb 2007 11:46:09 +0100 Nick Piggin [EMAIL PROTECTED] wrote

Re: [patch 1/9] fs: libfs buffered write leak fix

2007-02-03 Thread Nick Piggin
On Sat, Feb 03, 2007 at 05:49:47PM +, Jörn Engel wrote: On Sat, 3 February 2007 02:33:16 +0100, Nick Piggin wrote: If doing a partial-write, simply clear the whole page and set it uptodate (don't need to get too tricky). That sounds just like a bug I recently fixed in logfs

Re: [patch 0/9] buffered write deadlock fix

2007-02-02 Thread Nick Piggin
On Fri, Feb 02, 2007 at 03:52:32PM -0800, Andrew Morton wrote: On Mon, 29 Jan 2007 11:31:37 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: The following set of patches attempt to fix the buffered write locking problems (and there are a couple of peripheral patches and cleanups

Re: [patch 1/9] fs: libfs buffered write leak fix

2007-02-02 Thread Nick Piggin
On Fri, Feb 02, 2007 at 03:52:36PM -0800, Andrew Morton wrote: On Mon, 29 Jan 2007 11:31:46 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: simple_prepare_write and nobh_prepare_write leak uninitialised kernel data. They do? Under what situation? Yes, I have at least reproduced

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-02-02 Thread Nick Piggin
On Fri, Feb 02, 2007 at 03:53:11PM -0800, Andrew Morton wrote: On Mon, 29 Jan 2007 11:33:03 +0100 (CET) Nick Piggin [EMAIL PROTECTED] wrote: Modify the core write() code so that it won't take a pagefault while holding a lock on the pagecache page. There are a number of different

Re: [patch 1/9] fs: libfs buffered write leak fix

2007-02-02 Thread Nick Piggin
On Fri, Feb 02, 2007 at 05:58:01PM -0800, Andrew Morton wrote: On Sat, 3 Feb 2007 02:33:16 +0100 Nick Piggin [EMAIL PROTECTED] wrote: I think just setting page uptodate in commit_write might do the trick? (and getting rid of the set_page_dirty there). Yes, the page just isn't uptodate

Re: [patch 1/9] fs: libfs buffered write leak fix

2007-02-02 Thread Nick Piggin
On Fri, Feb 02, 2007 at 06:19:55PM -0800, Andrew Morton wrote: On Sat, 3 Feb 2007 03:09:26 +0100 Nick Piggin [EMAIL PROTECTED] wrote: From: Nick Piggin [EMAIL PROTECTED] To: Andrew Morton [EMAIL PROTECTED] argh. Yesterday all my emails were getting a mysterious s/osdl/linux-foundation

Re: [patch 0/9] buffered write deadlock fix

2007-01-30 Thread Nick Piggin
On Tue, Jan 30, 2007 at 03:21:19PM -0800, Andrew Morton wrote: On Tue, 30 Jan 2007 12:55:58 -0800 Andrew Morton [EMAIL PROTECTED] wrote: y'know, four or five years back I fixed this bug by doing current-locked_page = page; in the write() code, and then teaching the pagefault

[patch 0/9] buffered write deadlock fix

2007-01-29 Thread Nick Piggin
The following set of patches attempt to fix the buffered write locking problems (and there are a couple of peripheral patches and cleanups there too). Patches against 2.6.20-rc6. I was hoping that 2.6.20-rc6-mm2 would be an easier diff with the fsaio patches gone, but the readahead rewrite

[patch 1/9] fs: libfs buffered write leak fix

2007-01-29 Thread Nick Piggin
simple_prepare_write and nobh_prepare_write leak uninitialised kernel data. Fix the former, make a note of the latter. Several other filesystems seem to be iffy here, too. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/fs/libfs.c

[patch 2/9] mm: revert generic_file_buffered_write(): handle zero length iovec segments

2007-01-29 Thread Nick Piggin
From: Andrew Morton [EMAIL PROTECTED] Revert 81b0c8713385ce1b1b9058e916edcf9561ad76d6. This was a bugfix against 6527c2bdf1f833cc18e8f42bd97973d583e4aa83, which we also revert. Signed-off-by: Andrew Morton [EMAIL PROTECTED] Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm

[patch 4/9] mm: generic_file_buffered_write cleanup

2007-01-29 Thread Nick Piggin
From: Andrew Morton [EMAIL PROTECTED] Clean up buffered write code. Rename some variables and fix some types. Signed-off-by: Andrew Morton [EMAIL PROTECTED] Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.c

[patch 3/9] mm: revert generic_file_buffered_write(): deadlock on vectored write

2007-01-29 Thread Nick Piggin
just back that patch out - we'll be fixing the deadlock by other means. Signed-off-by: Andrew Morton [EMAIL PROTECTED] Nick says: also it only ever actually papered over the bug, because after faulting in the pages, they might be unmapped or reclaimed. Signed-off-by: Nick Piggin [EMAIL PROTECTED

[patch 6/9] mm: be sure to trim blocks

2007-01-29 Thread Nick Piggin
If prepare_write fails with AOP_TRUNCATED_PAGE, or if commit_write fails, then we may have failed the write operation despite prepare_write having instantiated blocks past i_size. Fix this, and consolidate the trimming into one place. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6

[patch 5/9] mm: debug write deadlocks

2007-01-29 Thread Nick Piggin
-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.c === --- linux-2.6.orig/mm/filemap.c +++ linux-2.6/mm/filemap.c @@ -1894,6 +1894,7 @@ generic_file_buffered_write(struct kiocb if (maxlen bytes

[patch 9/9] mm: fix pagecache write deadlocks

2007-01-29 Thread Nick Piggin
via the kernel address space. (also, rename maxlen to seglen, because it was confusing) Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.c === --- linux-2.6.orig/mm/filemap.c +++ linux-2.6/mm/filemap.c

[patch 7/9] mm: cleanup pagecache insertion operations

2007-01-29 Thread Nick Piggin
, in contrast with the per-CPU pagevecs that are persistent. Net result: 7.3 times fewer lru_lock acquisitions required to add the pages to pagecache for a bulk write (in 4K chunks). Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.c

[patch 8/9] mm: generic_file_buffered_write iovec cleanup

2007-01-29 Thread Nick Piggin
Hide some of the open-coded nr_segs tests into the iovec helpers. This is all to simplify generic_file_buffered_write, because that gets more complex in the next patch. Signed-off-by: Nick Piggin [EMAIL PROTECTED] Index: linux-2.6/mm/filemap.h

Re: [patch 9/9] mm: fix pagecache write deadlocks

2007-01-29 Thread Nick Piggin
On Mon, Jan 29, 2007 at 11:33:03AM +0100, Nick Piggin wrote: + } else { + char *src, *dst; + src = kmap(src_page); + dst = kmap(page); + memcpy(dst + offset, + src

Re: [patch 6/10] mm: be sure to trim blocks

2007-01-19 Thread Nick Piggin
On Sun, Jan 14, 2007 at 05:25:44PM +0300, Dmitriy Monakhov wrote: Nick Piggin [EMAIL PROTECTED] writes: If prepare_write fails with AOP_TRUNCATED_PAGE, or if commit_write fails, then we may have failed the write operation despite prepare_write having instantiated blocks past i_size

Re: [patch 6/10] mm: be sure to trim blocks

2007-01-19 Thread Nick Piggin
On Tue, Jan 16, 2007 at 08:14:16PM +0100, Peter Zijlstra wrote: On Tue, 2007-01-16 at 18:36 +0100, Peter Zijlstra wrote: buf, bytes); @@ -1935,10 +1922,9 @@ generic_file_buffered_write(struct kiocb

<    1   2   3   4   >