Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Andrew Morton
Badari Pulavarty <[EMAIL PROTECTED]> wrote: > > On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: > > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > > > >Solaris, which does forcedirectio as a mount option, actually > > > > will do buffered I/O on the trailing part. Consider it

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Wed, 2005-03-09 at 14:39, Andrew Morton wrote: > Badari Pulavarty <[EMAIL PROTECTED]> wrote: > > > > On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: > > > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > > > > > > Solaris, which does forcedirectio as a mount option, actually > >

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > > Solaris, which does forcedirectio as a mount option, actually > > > will do buffered I/O on the trailing part. Consider it like a bounce > > > buffer. That way they don't DMA the

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Joel Becker
On Wed, Mar 09, 2005 at 11:53:48AM -0800, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > If writes/truncates take care of zeroing out the rest of the sector > > on disk, might we still be OK without having to do the bounce buffer > > thing ? > > We can probably rely

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > >Solaris, which does forcedirectio as a mount option, actually > > will do buffered I/O on the trailing part. Consider it like a bounce > > buffer. That way they don't DMA the trailing data and succeed the I/O. > > The I/O returns

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Tue, 2005-03-08 at 17:44, Daniel McNeil wrote: > On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: > > > Andrew, please don't apply the original patch. We shouldn't even attempt > > > to submit IO beyond the filesize. We should truncate the IO request to > > > filesize. I will send a patch

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 08:07:57PM -0800, Joel Becker wrote: > On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: > > > 1. return EINVAL if the DIO goes past EOF. > > > > > > 2. truncate the request to file size (which is what your patch does) > > > and if it works, it works. >

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 20:07 -0800, Joel Becker a écrit : > On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: > > > 1. return EINVAL if the DIO goes past EOF. > > > > > > 2. truncate the request to file size (which is what your patch does) > > > and if it works, it works. > >

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 20:07 -0800, Joel Becker a écrit : On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: 1. return EINVAL if the DIO goes past EOF. 2. truncate the request to file size (which is what your patch does) and if it works, it works. 3.

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 08:07:57PM -0800, Joel Becker wrote: On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: 1. return EINVAL if the DIO goes past EOF. 2. truncate the request to file size (which is what your patch does) and if it works, it works. 3.

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Tue, 2005-03-08 at 17:44, Daniel McNeil wrote: On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: Andrew, please don't apply the original patch. We shouldn't even attempt to submit IO beyond the filesize. We should truncate the IO request to filesize. I will send a patch today to

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Andrew Morton
Suparna Bhattacharya [EMAIL PROTECTED] wrote: Solaris, which does forcedirectio as a mount option, actually will do buffered I/O on the trailing part. Consider it like a bounce buffer. That way they don't DMA the trailing data and succeed the I/O. The I/O returns actual bytes

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Joel Becker
On Wed, Mar 09, 2005 at 11:53:48AM -0800, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: If writes/truncates take care of zeroing out the rest of the sector on disk, might we still be OK without having to do the bounce buffer thing ? We can probably rely on the

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: Solaris, which does forcedirectio as a mount option, actually will do buffered I/O on the trailing part. Consider it like a bounce buffer. That way they don't DMA the trailing data and

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Badari Pulavarty
On Wed, 2005-03-09 at 14:39, Andrew Morton wrote: Badari Pulavarty [EMAIL PROTECTED] wrote: On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: Solaris, which does forcedirectio as a mount option, actually will do buffered

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-09 Thread Andrew Morton
Badari Pulavarty [EMAIL PROTECTED] wrote: On Wed, 2005-03-09 at 11:53, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: Solaris, which does forcedirectio as a mount option, actually will do buffered I/O on the trailing part. Consider it like a bounce

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Joel Becker
On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: > > 1. return EINVAL if the DIO goes past EOF. > > > > 2. truncate the request to file size (which is what your patch does) > > and if it works, it works. > > > > 3. truncate the request to a size that actually works - like a

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Daniel McNeil
On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: > > Andrew, please don't apply the original patch. We shouldn't even attempt > > to submit IO beyond the filesize. We should truncate the IO request to > > filesize. I will send a patch today to fix this. > > > > Well, spoke too soon. This is

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Daniel McNeil
On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: > > Andrew, please don't apply the original patch. We shouldn't even attempt > > to submit IO beyond the filesize. We should truncate the IO request to > > filesize. I will send a patch today to fix this. > > > > Well, spoke too soon. This is

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
On Tue, 2005-03-08 at 15:27, Daniel McNeil wrote: > On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: > > > Andrew, please don't apply the original patch. We shouldn't even attempt > > > to submit IO beyond the filesize. We should truncate the IO request to > > > filesize. I will send a patch

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
> Andrew, please don't apply the original patch. We shouldn't even attempt > to submit IO beyond the filesize. We should truncate the IO request to > filesize. I will send a patch today to fix this. > Well, spoke too soon. This is an ugly corner case :( But I have a ugly hack to fix it :) Let

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
On Tue, 2005-03-08 at 01:09, Suparna Bhattacharya wrote: > > Hmm, shouldn't dio->result ideally have been adjusted to be within > i_size at the time of io submission, so we don't have to deal with > this during completion ? We are creating bios with the right size > after all. > > We have

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > But, I'm wondering if this solution just covers up the real reason > for the single thread problem ? We shouldn't even be issuing IOs > beyond i_size into the user-space buffer ! That's true. - To unsubscribe from this list: send the line

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 01:41:41AM -0800, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > > > can you spot what is going wrong here that we have to try and > > > > workaround this later ? > > > > > > Good question. Do we have the i_sem coverage to prevent a

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 01:18 -0800, Andrew Morton a écrit : > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > ... > > > > Bugs in this area seem never-ending don't they - plug one, open up > > another - hard to be confident/verify :( - someday we'll have to > > rewrite a part of this

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 01:18 -0800, Andrew Morton a écrit : > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > ... > > > > Bugs in this area seem never-ending don't they - plug one, open up > > another - hard to be confident/verify :( - someday we'll have to > > rewrite a part of this

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > can you spot what is going wrong here that we have to try and > > > workaround this later ? > > > > Good question. Do we have the i_sem coverage to prevent a concurrent > > truncate? > > > > But from Sebastien's description it

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 01:18:14AM -0800, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > ... > > > > Bugs in this area seem never-ending don't they - plug one, open up > > another - hard to be confident/verify :( - someday we'll have to > > rewrite a part of this

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > ... > > Bugs in this area seem never-ending don't they - plug one, open up > another - hard to be confident/verify :( - someday we'll have to > rewrite a part of this code. It's solving a complex problem. Any rewrite would probably end up just

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Mon, Mar 07, 2005 at 10:39:17PM -0800, Andrew Morton wrote: > 6 > Lines: 76 > > Sébastien Dugué <[EMAIL PROTECTED]> wrote: > > > > When reading a file in async mode (using kernel AIO), and the file > > size is lower than the requested size (short read), the direct IO > > layer reports an

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Mon, Mar 07, 2005 at 10:39:17PM -0800, Andrew Morton wrote: 6 Lines: 76 Sébastien Dugué [EMAIL PROTECTED] wrote: When reading a file in async mode (using kernel AIO), and the file size is lower than the requested size (short read), the direct IO layer reports an incorrect number

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya [EMAIL PROTECTED] wrote: ... Bugs in this area seem never-ending don't they - plug one, open up another - hard to be confident/verify :( - someday we'll have to rewrite a part of this code. It's solving a complex problem. Any rewrite would probably end up just as

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 01:18:14AM -0800, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: ... Bugs in this area seem never-ending don't they - plug one, open up another - hard to be confident/verify :( - someday we'll have to rewrite a part of this code. It's

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya [EMAIL PROTECTED] wrote: can you spot what is going wrong here that we have to try and workaround this later ? Good question. Do we have the i_sem coverage to prevent a concurrent truncate? But from Sebastien's description it doesn't soound as if a

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 01:18 -0800, Andrew Morton a écrit : Suparna Bhattacharya [EMAIL PROTECTED] wrote: ... Bugs in this area seem never-ending don't they - plug one, open up another - hard to be confident/verify :( - someday we'll have to rewrite a part of this code. It's

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Sébastien Dugué
Le mardi 08 mars 2005 à 01:18 -0800, Andrew Morton a écrit : Suparna Bhattacharya [EMAIL PROTECTED] wrote: ... Bugs in this area seem never-ending don't they - plug one, open up another - hard to be confident/verify :( - someday we'll have to rewrite a part of this code. It's

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Suparna Bhattacharya
On Tue, Mar 08, 2005 at 01:41:41AM -0800, Andrew Morton wrote: Suparna Bhattacharya [EMAIL PROTECTED] wrote: can you spot what is going wrong here that we have to try and workaround this later ? Good question. Do we have the i_sem coverage to prevent a concurrent

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Andrew Morton
Suparna Bhattacharya [EMAIL PROTECTED] wrote: But, I'm wondering if this solution just covers up the real reason for the single thread problem ? We shouldn't even be issuing IOs beyond i_size into the user-space buffer ! That's true. - To unsubscribe from this list: send the line

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
On Tue, 2005-03-08 at 01:09, Suparna Bhattacharya wrote: Hmm, shouldn't dio-result ideally have been adjusted to be within i_size at the time of io submission, so we don't have to deal with this during completion ? We are creating bios with the right size after all. We have this:

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
Andrew, please don't apply the original patch. We shouldn't even attempt to submit IO beyond the filesize. We should truncate the IO request to filesize. I will send a patch today to fix this. Well, spoke too soon. This is an ugly corner case :( But I have a ugly hack to fix it :) Let me

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Badari Pulavarty
On Tue, 2005-03-08 at 15:27, Daniel McNeil wrote: On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: Andrew, please don't apply the original patch. We shouldn't even attempt to submit IO beyond the filesize. We should truncate the IO request to filesize. I will send a patch today to

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Daniel McNeil
On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: Andrew, please don't apply the original patch. We shouldn't even attempt to submit IO beyond the filesize. We should truncate the IO request to filesize. I will send a patch today to fix this. Well, spoke too soon. This is an ugly

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Daniel McNeil
On Tue, 2005-03-08 at 11:18, Badari Pulavarty wrote: Andrew, please don't apply the original patch. We shouldn't even attempt to submit IO beyond the filesize. We should truncate the IO request to filesize. I will send a patch today to fix this. Well, spoke too soon. This is an ugly

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-08 Thread Joel Becker
On Tue, Mar 08, 2005 at 03:54:04PM -0800, Badari Pulavarty wrote: 1. return EINVAL if the DIO goes past EOF. 2. truncate the request to file size (which is what your patch does) and if it works, it works. 3. truncate the request to a size that actually works - like a multiple

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-07 Thread Andrew Morton
Sébastien Dugué <[EMAIL PROTECTED]> wrote: > > When reading a file in async mode (using kernel AIO), and the file > size is lower than the requested size (short read), the direct IO > layer reports an incorrect number of bytes read (transferred). > > That case is handled for the sync path in

[PATCH] 2.6.10 - direct-io async short read bug

2005-03-07 Thread Sébastien Dugué
Hi, When reading a file in async mode (using kernel AIO), and the file size is lower than the requested size (short read), the direct IO layer reports an incorrect number of bytes read (transferred). That case is handled for the sync path in 'direct_io_worker' (fs/direct-io.c) where a check

[PATCH] 2.6.10 - direct-io async short read bug

2005-03-07 Thread Sébastien Dugué
Hi, When reading a file in async mode (using kernel AIO), and the file size is lower than the requested size (short read), the direct IO layer reports an incorrect number of bytes read (transferred). That case is handled for the sync path in 'direct_io_worker' (fs/direct-io.c) where a check

Re: [PATCH] 2.6.10 - direct-io async short read bug

2005-03-07 Thread Andrew Morton
Sébastien Dugué [EMAIL PROTECTED] wrote: When reading a file in async mode (using kernel AIO), and the file size is lower than the requested size (short read), the direct IO layer reports an incorrect number of bytes read (transferred). That case is handled for the sync path in