Re: nfsv3d wrong truncates over 4G
> " " == Matti Aarnio <[EMAIL PROTECTED]> writes: > The NFSv3 CLIENT code must do post-open verification, that if > the opened file has size exceeding 2G-1, and O_LARGEFILE flag > is not set, the opened file must be closed, and caller must be Just out of curiosity: why can this particular check not be done by the VFS? Surely this applies to all filesystems... Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
> " " == Andrea Arcangeli <[EMAIL PROTECTED]> writes: > On Wed, Oct 25, 2000 at 09:34:49PM +0200, Trond Myklebust > wrote: >> All NFSv3 operations are 64-bit and LFS-compliant. There's >> therefore no need for an O_LARGEFILE flag. > I don't understand what you mean, sorry. > Userspace programs must open with O_LARGEFILE to go over > 2G. This is true regardless of the underlying FS. You can find > the details in the specs. Sure, but I understood Matti's question as being about how NFS passes O_LARGEFILE over the wire. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Thu, Oct 26, 2000 at 03:56:46AM +0200, Andi Kleen wrote: > a 4GB-1 file. Side note: 4GB-2 since -1 means "ignore". Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Thu, Oct 26, 2000 at 12:06:17AM +0200, Andrea Arcangeli wrote: > 2.2.18pre17aa1 allows anybody to open a file larger than 2G (that seems not to > be completly in sync with the specs) but it has all the necessary runtime > checks during read/write. Just search for O_LARGEFILE in the lfs patch. That > needs to be forward ported to 2.4.x. 2.4 nfsd should already use O_LARGEFILE, I submitted a patch for it some time ago and Linus included it. It doesn't do the >2GB checking for v2, so a v2 client could in theory write a 4GB-1 file. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 10:49:52PM +0200, Trond Myklebust wrote: > This should already be done IIRC. At least nfsd_open() always sets > O_LARGEFILE. Yes, that is just working right. > we should be returning EOVERFLOW rather than EBIG. Right. 2.4.0-test10-pre4 is also buggy because it doesn't check for O_LARGEFILE while reading and writing (it should return EOVERFLOW from read and EBIG during writes). 2.2.18pre17aa1 allows anybody to open a file larger than 2G (that seems not to be completly in sync with the specs) but it has all the necessary runtime checks during read/write. Just search for O_LARGEFILE in the lfs patch. That needs to be forward ported to 2.4.x. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 11:25:14PM +0300, Matti Aarnio wrote: > The NFSv3 SERVER (for which Andrea made his patch) > actually MUST open local files with O_LARGEFILE > flag set. (Otherwise e.g. EXT2 may reject the open.) Exactly, that is the code-sharing implementation detail I was mentioning. That must be true not only for ext2 of course but for every other fs that nfsd is sitting upon. That was just fine in 2.4.0-test10-pre4. Andi spotted and fixed that some week ago. > The NFSv3 CLIENT code must do post-open verification, > that if the opened file has size exceeding 2G-1, and > O_LARGEFILE flag is not set, the opened file must be > closed, and caller must be informed of the rejection > with -EFBIG status. > (If possible, do that size verification before open.) 2.2.18pre17aa1 lfs is completly complaint in this sense. > Am I making any sense ? You're completly right. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 09:34:49PM +0200, Trond Myklebust wrote: > All NFSv3 operations are 64-bit and LFS-compliant. There's therefore > no need for an O_LARGEFILE flag. I don't understand what you mean, sorry. Userspace programs must open with O_LARGEFILE to go over 2G. This is true regardless of the underlying FS. You can find the details in the specs. On the server side nfsd must open files with O_LARGEFILE as well if it wants to return data over 2G to the nfsv2/3 clients that are asking for that (see it as a kernel internals implementation detail: we for sure want to re-use the common routines inside nfsd even if it's in kenrel space). The |O_LARGEFILE at line 472 and 476 of linux/fs/nfsd/vfs.c of 2.4.0-test10-pre4 are strictly necessary (they're present in 2.2.18pre17aa1 as well indeed). Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
> Matti Aarnio <[EMAIL PROTECTED]> writes: > The NFSv3 SERVER (for which Andrea made his patch) actually > MUST open local files with O_LARGEFILE flag set. (Otherwise > e.g. EXT2 may reject the open.) This should already be done IIRC. At least nfsd_open() always sets O_LARGEFILE. > The NFSv3 CLIENT code must do post-open verification, that if > the opened file has size exceeding 2G-1, and O_LARGEFILE flag > is not set, the opened file must be closed, and caller must be > informed of the rejection with -EFBIG status. (If possible, > do that size verification before open.) The exact same code from ext2_open_file() should work fine in nfs_open(). BTW: There seems to be a bug there: according to the LFS docs, on http://ewe3.sas.com/standards/large.file/x_open.20Mar96.html we should be returning EOVERFLOW rather than EBIG. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 09:34:49PM +0200, Trond Myklebust wrote: > > " " == Matti Aarnio <[EMAIL PROTECTED]> writes: > >When the NFS server does file open, does it do it with > >O_LARGEFILE, or not ? Is there a standardized way to pass > >that flag over NFSv3 ? > > All NFSv3 operations are 64-bit and LFS-compliant. There's therefore > no need for an O_LARGEFILE flag. Ok, so there is no such flag over the NFSv3 XDR protocol, however: The NFSv3 SERVER (for which Andrea made his patch) actually MUST open local files with O_LARGEFILE flag set. (Otherwise e.g. EXT2 may reject the open.) The NFSv3 CLIENT code must do post-open verification, that if the opened file has size exceeding 2G-1, and O_LARGEFILE flag is not set, the opened file must be closed, and caller must be informed of the rejection with -EFBIG status. (If possible, do that size verification before open.) Am I making any sense ? > Cheers, > Trond /Matti Aarnio - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
> " " == Matti Aarnio <[EMAIL PROTECTED]> writes: >When the NFS server does file open, does it do it with >O_LARGEFILE, or not ? Is there a standardized way to pass >that flag over NFSv3 ? All NFSv3 operations are 64-bit and LFS-compliant. There's therefore no need for an O_LARGEFILE flag. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 06:26:25PM +0300, Matti Aarnio wrote: > On Wed, Oct 25, 2000 at 03:57:39PM +0200, Andrea Arcangeli wrote: > > This bug causes nfsd kernel based server to wrongly truncate files > > while using offsets over 4G. With patch applied it starts to work right. > >Is it correct to limit it in anyway at the server ? >Let the CLIENT to handle the O_LARGEFILE testing, and let the SERVER >to just assume it being the situation ? I don't follow. The patch avoids to lose the high 32bit of information during the setattr call. I'm not limiting anything, I'm just allowing the server to see the whole information that cames from the client. >When the NFS server does file open, does it do it with O_LARGEFILE, >or not ? Is there a standardized way to pass that flag over NFSv3 ? NFS server needs to open with O_LARGEFILE to go over 2G. Currently it does that unconditionally (nfsv2 can't ask over 4G by design anyways, and in lfs in 2.2.18pre17aa1 we put an hard limit of 2G for nfsv2 clients to avoid triggering bugs with any other non-linux server, if you need more than 2G just use nfsv3). Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 03:57:39PM +0200, Andrea Arcangeli wrote: > This bug causes nfsd kernel based server to wrongly truncate files > while using offsets over 4G. With patch applied it starts to work right. Is it correct to limit it in anyway at the server ? Let the CLIENT to handle the O_LARGEFILE testing, and let the SERVER to just assume it being the situation ? When the NFS server does file open, does it do it with O_LARGEFILE, or not ? Is there a standardized way to pass that flag over NFSv3 ? ... > Andrea /Matti Aarnio - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 03:57:39PM +0200, Andrea Arcangeli wrote: This bug causes nfsd kernel based server to wrongly truncate files while using offsets over 4G. With patch applied it starts to work right. Is it correct to limit it in anyway at the server ? Let the CLIENT to handle the O_LARGEFILE testing, and let the SERVER to just assume it being the situation ? When the NFS server does file open, does it do it with O_LARGEFILE, or not ? Is there a standardized way to pass that flag over NFSv3 ? ... Andrea /Matti Aarnio - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
Matti Aarnio [EMAIL PROTECTED] writes: The NFSv3 SERVER (for which Andrea made his patch) actually MUST open local files with O_LARGEFILE flag set. (Otherwise e.g. EXT2 may reject the open.) This should already be done IIRC. At least nfsd_open() always sets O_LARGEFILE. The NFSv3 CLIENT code must do post-open verification, that if the opened file has size exceeding 2G-1, and O_LARGEFILE flag is not set, the opened file must be closed, and caller must be informed of the rejection with -EFBIG status. (If possible, do that size verification before open.) The exact same code from ext2_open_file() should work fine in nfs_open(). BTW: There seems to be a bug there: according to the LFS docs, on http://ewe3.sas.com/standards/large.file/x_open.20Mar96.html we should be returning EOVERFLOW rather than EBIG. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 11:25:14PM +0300, Matti Aarnio wrote: The NFSv3 SERVER (for which Andrea made his patch) actually MUST open local files with O_LARGEFILE flag set. (Otherwise e.g. EXT2 may reject the open.) Exactly, that is the code-sharing implementation detail I was mentioning. That must be true not only for ext2 of course but for every other fs that nfsd is sitting upon. That was just fine in 2.4.0-test10-pre4. Andi spotted and fixed that some week ago. The NFSv3 CLIENT code must do post-open verification, that if the opened file has size exceeding 2G-1, and O_LARGEFILE flag is not set, the opened file must be closed, and caller must be informed of the rejection with -EFBIG status. (If possible, do that size verification before open.) 2.2.18pre17aa1 lfs is completly complaint in this sense. Am I making any sense ? You're completly right. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Wed, Oct 25, 2000 at 10:49:52PM +0200, Trond Myklebust wrote: This should already be done IIRC. At least nfsd_open() always sets O_LARGEFILE. Yes, that is just working right. we should be returning EOVERFLOW rather than EBIG. Right. 2.4.0-test10-pre4 is also buggy because it doesn't check for O_LARGEFILE while reading and writing (it should return EOVERFLOW from read and EBIG during writes). 2.2.18pre17aa1 allows anybody to open a file larger than 2G (that seems not to be completly in sync with the specs) but it has all the necessary runtime checks during read/write. Just search for O_LARGEFILE in the lfs patch. That needs to be forward ported to 2.4.x. Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Thu, Oct 26, 2000 at 12:06:17AM +0200, Andrea Arcangeli wrote: 2.2.18pre17aa1 allows anybody to open a file larger than 2G (that seems not to be completly in sync with the specs) but it has all the necessary runtime checks during read/write. Just search for O_LARGEFILE in the lfs patch. That needs to be forward ported to 2.4.x. 2.4 nfsd should already use O_LARGEFILE, I submitted a patch for it some time ago and Linus included it. It doesn't do the 2GB checking for v2, so a v2 client could in theory write a 4GB-1 file. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
On Thu, Oct 26, 2000 at 03:56:46AM +0200, Andi Kleen wrote: a 4GB-1 file. Side note: 4GB-2 since -1 means "ignore". Andrea - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
" " == Andrea Arcangeli [EMAIL PROTECTED] writes: On Wed, Oct 25, 2000 at 09:34:49PM +0200, Trond Myklebust wrote: All NFSv3 operations are 64-bit and LFS-compliant. There's therefore no need for an O_LARGEFILE flag. I don't understand what you mean, sorry. Userspace programs must open with O_LARGEFILE to go over 2G. This is true regardless of the underlying FS. You can find the details in the specs. Sure, but I understood Matti's question as being about how NFS passes O_LARGEFILE over the wire. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: nfsv3d wrong truncates over 4G
" " == Matti Aarnio [EMAIL PROTECTED] writes: The NFSv3 CLIENT code must do post-open verification, that if the opened file has size exceeding 2G-1, and O_LARGEFILE flag is not set, the opened file must be closed, and caller must be Just out of curiosity: why can this particular check not be done by the VFS? Surely this applies to all filesystems... Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
