Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 that's where the boot-nfsroot-options comes from:-) if you notice, the filehandle for v3 is 64 bytes, but only 28 are used. but as I mentioned initially, this ONLY works when the server is FreeBSD, and breaks for other servers, ie NetAPP. AND the initial question stands: what's in a filehandle, or can it be > 28bytes. Yea, FreeBSD is making the assumption that all NFS servers will use the same size FH for NFSv3. That is just wrong. carful, I think this is the case only if fsb is the server, it will 'probably' accept filehandles of other sizes from other servers. I'm talking about the diskless root mounting code only at this point.. The FH is a server created opaque handle that it can create however it wishes. Most servers use information like inode, generation, fsid, etc to create it, but it's something that you can't necessarily decode. yes, but the FH has information that the server can/must use to figure out which local filesystem it refers to - remember that v2/v3 are stateless. Right, see my list right above your comment: inode, generation, fsid. Those three can uniquely identify a file on a filesystem on a server. There can be anything the server wants to stuff in the FH, or the FH can be a random number assigned to that file, etc. I've created a patch that might fix this, but I'm still testing and QEMU (which I use for my testing) keeps making my system either panic or lock up, so hopefully I should have something for you to try tonight. Also - can you tell me the exact 'mount' command you tried to do the remount/update? it's only in the diskless boot, where setting boot-nfsroot-options = "nfsv3" in /boot/loader.conf will do the remount. Ok - I'll do a little more testing on my patch tonight and let you know. Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
> Danny Braniss wrote: > >> Danny Braniss wrote: > Danny Braniss wrote: > > there is an undocumented option: > > boot-nfsroot-options > > that the diskeless boot can use. I tried > > boot-nfsroot-options = "nfsv3" > > since the pxeboot does the initial mount via nfsv2, and this has at > > least > > one problem: removing a file from the readonly / will hang the system. > > > > so, the remount to v3 works in the case that the root is served by a > > Freebsd > > nfs server, but fails if it's NetAPP. The reason is that the v2 > > filehandle > > is 32 bytes, and when switching to V3 it becomes 28bytes - > > sizeof(fhandle_t). > > This is not liked by the NetApp, which correctly gives error 1001: > > BADHANDLE > > :-) > > > > While I'm trying to come up with a solution, I am wondering if someone > > can shed some light: > > - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to > >32 bytes will start WW3? > NFSv3 file handles (by spec) can be up to 64bytes. > >>> true, but in freebsd, look at sys/nfs/nfsproto.h > >>> #define NFSX_V2FH 32 > >>> #define NFSX_V3FH (sizeof (fhandle_t)) > >>> #define NFSX_V4FH 128 > >>> > >>> so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > >>> > >>> > I'm not 100% sure what is happening, but it sounds like the file handle > for the mount point or maybe one of the directories is not getting reset > on remount. > > When do you get the BADHANDLE error? Can you capture a > tshark/wireshark/tcpdump of the remount and error? > >>> I did, and if you look in sys/nfsclient/nfs_vfsops.c, > >>> nfs_convert_diskless is responsible > >>> for chopping off the 4 extra bytes. BTW, I tried to change the bcopy > >>> count to NFSX_V2FH/32, and > >>> it panics the kernel :-( > >>> > >>> danny > >> > >> oh - looks like this says it all: > >> http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 > >> > > that's where the boot-nfsroot-options comes from:-) > > if you notice, the filehandle for v3 is 64 bytes, but > > only 28 are used. > > > > but as I mentioned initially, this ONLY works when the server is FreeBSD, > > and > > breaks for other servers, ie NetAPP. AND the initial question stands: > > what's in a filehandle, or can it be > 28bytes. > > > Yea, FreeBSD is making the assumption that all NFS servers will use the > same size FH for NFSv3. That is just wrong. > carful, I think this is the case only if fsb is the server, it will 'probably' accept filehandles of other sizes from other servers. > The FH is a server created opaque handle that it can create however it > wishes. Most servers use information like inode, generation, fsid, etc > to create it, but it's something that you can't necessarily decode. > yes, but the FH has information that the server can/must use to figure out which local filesystem it refers to - remember that v2/v3 are stateless. > I've created a patch that might fix this, but I'm still testing and QEMU > (which I use for my testing) keeps making my system either panic or lock > up, so hopefully I should have something for you to try tonight. > > Also - can you tell me the exact 'mount' command you tried to do the > remount/update? > it's only in the diskless boot, where setting boot-nfsroot-options = "nfsv3" in /boot/loader.conf will do the remount. cheers, danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 that's where the boot-nfsroot-options comes from:-) if you notice, the filehandle for v3 is 64 bytes, but only 28 are used. but as I mentioned initially, this ONLY works when the server is FreeBSD, and breaks for other servers, ie NetAPP. AND the initial question stands: what's in a filehandle, or can it be > 28bytes. Yea, FreeBSD is making the assumption that all NFS servers will use the same size FH for NFSv3. That is just wrong. The FH is a server created opaque handle that it can create however it wishes. Most servers use information like inode, generation, fsid, etc to create it, but it's something that you can't necessarily decode. I've created a patch that might fix this, but I'm still testing and QEMU (which I use for my testing) keeps making my system either panic or lock up, so hopefully I should have something for you to try tonight. Also - can you tell me the exact 'mount' command you tried to do the remount/update? Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
> Danny Braniss wrote: > >> Danny Braniss wrote: > >>> there is an undocumented option: > >>> boot-nfsroot-options > >>> that the diskeless boot can use. I tried > >>> boot-nfsroot-options = "nfsv3" > >>> since the pxeboot does the initial mount via nfsv2, and this has at least > >>> one problem: removing a file from the readonly / will hang the system. > >>> > >>> so, the remount to v3 works in the case that the root is served by a > >>> Freebsd > >>> nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle > >>> is 32 bytes, and when switching to V3 it becomes 28bytes - > >>> sizeof(fhandle_t). > >>> This is not liked by the NetApp, which correctly gives error 1001: > >>> BADHANDLE > >>> :-) > >>> > >>> While I'm trying to come up with a solution, I am wondering if someone > >>> can shed some light: > >>> - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to > >>>32 bytes will start WW3? > >> > >> NFSv3 file handles (by spec) can be up to 64bytes. > > > > true, but in freebsd, look at sys/nfs/nfsproto.h > > #define NFSX_V2FH 32 > > #define NFSX_V3FH (sizeof (fhandle_t)) > > #define NFSX_V4FH 128 > > > > so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > > > > > >> I'm not 100% sure what is happening, but it sounds like the file handle > >> for the mount point or maybe one of the directories is not getting reset > >> on remount. > >> > >> When do you get the BADHANDLE error? Can you capture a > >> tshark/wireshark/tcpdump of the remount and error? > > > > I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless > > is responsible > > for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count > > to NFSX_V2FH/32, and > > it panics the kernel :-( > > > > danny > > > oh - looks like this says it all: > http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 > that's where the boot-nfsroot-options comes from:-) if you notice, the filehandle for v3 is 64 bytes, but only 28 are used. but as I mentioned initially, this ONLY works when the server is FreeBSD, and breaks for other servers, ie NetAPP. AND the initial question stands: what's in a filehandle, or can it be > 28bytes. danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 Snippet: 51 /* 52 * I have defined a new structure that can handle an NFS Version 3 file handle 53 * but the kernel still expects the old Version 2 one to be provided. The 54 * changes required in nfs_vfsops.c for using the new are documented there in 55 * comments. (I felt that breaking network booting code by changing this 56 * structure would not be prudent at this time, since almost all servers are 57 * still Version 2 anyhow.) 58 */ Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
> Danny Braniss wrote: > > there is an undocumented option: > > boot-nfsroot-options > > that the diskeless boot can use. I tried > > boot-nfsroot-options = "nfsv3" > > since the pxeboot does the initial mount via nfsv2, and this has at least > > one problem: removing a file from the readonly / will hang the system. > > > > so, the remount to v3 works in the case that the root is served by a Freebsd > > nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle > > is 32 bytes, and when switching to V3 it becomes 28bytes - > > sizeof(fhandle_t). > > This is not liked by the NetApp, which correctly gives error 1001: > > BADHANDLE > > :-) > > > > While I'm trying to come up with a solution, I am wondering if someone > > can shed some light: > > - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to > >32 bytes will start WW3? > > > NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > > I'm not 100% sure what is happening, but it sounds like the file handle > for the mount point or maybe one of the directories is not getting reset > on remount. > > When do you get the BADHANDLE error? Can you capture a > tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
nfs v2/v3 and diskless boot problem
there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? cheers danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
