rpc.lockd
Once, long ago, some people emailed me a set of issues regarding our lockd implementation. Would people be willing to re-email me those, or take a fresh look? I once again have time and people to do development on this; and it was 99% there last time; the only issues being some byte-swapping in the RPC code under 64bits... but I don't remember where. And perhaps some other minor problems that I forget. Of particular interest would be little-endian 64 to little-endian 64 testing. The code is at http://www.cs.rpi.edu/~crossd/FreeBSD/lockd-0.2a.tar.gz Thank you -- David Cross | email: [EMAIL PROTECTED] Lab Director | Rm: 308 Lally Hall Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
Hi, I'm making progress with the code and I have also a new diff available. http://www.attic.ch/patches/rpc.diff_01052001-05.tgz > Issues with the code: > > 1.) NETBSD sets in svc_tcp.c some LOCAL_CREDS which we don't have in our > src/sys/kern/uipc_usrreq.c. They have a FLAG which - if set - > automatically sends the credentials on AF_UNIX sockets connections > if we do a recvmsg(). We have to implement this to have rpcbind properly > working. AF_UNIX socket operations are broken at the moment, but > with compability-mode 'rpcbind -Li' rpcbind works. Not solved yet. Bill, can you help me with the code ? I tried to send first a scm.type of SCM_CREDS with sendmsg(), and to recieve the Creds after with recvmsg() but it does not work as expected. Can you help me to resolve this problem ? I'm I doing something completly wrong ? I'm often on IRC, you know me as 'mbr'. > > 2. Maybe it's related to 1., but unregistring the rpc-services does not > work at the moment. > not solved yet. > 3.) It breaks 'make world', some rpc based programms do not find the > symbol for a extern defined variable from rpcgen generated code. NetBSD > has the same code generated from rpcbind, so it has to be somewhere > defined. Can someone with a netbsd account look if they have the same > problem ? > > cc -O -pipe -I.-o rpc.statd file.o sm_inter_svc.o statd.o procs.o > -lrpcsvc > sm_inter_svc.o: In function `sm_prog_1': > sm_inter_svc.o(.text+0xb): undefined reference to `_rpcsvcdirty' Fixed. This was solved my looking a little bit closer to the NetBSd rpcgen code which was infact a old version and completly messed. We use at the moment the newest rpcgen code available and generate the better code than NetBSD does (Thread-Safe, some fixes for BSD) > > 4.) The threaded version of the libs may not work. As I have seen > it was untested by NetBSD people and I fixed some bugs myself. One thing > I'm not sure about the way I changed pthread_getspecific(key) in several > files. > > Some time ago this function has changed from > > void * pthread_getspecific(pthread_key_t key, void **value_ptr) to > void * pthread_getspecific(pthread_key_t key); > > And I wonder where the **value_ptr now is. Is this the right way to do > this ? > > value_ptr = (int*)pthread_getspecific(key); > Solved, there is a _pthread_getspecific which has old semantics. > 5.) The semantics of authdes_create() are not the same as in TI-RPC > code of Sun. Can someone look a little bit at this changes ? Fixed, I Imported the newest version of auth_des and it should work now. Made the code thread-safe. Martin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
[chop] > Switching gears back to the BSD/OS approach you mentioned, UNPv1 says, > > On a datagram socket, the credentials accompany every datagram. On > a stream socket, the credentials are sent only once, the first time > data is sent. > > So as long as one is using a SOCK_DGRAM socket, the BSD/OS-NetBSD > approach should be analogous to what we have currently in FreeBSD? What if the client creates a TCP socket, the credentials are sent, then the process fork()s. Yeah, I know: why in the hell would you do that? Who knows, but you have to account for all cases. Even though you may have a TCP stream socket, you're still sending discrete requests and replies over it. -Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
On Tue, Dec 26, 2000 at 03:33:53PM -0800, Bill Paul wrote: > I'm responsible for implementing this feature. Thanks for that! > Using the SCM_CREDS 'hack' was > a) expedient, as it only involved a minor change to the kernel and > b) it seemed to agree with the way RPC worked, i.e. each RPC needs > the credential info for authentication. > Anyway, imagine my surprise when, after going to all the trouble of > thinking up the SCM_CREDS hack, making it work, and then patting myself > on the back for being clever, I opened up my brand new copy of _UNIX > Network Programming, 2nd Edition, Vol I_ and found that some fool at > BSDi had come up with the idea first. :) NetBSD uses the BSD/OS approach > rather than the FreeBSD approach. In theory, you could have both. I > still say the per-message credential mechanism works better with RPC, > but I'm just a crotchety old fart anyway. With regards to `the per-message credential mechanism works better with RPC': in fact, the way Solaris handles this (now?) is a per-message credential mechanism. Local RPC is implemented on top of doors (see UNPv2 chapter 15) rather than sockets. A doors procedure can use door_cred() to get client creditials each time it is invoked (i.e. per message). Switching gears back to the BSD/OS approach you mentioned, UNPv1 says, On a datagram socket, the credentials accompany every datagram. On a stream socket, the credentials are sent only once, the first time data is sent. So as long as one is using a SOCK_DGRAM socket, the BSD/OS-NetBSD approach should be analogous to what we have currently in FreeBSD? -- Jacques Vidrine / [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
> On Tue, Dec 26, 2000 at 02:20:46AM +0100, Martin Blapp wrote: > [snip] > > Issues with the code: > > > > 1.) NETBSD sets in svc_tcp.c some LOCAL_CREDS which we don't have in our > > src/sys/kern/uipc_usrreq.c. They have a FLAG which - if set - > > automatically sends the credentials on AF_UNIX sockets connections > > if we do a recvmsg(). We have to implement this to have rpcbind properly > > working. AF_UNIX socket operations are broken at the moment, but > > with compability-mode 'rpcbind -Li' rpcbind works. > > We have something analogous ... look for SCM_CREDS. It's a shame these > aren't the same on both (Net|Free)BSD. I'm responsible for implementing this feature. When I sat down to try and make secure RPC work, I was unaware of the existence of the LOCAL_CREDS feature that had been implemented in BSD/OS at about the same time. What I wanted was a way to provide credentials for each *message* rather than for each socket, since RPC is more or less message-based. I was also concerned with avoding problems that might arise if a client process fork()ed while holding open a socket to which credential info had been assigned. You obviously don't want the parent and the child process to return the same credential info. Using the SCM_CREDS 'hack' was a) expedient, as it only involved a minor change to the kernel and b) it seemed to agree with the way RPC worked, i.e. each RPC needs the credential info for authentication. The reason you need the LOCAL_CREDS/SCM_CREDS stuff at all is that keyserv needs to know the identity of the user that's talking to it. It must not allow access to a user's diffie-helman key pair to anyone other than the user to whom it belongs (and, potentially, the superuser). The problem is the original sockets API did not provide any way for this authentication do be done. Many alternatives were discussed and rejected because they were too complex or just plain didn't work. The notion of using credentials was new in TI-RPC because STREAMS/TLI offers a way to do it. In SunOS 4, there was instead a terrible kludge based on the ugly and bletcherous keyenvoy program. I made keyenvoy work, but it struck me that it had a potentially serious weakness: it depended on the "only root can bind to port numbers less than 1024" property of UNIX TCP/IP networking, and it distinguished local connections from remote ones by comparing the origin IP address with 127.0.0.1. (Can you say IP spoofing? I knew you could.) Anyway, imagine my surprise when, after going to all the trouble of thinking up the SCM_CREDS hack, making it work, and then patting myself on the back for being clever, I opened up my brand new copy of _UNIX Network Programming, 2nd Edition, Vol I_ and found that some fool at BSDi had come up with the idea first. :) NetBSD uses the BSD/OS approach rather than the FreeBSD approach. In theory, you could have both. I still say the per-message credential mechanism works better with RPC, but I'm just a crotchety old fart anyway. Relatively speaking. -Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
On Tue, Dec 26, 2000 at 02:20:46AM +0100, Martin Blapp wrote: [snip] > Issues with the code: > > 1.) NETBSD sets in svc_tcp.c some LOCAL_CREDS which we don't have in our > src/sys/kern/uipc_usrreq.c. They have a FLAG which - if set - > automatically sends the credentials on AF_UNIX sockets connections > if we do a recvmsg(). We have to implement this to have rpcbind properly > working. AF_UNIX socket operations are broken at the moment, but > with compability-mode 'rpcbind -Li' rpcbind works. We have something analogous ... look for SCM_CREDS. It's a shame these aren't the same on both (Net|Free)BSD. To narrow it down for you, here are the relevant files in -CURRENT: src/sys/kern/uipc_usrreq.c src/lib/libc/rpc/clnt_unix.c src/lib/libc/rpc/svc_unix.c keyserv and rpc.yppasswd are example applications that use this feature. -- Jacques Vidrine / [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: [PATCH available] TI-RPC and NetBSD's rpc.lockd
I have a second version available. I killed many whitespace diffs, but there are still a lot. http://www.attic.ch/patches/rpc.diff_26122000.tgz WARNING ! THIS BREAKS MAKE WORLD BECAUSE SOME OLD RPC PROGRAMS DO NOT COMPILE AT THE MOMENT. To build do the following: 1.) patch the codebase 2.) cd /usr/src/usr.bin/rpcgen && make && make install 3.) cd /usr/src/include && make clean & make && make install 4.) cd /usr/src/lib && make clean & make && make install 5.) cd /usr/src/usr.sbin/rpcbind && make && make install 6.) copy /usr/src/etc/netconfig to /etc 7.) Adjust your rc.conf for rpcbind And if you like to test the NetBSD rpc.lockd: 8.) cd /usr/src/usr.sbin/rpc.lockd && make && make install I'm off one week and read the emails when I'm back. Martin Martin Blapp, [EMAIL PROTECTED] Improware AG, UNIX solution and service provider Zurlindenstrasse 29, 4133 Pratteln, Switzerland Phone: +41 79 370 26 05, Fax: +41 61 826 93 01 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
[PATCH available] TI-RPC and NetBSD's rpc.lockd
Hi, I have a first working (but the code has many many whitespaces diff's and I'm sure some errors) version of the TI-NetBSD-RPC code available. It's a diff against CURRENT, but it'll work also on STABLE if you adjust src/include/Makefile - this file is different from CURRENT. The diff is uncompressed 1.5MB, compressed 290kb. You can get it on http://www.attic.ch/patches/rpc.diff.tgz Please send me a diff if you clean up the code. I'll make a new version available. I'm still testing the new rpcbind, and I can normaly use nfsv3 and also YP seems to work as it has before. Can someone test the rpc.lockd ? I'll clean up the code (I have already done a lot !) as I come back from my vacation this week. Issues with the code: 1.) NETBSD sets in svc_tcp.c some LOCAL_CREDS which we don't have in our src/sys/kern/uipc_usrreq.c. They have a FLAG which - if set - automatically sends the credentials on AF_UNIX sockets connections if we do a recvmsg(). We have to implement this to have rpcbind properly working. AF_UNIX socket operations are broken at the moment, but with compability-mode 'rpcbind -Li' rpcbind works. 2. Maybe it's related to 1., but unregistring the rpc-services does not work at the moment. 3.) It breaks 'make world', some rpc based programms do not find the symbol for a extern defined variable from rpcgen generated code. NetBSD has the same code generated from rpcbind, so it has to be somewhere defined. Can someone with a netbsd account look if they have the same problem ? cc -O -pipe -I.-o rpc.statd file.o sm_inter_svc.o statd.o procs.o -lrpcsvc sm_inter_svc.o: In function `sm_prog_1': sm_inter_svc.o(.text+0xb): undefined reference to `_rpcsvcdirty' 4.) The threaded version of the libs may not work. As I have seen it was untested by NetBSD people and I fixed some bugs myself. One thing I'm not sure about the way I changed pthread_getspecific(key) in several files. Some time ago this function has changed from void * pthread_getspecific(pthread_key_t key, void **value_ptr) to void * pthread_getspecific(pthread_key_t key); And I wonder where the **value_ptr now is. Is this the right way to do this ? value_ptr = (int*)pthread_getspecific(key); 5.) The semantics of authdes_create() are not the same as in TI-RPC code of Sun. Can someone look a little bit at this changes ? - Here is a list of rpcinfo (and rpcinfo -p) su-2.03# rpcinfo localhost program version netid addressserviceowner 104tcp 0.0.0.0.0.111 portmapper superuser 103tcp 0.0.0.0.0.111 portmapper superuser 102tcp 0.0.0.0.0.111 portmapper superuser 104udp 0.0.0.0.0.111 portmapper superuser 103udp 0.0.0.0.0.111 portmapper superuser 102udp 0.0.0.0.0.111 portmapper superuser 104tcp6 ::.0.111 portmapper superuser 103tcp6 ::.0.111 portmapper superuser 104udp6 ::.0.111 portmapper superuser 103udp6 ::.0.111 portmapper superuser 172udp 0.0.0.0.3.140 ypbind unknown 172tcp 0.0.0.0.3.154 ypbind unknown 1000210udp 0.0.0.0.2.107 nlockmgr unknown 1000211udp 0.0.0.0.2.107 nlockmgr unknown 1000213udp 0.0.0.0.2.107 nlockmgr unknown 1000214udp 0.0.0.0.2.107 nlockmgr unknown 1000210tcp 0.0.0.0.3.144 nlockmgr unknown 1000211tcp 0.0.0.0.3.144 nlockmgr unknown 1000213tcp 0.0.0.0.3.144 nlockmgr unknown 1000214tcp 0.0.0.0.3.144 nlockmgr unknown 1000210udp6 :::0.0.0.0.2.106 nlockmgr unknown 1000211udp6 :::0.0.0.0.2.106 nlockmgr unknown 1000213udp6 :::0.0.0.0.2.106 nlockmgr unknown 1000214udp6 :::0.0.0.0.2.106 nlockmgr unknown 1000210tcp6 :::0.0.0.0.3.135 nlockmgr unknown 1000211tcp6 :::0.0.0.0.3.135 nlockmgr unknown 1000213tcp6 :::0.0.0.0.3.135 nlockmgr unknown 1000214tcp6 :::0.0.0.0.3.135 nlockmgr unknown 1000241udp 0.0.0.0.2.229 status unknown 1000241tcp 0.0.0.0.3.121 status unknown 153udp 0.0.0.0.3.123 mountd unknown 153tcp 0.0.0.0.3.115 mountd unknown 151udp 0.0.0.0.3.123 mountd unknown 151tcp 0.
Re: rpc.lockd and true NFS locks?
On Sat, Dec 16, 2000 at 04:27:20PM -0600, Dan Nelson wrote: > In the last episode (Dec 16), Axel Thimm said: > > Wouldn't that mean, that you might cause data corruption if, say, I was to > > read my mail from a FreeBSD box over an NFS mounted spool directory > > (running under OSF1 in our case), and I decided to write back the mbox to > > the spool dir the same moment new mail is delivered? > That's why dotlocking is recommended for locking mail spools. Both procmail > and mutt will dotlock your mail file while it's being accessed. This was just a test case above. Not all programs are kind enough to allow control of their locking strategy. What about samba accessing NFS volumes in a transparent net or pure sendmail w/o procmail? Especially if your mail server is already at heavy load serving O(1000) users, forcing each incomming mail to be passed to procmail would must certainly increase the load too much. (Maybe sendmail and samba can also be compiled with dotlocking methods, these are also just examples). Also not all our users want to switch to mutt, we have to support a wide range of mail readers. Axel. -- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
Don Coleman wrote: > > David, > > I wrote the NFS lockd code for BSD/OS (it's based on some user land > stuff Keith Bostic did, and then Kirk McKusick helped clean up my > basic design and the VFS layering for the server/kernel side). We have an application that is desperately in need of client side NFS locks, so I'm highly motivated to test this out if it can be ported to either -stable or -current. Doug -- "The most difficult thing in the world is to know how to do a thing and to watch someone else do it wrong without comment." -- Theodore H. White Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
On Sat, Dec 16, 2000 at 04:26:58PM -0600, Dan Nelson wrote: > That's why dotlocking is recommended for locking mail spools. Both > procmail and mutt will dotlock your mail file while it's being > accessed. Or Maildirs. -- Jos Backus _/ _/_/_/"Modularity is not a hack." _/ _/ _/-- D. J. Bernstein _/ _/_/_/ _/ _/ _/_/ [EMAIL PROTECTED] _/_/ _/_/_/use Std::Disclaimer; To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
In the last episode (Dec 16), Axel Thimm said: > Wouldn't that mean, that you might cause data corruption if, say, I > was to read my mail from a FreeBSD box over an NFS mounted spool > directory (running under OSF1 in our case), and I decided to write > back the mbox to the spool dir the same moment new mail is delivered? That's why dotlocking is recommended for locking mail spools. Both procmail and mutt will dotlock your mail file while it's being accessed. -- Dan Nelson [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
David, I wrote the NFS lockd code for BSD/OS (it's based on some user land stuff Keith Bostic did, and then Kirk McKusick helped clean up my basic design and the VFS layering for the server/kernel side). It has passed the connectathon tests, and has been being used by BSD/OS customers for a while, but mostly for locking mail files (why they don't use POP or IMAP, I've never been able to figure out), and locking files being edited by vi over an NFS export. I think we've had 1 bug report/fix, at least that got back to me ... it's been in BSD/OS for like 2 years now, but given the lack of bug reports, I doubt it's limits are being pushed. The main feature the BSD/OS lockd code is missing, is the client side of server side recovery... BSD/OS never crashes ;->, so our clients have never reported this problem (we mention it in our man page). If the lockd server crashes/reboots, we do go through a grace period, and we notify the clients they need to re-establish their locks, but our client side doesn't track the current lock states (even though the client kernel has that complete information, the user mode lock daemon on the client side doesn't keep a copy). So a BSD/OS server, with non-BSD/OS clients is fully functional. This problem isn't hard to fix... there is a two step fix, all user-land. First, make the client side user level lockd a single process (under BSD/OS, it is two processes)... the problem is that each process has a piece of the responsibility/knowledge you need to re-establish locks on your server when it crashes. Second, the client side user level lockd needs to be able to figure out what lockd locks the client as a whole has been given... the lockd client needs to keep it's own idea of what lock the client still holds (just keep it in memory, ordered by server, simple and quick). I already have the first part done, it just hasn't been fully tested, and since BSD/OS is customer driven, and no customers have been pushing us, I haven't committed it. Anyone who wants it, let me know, and I'll give it to you. There are also some improvements I'd suggest as you merge the code (a pair of fresh eyes and fingers is always an opportunity!) 1) we use a FIFO to pass data from the kernel to the user land process. we should at least use a socket... 2) we use a private field in the proc structure, even though it is only used by the lockd process... this was so we could clean up all the server side data if the user level process of the server side lockd crashed or was killed (Solaris admins kill lockd all the time). Using a kernel level on_exit(), or perhaps making a lockd vfs (so a lockd_close() would be called when the lockd process exits), are both possibilities. The general idea is that if you kill and restart the lockd process, it should behave just as if the server was rebooted. And of course there are the little stuff, like we don't back-off and re-try as well as we should, and errors can be dropped, and little stuff like that. I'd be happy to answer any questions, and help in any way I can. don ps: I'll be travelling in South East Asia (Bangkok, Mayanmar, Bali) from Jan 9 to Feb 15, so I will not be able to help during that time... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
Thanks for the fast reply. On Thu, Dec 14, 2000 at 05:45:15PM -0500, David E. Cross wrote: > As for "client" vs. "server", that is quite tricky since WRT NFS locking > they are both client and server. The "server" side is done and requires no > modifcations to the kernel. However a FreeBSD kernel is still unable to > acquire an NFS lock. This latter case is quite likely what your users are > seeing the affects of. Just to understand it right: The current rpc.lockd is neither requesting locks, if FreeBSD is an NFS client to whatever NFS server, nor serving such requests as an NFS server to whatever client. Your (David Cross's) uncommited code does implement NFS locks for a FreeBSD NFS server. Perhaps in a development stage, but better than not having locks at all. Now I am quite surprised to learn that FreeBSD apparently is not able to request locks over NFS. Am I right? Wouldn't that mean, that you might cause data corruption if, say, I was to read my mail from a FreeBSD box over an NFS mounted spool directory (running under OSF1 in our case), and I decided to write back the mbox to the spool dir the same moment new mail is delivered? I can't imagine that, I must have misunderstood something, most probably the role of the client part of NFS locks. Could someone clarify? If I were right, then FreeBSD would only be good for read only NFS access, and we were using FreeBSD as NFS clients in our department since before 2.2.x. > In the end: the code is there and available for those who want to test and > play with it. It has not been committed because it is still "broken". > I could do it to -current or make it a port, if someone were to tell me that > it would be "ok" to do so. I would vote for port. Thanks, Axel. P.S. please reply not only to freebsd-hackers, but also Cc: me, as I am only subscribed to freebsd-current and freebsd-stable. -- [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
> David did say that it pretty much works, and preliminary reports > from a while back started getting him some feedback which quickly > died off after people forgot about the announcement. The only confirmed bug (reported from Drew Gallatin @ Duke) appears to be a result of broken RPC64. I _thought_ I had fixed it, but it would appear not. > Actually, that would do us well for the client side, however since > we don't have anyone (so far) from BSD/os to explain the intracate > parts of it, I'd rather see something come in that has someone > familiar with the code rather than something we'd have to grok. This is kinda funny... we both make the same argument, but on different sides :) > Then again, I should take a look at the BSD/os rpc.lockd one weekend. > > David, you have builder access, what do you think about the BSD/os > version? >From what I recall (it has been far too long). It was very similar in design to the epics one. Much of this is dictated via the RPC interface, but we also both picked using the various QUEUE(3) macros. I think the huge benefit with their code is greater testing, and likely better overall design. I don't recall if their code has the stuff for DOS "share" NFS access. Adding that to the rpics code was never finished due to a couple of poor assumptions early on and needing to rework it later to fit. -- David Cross | email: [EMAIL PROTECTED] Lab Director | Rm: 308 Lally Hall Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
Going with the lockd code on builder is great with me. The last I had looked it had some of the same issues as the lockd developed here (no handling of grace periods, etc.), so on a featureset we are even. The rpics lockd has the advantage of being known by some of us to a much greater extent than the BSDI code. _However_ the BSDI code has undergone much more testing and design work than the rpics one. Given this I think the clear choice is with the BSDI code. >sigh< now, if I wasn't always getting buried with stuff. -- David Cross | email: [EMAIL PROTECTED] Lab Director | Rm: 308 Lally Hall Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
* Jordan Hubbard <[EMAIL PROTECTED]> [001214 16:11] wrote: > [-current mailing list pruned; I think -hackers is enough] > > > I would like to see it in both -current and -stable. > > I think that would be wrong, at least given the current state of > the lockd stuff. > > First off, as David himself points out, there are issues with this > code and we'd be well off dealing with those *before* committing it to > -stable. I also don't think that this would be achieved simply by > having more eyes on it, as you intimate, but by actually having a > coherent set of code to work on and the Right Developers(tm) hacking > on it. I agree with Bill Joy's assertion that all bugs are NOT > shallow through having enough eyes, as Linus likes to say, but through > having one or two really bright people practically killing themselves > to fix them. My argument against this is that giving ample warning is a far cry from the Linux mantra "release early, release often, ship the system with async files, we'll let them know how to not loose data _next_ time". Here I'm proposing that we be more than honest. The current fake lockd doesn't even do fake NLMv4 locks (although there's patches that I did do it so that it would floating around). It's also a lot harder to find bugs when you're looking at your own code versus when someone sends you a crashdump because what they were doing is able to tickle a bug you'd never assume was possible. David did say that it pretty much works, and preliminary reports from a while back started getting him some feedback which quickly died off after people forgot about the announcement. > We've also had working NFS lockd code in the BSD/OS tree on builder, > along with full permission to grab it, for some time now but that > hasn't made it happen because the right developers have yet to take > that active an interest. Actually, that would do us well for the client side, however since we don't have anyone (so far) from BSD/os to explain the intracate parts of it, I'd rather see something come in that has someone familiar with the code rather than something we'd have to grok. Then again, I should take a look at the BSD/os rpc.lockd one weekend. David, you have builder access, what do you think about the BSD/os version? -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
:I'm not going to take such an action w/o the blessing of -core. :) : :-- :David Cross | email: [EMAIL PROTECTED] :Lab Director | Rm: 308 Lally Hall In regards to Jordan's message just a moment ago... you know, I *total* forgot that the BSDI working lockd code was now available. On the otherhand I know that Dave and a lot of people spent a lot of hard work on lockd, and also on the kernel-side implementation. If we were to use the BSDI code the kernel-side implementation would almost certainly be retained with only minor (if any) modification. But the experiemental lockd code would be completely replaced. David, how do you feel about that potentially occuring? Would you like to make a go of it with the BSDI lockd code on builder? -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
[-current mailing list pruned; I think -hackers is enough] > I would like to see it in both -current and -stable. I think that would be wrong, at least given the current state of the lockd stuff. First off, as David himself points out, there are issues with this code and we'd be well off dealing with those *before* committing it to -stable. I also don't think that this would be achieved simply by having more eyes on it, as you intimate, but by actually having a coherent set of code to work on and the Right Developers(tm) hacking on it. I agree with Bill Joy's assertion that all bugs are NOT shallow through having enough eyes, as Linus likes to say, but through having one or two really bright people practically killing themselves to fix them. We've also had working NFS lockd code in the BSD/OS tree on builder, along with full permission to grab it, for some time now but that hasn't made it happen because the right developers have yet to take that active an interest. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
On Fri, Dec 15, 2000 at 12:09:32AM +0100, Thierry Herbelot wrote: > Hello, > > I've recently seen in the NetBSD 1.5 release Notes that *they* claim to > have a fully functional rpc.lockd manager : "Server part of NFS locking > (implemented by rpc.lockd(8)) now works." > > could someone have a look at what our cousins have done and perhaps > import it in -current ? according to http://mail-index.netbsd.org/cgi-bin/projects.cgi?token=&mode=viewproj&projnum=70 code to do this was committed to netbsd on jun 7 2000. -- garrett rooney my pid is inigo montoya. [EMAIL PROTECTED] you kill -9 my parent process. http://electricjellyfish.net/prepare to vi. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
I'm not going to take such an action w/o the blessing of -core. :) -- David Cross | email: [EMAIL PROTECTED] Lab Director | Rm: 308 Lally Hall Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
Hello, I've recently seen in the NetBSD 1.5 release Notes that *they* claim to have a fully functional rpc.lockd manager : "Server part of NFS locking (implemented by rpc.lockd(8)) now works." could someone have a look at what our cousins have done and perhaps import it in -current ? TfH -- Thierry Herbelot To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
* David E. Cross <[EMAIL PROTECTED]> [001214 14:45] wrote: > I pruned the Cc: list a bit... > > One of the email messages that you quoted has the URL for the latest > development of the lockd code. As far as tests go it appears to be mostly > complete (there appears to be an issue with RPC64 on little endian machines, > but I have not yet had a chance to crawl through the librpc code). > > As for "client" vs. "server", that is quite tricky since WRT NFS locking > they are both client and server. The "server" side is done and requires no > modifcations to the kernel. However a FreeBSD kernel is still unable to > acquire an NFS lock. This latter case is quite likely what your users are > seeing the affects of. > > In the end: the code is there and available for those who want to test and > play with it. It has not been committed because it is still "broken". > I could do it to -current or make it a port, if someone were to tell me that > it would be "ok" to do so. I would like to see it in both -current and -stable. Please take a deep breath and keep reading. :) I think that although it's partially broken if we gave appropriate warning to the users about the experimental nature of the code we'd be doing them a favor by getting the code out so that _early adopters_ so that they can give us feedback. I do not support removing the current "fake" lockd until we've had ironed out the issues with the experimental lockd. I do not like _only_ having it in -current because then people will never consider it, I have confidence that academic installations and hobbiests would give it a shot, and who knows, maybe we'll get a knock on the door from someone who's completed the client, after all, what use is the client code without the server code? As an interim solution we could put the lockd into the system as rpc.lockd-experimental. I think had we done this over six months ago when you made the initial announcement we'd have enough feedback to begin ironing out the kinks. I want this in the user's faces, taunting and enticing them to use it and give us feedback. :) So can you commit this? -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd and true NFS locks?
I pruned the Cc: list a bit... One of the email messages that you quoted has the URL for the latest development of the lockd code. As far as tests go it appears to be mostly complete (there appears to be an issue with RPC64 on little endian machines, but I have not yet had a chance to crawl through the librpc code). As for "client" vs. "server", that is quite tricky since WRT NFS locking they are both client and server. The "server" side is done and requires no modifcations to the kernel. However a FreeBSD kernel is still unable to acquire an NFS lock. This latter case is quite likely what your users are seeing the affects of. In the end: the code is there and available for those who want to test and play with it. It has not been committed because it is still "broken". I could do it to -current or make it a port, if someone were to tell me that it would be "ok" to do so. -- David Cross | email: [EMAIL PROTECTED] Lab Director | Rm: 308 Lally Hall Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
rpc.lockd and true NFS locks?
Dear all, rpc.lockd in FreeBSD suffers from a pubic server's lazyness --- It says it's done the job, but never did anything besides talking... Searching through the lists gives different stories. Some say that NFS locking isn't really necessary, but what about locking critical situations like delivering mail over NFS to FreeBSD homes? Procmail & fcntl made our computing department especially unhappy, and we are wondering whether we can keep our migration strategy (moving our homes to backuped FreeBSD boxes). Some of the following quoted mails (consider this mail as a review, if you like) give hope that some people were working on this (without obviously having commited anything, as one can check in cvsweb). Is this true? Has anyone any server side patches for FreeBSD? Is he/she looking for guinea pigs? Anything is better than the current situation. Our users are running away from our otherwise very comfortable FreeBSD homes. :( On Mon, Apr 03, 2000 at 02:07:54PM +0200, Brad Knowles wrote: > [...] > Besides, file locking becomes impossible in -STABLE once you've > mounted it with NFS (we don't have a working lockd, although work in this > area is progressing in -CURRENT), and NFS writes generally suck when > compared to local writes. > [...] On Fri, Apr 07, 2000 at 08:07:40PM -0400, David E. Cross wrote: > I apologize profusely for the delay of this, but lockd-0.2 is out. > The URL is: http://www.cs.rpi.edu/~crossd/FreeBSD/lockd-0.2.tar.gz > [...] > 5) this does not add the code to FreeBSD's kernel to request the NFS locks >(that is a job for people more skilled than I ;) > [...] On Sat, Apr 08, 2000 at 12:23:14AM -0400, David E. Cross wrote: > [...] > http://www.cs.rpi.edu/~crossd/FreeBSD/lockd-0.2a.tar.gz > [...] On Fri, Apr 07, 2000 at 08:44:33PM -0400, Andrew Gallatin wrote: > This might be a bit touchy, but I'm rather curious -- how will the BSDI > merger affect your lockd work? It seems like your work on lockd > (esp. client side & statd interoperation issues) could be speeded up if you > had access to the BSDI sources.. On Tue, Sep 19, 2000 at 12:38:51PM +0200, Roman Shterenzon wrote: > Quoting Andrew Gordon <[EMAIL PROTECTED]>: > > On Mon, 4 Sep 2000, Roman Shterenzon wrote: > > > The rpc.lockd(8) is marked as broken in /etc/defaults/rc.conf in 4.1-R > > > My question is - how bad is it broken? > > The rpc.lockd in 4.x simply answers "Yes" to all locking requests, and > > does not maintain any state. This means that if your programs actually > > need locking, running rpc.lockd will cause problems (file corruption etc). > > > > On the other hand, if your programs don't need locking and are just making > > the locking calls for the hell of it, rpc.lockd will allow these programs > > to run rather than just hanging up. > > > > There was talk a few months ago about someone having implemented NFS > > locking properly, but I haven't heard any more since - check the mailing > > list archives. > > > > [I wrote the existing 'hack' implementation]. On Wed, Sep 20, 2000 at 09:28:36AM -0500, Scot W. Hetzel wrote: > From: "Roman Shterenzon" <[EMAIL PROTECTED]> > > On Tue, 19 Sep 2000 [EMAIL PROTECTED] wrote: > > > [...] Someone (from something.edu, perhaps rpi.edu) posted a URL to one > > > of the lists of a working but untested rpc.lockd. [...] I believe that Andrew means "David E. Cross" <[EMAIL PROTECTED]>, but his citation some lines above show that he hadn't worked in that direction. > I kind of remember reading about it on the current mailing list. > Current-Users: Has a working rpc.lockd been imported into CURRENT. If it > has, is there a possibility of getting it MFC'd to STABLE. On Thu, Sep 21, 2000 at 11:02:25AM +0930, Greg Lewis wrote: > Look through the freebsd-hackers archive. There was an rpc.lockd > implementation announced there looking for testers about a month or so > before the 4.0 release. The person who wrote it is David Cross who is now a > FreeBSD committer I believe. > > Thats my recollection anyway. Unfortunately I haven't seen any recent > followups. At the time it was deemed too close to the 4.0 release. If you > do test it maybe you can prod David with the results and get it committed to > -current. On Wed, Nov 08, 2000 at 05:45:21PM +0100, Erik Trulsson wrote: > On Wed, Nov 08, 2000 at 02:53:47PM +0100, cam wrote: > > I have to use rpc.lockd on my NFS server (FreeBSD 4.0-STABLE) and I've > > notice that it is broken with this line in /etc/defaults/rc.conf: > > 113: rpc_lockd_enable="NO" # Run NFS rpc.lockd (*broken!*) if >nfs_server. > > You
Re: rpc.lockd and xdr.
On Mon, 6 Mar 2000, David E. Cross wrote: :Version 2 of the lock manager is ready to be released. Amitha :says that it passes all of the tests in the suite posted by Drew (thanks :Drew). A noteable exception to this is on SGI where some lock requests :are never even received from the remote host. Also DOS sharing is not :yet complete. Any idea why this is? I only ask because: 12:12pm banshee /home/jamie %uname -aR IRIX64 banshee 6.5 6.5.5m 07151433 IP25 I have some PC's that I'm interested in putting FreeBSD on (other than my laptop) that would be on the network fulltime, and working NFS locking is really the only stumbling bock for me now. Jamie Bowden -- "Of course, that's sort of like asking how other than Marketing, Microsoft is different from any other software company..." Kenneth G. Cavness To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
rpc.lockd and xdr.
Version 2 of the lock manager is ready to be released. Amitha
says that it passes all of the tests in the suite posted by Drew (thanks
Drew). A noteable exception to this is on SGI where some lock requests
are never even received from the remote host. Also DOS sharing is not
yet complete.
On a side note, it would appear that at least some of the problems of the
previous version were in FreeBSD's XDR library. The xdr_*_int64 routines
do not correctly do network byte order conversions. Included below is
Amitha's 'hack fix'.
My hack fix (this is to /usr/src/lib/libc/xdr/xdr.c) is below. A similar
fix needs to be applied to xdr_int64_t. Note that xdr_opaque takes care of
swapping the bits in the byte.
#define SWAP(a,b,t) t=a;a=b;b=t
bool_t
my_xdr_u_int64_t(xdrs, uint64_p)
register XDR *xdrs;
u_int64_t *uint64_p;
{
u_int64_t x;
unsigned char* b= &x;
unsigned char t;
switch (xdrs->x_op) {
case XDR_ENCODE:
SWAP(b[0], b[7], t);
SWAP(b[1], b[6], t);
SWAP(b[2], b[5], t);
SWAP(b[3], b[4], t);
return (xdr_opaque(xdrs, (caddr_t)uint64_p, sizeof(u_int64_t)))
;
case XDR_DECODE:
if (!xdr_opaque(xdrs, (caddr_t)&x, sizeof x)) {
return (FALSE);
}
SWAP(b[0], b[7], t);
SWAP(b[1], b[6], t);
SWAP(b[2], b[5], t);
SWAP(b[3], b[4], t);
*uint64_p = x;
return (TRUE);
case XDR_FREE:
return (TRUE);
}
return (FALSE);
}
=
--
David Cross | email: [EMAIL PROTECTED]
Acting Lab Director | NYSLP: FREEBSD
Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd
Rensselaer Polytechnic Institute, | Ph: 518.276.2860
Department of Computer Science| Fax: 518.276.4033
I speak only for myself. | WinNT:Linux::Linux:FreeBSD
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd
First of all, don't crosspost to both -hackers and -current. They tend to cover the same audience. On Fri, 11 Feb 2000, David E. Cross wrote: > I realize that we are all very busy and the coming 4.0-RELEASE has also > compounded things, but I have heard nothing back on the rpc.lockd that > was released just a short time ago. I take it no news is good news and > we can start the process of bringing it into the source tree? :) 4.0-RELEASE is in code freeze, so it won't make in there. I suspect someone will be available to get it into -CURRENT after 4.0 ships. Doug White| FreeBSD: The Power to Serve [EMAIL PROTECTED] | www.FreeBSD.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: rpc.lockd
:I realize that we are all very busy and the coming 4.0-RELEASE has also :compounded things, but I have heard nothing back on the rpc.lockd that :was released just a short time ago. I take it no news is good news and :we can start the process of bringing it into the source tree? :) : :-- :David Cross | email: [EMAIL PROTECTED] :Acting Lab Director | NYSLP: FREEBSD :Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd :Rensselaer Polytechnic Institute, | Ph: 518.276.2860 I've been very interested in it, but I was going to wait until after the release before starting to mess with it. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
rpc.lockd
I realize that we are all very busy and the coming 4.0-RELEASE has also compounded things, but I have heard nothing back on the rpc.lockd that was released just a short time ago. I take it no news is good news and we can start the process of bringing it into the source tree? :) -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
rpc.lockd... is done.
Amitha (the person who has been working on the lockd code) has finished most of his work. There are still some issues with handling async locks and cancel messages. Also we were not able to implement the full NLM protocol as the FreeBSD kernel does not currently request NFS locks (we should fix that ASAP). This code is *ALPHA*. Even we will not be running it on production servers in the near future. PS: the tarball is at "http://www.cs.rpi.edu/~crossd/FreeBSD/lockd-2208.tar.gz" PPS: I would like to set this up in CVS for everyone's ease, could someone please tell em how to do this, and to make it available via cvsup? (We already have a complete FreeBSD cvsup mirror at cvsup.cs.rpi.edu, this would just be another "module", right? -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
rpc.lockd
It is almost done. A working and very lightly tested version of the code will be made available on Monday (Jan 24). It should be considered alpha quality, I would not recommend running important NFS servers with this code. -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Tue, 30 Nov 1999, Dan Moschuk wrote: > > | I reviewed the NFSv4 specs recently and came to the same conclusion. To do > | it right will be quite a bit of work and would include a decent kernel > | side implementation of rpc and gssapi. > > Cool! I can take that "I volunteer" ? :-) Not with my current workload :-(. -- Doug Rabson Mail: [EMAIL PROTECTED] Nonlinear Systems Ltd. Phone: +44 181 442 9037 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
| I reviewed the NFSv4 specs recently and came to the same conclusion. To do | it right will be quite a bit of work and would include a decent kernel | side implementation of rpc and gssapi. Cool! I can take that "I volunteer" ? :-) -- Dan Moschuk ([EMAIL PROTECTED]) "Cure for global warming: One giant heatsink and dual fans!" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Mon, 29 Nov 1999, Dan Moschuk wrote: > > | >I'm fairly certain that rpc.lockd is included with Darwin from Apple, > | >I've not closely compared it to what we have in -STABLE or -CURRENT > | >to see if it actually works. > | > | It doesn't, sorry... if someone gets a *BSD version of NFS locking operating > | I'd help see it into Darwin. > | > | BTW, is anyone working on NFS Version 4 for BSD? Rick? > > A few of us talked about this on IRC a few weeks ago and it was decided that > when/if we implement NFSv4 it would have to be from scratch. A pretty big > task. I reviewed the NFSv4 specs recently and came to the same conclusion. To do it right will be quite a bit of work and would include a decent kernel side implementation of rpc and gssapi. -- Doug Rabson Mail: [EMAIL PROTECTED] Nonlinear Systems Ltd. Phone: +44 181 442 9037 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
| >I'm fairly certain that rpc.lockd is included with Darwin from Apple, | >I've not closely compared it to what we have in -STABLE or -CURRENT | >to see if it actually works. | | It doesn't, sorry... if someone gets a *BSD version of NFS locking operating | I'd help see it into Darwin. | | BTW, is anyone working on NFS Version 4 for BSD? Rick? A few of us talked about this on IRC a few weeks ago and it was decided that when/if we implement NFSv4 it would have to be from scratch. A pretty big task. -- Dan Moschuk ([EMAIL PROTECTED]) "Cure for global warming: One giant heatsink and dual fans!" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
:I can't help with the code either - what little time I have available :is going towards the SMP project. I will be shorting beginning work ^ shortly. Bleh. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
:Well, I am starting to get pretty seriously involved. It looks pretty :easy, just a lot of small details (this is the kind of coding I like :) : :A couple of issues need to be worked out. First I need to backport :the FH open/stat/etc. calls to -STABLE. The main reason for this is that :... I can't help with the code either - what little time I have available is going towards the SMP project. I will be shorting beginning work work on another startup and consulting on a second (but that won't stop me from continuing to badger core to remove the idiotic extra restrictions they still have on my commit privs). But I do I have time to test patchsets as your work progresses. My test boxes are currently split into a -stable test box and a -current test box so I can test under both. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
Well, I am starting to get pretty seriously involved. It looks pretty easy, just a lot of small details (this is the kind of coding I like :) A couple of issues need to be worked out. First I need to backport the FH open/stat/etc. calls to -STABLE. The main reason for this is that I am developing rpc.lockd for out main servers, which for obvious reasons run -STABLE. Second I need some input and advice. Looking at Solaris' lockd, they use NLM version 2 as a method of IPC between lockd and statd this is an undocumented interface. I would suggest a unix domain socket as an alternate method (any objections). This I feel has a number of advantages, such as providing an easier Kernel interface to the user-land lock manager (it is my reading that rpc.lockd should perform the tasks of maintaining local locks on remote servers as well as managing locks on local filesystems from remote servers.) Thank you to the person who provided the Open-Group Spec. I had read that it was available from them in the NFS3 RFC, I just figured I would need to pay much $$$ to actually get it :) -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
At 1:36 PM -0800 11/25/99, C. Stephen Gunn wrote: >On Mon, Nov 22, 1999 at 02:07:58PM -0800, Alfred Perlstein wrote: >> > Does NetBSD have a working rpc.lockd... that would make this much easier. >> >> at a glance at http://cvsweb.netbsd.org/... no. >I'm fairly certain that rpc.lockd is included with Darwin from Apple, >I've not closely compared it to what we have in -STABLE or -CURRENT >to see if it actually works. It doesn't, sorry... if someone gets a *BSD version of NFS locking operating I'd help see it into Darwin. BTW, is anyone working on NFS Version 4 for BSD? Rick? -- Conrad Minshall ... [EMAIL PROTECTED] ... 408 974-2749 Apple Computer ... Mac OS X Core Operating Systems ... NFS/UDF/etc Alternative email address: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
Alfred Perlstein wrote: > > On Mon, 22 Nov 1999, David E. Cross wrote: > > > Ok... I have *had* it with the meta, but not really, lockd. Are there any > > kernel issues with correctly implimenting rpc.lockd? I can't help with the code, but put me down as both very interested, and willing to help test. We have a mixed sun, netapp and freebsd nfs environment, and I have some freebsd machines that I can sacrifice to the cause. We are pretty desperate for working NFS file locking at work as well. FWIW, we tried the linux version of lockd on linux and not only did it suck potatoes, it was frequently the cause of that server crashing. I would stay completely away from it. Thanks, Doug -- "Welcome to the desert of the real." - Laurence Fishburne as Morpheus, "The Matrix" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Mon, Nov 22, 1999 at 02:07:58PM -0800, Alfred Perlstein wrote: > On Mon, 22 Nov 1999, David E. Cross wrote: > > > Does NetBSD have a working rpc.lockd... that would make this much easier. > > at a glance at http://cvsweb.netbsd.org/... no. > > Linux may have one, a temporary GPL'd port would be interesting perhaps. I'm fairly certain that rpc.lockd is included with Darwin from Apple, I've not closely compared it to what we have in -STABLE or -CURRENT to see if it actually works. Just thought I'd mention it. - Steve -- C. Stephen Gunn URL: http://www.waterspout.com/ WaterSpout Communications, Inc.Email: [EMAIL PROTECTED] 427 North 6th Street Phone: +1 765.742.6628 Lafayette, IN 47901 Fax: +1 765.742.0646 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: wacky rpc.lockd idea...
On Tue, 23 Nov 1999, Ronald G. Minnich wrote: :On Mon, 22 Nov 1999, David E. Cross wrote: :> I've noticed about 99% of the panics on our machines are the result of NFS, :> more often than not it is the result of a backing store file being blown :> away underneath the client. ie. person editing a file on one machine, :> compiling and running on a second, then removing the binary on the first :> machine. If we had a working lock manager could we not have the kernel open :> a shared lock on anything it had in backing store, would that not assure that :> files didn't go poof in the night? : :I think you're really proposing to add state to NFS, but add it via a :'back door', the lockd. I think this is not as good an idea as getting :coda or intermezzo working -- for the latter, www.inter-mezzo.org : :nfs is just plain broken for this sort of thing, and has been forever. I'm :not sure you want to start grafting on fixes of this sort. How would this be different that what Sun has done? They designed NFS stateless (do to stupid utopian visions of 0 latency infinite bandwidth networks), and then added state via rpc.lockd to try and fix their silly design flaw. Sure, locking still has problems due to race conditions and deadlocks, but it beats what you have without it. Jamie Bowden -- If we've got to fight over grep, sign me up. But boggle can go. -Ted Faber (on Hasbro's request for removal of /usr/games/boggle) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
> : Ok... I have *had* it with the meta, but not really, lockd. Are there any > : kernel issues with correctly implimenting rpc.lockd?How can I take a > : filehandle and map it into a filename, with path, so I may open it and lock > : it on the server? Are there any protocol specs? I downloaded the RFC for > : version 4 nlm (which we do not supoprt at *all*), but it only lists diffs to > : the version 3 spec, which I cannot find, and the source is not a whole lot > : of help on this issue. > > One area that Solbourne had lots and lots of problems with years ago > when it tried to implenent rpc.lockd was that Sun, at the time, has 5! > incompatible versions that had to be interoperated with. Don't know > if things have changed in the ensuing years or not... Not really, no. Insofar as I know, the only distributed open source lock manager that might ever have a chance of being usable is the one the GFS guys are working on now, and naturally that will be tied to GFS, etc... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
In message <[EMAIL PROTECTED]> "David E. Cross" writes: : Ok... I have *had* it with the meta, but not really, lockd. Are there any : kernel issues with correctly implimenting rpc.lockd?How can I take a : filehandle and map it into a filename, with path, so I may open it and lock : it on the server? Are there any protocol specs? I downloaded the RFC for : version 4 nlm (which we do not supoprt at *all*), but it only lists diffs to : the version 3 spec, which I cannot find, and the source is not a whole lot : of help on this issue. One area that Solbourne had lots and lots of problems with years ago when it tried to implenent rpc.lockd was that Sun, at the time, has 5! incompatible versions that had to be interoperated with. Don't know if things have changed in the ensuing years or not... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: wacky rpc.lockd idea...
On Mon, 22 Nov 1999, David E. Cross wrote: > I've noticed about 99% of the panics on our machines are the result of NFS, > more often than not it is the result of a backing store file being blown > away underneath the client. ie. person editing a file on one machine, > compiling and running on a second, then removing the binary on the first > machine. If we had a working lock manager could we not have the kernel open > a shared lock on anything it had in backing store, would that not assure that > files didn't go poof in the night? I think you're really proposing to add state to NFS, but add it via a 'back door', the lockd. I think this is not as good an idea as getting coda or intermezzo working -- for the latter, www.inter-mezzo.org nfs is just plain broken for this sort of thing, and has been forever. I'm not sure you want to start grafting on fixes of this sort. ron To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: wacky rpc.lockd idea...
> That's really up to the server lockd/nfsd implementation, but considering > that more likely than not the server's lockd will have an open reference > to the file until the lock is gone the answer is probably yes. Hmm... I wold think even without having the file "open" a lock would be enough. Seems kinda pointless of a lock if it doesn't protect something as trivial as an rm ;) > How this would break semantics (auto-locking executables over NFS) is > another matter, I would make sure it stays as an option. As long as you make it a shared lock, read-only, I think those are always guaranteed to suceed, and to never affect other locks. Perhaps I am wrong in this. > Another issue is that it's possible that an in kernel implementation of > lockd may not follow those semantics so that even if locks are held on > the executeable, it may still disapear. It would most certainly be > broken behaviour, but I think NFS owns the arena on broken behavour. :) > > I think that nfs_bio ought to handle the loss of backing store a bit > more gracefully, kill -9 wouldn't be unreasonable in such circumstances. I agree that nfs_bio should be more tolerant of these types of faults. However a process in the middle of a page-in is not killable, and leaving it stuck in disk-wait is also not a viable option IMO. -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: wacky rpc.lockd idea...
On Mon, 22 Nov 1999, David E. Cross wrote: > I've noticed about 99% of the panics on our machines are the result of NFS, > more often than not it is the result of a backing store file being blown > away underneath the client. ie. person editing a file on one machine, > compiling and running on a second, then removing the binary on the first > machine. If we had a working lock manager could we not have the kernel open > a shared lock on anything it had in backing store, would that not assure that > files didn't go poof in the night? That's really up to the server lockd/nfsd implementation, but considering that more likely than not the server's lockd will have an open reference to the file until the lock is gone the answer is probably yes. How this would break semantics (auto-locking executables over NFS) is another matter, I would make sure it stays as an option. Another issue is that it's possible that an in kernel implementation of lockd may not follow those semantics so that even if locks are held on the executeable, it may still disapear. It would most certainly be broken behaviour, but I think NFS owns the arena on broken behavour. :) I think that nfs_bio ought to handle the loss of backing store a bit more gracefully, kill -9 wouldn't be unreasonable in such circumstances. -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
wacky rpc.lockd idea...
I've noticed about 99% of the panics on our machines are the result of NFS, more often than not it is the result of a backing store file being blown away underneath the client. ie. person editing a file on one machine, compiling and running on a second, then removing the binary on the first machine. If we had a working lock manager could we not have the kernel open a shared lock on anything it had in backing store, would that not assure that files didn't go poof in the night? -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
> Linux may have one, a temporary GPL'd port would be interesting perhaps. "There is nothing as permanent as a temporary decision." No thanks :) -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Mon, 22 Nov 1999, David E. Cross wrote: > Does NetBSD have a working rpc.lockd... that would make this much easier. at a glance at http://cvsweb.netbsd.org/... no. Linux may have one, a temporary GPL'd port would be interesting perhaps. -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
Does NetBSD have a working rpc.lockd... that would make this much easier. -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Mon, 22 Nov 1999, Ronald G. Minnich wrote: > Actually I wrote a system call for opening a file given a file handle for > freebsd a while back (oh, gee, has it really been 5 years ...), as part of > mnfs i'll try to find it. You don't need to map it to a filename to > make it go. i forgot to include that in my last email, the syscall is availble in -current for some time now. I brought fhopen, fhstat, and fhstatfs all over from NetBSD several months ago. -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
Actually I wrote a system call for opening a file given a file handle for freebsd a while back (oh, gee, has it really been 5 years ...), as part of mnfs i'll try to find it. You don't need to map it to a filename to make it go. ron To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Ok, that's it, enough is enough! (rpc.lockd)
On Mon, 22 Nov 1999, David E. Cross wrote: > Ok... I have *had* it with the meta, but not really, lockd. Are there any > kernel issues with correctly implimenting rpc.lockd?How can I take a > filehandle and map it into a filename, with path, so I may open it and lock > it on the server? Are there any protocol specs? I downloaded the RFC for > version 4 nlm (which we do not supoprt at *all*), but it only lists diffs to > the version 3 spec, which I cannot find, and the source is not a whole lot > of help on this issue. here's a url to some of the stuff I have in the works before work utterly consumed me: http://www.freebsd.org/~alfred/misc-patches/ (lockd.diff) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Ok, that's it, enough is enough! (rpc.lockd)
Ok... I have *had* it with the meta, but not really, lockd. Are there any kernel issues with correctly implimenting rpc.lockd?How can I take a filehandle and map it into a filename, with path, so I may open it and lock it on the server? Are there any protocol specs? I downloaded the RFC for version 4 nlm (which we do not supoprt at *all*), but it only lists diffs to the version 3 spec, which I cannot find, and the source is not a whole lot of help on this issue. -- David Cross | email: [EMAIL PROTECTED] Acting Lab Director | NYSLP: FREEBSD Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
