Re: [HACKERS] On file locking
That same documentation mentions that locks acquired using flock() will *not* invoke the mandatory lock semantics even if on a file marked for it, so I guess flock() isn't implemented on top of fcntl() in Linux. They're not. And there's another difference between fcntl and flock in Linux: although fork(2) states that file locks are not inherited, locks made by flock are inherited to children and they keep the lock even when the parent process is killed with SIGKILL. Tested this. Just see man syscall, there exists both flock(2) and fcntl(2) -- Antti Haapala +358 50 369 3535 ICQ: #177673735 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
Tom Lane wrote: On HPUX 10.20, flock doesn't seem to exist (hasn't got a man page nor any mention in /usr/include). Correct. Still isn't there in later releases. lockf says All locks for a process are released upon the first close of the file, even if the process still has the file opened, and all locks held by a process are released when the process terminates. and When a file descriptor is closed, all locks on the file from the calling process are deleted, even if other file descriptors for that file (obtained through dup() or open(), for example) still exist. which seems to imply (but doesn't actually say) that HPUX keeps track of exactly which process took out the lock, even if the file is held open by multiple processes. Having done some testing today, I now understand what the standards are trying to say when they talk about locks being inherited. Or at least I think I understand: standards are tricky, locking is subtle, and I'm prepared to be corrected if I'm wrong! All of these lock functions succeed when the same process asks for a lock that it already has. That is: fcntl(fd, ...); fcntl(fd, ...); /* success -- no error returned */ For flock() only, the lock is inherited by a child process along with the file descriptor so the child can re-issue the flock() call and that will pass, too: flock(fd, ...); pid = fork(); if (pid == 0) flock(fd, ...); /* success -- no error returned */ For fcntl() and lockf() the locks are not inherited, and the call in a child fails: fcntl(fd, ...); pid = fork(); if (pid == 0) fcntl(fd, ...); /* will fail and return -1 */ In no case does just closing the file descriptor in the child lose the parent's lock. I rationalise this as follows: 1. flock() is using a last close semantic, so closing the file descriptor is documented not to lose the lock 2. lockf() and fcntl() use a first close, but because the locks are not inherited by the child process the child can't unlock them This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. I conjecture that Tom was looking for a facility to lock a file and have it stay locked if the postmaster or any child process was still running. flock() fits the bill, but it's not portable everywhere. One additional warning: this stuff *is* potentially filesystem dependent, per the source code I looked at, which would call filesystem specific routines. I tested with HP-UX 11.00 (VxFS), NetBSD (FFS) and Linux (ext3). I've put the rough and ready test code up for FTP, if anyone wants to check my working: ftp://ftp.nemeton.com.au/pub/pgsql/ Limitations in the testing: I only used whole file locking (no byte ranges) and didn't prove that a lock taken by flock() is still held after a child calls close() as it is documented to be. Regards, Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
Giles Lean [EMAIL PROTECTED] writes: Boring reference material follows. Couldn't help noticing that you omitted HPUX ;-) On HPUX 10.20, flock doesn't seem to exist (hasn't got a man page nor any mention in /usr/include). lockf says All locks for a process are released upon the first close of the file, even if the process still has the file opened, and all locks held by a process are released when the process terminates. and When a file descriptor is closed, all locks on the file from the calling process are deleted, even if other file descriptors for that file (obtained through dup() or open(), for example) still exist. which seems to imply (but doesn't actually say) that HPUX keeps track of exactly which process took out the lock, even if the file is held open by multiple processes. This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] On file locking
On Sun, 2 Feb 2003, Tom Lane wrote: This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. Definitely. I wonder about the NetBSD manpage quotes in the post you followed up to, given that last time I checked flock() was implmented, in the kernel, using fcntl(). Either that's changed, or the manpages are unclear or lying. This has been my experience in the past; locking semantics are subtle and unclear enough that you really need to test for exactly what you want at build time on every system, and you've got to do this testing on the filesystem you intend to put the locks on. (So you don't, e.g., test a local filesystem but end up with data on an NFS filesystem with different locking semantics.) That's what procmail does. Given this, I'm not even sure the whole idea is worth persuing. (Though I guess I should find out what NetBSD is really doing, and fix the manual pages correspond to reality.) cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] On file locking
Curt Sampson [EMAIL PROTECTED] wrote: On Sun, 2 Feb 2003, Tom Lane wrote: This all doesn't look good for using file locks in the way I had in mind :-( ... but considering that all these man pages seem pretty vague, maybe some direct experimentation is called for. Definitely. I wonder about the NetBSD manpage quotes in the post you followed up to, given that last time I checked flock() was implmented, in the kernel, using fcntl(). Either that's changed, or the manpages are unclear or lying. Using the same kernel code != same semantics. I think the NetBSD manual pages are trying to say that it's safe to have lockf(), fcntl(), and flock() locking playing together. That needn't be the case on all operating systems and the standards don't require it. This has been my experience in the past; locking semantics are subtle and unclear enough that you really need to test for exactly what you want at build time on every system, and you've got to do this testing on the filesystem you intend to put the locks on. What he said ... Giles ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
Curt Sampson wrote: On Fri, 31 Jan 2003, Shridhar Daithankar[EMAIL PROTECTED] wrote: Besides file locking is implemented using setgid bit on most unices. And everybody is free to do what he/she thinks right with it. I don't believe it's implemented with the setgid bit on most Unices. As I recall, it's certainly not on Xenix, SCO Unix, any of the BSDs, Linux, SunOS, Solaris, and Tru64 Unix. (I'm talking about the flock system call, here.) Linux, at least, supports mandatory file locks. The Linux kernel documentation mentions that you're supposed to use fcntl() or lockf() (the latter being a library wrapper around the former) to actually lock the file but, when those operations are applied to a file that has the setgid bit set but without the group execute bit set, the kernel enforces it as a mandatory lock. That means that operations like open(), read(), and write() initiated by other processes on the same file will block (or return EAGAIN, if O_NONBLOCK was used to open it) if that's what the lock on the file calls for. That same documentation mentions that locks acquired using flock() will *not* invoke the mandatory lock semantics even if on a file marked for it, so I guess flock() isn't implemented on top of fcntl() in Linux. So if we wanted to make use of mandatory locks, we'd have to refrain from using flock(). -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] On file locking
Tom Lane wrote: Kevin Brown [EMAIL PROTECTED] writes: So if we wanted to make use of mandatory locks, we'd have to refrain from using flock(). We have no need for mandatory locks; the advisory style will do fine. This is true because we have no desire to interoperate with any non-Postgres code ... everyone else is supposed to stay the heck out of $PGDATA. True. But, of course, mandatory locks could be used to *make* everyone else stay out of $PGDATA. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] On file locking
On Fri, 31 Jan 2003, Tom Lane wrote: Antti Haapala [EMAIL PROTECTED] writes: And from SunOS 5.8 flock Locks are on files, not file descriptors. That is, file descriptors duplicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock. Locks are not inherited by a child process. That seems self-contradictory. Yes. I note that in NetBSD, that paragraph of the manual page is identical except that the last sentence has been removed. At any rate, it seems to me highly unlikely that, since the child has the *same* descriptor as the parent had, that the lock would disappear. The other option would be that the lock belongs to the process, in which case one would think that a child doing an unlock should not affect the parent, because it's a different process cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] On file locking
Curt Sampson [EMAIL PROTECTED] wrote: At any rate, it seems to me highly unlikely that, since the child has the *same* descriptor as the parent had, that the lock would disappear. It depends on the lock function. After fork(): o with flock() the lock continues to be held, but will be unlocked if any child process explicitly unlocks it o with fcntl() the lock is not inherited in the child o with lockf() the standards and manual pages don't say Boring reference material follows. flock = From the NetBSD manual page: NOTES Locks are on files, not file descriptors. That is, file descriptors du- plicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock. The Red Hat Linux 8.0 manual page has similar wording. (No standards to check here -- flock() is not standardised in POSIX, X/Open, Single Unix Standard, ...) fcntl = The NetBSD manual page notes that these locks are not inherited by child processes: Another minor semantic problem with this interface is that locks are not inherited by a child process created using the fork(2) function. Ditto the Single Unix Standard versions 2 and 3. lockf() === The standards and manual pages that I've checked don't discuss fork() in relation to lockf(), which seems a peculiar ommission and makes me suspect that behaviour has varied historically. In practice I would expect lockf() semantics to be the same as fcntl(). Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] On file locking
But this only wins if a child process inheriting an open file also inherits copies of any locks held by the parent. If not, then the issue is moot. Anybody have any idea if file locks work that way? Is it portable?? From RedHat 8.0 manages fork(2): SYNOPSIS #include sys/types.h #include unistd.h pid_t fork(void); DESCRIPTION fork creates a child process that differs from the parent process only in its PID and PPID, and in the fact that resource utilizations are set to 0. File locks and pending signals are not inherited. ^^ ^^ And from SunOS 5.8 flock Locks are on files, not file descriptors. That is, file descriptors duplicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock. Locks are not inherited by a child process. If I understand correctly it says that if parent dies, file is unlocked no matter if there's children still running? -- Antti Haapala ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] On file locking
Antti Haapala [EMAIL PROTECTED] writes: And from SunOS 5.8 flock Locks are on files, not file descriptors. That is, file descriptors duplicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock. Locks are not inherited by a child process. That seems self-contradictory. If the fork results in multiple references to the open file, then I should think that if the parent dies but the child still holds the file open, then the lock still exists. Seems that some experimentation is called for ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] On file locking
On Fri, 31 Jan 2003, Shridhar Daithankar[EMAIL PROTECTED] wrote: Besides file locking is implemented using setgid bit on most unices. And everybody is free to do what he/she thinks right with it. I don't believe it's implemented with the setgid bit on most Unices. As I recall, it's certainly not on Xenix, SCO Unix, any of the BSDs, Linux, SunOS, Solaris, and Tru64 Unix. (I'm talking about the flock system call, here.) cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] On file locking
I've been looking at the PID file creation mechanism we currently use. It goes through a loop in an attempt to create the PID file, and if one is there it attempts to remove it if the PID it contains no longer exists (there are checks for shared memory usage as well). This could be cleaned up rather dramatically if we were to use one of the file locking primitives supplied by the OS to grab an exclusive lock on the file, and the upside is that, when the locking code is used, the postmaster would *know* whether or not there's another postmaster running, but the price for that is that we'd have to eat a file descriptor (closing the file means losing the lock), and we'd still have to retain the old code anyway in the event that there is no suitable file locking mechanism to use on the platform in question. The first question for the group is: is it worth doing that? The second question for the group is: if we do indeed decide to do file locking in that manner, what *other* applications of the OS-level file locking mechanism will we have? Some of them allow you to lock sections of a file, for instance, while others apply a lock on the entire file. It's not clear to me that the former will be available on all the platforms we're interested in, so locking the entire file is probably the only thing we can really count on (and keep in mind that even if an API to lock sections of a file is available, it may well be that it's implemented by locking the entire file anyway). What I had in mind was implementation of a file locking function that would take a file descriptor and a file range. If the underlying OS mechanism supported it, it would lock that range. The interesting case is when the underlying OS mechanism did *not* support it. Would it be more useful in that case to return an error indication? Would it be more useful to simply lock the entire file? If no underlying file locking mechanism is available, it seems obvious to me that the function would have to always return an error. Thoughts? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] On file locking
Mmy problem is freebsd getting totally loaded at which point it sends kills to various processes. This sometime seems to end up with several actual postmasters running, and none of them working. Better existing process detection would help that greatly I'm sure. Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Kevin Brown Sent: Friday, 31 January 2003 11:24 AM To: PostgreSQL Development Subject: [HACKERS] On file locking I've been looking at the PID file creation mechanism we currently use. It goes through a loop in an attempt to create the PID file, and if one is there it attempts to remove it if the PID it contains no longer exists (there are checks for shared memory usage as well). This could be cleaned up rather dramatically if we were to use one of the file locking primitives supplied by the OS to grab an exclusive lock on the file, and the upside is that, when the locking code is used, the postmaster would *know* whether or not there's another postmaster running, but the price for that is that we'd have to eat a file descriptor (closing the file means losing the lock), and we'd still have to retain the old code anyway in the event that there is no suitable file locking mechanism to use on the platform in question. The first question for the group is: is it worth doing that? The second question for the group is: if we do indeed decide to do file locking in that manner, what *other* applications of the OS-level file locking mechanism will we have? Some of them allow you to lock sections of a file, for instance, while others apply a lock on the entire file. It's not clear to me that the former will be available on all the platforms we're interested in, so locking the entire file is probably the only thing we can really count on (and keep in mind that even if an API to lock sections of a file is available, it may well be that it's implemented by locking the entire file anyway). What I had in mind was implementation of a file locking function that would take a file descriptor and a file range. If the underlying OS mechanism supported it, it would lock that range. The interesting case is when the underlying OS mechanism did *not* support it. Would it be more useful in that case to return an error indication? Would it be more useful to simply lock the entire file? If no underlying file locking mechanism is available, it seems obvious to me that the function would have to always return an error. Thoughts? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] On file locking
file descriptor (closing the file means losing the lock), and we'd still have to retain the old code anyway in the event that there is no suitable file locking mechanism to use on the platform in question. What is the gain given the above statement? If what we currently do can cause issues (fail), then beefing it up where available may be useful -- but otherwise it's just additional code. -- Rod Taylor [EMAIL PROTECTED] PGP Key: http://www.rbt.ca/rbtpub.asc signature.asc Description: This is a digitally signed message part
Re: [HACKERS] On file locking
Kevin Brown [EMAIL PROTECTED] writes: This could be cleaned up rather dramatically if we were to use one of the file locking primitives supplied by the OS to grab an exclusive lock on the file, and the upside is that, when the locking code is used, the postmaster would *know* whether or not there's another postmaster running, but the price for that is that we'd have to eat a file descriptor (closing the file means losing the lock), Yeah, I was just thinking about that this morning. Eating one file descriptor in the postmaster is absolutely no problem --- the postmaster doesn't have all that many files open anyhow. What I was wondering was whether it was worth eating an FD for every backend process, by holding open the file inherited from the postmaster. If we did that, we would have a reliable way of detecting that the old postmaster died but left surviving child backends. (As I mentioned in a nearby flamefest, the existing interlock for this situation strikes me as mighty fragile.) But this only wins if a child process inheriting an open file also inherits copies of any locks held by the parent. If not, then the issue is moot. Anybody have any idea if file locks work that way? Is it portable?? The second question for the group is: if we do indeed decide to do file locking in that manner, what *other* applications of the OS-level file locking mechanism will we have? I can't see any use in partial-file locks for us, and would not want to design an internal API that expects them to work. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] On file locking
This could be cleaned up rather dramatically if we were to use one of the file locking primitives supplied by the OS to grab an exclusive lock on the file, ... ... The first question for the group is: is it worth doing that? In the past it has been proposed and declined -- there is some stuff in the archives. While it would be beneficial to installations using local data it would introduce new failure modes for installations using NFS. Regards, Giles ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
