Re: flock() and sendmail
On Fri, Nov 17, 2006 at 07:03:00PM +, John Kelly wrote: > On Fri, 17 Nov 2006 19:54:13 +0100, Kurt Roeckx <[EMAIL PROTECTED]> > wrote: > > >I actually see no good reason to want to use flock() over fcntl(). > > > Maybe because the fcntl() > > >interface follows the completely stupid semantics of System V and > >IEEE Std 1003.1-1988 (``POSIX.1'') that require that all locks associated > >with a file for a given process are removed when any file descriptor for > >that file is closed by that process. This semantic means that applica- > >tions must be aware of any files that a subroutine library may access. > >For example if an application for updating the password file locks the > >password file database while making the update, and then calls > >getpwnam(3) to retrieve a record, the lock will be lost because > >getpwnam(3) opens, reads, and closes the password database. The database I think this is a rather bad example. If you lock a file and then use something other than read() or write() that might access it, you're very likely having problems. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006 20:09:54 +0100, Michael Banck <[EMAIL PROTECTED]> wrote: >On Fri, Nov 17, 2006 at 06:12:44PM +, John Kelly wrote: >> Heil Hitler! >QED. Godwin's law only applies when the comparison is unfair.
Re: flock() and sendmail
On Fri, Nov 17, 2006 at 06:12:44PM +, John Kelly wrote: > Heil Hitler! QED. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006 19:54:13 +0100, Kurt Roeckx <[EMAIL PROTECTED]> wrote: >I actually see no good reason to want to use flock() over fcntl(). Maybe because the fcntl() >interface follows the completely stupid semantics of System V and >IEEE Std 1003.1-1988 (``POSIX.1'') that require that all locks associated >with a file for a given process are removed when any file descriptor for >that file is closed by that process. This semantic means that applica- >tions must be aware of any files that a subroutine library may access. >For example if an application for updating the password file locks the >password file database while making the update, and then calls >getpwnam(3) to retrieve a record, the lock will be lost because >getpwnam(3) opens, reads, and closes the password database. The database >close will release all locks that the process has associated with the >database, even if the library routine never requested a lock on the data- >base. Another minor semantic problem with this interface is that locks >are not inherited by a child process created using the fork(2) function. >The flock(2) interface has much more rational last close semantics and >allows locks to be inherited by child processes. Flock(2) is recommended >for applications that want to ensure the integrity of their locks when >using library routines or wish to pass locks to their children. http://leaf.dragonflybsd.org/cgi/web-man?command=fcntl§ion=2
Re: flock() and sendmail
On Fri, Nov 17, 2006 at 06:27:13PM +, John Kelly wrote: > On Fri, 17 Nov 2006 19:09:33 +0100, Kurt Roeckx <[EMAIL PROTECTED]> > wrote: > > >Anyway, from the linux/Documentation/locks.txt file: > >1.2.1 Typical Problems - Sendmail > >- > >Because sendmail was unable to use the old flock() emulation > > I believe flock() *emulation* is no longer used in 2.6 kernels. flock was changed to not use emulation in 1.3.x and the old flock emulation has been removed in 2.1.x, according to the same doc. > >installations use fcntl() instead of flock(). This is true of Slackware 3.0 > >for example. This gave rise to some other subtle problems if sendmail was > >configured to rebuild the alias file. Sendmail tried to lock the aliases.dir > >file with fcntl() at the same time as the GDBM routines tried to lock this > >file with flock(). With pre 1.3.96 kernels this could result in deadlocks > >that, > >over time, or under a very heavy mail load, would eventually cause the kernel > >to lock solid with deadlocked processes. > > Then I have to wonder why sendmail is still configured to use fcntl() > when running on linux. Sounds like the modern kernel implementation > of flock() would be better. I have to wonder why sendmail needs to use fcntl() or flock() at all to lock the database. I don't see why flock() would be better, it's just different. fcntl() is a standardised and can do more than flock(). I actually see no good reason to want to use flock() over fcntl(). Also note that this deadlock was caused by the flock() emulation. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006, John Kelly wrote: Then I have to wonder why sendmail is still configured to use fcntl() when running on linux. Sounds like the modern kernel implementation of flock() would be better. The most general solution wins ... and that is fcntl() ! afaict, flock() *still* does not work over NFS Just because you may not need that feature does not mean I can screw up the maildirs, databases, etc of those who happen to use NFS (and NFS v4 is looking *very* nice...) -- Rick Nelson <|Rain|> with sane code, maybe I could figure out the renderer :) rain: I'd probably be the one writing the renderer <|Rain|> well, er, uh -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006 19:09:33 +0100, Kurt Roeckx <[EMAIL PROTECTED]> wrote: >Anyway, from the linux/Documentation/locks.txt file: >1.2.1 Typical Problems - Sendmail >- >Because sendmail was unable to use the old flock() emulation I believe flock() *emulation* is no longer used in 2.6 kernels. >installations use fcntl() instead of flock(). This is true of Slackware 3.0 >for example. This gave rise to some other subtle problems if sendmail was >configured to rebuild the alias file. Sendmail tried to lock the aliases.dir >file with fcntl() at the same time as the GDBM routines tried to lock this >file with flock(). With pre 1.3.96 kernels this could result in deadlocks that, >over time, or under a very heavy mail load, would eventually cause the kernel >to lock solid with deadlocked processes. Then I have to wonder why sendmail is still configured to use fcntl() when running on linux. Sounds like the modern kernel implementation of flock() would be better.
Re: flock() and sendmail
On Fri, 17 Nov 2006 18:33:10 +0100, Michael Banck <[EMAIL PROTECTED]> wrote: >On Fri, Nov 17, 2006 at 03:37:56PM +, John Kelly wrote: >> I'm discussing flock() with the debain sendmail package and the linux >> 2.6 kernel. >> >> Or does that annoy you too? > >To be honest, I don't see why this should concern all other Debian >developers, yes. I didn't know "all other Debian developers" had appointed you as the topic Gestapo. Heil Hitler!
Re: flock() and sendmail
On Thu, Nov 16, 2006 at 11:24:34AM -0800, Richard A Nelson wrote: > On Thu, 16 Nov 2006, John Kelly wrote: > > >I don't need NFS with sendmail. Surely flock() is not *still* broken > >in 2.6 kernels? > > I doubt that flock is *still* broken - that was quite some time ago... >From the flock manpage: NOTES flock(2) does not lock files over NFS. Use fcntl(2) instead: that does work over NFS, given a sufficiently recent version of Linux and a server which supports locking. The reason flock() doesn't work over NFS probably has to do with that flock() and fcntl() locks don't interact with each other. > >** NOTE: Override HASFLOCK as you will but, as of 1.99.6, mixed-style > >** file locking is no longer allowed. In particular, make sure > >** your DBM library and sendmail are both using either flock(2) > >** *or* fcntl(2) file locking, but not both. > > This, indeed, is the important part ! DB has changed alot, and now > has its own locking system, and whatever it uses to lock the entire > DB is what should be used by its callers. > > Has DB changed from fcntl to flock ? > Are you trying to access the sendmail databases from another > program that has differing locking conventions ? > > You fail to provide any hints as to on why fcntl which is used instead > of flock is causing you grief. If sendmail is using libdb, I don't see why sendmail would need to use fcntl() or flock() on any libdb file. It's an internal problem of libdb how it should do it's locking before it updates the database. However, sendmail might want to make sure that it's only updating it once, and can do this is various ways. As long as sendmail properly uses libdb, I don't see why there should be any problem at all. Anyway, from the linux/Documentation/locks.txt file: 1.2.1 Typical Problems - Sendmail - Because sendmail was unable to use the old flock() emulation, many sendmail installations use fcntl() instead of flock(). This is true of Slackware 3.0 for example. This gave rise to some other subtle problems if sendmail was configured to rebuild the alias file. Sendmail tried to lock the aliases.dir file with fcntl() at the same time as the GDBM routines tried to lock this file with flock(). With pre 1.3.96 kernels this could result in deadlocks that, over time, or under a very heavy mail load, would eventually cause the kernel to lock solid with deadlocked processes. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, Nov 17, 2006 at 03:37:56PM +, John Kelly wrote: > I'm discussing flock() with the debain sendmail package and the linux > 2.6 kernel. > > Or does that annoy you too? To be honest, I don't see why this should concern all other Debian developers, yes. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006 15:42:21 +0100, Michael Banck <[EMAIL PROTECTED]> wrote: >On Fri, Nov 17, 2006 at 01:02:15PM +, John Kelly wrote: >> On Fri, 17 Nov 2006 11:14:21 +, Roger Leigh >> <[EMAIL PROTECTED]> wrote: >> >> >It's almost always a bad idea to use flock() instead of fcntl(). >> >fnctl() locking is effectively deprecating flock() >> >> I heard it was the other way around. Please explain ... > >This list is not for discussing different Un*x locking implementations, >please find advise on which you prefer elsewhere. Why are you telling me? Roger raised that issue. I'm discussing flock() with the debain sendmail package and the linux 2.6 kernel. Or does that annoy you too?
Re: flock() and sendmail
On Fri, Nov 17, 2006 at 01:02:15PM +, John Kelly wrote: > On Fri, 17 Nov 2006 11:14:21 +, Roger Leigh > <[EMAIL PROTECTED]> wrote: > > >It's almost always a bad idea to use flock() instead of fcntl(). > >fnctl() locking is effectively deprecating flock() > > I heard it was the other way around. Please explain ... This list is not for discussing different Un*x locking implementations, please find advise on which you prefer elsewhere. thanks, Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: flock() and sendmail
On Fri, 17 Nov 2006 11:14:21 +, Roger Leigh <[EMAIL PROTECTED]> wrote: >It's almost always a bad idea to use flock() instead of fcntl(). >fnctl() locking is effectively deprecating flock() I heard it was the other way around. Please explain ... >If you look at SUSv3/POSIX, you'll see that fcntl()/lockf() locking is >the only standardised form of locking. flock() isn't included in the >standard. I have no reason to limit my choices to "standards."
Re: flock() and sendmail
John Kelly <[EMAIL PROTECTED]> writes: > I need to make some local customizations, and it seemed like a good > idea, while I'm in there, to use flock() instead of fcntl(), if there > are no conflicts. It's almost always a bad idea to use flock() instead of fcntl(). fnctl() locking is effectively deprecating flock(), and you need a really good reason not to use it in favour of fcntl(). If you look at SUSv3/POSIX, you'll see that fcntl()/lockf() locking is the only standardised form of locking. flock() isn't included in the standard. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. pgpPS4cdUxCzU.pgp Description: PGP signature
Re: flock() and sendmail
On Thu, 16 Nov 2006 11:24:34 -0800 (PST), Richard A Nelson <[EMAIL PROTECTED]> wrote: >>** NOTE: Override HASFLOCK as you will but, as of 1.99.6, mixed-style >>** file locking is no longer allowed. In particular, make sure >>** your DBM library and sendmail are both using either flock(2) >>** *or* fcntl(2) file locking, but not both. > >This, indeed, is the important part ! DB has changed alot, and now >has its own locking system, and whatever it uses to lock the entire >DB is what should be used by its callers. >Has DB changed from fcntl to flock ? I don't know. >Are you trying to access the sendmail databases from another >program that has differing locking conventions ? Not yet. >You fail to provide any hints as to on why fcntl which is used instead >of flock is causing you grief. No grief yet. So far, I'm only studying your package to see what the possibilities are. I need to make some local customizations, and it seemed like a good idea, while I'm in there, to use flock() instead of fcntl(), if there are no conflicts. DBM is obsoleted by Berkeley DB if I understand correctly, so maybe the note about DBM won't matter to me. But there is still the question of how Berkeley DB locking interacts with sendmail's use of flock(), if at all.
Re: flock() and sendmail
On Thu, 16 Nov 2006, John Kelly wrote: I don't need NFS with sendmail. Surely flock() is not *still* broken in 2.6 kernels? I doubt that flock is *still* broken - that was quite some time ago... ** NOTE: Override HASFLOCK as you will but, as of 1.99.6, mixed-style ** file locking is no longer allowed. In particular, make sure ** your DBM library and sendmail are both using either flock(2) ** *or* fcntl(2) file locking, but not both. This, indeed, is the important part ! DB has changed alot, and now has its own locking system, and whatever it uses to lock the entire DB is what should be used by its callers. Has DB changed from fcntl to flock ? Are you trying to access the sendmail databases from another program that has differing locking conventions ? You fail to provide any hints as to on why fcntl which is used instead of flock is causing you grief. -- Rick Nelson anyone know if there is a version of dpkg for redhat? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
