Re: [Nmh-workers] Locking - which to use?
On Sun, 5 Feb 2012 15:38:08 -0800 Lyndon Nerenberg wrote - > > On 2012-02-05, at 3:13 PM, David Fellows wrote: > > > I find I have no need for /bin/mail so I have not installed a package = > that > > provides it. nmh, fetchmail, and ssmtp are sufficient for my needs. > > Okay, now you're just being a pain in the ass. So you didn't install = > binmail. This changes the API how? Stop wasting our time with this = > drivel.= Perhaps I did not make myself clear. Let me try again. I believe this thread is about configuration/build choices, not the API. The proposal was that the configuration and/or build and/or runtime determine in an as yet unspecified manner the locking mechansim in use for access to the system maildir and use it -- and that this behaviour be "unalterable". My request is that if this process in unable to determine what this mechanism is that it not fail to configure, build or run as the case may be, but do something reasonable. In particular, if there is no other program doing locking on maildir, nmh can pick its own locking mechanism. Alternatively, leave in an override option. I guess the logic would be something like: if there are ONE OR MORE programs locking system maildir then Assume they are playing nicely if determine their locking mechanism then use it else fail and seek human intervention fi else there is nobody here but me use a suitable locking mechanism, probably the same as for internal locking fi What I am asking for is the first test and last else clause. I admit I have no idea what the tests would be. Gentoo does have a basemail package that ensures that /var/spool/mail exists with what Gentoo believes are the correct properties. so testing for the presence or absence of /var/spool/mail doesn't help me. Dave F ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Lyndon wrote: > On 2012-02-05, at 4:04 PM, David Levine wrote: > > procmail can be configured to use whatever locking a user > > wants. Let's leave nmh that way, too. > > And again, an end user configuring their system into a > stupid state is not justification for nmh to follow suit. Shun fcntl, but when someone discards their /bin/mail that uses it: "stupid". And again, I'm not asking for nmh to be "stupid". I am all for sensible defaults. If someone wants to choose a different locking scheme for whatever reason, it doesn't bother me. If they can do it at runtime, even better. > How about submitting a patch set Your order has no effect on my priorities. > that makes nmh work reasonably under > cygwin. I'm not the only person who would rejoice. As a matter of fact, I have fixed some things. The build is reasonably clean. inc and folder operations seem to work. whatnow doesn't, maybe because of (v)fork. post doesn't, but I haven't looked at it even to determine if it's a nmh problem. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Valdis wrote: > On Sun, 05 Feb 2012 15:41:11 EST, David Levine said: > > > What if the user doesn't want to/can't use fcntl, and that's what > > /bin/mail uses, and the user removes it in favor of something else? > > If /bin/mail uses fcnt, but the user *can't* use fcntl, both the user or > the system should be taken out back and shot. A system where the > system-provided mail facility uses a locking scheme not available to the > user is just too broken to live. I agree. I never said that any of the locking schemes isn't available to the user. To try to save you a response (and I should know better by now): "can't use fcntl" could be for other reasons than "not available". I'm all for choosing more sensible defaults. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On Sun, 05 Feb 2012 15:41:11 EST, David Levine said: > What if the user doesn't want to/can't use fcntl, and that's what > /bin/mail uses, and the user removes it in favor of something else? If /bin/mail uses fcnt, but the user *can't* use fcntl, both the user or the system should be taken out back and shot. A system where the system-provided mail facility uses a locking scheme not available to the user is just too broken to live. And the user who chose such a broken system is just too stupid to live. (Sorry, am in a cranky mood today, I've spent far too much of my $DAYJOB lately fighting with too-broken-to-live software packages...) pgprUDrrjhHAN.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 4:04 PM, David Levine wrote: > Cygwin doesn't have a /bin/mail, either. I didn't bring it > up before because it obviously "doesn't count", so no need > to jump on it. And I don't use it for mail, but I don't want > to break nmh on it, either. For a system that doesn't have a binmail -- like windows -- your argument makes perfect sense. But your claim that 'I didn't install binmail' on a system that includes it, doesn't. > procmail can be configured to use whatever locking a user > wants. Let's leave nmh that way, too. And again, an end user configuring their system into a stupid state is not justification for nmh to follow suit. How about submitting a patch set that makes nmh work reasonably under cygwin. I'm not the only person who would rejoice. --lyndon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Personal attacks benefit no one. And certainly do not belong here. Cygwin doesn't have a /bin/mail, either. I didn't bring it up before because it obviously "doesn't count", so no need to jump on it. And I don't use it for mail, but I don't want to break nmh on it, either. procmail can be configured to use whatever locking a user wants. Let's leave nmh that way, too. I'm all for choosing more sensible defaults. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 3:13 PM, David Fellows wrote: > I find I have no need for /bin/mail so I have not installed a package that > provides it. nmh, fetchmail, and ssmtp are sufficient for my needs. Okay, now you're just being a pain in the ass. So you didn't install binmail. This changes the API how? Stop wasting our time with this drivel. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On Sun, 5 Feb 2012 13:25:43 -0800 Lyndon Nerenberg wrote - > > On 2012-02-05, at 1:23 PM, David Fellows wrote: > > > And some platforms (mine for example) do not have /bin/mail. And = > presumably=20 > > do not use the system mailbox. nmh should build nicely on such a = > system.=20 > > It could use its choice of locking for the system mailbox. > > Which platform is that?= Gentoo Linux. I find I have no need for /bin/mail so I have not installed a package that provides it. nmh, fetchmail, and ssmtp are sufficient for my needs. Dave F ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 1:23 PM, David Fellows wrote: > And some platforms (mine for example) do not have /bin/mail. And presumably > do not use the system mailbox. nmh should build nicely on such a system. > It could use its choice of locking for the system mailbox. Which platform is that? ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On Sun, 5 Feb 2012 11:33:45 -0800 Lyndon Nerenberg wrote - > > 1) System mailbox locking > > This must use the same scheme as the platform's /bin/mail and friends use. Th And some platforms (mine for example) do not have /bin/mail. And presumably do not use the system mailbox. nmh should build nicely on such a system. It could use its choice of locking for the system mailbox. Dave F ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 1:04 PM, David Levine wrote: > Then fcntl can't always be shunned. Given that a choice > must be made, let the user make it if they don't like the > nmh default. Shunned, not "not ever used." Obviously if the platform uses fcntl in /bin/mail and other system utilities then inc has to use the same. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Lyndon wrote: > On 2012-02-05, at 12:41 PM, David Levine wrote: > > > What if the user doesn't want to/can't use fcntl, and that's what > > /bin/mail uses, and the user removes it in favor of something else? > > What Valdis said. Then fcntl can't always be shunned. Given that a choice must be made, let the user make it if they don't like the nmh default. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 12:41 PM, David Levine wrote: > What if the user doesn't want to/can't use fcntl, and that's what > /bin/mail uses, and the user removes it in favor of something else? What Valdis said. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Valdis wrote: > On Sun, 05 Feb 2012 15:25:13 EST, David Levine said: > > > What if the user also uses an application that writes to the > > mail spool but uses a different locking scheme than /bin/mail? > > Then the other app is busticated, pure and simple. There's no really sane > choice for locking the mail spool than "exactly the same thing that the > system /bin/mail uses". What if the user doesn't want to/can't use fcntl, and that's what /bin/mail uses, and the user removes it in favor of something else? My point is that I don't see a problem leaving the knob here. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On Sun, 05 Feb 2012 15:25:13 EST, David Levine said: > What if the user also uses an application that writes to the > mail spool but uses a different locking scheme than /bin/mail? Then the other app is busticated, pure and simple. There's no really sane choice for locking the mail spool than "exactly the same thing that the system /bin/mail uses". pgpeFu3Ym2O5y.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
> 1) System mailbox locking > > This must use the same scheme as the platform's /bin/mail and friends > use. In some cases, that's fcntl. > The configure script needs to set the appropriate value based on > the platform it's being run on. configure can't do that unless it looks at all the applications that might write to the mail spool. > Configure also needs to determine > whether inc needs setgid on that platform. It already does. > The user should never have to override these, and I don't think we > should even provide a knob to let them to do so. What if the user also uses an application that writes to the mail spool but uses a different locking scheme than /bin/mail? David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
> >> configure should be able to set this on a per-platform basis. > > It can't because it depends on what other applications use. > > Fedora /bin/mail uses fcntl and Mutt recommends it. > procmail can use it. I haven't had any problems since I > starting using fcntl. I did see problems with dot locking, > I expect because other applications weren't using it. As Ken mentioned earlier, there are two types of locking at issue: 1) System mailbox locking This must use the same scheme as the platform's /bin/mail and friends use. The configure script needs to set the appropriate value based on the platform it's being run on. Configure also needs to determine whether inc needs setgid on that platform. The user should never have to override these, and I don't think we should even provide a knob to let them to do so. 2) nmh internal locking This can be whatever makes the most sense for nmh itself. Again, configure should have knowledge to set this to a reasonable platform-specific setting. My guess is it would default to flock, with platform specific overrides where it's known flock doesn't play nice. This one definitely needs a knob to allow the user to override the default. The reason to shun fcntl locking is this (from SUS v3): All locks associated with a file for a given process shall be removed when a file descriptor for that file is closed by that process or the process holding that file descriptor terminates. Locks are not inherited by a child process. This means that dup()ing a descriptor to a locked file, and then closing either of those fd's will kill *all* locks on that file, no matter which fd they were acquired on. This can wreak havoc when you have library routines that are doing file descriptor manipulations behind the scenes. And beware flock on Solaris, which is just a wrapper around fcntl. --lyndon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Lyndon wrote: > On 2012-02-05, at 8:49 AM, David Levine wrote: > > > Or to really > > play it safe, have configure determine which are available > > and use them all? > > At the same time? Sounds like a recipe for deadlock to me. And it won't > work correctly, if at all, on platforms that implement one or more of > fcntl/flock/lockf as wrappers around one of the others (e.g. Solaris). So it sounds like picking one default for each platform and allowing run-time selection is the way to go. > configure should be able to set this on a per-platform basis. It can't because it depends on what other applications use. > As for internal locking, it's a toss-up between flock and lockf. To me > the tie breaker is which of the two gets along best with network mounted > filesystems these days (and think AFP and CIFS in addition to NFS). > > And forget about fcntl -- the semantics are too broken for > consideration. If neither of flock/lockf are available, we can fall back > to dot locking. Fedora /bin/mail uses fcntl and Mutt recommends it. procmail can use it. I haven't had any problems since I starting using fcntl. I did see problems with dot locking, I expect because other applications weren't using it. David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Lyndon Nerenberg writes: >As for internal locking, it's a toss-up between flock and lockf. To >me the tie breaker is which of the two gets along best with network >mounted filesystems these days (and think AFP and CIFS in addition >to NFS). In case anyone thinks this is hypothetical, it's not. Both my home directory and my $MAIL are NFS mounted. I have no strong views on which locking mechanism is used, so long as it doesn't break for my setup. Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
On 2012-02-05, at 8:49 AM, David Levine wrote: > Or to really > play it safe, have configure determine which are available > and use them all? At the same time? Sounds like a recipe for deadlock to me. And it won't work correctly, if at all, on platforms that implement one or more of fcntl/flock/lockf as wrappers around one of the others (e.g. Solaris). configure should be able to set this on a per-platform basis. As for internal locking, it's a toss-up between flock and lockf. To me the tie breaker is which of the two gets along best with network mounted filesystems these days (and think AFP and CIFS in addition to NFS). And forget about fcntl — the semantics are too broken for consideration. If neither of flock/lockf are available, we can fall back to dot locking. --lyndon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Locking - which to use?
Ken wrote: > So Lyndon pointed out to me in private email that on BSD systems we > should be using flock for locking. (Specifically, the mail spool on > those systems uses flock). But this made me realize that locking in > nmh is a big mess. Not just in nmh. This is old but looks useful: https://bugzilla.mozilla.org/show_bug.cgi?id=239013#c9 > I think we can divide locking into two categories - the mail spool, and > everything else. "Everything else" includes annotations, the context, > sequences, and the MIME content cache. It occurs to me that it probably > doesn't matter which locking method we use for that as long as it's > consistent inside of nmh. I think that should be the "best" locking > available, which would be one of flock/lockf/fcntl. > > As for the mail spool ... well, I am wondering if that should be > runtime configurable. My thinking is that most people seem to be > using prebuilt packages and what makes sense on one system might > not make sense on another, so maybe runtime selection would be > better. Right now we default to dot locking unless you override > it, and that seems wrong to me. I agree. How about this: default to flock on FreeBSD, fcntl on Linux, and so on for the platforms we know. For platforms we don't know, use dot locking. Or to really play it safe, have configure determine which are available and use them all? David ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
[Nmh-workers] Locking - which to use?
So Lyndon pointed out to me in private email that on BSD systems we should be using flock for locking. (Specifically, the mail spool on those systems uses flock). But this made me realize that locking in nmh is a big mess. I think we can divide locking into two categories - the mail spool, and everything else. "Everything else" includes annotations, the context, sequences, and the MIME content cache. It occurs to me that it probably doesn't matter which locking method we use for that as long as it's consistent inside of nmh. I think that should be the "best" locking available, which would be one of flock/lockf/fcntl. As for the mail spool ... well, I am wondering if that should be runtime configurable. My thinking is that most people seem to be using prebuilt packages and what makes sense on one system might not make sense on another, so maybe runtime selection would be better. Right now we default to dot locking unless you override it, and that seems wrong to me. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
