Re: ctl_cyrusdb -r takes too long
Lawrence Greenfield wrote: --On Saturday, December 14, 2002 5:50 PM +0530 Jatin Nansi <[EMAIL PROTECTED]> wrote: [...] Will ctl_cyrusdb -r take 10 hours ? I dread the day when the system will need downtime again. This is on a decent 2 x P3 1 GHz with 100 GB scsi raid and 1 GB RAM, about 2000 users, 50 MB per user. I really need reboot times to be between 5 to 10 minutes, as against 10 hours. I will set the checkpoint period to 5 minutes as per the FAQ, but are there any other pointers ? The amount of time a Berkeley db recovery should take it proportional to the number of transactions since the second to last checkpoint. The more frequent the checkpoints and the less the trransactions, the faster recovery should go. 10 hours is an absurdly long time for the databases the size & activity Cyrus stores in Berkeley db. I would make sure that database checkpointing is happening regularly. Well yes, after this incident I have changed the checkpoint interval to 5 min. How many megabytes of log files do you have? Cyrus should automatically prune the log files down, so there should never be more than 20 megabytes or so in the db/ directory. Today the db directory size is about 16 MB. On the day of reporting, the size of db/ was 217 MB. The full size of config_dir was 650+ MB. So I guess that was the reason. Larry
Re: Aborting locker errors...
> This should be due to internal deadlocks in the berkeley db code. (This can > happen when two different processes are trying to upgrade read locks to > write locks or split a page.) Generally the bigger the database is the less > likely this is to happen, which is why you might see it only on your less > loaded server. > > It's one of those "if it isn't happening constantly, it isn't a problem" > deals. Given that it's for the deliver.db, which is used for duplicate delivery suppression and sieve, will an aborted locker result in sieve not correctly processing an email? Rob
Re: Updating /seen from concurrent sessions
--On Friday, December 13, 2002 12:52 AM -0500 Jay Levitt <[EMAIL PROTECTED]> wrote: I believe I've fixed this bug in CVS (did it a few days ago, actually) and it'll be in the next release. If I understand correctly, this fixes the flat-file seen implementation, but not the underlying problem, which is that updates to seen are deferred until the connection is closed. Am I following? If so, is there any chance of updating more frequently in a future release? OE is certainly a popular client, and it sounds like Mozilla has the same problem. I can duplicate this at will in OE with a skiplist seen db. I have no plans on changing how seen state is currently cached. We haven't had any complaints from our users and (now that we found the cyrusdb_flat problem) it doesn't seem like it is that huge an uissue for most other sites, either. The only time I can see this changing is if someone decides to abstract out all flags equally so that we can have shared seen or private deleted flags, etc. How flags are written back would be an obvious thing to add to such a project. Larry
Re: Sieve Script - two actions?
Whoops - Sorry folks
The script wasn't active.
Syntax below works just fine!
>> Original Message <<
On 18/12/2002, 14:36:34, Andrew Pealing <[EMAIL PROTECTED]> wrote
regarding Sieve Script - two actions?:
> What's the sieve syntax to perform two actions for a message?
> I want to keep and redirect mail.
> So I have:
> if address :is :all ["to", "cc", "bcc"] ["[EMAIL PROTECTED]"]
> {
> keep;
> redirect "[EMAIL PROTECTED]";
> }
> but the redirect doesn't seem to happen.
> What's the correct syntax?
> Many thanks
> Andrew Pealing
Re: ctl_cyrusdb -r takes too long
--On Saturday, December 14, 2002 5:50 PM +0530 Jatin Nansi <[EMAIL PROTECTED]> wrote: [...] Will ctl_cyrusdb -r take 10 hours ? I dread the day when the system will need downtime again. This is on a decent 2 x P3 1 GHz with 100 GB scsi raid and 1 GB RAM, about 2000 users, 50 MB per user. I really need reboot times to be between 5 to 10 minutes, as against 10 hours. I will set the checkpoint period to 5 minutes as per the FAQ, but are there any other pointers ? The amount of time a Berkeley db recovery should take it proportional to the number of transactions since the second to last checkpoint. The more frequent the checkpoints and the less the trransactions, the faster recovery should go. 10 hours is an absurdly long time for the databases the size & activity Cyrus stores in Berkeley db. I would make sure that database checkpointing is happening regularly. How many megabytes of log files do you have? Cyrus should automatically prune the log files down, so there should never be more than 20 megabytes or so in the db/ directory. Larry
Re: Aborting locker errors...
--On Wednesday, December 18, 2002 12:39 PM +1100 Rob Mueller <[EMAIL PROTECTED]> wrote: I'm just wondering if anyone know what this might be about. Seems to be a BDB issue and it only happens a few times a day, but I don't know if it's harmless or not. Interestingly it occurs on our less loaded server. Dec 17 20:31:41 www lmtpd[17675]: DBERROR db3: Aborting locker 8011ca86 We use skiplist for mailboxdb and seen state, flat for sub, and db3_nosync for deliver.db. This should be due to internal deadlocks in the berkeley db code. (This can happen when two different processes are trying to upgrade read locks to write locks or split a page.) Generally the bigger the database is the less likely this is to happen, which is why you might see it only on your less loaded server. It's one of those "if it isn't happening constantly, it isn't a problem" deals. Larry
Re: cyrus-2.2-cvs: virtualdomains and sendmail virtusertable (cyrusv2as local mailer)
Joe Rhett schrieb: I hope that documenting how best to configure sendmail for use with Cyrus 2.2 in virtdomain mode will be part of the documentation cleanup that preceeds the 2.2 release. If I were sure what "the best" approach was, I'd happily submit patches to the Cyrus documentation files describing it. But I keep thinking that someone somewhere surely knows of a better way than making changes to proto.m4 :-) It's already in the docs for 2.2. I am running a 2.2 branch installation. I cannot find appropriate documentation concerning virtual domain support and how to configure the MTA under the doc/ directory! Is the documentation available in cvs ? I will have to change some things in my sendmail installation to get sendmail working with domains in local-user names. What I have running right now is working but with the cost of loosing the ability of configuring catchall accounts in virtusertable which would end up in some virtusertable-entry like: @domain.tld[EMAIL PROTECTED] which actually is a recursion! Also aliases are no longer expanded because after ending virtusertable-processing there will not be local users without a @domain.tld part. An alias-entry like: [EMAIL PROTECTED][EMAIL PROTECTED], [EMAIL PROTECTED] is not possible at all! So what I will try to do is an extension of the proto.m4 to be able to finish up virtusertable with entries for which an alias-entry exists without @domain.tld ! If there are some hints in the documentation it would be great if you could point me somewhere --Christian--
Re: cyrus 2.2 status
--On Tuesday, December 17, 2002 4:24 PM -0800 Jonathan Marsden <[EMAIL PROTECTED]> wrote: | So the question becomes: what, if anything can non-CMU people do that | would help cause a release of 2.2 (or 2.1 with virtdomains in it??) to | happen sooner rather than later? Just a guess... send Ken some money. Amos
Aborting locker errors...
I'm just wondering if anyone know what this might be about. Seems to be a BDB issue and it only happens a few times a day, but I don't know if it's harmless or not. Interestingly it occurs on our less loaded server. Dec 17 20:31:41 www lmtpd[17675]: DBERROR db3: Aborting locker 8011ca86 We use skiplist for mailboxdb and seen state, flat for sub, and db3_nosync for deliver.db. Rob
Sieve Script - two actions?
What's the sieve syntax to perform two actions for a message?
I want to keep and redirect mail.
So I have:
if address :is :all ["to", "cc", "bcc"] ["[EMAIL PROTECTED]"]
{
keep;
redirect "[EMAIL PROTECTED]";
}
but the redirect doesn't seem to happen.
What's the correct syntax?
Many thanks
Andrew Pealing
Re: cyrus-2.2-cvs: virtualdomains and sendmail virtusertable (cyrusv2 as local mailer)
> I hope that documenting how best to configure sendmail for use with > Cyrus 2.2 in virtdomain mode will be part of the documentation cleanup > that preceeds the 2.2 release. If I were sure what "the best" > approach was, I'd happily submit patches to the Cyrus documentation > files describing it. But I keep thinking that someone somewhere > surely knows of a better way than making changes to proto.m4 :-) It's already in the docs for 2.2. -- Joe Rhett Chief Geek [EMAIL PROTECTED] ISite Services, Inc.
Re: [CVS] pidfile support
--On Tuesday, December 17, 2002 1:12 PM -0200 Henrique de Moraes Holschuh <[EMAIL PROTECTED]> wrote: [A] 1. stat pidfile 2. file exists? 2a (yes) - read pid from file, kill(pid,0) it 2a1 - kill says process exists? fail to start if so No no no. You must check and see if the PID that exists is sane - e.g. has the correct process name. Otherwise, you can get into a situation where you never restart at reboot, if some other process has that PID. -- Carson
Re: cyrus-2.2-cvs: virtualdomains and sendmail virtusertable (cyrusv2 as local mailer)
On 16 Dec 2002, Christian Schulte writes: > By the way: > +`R$=L < @ $=w . >$#_LOCAL_ $: @ $1`@'$2 special local names > What defines class L ? It seems like I do not have a so called class L! In my proto.m4 and sendmail.cf it says: # class L: names that should be delivered locally, even if we have a relay But the definition of that class is commented out: #CL root So most likely that line is not really used in either of our configurations. We don't use relay hosts; if we did, this would probably matter. > And what does the first @ character in $#_LOCAL_ $: @ $1`@'$2 stand > for ? I don't know for sure! I think that is a 'marker' that later sendmail.cf rules later notice and remove. It was there in the original rules I modified, so I left it there :-) Jonathan -- Jonathan Marsden| Internet: [EMAIL PROTECTED] | Making electronic 1252 Judson Street | Phone: +1 (909) 795-3877 | communications work Redlands, CA 92374 | Fax: +1 (909) 795-0327 | reliably for Christian USA | http://www.xc.org/jonathan| missions worldwide
Re: cyrus-2.2-cvs: virtualdomains and sendmail virtusertable (cyrusv2 as local mailer)
On 16 Dec 2002, Christian Schulte writes: >>> +`R$=L < @ $=w . > $#_LOCAL_ $: @ $1`@'$2 special local names >>> +R$+ < @ $=w . >$#_LOCAL_ $: $1`@'$2regular local name') > I think the two lines you added should look like > `R$=L < @ $=w . > $#_LOCAL_ $: @ $1 < @ $2 > special local names > R$+ < @ $=w . > $#_LOCAL_ $: $1 < @ $2 >regular local name') > don't they ? For me thinks work after chaning them this way ! I'm glad you've found something that works for you. I'd have to run more tests to know if that same change to proto.m4 would work for me. It could be that you have found a more generic approach than mine. What I sent is in active use on multiple smallish servers under Sendmail 8.12.5, and seems to be working fine so far. Suggestions from others about using mailertable and avoiding the need to hack proto.m4 also sound like they could be worth trying; it would make upgrading sendmail versions easier, for example, to use only the standard provided m4 files. I hope that documenting how best to configure sendmail for use with Cyrus 2.2 in virtdomain mode will be part of the documentation cleanup that preceeds the 2.2 release. If I were sure what "the best" approach was, I'd happily submit patches to the Cyrus documentation files describing it. But I keep thinking that someone somewhere surely knows of a better way than making changes to proto.m4 :-) Jonathan -- Jonathan Marsden| Internet: [EMAIL PROTECTED] | Making electronic 1252 Judson Street | Phone: +1 (909) 795-3877 | communications work Redlands, CA 92374 | Fax: +1 (909) 795-0327 | reliably for Christian USA | http://www.xc.org/jonathan| missions worldwide
Re: cyrus 2.2 status
On 13 Dec 2002, Jure Pecar writes: > On Thu, 12 Dec 2002 20:31:41 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: >> I addition to what Rob already mentioned, there needs to be more >> work done on documenting the virtdomain support and tying some >> loose ends. > Yes, virtdomains are actually the #1 thing i'm interested in cyrus > 2.2 ... I'm sure there are more people interested, so i think it > would be nice to provide either a stable, known working cvs branch > of 2.2 or a patch with a backport of virtdomains stuff to 2.1. I'm > willing to help here, just give me some directions. I would also very much like to see this happen. To get Red Hat 7.x RPMs of cyrus-imapd with virtdomain support, I grabbed the CVS for 2.2 as of late September and turned it ito RPMs and since then I have stuck with that. I'm about to resync with CVS again to pick up the recent security fixes. But I'd be more comfortable with using a somewhat supported (or at least officially labelled!) version of the codebase. The issue on when releases happen seems to me to be that CMU has its own priorities, and tends to stick to them. Which is absolutely fine, and probably the right thing for them to do. The result is that, even if a backport patch of the virtdomain code to 2.1.x happens, I'm not sure it would really help in the supportedness/officialness stakes, since CMU would not be using it :-) So it may be that until CMU finds an internal need for something that is in 2.2 but not in 2.1, there's little anyone else can do to get virtdomain support "more official" than it just being there in CVS. So the question becomes: what, if anything can non-CMU people do that would help cause a release of 2.2 (or 2.1 with virtdomains in it??) to happen sooner rather than later? Jonathan -- Jonathan Marsden| Internet: [EMAIL PROTECTED] | Making electronic 1252 Judson Street | Phone: +1 (909) 795-3877 | communications work Redlands, CA 92374 | Fax: +1 (909) 795-0327 | reliably for Christian USA | http://www.xc.org/jonathan| missions worldwide
Re: cyrus-2.2-cvs: virtualdomains and sendmail virtusertable (cyrusv2 as local mailer)
Don't put the domains in $w, use mailertable and _VIRTUSER_NO_RECURSE_ or whatever the option name is. On Mon, Dec 16, 2002 at 09:32:20AM +0100, Christian Schulte wrote: > Hi, > > after changing the local mailer in my sendmail.mc from cyrus to cyrusv2 > I cannot get sendmail to correctly deliver the domain-part of > local-adresses to cyrusv2-lmtpd! Before, I had the cyrusv2-mailer set in > /etc/mail/mailertable but that way , I was not able to route my email as > I need to and as I do in /etc/mail/virtusertable. Ecspacially > catchall-accounts for domains which have more than one email-account in > cyrus are not possible with the mailertable approach. > > I have all my local domains in /etc/mail/local-host-names and do (want > to do) all email routing in /etc/mail/virtusertable like before. > > If I specify a final recipient (cyrus-account) in virtusertable as: > > @virtualdomain.it [EMAIL PROTECTED] > > where an account like [EMAIL PROTECTED] exists, sendmail > recognizes virtualdomain.it in /etc/mail/local-host-names as a local > domain and will strip the original virtualdomain.it from the recipient > replacing it with the localhost hostname. All domains defined in > /etc/mail/local-host-names will be recognized in virtusertable but the > local delivery will only go to the user@localhostname! > > Where can I change sendmail to not do that ? How do I tell sendmail to > never change the local-domain to the local hostname on succesfully > recognized /etc/mail/local-host-names domains ? > > --Christian-- -- Joe Rhett Chief Geek [EMAIL PROTECTED] ISite Services, Inc.
Re: [CVS] pidfile support
On Tue, 17 Dec 2002, Rob Siemborski wrote: > On Tue, 17 Dec 2002, Henrique de Moraes Holschuh wrote: > > I like it. It also allows us to keep [A] around for a little while if we > > wish to do so, to detect startup errors of other sorts... > Okay, I've committed this. Take a look and let me know what you think. Looks okay to me. I didn't merge it into the Debian tree and test it yet, though. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
ctl_cyrusdb DBERROR on NetBSD 1.6
I'm currently trying to setup cyrus-imapd-2.1.11 on a NetBSD 1.6 server. For some reason I keep seeing errors in the log files whenever the ctl_cyrusdb checkpoint event is issued. This is what I saw in the logs when I first started the master process.. Dec 17 04:15:21 kompressor master[1444]: process started Dec 17 04:15:21 kompressor ctl_cyrusdb[1446]: recovering cyrus databases Dec 17 04:15:23 kompressor ctl_cyrusdb[1446]: done recovering cyrus databases Dec 17 04:15:23 kompressor master[1444]: ready for work Then immediately.. Dec 17 04:15:23 kompressor ctl_cyrusdb[1447]: checkpointing cyrus databases Dec 17 04:15:23 kompressor ctl_cyrusdb[1447]: DBERROR: archive /var/imap/db: cyrusdb error Dec 17 04:15:23 kompressor ctl_cyrusdb[1447]: DBERROR: error archiving log file: /var/imap/db/log.01 Dec 17 04:15:23 kompressor ctl_cyrusdb[1447]: DBERROR: archive /var/imap/db: cyrusdb error Dec 17 04:15:23 kompressor ctl_cyrusdb[1447]: done checkpointing cyrus databases These errors keep showing up in the logs every 30 minutes. Here are the contents of the /var/imap/db directory.. drwxr-xr-x 2 cyrus mail 512 Dec 17 04:21 . drwxr-x--- 10 cyrus mail 512 Dec 17 12:51 .. -rw--- 1 cyrus mail 8192 Dec 17 04:21 __db.001 -rw--- 1 cyrus mail270336 Dec 17 04:21 __db.002 -rw--- 1 cyrus mail 98304 Dec 17 04:21 __db.003 -rw--- 1 cyrus mail 17063936 Dec 17 04:21 __db.004 -rw--- 1 cyrus mail 32768 Dec 17 04:21 __db.005 -rw--- 1 cyrus mail 17497 Dec 17 04:21 log.01 >From browsing the mailing list archives I noticed two other people are experiencing this problem. http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=DBERROR&msg=19541 http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=DBERROR&msg=19579 This is a brand new server so no imap user accounts have been created yet. Is there any reason to be concerned about these errors?
Re: [CVS] pidfile support
On Tue, 17 Dec 2002, Henrique de Moraes Holschuh wrote: > Kludgy it is :P but using kill to detect the presence of a process is > actually reasonably portable AFAIK, and safe :-) Well, yeah, it looked ugly enough to be portable. ;) > > I think this works no matter how the things are interleaved after forking. > > I like it. It also allows us to keep [A] around for a little while if we > wish to do so, to detect startup errors of other sorts... Okay, I've committed this. Take a look and let me know what you think. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: Return-path header control
On Tue, 17 Dec 2002, Jonathan Marsden wrote: > 3. Remove: removes any existing Return-Path and does not add a new > one. I suppose this one would be fine, although I wonder why would anyone use it instead of "overwrite". Sieve really can benefit from a Return-Path header. > 4. Ignore: does nothing, leaves any existing Return-Path header(s) in > place. Not a good idea, I am not doing that. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: Return-path header control
On 17 Dec 2002, Henrique de Moraes Holschuh writes: > I could write code to add a configuration option to Cyrus so that it > has two methods to deal with return-path: > 1. Override (should be the default one): trash any return-path >headers in the message, and add ours (from -r or MAIL FROM:) > 2. Add: Add our return-path header _if_ there ins't already one in >there. Messages with more than one return-path header are in >error. It might be good to add a couple of other options while you are about it: 3. Remove: removes any existing Return-Path and does not add a new one. 4. Ignore: does nothing, leaves any existing Return-Path header(s) in place. That way all reasonable possibilities (at least, that I can think up!) for handling Return-Path are available and documented. Jonathan -- Jonathan Marsden| Internet: [EMAIL PROTECTED] | Making electronic 1252 Judson Street | Phone: +1 (909) 795-3877 | communications work Redlands, CA 92374 | Fax: +1 (909) 795-0327 | reliably for Christian USA | http://www.xc.org/jonathan| missions worldwide
Re: Return-path header control
Henrique de Moraes Holschuh wrote: 1. Override (should be the default one): trash any return-path headers in the message, and add ours (from -r or MAIL FROM:) 2. Add: Add our return-path header _if_ there ins't already one in there. Messages with more than one return-path header are in error. I think that second behaviour must be by default. I will follow CMU's judgment on this one. You do understand that if you use (2), you must have one MTA in your administrative domain that kills all possibly-illegal return-paths? Adding one is optional, but you must not let false ones through. So (1) is a safer default, AND it is compatible to what Cyrus tries to do right now. (2) is to be used by people that know what they're doing, and need a MTA to do the return-path creation beforehand due to envelope rewriting, or somesuch. All right, may be, I think so because I use deliver that rewrites envelope... So I always set up MTA to add the Return-Path header to mail messages. For LMTP delivery probably first behaviour is the best choice. Also, should messages with multiple return-paths be flagged as illegal? The RFCs seem to imply that only _one_ return-path header is allowed. Doing this could cause severe headaches for people with spools with broken emails with more than one (which I think is a fairly common problem). No, they shouldn't. Why? I would like answers a bit more elaborate than that, please. To avoid problems with message import from other mail storages wich mostly do not have such strict checks. Or there should be an utility that will verify message headers and correct them if necessary. There is a good principle for software - to be much more conservative and restrictive in output and liberal in input.
Re: Return-path header control
On Tue, 17 Dec 2002, Oleg Derevenetz wrote: > Henrique de Moraes Holschuh wrote: > >Well, this Return-path issues got me thinking a bit about the issue... > > > >I could write code to add a configuration option to Cyrus so that it > >has two methods to deal with return-path: > > > >1. Override (should be the default one): trash any return-path headers > > in the message, and add ours (from -r or MAIL FROM:) > >2. Add: Add our return-path header _if_ there ins't already one in there. > > Messages with more than one return-path header are in error. > > I think that second behaviour must be by default. I will follow CMU's judgment on this one. You do understand that if you use (2), you must have one MTA in your administrative domain that kills all possibly-illegal return-paths? Adding one is optional, but you must not let false ones through. So (1) is a safer default, AND it is compatible to what Cyrus tries to do right now. (2) is to be used by people that know what they're doing, and need a MTA to do the return-path creation beforehand due to envelope rewriting, or somesuch. > >Also, should messages with multiple return-paths be flagged as illegal? The > >RFCs seem to imply that only _one_ return-path header is allowed. Doing > >this could cause severe headaches for people with spools with broken emails > >with more than one (which I think is a fairly common problem). > > No, they shouldn't. Why? I would like answers a bit more elaborate than that, please. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: Return-path header control
Henrique de Moraes Holschuh wrote: Well, this Return-path issues got me thinking a bit about the issue... I could write code to add a configuration option to Cyrus so that it has two methods to deal with return-path: 1. Override (should be the default one): trash any return-path headers in the message, and add ours (from -r or MAIL FROM:) 2. Add: Add our return-path header _if_ there ins't already one in there. Messages with more than one return-path header are in error. I think that second behaviour must be by default. Also, should messages with multiple return-paths be flagged as illegal? The RFCs seem to imply that only _one_ return-path header is allowed. Doing this could cause severe headaches for people with spools with broken emails with more than one (which I think is a fairly common problem). No, they shouldn't.
Return-path header control
Well, this Return-path issues got me thinking a bit about the issue... I could write code to add a configuration option to Cyrus so that it has two methods to deal with return-path: 1. Override (should be the default one): trash any return-path headers in the message, and add ours (from -r or MAIL FROM:) 2. Add: Add our return-path header _if_ there ins't already one in there. Messages with more than one return-path header are in error. Also, should messages with multiple return-paths be flagged as illegal? The RFCs seem to imply that only _one_ return-path header is allowed. Doing this could cause severe headaches for people with spools with broken emails with more than one (which I think is a fairly common problem). Thoughts about the issue? Ken, Rob? -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: [CVS] pidfile support
On Tue, 17 Dec 2002, Rob Siemborski wrote: > On Tue, 17 Dec 2002, Henrique de Moraes Holschuh wrote: > > It won't. We will have to transfer the lock to B without a > > race-condition. That means IPC, or signals, or something like that... > > Yeah, I was discussing this with someone in the office yesterday, and > there's a half-decent way to do this, but it does requrie signals (well, a > pipe), and two locks. Looks good to me... pipes quite sane in most unixes, AFAIK. > I'm not sure I like using the presence of the pidfile, and this method > feels a bit kudgy to me (using kill to detect the presence of the process, Kludgy it is :P but using kill to detect the presence of a process is actually reasonably portable AFAIK, and safe :-) > for example). Like you mention, there's also a minor race. I think we > can do better: > > [A] Create/lock pidfile.lock. If locked, exit(failure). > [A] Create a pipe > [A] Fork [B] > [A] Block on reading exit code from pipe > [B] Create/lock pidfile If locked, write failure code to pipe and exit(failure) > [B] write pid to pidfile > [B] write success code to pipe & finish starting up > [A] exit(code read from pipe) > > I think this works no matter how the things are interleaved after forking. I like it. It also allows us to keep [A] around for a little while if we wish to do so, to detect startup errors of other sorts... -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: [CVS] pidfile support
On Tue, 17 Dec 2002, Henrique de Moraes Holschuh wrote: > It won't. We will have to transfer the lock to B without a > race-condition. That means IPC, or signals, or something like that... Yeah, I was discussing this with someone in the office yesterday, and there's a half-decent way to do this, but it does requrie signals (well, a pipe), and two locks. > One other possile way to do it, is to use the _presence_ of the pidfile with > a running process with that pid (specified in the pidfile) as the lock. We > remove the file on exit (clean or otherwise), and we ignore it at startup if > there is no such a process running in the system. [snip] I'm not sure I like using the presence of the pidfile, and this method feels a bit kudgy to me (using kill to detect the presence of the process, for example). Like you mention, there's also a minor race. I think we can do better: [A] Create/lock pidfile.lock. If locked, exit(failure). [A] Create a pipe [A] Fork [B] [A] Block on reading exit code from pipe [B] Create/lock pidfile If locked, write failure code to pipe and exit(failure) [B] write pid to pidfile [B] write success code to pipe & finish starting up [A] exit(code read from pipe) I think this works no matter how the things are interleaved after forking. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: [CVS] pidfile support
On Mon, 16 Dec 2002, Rob Siemborski wrote: > > 1. There is no support for updating the pidfile. Thus, the lock is acquired > > only after forking, which means we lost the controlling terminal by the time > > we can complain about not being able to acquire a lock, and that we have > > already tried to go daemon and thus whatever is trying to start master is > > NOT told that we failed. > > I don't think you can fix this in a perfect way, giving unix locking > semantics (well, atleast fcntl). Well, I don't have SuS3 here, but according to SuS2, you're correct. fcntl alone won't do it. > [A] lock and write pidfile > [A] fork and create [B] (which doesn't hold a lock on the file) > [B] attempt to reacquire lock on pidfile, FAIL since A doesn't have it > [A] exit(success) > [B] exit(failure) Which is clearly broken and suboptimal :-) I'm (un)?lucky that linux likes to run A first, then B (after the fork), so I just got an useless race in there that avoids the always-fail situation... > I don't think the BSD flock() semantics even help us here (hold lock It won't. We will have to transfer the lock to B without a race-condition. That means IPC, or signals, or something like that... One other possile way to do it, is to use the _presence_ of the pidfile with a running process with that pid (specified in the pidfile) as the lock. We remove the file on exit (clean or otherwise), and we ignore it at startup if there is no such a process running in the system. [A] 1. stat pidfile 2. file exists? 2a (yes) - read pid from file, kill(pid,0) it 2a1 - kill says process exists? fail to start if so 2a2 - lock file, fail to start if locking fails 2b (no) - create locked file, fail to start if locking fails 3. write pid to file (just in case) 4. unlock it, and fork [B] 1. lock pidfile [B] 2. write pid to it [B] 3. do master work() [A] 5. exit There are minor race conditions if whatever is trying to start master is a forkbomb, and the system has a small pid space, but that's far better than the current scheme. Unless I overlooked something, that is :) -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: ctl_cyrusdb -r takes too long
Hello, I am also quite interested in this topic as I will be using Solaris 9 with UFS logging enabled on each slices. Can others from this mailing list confirm that disabling UFS logging on the Cyrus slice (partition) will speed up the "ctl_cyrusdb -r" process ? I guess if it's recommended to turn off UFS logging, it would be good to have this somewhere in the Cyrus documentation or maybe in the FAQ... Regards Marc John Havard To: [EMAIL PROTECTED] <[EMAIL PROTECTED]> cc: Jatin Nansi <[EMAIL PROTECTED]> Sent by: Subject: Re: ctl_cyrusdb -r takes too long [EMAIL PROTECTED] ew.cmu.edu 12/16/02 06:17 PM You wouldn't happen to have have /var/imap on a logging or journaling filesystem by any chance? A while back I noticed this under Solaris. The fix was to mount /var without UFS logging. IIRC, the db code does its own journaling/logging, so the only thing you'll be missing by turning it off in the FS is the long start times. Regards, John Havard Systems Administrator Internet Doorway, Inc. Internet Doorway, Inc. (NETDOOR) - "Mississippi's ISP" 601.969.1434 | 800.952.1570 | http://www.netdoor.com/ --On Saturday, December 14, 2002 05:50:33 PM +0530 Jatin Nansi <[EMAIL PROTECTED]> wrote: > Hi, > > This is for cyrus gurus and for record: > > --- > Dec 13 20:10:21 mail3 master[4574]: process started > Dec 13 20:10:21 mail3 master[4575]: about to exec > /usr/cyrus/bin/ctl_cyrusdb Dec 13 20:10:21 mail3 ctl_cyrusdb[4575]: > recovering cyrus databases Dec 14 06:15:11 mail3 ctl_cyrusdb[4575]: done > recovering cyrus databases Dec 14 06:15:11 mail3 master[4574]: ready for > work > Dec 14 06:15:11 mail3 master[4979]: about to exec > /usr/cyrus/bin/ctl_cyrusdb Dec 14 06:15:11 mail3 master[4982]: about to > exec /usr/cyrus/bin/imapd Dec 14 06:15:11 mail3 master[4983]: about to > exec /usr/cyrus/bin/imapd Dec 14 06:15:11 mail3 master[4984]: about to > exec /usr/cyrus/bin/lmtpd Dec 14 06:15:11 mail3 master[4985]: about to > exec /usr/cyrus/bin/lmtpd Dec 14 06:15:11 mail3 master[4981]: about to > exec /usr/cyrus/bin/tls_prune Dec 14 06:15:11 mail3 master[4980]: about > to exec /usr/cyrus/bin/ctl_deliver Dec 14 06:15:11 mail3 > ctl_cyrusdb[4979]: checkpointing cyrus databases Dec 14 06:15:11 mail3 > lmtpunix[4985]: executed > Dec 14 06:15:11 mail3 lmtp[4984]: executed > Dec 14 06:15:11 mail3 tls_prune[4981]: mydelete: starting txn 2147483653 > --- > > Will ctl_cyrusdb -r take 10 hours ? I dread the day when the system will > need downtime again. > > This is on a decent 2 x P3 1 GHz with 100 GB scsi raid and 1 GB RAM, > about 2000 users, 50 MB per user. > > I really need reboot times to be between 5 to 10 minutes, as against 10 > hours. I will set the checkpoint period to 5 minutes as per the FAQ, but > are there any other pointers ? > > Jatin > > > > > Jatin Nansi wrote: > >> Hi, >> >> I have a cyrus IMAP system here which is taking way too long to start >> up, stopping just at ctl_cyrusdb -r. There does not seem to be any >> activity going on, but ctl_cyrusdb just doesnot exit. >> >> I am running cyrus-imapd-2.1.8 on a redhat 7.3 system. The backend >> database is DB3. >> >> I need some directions on what could be wrong and where ? Is it normal >> for ctl_cyrusdb -r to take as much as 1 hr (and still running) to >> finish ? >> >>
³¬Î¢MBAºËÐĿγÌÖ¤Êé°à£¨µÚÈýÆÚ£©
¹á³¹ÊµÓÃʵЧƷÖÊ ¼á³ÖѧÒÔÖÂÓÃÔÔò¡ª¡ª ³¬Î¢MBAºËÐĿγÌÖ¤Êé°à£¨µÚÈýÆÚ£© ¿Î³Ì°²ÅÅ£º 1¡¢Î´¶¯¶øÏÈı¡ª¡ªÖªÊ¶¾¼Ãʱ´úµÄ¹ÜÀí˼ά 2¡¢Ã»ÓÐ×îºÃ £¬Ö»ÓÐÊÊÓ᪡ªÏÖ´úÈËÁ¦×ÊÔ´¹ÜÀí 3¡¢´´ÐÂΪ»ê¡ª¡ªÓªÏú¹ÜÀíÓ봴оӪ ¡ïÖ÷°ìµ¥Î»£ºÑÇÌ«ÊÀ¼ÍÖ°Òµ¾ÀíÈËÅàѵÖÐÐÄ ¡ïÅàѵʱ¼ä£º12ÔÂ26ÈÕ¡ª¡ª28ÈÕ£¨¹²ÈýÌ죩 ¡ïÅàѵµØµã£º±±¾©¡¤±±¾©´óѧ ¡ï¿Î³ÌÊÕ·Ñ£º1980Ôª/ÈË£¨º¬Åàѵ¡¢×ÊÁÏ¡¢Ö¤Ê顢ר¼Ò·Ñ¡¢Îç²Í·Ñ£© ¡ïסËÞ°²ÅÅ£ºÐÖú°²ÅÅÔÚ±±¾©´óѧ¸½½üסËÞ£¬·ÑÓÃ×ÔÀí¡£ ¡ïʦ×ÊÁ¦Á¿£ºÌ¨Í岩ʿ£»±±´ó½²Ê¦£»¹úÄÚʵս¹ÜÀíר¼Ò¡£ ¡ï²Î»á¶ÔÏ󣺶ʳ¤¡¢×ܾÀí£»ÏúÊÛ¡¢ÈËÁ¦×ÊÔ´µÈÖи߲ã¹ÜÀíÕߣ¬Ö°Òµ¾ÀíÈË£» ¡ï½áÒµÖ¤Ê飺½áÒµ°ä·¢½áÒµÖ¤Êé¡£ ¡ï×Éѯµç»°£º010¡ª 82057871ת84ÎâÀÏʦ 82057871ת87 ÌïÀÏʦ £¨Çë²»ÒªÖ±½Ó»Ø¸´±¾Óʼþ£© È˲»¿ÉÄÜÔÚÒ»ÌìÄÚ³ÉΪ¹ÜÀí´óʦ£¬ µ«¿ÉÒÔÓôóʦµÄÑÛ¹âÈ¥¿´´ýÎÊÌ⣻ È˲»¿ÉÄÜÔÚÒ»ÌìÄÚ³ÉΪMBA£¬ µ«¿ÉÒÔÓÃMBAµÄÊÓ½ÇÈ¥½â¾öÎÊÌâ¡£ --- Èç¹ûÕâ·âÓʼþ´òÈÅÄúÁË£¬·³ÇëËæÊÖɾµô£¬²¢Çë¼ûÁ¡£ ÈôÄú²»Ï£ÍûÔÙ´ÎÊÕµ½ÎÒÃǵÄÓʼþ£¬·³Çë·ÃÎÊÒÔÏÂÍøÖ·£º http://mailt.59i.net/DB_Agents/user_cancel.asp?id=596&language=gb2312
