Re: [PATCH] 2.1.4 first step in shutdown()ing sockets
Henrique de Moraes Holschuh wrote: The attached patch tries to play it safe. To the fastmail.fm crew, are you sure all those prot_fill calls are really needed? SHUT_RD should dump any unread data from the socket anyway... Not at all sure. I suspect they're not necessary. We added them because we had to get our server up and running again ASAP which didn't leave time to test what was necessary and what wasn't, so we added everything we could think of. I agree that SHUT_RD ought to correctly flush the receive buffer. The prot_flush()s are still necessary I think--IIRC we added a few where they seemed to be missing. Thanks for your hard work on this, Henrique.
Re: [PATCH] Updated master.c process counting patch
Henrique de Moraes Holschuh wrote: I am, thus, somewhat wary of adding SHUT_RDWR inconditionally. Maybe we could add a runtime-option that very busy sites can set if they need even faster socket recycling? That sounds like a good idea.
Re: [PATCH] Updated master.c process counting patch
Henrique de Moraes Holschuh wrote: On Thu, 16 May 2002, Jeremy Howard wrote: I *strongly* recommend also including shutdown.diff. This is important in Linux to avoid sockets handing around in CLOSE_WAIT state. Remove the ' !imapd_in-tls_conn' bit everywhere for general distribution--this is a workaround for a memory corruption problem that is unrelated to this patch. I see. What about all those commented lines in your original master-avail.diff patch? Were they past shutdown() experiments? That's correct, Henrique. The commented lines should not be included in a widely distributed version. I left them in, but commented out, in order to remind myself what experiments we had done. Only the lines that are *not* commented out should be included. I believe (?) that this issue is less important on Solaris, because I think that it handles close() differently to Linux. However on Linux it is vital to flush receive buffers and call shutdown() to avoid hanging connections.
Re: Repeatable IMAP crash. Was Re: [PATCH] Updated master.c processcounting patch
Rob, could you post a core backtrace as well? Rob Mueller (fastmail) wrote: Actually, a number of users having been saying that renaming folders seems to cause problems. I think I've found a completely reproduceable scenario that crashes the IMAP server for me every time. [root@server2 root]# telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK blahserver Cyrus IMAP4 v2.1.3 server ready . login testuser blahpassword . OK User logged in . select inbox * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1021369326] * OK [UIDNEXT 1] . OK [READ-WRITE] Completed . create inbox.t1 . OK Completed . create inbox.t2 . OK Completed . select inbox * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1021369326] * OK [UIDNEXT 1] . OK [READ-WRITE] Completed . rename inbox.t1 inbox.t1_ . OK Completed . rename inbox.t2 inbox.t2_ Connection closed by foreign host. And in the IMAP log: May 16 22:22:25 blahserver master[23269]: process 8649 exited, signaled to death by 11 Does this do the same thing for anyone else? I'm not sure of all our config details, I'm sure Jeremy can post them if you need them. Rob - Original Message - From: Ken Murchison [EMAIL PROTECTED] To: Jeremy Howard [EMAIL PROTECTED] Cc: Lawrence Greenfield [EMAIL PROTECTED]; [EMAIL PROTECTED]; Henrique de Moraes Holschuh [EMAIL PROTECTED] Sent: Thursday, May 16, 2002 11:31 PM Subject: Re: [PATCH] Updated master.c process counting patch Jeremy Howard wrote: Lawrence Greenfield wrote: Date: Wed, 15 May 2002 16:02:42 -0300 From: Henrique de Moraes Holschuh [EMAIL PROTECTED] [...] The point is, if that indeed happens, log or no log, master loses track of the number of children that can service requests. That would be a bug, and the patch supposedly fixes this bug. It really doesn't matter (for accepting or not the patch) why the child died. Yes, I understand that. However, if the master (in real life situations) is actually losing track of the number of available service processes without one of those service processes crashing (either by the sysadmin or otherwise) then there's some other problem in the child accounting. The child accounting is fine. The problem in our case was always caused by child segfaults, or failure to properly close TCP connections. We still see segfaults (about one per fifty thousand connections I'd guess), Can you send us a backtrace from a core? If you're not getting a core, please setup your system to dump one. Here are bits that I use in my Cyrus startup script on Linux: cd /var/imap/cores ulimit -c unlimited export MALLOC_CHECK_=2 $master If you have multiple services/processes the cores will overwrite each other, so you need to catch it fairly quickly (unless they all have the same failure). Ken -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp
Re: [PATCH] Updated master.c process counting patch
Henrique de Moraes Holschuh wrote: On Fri, 17 May 2002, Jeremy Howard wrote: I believe (?) that this issue is less important on Solaris, because I think that it handles close() differently to Linux. However on Linux it is vital to flush receive buffers and call shutdown() to avoid hanging connections. I am still studying the shutdown issue. BTW, why didn't you shutdown() all streams at once, and then sleeped, instead of shutdown() and sleep one at a time? Because it was late at night and I wasn't thinking straight... ;-)
Re: [PATCH] 2.1.4 master process counting
Henrique de Moraes Holschuh wrote: Still, I am seriously thinking about doing something to any unknown children master notices (with proper LOG_ERR logging, of course) because said children is messaging it. Such a thing (unknown children) can only happen if either the underlying OS is messed up, or if we have a bug or memory corruption in the linked list, AFAIK. Actually this can happen. The CHLD signal handler can run before master gets to process the child's message. We see this happen quite regularly on a busy server.
Re: [PATCH] Updated master.c process counting patch
Ken Murchison wrote: I'm running a config almost the same as you and have never seen this problem. AFAIK, the CMU guys have never seen this either. Do you have a core that you can run a backtrace on, or can you force a core by setting MALLOC_CHECK_=2 before starting master (see malloc(3) for details)? Most of the segfaults were due to the problem that imapd_out or imapd_in were corrupted. The workaround discussed in our patch has solved most of these. I'll try and get a core file for the rare segfaults that we still get to see what the unresolved issues are. The imapd_out corruption problem can't be solved by studying the core file AFAICT because we can't see where the corruption is occuring. What's your DB config look like? Are you using skiplist for everything by any chance? Yes, 2.1.3 with skiplist for everything. We have the tls cache turned off, however. We prune the delivery database with -E0 every hour to avoid it getting big (otherwise DB recovery takes too long).
Re: [PATCH] Updated master.c process counting patch
Ken Murchison wrote: If you set MALLOC_CHECK_=2, then imapd will abort() whenever it thinks that there might be a corruption. By examining this core, it is easier to track down these problems. I've done this a few times to track down the subtle errors that have baffled others. Great. We've just updated our init.d file to use this, and to spit cores into different files using the pid, so hopefully we'll have some more info over the next few days.
Re: [PATCH] Updated master.c process counting patch
Henrique de Moraes Holschuh wrote: I don't know what Ken and Lawrence think of these patches, but I just finished porting the child pid tracking of master-avail.diff to 2.1.4CVS, and will post that to this list soon. I will also include it in Debian, which will give some field-testing to the patch. I *strongly* recommend also including shutdown.diff. This is important in Linux to avoid sockets handing around in CLOSE_WAIT state. Remove the ' !imapd_in-tls_conn' bit everywhere for general distribution--this is a workaround for a memory corruption problem that is unrelated to this patch.
Re: [PATCH] Updated master.c process counting patch
Lawrence Greenfield wrote: Date: Wed, 15 May 2002 16:02:42 -0300 From: Henrique de Moraes Holschuh [EMAIL PROTECTED] [...] The point is, if that indeed happens, log or no log, master loses track of the number of children that can service requests. That would be a bug, and the patch supposedly fixes this bug. It really doesn't matter (for accepting or not the patch) why the child died. Yes, I understand that. However, if the master (in real life situations) is actually losing track of the number of available service processes without one of those service processes crashing (either by the sysadmin or otherwise) then there's some other problem in the child accounting. The child accounting is fine. The problem in our case was always caused by child segfaults, or failure to properly close TCP connections. We still see segfaults (about one per fifty thousand connections I'd guess), and occassional TCP closing problems, but they're much reduced with the other patches. However with the master.c patch these intermittent problems have no practical impact on our users, since the server handles them gracefully. The master.c patch is important because without it, any little problem in any daemon that causes a child to crash, will eventually bring down the whole server. With the patch, processes are still counted correctly and therefore a child segfaulting will not stop the server from accepting connections. Child segfaults still cause the problem to be logged as per usual--the only difference with the patch is that child processes are correctly counted in this event.
Re: [PATCH] 2.1.4 master process counting
Henrique de Moraes Holschuh wrote: Here is the 2.1.4 port of the master process counting patch. It will be in Debian's Cyrus IMAPd package 2.1.4-9, to be uploaded today or tomorrow. I'm not 100% sure, but I think the change you've made to return if the centry isn't found may mean the nconnections doesn't get updated correctly in some cases. You probably want to update nconnections in the case of the SERVICE_CONNECTION message regardless of whether you find the centry.
Re: [PATCH] Updated master.c process counting patch
Henrique de Moraes Holschuh wrote: On Wed, 15 May 2002, Scott Russell wrote: On Wed, May 15, 2002 at 10:35:04AM -0300, Henrique de Moraes Holschuh wrote: I don't know about all the other patches, though. I have included the safe_flock patches, and I *may* include the alarm and locking stuff in master-avail.diff later, but I must study and understand it first. Are the patches TLS sane / tested? At one point I remember reading The ones I am including in Debian are. No TLS is _not_ an option IMHO. The TLS issue was simply required to work around a memory corruption bug. The patches all work correctly with TLS, as long as you remove the ' !imapd_in-tls_conn' bits in imapd.c and pop3d.c. These extra tests are nothing to do with the other patches. The problem is that imapd_in et al sometimes get overwritten, causing prot_flush and prot_fill to result in a segfault when they are call during shut_down() or imapd_reset(). Therefore we wanted some way of checking whether imapd_in was still sane, and we happened to pick to check that -tls_conn==0. A better approach would be to make the first element of the record a known constant and check for that. An even better approach would be to work out why the structure is getting corrupted in the first place! To clarify--the memory corruption problem occurs regardless of whether the other patches are present (in our environment at least), and needs to be worked around regardless of whether the other patches are present. The issues are completely independent. The memory corruption bug is something we see every few thousand connections under Linux 2.4.18 with Cyrus 2.1.3 in a mixed client high load environment.
[PATCH] Updated master.c process counting patch
Thanks to Jaska Kivela, some patch formatting problems that caused the master.c process counting patch to not apply cleanly have been resolved. The patch set has been updated, and now also incorporates the master.c race condition patch: http://jhoward.fastmail.fm/patches/cyrus/imap-diff.tgz The only file changed in the patch set is master-avail.diff . master-avail.diff solves the problem that master fails to correctly keep count of child processes if processes do not exit cleanly. This manifests itself as Cyrus failing to accept new connections on one or more ports after a while, when using preforking in imapd.conf.
Re: 'in-lining' a content_filter with delivery ...
Marc G. Fournier wrote: Then again, someone mentioned in the other thread about having the perl check to see if a user wants the filter to do the checks or not ... but, in the content_filter itself, there is no concept *of* a user, so how would you do such a check? In our content_filter we look at the RCPT TO: and check whether any user in the list wants spam filtering. If they do, we run it through SpamAssassin and add appropriate headers. Then in Sieve the actual actions get taken. The only downside is that if you want per-user scores for each test, and a message has multiple RCPT TOs, you have to decide which scores to use. Generally you'd want to choose the lowest score for each test, to err on the safe side. We consider this an acceptable downside, because it means that we only have to run each SMTP transaction through SpamAssassin, rather than each recipient through SpamAssassin. To avoid this, you'd have to deliver to an LMTP proxy and use SpamAssassin there, rather than use an SMTP content_filter.
[PATCH] Race condition in master.c
In the last set of patches we sent to the list, we included a patch to
master.c to avoid losing track of child processes after a segfault. This
patch has a race condition that we saw triggered under high load, where
a child can be reaped before master has processed an
MASTER_SERVICE_UNAVAILABLE notification. The result of this is that the
process count is off by one for each time the race condition occurs,
causing the number of processes to increase indefinitely.
This fixes that problem, and has resulted in a stable number of
processes on FastMail.FM for the last few days:
--- master/master.c Thu May 9 19:36:03 2002
+++ master/master.c.new Thu May 9 19:35:21 2002
-814,13 +814,17
switch (msg-message) {
case MASTER_SERVICE_AVAILABLE:
- c-is_available = 1;
- s-ready_workers++;
+ if (c c-pid == msg-service_pid) {
+c-is_available = 1;
+s-ready_workers++;
+ }
break;
case MASTER_SERVICE_UNAVAILABLE:
- c-is_available = 0;
- s-ready_workers--;
+ if (c c-pid == msg-service_pid) {
+c-is_available = 0;
+s-ready_workers--;
+ }
break;
case MASTER_SERVICE_CONNECTION:
Re: 'in-lining' a content_filter with delivery ...
Marc G. Fournier wrote: Ya, saw someone's thought about that ... that would definitely work instead of the spam extensions, but I don't believe the lmtp proxy supports that yet, does it? Correct. A general LMTP proxy framework doesn't exist yet. It would only be a few hours work to take the MSDW smtpprox framework and migrate it to LMTP, if someone needs that...
[PATCH] 2.1.3 reliability patches (REPOST)
I forgot to mention where the patches are last time! Sorry!
The patches described here can be downloaded from:
http://jhoward.fastmail.fm/patches/cyrus/imap-diff.tgz
As I mentioned last week, we have now resolved the main reliability
problems with 2.1.3. The problems were:
- Failure to flush buffers and therefore failure to close some connections
- Failure to call shutdown() to safely close some connections
- Corruption of a pointer causing segfault on shutdown of some sessions
- Failure to maintain connection count correctly in master after a child
segfault
Attached is a set of diffs against 2.1.3 that fix these and other
problems, written by Jeremy Howard and Robert Mueller of FastMail.FM
(except for those taken from 2.1.4, of course!). However, these diffs
are not production-ready: they work in our specific environment, but you
should review them carefully before applying in your environment. Having
said that, in our particular environment they have now fixed all of the
problems we were having, and we have now had no problems for four days
(previously we had to restart at least once a day). If you are using
Linux 2.4 and not using TLS most patches will work directly.
In addition to the basic reliability fixes, I'm also attaching most of
the patches that we are using since we added a number of features and
fixes which may be useful for others. I'd be interested in hearing any
comments on how to achieve any of these goals better, or in seeing some
of the patches fleshed out more fully.
The diffs are against 2.1.3 instead of 2.1.4 because 2.1.4 includes some
experimental code and our focus is at this stage on reliability. It
would be great if Project Cyrus could start using a scheme like Linux
and Perl where experimental releases get odd-numbered versions. That way
we could focus on completely stablising a release while adding new
features to a different tree. In the attached I have included some
patches from 2.1.4 where they are for reliability rather than new
features. Here's a complete description of the attached patches, in
order of priority. The first two patches are vital for production
systems IMO.
master-avail.diff
This patch is very important. Without it, any error causing a process to
segfault will eventually cause master to stop accepting connections.
master previously kept track of available processes in ready_workers
by listening to notifications from children. If a child died, no
notification would be sent. Therefore there would be less ready_workers
than master actually knew about. Over time, the pool of processes would
shrink until eventually no new processes would be accepted.
To fix this problem master should instead use the CHLD signal to track
ready_workers, as well as being notified when a child is busy servicing
a request. However if a child dies while in the middle of servicing a
request, it would decrement ready_workers twice (once when starting to
service the request, one on SIGCHLD).
Therefore a per-process status is required to track which pids are busy
servicing a request. Then on SIGCHLD ready_workers is decremented,
unless this status shows that the pid had already notified master that
it was unavailable. We have found that this patch successfully tracks
process counts under high load and when killing children with a range of
signals.
master-avail.diff also includes a patch from 2.1.4 that correctly
handles ALRM signals.
shutdown.diff
imapd and popd did not call shutdown() before close(). shutdown() should
always be called first to flush buffers and notify that no further data
is expected. On some OSs this is called automatically, but this should
not be assumed. This patch adds shutdown() calls as appropriate.
A socket will not close correctly if data is waiting to be read. This
patch adds a prot_fill call to ensure that receive buffers are empty.
Without this we frequently saw sockets waiting in a CLOSE_WAIT state.
***IMPORTANT:
In shut_down() you'll notice the following patch (and an equivalent in
pop3d.c):
-if (imapd_out) {
+
+if (imapd_out !imapd_out-tls_conn) {
This is because we found that imapd_out would often be corrupted.
Because we are not using TLS, we could check for corruption by making
sure that tls_conn was false. This will *NOT* work if you are using TLS.
A real fix is to work out why imapd_out is getting corrupted--we have
not as yet succeeded at doing that.
linux-limit.c
Linux does not recognise RLIM_INFINITY. This patch uses 64000 instead.
The patch really should add a configure test instead of just using this
on all systems. Without using this patch imapd will run out of FDs on
moderately loaded systems.
skiplist.diff
This is the improved skiplist code from 2.1.4. It works well for us.
safe-flock.diff
This is jwade's safer flock code which resolves problems with waiting
forever for locks. Although not required on all platforms, we are not
aware of any downsides to using it and therefore
Re: [PATCH] 2.1.3 reliability patches (REPOST)
Henrique de Moraes Holschuh wrote: I have cleaned up the log messages, and written the fcntl patch to get the same timeout functionality. See attached file. Great! + * Modified by [EMAIL PROTECTED] 2002-05-06 to work around seen file locking + * problem. Added locking timeout parameter to allow processes that are + * waiting for a lock to eventually time out, based on patch by Jeremy Howard Note that the original patch was actually by jwade, not by me. I included it for completeness to show all the patches that we are using to improve reliability, but we did not write or change that particular patch.
Re: [PATCH] 2.1.3 reliability patches (REPOST)
Henrique de Moraes Holschuh wrote: Duh, I should awake first before I reply. This is a From(space)-type header, which is useless for Cyrus, and should not be stored inside the message either. I am NOT adding this patch to Debian's Cyrus package. The problem is that a lot of messages have this header. They shouldn't, of course, and it's certainly not RFC-compliant... A nice approach would be to strip this header, rather than just failing to deliver the message.
Re: [PATCH] 2.1.3 reliability patches (REPOST)
Henrique de Moraes Holschuh wrote: They REALLY shouldn't, and the MTA is supposed to trash them when told to not add them in the first place, and to rewrite them with the true information when told to add them. Most MTAs are pretty good about this. However, when using an IMAP client to move messages between accounts (e.g. drag and drag in Mozilla, Netscape, Outlook Express, etc) the source account may well store the From in the message even although the MTA did not send it like this. This is where our users were constantly complaining until we added this patch--they would try to drag messages to their Cyrus mailbox from another mailbox and get the cryptic (to them) Message contains invalid header error. While I can certainly understand that accepting non-RFC compliant messages is something that may not be acceptable in a major distribution (at least by default) I can tell you that doing so has substantially reduced our support load in this case. We probably see this problem more often than most sites, because we are a fast-growing public email provider so people are often migrating messages across to us, and also we have a lot of ex-Hotmail users (and Hotmail is one of the providers that store messages with the From prepended).
Re: Master prefork problems; CLOSE_WAIT problems
Helmut Apfelholz wrote: I've found a way to fix this. In master.c reap_child, add: c-s-ready_workers--; and t-s-ready_workers--; ...immediately after the corresponding nactive--. This resolves the problem for me in the limited testing I've done to date. I've applied the patch to our server and I must say it doesn't work. The master process creates more and more processes with this patch. This is not the right solution to the problem. That's correct. We've got a proper fix which I'll be posting in a few hours.
[PATCH] 2.1.3 reliability patches
As I mentioned last week, we have now resolved the main reliability
problems with 2.1.3. The problems were:
- Failure to flush buffers and therefore failure to close some connections
- Failure to call shutdown() to safely close some connections
- Corruption of a pointer causing segfault on shutdown of some sessions
- Failure to maintain connection count correctly in master after a child
segfault
Attached is a set of diffs against 2.1.3 that fix these and other
problems, written by Jeremy Howard and Robert Mueller of FastMail.FM
(except for those taken from 2.1.4, of course!). However, these diffs
are not production-ready: they work in our specific environment, but you
should review them carefully before applying in your environment. Having
said that, in our particular environment they have now fixed all of the
problems we were having, and we have now had no problems for four days
(previously we had to restart at least once a day). If you are using
Linux 2.4 and not using TLS most patches will work directly.
In addition to the basic reliability fixes, I'm also attaching most of
the patches that we are using since we added a number of features and
fixes which may be useful for others. I'd be interested in hearing any
comments on how to achieve any of these goals better, or in seeing some
of the patches fleshed out more fully.
The diffs are against 2.1.3 instead of 2.1.4 because 2.1.4 includes some
experimental code and our focus is at this stage on reliability. It
would be great if Project Cyrus could start using a scheme like Linux
and Perl where experimental releases get odd-numbered versions. That way
we could focus on completely stablising a release while adding new
features to a different tree. In the attached I have included some
patches from 2.1.4 where they are for reliability rather than new
features. Here's a complete description of the attached patches, in
order of priority. The first two patches are vital for production
systems IMO.
master-avail.diff
This patch is very important. Without it, any error causing a process to
segfault will eventually cause master to stop accepting connections.
master previously kept track of available processes in ready_workers
by listening to notifications from children. If a child died, no
notification would be sent. Therefore there would be less ready_workers
than master actually knew about. Over time, the pool of processes would
shrink until eventually no new processes would be accepted.
To fix this problem master should instead use the CHLD signal to track
ready_workers, as well as being notified when a child is busy servicing
a request. However if a child dies while in the middle of servicing a
request, it would decrement ready_workers twice (once when starting to
service the request, one on SIGCHLD).
Therefore a per-process status is required to track which pids are busy
servicing a request. Then on SIGCHLD ready_workers is decremented,
unless this status shows that the pid had already notified master that
it was unavailable. We have found that this patch successfully tracks
process counts under high load and when killing children with a range of
signals.
master-avail.diff also includes a patch from 2.1.4 that correctly
handles ALRM signals.
shutdown.diff
imapd and popd did not call shutdown() before close(). shutdown() should
always be called first to flush buffers and notify that no further data
is expected. On some OSs this is called automatically, but this should
not be assumed. This patch adds shutdown() calls as appropriate.
A socket will not close correctly if data is waiting to be read. This
patch adds a prot_fill call to ensure that receive buffers are empty.
Without this we frequently saw sockets waiting in a CLOSE_WAIT state.
***IMPORTANT:
In shut_down() you'll notice the following patch (and an equivalent in
pop3d.c):
-if (imapd_out) {
+
+if (imapd_out !imapd_out-tls_conn) {
This is because we found that imapd_out would often be corrupted.
Because we are not using TLS, we could check for corruption by making
sure that tls_conn was false. This will *NOT* work if you are using TLS.
A real fix is to work out why imapd_out is getting corrupted--we have
not as yet succeeded at doing that.
linux-limit.c
Linux does not recognise RLIM_INFINITY. This patch uses 64000 instead.
The patch really should add a configure test instead of just using this
on all systems. Without using this patch imapd will run out of FDs on
moderately loaded systems.
skiplist.diff
This is the improved skiplist code from 2.1.4. It works well for us.
safe-flock.diff
This is jwade's safer flock code which resolves problems with waiting
forever for locks. Although not required on all platforms, we are not
aware of any downsides to using it and therefore we recommend its use.
---Diffs below this point are not required; they provide features that
may be useful to some
Master prefork problems; CLOSE_WAIT problems
I've seen a couple of problems over the last few weeks with master apparently failing to correctly maintain the prefork pool. We particularly see this problem with pop3d, which has more connects/disconnects than IMAP because of the nature of the protocol. The first issue is that in shut_down() sockets are not closed. It seems that this can leave sockets in CLOSE_WAIT state in certain error situations where popd_reset() is not called. The second issue is that we sometimes see sockets remain in a CLOSE_WAIT state because there is still data to be read. It appears that prot_fill() should be called in popd_reset() and shut_down(). The third issue is that when a process fails to shutdown correctly, such as if it segfaults, master does not seem to correctly keep track of the child process count. As a result, eventually the pool runs out and no more connections are accepted. Do the resolutions to the first two issues sound correct (we have made these changes and it seems to have fixed things for us)? Does anyone have a fix for the third issue?
Re: Master prefork problems; CLOSE_WAIT problems
Jeremy Howard wrote: I've seen a couple of problems over the last few weeks with master apparently failing to correctly maintain the prefork pool. We particularly see this problem with pop3d, which has more connects/disconnects than IMAP because of the nature of the protocol. ... The third issue is that when a process fails to shutdown correctly, such as if it segfaults, master does not seem to correctly keep track of the child process count. As a result, eventually the pool runs out and no more connections are accepted. I've found a way to fix this. In master.c reap_child, add: c-s-ready_workers--; and t-s-ready_workers--; ...immediately after the corresponding nactive--. This resolves the problem for me in the limited testing I've done to date.
[BUG] lmtpproxyd.c verify_user()
In lmtpproxyd.c verify_user(): if (plus) l = plus - user; - else l = strlen(buf); if (plus) l = plus - user; + else l = strlen(user); Yes, I'm too lazy to do a real patch, but you get the idea... ;-) buf is actually uninitialized at this point, so this bug causes intermittent failures that are hard to track down.
Re: spam sieve extension
Ken Murchison wrote: http://www.ietf.org/internet-drafts/draft-segmuller-sieve-relation-01.txt Nice! Is this being added to Cyrus? I don't see anything in CVS as yet...
Re: spam sieve extension
Bob Finch wrote: Jeremy == Jeremy Howard [EMAIL PROTECTED] writes: Jeremy Doing it in a content_filter lets you add headers that you Jeremy can then handle in Sieve scripts on a per-user basis, Jeremy which is faster and cleaner. Would it really be faster? I haven't played with Postfix content_filters much yet, but FILTER_README says You can expect to lose about a factor of two in Postfix performance for transit mail that arrives and leaves via SMTP, provided that you create no temporary files. From the diagram, it appears that mail passes through Postfix twice, so it looks like the message will be written into the Postfix queue twice as well. The time it takes Postfix to write out a queue file is nominal, compared to the time it takes SpamAssassin to analyze an email. So the key to speeding up the process is to call SpamAssassin as few times as possible. In our content_filter we check if any of the recipients has spam protection turned on, and if so we add the spam headers. If there are multiple recipients with spam protection turned on, SpamAssassin still only has to run once for the message, rather than once per recipient.
Re: spam sieve extension
Marc G. Fournier wrote: I have a server with 5 000+ users on it ... if each of them receives 10 messages a day, then 'content_filter' gets the joy of processing 50k messages ... if 10% of those users want spam filtering enabled, its now processing 40k *more* messages then it has to ... which is going to affect performance of the server ... Then you need to fix your content_filter. We only apply SpamAssassin to messages where at least one of the recipients has spam protection enabled.
Re: spam sieve extension
Marc G. Fournier wrote: Oh, very very cool ... I've been doign it in postfix's content_filter, but then its doing *everyone*'s email ... this is soo much better ... Any chance we'll see this in CVS sometime soon? Maybe in time for 2.1.4?? I hope not. Doing it in a content_filter lets you add headers that you can then handle in Sieve scripts on a per-user basis, which is faster and cleaner. The only thing necessary in Sieve to make this easier is an ability to specify that some headers are expected to contain numeric data. That way you could have X-Spam-score :greater 5.0 as a Sieve test.
Re: using spam control tools
Bob Finch wrote: The spam test passes the message to spamassassin's spamd. The test returns true iff the spamassassin score is greater than the threshold. This way, only users that want to use spamassassin incur the extra overhead. Neat. Our approach is to use a Postfix content filter. It checks the recipients against a DB to see if anyone have spam protection turned on, and if one does then it adds the SpamAssassin headers. A benefit of that is that it only runs SpamAssassin once for a message with multiple recipients.
Re: Watchdogs (was: Re: Cyrus 2.0.16 seen file locking patch)
Henrique de Moraes Holschuh wrote: I'd like to see watchdogs in cyrus, just like we have in postfix for example. That would protect us of all bugs like the hanging imap and pop3d from causing resource starvation, while we track down and fix whatever issue is causing the subsystem to hang... That would be oh-so-cool! Our 2.1.3 server has been locking up once every day or two, with a wide variety of problems related to skiplist, file locking, and stuff that we're still tracking down. Currently we are running a cron job to check IMAP, LMTP, and POP login every two minute and restart it if it's locked up. Not ideal, obviously! A watchdog system would be far preferable for periods where bugs are getting ironed out. Heneique, please let us know if you'd like any help with testing or anything else, if you decide to work on this.
skiplist recovery error
When our Cyrus (2.1.3 with everything on skiplist) restarted after being killed, the very first IMAP login had this problem: imapd[11677]: DBERROR: skiplist recovery: 0168 should be ADD or DELETE imapd[11677]: DBERROR: opening /var/imap/user/s/xxx.seen: cyrusdb error No other users had this problem. Any ideas what might be happening here? I had to delete this user's .seen file. Also, how do I gracefully shutdown Cyrus so it doesn't have to do a skiplist recovery on every db next time I start it?
Re: moving from 2.0 to 2.1.3
Oh, and using BerkeleyDB for deliver and tls stuff seems to be fine. The number of lockers will climb during peak periods, but then taper back down when traffic subsides. It just seems like mailboxes.db was an application not terribly well suited for it. We got deadlocks using BDB for the TLS cache, BTW. I just disabled the TLS cache altogether to fix it...
Re: Problem with imapd 2.1.3 not closing TCP session
Mike Brady wrote: I am observing a problem where imapd occasionally does not close the TCP session properly. This only seems to occur with 2.1.3. ...original details at end... We are also seeing an odd problem with 2.1.3. It may or may not be related to Mike's issue. We are using the skiplist backend for mailboxes and seen state. We have disabled the TLS session cache. We are using Linux 2.4.18 with Ext3. 2 times in the last week our IMAP server has suddenly stopped accepting IMAP connections on port 143 on its ethernet interface. However, it continues to accept IMAP connections on port 993 (SSL) and the localhost interface. Our cyrus.conf lines are: imap cmd=imapd listen=www.fastmail.fm:imap prefork=20 imaplocal cmd=imapd listen=[127.0.0.1]:imap prefork=25 imaps cmd=imapd -s listen=www.fastmail.fm:imaps prefork=2 Restarting Cyrus clears the problem. There are no unusual messages in imapd.conf or log/messages. We have plenty of spare system file descriptors. Both times this has happened has been at the busiest time of the day. However we had plenty of IO and CPU capacity spare at the time. It looks to me like somehow that particular port on that interface got 'filled up'. I'm not a TCP guru so I don't know exactly what that might look like--is there some limit on concurrent connections or a queue of waiting connections? When the lockup occured, telneting to the port simply resulted in nothing at all--it just sat there waiting for 5 minutes... Could Mike's problem report of TCP connections not being closed correctly lead to this kind of lockup? The rest of Mike's message... The details are as follows. My system is RedHat 7.2 with all appropriate errata except the kernel ones. The kernel is 2.4.18 compiled from tarball, but this issue also occurs with the Redhat 2.4.7-10 kernel. I am currently using Outlook 2000 on W2K Pro as my mail client (sorry :-). In addition I have compiled Postfix 1.1.4, Cyrus SASL 2.1.1 and Cyrus IMAP 2.1.3 from their respective tarballs. The server is lightly loaded. This is my home server, I am the only user and it is only doing my e-mail at the moment. For the most part mail works. That is, I can send and receive e-mail with no problems. The symptom of the problem is that Outlook gets stuck while retrieving messages (I can't remember the exact message on the status bar). By stuck I mean that the Outlook UI is still accepting input, but it doesn't do anything. The Outlook UI can be closed, but the Outlook processes does not die. The only way to get rid of it is to use Task Manager to kill it. On the server side netstat shows that the tcp session state is CLOSE_WAIT. I have a tcpdump capture which shows that Outlook (well the tcp stack underneath it) has sent a FIN packet and imapd has acked it, which is why it is in the CLOSE_WAIT state. So far so good. However, to complete the tcp session close imapd should next send a FIN which Outlook should then ack. imapd never sends the FIN. I have waited for hours. The only way to get rid of the imapd process is to kill it, at which point it does send a RST packet. However, by this stage W2K/Outlook is so out to lunch the RST does not do anything. The only way to recover is to kill the Outlook.exe process. I have not been able to determine a way to make this happen. It has occurred within seconds of opening Outlook, but sometimes it takes hours. My initial figuring out Cyrus install/testing was done with 2.1.2 and I did not see this behaviour. 2.1.3 came out while I was doing through my learning phase, so when I did my real install I use that. I went back to 2.1.2 on my live server about 24 hours ago and this problem has not occurred at all. It has been many years since I did any C coding so I haven't tried to look at the source to figure this out. If there is something specific that I can look at or there is any more information that I can send please let me know. Thanks Mike Brady Auckland New Zealand
Re: multiple cyruses via SAN
Amos Gouaux wrote: ... Though, I will say this: we recently switched to skiplist for mailboxes.db and at least so far, it beats the pants off of Berkeley DB. Ditto. We switched to skiplist last week (from BDB3), and the performance improvement is amazing! Plus of course, no more deadlocks... Oh, and the upgrade process couldn't have been simpler. :-)
Re: cyrus-2-1-sasl-v1-tail
Ken Murchison wrote: Jeremy Howard wrote: ... To correct locking problems you may want to grab the skiplist backend from the current CVS, and use that with 2.0.16. CMU are now using this backend in production. This won't work out of the box, because the cyrusdb interface has changed (new function pointers have been added to structure). Could you summarise the main steps required to make it work with 2.0.16?
Re: cyrus-2-1-sasl-v1-tail
Tonight I am upgrading 4 mail servers currently running Cyrus 2.0.16 release versions to cyrus-2-1-sasl-v1-tail from cvs. ... I am hoping this will correct allot of locking problems on the server and add most of the bug fixes from 2.0.16 (which is from June of 01) To correct locking problems you may want to grab the skiplist backend from the current CVS, and use that with 2.0.16. CMU are now using this backend in production.
Re: How to get mail for unknown users?
Lorty wrote: I recently upgraded my server to a Postfix/Cyrus Imap installation. Everything works fine, but I'd like to know ho to get email sent to unknown users. The luser_relay postfix configuration parameter has a lower precedence than cyrus deliver... METOO I couldn't get this working the last time I tried. In Postfix I tried setting local_recipient_maps in the hope that luser_relay would be used for stuff that wasn't in local_recipient_maps. But that didn't seem to work... /METOO
Re: How to get mail for unknown users?
Victor Duchovni wrote: What determines whether a user's mail should go to cyrus or not (it cannot be a cyrus delivery attempt)? The answer seems simple. If a user is not in local_recipient_maps, it's not a local recipient. In my case, the reason I want this is for mail abuse control. Most messages to a non-existant mailbox in our domain is untargeted spam hoping that particular usernames exist. I want to redirect all undeliverable messages to a mailbox where a script searches the headers for IP addresses and automatically checks for open relays and submits as appropriate to ORDB and ORBZ.
Re: Skiplist in 2.0? (was Re: Cyrus IMAP 2.1.2 released)
What is skiplist? What kinds of sites might benefit from using skiplist? How do you implement skiplist? skiplist is an algorithm used in the same way as a balanced binary tree, but implemented differently. By using non-deterministic insertion rules it avoids the need for a re-balancing step. Search the archives for 'skiplist' to see Larry's recent discussion and a URL I posted to more info on the algorithm.
Re: who uses cyrus ?
We are evaluating cyrus imap sieve for our production servers I would be grateful if someone could name maybe a few large companies / organizations that are also running it. http://FastMail.FM uses Cyrus. The web interface talks directly to the IMAP servers, and everyone has direct access to IMAP too. Although specific statistics are not public, you can get an idea of popularity that the FastMail.FM forum gets around 1,000 posts a month at http://www.emaildiscussions.com. FastMail.FM provides free and upgraded paid accounts to consumers, and business outsourcing.
Skiplist in 2.0? (was Re: Cyrus IMAP 2.1.2 released)
It also includes a cyrusdb_skiplist backend which may work better than db3 or flat for some sites. We encourage people to try it out, though they should be cautious about deployment. Further instructions will be posted to the mailing list when I get a chance. We've been getting bitten lots by the BDB deadlock problem and would like to switch to skiplist ASAP. However we're not yet quite ready to upgrade to 2.1. Is there any reason the skiplist backend shouldn't/can't be used with 2.0.16? If not, could you give a quick summary of what's required to patch 2.0.16 appropriately? TIA, Jeremy
Re: DB vs. Flat File
Feel free to write a lib/cyrusdb_mysqlinno.c; I don't have the time or inclination to do this right now. What's the best way to learn about how to write a backend? Is there some docs somewhere? Or is it just a case of looking at an existing backend? If there's no docs, could you provide a really quick summary of what a backend has to do? I'm also interested in an InnoDB backend and would be happy to spend some time on it if I have some time free. How big a job would you guess this is, Larry?
Re: DB vs. Flat File
On Mon, 04 Feb 2002 20:15:22 -0500 Walter Wong [EMAIL PROTECTED] wrote: With the release of Cyrus IMAP 2.1.2 there will be a new backend option: skiplist. We have already moved our development system to use it and performance seems very good. I expect we'll be moving our production system to it within a few months. It should not suffer from the same problems with insert/modify operations that the flat file has and it has better list speed than DB. Wow! I'd never come across the skiplist before, but I've just done some reading and it looks like a great algorithm! :-) Not only is it faster than b-trees because of not having to rebalance, but the code can be way simpler because deletions are so much easier. Are you going to use an existing library, or write your own? Does this mean that BDB is not required for installs that use the skiplist backend? Will the backend be used for seen dbs? ...mailboxes db? ...header caches? ...anything else?
Re: Eudora Client with Cyrus
Theodore Knab wrote: I was wondering if anyone has sucessfully intergrated the 5.1 version of the Eudora client with CMU's Cyrus Server. We have a number of Eudora users. Numerous discussions about their findings can be found by searching the 'FastMail.FM' forum for 'Eudora' here: http://www.emailaddresses.com/forum/search.php It works, but not that well. In particular, it doesn't understand the concept of personal namespaces at all. I've sent references to the relevent RFC to their support people, but they refuse to listen.
Re: dothack and cyrus 2.0.16
We are locking for dothack patch in order to be able to create logins with dots... The alternate separator patch is incorporated into 2.1, which provides the same functionality.
Re: Bug report: LMTP/SIEVE redirect
Lawrence Greenfield wrote: As of Cyrus IMAP 2.1.0, we invoke Sendmail with sendmail -i, which tells it to ignore dots on input. Why? How can we work around that?
Re: Bug report: LMTP/SIEVE redirect
Christopher D. Audley wrote: There is no need to 'work around this'. When Lawrence says sendmail ignores dots on input, he means they aren't given any special meaning. It doesn't mean they are discarded. The bigger problem is that this does not completely solve the issues with redirecting mail through sendmail. That's what I meant by 'work around this'. To be more specific--what tips does anyone have for redirecting messages using sieve when using sendmail or Postfix's sendmail-clone, if the original message contains raw '.'s?
Re: DBerror
db3: /var/imap/db/__db.004: Too many open files in system You need to # echo 8 /proc/sys/fs/file-max to increase the max number of open files allowed on your system (you use Linux, IIRC). Replace 8 with whatever number you require. /proc/sys/fs/file-nr tells you the max used in this session. Also, in master/master.c you may need to change the line limit_fds(RLIM_INFINITY) to limit_fds(6) (or some other particular number) because at least on my Linux (2.4.16) it doesn't know how to change to infinity.
Re: vacation auto responders
Ken Murchison wrote: Agreed. I'm going to add the Precedence header check momentarily. We are going to skip the List- header check because it would be difficult to implement given the current cmu-sieve architecture, and this check doesn't seem to be in wide use. Our system includes the following to test for lists: --- my %list_headers = ( 'Mailing-List' = 1, 'List-Unsubscribe' = 1, 'List-Post' = 1, 'Precedence' = 'bulk', 'Precedence' = 'junk', 'Precedence' = 'list' ); --- An RHS of '1' means header must exist whereas any other RHS means header must have this value. These criteria seem to catch pretty much all lists for us.
Mailbox does not exist error
I get intermittent errors Mailbox does not exist from deliver, for mailboxes which definitely do exist. The intermittent errors occur even on mailboxes which receive mail successfully just before and just after the error. I'm using Cyrus 2.0.16 and BDB 3.2.9. What kinds of situations could create these temporary failures? Nothing appears in imapd.log around the time that the bounces occur. Any suggestions on how to avoid this problem?
Re: lmtpd: how to send auth ?
Peter Pilsl wrote: ltmpd offers only one auth-method to me : external LHLO localhost 250-server.local 250-IGNOREQUOTA 250-8BITMIME 250-ENHANCEDSTATUSCODES 250-AUTH EXTERNAL 250 PIPELINING I just remembered something. IIRC, if you use a Unix socket rather than a TCP socket, LMTP doesn't make you authenticate. I dunno if you'll have to patch Net::LMTP to use a Unix socket--if so it's a simple module and I'm sure you'll have no trouble. Sorry if I've got this all wrong--this is just a vague recollection, and I'm too busy on other stuff right now to check...
Re: lmtpd: how to send auth ?
I want an external application deliver mails to my cyrus-box. I use LMTP but I dont know how to authenticate. After I send the 'MAIL FROM'-command I get the response: '530 Authentication required' I can skip this problem by specifying 'lmtpd -a' in cyrus.conf to turn off auth, but this is not what I want. What commands are there to authenticate the sender ? I assume that you use the same commands as you would with SMTP. Here's a summary: http://groups.google.com/groups?selm=192699727.1006985626%40bacardi Please post (and cc me) if you get a working Perl script. I'm interested in doing this too. You should also look at Bennett Todd's smtpprox; perhaps you can derive an lmtpprox?: http://bent.latency.net/smtpprox/ This is handy for writing filters in Perl that inject headers and accept/deny based on statistics about the message being delivered.
Global Sieve script?
I'd love to set some site-wide policies using a Sieve script. Is it possible to create a Sieve script that is applied to every incoming mail to all users, which is run before per-user scripts (even for users without a script)? I guess at the moment that would be kinda slow, because scripts are reparsed on every delivery. Has anyone made any progress on that idea of parsing scripts once and storing a byte code?
Re: db-4.0.14
Looks like there's a new release for Berkeley DB Any great new enhancements? Any experience using it with Cyrus and/or Postfix?
Re: [PATCH] daemonization support for master
Henrique de Moraes Holschuh wrote: This patch should be applied on top of the previous one in this thread (in message 20011128123038.K13192@khazad-dum). It adds configure support for --enable-pidfile (which enables pidfile and daemon mode). The full filename (with path info) for the pidfile can be sent as the optional argument for --enable-pidfile. The default is the old behaviour (no pidfile). The default pidfile is /var/run/cyrus-master.pid (unlikely to cause colisions). Nice one! :-) I'd love to see this added to 2.1.
Re: formail creates deliver-unreadable format
On 2 December 2001, Christopher D. Audley [EMAIL PROTECTED]
wrote:
deliver is correct in reporting the error. The 'From pilsl' line
with no colon after the From is how the beginning of a new message is
marked in an mbox file. It is not a correct rfc822 header, and as the
first line in the message, it is not a valid rfc822 message.
True. However, this behaviour is an unnecessary PITA. How about an
option for deliver to silently ignore From and From lines? (In
case you wonder, MH utilities sometimes stick From in the middle of
the headers.)
It is a PITA. It bit me because when a user drags a message from a Hotmail
account to a Cyrus IMAP account in Outlook Express the 'From ' header is
included, so Cyrus won't accept it. My horrible workaround was in message.c:
/* Check for valid header name */
if (sawnl buf[0] != ' ' buf[0] != '\t') {
if (buf[0] == ':') return IMAP_MESSAGE_BADHEADER;
for (p = (unsigned char *)buf; *p != ':'; p++) {
if (*p = ' ') return IMAP_MESSAGE_BADHEADER;
-^
Get rid of the marked '=', so that ' ' is not considered a bad header.
Re: Problem with cyrus imap 2.0.16 and Outlook Express 5.0: Seen flag does not stick
Cyrus caches seen state in memory for a time before flushing it to disk. Generally this works quite well; I use Outlook Express and don't seem to have this problem, but perhaps I just don't do this exact sequence of clicks. It's possible to force Cyrus to synchronize seen state more quickly with the comparable loss of performance/scalability. I certainly see this problem. Not regularly enough to have made me try and actually do something about it, but certainly I respond to a few support requests per week over this. Our web app at fastmail.fm runs multiple processes (being Apache 1.x + mod_perl) and I've written a socket pooling daemon to hand authenticated IMAP sockets to authenticated httpd processes. Because of how things are set up it's not unusual for multiple processes to end up with different IMAP connections and just occassionally they will end up showing the user the incorrect read/unread state as a result. I also see this problem in OE 5.0, 5.5, and 6.0. Again, only enough to be mildly annoying. I guess whether syncing should occur more quickly depends on the detail of just how much scalability is lost as a result.
Re: [PATCH] preliminary daemonization support for master
Thanks for this patch, Henrique. Can you explain what is actually causing those warnings (I get them under kernel 2.4 too)? What glibc version do you have? The 2.4 kernels and older glibc versions disagree on the value of RLIM_INFINITY. Glibc 2.2 should be OK. Thanks for the suggestion. It looks like that's not the problem in this case, since I've got 2.2: # rpm -qa | grep -i glib glibc-common-2.2.2-10 glibc-2.2.2-10 glib-1.2.9-1 glibc-devel-2.2.2-10 glib-devel-1.2.9-1
Re: [PATCH] preliminary daemonization support for master
Sideeffects of this patch are a few less gcc 2.95.4 warnings, and that
cyrus
will shut up about not being able to set the file handlers limit to
infinity
all the time (this is useful for Linux 2.2.x; feel free to rip it out...
it
was in my mods to master.c, so I left it in there).
rl.rlim_cur = x;
rl.rlim_max = x;
-if (setrlimit(RLIMIT_NUMFDS, rl) 0) {
+if (setrlimit(RLIMIT_NUMFDS, rl) 0 x != RLIM_INFINITY) {
syslog(LOG_ERR, setrlimit: Unable to set file descriptors limit to %d:
%m, x);
}
-if (verbose 1) {
- r = getrlimit(RLIMIT_NUMFDS, rl);
+if (verbose 1 getrlimit(RLIMIT_NUMFDS, rl) = 0) {
syslog(LOG_DEBUG, set maximum file descriptors to %d/%d, rl.rlim_cur,
rl.rlim_max);
}
Thanks for this patch, Henrique. Can you explain what is actually causing
those warnings (I get them under kernel 2.4 too)? Are they correct--is the
max FD limit actually not being increase? If so, does your patch allow the
max FD limit to be set correctly, or does it just quieten the warnings?
Sorry I couldn't figure this out from the code--this is stuff I'm a bit
unfamiliar with...
Re: [PATCH] preliminar run-time config of notification
Another patch, this one for the notify-unix or notify-zephyr services. ... The new configuration directive is named notify, and takes a boolean value. The default is to do notification (as Cyrus would do without the patch). Set it to false to disable notification. SNMP statistics are not affected. I am thinking about changing that to an string, with none being used for no notification at all, zephyr to use notify_zephyr (if support compiled in), and unix for notify_unix (support always compiled in). More importantly, we need separate conf entries for 'SIEVE' versus 'MAIL' class notification. I always have to comment out the 'MAIL' class notification, because I only want people to be notified if they have specifically requested it in their Sieve script. See: http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1018 BTW, I'd _really_ love to see support of the new(ish) notify extension draft in Cyrus IMAP 2.1: http://www.imc.org/draft-martin-sieve-notify This allows Sieve scripts to include everything required to work out what notification action to take in the script itself. Currently we have to do a DB lookup in our notify daemon to find the SMS number etc based on the mailbox name. With the 0.01 draft, this kind of info can be included in the script and passed to the daemon.
Re: What File Types does Cyrus use?
The .index, .cache, ... files are Berkeley DB files, I believe (www.sleepycat.com). I'd also like to know more about what's stored where and how, so that our webmail app can use them directly rather than going through the IMAP server to speed things up. - Original Message - From: Ashley Breeden [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, November 22, 2001 1:11 PM Subject: What File Types does Cyrus use? Hi All, I have been trying to find this answer out for days. I am writting a perl script for use with our Cyrus POP3/IMAP4 server and I need to interigate the Cyrus.index. Cyrus.header, Cyrus.cache files. However I have no idea what type of files they are? Does anyone know How I can find out or what the will be. If I use file cyrus.header I get cyrus.header: data As you can see that doesn't give me a lot to work with. Also if possible does anyone have anything that explains how each file is layed out, by this I mean what data is stored and for what. Thanks heaps! Ashley Breeden
Re: What File Types does Cyrus use?
Devdas Bhagat wrote: Hmmm, cyrus is supposed to be a black box solution. This means that access should only be through the IMAP/POP/KPOP interface and not directly over the filesystem. What cyrus does internally should not be the concern of the administrator/user. I'm sure we all understand the dangers of hacking at internal structures. There's also performance benefits associated with this. It's up to solution developers to decide whether that trade-off makes sense in their particular case, and would be based on benchmarking and analysis of the level of maintenance required if internal structures change. Obviously this voids the warranty ;-) so I'm not looking for support, but if anyone can summarise the file structures I'd appreciate. Otherwise I'll just work it out for myself by looking at the code, but that seems awfully inefficient, and will mean I'm just more likely to muck something up...
Re: [POLL] Cyrus v2.1 and SASL v2
Regarding complaints from those who don't want to mess with different software systems using different versions of SASL... DONT! Immediate upgrades to the latest version are not compulsory. How many recent messages start with I'm running Cyrus 1.6.x on If you have to wait for sendmail and OpenLDAP to catch up, don't worry, they will. I'd like to look at porting Postfix to SASL2 at some time shortly after Cyrus for SASL 2 comes out. Is there a porters guide someplace, or can someone post a summary of required steps?
Re: [POLL] Cyrus v2.1 and SASL v2
Additionally, how hard is it to write a pwcheck method in SASL 2? I had a couple I did in SASL 1.5, but they required patches to the source. Can you just drop in a plugin, do you have to write a pwcheck/saslauthd daemon, or is there a simpler method I'm just overlooking? You can plug in a C function into saslauthd pretty easily. I'll also be offering an alternative Perl framework using the same wire protocol a couple of months after release.
The FAQ (was Re: Storing sent-messages on Cyrus IMAP)
It used to have a section on clients, I do not know what it has at the moment. Somehow faq-o-matic crashed and wups I loost the 15 last answers I put there. ;( Hopefully they're not gone. I've just been really busy on other projects for a few weeks and haven't got around to fixing up the FAQ-o-matic. With a bit of luck I'll have something to announce this weekend and Tarjei's hard work will reappear. Also, I'm wondering about moving cyrus-utils off Sourceforge, because we've all found it very unreliable for managing file releases. The only alternative I know of is http://savannah.gnu.org/ . If anyone has any suggestions or experience with other free project hosting services please share them.
Re: The FAQ (was Re: Storing sent-messages on Cyrus IMAP)
AFAIK Savannah is a bit interested in people hosting their projects off savannah because of bandwith limmitations. I dunno, but if the CMU folks agree, why not move verything into the official CMU pages? I presume you mean 'uninterested' above! Anyhoo, the idea of integrating with CMU is great, but the whole reason I started the SF thing is because I want to let the burgeoning Cyrus community do what it can without disturbing the CMU folks who are busy coding and also running a large installation. Obviously if they don't mind providing access to the CVS/Bugzilla/FTP/Web areas for non-CMU / non-core folks, that would be great, but I want to continue to look at other alternatives because Larry and team might well prefer to keep the official Cyrus project fairly tightly controlled, whereas I'd like to see an informal, flexible, and only loosely controlled project for non-core stuff like utilities and docs.
Re: Performance issues
Every account I create sits under /my/imap/partition/user/account Is it a good idea to create several partitions to split access to this directory? Maybe even on separate physical discs? At our current (POP3 only) installation, all user files are sitting in the same directory, which makes access quite slow. There's an improved directory hashing algorithm at http://mail.cc.umanitoba.ca/source/ . I believe that it's scheduled for inclusion in the upcoming v2.1.
Re: Sieve
No. Sieve only scans headers, not bodies. Body checks should be done by your MTA. Search google for 'sieve filtering' for info on Sieve. - Original Message - From: Tym Rehm [EMAIL PROTECTED] To: info-cyrus [EMAIL PROTECTED] Sent: Wednesday, October 24, 2001 1:51 AM Subject: Sieve Can sieve filter exe and vbs files from email? I would want emails with exe attachments dropped, but I would like to know that it was dropped. Is there a way to do that with sieve?
Re: Sieve error in :matches clause
William K. Hardeman wrote: You've probably noticed this yourself, but I'm suspecting cyrus is treating the [ and ] as standard regex grouping characters, which would explain the number rejects. I don't really know anything about the Cyrus codebase, not being a programmer, but I do know that sieve can support regexes, from some things I've seen on the list and from my perusal of websieve. However, it was my understanding that regex support in a sieve script had to be explicitly enabled before it works. I suspect you're right, William. AFAICT this goes against the RFC however, since you should only get regex when you use the :regex comparator. Ken--is this a bug... a feature... a misdiagnosis...?
Re: Cyrus and very large folders
Jurgen Botz wrote: I just installed Cyrus for the first time to evaluate it as a possible mailserver to use at my company. I was surprised to find that each folder/mailbox is a single directory... my experience is that on typical Unix filesystems (UFS, ext2) this would cause severe performance degradation once a mailbox has a few thousand messages. I believe that Cyrus 2.1 includes Gary Mill's improved directory hashing scheme. Check the CVS tree--I haven't tried it so I don't really know what I'm talking about...
Sieve error in :matches clause
One of my users has this script:
if anyof(
header :matches subject *[spam score 10.0/10.0 -pobox]*,
header :matches subject **) {
reject Message bounced by server content filter;
stop;
}
It is rejecting a lot of messages that do not match either test. When I
changed the first test to *spam score 10.0*10.0 *pobox* (i.e. I got rid of
all 'special' characters) it worked fine.
But according to the RFC only '*' and '?' are special. Does Cyrus treat some
other chars specially?
CRAM-MD5 SASL auth?
Well, I've happily been supporting nothing but plain/login at fastmail.fm during our 3 years of development and beta testing, but now that we've just gone live I'm starting to see new mail clients with behaviours I haven't seen before. Some mail clients are trying to use CRAM-MD5, according to our logs. Can anyone give me a quick overview of what this is, and what I need to do to support it? I use a custom pwcheck daemon for authentication--am I going to have to change this in any way, or do I just have to compile something into SASL? TIA, Jeremy
Re: CRAM-MD5 SASL auth?
You're going to have to compile CRAM-MD5 support into SASL (if you don't already have it). Then you're going to have to allow your users to create a CRAM-MD5 secret in /etc/sasldb. If you search the archives, I think you'll find a PHP-based front-end for doing this. If you go ahead with this, you might want to just use sasldb for plaintext passwords as well. Is it possible to get my pwcheck daemon to handle this? Or is there no way for SASL to get the CRAM-MD5 secret from pwcheck?
Re: Cyrus Sieve/Notifications
The little Perl scripts are Unix socket daemons. You need to run one of them (start with the one that just does logging) and then check that stuff appears in your logs when you deliver mail. Also make sure that the directory exists that the Unix socket daemon is using (there are constants at the top of the Perl code that specify the directory), and that Cyrus has read/write permissions there. BTW, although I contributed that code I've never used it in production, and it has a known problem if headers are 2k (just fix the printf() in unix_notify.c if this is a problem for you). I'm planning on finally rolling out something that uses this in a couple of months, so it will be more actively supported then. *** And now a question to anyone who's actually using this... How do I get notify() called only when requested in a Sieve script, rather than every time like happens with Zephyr. I know that I can just check in my Perl daemon how it's been called, but this is a lot of extra overhead when the fast majority of users will not have any notification set up in their Sieve script. - Original Message - From: Jewfish [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, October 19, 2001 6:05 AM Subject: Cyrus Sieve/Notifications Hello. I am new to this -info list, but I have been using Cyrus IMAPD for several months now, and really enjoy it. Just a few days ago, I was able to work on getting Sieve working, and it works quite well. The only problem is that I wish to get notifications working. I compiled cyrus --with-notify=unix, but I have no idea where to go from here! I have the simple perl scripts from the contrib/notify_unix directory, also, but whenever I send an e-mail to myself, I don't see any notification happening. Any hints/suggestions/solutions/sample code? I am quite lost right now, and I hope some of you can be helpful. Thanks in advance! James
Re: Cyrus Sieve/Notifications
Try 'denotify' in your sieve script, you can then explicitly call 'notify' when you want. I *think* that cmu-sieve has an implicit 'notify' unless explicitly turned off. But doesn't the notify daemon get called for every message regardless of what Sieve says? ...Because that's how it works for Zephyr? I might be wrong here--corrections welcome. I'm particularly thinking of what happens for users that have no Sieve script at all (which is 99% of our users).
Extending Sieve
I'd like to allow our users to respond to messages with particular subjects/headers by emailing back files. I don't think that I can do this with Sieve vacation because vacation creates a plain text part rather than expecting a raw MIME structure. What's the best way to achieve this? Is it to create some kind of Sieve vacation extension that can take a file name instead of text as an argument, and treat that file as a raw message to send (or as a file to attach to the email)? Where would I look in the code to do this? Or is there an altogether different place to implement this?
Sender address for Sieve vacation messages
How does Cyrus decide what from/reply-to address to use when sending a vacation reply? My users can choose from quite a few domains, and they want to make sure that when a vacation reply is sent, that the From address is the same address that the original message was sent to.
Re: Sender address for Sieve vacation messages
Ken Murchison wrote: Jeremy Howard wrote: How does Cyrus decide what from/reply-to address to use when sending a vacation reply? My users can choose from quite a few domains, and they want to make sure that when a vacation reply is sent, that the From address is the same address that the original message was sent to. If you specify the :addresses parameter in the vacation action, it uses the first address in the list, otherwise it uses the envelope To address. Take a look at eval() in sieve/script.c if you are interested. Thanks for the pointer, Ken. If I patched eval() to choose the envelope address if it appears anywhere in the :addresses parameter, and then fall back to the existing behaviour, would that be something you'd consider incorporating? PS: luck level=pushed I'm a hopeless C coder (as regulars here will know ;-). Any chance of providing a patch that does this? /luck
Re: excuse my ignorance (auth methods)
Michael Bartosh wrote: this question is far simpler than most I see across this list- That's fine--we all start somewhere :-) But I can not get any method of authentication except for sasldb to work. I tied pam, I tried shadow (and added cyrus to the shadow group) and nothing worked until I gave the user a saslpasswd and changed auth to sasldb. How do I authenticate against pre-existing passwords? Or is there a way to keep a user's password and saslpassword in sync? Where do I start to look and see where my problem is? Start by reading the FAQ: http://cyrus-utils.sf.net/fom and the /docs directory in the source distribution (the install-*.html, overview.html, and faq.html files). You can use PAM or pwcheck to authenticate against shadow passwords. pwcheck is a little authentication daemon that comes with the Cyrus source distribution. After you've done that, try again from scratch (so that you don't have any bits and pieces from when you first started floating around). If it's not working, post detailed diagnostics here--there's been some good posts recently showing what this looks like. You should at least include contents and permissions of imapd.conf, cyrus.conf, pwcheck dir or pam config as appropriate, input and output to cyradm, imtest, and `telnet localhost imap`, and contents of /var/log/imapd.conf, plus your ./configure command and anything else relevent. HTH, Jeremy
Re: excuse my ignorance (auth methods)
Michael Bartosh wrote: OK. ran ldd on libgssapiv2.so and found that for some reason I had to copy a lot of kerb libs to /usr/lib, which fixed Oct 10 19:48:25 4am imapd[22837]: unable to dlopen /usr/lib/sasl/libgssapiv2.so: libgssapi.so.1: cannot load shared object file: No such file or directory My SASL configure entry on the following page disables stuff like GSSAPI so you don't have to both with this if you don't need it: http://cyrus-utils.sourceforge.net/faq/install/redhatinstall.php A lot of people seem to have trouble with PAM/shadow. I don't use it so I don't know the answer to your specific question. I use pwcheck, which is another option for you. and contents of /var/log/imapd.conf, For some reason nothing is being written to the file? I meant '/var/log/imapd.log', sorry. Search for 'syslog' in doc/install-configure.html to see how to turn logging on.
Re: [ANN] Publicly accessible Sieve-enabled IMAP
Ken Murchison wrote: John C. Amodeo wrote: ... And, are wildcards supported yet? In regex, yes. The extension follows the POSIX spec with the exception of disallowing backreferences (no variables currently allowed in Sieve). If you're really interested, have a look at the draft: http://www.imc.org/draft-murchison-sieve-regex Also note that you don't need to use regex to support wild-cards. Just use the ':matches' comparator, which is part of the Sieve RFC, IIRC. BTW Ken, what are the performance implications of using regex instead of :matches?
[ANN] Publicly accessible Sieve-enabled IMAP
One thing notably missing from most webmail providers is access to IMAP, and access to server-side filtering. Well, thanks to Cyrus we are now providing just that at http://fastmail.fm -- an IMAP accessible, Sieve enabled, email system. I just finished the Sieve interface yesterday. If you can't be bothered getting a FastMail account to see it, I've got a screen-shot here: http://www.fastmail.fm/docs/DefineRules.htm The nice thing about Sieve was that I didn't have to worry about users creating insecure scripts, since Sieve scripts have limited ability to do damage. The interface supports wildcards by creating the 'match' comparator as appropriate, although I'd like to support regexs--is that available in Cyrus at the moment (I know there's an internet draft around for it)?
Re: Per-user receive rate controls
Michael Fair wrote: If you do it any later then the initial attempt to send mail into the users inbox you have not gained anything as the mail has already gone through the pipeline. This is exactly right in a sense... but it's OK to _catch_ it later in the pipeline, and then as soon as some 'probable abuse' threshold is hit, add the user name to the check_client_access table. This can then be used to restrict RCPT TO. ... you will want to hack the Postfix daemon to check/update a counter and timestamp associated to the email address each time it receives the SMTP RCPT TO command. This integration would actually be really useful for stopping delivery for over quota users as well. Well, since I posted my plea for help, I've had a few beers with our webmaster and we've come up with a compromise which is lower resource usage but stops the worst of the abuse. Basically, the plan is to run a cron job (or daemon that sleeps for a few minutes after each loop) that checks for which accounts have been updated since last run (a quick hack is to look at the last update time of the directories in the imap message folder, although I'm sure others can suggest better ways), and do an IMAP STATUS on each to check for new messages, storing the result in a table and checking the delta against the last run to see whether the number of new messages is over a reasonble-use-threshold. We'd run this with a low 'nice' priority to ensure that it doesn't get lost if a mound of spam is arriving. Otherwise, I would either pass it off as anomolous hardly worth the resources and engineering efforts to defend against, and then wait to see if this practice actually became a larger nuisance than a one time event. Michael--many thanks for the thoughtful response. Interesting to hear that this behaviour is anomolous in your experience--we've only recently started publicising our system after 2 years beta testing with a hand-selected group, so we don't really know what level of abuse to expect now that we're out in the big wide world. I'm going to at least put in place the basic response outlined above--even if it is not really necessary; our users were subjected to a few hours of very patchy response from our server, so the least I can do is to show that they shouldn't have to put up with this again... For good measure, now that his account has been blocked I would send him an email threatening with abuse of resources and a more stringent quota as a result and request a response informing me of the correction within 72 hours. Check the logs every so often to see if the end user logs in to receive the warning and if not, nuke the account. Since the case tends to be that once you are on the spam list, you aren't getting off of it, there will most likely be nothing the end user can do about it and therefore have their account nuked for abuse anyway. Yeah, I've already sent him a message to his alternate account, but I didn't directly threaten him but rather offered help in case he's just been an unlucky target (I don't want to offend someone and just end up on the end of a DOS attack). But now that I've built a little Perl script to scan the received email headers I see that they were sent to over 500 yahoogroups mailing lists, with names like '[EMAIL PROTECTED]' and '[EMAIL PROTECTED]'... It makes me wonder if this guy was actually maintaining these lists as a way to get cheap mass mailings, and subscribed himself through his account on my system as a way of checking that they were all running smoothly. I've sent this list to abuse@yahoogroups which hopefully they'll find handy... Heh--our TC has a clause saying that damages for SPAM are assumed to be $5 per message, so we could make a good profit from this ;-) I think I've got better things to do than to get involved in this nasty business though...
Re: pop3d auth
Pretty strange -- I have tuned pam_mysql to be case insensitive for usernames. If I telnet to imap port and do . login UsEr password it logs me in. If I do the same with pop3 (user UsEr pass password) according to log it says user UsEr logged in, however the response is -- ERR Invalid login Just to check--are you sure that pop3 is working OK when you test matching case?
Re: Per-user receive rate controls
Ralf Hildebrandt wrote: On Wed, Oct 03, 2001 at 01:15:23PM +1000, Jeremy Howard wrote: What I'd like to do is avoid this happening in the future. I've manually added this address with REJECT to check_client_access for now. Now what I'd You mean check_recipient_access? Yes I do--sorry. Something like pop-before-smtp.pl will do the trick: It will tail the maillog and then you can build an in memory database (a queue) that stores recipients, number of mails they recieved and timestamps. Nice--I'll do this, at least until I get around to hooking into the delivery process directly. However my logging is currently set in syslog to be only 'notice' or above. If I change it back to * I get much more logging than I want. Is it possible to just log the lines that say: postfix/smtp[10332]: CDA86E382: to=[EMAIL PROTECTED], relay=domain.com[0.0.0.0], delay=5, status=sent (250 TAA10932 Message accepted for delivery) This would be just enough to know how many messages were being sent out. I know how to add _more_ logging with -v, and change the _overall_ logging amount with syslog.conf, but I haven't found any info on finer-grain log control...
Re: Per-user receive rate controls
Chris Audley wrote: Create a named pipe in some appropriate location such as /var/local for the perl script to read from, mkfifo /var/local/lmtpmon Then add an entry to the syslog.conf file to send mail.info messages to the pipe separate from the entry currently sending mail.notice to maillog mail.=info|/var/local/lmtpmon Now your perl script will get the info messages it needs to work without cluttering your maillog file. Brilliant Chris--I knew in theory that you could put a pipe into syslog, but I'd never thought to try it... This is a fantastic solution!
Per-user receive rate controls
Sorry for the x-post, but I'm not sure if this is best done by Postfix or deliver... Last night we had a user sign up who for some reason used their account to receive a _lot_ of spam (thousands of messages per minute). I'm curious as to why this might happen--any suggestions via private email would be welcome. But that's not the purpose of this post... What I'd like to do is avoid this happening in the future. I've manually added this address with REJECT to check_client_access for now. Now what I'd like to do is add something that checks how many messages a user has received in the last n minutes, or in some other way is triggered by an unusual rate of email to a particular user. I'd want to be able to trigger a little script based on this hook, which would automatically add the user to our check_client_access table and notify me that something fishy was going on. That way my other users won't be effected because Postfix will block the deluge of messages early on. What would be the easiest way to do this? I could use unix_notify in Cyrus to update a table and check rate with a little Perl daemon, but this seems like a resource intensive way to do such a simple check... Another extreme would be a cron job that checks somehow the message rate--but what log to check? How are others dealing with this, if at all? TIA, Jeremy
Re: Migrating from wu-imap to Cyrus
[EMAIL PROTECTED] wrote: Cillian Sharkey writes: Marko Cuk [EMAIL PROTECTED] wrote: I have working cyrus, but I have problems with migration. If I try to move messages in Netscape client, it moves some folders, but on some folders I get an error, that message contains invalid header... I got that too when doing a mass-conversion a while back, but my script simply 'ignored' troublesome mails and carried on. I never did track down what header Cyrus didn't like as the number of dropped mails was small. I'd recommend tracking down which message is causing the problem then look for any invalid headers (or post it here nothing seems amiss). Here's an example of a messages that Cyrus rejected when I was testing loading Cyrus mailboxes from Unix mailboxes: From IMAP4.psuedo.sims Fri Sep 28 10:09:00 2001 Date: Fri, 28 Sep 2001 10:09:00 -0500 (CDT) From: Postmaster Subject: Message from mail server Content-Length: 94 Mime-Version: 1.0 Status: RO X-IMAP: 923336730 178 The problem is the 'From' header. I patched imap/message.c so that rather than saying if (*p = ' ') return IMAP_MESSAGE_BADHEADER; it says if (*p ' ') return IMAP_MESSAGE_BADHEADER; I haven't seen any bad side-effects from this. I've been running it for over a month. I made this change because HotMail adds this header, and dragging messages in Outlook Express from a HotMail account to an IMAP account fails without the patch.
Re: overview.html
Tarjei Huse wrote: How up to date is the overview.html part of the docs that come with imapd? Its pretty good. I fyou see something that is clearly wrong/old, let us know. I find the Cyrus docs pretty reliable. I suggest that in the FAQ you remove the comment about suggesting that users follow the HOWTO but be aware of differences with v2, and instead suggest that they ignore the HOWTO and instead follow the install*.html in the distribution. BTW, can we mirror the Cyrus docs on the FAQ site? Not everyone has the docs in their binary distribution, and it would also be nice to link to the official docs from the FAQ.
Re: LMTPD signaled to death by 11 - neverending story [the end]
Amos Gouaux wrote: On Mon, 1 Oct 2001 05:56:08 +0200, Szymon Juraszczyk [EMAIL PROTECTED] (sj) writes: sj I spent a few days figuring out why this beast was crashing. And all sj because lots of people still are unaware of elementary secure programing sj issues, hence they make trivial mistakes such as sprintf()-ing variable sj length string into a fixed size buffer. Sigh... Looks like this was contributed to CMU. Wait, did you use the --with-notify option to configure? If so, what did you specify it as? Yes, that is contributed code. By me! When I posted it to the list I warned that I'm not a regular C coder and that this is a quick hack mainly copied and pasted from someone else's code, but I guess that that warning didn't get through to the docs. Anyway, this is an obvious and stupid mistake, and I'm sorry that you've been caught by it. I haven't actually got around to using notify_unix at our site, which is why I haven't checked the code more carefully, and haven't been caught by this bug.
Re: Cyrus 2.0.16 with RedHat 7.1
I have compiled the sources with the --with-auth=unix and I also tried with --with-pwcheck_method=pam but still does not work. ... Hey, nice set of diagnostics :-) We might put this in the FAQ as a good example of how to get your question answered :-) Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. +OK magenta.omnisig.com Cyrus POP3 v2.0.16 server ready user test +OK Name is a valid mailbox pass -ERR Invalid login Try creating a sasldb file--I seem to remember Cyrus getting confused without one, even although you're not actually using it for authentication. Create one with utils/saslpasswd, and add a 'cyrus' user, since that's who you have listed as an admin. I don't do PAM, so I can comment on the rest of the config.
Re: Cyrus 2.0.16 with RedHat 7.1
Ok I added the following line in my /etc/pam.d/pop: auth required pam_warn.so And here is the log when I tried to connect using pop: Sep 28 10:01:22 magenta PAM-warn[2069]: user: (uid=76) - test [remote: ?nobody@?nowhere] So PAM seems to work correctly but why do I still get an Invalid Login?? What is cyrus doing after a user is authenticated??? You only showed your telnet session using POP. How about IMAP? Try: telnet localhost imap . login user password . logout If that works but POP doesn't, it's probably just that you haven't created a mailbox for your user using cyradm yet.
Re: LMTPD signaled to death by 11 - neverending story
I experience often lmtpd process segfaulting with signal 11. I tried to find a solution in the archivers the list, unfortunately with no luck. What's strange is that sometimes it delivers the mail, the other time it crashes, so deliver returns error 75 (temporary unavailable, try again). Has anyone experienced this bug and found a proper solution? I use Cyrus-2.0.16 and authenticate to an LDAP server (authentication works fine for me). Are you using PAM/LDAP? If so check the recent archives (about 4 weeks ago) for a very long thread regarding a memory allocation problem between OpenLDAP and Cyrus. I'm not sure what the resolution was in the end... Using saslauthd from the new SASL version would almost certainly fix it though.
Re: Cyrus 2.0.16 with RedHat 7.1
Eric L'Heureux wrote: I need help! I'm trying to install Cyrus 2.0.16 on Red Hat 7.1. I keep getting Invalid login errors when trying to connect from pop or imap. I've set-up Cyrus to use PAM for authentication but it seems to try looking for a sasldb file. I DO NOT want to use sasldb, I have already a huge passwd/shadow database and I'm not planning to convert it to sasldb. What configure command did you use? What do your cyrus.conf and imapd.conf files look like? What is in your imap log when you fail to authenticate?
Re: Microsoft Outlook Express Logon using Secure Password Authentication option.
James Courtier-Dutton wrote: Can the above option be used with cyrus imap ? If so, how ? I believe that it is proprietary. How ever you can enable SSL in OE, which works with Cyrus fine.
Re: quota abstraction idea, comments ?
Helmut Apfelholz wrote: I was thinking about creating a deamon that would do quota operations in the cyrus system. One could then write the functions used by deamon for seting and reading the quota. In such a configuration one could store the information in: - files as it happens now - SQL database and so on. Such a setup would for example allow for storing quota information in the central database for separate servers. Sounds great. If you do this, have a look at unix_notify.c to see in general how to get Cyrus to call out to a daemon during a delivery phase. Consider implementing a quota check in a similar way--eg a compile time hook that can be easily configured, with a specific hook included that does the Unix socket daemon callout.
Re: AARRGGGHHHHHHH!!!! deliver - lmtp - cyrus still not working!
Louis LeBlanc wrote: What the heck happened to my leblanc+root? It looks like lmtp is dropping it! What gives? BTW, if no one remembers from a previous post, I have built this on FreeBSD 4.3, from the ports - default build as defined by the port maintainer. Can you show the exact output of: cyradm localhost -u admin_name lm user.leblanc.* lam user.leblanc.root Also, have you tried subfolders other than 'root'? Maybe 'root' is special in some way...
Re: Can't deliver to INBOX.folder
Louis LeBlanc wrote:
Here is what I have in that conf:
SERVICES {
# add or remove based on preferences
imap cmd=imapd listen=imap prefork=0
imaps cmd=imapd -s listen=imaps prefork=0
pop3 cmd=pop3d listen=pop3 prefork=0
pop3s cmd=pop3d -s listen=pop3s prefork=0
# sieve cmd=timsieved listen=sieve prefork=0
# at least one LMTP is required for delivery
lmtp cmd=lmtpd listen=lmtp prefork=0
lmtpunix cmd=lmtpd listen=/var/imap/socket/lmtp prefork=0
}
Now, I am starting to suspect that I might want to kill one of those
lmtp* entries. Any thoughts?
No, that shouldn't be necessary. Still if you're not using both the Unix
socket and the TCP socket you may as well comment out the one you're not
using.
Re: Can't deliver to INBOX.folder
Tarjei Huse wrote: re [anyone p] right Hmm. This has to be set user by user, right? Yes, I believe so. Whould it be possible to have an option in imapd.conf to let all users that are created get this flag? Maybe I'mwrong, but I seem to remember that this does not exixst today (read through imapd.conf last night). I don't think you're wrong. But you must be creating your users somewhere--can't you just edit that script so that it sets the [anyone p] right when they're created? Tarjei sam mailbox anyone p I tried looking through the imapd.conf manpage, but I didn't find a option that all subfolders of a user got this flag set. AFAIK the flag is set when the user creaes a new subfolder. I believe that you have to set [anyone p] on users.username, which means that then any subfolder they create will have these rights assigned.
