Patch: add new lmtptarget annotation
All,
Just submitting a patch I'm supporting locally for consideration.
We use shared mailboxes quite extensively for role-based communication.
For quite some time we've had a problem with users deleting or renaming
mailboxes into which we deliver mail. We can, and do, use IMAP ACLs to
dissallow users from deleting the delivery target mailbox. But when a
user creates a child mailbox it inherits the ACLs of the parent, and the
user is then not able to delete or rename the sub folder.
As a fix, I have written a patch against 2.3.16 to add a new lmtptarget
mailbox annotation. When enabled, Cyrus won't allow the mailbox to be
deleted or renamed. We can then set whatever ACLs we want inherited by
child mailboxes, happy in the knowledge the user won't blat the mailbox
and cause mail to bounce.
The rationale here is that Cyrus treats user.foo with special
significance as a delivery target, but does not do the same for shared
mailboxes because there is no way for Cyrus to know which shared
mailboxes we intend to deliver mail into. Using a mailbox annotation
seems a nice way of flagging this.
Patch attached. Comments welcome.
Cheers,
Stephen
--
Stephen Grier
Systems Developer
IT Services
Queen Mary, University of London
diff -Naur cyrus-imapd-2.3.16.old/imap/annotate.c cyrus-imapd-2.3.16/imap/annotate.c
--- cyrus-imapd-2.3.16.old/imap/annotate.c 2009-12-21 11:25:22.0 +
+++ cyrus-imapd-2.3.16/imap/annotate.c 2010-05-18 10:54:42.069620739 +0100
@@ -1874,6 +1874,9 @@
{ /vendor/cmu/cyrus-imapd/duplicatedeliver, ATTRIB_TYPE_BOOLEAN, BACKEND_ONLY,
ATTRIB_VALUE_SHARED | ATTRIB_CONTENTTYPE_SHARED,
ACL_ADMIN, annotation_set_mailboxopt, NULL },
+{ /vendor/qmul/cyrus-imapd/lmtptarget, ATTRIB_TYPE_BOOLEAN, BACKEND_ONLY,
+ ATTRIB_VALUE_SHARED | ATTRIB_CONTENTTYPE_SHARED,
+ ACL_ADMIN, annotation_set_todb, NULL },
{ NULL, 0, ANNOTATION_PROXY_T_INVALID, 0, 0, NULL, NULL }
};
diff -Naur cyrus-imapd-2.3.16.old/imap/mboxlist.c cyrus-imapd-2.3.16/imap/mboxlist.c
--- cyrus-imapd-2.3.16.old/imap/mboxlist.c 2009-11-17 03:34:30.0 +
+++ cyrus-imapd-2.3.16/imap/mboxlist.c 2010-05-18 11:18:15.509634066 +0100
@@ -1028,6 +1028,7 @@
int mbtype;
const char *p;
mupdate_handle *mupdate_h = NULL;
+struct annotation_data attrib;
if(!isadmin force) return IMAP_PERMISSION_DENIED;
@@ -1048,6 +1049,14 @@
if (!isadmin) { r = IMAP_PERMISSION_DENIED; goto done; }
}
+/* Does mailbox have the lmtptarget annotation set? */
+if (annotatemore_lookup(name, /vendor/qmul/cyrus-imapd/lmtptarget, , attrib) == 0
+attrib.value !strcasecmp(attrib.value, true)) {
+/* Even admins can't delete a mailbox with the lmtptarget annotation set. */
+r = IMAP_MAILBOX_NOTSUPPORTED;
+goto done;
+}
+
r = mboxlist_mylookup(name, mbtype, path, mpath, NULL, acl, tid, 1);
switch (r) {
case 0:
@@ -1193,6 +1202,7 @@
char *newpartition = NULL;
char *mboxent = NULL;
char *p;
+struct annotation_data attrib;
mupdate_handle *mupdate_h = NULL;
int madenew = 0;
@@ -1299,6 +1309,13 @@
goto done;
}
}
+/* Does mailbox have the lmtptarget annotation set? */
+if (annotatemore_lookup(oldname, /vendor/qmul/cyrus-imapd/lmtptarget, , attrib) == 0
+attrib.value !strcasecmp(attrib.value, true)) {
+/* Even admins can't rename a mailbox with the lmtptarget annotation set. */
+r = IMAP_MAILBOX_NOTSUPPORTED;
+goto done;
+}
r = mboxlist_mycreatemailboxcheck(newname, 0, partition, isadmin,
userid, auth_state, NULL,
newpartition, 1, 0, forceuser, tid);
diff -Naur cyrus-imapd-2.3.16.old/perl/imap/IMAP/Admin.pm cyrus-imapd-2.3.16/perl/imap/IMAP/Admin.pm
--- cyrus-imapd-2.3.16.old/perl/imap/IMAP/Admin.pm 2008-04-04 13:47:11.0 +0100
+++ cyrus-imapd-2.3.16/perl/imap/IMAP/Admin.pm 2010-05-18 11:30:54.437108440 +0100
@@ -789,6 +789,7 @@
expire = /vendor/cmu/cyrus-imapd/expire,
news2mail = /vendor/cmu/cyrus-imapd/news2mail,
sharedseen = /vendor/cmu/cyrus-imapd/sharedseen,
+ lmtptarget = /vendor/qmul/cyrus-imapd/lmtptarget,
sieve = /vendor/cmu/cyrus-imapd/sieve,
squat = /vendor/cmu/cyrus-imapd/squat );
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Patch: add new lmtptarget annotation
On 18/05/10 18:47, Wesley Craig wrote: Seems like a reasonable approach and a good patch. I might suggest some feedback, preferably to the deleting / renaming user but a syslog might also do, when the delete / rename failed. My patch returns IMAP_MAILBOX_NOTSUPPORTED to the client. I can add a syslog call if people think that would be useful. Is there a bugzilla number? #3220. :wes On 18 May 2010, at 12:38, Stephen Grier wrote: Just submitting a patch I'm supporting locally for consideration. We use shared mailboxes quite extensively for role-based communication. For quite some time we've had a problem with users deleting or renaming mailboxes into which we deliver mail. We can, and do, use IMAP ACLs to dissallow users from deleting the delivery target mailbox. But when a user creates a child mailbox it inherits the ACLs of the parent, and the user is then not able to delete or rename the sub folder. As a fix, I have written a patch against 2.3.16 to add a new lmtptarget mailbox annotation. When enabled, Cyrus won't allow the mailbox to be deleted or renamed. We can then set whatever ACLs we want inherited by child mailboxes, happy in the knowledge the user won't blat the mailbox and cause mail to bounce. The rationale here is that Cyrus treats user.foo with special significance as a delivery target, but does not do the same for shared mailboxes because there is no way for Cyrus to know which shared mailboxes we intend to deliver mail into. Using a mailbox annotation seems a nice way of flagging this. Patch attached. Comments welcome. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Stephen Grier Systems Developer IT Services Queen Mary, University of London Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
NULL pointer bug in 2.3.13rc3 when deleting top-level mailboxes
Hi,
While testing cyrus-imapd-2.3.13rc3, I noticed log entries like the
following when deleting a top-level mailbox:
Oct 14 17:39:21 machine master[24378]: process 24391 exited, signaled to
death by 11
The mailbox had been removed from the filesystem, but the imapd process
appears to have segfaulted. Trussing the imapd showed:
stat(/var/imap/mailboxes.db, 0xFFBFB5B0) = 0
fcntl(6, F_SETLKW, 0xFFBFB628) = 0
rmdir(/var/spool/imap/v/wobble) = 0
Incurred fault #6, FLTBOUNDS %pc = 0x00047850
siginfo: SIGSEGV SEGV_MAPERR addr=0x
Received signal #11, SIGSEGV [default]
siginfo: SIGSEGV SEGV_MAPERR addr=0x
Looking at the code there is a clear NULL pointer bug in the
mailbox_delete function in imap/mailbox.c. The patch is:
$ diff -u imap/mailbox.c.orig imap/mailbox.c
--- imap/mailbox.c.orig Wed Oct 8 16:47:08 2008
+++ imap/mailbox.c Tue Oct 14 17:49:05 2008
@@ -2919,7 +2919,9 @@
r = mboxlist_lookup(nbuf, NULL, NULL);
} while(r == IMAP_MAILBOX_NONEXISTENT);
-*ntail = '\0';
+if (ntail != NULL) {
+*ntail = '\0';
+}
if (updatenotifier) updatenotifier(nbuf);
--
Stephen Grier
Systems Developer
Computing Services
Queen Mary, University of London
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
imclient broken on Solaris 9
Dear all, We've got Cyrus-imapd-2.2.10 running on on a Solaris 9 box, and everything is fine, except for the cyradm utility, which fails with: cyradm --user admin imap-test imclient_connect: unknown host imap-test at /usr/local/lib/perl5/site_perl/5.8.5/sun4-solaris/Cyrus/IMAP/Admin.pm line 71. Having done some debugging, the problem appears to be the getaddrinfo() call of the imclient_connect() function of lib/imclient.c. It does not appear to like the AI_CANONNAME flag as defined in netdb.h, and returns EAI_BADFLAGS invalid ai_flags. If I change line 238 to: hints.ai_flags = 2; (which is how AI_CANONNAME is defined in lib/gai.h) cyradm works as expected. So the getaddrinfo() call in imclient_connect seems to expect AI_CANONNAME to be as defined in lib/gai.h rather than as defined in netdb.h. And this only seems to happen on Solaris 9, not Solaris 7/8. Does anyone know why this is happening? I have found a couple of references on this issue in the info-cyrus archives, but nothing conclusive. We can work with the above fix for now, but I'd rather know exactly what's going on here. Regards, Stephen -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus Murder patch
Dear all,
I'm submitting a patch which fixes a slightly irritating (although not
serious) problem, whereby the Cyrus Murder daemons proxyd and lmtpproxyd
do not bind to the correct interface when opening tcp connections to the
backend servers. This happens when the servername of the Murder
front-end is not the primary interface on the host. This leads to
inconsistent Cyrus logs and Received headers on delivered mail.
The patch simply adds a bind() call to the backend_connect() function of
imap/backend.c. It tries to bind the socket to the interface specified
by the servername imapd.conf option.
I'd be grateful if someone could take a look at this. It should patch OK
against the CVS HEAD.
Regards,
Stephen Grier
--
Stephen Grier
Systems Developer
Computing Services
Queen Mary, University of London
diff -Naur cyrus-imapd-2.2.10.old/imap/backend.c cyrus-imapd-2.2.10/imap/backend.c
--- cyrus-imapd-2.2.10.old/imap/backend.c 2004-10-27 17:53:35.0 +0100
+++ cyrus-imapd-2.2.10/imap/backend.c 2005-02-24 14:39:46.542462000 +
@@ -272,7 +272,7 @@
int sock = -1;
int r;
int err;
-struct addrinfo hints, *res0 = NULL, *res;
+struct addrinfo hints, *res0 = NULL, *res1 = NULL, *res;
struct sockaddr_un sunsock;
char buf[2048], *mechlist = NULL;
struct sigaction action;
@@ -315,6 +315,12 @@
free(ret);
return NULL;
}
+/* Get addrinfo struct for local interface. */
+err = getaddrinfo(config_servername, NULL, hints, res1);
+if(err) {
+syslog(LOG_ERR, getaddrinfo(%s) failed: %s,
+ config_servername, gai_strerror(err));
+}
}
/* Setup timeout */
@@ -331,6 +337,15 @@
sock = socket(res-ai_family, res-ai_socktype, res-ai_protocol);
if (sock 0)
continue;
+/* Bind to local interface. */
+if (!err) {
+if (bind(sock, res1-ai_addr, res1-ai_addrlen) 0) {
+struct sockaddr_in *local_sockaddr = (struct sockaddr_in *) res1-ai_addr;
+syslog(LOG_ERR, failed to bind to address %s: %s,
+ inet_ntoa(local_sockaddr-sin_addr), strerror(errno));
+}
+freeaddrinfo(res1);
+}
alarm(config_getint(IMAPOPT_CLIENT_TIMEOUT));
if (connect(sock, res-ai_addr, res-ai_addrlen) = 0)
break;
Re: Sieve for shared mailboxes
On Fri, 2004-03-19 at 01:09, Ken Murchison wrote: Stephen Grier wrote: What is the current situation with Sieve for shared mailboxes? I've read the discussion from last July on the cyrus-devel list, subject [PROPOSAL] Sieve for shared mailboxes, but I can't find any anything on the subject more recent than this. Is it still the intention to use the IMAP ANNOTATE extension to hold a /sieve value indicating the Sieve script to run for a particular shared mailbox? The biggest issue with sieve opn shared folders is how to use them. Are they only executed when mail is delivered via LMTP, or are they executed any time a message is inserted into the mailbox? Isn't it the case that for user-space mailboxes, sieve scripts are only applied on delivery via LMTP, and not when a message is copied or fileintod it? I don't see why the same behaviour wouldn't be adequate for shared mailboxes. Are sieve scripts inherited by subfolders, etc? This is effectively the case with user-space mailboxes, although I realise this is because sieve scripts are applied to users rather than mailboxes. Again, the same behaviour would be appropriate for shared mailboxes. This might be a bit tricky using IMAP annotations though. I think for the sort of sieve functionality we need on shared mailboxes here, I would be happy to just have lmtpd look for scripts owned by the postuser user, and apply that to all mail destined for a shared mailbox. Are there any complicating factors in implementing this? I'm looking at the deliver function of lmtpd.c, and it appears to be fairly straight forward to do this. -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Sieve for shared mailboxes
What is the current situation with Sieve for shared mailboxes? I've read the discussion from last July on the cyrus-devel list, subject [PROPOSAL] Sieve for shared mailboxes, but I can't find any anything on the subject more recent than this. Is it still the intention to use the IMAP ANNOTATE extension to hold a /sieve value indicating the Sieve script to run for a particular shared mailbox? -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Sieve script and fileinto encoding
Hi, I have a question regarding the charset encoding of Sieve scripts which I have not been to find a definite answer to elsewhere. As I understand it, Cyrus expects Sieve scripts to be UTF-8 encoded. Is this also true of mailbox names in a fileinto action? Cyrus supplies mailbox names in modified UTF-7. Would it be correct, then, to say that mailbox names need to be converted from MUTF-7 to UTF-8 when writing a Sieve script? If so, then is it the case that lmtpd coverts the mailbox name in a fileinto action back from UTF-8 to MUTF-7 when parsing the script? Also, am I right in thinking that header string matching is always done UTF-8, and non-UTF-8 mail headers are converted to UTF-8 first? Thanks, Stephen
sieve redirect mail loop causing mail to be lost
Hi, We are running Cyrus-imapd-2.1.11 with sieve and duplicate delivery suppression. One of our users complained that they had not been receiving mail. It appears that this is because he has set up a sieve script with a redirect rule which redirects all his mail to himself (his own email address). As far as I can tell, his mail is initially delivered to his Inbox, at which point lmtpd redirects the mail back via the MTA to his email address. The redirected mail is delivered a second time, at which point it appears that the mail is suppressed by Cyrus' duplicate delivery suppression. I have seen a few references to this behaviour on the list from a while back, but I'm not sure if this problem has been addressed in 2.1.11 or later. When a redirect rule redirects mail to the same mailbox in this way, will this always result in mail being silently discarded? Would this problem be solved by disabling duplicate delivery suppression? I am slightly confused by the fact that lmtpd is not actually logging that it has suppressed the message when this occurs. Yet the messages never appear in the user's Inbox. Obviously the user's sieve script is broken and I would expect it to produce a mail loop, which our MTA should handle. But it is very likely that other users will inadvertently redirect mail to themselves, and if this is going to cause mail to be lost we may have to disable duplicate delivery suppression. -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London
Re: empty line Cyrus' reject/vacation messages (was: Re: Sievewrites an empty line in headers for reject/vacation)
On Sat, 2003-07-19 at 21:41, Bernhard Erdmann wrote: Hi, if Cyrus 2.1.13 sends reject or vacation messages generated by Sieve on Exim 3.36 Linux systems and being filed back from Exim into Cyrus, a blank line is added in the middle of the header lines causing the second part of the header being treated as the body of the mail. We were seeing this with Cyrus and exim-3.22. Exim 4.20 has an option drop_cr [5] but not Exim 3.36. exim-3.22 has a -dropcr command line option. Would it be advisable to - patch Cyrus not to use \r\n as a line terminator - patch Cyrus to use a wrapper for the sendmail binary stripping \r\n down to \n - upgrade to Exim 4.20 and use the option drop_cr? We created a simple wrapper to exim using the -dropcr option and specified this in the sendmail: line of imapd.conf. -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London
Re: non-existent msgids duplicate delivery suppression
On Fri, 2003-06-27 at 14:10, Sebastian Hagedorn wrote: -- Stephen Grier [EMAIL PROTECTED] is rumored to have mumbled on Freitag, 27. Juni 2003 13:47 Uhr +0100 regarding non-existent msgids duplicate delivery suppression: I have noticed that this has been happening on other occasions, where lmtpd does a duplicate_check and then a duplicate_mark on zero length message-ids. This raises the possibility that the server is suppressing messages that are not duplicates, but merely have no message-id header. That in itself is very weird. Every MTA along the way should add a message-id if there isn't one. Or are you saying there *is* a message-id, but it's empty? That would be horribly broken, but might explain why one isn't added by the MTAs ... Yes, the messages do have a Message-Id: header with an empty value. I've identified 2 domains which appear to be doing this. Would it be reasonable to have lmtpd not add an entry to the deliver db for zero-length message-ids? -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London
read-receipts duplicate delivery suppression
Hi, We're currently running cyrus-imapd-2.1.11 with duplicate delivery suppression enabled. We recently received a complaint from a user who reported not receiving a message he was sent. On investigation, it seems that he initially sent a message to a third party along with a request for a read-receipt. The third party returned the read-receipt. It seems that when the third party then replied to the original message, lmtpd suppressed it because it contained a message-id identical to the one on the read-receipt sent earlier: Jun 10 15:23:31 machine lmtpd[15702]: duplicate_check: [EMAIL PROTECTED] user.XX 0 Jun 10 15:23:31 machine lmtpd[15702]: duplicate_mark: [EMAIL PROTECTED] user.XX 1055255011 ... Jun 10 15:56:45 machine lmtpd[17525]: duplicate_check: [EMAIL PROTECTED] user.XX 1055255011 Jun 10 15:56:45 machine lmtpd[17525]: dupelim: eliminated duplicate message to user.XX Has anyone else observed this happening? As far as I understand rfc-822 message-ids are supposed to be unique, but read-receipts are non-standard, so I don't know if the client which did this, MS Outlook, Build 10.0.3416, behaved sensibly or not. We may have to disable duplicate delivery suppression if this is likely to happen again. Thanks -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London
Re: Problems with cyrus-imapd 2.1.11 under Solaris 8
Henrique de Moraes Holschuh wrote: There is a more complete solution to the SIGCHILD problems in master, that fixes all the race conditions that cause the process count to be lost. I call it the pid morgue :-) It is in the bugzilla, and it is being used in production by the fastmail.fm people, AND all Debian users without a glitch for a long while now... Is there any reason why this SIGCHILD patch should not be applied to 2.1.11 under Solaris 8? Why is this not in recent releases of cyrus-imapd? The patch dates to May 02. http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1261 -- Stephen Grier Systems Developer Computing Services Queen Mary, University of London
lmtpd and forwarding MTA error messages
We have recently noticed strange behaviour with our cyrus-imapd-1.0.14
installation. Lmtpd does not seem to forward auto-generated error messages
properly. When a cyrus-imap user sends a mail to a non-existent address, the other
MTA will typically respond with an error message with a null envelope sender
address: ''. Suppose now that this cyrus-imap user also has a sieve forward rule
which will forward this error message to another address. In our case, when lmtpd
forwards the mail by calling sendmail, it seems to be trying to set the sender
address (with the -f option) to the empty return-path: value, generating something
like:
sendmail -f -- rcpt to ...
Consequently, the mail is sent with --@ourimap as the envelope sender. A local
MTA rejects this mail, sending another error message back to --@ourimap, which
of course does not exist. I imaging it should be forwarded with sendmail -f --
rcpt to.
I think there may be two bugs here. Firstly, in send_forward() in lmtpd.c we have:
smbuf[0] = sendmail;
if (return_path != NULL) {
smbuf[1] = -f;
smbuf[2] = return_path;
} else {
smbuf[1] = -f;
smbuf[2] = ;
}
smbuf[3] = --;
smbuf[4] = forwardto;
Should the second line not be if (return_path != '') ? I think I'm right in
saying that if the original message had an envelope sender, then return_path
will be empty here, and hence the problem.
Secondly, in savemsg() in lmtpengine.c we have:
if (!strchr(rpath, '@')) {
hostname = config_servername;
}
fprintf(f, Return-Path: %s%s%s\r\n,
rpath, hostname ? @ : , hostname ? hostname : );
which explains why messages generated by a sieve vacation rule (where the envelope
sender will be ) have:
Return-Path: @imapdomain
I think this might be fixed in the cvs, but I'm not sure.
Thanks.
--
Stephen Grier [EMAIL PROTECTED]
Systems Developer
Computing Services
Queen Mary, University of London
