Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server. [WARNING: DKIM validation failed]
On Fri, Oct 23, 2015 at 07:38:56PM -0400, Matt Elson wrote: > > So the exact text after the OK response doesn't matter, but it MUST > > be SP followed by at least 1 TEXT-CHAR. > > > > This is pretty easy to patch in 2.3.x if you're forced to remain > > there for reasons. It is, of course, fixed in later versions. > > > > Bron. > > > > > > I'm seeing this issue in my environment.. I think. I'm only *fairly* > sure - I see clients that are identifying themselves as 10.11 causing > lots of entries showing up in the sync log and a whole lot of index > churning on.. usually smaller folders, luckily. I've been trying to > track down a test machine to validate, but resources are quite limited > where I am. > > But I'm a bit confused about the current working theory as to what's > causing Mail.app to go into its strange loop. > > My Cyrus (RedHat provided 2.3.16-13.el6_6 with a very minor local patch) > responds to EXPUNGE with " OK COMPLETED" as near as I can tell.. > which seems spec compliant if I understand the above. > > * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID > MUPDATE=mupdate://redacted.example.com/ AUTH=GSSAPI AUTH=PLAIN > AUTH=LOGIN SASL-IR COMPRESS=DEFLATE] redacted.example.com Cyrus IMAP > Murder v2.3.16-Fedora-RPM-2.3.16-13.el6_6 server ready > A0001 login REDACTED PASSWORD > A0001 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID > MUPDATE=mupdate://redacted.example.com/ LOGINDISABLED AUTH=GSSAPI > AUTH=PLAIN AUTH=LOGIN COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA > MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN > MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT > THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE SCAN IDLE LISTEXT > LIST-SUBSCRIBED X-NETSCAPE URLAUTH] User logged in > A0002 SELECT Trash > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] > * 4560 EXISTS > * 0 RECENT > * OK [UNSEEN 1] > * OK [UIDVALIDITY 1347582296] > * OK [UIDNEXT 6437] > * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox > * OK [URLMECH INTERNAL] > A0002 OK [READ-WRITE] Completed > A0003 idle > + idling > done > A0003 OK Completed > A0004 EXPUNGE > * 4560 EXISTS > * 0 RECENT > A0004 OK Completed > > > In fact a 2.4.18 install I'm building (I'm using this issue as an excuse > to prioritize a long time coming upgrade) seems to respond to expunge in > the same way. > > > * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE > MUPDATE=mupdate://redacted.example.com/ STARTTLS AUTH=LOGIN AUTH=PLAIN > AUTH=DIGEST-MD5 SASL-IR] redacted.example.com Cyrus IMAP Murder > v2.4.18-Invoca-RPM-2.4.18-1.el7.centos server ready > A0001 login redacted redacted > A0001 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxte QUOTA > MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN > MULTIAPPEND BINARY CATENATE CONDSTORE SORT SORT=MODSEQ SORT=DISPLAY > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE WITHIN SCAN XLIST > URLAUTH X-NETSCAPE MUPDATE=mupdate://redacted.example.com/ LOGINDISABLED > COMPRESS=DEFLATE IDLE] User logged in > SESSIONID= > A0002 SELECT Trash > * 0 EXISTS > * 0 RECENT > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok > * OK [UIDVALIDITY 144503] Ok > * OK [UIDNEXT 1] Ok > * OK [HIGHESTMODSEQ 1] Ok > * OK [URLMECH INTERNAL] Ok > A0002 OK [READ-WRITE] Completed > A0003 idle > + idling > done > A0003 OK Completed > A0004 EXPUNGE > A0004 OK Completed > > (And fastmail's IMAP responds with a OK Completed, and I assume it > represents the most up to date Cyrus :P) > > Am I misunderstanding the theoretical issue? My IMAP-fu is a little > rusty these days and I'm a bit bleary eyed so apologies in advance if > I'm missing something obvious - I'm just testing by typing the commands > in raw since imaptest is panicking on me and I've not had time to sort > that out (though my Thunderbird debug log also shows an OK Completed as > a repsonse). > > It does seem from one of the trace logs from the Apple discussion that > there is a version of Cyrus that responds with just OK, but.. it's not > all 2.3.x 's at the very least :P. > > Sorry again if this noise and thanks in advance for any clarification > someone can give. > > Matt > Hi Matt, I just confirmed your result. It looks like Redhat had patched that in there release. I will try and get some El Capitan telemetry on Monday and see what it is doing. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server. [WARNING: DKIM validation failed]
On Fri, Oct 23, 2015 at 03:34:22PM -0500, k...@rice.edu wrote:
> On Fri, Oct 23, 2015 at 09:37:28AM +1100, Bron Gondwana wrote:
> > >
> > > If they are thinking that the example in the RFC is the specification,
> > > they are
> > > not correct. The IMAP server responses to determine completion of the
> > > EXPUNGE
> > > command are "OK" for a completed EXPUNGE, "NO" for a failure, and "BAD
> > > for unknown
> > > command or invalid arguments. Everything after those words are not
> > > germane to
> > > whether or not the command completed. The tag is what links the OK to the
> > > EXPUNGE
> > > command and not the "EXPUNGE completed". Sigh, they had it right and now
> > > it is
> > > broken.
> >
> > Actually, the ImapTest command complained about this too:
> >
> > http://imapwiki.org/ImapTest
> >
> > >From RFC3501:
> >
> > response-tagged = tag SP resp-cond-state CRLF
> >
> >
> > resp-cond-state = ("OK" / "NO" / "BAD") SP resp-text
> > ; Status condition
> >
> > resp-text = ["[" resp-text-code "]" SP] text
> >
> > text= 1*TEXT-CHAR
> >
> > So the exact text after the OK response doesn't matter, but it MUST be SP
> > followed by at least 1 TEXT-CHAR.
> >
> > This is pretty easy to patch in 2.3.x if you're forced to remain there for
> > reasons. It is, of course, fixed in later versions.
> >
> > Bron.
> >
>
> Hi Bron,
>
> I don't suppose you have a patch? If not, I will work one up.
>
> Regards,
> Ken
Hi Bron,
I should have looked at the telemetry. The Redhat version I am running
2.3.16-13.el6_6 does send the appropriate response to the EXPUNGE command,
at least for mutt. I will have to get some telemetry from an El Capitan
connection to see what it is doing:
mutt
a0010 EXPUNGE
>1445702635>a0009 OK Completed
>1445702635>* 1524 EXPUNGE
* 1601 EXISTS
* 0 RECENT
a0010 OK Completed
<14457026381445702638>a0011 OK Completed
>1445702638>* BYE LOGOUT received
a0012 OK Completed
I will dig into it further on Monday.
Regards,
Ken
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server.
On Fri, Oct 23, 2015 at 10:59:08PM +0200, Eric Luyten wrote: > On Tue, October 13, 2015 3:26 pm, k...@rice.edu wrote: > > > The El Capitan Mac Mail was recently released. It definitely has a problem. > > We saw our typical sync log size go from 10k when busy to 100k+. We are > > going > > to recommend that people wait to upgrade to that release until the problem > > is > > addressed. > > > > Regards, > > Ken > > > Earlier today I used DTrace to capture I/O operations on our Solaris 10 mail > server. > The imapd processes hammered by those El Capitan Mail.app clients perform > roughly 500 (five hundred) cyrus.index + cyrus.cache rewrites per second. > Multiply by the number of affected users, now numbering beyond one hundred. > Cool. > > > Eric. > Hi Eric, A bug in cyrus-imapd version 2.3.x. I am looking into generating an rpm with a patch. In the meantime, we are disabling the automatic folder compaction option to keep from hammering the server. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server. [WARNING: DKIM validation failed]
On Fri, Oct 23, 2015 at 09:37:28AM +1100, Bron Gondwana wrote:
> >
> > If they are thinking that the example in the RFC is the specification, they
> > are
> > not correct. The IMAP server responses to determine completion of the
> > EXPUNGE
> > command are "OK" for a completed EXPUNGE, "NO" for a failure, and "BAD for
> > unknown
> > command or invalid arguments. Everything after those words are not germane
> > to
> > whether or not the command completed. The tag is what links the OK to the
> > EXPUNGE
> > command and not the "EXPUNGE completed". Sigh, they had it right and now it
> > is
> > broken.
>
> Actually, the ImapTest command complained about this too:
>
> http://imapwiki.org/ImapTest
>
> >From RFC3501:
>
> response-tagged = tag SP resp-cond-state CRLF
>
>
> resp-cond-state = ("OK" / "NO" / "BAD") SP resp-text
> ; Status condition
>
> resp-text = ["[" resp-text-code "]" SP] text
>
> text= 1*TEXT-CHAR
>
> So the exact text after the OK response doesn't matter, but it MUST be SP
> followed by at least 1 TEXT-CHAR.
>
> This is pretty easy to patch in 2.3.x if you're forced to remain there for
> reasons. It is, of course, fixed in later versions.
>
> Bron.
>
Hi Bron,
I don't suppose you have a patch? If not, I will work one up.
Regards,
Ken
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server. [WARNING: DKIM validation failed]
On Thu, Oct 22, 2015 at 10:54:28PM +0200, Eric Luyten wrote: > On Thu, October 22, 2015 10:39 pm, k...@rice.edu wrote: > > On Thu, Oct 22, 2015 at 10:32:40AM -0400, bacon wrote: > > > >> Apple released OS 10.11.1 update yesterday. It shows that it includes, > >> > >> > >> - Fixes an issue where outgoing server information may be missing from > >> Mail > >> - Resolves an issue that prevented display of messages and mailboxes in > >> Mail > >> > >> > >> Does anyone know if this fixed the problem. I've asked my users not to > >> use Mail on El Capitan until this bug is fixed. > >> > >> Meanwhile, I'm working in my spare time to upgrade my cyrus-imapd on > >> RedHat Enterprise Linux 6 from the 2.3 series distributed by RedHat to > >> the 2.5 series which I'll have to build and maintain myself. That's an > >> ugly > >> job and one that I don't really want to do. > >> > >> Fred > >> > >> > > > > Hi Fred, > > > > > > It does not appear to have addressed the problem. We are trying to gather > > more information. > > > In > >https://discussions.apple.com/thread/7267399 > > someone states older Cyrus servers are returning a syntactically not quite > correct response to the EXPUNGE command but I'm having a hard time buying > this for an explanation. > > Oh, and someone in there asks for help to get the Apple bug report (22996765) > escalated. > > I think at our site we have reached the count of 100 (one hundred) El Capitan > Mail.app users. > > > Eric. > Hi Eric, If they are thinking that the example in the RFC is the specification, they are not correct. The IMAP server responses to determine completion of the EXPUNGE command are "OK" for a completed EXPUNGE, "NO" for a failure, and "BAD for unknown command or invalid arguments. Everything after those words are not germane to whether or not the command completed. The tag is what links the OK to the EXPUNGE command and not the "EXPUNGE completed". Sigh, they had it right and now it is broken. Regards, Ken > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server. [WARNING: DKIM validation failed]
On Thu, Oct 22, 2015 at 10:32:40AM -0400, bacon wrote: > Apple released OS 10.11.1 update yesterday. It shows that it includes, > > - Fixes an issue where outgoing server information may be missing from > Mail > - Resolves an issue that prevented display of messages and mailboxes in > Mail > > Does anyone know if this fixed the problem. I've asked my users not to > use Mail on El Capitan until this bug is fixed. > > Meanwhile, I'm working in my spare time to upgrade my cyrus-imapd on > RedHat Enterprise Linux 6 from the 2.3 series distributed by RedHat to > the 2.5 series which I'll have to build and maintain myself. That's an > ugly job and one that I don't really want to do. > > Fred > Hi Fred, It does not appear to have addressed the problem. We are trying to gather more information. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server.
On Wed, Oct 14, 2015 at 07:43:47AM +1100, Bron Gondwana wrote: > On Wed, Oct 14, 2015, at 00:26, k...@rice.edu wrote: > > The El Capitan Mac Mail was recently released. It definitely has a problem. > > We saw our typical sync log size go from 10k when busy to 100k+. We are > > going > > to recommend that people wait to upgrade to that release until the problem > > is > > addressed. > > I'll look into whether this is a problem (or as much of a problem) on later > releases! > Just to be clear - have you tracked the issue down to the EXPUNGE command > using > IO? > > (It's unlikely that 2.3 will be patched to make EXPUNGE more efficient unless > someone > else wants to do it. I've already spent a year fixing that code, and it's > called 2.4) > > Bron. Hi, It has not been much of a problem here, but out servers are spec-ed for peak load and not average load. In the sync log, there are hundreds of lines like: 202 MAILBOX user.xxx.Processed 217 MAILBOX "user.yyy.Deleted Messages" 219 MAILBOX "user.zzz.Sent Messages" all from El Capitan Mac Mail. The sync process collapses most of the duplicates so the replication is less impacted than the extra work the primary has to do. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server.
On Tue, Oct 13, 2015 at 11:13:43AM +0200, Eric Luyten wrote: > Howdy, > > > In recent weeks I have noticed a number of clients entering seemingly > endless loops writing the cyrus.index on a sometimes empty mailbox, > resulting in several dozen MB/s writing to our mail spool for hours on > end. > > Always IMAP, never POP, sometimes TCP/143, mostly TCP/995 > > I enabled telemetry logging on several of these accounts and here's a > typical snippet : > > <1444726821<1<1444726821<804.3507 IDLE > >1444726821>+ idling > <1444726821 >1444726821>1804.3507 OK Completed > <1444726821<1<1444726821<805.3507 EXPUNGE > >1444726821>* 0 EXISTS > * 0 RECENT > 1805.3507 OK Completed > <1444726821<1<1444726821<806.3507 IDLE > >1444726821>+ idling > <1444726821 >1444726821>1806.3507 OK Completed > <1444726821<1<1444726821<807.3507 EXPUNGE > >1444726822>* 0 EXISTS > * 0 RECENT > 1807.3507 OK Completed > <1444726822<1<1444726822<808.3507 IDLE > >1444726822>+ idling > <1444726822 >1444726822>1808.3507 OK Completed > <1444726822<1<1444726822<809.3507 EXPUNGE > >1444726918>* 0 EXISTS > * 0 RECENT > 1809.3507 OK Completed > > > > Clues ? > > > Eric Luyten, Computing Centre VUB/ULB. > Hi Eric, The El Capitan Mac Mail was recently released. It definitely has a problem. We saw our typical sync log size go from 10k when busy to 100k+. We are going to recommend that people wait to upgrade to that release until the problem is addressed. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus aggregate compatibility.
On Mon, Apr 20, 2015 at 05:11:00PM -0400, Michael D. Sofka wrote: > Under the scenario, would 2.5 work better? > > Mike > Hi Mike, In our case, the unconstrained I/O caused by the mandatory mailbox format conversion on first use would have necessitated a prolonged service outage to prevent overloading the system. 2.5 will allow you to schedule your conversions while the system is functional. This may not be a concern for you. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus aggregate compatibility.
On Mon, Apr 20, 2015 at 04:23:07PM -0400, Michael D. Sofka wrote: > We currently have: > >Cyrus Front-End servers running 2.2.12 >Cyrus Back-End running 2.3.16 > > I have built a new back-end server running 2.4.17. > > > I plan on adding the new, 2.4 server, to the aggregate, and move all the > mailboxes. I would rather not upgrade the front-end servers, since they > are be retired (and replaced by perdition or nginx). This move > motivated by the status of the old SAN disks. > > I know that there is a NOSELECT issue, but any shared folders will be > moved as a group. Are there other issues? > > Specifically, is the proxy/mupdate protocol compatible? > > I'm ready to add the new back-end to the aggregate, and run some tests. > Thought it might be good to ask here first. > > Thank You. > > Mike > Hi Mike, I am just curious, but why version 2.4 and not version 2.5? Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sieve vacation with start and end date
On Tue, Dec 30, 2014 at 09:25:48AM -0500, James Cassell wrote:
>
> On Tue, Dec 30, 2014, at 08:51 AM, k...@rice.edu wrote:
> > Quoting Adam Tauno Williams :
> >
> > > On Sun, 2014-12-28 at 10:20 +0100, Marcus Schopen wrote:
> > >> does sieve vacation understand a start and end date? Something like this
> > >> does not work:
> > >> ---
> > >> require ["date", "relational", "vacation"];
> > >> if allof(currentdate :value "ge" "date" "2007-06-30",
> > >> currentdate :value "le" "date" "2007-07-07")
> > >> { vacation :days 7 "I'm away during the first week in July."; }
> > >> ---
> > >> System: cyrus 2.4.12 on Ubuntu 12.04 LTS
> > >
> > > It may or may not; depends on what extensions/plugins are activated in
> > > your SIEVE. Is the above documented syntax from somewhere?
> > >
> > > Horde's Ingo application uses regular expressions to match dates in
> > > order to implement vacation start/end. I believe date matching in SIEVE
> > > is a relatively recent thing, and I am not sure to what level it is
> > > implemented [anywhere].
> > >
> > Cyrus sieve does not have the date extension. I wish it did. :)
> >
>
> cyrus 2.5 will have the date extension. It has already been implemented in
> the master git branch. Likely, it could be backported to 2.4 if anyone is up
> to the task.
>
> > Regards,
> > Ken
> >
Very, very cool!
Regards,
Ken
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: High avaliabilty for IMAP/PROXY
On Mon, Sep 22, 2014 at 03:09:13PM +0200, Marc Patermann wrote: > k...@rice.edu schrieb (18.09.2014 21:43 Uhr): > > > >These are all located behind our Citrix Netscaler boxes. You should > >be able to replicate their function with either haproxy or nginx. > What does the Netscaler do in this scenario? > > Marc > The Netscaler provides redundant and load-balanced access to the IMAP/POP3 backends with automated fail-over. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: High avaliabilty for IMAP/PROXY
On Thu, Sep 18, 2014 at 12:07:57PM -0700, Vincent Fox wrote: > On 9/18/2014 11:58 AM, Fabio S. Schmidt wrote: > > Hi, > > > > - Sorry if it seems to be a little off-topic - > > > > We have deployed Cyrus Aggregator and currently we provide load > > balancing and high availability for the Cyrus Front Ends through DNS. > > With this scenario, if a Frontend is unavailable it will receive > > connections unless we remove it from the DNS record for the IMAP > > service. > > > > Does anyone have any better ideas to improve the high availability? I > > was wondering about using HAPROXY vs NGINX but I do not know their > > behaviours in cases like I mentioned above. > > > We have for about 8 years used Perdition for POP/IMAP proxy. > > 3 simple Linux boxes in a load balanced pool. > > Friends don't let friends do Round Robin DNS. You can't count > on removing DNS entries, since propagation can be very slow and > some clients don't even respect TTL. > We also used Perdition here for our POP3/IMAP proxy. Unfortunately, its process per connection resulted in an enormous resource footprint when everyone was connected to the server. In addition, the startup stampede of processes completely swamped the frontends crippling the performance until a steady state was reached. As a result, we moved to using NGINX as our POP3/IMAP proxy. Now a single-box can carry the connection load that 4 or more boxes struggled with along with better responsiveness and performance to boot. These are all located behind our Citrix Netscaler boxes. You should be able to replicate their function with either haproxy or nginx. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: SIEVE vacation not works with "email aliases"
On Fri, Apr 04, 2014 at 04:26:43PM +0200, Niels Dettenbach wrote:
> Hi all,
>
>
> it seems i'm not alone with this "effect" or "bug" in the SIEVE vacation
> module.
>
> On a cyrus-imap ("feeded" by EXIM) with virtual domains where usernames are
> "equal" to email adresses i have the effect, that vacations are not sent out
> when an incoming email is addressed to an "alias" of the mailbox (means to an
> email address which is NOT the same as the IMAP user name / mailbox name).
>
> On another machine where user accounts arr in the format user.domain.tld
> (instead of u...@domain.tld) under NetBSD it works without any problems. So i
> assume slightly that it may be related to the aliasing mechanism.
>
> There is no change if i catch all incoming emails or define each by hand in
> sieve.
>
>
> Is anyone out there faced to the same problem? Is there any solution?
>
>
> many thanks in advance.
> cheerioh,
>
> Niels.
> --
Hi Niels,
You need to include the addresses that the vacation is to reply to. Here is
an example:
require ["vacation","regex"];
vacation :days 7 :addresses ["addre...@example.com", "addre...@example.com"]
text:
testing vacation
.
;
Regards,
Ken
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: IMAP proxy recommendations.
On Thu, Dec 12, 2013 at 01:11:19PM -0500, sofkam wrote: > I am in the process of replacing our Cyrus Front-End servers with a > proxy server that can front both Exchange and Cyrus IMAP, as well as > future IMAP hosts as we move and migrate users. The two that stand out > are: Perdition, and NGinX. I'm looking for recommendations, or > experience with either, or additional suggestions to examine. > > Initially it will replace the old (and back-leveled) Cyrus Front-End > servers in our murder aggregate, and I would like that process to be > smooth and transparent. But the proxy will then be switching back-ends > based on Exchange vs Cyrus hosted IMAP (Currently Exchange users are > told to configure their clients with a different server, a practice we > would like to discontinue.) We will also be migrating users (usually > from Cyrus to Exchange, but the other direction happens as well), and we > would like this process to be as transparent as possible. Finally, our > Webmail server will also be using the proxy, so a proxy that maintains > state-full connections would be a plus, but this takes second place to > location transparency. > > Because of the need to host multiple back-ends based on account, I have > not so far been considering imapproxy, even though this provides the > state-full connections. > > Any thoughts or suggestions welcome. > > Thank You, > Mike > > -- > Michael D. Sofka sof...@rpi.edu > C&MT Sr. Systems Programmer, Email, TeX, Epistemology > Rensselaer Polytechnic Institute, Troy, NY. > http://www.rpi.edu/~sofkam/ Hi Mike, We started with the perdition IMAP proxy and while it worked well, its process-based architecture resulted in a very heavy resource footprint -- a process per connection. When we upgraded our frontends we moved to nginx as the IMAP proxy, and it had a much lower resource footprint, a thread per connection, and better high load performance. We also use imapproxy to provide a stateful connection for our webmail system to our IMAP proxy. We can definitely recommend this setup. In fact, when we moved our student to Google mail, the transition was made without the need to reconfigure any end user clients and our webmail worked as well. Please let me know if you have any questions about our use. Regards, Kenneth Marshall, PhD -- Mgr./Middleware, Infrastructure & Development k...@rice.edu / 713-348-5294 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Sieve based a day of week
On Wed, Sep 18, 2013 at 08:20:07PM +0200, Stefan G. Weichinger wrote: > Am 13.03.2013 15:04, schrieb André Schild: > > Hello, > > > > we have a customer where they have two persons working each 4 days a week. > > > > On friday in the email of User1 there should be a auto answer for friday > > On Monday in the mail of User2, there should be a auto answer for monday > > > > I think this should be possible when RFC 5260 is implemented, > > but according to this, we don't have it yet. > > > > Is there another way I could activate/deactivate the auto answers > > on day-per-week automatically ? > > > > I also need that ... any answer found yet? > > AFAI googled the "date" extension would be able to do that ... but I > don't have that in my gentoo installation. > > Stefan > Hi Andre, You should be able to use a regex on the Date: header to get the day of the week specific reply. Cheers, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus IMAP 2.3 EOL?
On Wed, Nov 21, 2012 at 01:51:28PM -0500, Adam Tauno Williams wrote: > On Wed, 2012-11-21 at 18:21 +0100, Bron Gondwana wrote: > > On Wed, Nov 21, 2012, at 05:31 PM, Egoitz Aurrekoetxea Aurre wrote: > > > Good morning, > > > Does Cyrus IMAP 2.3.18 get still supported by Cyrus IMAP team?. > > Just security updates really, why? > > Cyrus IMAP 2.4.x is much nicer all around; if you are on 2.3.x I'd > recommend upgrading. Other than some re-indexing pain, it is seemless > and leaves you on a considerably better product. > Hi Adam, I would love to be running 2.4.x. Unfortunately, because replication does not work from old version 2.3.x to newer 2.4.x "some re-indexing pain" is from everything I have read here a completely uncontrolled I/O storm on your disk subsystem unless you have a prolonged service outage. I really, really, really hope that 2.4.x will be able to replicate to 2.5.x. Then you can upgrade the relatively unloaded replica, fail-over to the new version and then upgrade the replica for a virtually seamless procedure. Now we are looking for idle usage periods together with a folder tickle script to stagger the upgrade process I/O. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: CAPABILITY response in banner, how to disable it?
On Wed, Oct 31, 2012 at 02:12:36PM +0100, Michael Neumann wrote: > Hello, > > we recently switched from imapd version 2.2.12 to 2.4.12. Now my > cellphone with bada-os 2.0 wont use the idle feature anymore. I assume > the problem lies in the change that happened in version 2.3.4, the > changelog states: > "Implemented CAPABILITY response in banner and after authentication." > > The old version responded something like this: > > TLS connection established: TLSv1 with cipher AES256-SHA (256/256 bits) > > S: * OK Cyrus IMAP4 v2.2.13-Debian-2.2.13-13ubuntu3 server ready > > C: C01 CAPABILITY > > S: * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS ID > > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT > > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE IDLE AUTH=PLAIN SASL-IR > > S: C01 OK Completed > > Please enter your password: > > The new version responds like this: > > TLS connection established: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 > > bits) > > S: * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN AUTH=LOGIN > > SASL-IR] mail Cyrus IMAP v2.4.12-Debian-2.4.12-2 server ready > > Please enter your password: > > C: A01 AUTHENTICATE PLAIN string > > S: A01 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxte QUOTA > > MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN > > MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ SORT=DISPLAY > > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE LIST-EXTENDED WITHIN > > QRESYNC SCAN XLIST URLAUTH URLAUTH=BINARY LOGINDISABLED IDLE] Success (tls > > protection) > > So it seems since the change in version 2.3.4 the imapd announces some > CAPABILITYs already in the greeting/banner, but only a subset of the > CAPABILITYs missing "idle" for example. The full CAPABILITYs string is > presented after login. Now i guess that is the reason that i cant select > the idle feature (push-sync) on my bada 2 device anymore. There is the > option "serverinfo" in imapd.conf, but using this option has no > influence on the CAPABILITYs string in the greeting. Is there a way to > return to the old behaviour, or is there a good reason not to return to > the old behaviour? > > Best regards > Michael Hi Michael, The new behavior, if something that changed 7 years ago when 2.3.0 was released, announces the available capabilities. Until you have logged in, you actually cannot use IDLE. Then when you have logged in to the server, the additional available capabilities are displayed. The previous behavior allowed unauthenticated accesses to gain information about the underlying services and systems that could be used to facilitate an exploit. The new behavior is a big improvement but I do understand the problem of working with poor client-side implementations. :( Maybe you could put a proxy in front of the server that could provide the previous behavior just for the use of your device. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
AUTHENTICATE PLAIN and authz
Hi Cyrus community,
I am having a problem getting AUTHN/AUTHZ to work with a cyrus
priviledged user. It fails to authenticate. Using LOGIN it works
but that does not allow you to proxy. I have the account listed
in proxyservers:
imapd.conf-
proxyservers: bigadmin
imapd.conf-
Then with telnet:
1 AUTHENTICATE PLAIN
+
base64{bigadmin\0bigadmin\0bigadminpassword}
1 NO authentication failure
2 LOGIN bigadmin bigadminpassword
2 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED AUTH=PLAIN
COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS
NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE SCAN
LISTEXT LIST-SUBSCRIBED URLAUTH] User logged in
This works fine with a normal user:
1 AUTHENTICATE PLAIN
+
base64{user\0user\0userpassword}
1 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED COMPRESS=DEFLATE
ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME
UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT
THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE SCAN LISTEXT LIST-SUBSCRIBED
URLAUTH] Success (tls protection)
Does anyone have any ideas about how to debug this problem?
Thank you,
Ken
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: 4096 file descriptors
On Wed, Aug 22, 2012 at 02:43:09PM -0400, Ron Vachiyer wrote: > Quick question about filedescriptors. On Centos6, cyrus 2.3.16 seems to be > able to open 4096 FDs ; > > master[27121]: retrying with 4096 (current max) > > ulimit -a says 1024; > > open files (-n) 1024 > > I am looking to increase this, and have found some documentation saying to > increse file-max in /proc. However, file-max already has a much larger > number; > > cat /proc/sys/fs/file-max > 1201105 > > The only way I have found so far is to add a ulimit -n 8192 in > /etc/rc.d/init.d/cyrus-imapd > > > Is there a more generic/cleaner way to do this? > > Thanks, > > Ron > > Hi Ron, I do not know if this is applicable to Centos6, but Redhat 6 shipped with this file: /etc/security/limits.d/90-nproc.conf # Default limit for number of user's processes to prevent # accidental fork bombs. # See rhbz #432903 for reasoning. * softnproc 65535 except the limit was 1024. Argh! We bumped it here and we were good. Note, do NOT remove the file or a patch/update will re-create it with the expected negative consequences for a busy system. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Compiling latest version for Redhat 6.
On Tue, Mar 20, 2012 at 02:14:16PM -0400, Mark London wrote: > Hi - I'm about to try installing the latest version of Cyrus on Redhat > 6, and wanted to get any feedback from anyone, about any problems that I > might run into. > > The reason I'm dgo this, is that we've recently been experiencing a > problem with the older supplied version of cyrus that comes with > redhat. Occasionally, a user's account will get "stuck". It appears it > might involve the user's ".seen" file, as attempts to access the user's > account gets stuck in the code that tries to open that file. The > problem might have not be "new", but we're only seeing it now, because > of much heavier usage of the mail server. > > I don't have the time right now to fully debug the problem, so I figured > it might be easier to first simply install the latest version. I have > compiled it on a test redhat 6 server, and it seems to work fine. I'll > be porting to our real server tonight, but I wonder if there are any > "gotchas", that I might run into, that won't be noticable until it > actually starts to heavy use. > > Thanks. - Mark > Hi Mark, Are you talking about the latest point release of the cyrus 2.3 that ships with Redhat 6 or the latest 2.4 release? A major version bump has many upgrade implications and should be avoided. We are running the same Redhat 6 release here and we have seen a problem like this being caused by the stalled pop3d process. The easy fix was to kill the pop3d process that was holding the lock and then everything started to work again. A simple check is to kill all the old pop3d processes since pop3 does not maintain persistent connections, the old ones are the culprit. You can use 'lsof -p process-id' to see which cyrus user it is locked on. That is certainly a lot simpler than performing a software upgrade on your production environment. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Minimum days between sieve vacation responses
On Fri, Nov 04, 2011 at 07:59:07AM +0100, Bron Gondwana wrote: > On Fri, Nov 04, 2011 at 12:22:54PM +0530, Ram wrote: > > Can I configure sieve to send vacation responses for every message .. > > rather than waiting for "n" days before responding again to the same sender > > Ouch - why? There's a danger of mail loops, amongst other things. > > Bron. Double ouch, ouch! That is a really, really, REALLY bad idea. We actually made a work-around within the mail system to make this happen and sooner or later, in EVERY case, we had a massive mail loop or other type of self- inflicted DoS against our mail infrastructure. In most of them, the users themselves received so much mail that their mail clients were unable to handle the volume and mail administrators were needed to clean things enough for them to get back to work. The moral is, if you need this, the implementation is far from trivial and you will need to implement internal controls and feedback loops to prevent these types of problems. Unless you are legally required to do this, figure out another way. Cheers, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus Postfix LMTP tuning
On Fri, Oct 14, 2011 at 11:01:13AM -0700, Andrew Morgan wrote: > On Fri, 14 Oct 2011, Henrique de Moraes Holschuh wrote: > > > On Thu, 13 Oct 2011, John Madden wrote: > >>> Our Postfix relays (there are 3) seem to make one lmtp connection per > >>> message, rather than sending multiple messages down a single connection. > >>> > >>> Do any Cyrus+Postfix users out there have tuning recommendations? I see a > >>> lot of postfix lmtp_* config options, but I know little about Postfix. > >> > >> I make no guarantees about these, but try this out: > >> > >> lmtp_destination_concurrency_limit = 50 > >> lmtp_destination_recipient_limit = 5000 > >> lmtp_connection_cache_on_demand = no > >> lmtp_data_done_timeout = 3600s > >> > >> It's been a couple years, but I definitely had oddball issues with LMTP > >> under load and somewhere along the line these options fixed them. YMMV. > >> The usual RTFineM about these applies too: > >> http://www.postfix.org/postconf.5.html > > > > You should name an specific LMTP transport for cyrus deliveries in postfix' > > master.cf. That way, you can use _ to > > configure it, without affecting other potential users of the same transport > > binary. > > > > E.g. if you created a cyruslmtp transport (which uses the lmtp binary), you > > could have: > > > > cyruslmtp_destination_concurrency_limit = 50 > > cyruslmtp_destination_recipient_limit = 5000 > > cyruslmtp_connection_cache_on_demand = no > > cyruslmtp_data_done_timeout = 3600s > > > > Obviously you must not do something that would make it impossible to batch > > the deliveries into a single ESMTP/LMTP transaction. If you use a mailing > > list, that means you must not enable VERP. > > > > You also want to let cyrus hardlink multi-recipient deliveries when possible > > (but watch out for your backups if they duplicate hardlink files, backup > > space will be much larger than spool space): > > > > in imapd.conf, add: > > > > singleinstancestore: 1 > > Thanks for the tips John and Henrique. Our Postfix admin is going to take > a look. Do you have any guidelines for setting the number of lmtp > children allowed on the Cyrus side? We have 3 frontends set to > maxchild=25 each. Is it better to keep maxchild low, or would throughput > improve with a higher limit? > > Thanks, > Andy Also, if you are using an aliases table for delivery in postfix you will not take advantage of the single instance store because the aliases table is handle by the local delivery agent which delivers everything one addess at a time. You will need to use virtual aliases to take advantage of the single instance store. We are still in the process of moving to virtual aliases from the regular aliases table for that reason. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Index upgrade problems after cyrus upgrade
On Thu, Oct 13, 2011 at 07:46:17AM +0200, Bron Gondwana wrote: > On Thu, Oct 13, 2011 at 07:00:42AM +0530, Ramprasad wrote: > > On 10/13/2011 6:39 AM, Adam Tauno Williams wrote: > > > On Thu, 2011-10-13 at 05:33 +0530, Ramprasad wrote: > > >> I just upgraded cyrus on a busy Imap server from 2.3.x to 2.4.6 , > > >> problem is with lmtp delivery. > > >> All lmtp deliveries from postfix are hanging forever. > > >> I think this is due to the "Index upgrade" that is going on .. Is there > > >> a way I can speed up the index upgrade > > >> I have 2000 users and only 200 users index upgrade is complete till now > > >> in last 4 hours > > > Are you letting index reconstructs happen on-the-fly or are you doing a > > > reconstruct of the mailbox(es). A reconstruct will rebuild the indexes. > > > > I just sent a mail to all mailboxes , so lmtp delivery will do a Index > > upgrade. > > Reconstructing will be much heavier , I think. > > > > Is there a way I can do a Index-upgrade only ? > > > > Also I have a few more servers to upgrade .. but if this is going to > > take so long , upgrade may not be possible. Only if I could upgrade the > > Index before really upgrading cyrus then this would work > > Actually, reconstruct isn't much heavier. > > Bloody 2.4.6, it's half a year old now. There are quite a few known bugs > solved since, but your upgrade cost isn't one of them. The only real > way to do save the upgrade cost would be a delayed rebuild, which is... > tricky. > > Bron. Yes, the upgrade problem is what is keeping us on 2.3.16 because of the impact on production services. I had hoped that we could replicate from 2.3.16 to 2.4.x which would hide the rebuild cost on the non-production replica and then failover to the replica to finish the upgrade. If you can think of any way to keep from killing production imap services when upgrading, it would be really helpful. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
