Re: messages removed before expire
> On Wed, Sep 17, 2014, at 08:24 AM, Stephen Ingram wrote: >> On Tue, Sep 16, 2014 at 3:02 PM, Bron Gondwana wrote: >> >> Dumb question - but I don't suppose anything happened to your clock recently? >> >> More non-dumb question, what version of Cyrus? >> >> Sorry, I should have included the version. I'm using 2.4.16 rpms from Invoca >> Systems. Nothing has happened with the clock. This is the only mailbox that >> seems to have the problem too, user.ken.Trash. We are running a murder >> configuration with 2 backends and 2 front ends. Not may users accounts, just >> large mailboxes. Maybe there is some setting on the mupdate master or >> somewhere else that is set for 14 days? > > I don't think so. Do you have any entries in the syslog showing the deletes? > I'm pretty sure 'auditlog: 1' works in imapd.conf in 2.4.16, so you should > be able to add that to see more logging. Just yet again one more dumb suggestion: are you sure it’s not on the clients side? Some “delete in 14 days” setting in the mail client? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: High avaliabilty for IMAP/PROXY
On Thu, Sep 18, 2014 at 12:07:57PM -0700, Vincent Fox wrote: > On 9/18/2014 11:58 AM, Fabio S. Schmidt wrote: > > Hi, > > > > - Sorry if it seems to be a little off-topic - > > > > We have deployed Cyrus Aggregator and currently we provide load > > balancing and high availability for the Cyrus Front Ends through DNS. > > With this scenario, if a Frontend is unavailable it will receive > > connections unless we remove it from the DNS record for the IMAP > > service. > > > > Does anyone have any better ideas to improve the high availability? I > > was wondering about using HAPROXY vs NGINX but I do not know their > > behaviours in cases like I mentioned above. > > > We have for about 8 years used Perdition for POP/IMAP proxy. > > 3 simple Linux boxes in a load balanced pool. > > Friends don't let friends do Round Robin DNS. You can't count > on removing DNS entries, since propagation can be very slow and > some clients don't even respect TTL. > We also used Perdition here for our POP3/IMAP proxy. Unfortunately, its process per connection resulted in an enormous resource footprint when everyone was connected to the server. In addition, the startup stampede of processes completely swamped the frontends crippling the performance until a steady state was reached. As a result, we moved to using NGINX as our POP3/IMAP proxy. Now a single-box can carry the connection load that 4 or more boxes struggled with along with better responsiveness and performance to boot. These are all located behind our Citrix Netscaler boxes. You should be able to replicate their function with either haproxy or nginx. Regards, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: High avaliabilty for IMAP/PROXY
Thanks Vincent and Simon for the answers ! I am studying the Perdition and LVS-DR solutions ! Kind regards, Fabio On 18 September 2014 16:38, Simon Amor wrote: > On 18 Sep 2014, at 19:58, Fabio S. Schmidt wrote: > >> Hi, >> >> - Sorry if it seems to be a little off-topic - >> >> We have deployed Cyrus Aggregator and currently we provide load >> balancing and high availability for the Cyrus Front Ends through DNS. >> With this scenario, if a Frontend is unavailable it will receive >> connections unless we remove it from the DNS record for the IMAP >> service. >> >> Does anyone have any better ideas to improve the high availability? I >> was wondering about using HAPROXY vs NGINX but I do not know their >> behaviours in cases like I mentioned above. > > We use LVS-DR with a cluster of 3 Cyrus pop/imap servers where servers 1 and > 2 use heartbeat to failover the inbound IP in the event of an outage. They > also handle outbound authenticated SMTP, again with LVS-DR. > > http://www.linuxvirtualserver.org/VS-DRouting.html > > Simon > -- > Simon Amor > Daily Internet Services Ltd > T: +44 (0)115 973 7260 > W: http://www.daily.co.uk/ > -- My best regards, Fabio Soares Schmidt Linux Professional Institute - LPIC-3 Microsoft Certified Technology Specialist: Active Directory Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: High avaliabilty for IMAP/PROXY
On 9/18/2014 11:58 AM, Fabio S. Schmidt wrote: > Hi, > > - Sorry if it seems to be a little off-topic - > > We have deployed Cyrus Aggregator and currently we provide load > balancing and high availability for the Cyrus Front Ends through DNS. > With this scenario, if a Frontend is unavailable it will receive > connections unless we remove it from the DNS record for the IMAP > service. > > Does anyone have any better ideas to improve the high availability? I > was wondering about using HAPROXY vs NGINX but I do not know their > behaviours in cases like I mentioned above. > We have for about 8 years used Perdition for POP/IMAP proxy. 3 simple Linux boxes in a load balanced pool. Friends don't let friends do Round Robin DNS. You can't count on removing DNS entries, since propagation can be very slow and some clients don't even respect TTL. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
High avaliabilty for IMAP/PROXY
Hi, - Sorry if it seems to be a little off-topic - We have deployed Cyrus Aggregator and currently we provide load balancing and high availability for the Cyrus Front Ends through DNS. With this scenario, if a Frontend is unavailable it will receive connections unless we remove it from the DNS record for the IMAP service. Does anyone have any better ideas to improve the high availability? I was wondering about using HAPROXY vs NGINX but I do not know their behaviours in cases like I mentioned above. -- My best regards, Fabio Soares Schmidt Linux Professional Institute - LPIC-3 Microsoft Certified Technology Specialist: Active Directory Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How to improve mail search from iPad and other mobile devices
Hi the performance hit with usage of incremental squatter update should not by so high as with full reindex. Cheers Kleo -- _ | You have moved the mouse. # | Windows must be restarted for the changes to take effect. # | # ##/ ~~ ~~ ~~ ~~ ~~ ~~ ~~ Vladimir `KLEO' Klejch Kleo'at'netbox.cz ... ... ... ... Hi, Quoting Sebastian Hagedorn : Hi, [...] Search for addresses and subject should in theory be reasonably fast because the results should mostly come from the cache. We also use squatter, which should help with the body searches. However, each mailbox is currently squatted roughly every 24 hours. With busy mailboxes I presume that leaves a large window where search will have to fall back to searching every message in that mailbox, right? AFAIK the index created by squatter is used to exclude mails that don't contain the search pattern, so an outdated index is still of use. Only the new messages and the files the index could not exclude will be searched. Running squatter more often will result in more IO traffic during the daytime. We are looking into using metapartitions to move all the relevant files to SSD to make access to cache and squat files faster, but that won't help with "outdated" squat files, right? Would it make sense to squat his mailboxes more often? Other suggestions? Cheers Sebastian -- .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:. M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How to improve mail search from iPad and other mobile devices
Hi Bron, It sounds remarkably like you want the full text xapian indexing that exists in the FastMail branch! how does that differ from what squatter does, and will it be part of the 2.5 release you're currently planning? Thanks Sebastian -- Sebastian Hagedorn - Weyertal 121, Zimmer 2.02 Regionales Rechenzentrum (RRZK) Universität zu Köln / Cologne University - Tel. +49-221-470-89578 p7sho9nM7C3AI.p7s Description: S/MIME cryptographic signature Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How to improve mail search from iPad and other mobile devices
On Thu, Sep 18, 2014, at 11:51 PM, Sebastian Hagedorn wrote: > Hi, > > my boss has tasked me with improving the performance of IMAP searches using > mail clients on mobile devices, mostly Apple's Mail on iPhone and iPad, > because that's what he uses. The bad performance he experiences is > exacerbated by the fact that he has one of the largest mail accounts we > have: currently he has 23 GB of mails ... at least they're not all in his > INBOX – he has about 1,750 folders. > > According to Apple's manual, Mail searches all folders like this: > "Searching looks at the address fields, the subject, and the message body." > > Search for addresses and subject should in theory be reasonably fast > because the results should mostly come from the cache. We also use > squatter, which should help with the body searches. However, each mailbox > is currently squatted roughly every 24 hours. With busy mailboxes I presume > that leaves a large window where search will have to fall back to searching > every message in that mailbox, right? > > We are looking into using metapartitions to move all the relevant files to > SSD to make access to cache and squat files faster, but that won't help > with "outdated" squat files, right? Would it make sense to squat his > mailboxes more often? It sounds remarkably like you want the full text xapian indexing that exists in the FastMail branch! Bron. -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How to improve mail search from iPad and other mobile devices
Hi, Quoting Sebastian Hagedorn : Hi, [...] Search for addresses and subject should in theory be reasonably fast because the results should mostly come from the cache. We also use squatter, which should help with the body searches. However, each mailbox is currently squatted roughly every 24 hours. With busy mailboxes I presume that leaves a large window where search will have to fall back to searching every message in that mailbox, right? AFAIK the index created by squatter is used to exclude mails that don't contain the search pattern, so an outdated index is still of use. Only the new messages and the files the index could not exclude will be searched. Running squatter more often will result in more IO traffic during the daytime. We are looking into using metapartitions to move all the relevant files to SSD to make access to cache and squat files faster, but that won't help with "outdated" squat files, right? Would it make sense to squat his mailboxes more often? Other suggestions? Cheers Sebastian -- .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:. M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen smime.p7s Description: S/MIME Signatur Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
How to improve mail search from iPad and other mobile devices
Hi, my boss has tasked me with improving the performance of IMAP searches using mail clients on mobile devices, mostly Apple's Mail on iPhone and iPad, because that's what he uses. The bad performance he experiences is exacerbated by the fact that he has one of the largest mail accounts we have: currently he has 23 GB of mails ... at least they're not all in his INBOX – he has about 1,750 folders. According to Apple's manual, Mail searches all folders like this: "Searching looks at the address fields, the subject, and the message body." Search for addresses and subject should in theory be reasonably fast because the results should mostly come from the cache. We also use squatter, which should help with the body searches. However, each mailbox is currently squatted roughly every 24 hours. With busy mailboxes I presume that leaves a large window where search will have to fall back to searching every message in that mailbox, right? We are looking into using metapartitions to move all the relevant files to SSD to make access to cache and squat files faster, but that won't help with "outdated" squat files, right? Would it make sense to squat his mailboxes more often? Other suggestions? Cheers Sebastian -- .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:. p7stgxcqb4f4f.p7s Description: S/MIME cryptographic signature Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
