Re: messages removed before expire

[Stephen Ingram]

On Thu, Oct 2, 2014 at 8:48 AM, Stephen Ingram  wrote:

> On Tue, Sep 16, 2014 at 11:15 PM, Bron Gondwana  wrote:
>
>>  On Wed, Sep 17, 2014, at 08:24 AM, Stephen Ingram wrote:
>>
>> On Tue, Sep 16, 2014 at 3:02 PM, Bron Gondwana  wrote:
>>
>>
>> Dumb question - but I don't suppose anything happened to your clock
>> recently?
>>
>> More non-dumb question, what version of Cyrus?
>>
>>
>> Sorry, I should have included the version. I'm using 2.4.16 rpms from
>> Invoca Systems. Nothing has happened with the clock. This is the only
>> mailbox that seems to have the problem too, user.ken.Trash. We are running
>> a murder configuration with 2 backends and 2 front ends. Not may users
>> accounts, just large mailboxes. Maybe there is some setting on the mupdate
>> master or somewhere else that is set for 14 days?
>>
>>
>> I don't think so.  Do you have any entries in the syslog showing the
>> deletes?  I'm pretty sure 'auditlog: 1' works in imapd.conf in 2.4.16, so
>> you should be able to add that to see more logging.
>>
>
> I finally got a chance to put in auditlog and I see several entries like
> this everyday:
>
> Oct  1 09:45:41 imap1 imap[7357]: auditlog: expunge
> sessionid= mailbox=
> uniqueid=<5f184bff4ee3c346> uid=<170665>
> guid=<5bbe70214bd4c7979f72977adda664afaa8ebe24>
>
> and then:
>
> Oct  1 09:45:41 imap1 imap[7357]: Expunged 19 messages from user.ken.Trash
>
> at then end. The system is now removing everything down to 7 days when
> "info user.ken.Trash" yields:
>
> {user.ken.Trash}:
>   duplicatedeliver: false
>   expire: 30
>   lastpop:
>   lastupdate:  2-Oct-2014 10:39:17 -0500
>   partition: default
>   pop3newuidl: true
>   sharedseen: false
>   size: 7749319
>
> I did check the client as Alexei had suggested too as I thought the 14
> days (and now 7) were a little coincidental, but there was nothing there
> causing the issue. Now after turning on auditlog, it does show the system
> removing these mails. Is this expire 30 number stored somewhere that has
> become corrupt or changed? This is just baffling to me as it seems to only
> affect this mailbox.
>

After another log search, I see there are messages removed every night.
There are dates stretching back 14 days on the file system and 7 days in
the actual Trash folder. I can't imagine that this could be a bug in Cyrus
as I would have thought others to have seen it by now. I'm guessing perhaps
under some unique set of circumstances, the file that contains the
expiration information has become corrupt causing the mail to be removed
pre-maturely. Does anyone know what file that information is contained
within and if it can somehow be regenerated?

Steve

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Upgrading from 2.2 to 2.4 (slow cyr_expire)

[Andrew Morgan]

On Thu, 16 Oct 2014, Jay Sekora wrote:

> Hi.  I recently tried to upgrade/migrate our Cyrus deployment from
> 2.2.13 (on Debian) to 2.4.17 (on Ubuntu 14.04).  In our environment,
> user mailboxes (about 3TB of them) are on iSCSI volumes; everything else
> is on local disk (which I rsync'ed).
>
> I ran into some delays to do with the storage backend configuration, so
> I didn't actually get to the point of starting imapd on the new server
> until disturbingly close to the end of our announced downtime window.
>
> When I did, I saw that imapd wasn't responding and cyr_expire was
> running.  I was expecting that, but eyeballing what it was doing via
> strace suggested that it would have taken *over twenty hours* to walk
> all the mailboxes.  (I'm guessing that cyr_expire was doing, perhaps as
> a side effect, the "full re-parse of all messages, which may take a
> while" mentioned under "Upgrading from 2.4.3" at
> http://cyrusimap.org/docs/cyrus-imapd/2.4.17/install-upgrade.php.)  So
> we announced that we were backing out of the upgrade.
>
> However, as I was getting ready to back out, I killed the cyr_expire by
> hand, and at that point imapd started responding and I was able to log
> in and see my own mail (which I know cyr_expire hadn't gotten to).  It
> was a little slow to initially show my my mail, which suggests that
> maybe Cyrus was running cyr_expire or its equivalent after I
> authenticated and before showing my my inbox, but that led me to wonder
> whether it might be safe (when we repeat the migration) to kill the
> cyr_expire on initial startup so that Cyrus will start talking IMAP
> right away, and run it in the background.
>
> In case it matters, we have a bunch of emeritus users who occasionally
> check their mail at our site but don't use it on a day-to-day basis, and
> a bunch of users who forward their mail elsewhere and leave a copy on
> our IMAP server as a backup, and a lot of our heavy users are
> sophisticated enough not to leave all their mail in their inboxes so
> when we open the floodgates a very large fraction of that ~3TB is not
> going to be looked at immediately.

You could comment cyr_expire out of cyrus.conf before you upgrade.  After 
a few days, you could uncomment cyr_expire and send a HUP signal to the 
Cyrus master process to have it re-read cyrus.conf.

Remember, the mailbox will be upgraded anytime it is opened.  That will 
occur when a user checks their mailbox AND when new mail is delivered. 
Still, it seems reasonably safe.  Your best bet is to schedule the upgrade 
at a time of low usage and try to "touch" as many mailboxes as possible 
before things get really busy.

Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Upgrading from 2.2 to 2.4 (slow cyr_expire)

[Jay Sekora]

Hi.  I recently tried to upgrade/migrate our Cyrus deployment from
2.2.13 (on Debian) to 2.4.17 (on Ubuntu 14.04).  In our environment,
user mailboxes (about 3TB of them) are on iSCSI volumes; everything else
is on local disk (which I rsync'ed).

I ran into some delays to do with the storage backend configuration, so
I didn't actually get to the point of starting imapd on the new server
until disturbingly close to the end of our announced downtime window.

When I did, I saw that imapd wasn't responding and cyr_expire was
running.  I was expecting that, but eyeballing what it was doing via
strace suggested that it would have taken *over twenty hours* to walk
all the mailboxes.  (I'm guessing that cyr_expire was doing, perhaps as
a side effect, the "full re-parse of all messages, which may take a
while" mentioned under "Upgrading from 2.4.3" at
http://cyrusimap.org/docs/cyrus-imapd/2.4.17/install-upgrade.php.)  So
we announced that we were backing out of the upgrade.

However, as I was getting ready to back out, I killed the cyr_expire by
hand, and at that point imapd started responding and I was able to log
in and see my own mail (which I know cyr_expire hadn't gotten to).  It
was a little slow to initially show my my mail, which suggests that
maybe Cyrus was running cyr_expire or its equivalent after I
authenticated and before showing my my inbox, but that led me to wonder
whether it might be safe (when we repeat the migration) to kill the
cyr_expire on initial startup so that Cyrus will start talking IMAP
right away, and run it in the background.

In case it matters, we have a bunch of emeritus users who occasionally
check their mail at our site but don't use it on a day-to-day basis, and
a bunch of users who forward their mail elsewhere and leave a copy on
our IMAP server as a backup, and a lot of our heavy users are
sophisticated enough not to leave all their mail in their inboxes so
when we open the floodgates a very large fraction of that ~3TB is not
going to be looked at immediately.

Jay



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Patch for adding tls_honor_cipher_order

[Kristian Kræmmer Nielsen]

Hi,

Patch attached.

While at it we might as well also let the user set tls_honor_cipher_order  
if they want to so that the order of cipher specified using  
tls_cipher_list is honored.


By default false, so changes nothing.

For expert uses might give clients a bit of extra performance by using the  
cheaper but still safe ciphers.


I would recommend going for a list as the one Mozilla have research for  
browsers since most clients use same SSL-libraries for both their browser  
and mail client. This is often the case on unix (openssl) and Windows.


Hope you'll merge,
Kristian

--

My configuration for reference:

#https://wiki.mozilla.org/Security/Server_Side_TLS
tls_cipher_list:  
ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-A

ES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA
-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-R
SA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-S
HA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:D
ES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-C
BC3-SHA

#tls_ec: prime256v1
tls_tlsonly: true
tls_honor_cipher_order: true

patch-tls_honor_cipher_order
Description: Binary data

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imap patching POODLE and another for adding perfect forward secrecy (PFS)

[Kristian Kræmmer Nielsen]

Hi Geoff,

I am basically not trying to take any stand on this. I just think it is  
time for the users to be able to disable the older protocols if they want  
to - as the old protocols are really no longer necessary for the wide  
majority of clients - and that is the main reasoning by my patches.


Notice that is also way I leave it false (changing nothing) by default in  
the patch.


For reference, you can see this for Ubuntu, they recommend total disabling  
SSLv3:

http://askubuntu.com/questions/537196/how-do-i-patch-workaround-sslv3-poodle-vulnerability-cve-2014-3566
"Conclusion: disable SSLv3 for HTTPS now, disable SSLv3 for other services  
in your next service window."


So, hope my patches get merged soon :)

/Kristian

On Thu, 16 Oct 2014 11:34:21 +0200, Geoff Winkless  wrote:


Hi Kristian

Firstly, many thanks for your work :)

Can you share the source for those recommendations? While I fully agree  
that using something that is shown to be >vulnerable is not ideal I'd be  
interested to see how they think a similar attack to POODLE could be  
implemented for >imap. As I posted to the info list, I've not seen  
anything that would suggest that IMAPS/SSLv3 is any less secure than >it  
was 10 years ago.


Thanks


On 16 October 2014 02:55, Kristian Kræmmer Nielsen  wrote:

Hi,

Two patches for merging

Thanks for the great work on cyrus imapd.

I have just read various recommendations that we now should disable  
SSLv3 not just on HTTPS as POODLE-attack demonstrates but we should  
>>expect to see exploits on other services as well like IMAPS and POPS.


I saw that disabling SSLv2 and SSLv3 in fact is already available in  
the tls-code but not made available to the user so therefore I have  
written the >>attached patch to do just that using a configuration  
variable named "tls_tlsonly". It is still by default false, so the  
patch should change nothing for >>users that still want to use the old  
protocols and may stay that way until an actual imaps-attack is proven.


Also I am including a cleaned up version of Chris Panayis' old patch  
for adding tls_ec for Perfect Forward Secrecy:

https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2013-January/002729.html

Using PFS is also a security recommendation we should follow. The  
default is set to prime256v1 just as sendmail and apache does this.


The patches are made against cyrus-imap-2.4.17 - but they also cleanly  
patch against the tip of the git repository of cyrus-imapd if skipping  
the >>patch of the man-page.


PFS: https://scotthelme.co.uk/perfect-forward-secrecy/
POODLE:  
https://www.dfranke.us/posts/2014-10-14-how-poodle-happened.html and  
https://www.openssl.org/~bodo/ssl-poodle.pdf


Regards
Kristian Kræmmer Nielsen,
Odense, Denmark

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How to prevent SSLv3/Poodle attack?

[Geoff Winkless]

On 16 October 2014 11:14, Sven Schwedas  wrote:

> On 2014-10-15 18:20, Geoff Winkless wrote:
> > Well the only thing new about POODLE versus previous known
> > vulnerabilities is the way to manipulate the known vulnerability to gain
> > the session cookie, which you can then re-use to log on to the site for
> > yourself without needing to authenticate.
>
> I think the more important new concept is that arbitrary sessions can be
> downgraded to use a known vulnerable cipher/protocol version, even if
> more secure are available and servers/clients use cipher suite pinning
> and all the other tricks we came up with to mitigate BEAST et. al.
>

​Ahhh. Thanks, I figured I must have missed the point :)

Although it isn't exactly news - referenced from the article:

http://jbp.io/2013/07/07/tls-downgrade/

Geoff​

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How to prevent SSLv3/Poodle attack?

[Sven Schwedas]

On 2014-10-15 18:20, Geoff Winkless wrote:
> Well the only thing new about POODLE versus previous known
> vulnerabilities is the way to manipulate the known vulnerability to gain
> the session cookie, which you can then re-use to log on to the site for
> yourself without needing to authenticate.

I think the more important new concept is that arbitrary sessions can be
downgraded to use a known vulnerable cipher/protocol version, even if
more secure are available and servers/clients use cipher suite pinning
and all the other tricks we came up with to mitigate BEAST et. al.

This makes the current "add new protocols for secure clients, but keep
backwards compatibility anyway" approach for handling SSL much more
dangerous.

> There's no such thing as a session cookie in IMAP, so I'd be very
> surprised to see it usable. That doesn't mean that IMAP/SSL3 is secure,
> it just means it's no less secure today than it was 10 years ago.

The current exploit is quite HTTP(S) specific and I can't think of a way
to apply it to IMAP, but it's probably not the last SSL3 problem.

> https://www.dfranke.us/posts/2014-10-14-how-poodle-happened.html is
> really good description, read especially the bit above "The workaround".
> 
> Hope this helps
> 
> Geoff
> 
> On 15 October 2014 17:03,  > wrote:
> 
> 
> Zitat von Geoff Winkless mailto:cy...@geoff.dj>>:
> 
> 
> Genuine question: is it shown that POODLE impacts on IMAPS?
> 
> I don't see how POODLE could affect an IMAPS session, since it
> only works
> if you can MITM a non-SSL session on the user's browser and
> force it to
> request the same target page over and over.
> 
> Cheers
> 
> Geoff
> 
> 
> As said i'm still reading on the details, so thanks for the pointer.
> Nonetheless it might be time to give up on SSLv3 because of protocol
> design errors/weakness. Unfortunately it looks like Cyrus can not
> disable SSLv3 protocol without disabling ciphers also used in
> TLSv1.x, no?
> 
> Regards
> 
> Andreas
> 
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 

-- 
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas
Systemadministrator
TAO Beratungs- und Management GmbH | Lendplatz 45 | A - 8020 Graz
Mail/XMPP: sven.schwe...@tao.at | +43 (0)680 301 7167
http://software.tao.at



signature.asc
Description: OpenPGP digital signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus