Re: SASL minimum layer used to work at 256, but now requires 1 [Resolved]
I just thought I would answer my own email in case anyone else has the same setup. I found out there is obviously some type of bug in the -23 versions of the SASL packages so they just won't work. Once reverting to the -20 or -21 versions, everything works great. My suspicion about the faulty SASL was correct, but since there was only a problem with the daemons themselves and not mupdatetest and imtest, it was difficult to see this. I suppose I should report this to the SASL list, but I'm really not sure what to say since the logging information is rather incomplete. Steve On Fri, Jun 1, 2018 at 5:16 PM, Stephen Ingram wrote: > I recently upgraded a CentOS 7 Cyrus 2.4.17 system with Murder and > Kerberos and ran into lots of issues with the new packages. What's really > puzzling though is although I used to be able to use a SASL minimum layer > of 256 (I'm using TLS with GSSAPI for auth), I now must use 1 for the > front-ends and backends communicating to the mupdate server. I've run into > SASL package issues before (2.1.23 to 2.1.26 was a mess) and had to > actually revert to an older version so I'm thinking that might be the > problem. However, when I use mupdatetest and imtest, everything works > perfectly. It's only when I fire up the daemons themselves that I see: > > Jun 1 23:53:21 imap mupdate[19865]: couldn't authenticate to backend > server: mechanism too weak for this user > Jun 1 23:53:21 imap mupdate[19865]: mupdate_connect failed: no auth status > Jun 1 23:53:21 imap mupdate[19865]: couldn't connect to mupdate server > Jun 1 23:53:21 imap mupdate[19865]: retrying connection to mupdate server > in 27 seconds > > If I grab a ticket and mupdatetest -m GSSAPI -t '' mupdate.test.net with > a 256 min layer, I get in perfectly. So perhaps it's a change/new config > issue in cyrus-imap this time around? > > I'm using cyrus-imapd-2.4.17-13 and cyrus-sasl-2.1.26-23 versions of the > various package sets. > > Steve > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
SASL minimum layer used to work at 256, but now requires 1
I recently upgraded a CentOS 7 Cyrus 2.4.17 system with Murder and Kerberos and ran into lots of issues with the new packages. What's really puzzling though is although I used to be able to use a SASL minimum layer of 256 (I'm using TLS with GSSAPI for auth), I now must use 1 for the front-ends and backends communicating to the mupdate server. I've run into SASL package issues before (2.1.23 to 2.1.26 was a mess) and had to actually revert to an older version so I'm thinking that might be the problem. However, when I use mupdatetest and imtest, everything works perfectly. It's only when I fire up the daemons themselves that I see: Jun 1 23:53:21 imap mupdate[19865]: couldn't authenticate to backend server: mechanism too weak for this user Jun 1 23:53:21 imap mupdate[19865]: mupdate_connect failed: no auth status Jun 1 23:53:21 imap mupdate[19865]: couldn't connect to mupdate server Jun 1 23:53:21 imap mupdate[19865]: retrying connection to mupdate server in 27 seconds If I grab a ticket and mupdatetest -m GSSAPI -t '' mupdate.test.net with a 256 min layer, I get in perfectly. So perhaps it's a change/new config issue in cyrus-imap this time around? I'm using cyrus-imapd-2.4.17-13 and cyrus-sasl-2.1.26-23 versions of the various package sets. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Jason- That came up clean for my latest config, but it did highlight problems when I didn't have the auth working. Thank you so much as this is a great search command to help sleuth out those SELinux issues. I did turn it off just to see what happened, but it was not the problem. Nice though, because I learned how to relabel a volume to get back in the good graces of SELinux. I have this mostly working now, but I'm not exactly sure how so I don't really have any tips to post. I am going back through the system though piece by piece to try and find any one out of place configuration/package that may have caused the problem. My thought is that is was some strange cacheing issue, but I'm not totally sure. Steve On Fri, Jun 1, 2018 at 11:37 AM, Jason L Tibbitts III wrote: > If you suspect selinux, please do 'ausearch -m avc -ts today' and see what > you get. You may also wish to do 'setenforce 0' and try again, just to > make sure. > > I can provide some basic help with selinux if that's needed. > > - J< > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Well, I finally figured out how to turn up the logging. Alas, the logging inside cyrus tells me less than postfix. How can that be? Jun 1 17:58:56 imap lmtp[20994]: connection from mx.x.x [10.0.13.69] Jun 1 17:58:56 imap lmtp[20994]: SSL_accept() incomplete -> wait Jun 1 17:58:56 imap lmtp[20994]: Doing a peer verify Jun 1 17:58:56 imap lmtp[20994]: Doing a peer verify Jun 1 17:58:56 imap lmtp[20994]: Doing a peer verify Jun 1 17:58:56 imap lmtp[20994]: Doing a peer verify Jun 1 17:58:56 imap lmtp[20994]: SSL_accept() succeeded -> done Jun 1 17:58:56 imap lmtp[20994]: received client certificate Jun 1 17:58:56 imap lmtp[20994]: subject=/OU=Domain Control Validated/OU=PositiveSSL/CN=mx.x.x Jun 1 17:58:56 imap lmtp[20994]: starttls: TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits new) authenticated as mx.x.x Jun 1 17:58:56 imap lmtp[20994]: login: mx.x.x [10.0.13.69] smtp/mx.x.x GSSAPI+TLS User logged in Jun 1 17:58:56 imap lmtp[20994]: USAGE x user: 0.009004 sys: 0.002318 Jun 1 17:58:56 imap lmtp[20994]: telling master 1 Steve On Fri, Jun 1, 2018 at 10:48 AM, Stephen Ingram wrote: > Patrick- > > My thought too, but nothing but success entries for all of the cyrus > commands. Is there a way to increase logging on the frontend? I seem to be > getting details on the backends, but not the frontend. That might help > diagnose this. Ken asked for cyrus logs, but I don't really have anything > to give him because I don't see anything of consequence there. > > Steve > > On Fri, Jun 1, 2018 at 10:41 AM, Patrick Boutilier > wrote: > >> Anything in /var/log/audit/audit.log ? >> >> >> On June 1, 2018 2:29:06 PM ADT, Stephen Ingram >> wrote: >>> >>> Patrick- >>> >>> I'm also trying to get more debugging about he system I/O error, but >>> never see it in the cyrus logs, only in the postfix logs. >>> >>> Steve >>> >>> On Fri, Jun 1, 2018 at 9:37 AM, Patrick Boutilier >>> wrote: >>> >>>> On 06/01/2018 01:31 PM, Stephen Ingram wrote: >>>> >>>>> Patrick- >>>>> >>>>> Actually, nothing. I've got everything piped into /var/log/maillog and >>>>> not too much there either beyond the actual error message. >>>>> >>>>> >>>> Hmmm... Usually when I have seen the System I/O error the log entry >>>> also records what the actual directory/file that it wants to write to. >>>> Going by memory here since it has been a long time since I have seen the >>>> error. >>>> >>>> >>>> >>>> >>>> Steve >>>>> >>>>> >>>>> On Fri, Jun 1, 2018 at 9:23 AM, Patrick Boutilier < >>>>> bouti...@ednet.ns.ca <mailto:bouti...@ednet.ns.ca>> wrote: >>>>> >>>>> On 06/01/2018 01:21 PM, Stephen Ingram wrote: >>>>> >>>>> I'm receiving a 451 4.3.0 System I/O error (in reply to end of >>>>> DATA command) error from Postfix when trying to deliver to >>>>> cyrus-imap and not really sure why. I'm on CentOS 7 (2.4.17-8) >>>>> after downgrading from current version. I'm using Kerberos >>>>> GSSAPI to connect to the front end, but authentication appears >>>>> to be working fine as I can see authenticated when enabling >>>>> LMTP >>>>> debugging in Postifx. All messages are refused for delivery >>>>> though. I'm not sure what to do. Any suggestions? >>>>> >>>>> >>>>> Should be something of value in /var/log/messages . >>>>> >>>>> >>>>> >>>>> >>>>> Steve >>>>> >>>>> >>>>> >>>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>>> List Archives/Info: >>>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>>> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >>>>> To Unsubscribe: >>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>>> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >>>>> >>>>> >>>>> >>>>> >>>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>>> List Archives/Info: >>>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>>> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >>>>> To Unsubscribe: >>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>>> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >>>>> >>>>> >>>>> >>>> >>> > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Patrick- My thought too, but nothing but success entries for all of the cyrus commands. Is there a way to increase logging on the frontend? I seem to be getting details on the backends, but not the frontend. That might help diagnose this. Ken asked for cyrus logs, but I don't really have anything to give him because I don't see anything of consequence there. Steve On Fri, Jun 1, 2018 at 10:41 AM, Patrick Boutilier wrote: > Anything in /var/log/audit/audit.log ? > > > On June 1, 2018 2:29:06 PM ADT, Stephen Ingram wrote: >> >> Patrick- >> >> I'm also trying to get more debugging about he system I/O error, but >> never see it in the cyrus logs, only in the postfix logs. >> >> Steve >> >> On Fri, Jun 1, 2018 at 9:37 AM, Patrick Boutilier >> wrote: >> >>> On 06/01/2018 01:31 PM, Stephen Ingram wrote: >>> >>>> Patrick- >>>> >>>> Actually, nothing. I've got everything piped into /var/log/maillog and >>>> not too much there either beyond the actual error message. >>>> >>>> >>> Hmmm... Usually when I have seen the System I/O error the log entry also >>> records what the actual directory/file that it wants to write to. Going by >>> memory here since it has been a long time since I have seen the error. >>> >>> >>> >>> >>> Steve >>>> >>>> >>>> On Fri, Jun 1, 2018 at 9:23 AM, Patrick Boutilier >>> <mailto:bouti...@ednet.ns.ca>> wrote: >>>> >>>> On 06/01/2018 01:21 PM, Stephen Ingram wrote: >>>> >>>> I'm receiving a 451 4.3.0 System I/O error (in reply to end of >>>> DATA command) error from Postfix when trying to deliver to >>>> cyrus-imap and not really sure why. I'm on CentOS 7 (2.4.17-8) >>>> after downgrading from current version. I'm using Kerberos >>>> GSSAPI to connect to the front end, but authentication appears >>>> to be working fine as I can see authenticated when enabling LMTP >>>> debugging in Postifx. All messages are refused for delivery >>>> though. I'm not sure what to do. Any suggestions? >>>> >>>> >>>> Should be something of value in /var/log/messages . >>>> >>>> >>>> >>>> >>>> Steve >>>> >>>> >>>> >>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>> List Archives/Info: >>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >>>> To Unsubscribe: >>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >>>> >>>> >>>> >>>> >>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>> List Archives/Info: >>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >>>> To Unsubscribe: >>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >>>> >>>> >>>> >>> >> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Patrick- I'm also trying to get more debugging about he system I/O error, but never see it in the cyrus logs, only in the postfix logs. Steve On Fri, Jun 1, 2018 at 9:37 AM, Patrick Boutilier wrote: > On 06/01/2018 01:31 PM, Stephen Ingram wrote: > >> Patrick- >> >> Actually, nothing. I've got everything piped into /var/log/maillog and >> not too much there either beyond the actual error message. >> >> > Hmmm... Usually when I have seen the System I/O error the log entry also > records what the actual directory/file that it wants to write to. Going by > memory here since it has been a long time since I have seen the error. > > > > > Steve >> >> >> On Fri, Jun 1, 2018 at 9:23 AM, Patrick Boutilier > <mailto:bouti...@ednet.ns.ca>> wrote: >> >> On 06/01/2018 01:21 PM, Stephen Ingram wrote: >> >> I'm receiving a 451 4.3.0 System I/O error (in reply to end of >> DATA command) error from Postfix when trying to deliver to >> cyrus-imap and not really sure why. I'm on CentOS 7 (2.4.17-8) >> after downgrading from current version. I'm using Kerberos >> GSSAPI to connect to the front end, but authentication appears >> to be working fine as I can see authenticated when enabling LMTP >> debugging in Postifx. All messages are refused for delivery >> though. I'm not sure what to do. Any suggestions? >> >> >> Should be something of value in /var/log/messages . >> >> >> >> >> Steve >> >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: >> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >> >> >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: >> http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> <http://lists.andrew.cmu.edu/pipermail/info-cyrus/> >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus> >> >> >> > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Ken- Here is the only entry in the cyrus logs I can see from that postfix server: Jun 1 17:01:19 imap lmtp[12597]: starttls: TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits new) authenticated as x.x.x Jun 1 17:01:19 imap lmtp[12597]: login: x.x.x [10.0.13.69] smtp/x.x.x GSSAPI+TLS User logged in Steve On Fri, Jun 1, 2018 at 9:58 AM, Stephen Ingram wrote: > Ken- > > That could be. I noticed that the SELinux policy updated on the server as > well and Redhat might have changed something that is not allowing Cyrus to > write there?? It is able to write the Kerberos credential cache there > though. I thought it wrote everything else to /var/lib/imap. > > I don't see anything in the cyrus syslog besides the postfix server > logging into the cyrus server successfully. Is there a way to increase the > debugging? > > Steve > > On Fri, Jun 1, 2018 at 9:54 AM, Ken Murchison wrote: > >> OTH, my guess is that Cyrus is failing to create the tmpfile to stage the >> message. Is /tmp (or its equivalent) full, or not writeable? >> >> Without something from the Cyrus syslog, this will be hard to diagnose. >> >> On 06/01/2018 12:51 PM, Stephen Ingram wrote: >> >> Ken- >> >> That all appears to be working correctly. Here's a cut from the Postfix >> debug with addresses obfuscated: >> >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr offset = 682 >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr dsn_orig_rcpt = >> rfc822;x...@x.com >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr notify_flags = 0 >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr status = 4.3.0 >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_type = smtp >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_text = 451 4.3.0 >> System I/O error >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_type = dns >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_mname = imap.x.x >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr action = delayed >> Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr reason = host >> imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of >> DATA command) >> Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted >> attribute: status >> Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: status >> Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute value: 0 >> Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted >> attribute: (list terminator) >> Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: (end) >> Jun 1 16:48:52 mx postfix/lmtp[18136]: B7B0B9A2EB0: to= , >> orig_to= , relay=imap.x.x[10.0.13.83]:24, delay=0.3, >> delays=0.19/0/0.09/0.02, dsn=4.3.0, status=deferred (host >> imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of >> DATA command)) >> >> Steve >> >> On Fri, Jun 1, 2018 at 9:33 AM, Ken Murchison wrote: >> >>> On 6/1/18 12:21 PM, Stephen Ingram wrote: >>> >>>> I'm receiving a 451 4.3.0 System I/O error (in reply to end of DATA >>>> command) error from Postfix when trying to deliver to cyrus-imap and not >>>> really sure why. I'm on CentOS 7 (2.4.17-8) after downgrading from current >>>> version. I'm using Kerberos GSSAPI to connect to the front end, but >>>> authentication appears to be working fine as I can see authenticated when >>>> enabling LMTP debugging in Postifx. All messages are refused for delivery >>>> though. I'm not sure what to do. Any suggestions? >>>> >>> >>> Any chance that Kerberos is negotiating a security layer? >>> >>> -- >>> Kenneth Murchison >>> Cyrus Development Team >>> FastMail Pty Ltd >>> >>> >>> >>> Cyrus Home Page: http://www.cyrusimap.org/ >>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>> To Unsubscribe: >>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>> >> >> >> -- >> Ken Murchison >> Cyrus Development Team >> FastMail US LLC >> >> > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Ken- That could be. I noticed that the SELinux policy updated on the server as well and Redhat might have changed something that is not allowing Cyrus to write there?? It is able to write the Kerberos credential cache there though. I thought it wrote everything else to /var/lib/imap. I don't see anything in the cyrus syslog besides the postfix server logging into the cyrus server successfully. Is there a way to increase the debugging? Steve On Fri, Jun 1, 2018 at 9:54 AM, Ken Murchison wrote: > OTH, my guess is that Cyrus is failing to create the tmpfile to stage the > message. Is /tmp (or its equivalent) full, or not writeable? > > Without something from the Cyrus syslog, this will be hard to diagnose. > > On 06/01/2018 12:51 PM, Stephen Ingram wrote: > > Ken- > > That all appears to be working correctly. Here's a cut from the Postfix > debug with addresses obfuscated: > > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr offset = 682 > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr dsn_orig_rcpt = > rfc822;x...@x.com > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr notify_flags = 0 > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr status = 4.3.0 > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_type = smtp > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_text = 451 4.3.0 > System I/O error > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_type = dns > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_mname = imap.x.x > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr action = delayed > Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr reason = host > imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of > DATA command) > Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted > attribute: status > Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: status > Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute value: 0 > Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted > attribute: (list terminator) > Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: (end) > Jun 1 16:48:52 mx postfix/lmtp[18136]: B7B0B9A2EB0: to= , > orig_to= , relay=imap.x.x[10.0.13.83]:24, delay=0.3, > delays=0.19/0/0.09/0.02, dsn=4.3.0, status=deferred (host > imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of > DATA command)) > > Steve > > On Fri, Jun 1, 2018 at 9:33 AM, Ken Murchison wrote: > >> On 6/1/18 12:21 PM, Stephen Ingram wrote: >> >>> I'm receiving a 451 4.3.0 System I/O error (in reply to end of DATA >>> command) error from Postfix when trying to deliver to cyrus-imap and not >>> really sure why. I'm on CentOS 7 (2.4.17-8) after downgrading from current >>> version. I'm using Kerberos GSSAPI to connect to the front end, but >>> authentication appears to be working fine as I can see authenticated when >>> enabling LMTP debugging in Postifx. All messages are refused for delivery >>> though. I'm not sure what to do. Any suggestions? >>> >> >> Any chance that Kerberos is negotiating a security layer? >> >> -- >> Kenneth Murchison >> Cyrus Development Team >> FastMail Pty Ltd >> >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> > > > -- > Ken Murchison > Cyrus Development Team > FastMail US LLC > > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Ken- That all appears to be working correctly. Here's a cut from the Postfix debug with addresses obfuscated: Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr offset = 682 Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr dsn_orig_rcpt = rfc822;x...@x.com Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr notify_flags = 0 Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr status = 4.3.0 Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_type = smtp Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr diag_text = 451 4.3.0 System I/O error Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_type = dns Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr mta_mname = imap.x.x Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr action = delayed Jun 1 16:48:52 mx postfix/lmtp[18136]: send attr reason = host imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of DATA command) Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted attribute: status Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: status Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute value: 0 Jun 1 16:48:52 mx postfix/lmtp[18136]: private/defer socket: wanted attribute: (list terminator) Jun 1 16:48:52 mx postfix/lmtp[18136]: input attribute name: (end) Jun 1 16:48:52 mx postfix/lmtp[18136]: B7B0B9A2EB0: to=, orig_to=, relay=imap.x.x[10.0.13.83]:24, delay=0.3, delays=0.19/0/0.09/0.02, dsn=4.3.0, status=deferred (host imap.x.x[10.0.13.83] said: 451 4.3.0 System I/O error (in reply to end of DATA command)) Steve On Fri, Jun 1, 2018 at 9:33 AM, Ken Murchison wrote: > On 6/1/18 12:21 PM, Stephen Ingram wrote: > >> I'm receiving a 451 4.3.0 System I/O error (in reply to end of DATA >> command) error from Postfix when trying to deliver to cyrus-imap and not >> really sure why. I'm on CentOS 7 (2.4.17-8) after downgrading from current >> version. I'm using Kerberos GSSAPI to connect to the front end, but >> authentication appears to be working fine as I can see authenticated when >> enabling LMTP debugging in Postifx. All messages are refused for delivery >> though. I'm not sure what to do. Any suggestions? >> > > Any chance that Kerberos is negotiating a security layer? > > -- > Kenneth Murchison > Cyrus Development Team > FastMail Pty Ltd > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: System I/O error (in reply to end of DATA command) for LMTP delivery
Patrick- Actually, nothing. I've got everything piped into /var/log/maillog and not too much there either beyond the actual error message. Steve On Fri, Jun 1, 2018 at 9:23 AM, Patrick Boutilier wrote: > On 06/01/2018 01:21 PM, Stephen Ingram wrote: > >> I'm receiving a 451 4.3.0 System I/O error (in reply to end of DATA >> command) error from Postfix when trying to deliver to cyrus-imap and not >> really sure why. I'm on CentOS 7 (2.4.17-8) after downgrading from current >> version. I'm using Kerberos GSSAPI to connect to the front end, but >> authentication appears to be working fine as I can see authenticated when >> enabling LMTP debugging in Postifx. All messages are refused for delivery >> though. I'm not sure what to do. Any suggestions? >> > > Should be something of value in /var/log/messages . > > > > >> Steve >> >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> >> > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
System I/O error (in reply to end of DATA command) for LMTP delivery
I'm receiving a 451 4.3.0 System I/O error (in reply to end of DATA command) error from Postfix when trying to deliver to cyrus-imap and not really sure why. I'm on CentOS 7 (2.4.17-8) after downgrading from current version. I'm using Kerberos GSSAPI to connect to the front end, but authentication appears to be working fine as I can see authenticated when enabling LMTP debugging in Postifx. All messages are refused for delivery though. I'm not sure what to do. Any suggestions? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Default value changes in Cyrus 3
I see now that the warning applies to change from altnamespace to not. I already use the altnamespace, but I want to change to the unix hierarchy separator where it says the altnamespace doesn't matter. Does that mean it will keep the same hierarchy? Steve On Thu, Oct 19, 2017 at 6:15 PM, Stephen Ingram <sbing...@gmail.com> wrote: > I'm not sure I really know what that means. Would that change the > hierarchy? Force a download of all messages again on the client side? > > Steve > > On Thu, Oct 19, 2017 at 6:04 PM, Patrick Boutilier <bouti...@ednet.ns.ca> > wrote: > >> On 10/19/2017 08:56 PM, Janos Dohanics wrote: >> >>> On Thu, 19 Oct 2017 16:43:48 -0700 >>> Stephen Ingram <sbing...@gmail.com> wrote: >>> >>> While we are talking about it, can this just be switched on the fly if >>>> someone is using the "." namespace? >>>> >>> >>> Never tried it, but I read this page that the answer is yes. >>> >>> https://www.cyrusimap.org/imap/reference/admin/sop/altnamespace.html >>> >>> >> Yes, seems possible but there is this warning on >> https://www.cyrusimap.org/imap/concepts/features/namespaces. >> html#imap-admin-namespaces-mode >> >> >> >> Warning >> >> Changing altnamespace in an active operating environment will cause all >> IMAP clients to need to resync the entire hierarchy. >> >> >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> > > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Default value changes in Cyrus 3
I'm not sure I really know what that means. Would that change the hierarchy? Force a download of all messages again on the client side? Steve On Thu, Oct 19, 2017 at 6:04 PM, Patrick Boutilier <bouti...@ednet.ns.ca> wrote: > On 10/19/2017 08:56 PM, Janos Dohanics wrote: > >> On Thu, 19 Oct 2017 16:43:48 -0700 >> Stephen Ingram <sbing...@gmail.com> wrote: >> >> While we are talking about it, can this just be switched on the fly if >>> someone is using the "." namespace? >>> >> >> Never tried it, but I read this page that the answer is yes. >> >> https://www.cyrusimap.org/imap/reference/admin/sop/altnamespace.html >> >> > Yes, seems possible but there is this warning on > https://www.cyrusimap.org/imap/concepts/features/namespaces. > html#imap-admin-namespaces-mode > > > > Warning > > Changing altnamespace in an active operating environment will cause all > IMAP clients to need to resync the entire hierarchy. > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Default value changes in Cyrus 3
While we are talking about it, can this just be switched on the fly if someone is using the "." namespace? Does it only affect the admin in that you have to type user/mailbox now instead of user.mailbox, or does the change also affect users? Steve On Thu, Oct 19, 2017 at 4:36 PM, Janos Dohanicswrote: > On Fri, 20 Oct 2017 08:02:04 +1100 > Bron Gondwana wrote: > > > Basically: > > > > 1) most of the rest of the world is using '/', so people expect it. > >Also, it means you can use '.' in mailbox names and in usernames, > >both of which people expect to be able to do due to other systems > >allowing it. > > 2) various clients expect to be able to create top level names and get > >sad if they don't. The reason not to use altnamespace in the past > >was that it didn't allow subfolders of INBOX, and there were a few > >other broken things about sort. They have now been fixed, so it's > >safe to make it the default. > > Both changes are to make the "out of the box" behaviour closer to what > > most people expect. > > Cheers, > > > > Bron. > > Thanks. May be adding your comments to the Cyrus web site would be > useful to people who wonder whether to keep the < 3 defaults or go with > the new ones. > > -- > Janos Dohanics > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: list of IMAP extensions
On Thu, Jun 22, 2017 at 1:08 PM, Stephen Ingram <sbing...@gmail.com> wrote: > On Thu, Jun 22, 2017 at 12:55 PM, Adam Tauno Williams < > awill...@whitemice.org> wrote: > >> Quoting Stephen Ingram <sbing...@gmail.com>: >> >>> Is there a comprehensive list of IMAP extensions supported by Cyrus-IMAP >>> 2.4.x and 3.x? Not the RFCs, but the actual extensions like QRESYNC and >>> THREAD=REFS. >>> >> >> So not https://cyrusimap.org/imap/rfc-support.html ? >> > > No. I had already found that, but would love it if the extensions were > listed directly. I'm trying to verify product support for Cyrus-IMAP and > would love to have a quick checklist. I could sworn I've seen something > like this before, but can't seem to locate it now. > I forgot to include exactly the extensions I'm looking for: IDLE QRESYNC CONDSTORE CHILDREN THREAD=REFS UIDPLUS SORT UID SORT LIST-STATUS SPECIAL-USE MOVE I know that IDLE is supported and it looks like CONDSTORE can be turned on using cyradm on an account-by-account basis, but I'm not totally sure about the rest. Does anyone have an idea? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: list of IMAP extensions
On Thu, Jun 22, 2017 at 12:55 PM, Adam Tauno Williams < awill...@whitemice.org> wrote: > Quoting Stephen Ingram <sbing...@gmail.com>: > >> Is there a comprehensive list of IMAP extensions supported by Cyrus-IMAP >> 2.4.x and 3.x? Not the RFCs, but the actual extensions like QRESYNC and >> THREAD=REFS. >> > > So not https://cyrusimap.org/imap/rfc-support.html ? > No. I had already found that, but would love it if the extensions were listed directly. I'm trying to verify product support for Cyrus-IMAP and would love to have a quick checklist. I could sworn I've seen something like this before, but can't seem to locate it now. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
list of IMAP extensions
Is there a comprehensive list of IMAP extensions supported by Cyrus-IMAP 2.4.x and 3.x? Not the RFCs, but the actual extensions like QRESYNC and THREAD=REFS. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
debugging on CentOS 7
With the switch to systemd and SELinux on CentOS 7, I've run into some Kerberos issues. I'm trying to debug them, like to see if the KRB5_KTNAME environment variable is loading properly, but can't seem to turn on more debugging. I see from the documentation that adding CYRUS_VERBOSE=x, where x is higher than 1, to the environment will result in more debugging. Yet, it doesn't seem to work. There is such a dearth of information about CyrusIMAP running on RHEL7/CentOS7 out there that I'm wondering if anyone is even running a Kerberos-based system out there. Does anyone know how to turn on the debugging? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
kerberos credentials on systemd-based CentOS 7
I'm trying to setup a kerberos connection to an mupdate server using gssapi authentication. I'm creating a credentials cache using a keytab file on the system for user imap/machine1.domain.com. In the old init.d-based system, I specified the KRB5_KTNAME and KRB5CCNAME environment variables, then when the cyrus-master program ran, the ticket was fetched and the system was able to connect. However, with systemd, it appears as though the server should maybe use a persistent keyring to store the credentials. Even if I try to use a file, say inside /var/lib/imap to escape selinux, the system still fails to authenticate. Does anyone have this setup working that allows a cyrus client to connect to an mupdate server to fetch mailbox information? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: kerberos credentials on systemd-based CentOS 7
On Sun, Sep 20, 2015 at 6:00 PM, Stephen Ingram <sbing...@gmail.com> wrote: > I'm trying to setup a kerberos connection to an mupdate server using > gssapi authentication. I'm creating a credentials cache using a keytab file > on the system for user imap/machine1.domain.com. In the old init.d-based > system, I specified the KRB5_KTNAME and KRB5CCNAME environment variables, > then when the cyrus-master program ran, the ticket was fetched and the > system was able to connect. However, with systemd, it appears as though the > server should maybe use a persistent keyring to store the credentials. Even > if I try to use a file, say inside /var/lib/imap to escape selinux, the > system still fails to authenticate. Does anyone have this setup working > that allows a cyrus client to connect to an mupdate server to fetch mailbox > information? > Looks like I got bit by Bug 3480 <https://bugzilla.cyrusimap.org/show_bug.cgi?id=3480> again. I wrongly assumed this had been fixed by now, but I guess not, so RHEL 7 cyrus is still broken for those using sasl with GSSAPI. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
change to UNIX hierarchy
Since we support Kerberos, we use standard usernames on our system without any domain endings and we also use the Alternate namespace. This being the case, can we turn on UNIX hierarchy without any changes in the user's mail client or the filesystem itself? From the documentation, it looks like the only change would be in the management of the mailboxes (cyradm) where we would now use a / instead of a .. For instance, the cyradm command: cm user/john/Sent instead of cm user.john.Sent. Am I correct or off base here? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
telnet test of cyrus sync server
I'm trying to write a quick routine that will test connectivity to a cyrus sync server for monitoring purposes. I can connect to port 2005 using telnet, however, once connected, I cannot disconnect using A QUIT or anything else I can think of that might work with the IMAP server itself. Every time I issue a command I receive: NO Please authenticate first I would really like to avoid authenticating just for this quick connection test. Is there anyway to quit without authenticating or just punching up the telnet escape sequence? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: POLL: per-domain shared folder/sieve/etc
On Thu, Oct 23, 2014 at 6:18 PM, Bron Gondwana br...@fastmail.fm wrote: Yeah, I'm afraid so. It's going to kind of suck for FastMail customers as much as anyone actually - since that's what we use! But here's the thing: a) it will be possible to switch to the netnews way if you want b) but if you have virtdomains AND netnews separator, then that will mean that you need to switch. Hopefully most clients will cope - I haven't tested it. If you DON'T have virtdomains, or the users are all in the default domain, then it will keep working just the same. I guess we could have a config option strip domain if same or something, and get the same display as what we have now despite the different representation internally. The only thing is, you _could_ share with users in other domains if you wanted. Since everything is in one domain on our system, it sounds like you are saying we can stay with what we have. I'm wondering though how difficult it would be to change to unixheirarchysep method. Does this simply change the way that the server presents the mailbox information to the client, and, thus, require changes to the cyrus database files or does it actually require changes to how the files/directories are stored on the disk? If it's just the cyrus database, is this difficult to convert? I'm guessing there is as you would want to deploy this at Fastmail, right? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: POLL: per-domain shared folder/sieve/etc
On Wed, Oct 22, 2014 at 2:02 PM, Bron Gondwana br...@fastmail.fm wrote: So Cyrus has three different types of domain split: * none at all * on/yes = weird reverse DNS hackery * userid = login with domain. As part of finally switching to unixheirarchysep: on (yay) and better key format (double yay) I want to change the overarching split users into separate domains logic we have right now. Yes, that means a massive change, instead of internally: example.com!user.foo.bar = user/foo/b...@example.com (which is a million ways of bogus) we would have: user.foo@example^com.bar = user/f...@example.com/bar Or in alt namspace: Other Users/f...@example.com/bar This means we will finally be able to share things across domains. It creates a single consistent way to access everything. The problem is, it means you can't set quotas per domain, you can't have sieve scripts per domain, and most of all - you can't have shared folders in a domain. example.com!shared.stuff worked fine, but shared.example^com.stuff would be weird. It's just a folder, and wouldn't be treated specially in any way. The domain would have no special meaning. This is all, obviously, Cyrus 3.0 stuff. It's a significant change in how folder naming works. It's really good for removing some inconsistencies though. I just want to have an idea of whether it will mess up anyone's existing workflows - and if so how we can make sure you can still achieve a similar result, even if it doesn't look quite the same in the new world. We only set quotas on individual mailboxes so that wouldn't be a problem. We also don't have sieve scripts except per-mailbox, so ditto there. I'm not quite clear though about the global sharing thing. Does this mean, for example, that if one user wants to share a mailbox with another user, its name has to be unique on the entire system? We would have users who would want to only share with other users in their domain. Since we support a single-realm Kerberos setup we only use usernames not email address login. Does that make any difference here since there appears to be an issue with the domain part? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: POLL: per-domain shared folder/sieve/etc
On Thu, Oct 23, 2014 at 5:59 PM, Bron Gondwana br...@fastmail.fm wrote: On Thu, Oct 23, 2014, at 08:55 PM, Stephen Ingram wrote: On Wed, Oct 22, 2014 at 2:02 PM, Bron Gondwana br...@fastmail.fm wrote: We only set quotas on individual mailboxes so that wouldn't be a problem. We also don't have sieve scripts except per-mailbox, so ditto there. Sounds like you'll be fine. I'm not quite clear though about the global sharing thing. Does this mean, for example, that if one user wants to share a mailbox with another user, its name has to be unique on the entire system? We would have users who would want to only share with other users in their domain. No, the per-user namespace is still fine - users can still share with other users in their own domain - just currently it is technically impossible to share with users in other domains right now - because the mailbox naming is not RFC compliant, so it's not compatible with real IMAP client, only with Cyrus management tools. Since we support a single-realm Kerberos setup we only use usernames not email address login. Does that make any difference here since there appears to be an issue with the domain part? There's nothing wrong with running without domains still - there would still be support for virtdomains: off, or else for a single defaultdomain: example.com which would be appended/stripped as appropriate. Great. I forgot to ask about unixheirarchysep. Does that mean that the default netnews . way of doing things is going away? If so, will there be an easy way to convert? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: messages removed before expire
On Fri, Oct 17, 2014 at 7:50 AM, Bron Gondwana br...@fastmail.fm wrote: iOS is known for deleting messages from the Trash folder on its own behalf. On Fri, Oct 17, 2014, at 10:36 AM, Joseph Brennan wrote: Oct 1 09:45:41 imap1 imap[7357]: Expunged 19 messages from user.ken.Trash The above shows *imap* doing an expunge. The imap client did this. Given delayed expunge the messages would still be there on the filesystem after this. The real expunge done by the expire job does not say imap: Oct 17 05:16:28 salmon cyr_expire[11566]: Expunged 2 messages from user.xxx Joseph Brennan Columbia University Information Technology Thank you so much! This is a relief to know that Cyrus is not at fault here as it wouldn't make sense that it works for everyone else, but not this person. We just called the user again and asked if there was a phone or tablet that he hadn't disclosed previously. Indeed there was an iPad that he didn't consider important to mention that was set to delete after 7 days. I guess I need to be more thorough about checking all of the clients. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: messages removed before expire
On Thu, Oct 2, 2014 at 8:48 AM, Stephen Ingram sbing...@gmail.com wrote: On Tue, Sep 16, 2014 at 11:15 PM, Bron Gondwana br...@fastmail.fm wrote: On Wed, Sep 17, 2014, at 08:24 AM, Stephen Ingram wrote: On Tue, Sep 16, 2014 at 3:02 PM, Bron Gondwana br...@fastmail.fm wrote: Dumb question - but I don't suppose anything happened to your clock recently? More non-dumb question, what version of Cyrus? Sorry, I should have included the version. I'm using 2.4.16 rpms from Invoca Systems. Nothing has happened with the clock. This is the only mailbox that seems to have the problem too, user.ken.Trash. We are running a murder configuration with 2 backends and 2 front ends. Not may users accounts, just large mailboxes. Maybe there is some setting on the mupdate master or somewhere else that is set for 14 days? I don't think so. Do you have any entries in the syslog showing the deletes? I'm pretty sure 'auditlog: 1' works in imapd.conf in 2.4.16, so you should be able to add that to see more logging. I finally got a chance to put in auditlog and I see several entries like this everyday: Oct 1 09:45:41 imap1 imap[7357]: auditlog: expunge sessionid=imap1.4test.net-7357-1412174740-1 mailbox=user.ken.Trash uniqueid=5f184bff4ee3c346 uid=170665 guid=5bbe70214bd4c7979f72977adda664afaa8ebe24 and then: Oct 1 09:45:41 imap1 imap[7357]: Expunged 19 messages from user.ken.Trash at then end. The system is now removing everything down to 7 days when info user.ken.Trash yields: {user.ken.Trash}: duplicatedeliver: false expire: 30 lastpop: lastupdate: 2-Oct-2014 10:39:17 -0500 partition: default pop3newuidl: true sharedseen: false size: 7749319 I did check the client as Alexei had suggested too as I thought the 14 days (and now 7) were a little coincidental, but there was nothing there causing the issue. Now after turning on auditlog, it does show the system removing these mails. Is this expire 30 number stored somewhere that has become corrupt or changed? This is just baffling to me as it seems to only affect this mailbox. After another log search, I see there are messages removed every night. There are dates stretching back 14 days on the file system and 7 days in the actual Trash folder. I can't imagine that this could be a bug in Cyrus as I would have thought others to have seen it by now. I'm guessing perhaps under some unique set of circumstances, the file that contains the expiration information has become corrupt causing the mail to be removed pre-maturely. Does anyone know what file that information is contained within and if it can somehow be regenerated? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: messages removed before expire
On Tue, Sep 16, 2014 at 11:15 PM, Bron Gondwana br...@fastmail.fm wrote: On Wed, Sep 17, 2014, at 08:24 AM, Stephen Ingram wrote: On Tue, Sep 16, 2014 at 3:02 PM, Bron Gondwana br...@fastmail.fm wrote: Dumb question - but I don't suppose anything happened to your clock recently? More non-dumb question, what version of Cyrus? Sorry, I should have included the version. I'm using 2.4.16 rpms from Invoca Systems. Nothing has happened with the clock. This is the only mailbox that seems to have the problem too, user.ken.Trash. We are running a murder configuration with 2 backends and 2 front ends. Not may users accounts, just large mailboxes. Maybe there is some setting on the mupdate master or somewhere else that is set for 14 days? I don't think so. Do you have any entries in the syslog showing the deletes? I'm pretty sure 'auditlog: 1' works in imapd.conf in 2.4.16, so you should be able to add that to see more logging. I finally got a chance to put in auditlog and I see several entries like this everyday: Oct 1 09:45:41 imap1 imap[7357]: auditlog: expunge sessionid=imap1.4test.net-7357-1412174740-1 mailbox=user.ken.Trash uniqueid=5f184bff4ee3c346 uid=170665 guid=5bbe70214bd4c7979f72977adda664afaa8ebe24 and then: Oct 1 09:45:41 imap1 imap[7357]: Expunged 19 messages from user.ken.Trash at then end. The system is now removing everything down to 7 days when info user.ken.Trash yields: {user.ken.Trash}: duplicatedeliver: false expire: 30 lastpop: lastupdate: 2-Oct-2014 10:39:17 -0500 partition: default pop3newuidl: true sharedseen: false size: 7749319 I did check the client as Alexei had suggested too as I thought the 14 days (and now 7) were a little coincidental, but there was nothing there causing the issue. Now after turning on auditlog, it does show the system removing these mails. Is this expire 30 number stored somewhere that has become corrupt or changed? This is just baffling to me as it seems to only affect this mailbox. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
messages removed before expire
Each of our user's Trash folders have an expiration of 30 days set using cyr_adm. Recently, one of the mailboxes began losing messages before the 30 days. Viewing mboxconfig inside cyradm for the user's Trash folder clearly shows: expire 30. I tried removing the value and re-setting it to no avail. The mailbox now only holds 14 days worth of email. We don't use delayed expunge on the system. Is there any other setting I might be missing that could be causing the mail to be removed prematurely? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: messages removed before expire
On Tue, Sep 16, 2014 at 3:02 PM, Bron Gondwana br...@fastmail.fm wrote: Dumb question - but I don't suppose anything happened to your clock recently? More non-dumb question, what version of Cyrus? Sorry, I should have included the version. I'm using 2.4.16 rpms from Invoca Systems. Nothing has happened with the clock. This is the only mailbox that seems to have the problem too, user.ken.Trash. We are running a murder configuration with 2 backends and 2 front ends. Not may users accounts, just large mailboxes. Maybe there is some setting on the mupdate master or somewhere else that is set for 14 days? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: frequently corrupt tis_sessions.db files
On Wed, Feb 26, 2014 at 12:27 AM, Willy Offermans wi...@offermans.rompen.nl wrote: Dear Steve and Cyrus friends, On Tue, Feb 25, 2014 at 11:03:30AM -0800, Stephen Ingram wrote: I'm running a very small murder setup using Simon Matter's RPM packages on CentOS 6.x. Frequently, the tis_sessions.db file on the update master becomes corrupt such that one or more of the nodes can no longer establish a connection. Of course, this results in folders not reserved properly on the master and a long list of issues from there. Each time I see the behavior in the logs: tlsv1 alert decrypt error in SSL_accept() - fail STARTTLS negotiation failed: imap1.xxx.xxx Connection reset by peer, closing connection Stopping cyrus-imapd, removing tls_sessions.db and then restarting cyrus-imapd always solves the problem. Is the tls database typically this unreliable (can't imagine why as it's the same db used for the mail, right?) and perhaps I should just not cache these connections or is there something else that could be wrong? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus How do you know that tls_sessions.db is corrupt? Is this really the right order: 1) tls_sessions.db becomes corrupt 2) tlsv1 alert decrypt error in SSL_accept() - fail STARTTLS negotiation failed: imap1.xxx.xxx Connection reset by peer, closing connection To me, the one is independent of the other, meaning that a corrupted tls_sessions.db is not related to STARTTLS and a STARTTLS negotiation failed does not necessarily leads to a corrupted tls_sessions.db. It very well could be; I'm really not sure. However, removing the tls_sessions.db file and restarting Cyrus solves the issue 100% of the time. Simply restarting Cyrus without changing anything does not. I'm certainly open to other possibilities, but in the absence of anything else, I'm thinking there is something to this. I suppose the next step might be trying to export the db to text format. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
frequently corrupt tis_sessions.db files
I'm running a very small murder setup using Simon Matter's RPM packages on CentOS 6.x. Frequently, the tis_sessions.db file on the update master becomes corrupt such that one or more of the nodes can no longer establish a connection. Of course, this results in folders not reserved properly on the master and a long list of issues from there. Each time I see the behavior in the logs: tlsv1 alert decrypt error in SSL_accept() - fail STARTTLS negotiation failed: imap1.xxx.xxx Connection reset by peer, closing connection Stopping cyrus-imapd, removing tls_sessions.db and then restarting cyrus-imapd always solves the problem. Is the tls database typically this unreliable (can't imagine why as it's the same db used for the mail, right?) and perhaps I should just not cache these connections or is there something else that could be wrong? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
murder system folders on different servers
I see in the documentation that it is better to leave trees of mailboxes on the same server. I'm wondering if this is an absolute requirement and I will experience problems if I do not follow this, or if it is more of a housekeeping issue. The reason I'm asking is that I would like to create an archive folder that resides on a less available system where mail could be moved for long term storage. Will I have problems if I do this? If so, is there any other recommendation to implement this sort of system? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Troubleshooting GSSAPI
On Fri, Sep 6, 2013 at 1:10 PM, Lorenzo Marcantonio l.marcanto...@logossrl.com wrote: I can't find a way to make GSSAPI authentication working with cyrus IMAP... (even tried the latest 'unstable' heimdal release). Configuration: - Cyrus SASL 2.1.26 - Cyrus IMAP 2.4.17 - Heimdal 1.5.2 or 1.6 (from git) - Latest mutt as an IMAP client (and imtest, of course) All of this on Linux x64. What does work: - IMAP on TLS using plaintext (in the log it says plaintext+TLS User logged in) - ssh authenticated with GSSAPI is ok (and delegates the tickets, too) - the two sample programs in cyrus-sasl correctly authenticate with GSSAPI (passing service imap and pointing to the keytab using the environment) So I am pretty sure that at least the easy stuff works. The principal is configured and exported in the keytab as realname.domain/REALM, the DNS has a CNAME record for imap.domain pointing to realname (doesn't work either, anyway...). Is this correct? When I try something like imtest -m GSSAPI realname.domain I get the capability banner with AUTH=GSSAPI available, then it goes A01 AUTHENTICATE GSSAPI (stuff) and it gets A01 NO generic failure. In the process the client actually acquired a ticket for the imap service. On the server side I see logged as following: imtest GSSAPI client step 1 kdc TGS-REQ (for the imap service ticket) imapo GSSAPI server step 1 imapo GSSAPI Error: No credentials were supplied, or the credentials were unavailable or inaccessible. (unknown mech-code 0 for mech unknown) imapo badlogin: host.from.where.im.trying GSSAPI [SASL(-1): generic failure: GSSAPI Error: (same as above) It seems the same error for a missing keytab or similar (however I straced imapd and it reads the right keytab file). The keytab of course contains the right key (I tested it using the SASL test programs). The relevant options in imapd.conf are: auth_mech: unix sasl_pwcheck_method: saslauthd sasl_mech_list: gssapi plain sasl_keytab: /data/imap/krb5.keytab sasl_allow_plaintext: true sasl_log_level: 7 log_level: 7 I would change auth_mech to krb5. I'm not sure what distro you are using, but you also need to export environment variables KRB5_KTNAME and KRB5CCNAME. I do not include the sasl_keytab or sasl_allow_plaintext settings in my config either, but I do have allowplaintext: no. I do allow plain text auth too, but only over TLS or SSL encrypted link. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Murder - MTA -- Auth required on frontend
On Tue, Aug 14, 2012 at 12:16 AM, shabahang elmian eshabah...@yahoo.com wrote: Hello, I have a problem on murder env. Env: 1 MTA on postfix (redhat221) 1 frontend+Mupdate (redhat101) 2 backends (redhat103, redhat112) if i pont the MTA to backend(mailbox_transport = lmtp:redhat101.example.com:2003), it works on the backend mail boxes. but when pointing MTA to frontend ,its getting a error as follow: Aug 14 10:39:59 localhost postfix/lmtp[5717]: 31E182408EF: to=test11...@example.com, relay=redhat101.example.com[10.131.57.101]:2003, delay=0.29, delays=0.12/0.01/0.16/0, dsn=4.0.0, status=deferred (host redhat101.example.com[10.131.57.101] said: 430 Authentication required (in reply to MAIL FROM command)) I would be thankful if you could help me on the problem. Best regards, Shabahang - Config on frontend : [root@redhat101 ~]# egrep -v ^#|^$ /etc/cyrus.conf START { # do not delete this entry! recover cmd=ctl_cyrusdb -r # this is only necessary if using idled for IMAP IDLE idled cmd=/usr/lib/cyrus-imapd/idled } SERVICES { # add or remove based on preferences mupdate cmd=/usr/cyrus/bin/mupdate -m listen=3905 prefork=1 imap cmd=proxyd listen=imap prefork=5 imaps cmd=proxyd -s listen=imaps prefork=1 pop3 cmd=pop3d listen=pop3 prefork=0 pop3s cmd=pop3d -s listen=pop3s prefork=0 kpop cmd=pop3d -k listen=kpop prefork=0 nntp cmd=/usr/lib/cyrus-imapd/nntpd listen=nntp prefork=0 nntps cmd=/usr/lib/cyrus-imapd/nntpd -s listen=nntps prefork=0 sieve cmd=timsieved listen=sieve prefork=0 lmtp cmd=/usr/cyrus/bin/lmtpproxyd listen=0.0.0.0:lmtp prefork=0 You don't need 0.0.0.0, just lmtp cmd=lmtpproxyd listen:lmtp will do. # these are only necessary if receiving/exporting usenet via NNTP # at least one LMTP is required for delivery # this is only necessary if using notifications } EVENTS { # this is required checkpoint cmd=ctl_cyrusdb -c period=30 # this is only necessary if using duplicate delivery suppression, # Sieve or NNTP delprune cmd=cyr_expire -E 3 at=0400 # this is only necessary if caching TLS sessions tlsprune cmd=tls_prune at=0400 } [root@redhat101 ~]# egrep -v ^#|^$ /etc/imapd.conf configdirectory: /var/lib/imap partition-default: /var/spool/imap admins: cyrus sievedir: /var/lib/imap/sieve sendmail: /usr/sbin/sendmail hashimapspool: true sasl_pwcheck_method: saslauthd sasl_mech_list: PLAIN PLAIN+TLS LOGIN DIGEST-MD5 CRAM-MD5 tls_cert_file: /etc/pki/cyrus-imapd/server.pem tls_key_file: /etc/pki/cyrus-imapd/server.pem tls_ca_file: /etc/pki/cyrus-imapd/server.pem tls_ca_path: /etc/pki/cyrus-imapd/ allowplaintext: yes redhat112_password: password redhat103_password: password proxy_authname: cyrus allowanonymouslogin: yes lmtp_admins: cyrus [root@redhat101 ~]# config on backend : [root@redhat103 ~]# egrep -v ^#|^$ /etc/cyrus.conf START { # do not delete this entry! recover cmd=ctl_cyrusdb -r # this is only necessary if using idled for IMAP IDLE idled cmd=idled } SERVICES { # add or remove based on preferences imap cmd=imapd listen=imap prefork=5 imaps cmd=imapd -s listen=imaps prefork=1 pop3 cmd=pop3d listen=pop3 prefork=3 pop3s cmd=pop3d -s listen=pop3s prefork=1 sieve cmd=timsieved listen=sieve prefork=0 # these are only necessary if receiving/exporting usenet via NNTP # at least one LMTP is required for delivery lmtp cmd=lmtpd -a listen=0.0.0.0:lmtp prefork=1 lmtpunix cmd=lmtpd listen=/var/lib/imap/socket/lmtp prefork=1 # this is only necessary if using notifications Two things wrong here. First, you only need one line. lmtpunix is for listening on a unix socket. lmtp is for listening on a tcp socket--this is the one you need if postfix is on another server. Also, the -a is telling lmtp to use preauthorized connections. This is not what you've told Postfix. Get rid of it. Make sure you tell Cyrus your lmtp password in imapd.conf. So, lmtp cmd=lmtpd listen:lmtp prefork=1 will work. } EVENTS { # this is required checkpoint cmd=ctl_cyrusdb -c period=30 # this is only necessary if using duplicate delivery suppression, # Sieve or NNTP delprune cmd=cyr_expire -E 3 at=0400 # this is only necessary if caching TLS sessions tlsprune cmd=tls_prune at=0400 } [root@redhat103 ~]# egrep -v ^#|^$ /etc/imapd.conf configdirectory: /var/lib/imap partition-default: /var/spool/imap admins: cyrus mupdateslave1 backend1 sievedir: /var/lib/imap/sieve sendmail: /usr/sbin/sendmail hashimapspool: true sasl_pwcheck_method: saslauthd sasl_mech_list: PLAIN PLAIN+TLS tls_cert_file: /etc/pki/cyrus-imapd/server.pem tls_key_file: /etc/pki/cyrus-imapd/server.pem tls_ca_file: /etc/pki/cyrus-imapd/server.pem tls_ca_path: /etc/pki/cyrus-imapd/ allowplaintext: yes allowanonymouslogin: yes
mailbox vs login referrals
I'm attempting to get another program to interface with Cyrus-IMAP and was asked should we support mailbox or login referrals. I'm thinking that Cyrus supports mailbox referrals, but does it support login as well? If so, is there an advantage to one over the other? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
lmtp user lmtp_admins or admins on murder front-end system
I'm trying to test an lmtp connection from the MTA using lmtptest. I'm connecting to a murder frontend system which then lmtpproxy's to the backend. On the frontend system I have: lmtp_admins: lmtp-kerberos-principal On the backend system I have: lmtp_admins: lmtp-kerberos-principal Every time I try to connect using lmtptest -m GSSAPI frontend.system.com I get: badlogin: x.x.x.x GSSAPI SASL(-13): authentication failure: only admins may authenticate. When I switch the frontend configuration to: admins: lmtp-kerberos-principal everything works. I'm trying to limit admin access as much as possible. While the backend is still protected, the proxy is not. Is this possible to do or is it not a concern any way since it's only a frontend proxy? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Sat, Jun 23, 2012 at 10:55 PM, Stephen Ingram sbing...@gmail.com wrote: On Thu, Jun 14, 2012 at 9:14 PM, Dan White dwh...@olp.net wrote: ...snip... You can control whether clients will get referrals via the proxyd_disable_mailbox_referrals option. When proxying, you would configure the 'cyrus-hostname' user within the proxyservers option on the backend. When the frontend authenticates to the backend, it will send an authorization identity of the previously authenticated frontend user. Like: authcid: none (derived from your kerberos identity) authzid: jsmith Then, from the backend's perspective, jsmith performed the authentication, and gets all the proper ACL permissions applied. The frontend *might* have all the appropriate service principals in place to support client gssapi authentication, however that's not necessary. The client authentication to the frontend, and the frontend's proxy authentication to the backend are distinct authentications. The frontend *will* need to have a non-service principal ticket initialized when performing gssapi authentication to the backend. Well, while I'm still not sure which way to go on the issue of where to place the service keytabs, your assertion that one *must* use a user to authenticate from the frontend to the backend in order to proxy the user authentication through seems to be correct. However, I'm not sure exactly why. When I test with imtest as user cyrus using the service principals in the same credential cache as fetched by the program, it works great. Even when testing with the service principals in place with the processes running, examining the caches, all the tickets appear to be properly granted and in the cache, however, every time there is a: couldn't authenticate to backend server: authentication failure error. Is there something specific in the cyrus-imapd code the ensures only a user principal will work? Is there some rationale to this? I've been told by everyone I've asked that there is no difference between user and service principals. Is it as something as silly as the / you alluded to omitting from your user principals before so you could satisfy libsasl2? After a little more testing, yes, it appears as though the / is disallowed. But Dave you said you are using imap/`hostname`@ANDREW.CMU.EDU. What happens if you want to use an admin instance of a user principal (e.g. steve/ad...@andrew.cmu.edu)? Has this changed and Dave, you are using a different version than Dan and I? Sorry to keep pounding on this issue, but I don't want to write documentation unless I really understand what is going on. BTW, the service principal works perfectly with the mupdate client to server auth. I have a feeling that the sync would work too. It seems to have something to do with the user proxy that Dan was describing. Maybe only a user can proxy another user and not a service? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Thu, Jun 14, 2012 at 9:14 PM, Dan White dwh...@olp.net wrote: ...snip... You can control whether clients will get referrals via the proxyd_disable_mailbox_referrals option. When proxying, you would configure the 'cyrus-hostname' user within the proxyservers option on the backend. When the frontend authenticates to the backend, it will send an authorization identity of the previously authenticated frontend user. Like: authcid: none (derived from your kerberos identity) authzid: jsmith Then, from the backend's perspective, jsmith performed the authentication, and gets all the proper ACL permissions applied. The frontend *might* have all the appropriate service principals in place to support client gssapi authentication, however that's not necessary. The client authentication to the frontend, and the frontend's proxy authentication to the backend are distinct authentications. The frontend *will* need to have a non-service principal ticket initialized when performing gssapi authentication to the backend. If I'm reading this correctly, you are saying that you really don't need any of the services (imap,sieve,nntp,pop) in the keytab on the frontend, but only the backend. The frontend authenticates to the backend using it's own credentials (in my case the credential cache from imap/imap.example.com) and proxies the user ticket to the backend services (even with proxyd_disable_mailbox_referrals turned on). It looks like Dave is authenticating on the frontend instead. Is this just a different way of doing things or does each come with advantages/disadvantages? I would think that you *would* need to make the authcid to authzid determination on the backend, so I wonder how this is working for him? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Thu, Jun 14, 2012 at 9:14 PM, Dan White dwh...@olp.net wrote: ...snip... You can control whether clients will get referrals via the proxyd_disable_mailbox_referrals option. When proxying, you would configure the 'cyrus-hostname' user within the proxyservers option on the backend. When the frontend authenticates to the backend, it will send an authorization identity of the previously authenticated frontend user. Like: authcid: none (derived from your kerberos identity) authzid: jsmith Then, from the backend's perspective, jsmith performed the authentication, and gets all the proper ACL permissions applied. The frontend *might* have all the appropriate service principals in place to support client gssapi authentication, however that's not necessary. The client authentication to the frontend, and the frontend's proxy authentication to the backend are distinct authentications. The frontend *will* need to have a non-service principal ticket initialized when performing gssapi authentication to the backend. Well, while I'm still not sure which way to go on the issue of where to place the service keytabs, your assertion that one *must* use a user to authenticate from the frontend to the backend in order to proxy the user authentication through seems to be correct. However, I'm not sure exactly why. When I test with imtest as user cyrus using the service principals in the same credential cache as fetched by the program, it works great. Even when testing with the service principals in place with the processes running, examining the caches, all the tickets appear to be properly granted and in the cache, however, every time there is a: couldn't authenticate to backend server: authentication failure error. Is there something specific in the cyrus-imapd code the ensures only a user principal will work? Is there some rationale to this? I've been told by everyone I've asked that there is no difference between user and service principals. Is it as something as silly as the / you alluded to omitting from your user principals before so you could satisfy libsasl2? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Wed, Jun 20, 2012 at 6:29 AM, Dan White dwh...@olp.net wrote: On 06/19/12 19:04 -0700, Stephen Ingram wrote: Thank you for your continued help with this. I really appreciate it and am determined to get to the end of this. I think I'm getting closer. I have successfully authenticated using mupdatetest from one of the backends to the mupdate server. I'm using service principals on both ends. I've even specified the imap/imap1.example.com part of the principal in the admins: section of the configuration and after solving several configuration issues on my end, it seems to work! I came across a post from you some time ago talking about /etc/krb.equiv. Would this be an easier way to do this? I tried placing that file on the mupdate server and loaded it with imap/imap1.example.com imap1 and then placed admins: imap1 in my imapd.conf file, but I'm not sure if it works. Do I have to tell cyrus about that file somewhere? I have not used /etc/krb.equiv before, but the last time I dug into the code trying to understand it, I came away with the impression that it's used for kerberosv4 only. Apparently it would be a way to map 'imap/imap1.example.com' to 'imap1'. It might work just as well to just place 'imap/imap1.example.com' or 'imap/imap1.example@example.com' into your proxyservers/*_admins entries. I know that this format works, because it's what I currently have in my config: cyrus-mail1.example@example.net Yes, I was able to get this to work too with mupdatetest, even without the @EXAMPLE.COM piece. I'm guessing if you don't include the realm, it just adds the default one. The krb.equiv does seem to work too, although since you can just specify the entire principal, I'm not sure that it is really that useful in this instance. One other question though, how often do you refresh your credential cache? From the few examples I've seen, most people seem to refresh very frequently (anywhere from 6 minutes to 1 hour). Given that most tickets can last up to 10 or 12 hours, I'm guessing the shorter life is for security or some other reason? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Sun, Jun 17, 2012 at 8:21 PM, Dan White dwh...@olp.net wrote: On 06/17/12 18:04 -0700, Stephen Ingram wrote: On Thu, Jun 14, 2012 at 9:14 PM, Dan White dwh...@olp.net wrote: ...snip... Another way to keep your principals straight is that you'll need a user principal where you will run the *test utilities, and a service principal on the server that the *test utility will connect to. The service principals will be initialized for you by libsasl2, and the user principals will need to be kinit'd via some other mechanism (like in your START/EVENTS section). ...snip... The frontend *will* need to have a non-service principal ticket initialized when performing gssapi authentication to the backend. This is *exactly* what I continue to be confused about. Can't a service principal be used on both client and server sides? To me a user should only be a physical person that would login, not a process. For example, can the authenticated (mupdate client and backend) mupdate/imap1.example@example.com connect to (mupdate server) mupdate/murder.example@example.com. Why couldn't this happen? That may work, however you'd need to kinit (or initialize by some other mechanism) on imap1 since the client GSSAPI mechanism won't do that for you. You can still authenticate from a keytab with kinit. You may end up with two different TGTs on imap1. It may be a nightmare to attempt to authenticate from the client side with different service principals, like: mupdate/imap1.example.com imap/imap1.example.com (for proxying) lmtp/imap1.example.com etc. The client side GSSAPI mechanism would need to be told which credentials cache to use for that particular type of authentication, such as with an environment variable. You could post to the cyrus-sasl list to see if someone there has a better recommendation. GS2 is a newer kerberos based authentication mechanism that may handle this in a more sensible way. Thank you for your continued help with this. I really appreciate it and am determined to get to the end of this. I think I'm getting closer. I have successfully authenticated using mupdatetest from one of the backends to the mupdate server. I'm using service principals on both ends. I've even specified the imap/imap1.example.com part of the principal in the admins: section of the configuration and after solving several configuration issues on my end, it seems to work! I came across a post from you some time ago talking about /etc/krb.equiv. Would this be an easier way to do this? I tried placing that file on the mupdate server and loaded it with imap/imap1.example.com imap1 and then placed admins: imap1 in my imapd.conf file, but I'm not sure if it works. Do I have to tell cyrus about that file somewhere? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Thu, Jun 14, 2012 at 9:14 PM, Dan White dwh...@olp.net wrote: ...snip... Another way to keep your principals straight is that you'll need a user principal where you will run the *test utilities, and a service principal on the server that the *test utility will connect to. The service principals will be initialized for you by libsasl2, and the user principals will need to be kinit'd via some other mechanism (like in your START/EVENTS section). ...snip... The frontend *will* need to have a non-service principal ticket initialized when performing gssapi authentication to the backend. This is *exactly* what I continue to be confused about. Can't a service principal be used on both client and server sides? To me a user should only be a physical person that would login, not a process. For example, can the authenticated (mupdate client and backend) mupdate/imap1.example@example.com connect to (mupdate server) mupdate/murder.example@example.com. Why couldn't this happen? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Thu, Jun 14, 2012 at 6:20 AM, Dave McMurtrie dav...@andrew.cmu.edu wrote: On 06/14/2012 12:02 AM, Stephen Ingram wrote: This is exactly the part I'm really confused about. For murder, I see connections from the frontends and backends to the mupdate server. I also see connections from the frontends to the backends. The connections to the mupdate server are, in a very simplistic sense, to spread information about the mailboxes. I was thinking these should be machine to machine connections using Kerberos service accounts. Let me try to run through which keys exist on various servers in our environment. I think that will possibly clear things up a bit. In the keytab on our Cyrus frontend servers: * imap/cyrus.andrew.cmu@andrew.cmu.edu service principal. This is used for end-user client authentication to the imap service running on cyrus.andrew.cmu.edu hosts. * pop/cyrus.andrew.cmu@andrew.cmu.edu service principal. This is used for end-user client authentication to the pop service running on cyrus.andrew.cmu.edu hosts. * sieve/cyrus.andrew.cmu@andrew.cmu.edu service principal. This is used for end-user client authentication to the sieve service running on cyrus.andrew.cmu.edu hosts. * nntp/cyrus.andrew.cmu@andrew.cmu.edu service principal. This is used for end-user client authentication to the nntp service running on cyrus.andrew.cmu.edu hosts. I notice for these 4 service principals you are using cyrus.andrew.cmu.edu for the fqdn. Does this mean that you are using the same principals/keys for each of your frontend systems, your frontend systems are all called cyrus with some sort of load balancing or do you actually place a host name in front of cyrus? * imap/`hostname`@ANDREW.CMU.EDU. The Cyrus master process runs authenticated as this principal. We accomplish this by having a simple cyrus.auth script run on startup from cyrus.conf, and also as a recurring event in cyrus.conf. It does nothing more than set KRB5CCNAME and run kinit. These credentials are used to authenticate to our mupdate server and to each of our Cyrus backend servers. So since this service principal is always authenticated, you can use it without further intervention to connect to the mupdate and backend servers? * host/cyrus.andrew.cmu@andrew.cmu.edu. This could really use some documentation. It's used by saslauthd when saslauthd is configured to use kerberos5. The idea is that saslauthd takes the user credentials via a socket and uses those to request a tgt. To make sure it wasn't talking to a spoofed KDC, it then uses that tgt to request a host service ticket. host is hard-coded in saslauthd. I already have this setup. It was the easiest for me to understand since it starts with an authenticated service principal and is more like the user-login kerberos I'm used to setting up. I'm sure this will take on a whole new light once I fully understand the always-authenticated service principals. In the keytab on our Cyrus backend servers: --- * imap/`hostname`@ANDREW.CMU.EDU. The Cyrus master process runs authenticated as this principal. We accomplish this by having a simple cyrus.auth script run on startup from cyrus.conf, and also as a recurring event in cyrus.conf. It does nothing more than set KRB5CCNAME and run kinit. These credentials are used to authenticate to our mupdate server. If a client were to connect directly to one of our backends, it would use this service principal to authenticate. If you disable referrals, you won't need to account for clients connecting directly to your backends and you therefore won't need any service principals for client authentication. Again, this is running authenticated so that it can connect to the mupdate server? I don't think we want users logging directly into the backends, but if we did, then we would also need imap, pop (if using), sieve and nntp principals too, right? However, I'm not really sure, should only the mupdate server have an mupdate service principals and then the frontend clients and backend clients connect to mupdate using user kerberos principals, or if all servers involved have service principals. Also when proxying a mail connection from frontend to backend, how should this be done? The frontends authenticate to the backends using their imap/`hostname`@ANDREW.CMU.EDU credentials (remember, our master process runs authenticated). Proxy authentication is supported in Cyrus-SASL for GSSAPI, so the imap/`hostname` credentials are used for authentication, but the connection is authorized as the real user, or the user who authenticated to the frontend. Hence, in the Cyrus logs on the backend you'll see the real username as having logged in, not your imap/`hostname` principal. And is it correct that this is where the warning comes in about being careful of the admins: setting
Re: GSSAPI for various murder component setups
On Thu, Jun 14, 2012 at 7:05 AM, Dan White dwh...@olp.net wrote: On 06/13/12 21:02 -0700, Stephen Ingram wrote: On Wed, Jun 13, 2012 at 1:23 PM, Dan White dwh...@olp.net wrote: The other issue is that where your systems are acting as clients (such as when a frontend server is connecting to an mupdate server), your client will need to initialize a kerberos ticket cache, and in my experience cannot use the kerberos credentials used to accept connections. Or in other words, your frontends might have an imap/mail.example.net service ticket for accepting client imap connections, but then may need a separate ticket, such as cyrus/mail.example.net, for backend/mupdate connections. I use cronjobs, running as the cyrus user, to initialize those crendential caches. This is exactly the part I'm really confused about. For murder, I see connections from the frontends and backends to the mupdate server. I also see connections from the frontends to the backends. The connections to the mupdate server are, in a very simplistic sense, to spread information about the mailboxes. I was thinking these should be machine to machine connections using Kerberos service accounts. However, I'm not really sure, should only the mupdate server have an mupdate service principals and then the frontend clients and backend clients connect to mupdate using user kerberos principals, or if all servers involved have service principals. Also when proxying a mail connection from frontend to backend, how should this be done? And then there is replication Every service listed within your SERVICES section in cyrus.conf will potentially need it's own service principal, particularly on your backends and mupdate master. Your frontends may not need service principals if your users are not performing GSSAPI authentication. libsasl2 will search for for service principals starting with: imap/ lmtp/ mupdate/ csync/ pop/ nntp/ sieve/ Wouldn't the front ends need these connections worse than the backends (assuming I'm not supporting referrals)? I'm guessing the lmtp is for Postfix connecting to the frontend/proxy to backend to deliver the message? The csync is for replication? when initialized during service startup. Within your imapd.conf, you can restrict authentication only to gssapi with: imap_sasl_mech_list: gssapi etc. The *test utilities (lmtptest, imtest, mupdatetest, etc.) are invaluable for validating the server side of your setup. Every server in your murder, except perhaps your replica server, will likely need an additional client/user principal. Why wouldn't the replica server need a service principal since the backend connects to it to sync? When proxying from the frontend to the backend, the frontend will make a gssapi connection to the backend regardless of the authentication method the client used when connecting to the frontend. If the client supports referrals, then the client will then make it's own connection to the backend using which ever authentication method it's configured to use. But only if the backend is configured for that authentication method, no? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
GSSAPI for various murder component setups
There seems to be quite a bit of information on the Website about setting up a murder configuration. Most of the documentation, however, seems to be centered on basic authentication. Is there a good resource somewhere to using Kerberos to setup the communication between the mupdate, frontend and backend servers for mupdate, imap and replication processes? I see some configs in the distribution conf directory from CMU setups, but they are only partially complete and based on Kerberos 4. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: GSSAPI for various murder component setups
On Wed, Jun 13, 2012 at 1:23 PM, Dan White dwh...@olp.net wrote: On 06/13/12 12:57 -0700, Stephen Ingram wrote: There seems to be quite a bit of information on the Website about setting up a murder configuration. Most of the documentation, however, seems to be centered on basic authentication. Is there a good resource somewhere to using Kerberos to setup the communication between the mupdate, frontend and backend servers for mupdate, imap and replication processes? I see some configs in the distribution conf directory from CMU setups, but they are only partially complete and based on Kerberos 4. There are two differences that come to mind: When configuring authentication, you can simply leave the authname and password out of your configuration, such as: mupdate_server: mupdate.example.net # mupdate_port # mupdate_username: # mupdate_authname # mupdate_realm # mupdate_password # mupdate_retry_delay mupdate_config: standard The other issue is that where your systems are acting as clients (such as when a frontend server is connecting to an mupdate server), your client will need to initialize a kerberos ticket cache, and in my experience cannot use the kerberos credentials used to accept connections. Or in other words, your frontends might have an imap/mail.example.net service ticket for accepting client imap connections, but then may need a separate ticket, such as cyrus/mail.example.net, for backend/mupdate connections. I use cronjobs, running as the cyrus user, to initialize those crendential caches. This is exactly the part I'm really confused about. For murder, I see connections from the frontends and backends to the mupdate server. I also see connections from the frontends to the backends. The connections to the mupdate server are, in a very simplistic sense, to spread information about the mailboxes. I was thinking these should be machine to machine connections using Kerberos service accounts. However, I'm not really sure, should only the mupdate server have an mupdate service principals and then the frontend clients and backend clients connect to mupdate using user kerberos principals, or if all servers involved have service principals. Also when proxying a mail connection from frontend to backend, how should this be done? And then there is replication I guess I'm mostly confused about whether and where to use user and/or service principals and how does the other end know that it is being connected to correctly. For instance is the mupdate server expecting a user in the admins group to connect to retrieve the mailbox list or simply a machine and where is that specified in the configuration files? I've found a couple of configuration files floating around in the mailing list archives and was confused even more after looking at it for they only seem to cache credentials for service principal type accounts by inserting lines into the cyrus.conf file to create and then renew credentials on a regular basis. I'm really shocked that there is no good documentation on this. Am I going down a road that hardly anyone travels on? Or, are those who have ventured there simply too exhausted to write about it? Considering how great this all seems, I can't believe more people don't deploy this type of setup as it seems much more secure than using plain text passwords. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
mailboxes.db vs IMAP client irregularities
I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had an issue with a folder in a mailbox that would not show any subfolders. I created a new folder 'folder2' and moved all of the subfolders to it and then performed a reconstruct on the new set of folders and everything worked. Now I deleted the old folder 'folder' from the file system and then (after it wouldn't go away from the cyradm listing) used cyr_dbtool to manually remove it (and the subfolders) from the mailboxes.db file. The old folders and subfolders are now gone, however, I can't (using the IMAP client) rename 'folder2' back to 'folder' as when I do, the subfolders are not visible. I've dumped the mailboxes.db file to a flat file to look and see if there is anything in there that wasn't visible in cyradm or using cyr_dbtool show. Everything is as expected except there are some DELETED.user.xxx.folder entries at the top. Are you not allowed the create folders with the same name you've just deleted? Where are these DELETED folders actually stored and how long does it take them to go away? (I'm not using delayed expunge.) If that's not the issue, then is there some other file besides mailboxes.db that might contain bad information or is this a bug in the system? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailboxes.db vs IMAP client irregularities
On Sat, May 19, 2012 at 10:59 AM, Bron Gondwana br...@fastmail.fm wrote: On Sat, May 19, 2012 at 09:51:38AM -0700, Stephen Ingram wrote: I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had Could be sort order bugs with 2.4.13 if you don't have improved_mboxlist_sort turned on. I've dumped the mailboxes.db file to a flat file to look and see if there is anything in there that wasn't visible in cyradm or using cyr_dbtool show. Everything is as expected except there are some DELETED.user.xxx.folder entries at the top. Are you not allowed the create folders with the same name you've just deleted? Where are these DELETED folders actually stored and how long does it take them to go away? (I'm not using delayed expunge.) Do you have folder names like: user.xxx user.xxx.fol der user.xxx.fol.stuff rather than: user.xxx user.xxx.fol.stuff user.xxx.fol der A fix is to upgrade to a new version of 2.4.x. Are you saying that the sort order could be wrong or that the incorrect sort order could be causing the other issues I'm seeing? Should I just remove those DELETED folder entries? They don't seem to exist anywhere in the file system. With your mention of bugs though, I went through the bugs fixed from 2.14-2.16 and I see bug 2685. That sort of sounds like what I'm experiencing. It sounds like I should upgrade before spending too much more time on this. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailboxes.db vs IMAP client irregularities
On Sat, May 19, 2012 at 10:58 AM, Patrick Boutilier bouti...@ednet.ns.ca wrote: On 05/19/2012 01:51 PM, Stephen Ingram wrote: I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had an issue with a folder in a mailbox that would not show any subfolders. I created a new folder 'folder2' and moved all of the subfolders to it and then performed a reconstruct on the new set of folders and everything worked. Now I deleted the old folder 'folder' from the file system and then (after it wouldn't go away from the cyradm listing) used cyr_dbtool to manually remove it (and the subfolders) from the mailboxes.db file. The old folders and subfolders are now gone, however, I can't (using the IMAP client) rename 'folder2' back to 'folder' as when I do, the subfolders are not visible. I've dumped the mailboxes.db file to a flat file to look and see if there is anything in there that wasn't visible in cyradm or using cyr_dbtool show. Everything is as expected except there are some DELETED.user.xxx.folder entries at the top. Are you not allowed the create folders with the same name you've just deleted? Where are these DELETED folders actually stored and how long does it take them to go away? (I'm not using delayed expunge.) Sounds like you are using delayed delete. Mine show up in /imap/mail/C/DELETED/ . How long they stay around depends on when you run cyr_expire and what parameters you give it. Man page entries: deletedprefix: DELETED If delete_mode set to be delayed, the prefix for the deleted mailboxes hierarchy. The hierarchy delimiter will be automatically appended. delete_mode: immediate The manner in which mailboxes are deleted. immediate mode is the default behavior in which mailboxes are removed immediately. In delayed mode, mailboxes are renamed to a special hiearchy defined by the deletedprefix option to be removed later by cyr_expire. Allowed values: immediate, delayed I don't have any entry in imapd.conf for delete_mode so I'm thinking that it's using the default of immediate. I did check the file system again and I can't find those DELETED folders anywhere, so I'm guessing that mailboxes.db is just wrong probably because of some bug. At your suggestion, I checked cyr_expire and it is set to 3 days, so perhaps that's my issue. Maybe those entries in mailboxes.db will go away on their own after 3 days. I think I might wait just out of curiosity and then upgrade to 2.4.16 to see if these mailbox naming issue (bug 2685?) is solved. I wonder if there is any harm in removing those DELETED entries in mailboxes.db if I can't find them anywhere in the file system? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailboxes.db vs IMAP client irregularities
On Sat, May 19, 2012 at 12:40 PM, Patrick Boutilier bouti...@ednet.ns.ca wrote: On 05/19/2012 04:02 PM, Stephen Ingram wrote: On Sat, May 19, 2012 at 10:58 AM, Patrick Boutilier bouti...@ednet.ns.ca wrote: On 05/19/2012 01:51 PM, Stephen Ingram wrote: I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had an issue with a folder in a mailbox that would not show any subfolders. I created a new folder 'folder2' and moved all of the subfolders to it and then performed a reconstruct on the new set of folders and everything worked. Now I deleted the old folder 'folder' from the file system and then (after it wouldn't go away from the cyradm listing) used cyr_dbtool to manually remove it (and the subfolders) from the mailboxes.db file. The old folders and subfolders are now gone, however, I can't (using the IMAP client) rename 'folder2' back to 'folder' as when I do, the subfolders are not visible. I've dumped the mailboxes.db file to a flat file to look and see if there is anything in there that wasn't visible in cyradm or using cyr_dbtool show. Everything is as expected except there are some DELETED.user.xxx.folder entries at the top. Are you not allowed the create folders with the same name you've just deleted? Where are these DELETED folders actually stored and how long does it take them to go away? (I'm not using delayed expunge.) Sounds like you are using delayed delete. Mine show up in /imap/mail/C/DELETED/ . How long they stay around depends on when you run cyr_expire and what parameters you give it. Man page entries: deletedprefix: DELETED If delete_mode set to be delayed, the prefix for the deleted mailboxes hierarchy. The hierarchy delimiter will be automatically appended. delete_mode: immediate The manner in which mailboxes are deleted. immediate mode is the default behavior in which mailboxes are removed immediately. In delayed mode, mailboxes are renamed to a special hiearchy defined by the deletedprefix option to be removed later by cyr_expire. Allowed values: immediate, delayed I don't have any entry in imapd.conf for delete_mode so I'm thinking that it's using the default of immediate. I did check the file system again and I can't find those DELETED folders anywhere, so I'm guessing that mailboxes.db is just wrong probably because of some bug. At your suggestion, I checked cyr_expire and it is set to 3 days, so perhaps that's my issue. Maybe those entries in mailboxes.db will go away on their own after 3 days. I think I might wait just out of curiosity and then upgrade to 2.4.16 to see if these mailbox naming issue (bug 2685?) is solved. I wonder if there is any harm in removing those DELETED entries in mailboxes.db if I can't find them anywhere in the file system? I am thinking the worst that would happen is you get an I/O error if the DELETED hierarchy really doesn't exist. Have you tried mbpath? For example: [root@student ~]# /usr/local/cyrus/bin/mbpath DELETED.user.whoj.4FB1557F /imap/mail/C/DELETED/user/whoj/4FB1557F Wow, thanks! I didn't know about that command. It exposed them! Strange, they were in /var/spool/imap/u/DELETED/ (my mailbox root is /var/spool/imap). I'm not sure why they wouldn't be in /var/spool/imap/j/user/jmaxwell (jmaxwell is the user), but perhaps it's because I haven't defined deleted_prefix as you pointed out earlier. Maybe /var/spool/imap/u/DELETED is the default. Looking at all of the files in there, several are older than the 3 days they are supposed to be. I'm guessing that means there was a bug somewhere. I guess I should remove all of these, reconstruct the mailboxes.db to match and then probably upgrade as Bron suggested. Thanks, again. I never would have found those files. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailboxes.db vs IMAP client irregularities
On Sat, May 19, 2012 at 1:00 PM, Simon Matter simon.mat...@invoca.ch wrote: On 05/19/2012 01:51 PM, Stephen Ingram wrote: I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had an issue with a folder in a mailbox that would not show any subfolders. I created a new folder 'folder2' and moved all of the subfolders to it and then performed a reconstruct on the new set of folders and everything worked. Now I deleted the old folder 'folder' from the file system and then (after it wouldn't go away from the cyradm listing) used cyr_dbtool to manually remove it (and the subfolders) from the mailboxes.db file. The old folders and subfolders are now gone, however, I can't (using the IMAP client) rename 'folder2' back to 'folder' as when I do, the subfolders are not visible. I've dumped the mailboxes.db file to a flat file to look and see if there is anything in there that wasn't visible in cyradm or using cyr_dbtool show. Everything is as expected except there are some DELETED.user.xxx.folder entries at the top. Are you not allowed the create folders with the same name you've just deleted? Where are these DELETED folders actually stored and how long does it take them to go away? (I'm not using delayed expunge.) Sounds like you are using delayed delete. Mine show up in /imap/mail/C/DELETED/ . How long they stay around depends on when you run cyr_expire and what parameters you give it. Man page entries: deletedprefix: DELETED If delete_mode set to be delayed, the prefix for the deleted mailboxes hierarchy. The hierarchy delimiter will be automatically appended. delete_mode: immediate The manner in which mailboxes are deleted. immediate mode is the default behavior in which mailboxes are removed immediately. In delayed mode, mailboxes are renamed to a special hiearchy defined by the deletedprefix option to be removed later by cyr_expire. Just to clear this up, if he's using our invoca rpms then his man page reads: delete_mode: delayed The manner in which mailboxes are deleted. immediate mode is the the mode in which mailboxes are removed immediately. In delayed mode, mailboxes are renamed to a special hiearchy defined by the deletedprefix option to be removed later by cyr_expire. Note: This Invoca RPM build uses delayed by default instead of immedi- ate for delete_mode. Allowed values: immediate, delayed Oops, sorry to mislead. Yes, I was lazy and just read a man page from Google. I will change to delete_mode: immediate. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailboxes.db vs IMAP client irregularities
On Sat, May 19, 2012 at 1:28 PM, Bron Gondwana br...@fastmail.fm wrote: On Sat, May 19, 2012 at 12:56:44PM -0700, Stephen Ingram wrote: Wow, thanks! I didn't know about that command. It exposed them! Strange, they were in /var/spool/imap/u/DELETED/ (my mailbox root is /var/spool/imap). I'm not sure why they wouldn't be in /var/spool/imap/j/user/jmaxwell (jmaxwell is the user), but perhaps it's because I haven't defined deleted_prefix as you pointed out earlier. Maybe /var/spool/imap/u/DELETED is the default. Hashing is on the second part of the name, rather uninteligently, no matter what the name - hence it's hashing on: DELETE.user.x ^ :) Looking at all of the files in there, several are older than the 3 days they are supposed to be. I'm guessing that means there was a bug somewhere. I guess I should remove all of these, reconstruct the mailboxes.db to match and then probably upgrade as Bron suggested. No - the files will have the delivery date as their age. The interesting bit is the final part of the folder name, which is actually a 32 bit unix timestamp in hex format (yes, really). Wow, I never stopped being amazed by the really clever hidden things in cyrus-imap! cyr_expire will clean it up. Don't mess with the filesystem under cyrus if you don't have to. OK, this must be working or I would have tons of DELETED files in there. However, I see stuff from April that is not gone yet. And, yes, I actually converted the hex timestamp to a date an it says April too, certainly more than the 3 day expire time. Hopefully this stuff will be taken care of when I upgrade to 2.4.16 too! BTW, I love 2.4. I've been using now for several months and it is such a HUGE improvement from 2.3. Everything is faster, replication works better, it's like a whole new program! Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Compiling latest version for Redhat 6.
On Tue, Mar 20, 2012 at 11:14 AM, Mark London m...@psfc.mit.edu wrote: Hi - I'm about to try installing the latest version of Cyrus on Redhat 6, and wanted to get any feedback from anyone, about any problems that I might run into. The reason I'm dgo this, is that we've recently been experiencing a problem with the older supplied version of cyrus that comes with redhat. Occasionally, a user's account will get stuck. It appears it might involve the user's .seen file, as attempts to access the user's account gets stuck in the code that tries to open that file. The problem might have not be new, but we're only seeing it now, because of much heavier usage of the mail server. I don't have the time right now to fully debug the problem, so I figured it might be easier to first simply install the latest version. I have compiled it on a test redhat 6 server, and it seems to work fine. I'll be porting to our real server tonight, but I wonder if there are any gotchas, that I might run into, that won't be noticable until it actually starts to heavy use. I used the Invoca srpms from Simon Matter and built them for Redhat 5 and upgraded from the 2.3.x version supplied with Redhat. All of the databases update in place (new version uses all skiplist instead of some skiplist and some Berkeley DB). The only thing you need to look out for is that when users access their mailboxes for the first time after the upgrade, their indexes are updated. If you have several of these go off at once, it can send your load up quite high. Thus, if there is any way to stagger this, it can be helpful--especially if you have lots of mailboxes. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
TLS changes in 2.4.x
I just upgraded from 2.3.x to 2.4.13 using Simon Matter's rpms. The upgrade is going as expected from all of the comments on the list with one big exception. I'm wondering how TLS has changed from the 2.3 series. I have 2 different Postfix systems trying to connect (using LMTP) to one Cyrus-IMAP mailstore. Both Postfix systems were able to STARTTLS during LMTP to the Cyrus-IMAP mailstore before the upgrade. Only one (the first one that connects) is able to do so after the upgrade. I've actually set this up with virtual machines so I could test and rollback to see what was going on. I upgraded a test Cyrus-IMAP server, and, again, only the first Postfix server to connect could do so successfully. I've also verified the results using lmtptest which hangs with the errant server. The only thing I can imagine might be causing the problem is that I'm using the same wildcard certificate (3rd party signed) for each Postfix machine trying to connect to the Cyrus-IMAP mailstore, which also uses the same certificate (all in same domain). I notice that there is a note in the change logs regarding TLS session reuse. Could this TLS caching be the problem? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: cyrus-imap authorization confusion
On Sat, Mar 17, 2012 at 8:06 PM, Dan White dwh...@olp.net wrote: On 03/15/12 12:10 -0700, Stephen Ingram wrote: I see in the documents mention of the four types of authorization supported by Cyrus-IMAP. I also see a --with-auth compile option in older versions that no longer appear in newer versions. I understand that authentication is handled by Cyrus-SASL. Is authorization now also handled also by Cyrus-SASL with userid and authid being equal? I believe the compile time --with-auth option was replaced with the 'auth_mech' runtime (/etc/imapd.conf) option. Also see the 'unix_group_enable' option. Cyrus SASL will be used to resolve and canonicalize both the userid and authid, but it's left up to Cyrus IMAPD to: * figure out who belongs to what group (for group:staff type ACLs), via the auth_mech configuration * apply ACLs to determine what rights a user has to access another's mailbox * who can act *as* another user, via the 'proxyservers' and 'loginuseacl' config options. Thank you Dan, that's exactly what I was looking for. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
cyrus-imap authorization confusion
I see in the documents mention of the four types of authorization supported by Cyrus-IMAP. I also see a --with-auth compile option in older versions that no longer appear in newer versions. I understand that authentication is handled by Cyrus-SASL. Is authorization now also handled also by Cyrus-SASL with userid and authid being equal? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Newbie errors
On Mon, Dec 12, 2011 at 8:47 AM, Dominique Couot dco...@terra.es wrote: Hi, I've playing with Cyrus (+Postfix + SASL) for a while without any problem - and without any security (port143). I finally got around to get a certificate and installed it, modified the imap.conf file, and I can no longer receive any mail on port 993. Sending does work however on port 465. It worked for a while this morning until I screwed up. Can someone tell me what's where I might I have gone wrong... Promise, next time I'll take better notes of the changes I make The port 993 is forwarded to the server and the server seems to be listening to it (nmap results). However nothing in the mail.log as far as connection are concerned, only errors: Dec 12 17:06:23 www cyrus/imaps[19071]: imaps TLS negotiation failed: 16.Red-81-34-68.dynamicIP.rima-tde.net [81.34.68.16] Dec 12 17:06:23 www cyrus/imaps[19071]: Fatal error: tls_start_servertls() failed Dec 12 17:06:23 www cyrus/master[18687]: process 19071 exited, status 75 Dec 12 17:06:23 www cyrus/master[18687]: service imaps pid 19071 in BUSY state: terminated abnormally Any help more than welcome. Make sure that cyrus-imapd has access to your SSL certificate and key. It sounds like only your MTA has proper access. Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Newbie errors
On Mon, Dec 12, 2011 at 9:30 AM, Dominique Couot dco...@terra.es wrote: Steve, If by acces you mean the path is right, It does have access (see imapd.conf extract): # # SSL/TLS Options # # File containing the global certificate used for ALL services (imap, pop3, # lmtp, sieve) # tls_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem tls_cert_file: /etc/ssl/certs/server_mail_solipym_com.pem # File containing the private key belonging to the global server certificate. # tls_key_file: /etc/ssl/private/ssl-cert-snakeoil.key tls_key_file: /etc/ssl/private/server.key # File containing one or more Certificate Authority (CA) certificates. # tls_ca_file: /etc/ssl/certs/ca-certificates.crt tls_ca_file: /etc/ssl/CA/root.crt If you mean right to access, all files are read only except for root. I actually have a set just for cyrus-imap owned by the user that cyrus-imap runs as. Not sure if the cert_file should be pem or crt format though. PEM is fine. The weirdest thing, is that it worked till mid day, then nothing. Does the CA file have the necessary certificates to validate the cert on the connecting client? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Android, outgoing messages not saved on server
Paul- You typically have to tell the phone to use the Sent folder on the server rather than the local Sent folder. This is typically found in the configuration for the email account. As far as the INBOX.Sent, are you using altnamespace: yes in your imapd.conf? Steve On Fri, Aug 5, 2011 at 6:33 AM, Paul van der Vlis p...@vandervlis.nl wrote: Hello, I've a customer with an Samsung Galaxy S2 phone (Android 2.3), and he has a Cyrus IMAP 2.2.13 server. Connecting works fine, but e.g. outgoing messages are not saved in the Sent folder on the server, but local on the phone. Another less important problem is: he sees mailboxes like INBOX.Sent, he would prefer to see Sent. Is here somebody with Android experience? With regards, Paul van der Vlis. -- Linux systeembeheer Groningen http://www.vandervlis.nl Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Android, outgoing messages not saved on server
On Fri, Aug 5, 2011 at 7:05 AM, Paul van der Vlis p...@vandervlis.nl wrote Op 05-08-11 16:01, Stephen Ingram schreef: Paul- You typically have to tell the phone to use the Sent folder on the server rather than the local Sent folder. This is typically found in the configuration for the email account. My customer did not found this setting. He found an setting for a path, we've tried INBOX and INBOX.. As far as the INBOX.Sent, are you using altnamespace: yes in your imapd.conf? No, I am using the default on this server. You have to use altnamespace: yes to get rid of the INBOX.Sent. Many people are writing me to use the K-9 client. Yes, I thought that's what you were using. You can adjust the sent, drafts, deleted, etc. folders with the later versions of this (https://github.com/k9mail/k-9/wiki/FrequentlyAskedQuestions). Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
change virtual domain setup to standard
Is it possible to change a cyrus-imap installation that uses virtual domains to one that does not? If so, I'm guessing you would have to move the accounts (e.g. n...@domain.com to name). Should you move the accounts first and then remove the virtualdomains: on entry in imapd.conf or the other way around? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
login uid doesn't match mailbox name
Is it possible to login to cyrus-imap with a uid that doesn't match the mailbox name? I'm trying to use the virtual domains setup where mailbox names would be full email addresses, however, some of the users need to use kerberos for login so they would have uid of say u...@realm.com where realm.com would not necessarily match their email domain. I see that there is /etc/krb.equiv file that can be used to equate kerberos uid to local unix user. Would this also work for mailbox name? Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: login uid doesn't match mailbox name
Dan- Thanks. I'm already setting up the user canonicalization plugin using 2.1.24rc1. When I saw the krb.equiv today, I thought maybe I had found an easier solution. Steve On Thu, Mar 3, 2011 at 7:28 PM, Dan White dwh...@olp.net wrote: On 03/03/11 15:28 -0800, Stephen Ingram wrote: Is it possible to login to cyrus-imap with a uid that doesn't match the mailbox name? I'm trying to use the virtual domains setup where mailbox names would be full email addresses, however, some of the users need to use kerberos for login so they would have uid of say u...@realm.com where realm.com would not necessarily match their email domain. I see that there is /etc/krb.equiv file that can be used to equate kerberos uid to local unix user. Would this also work for mailbox name? I don't know. A quick look through the source would lead me to believe the /etc/krb.equiv file doesn't work with kerberos5/gssapi, but was written for kerberos_v4. Another way to accomplish username mapping is with a libsasl user canonicalization plugin, which would allow you to arbitrarily match usernames to mailboxes. There are two such plugins: ldapdb (in 2.1.24rc1) and sql (cyrus bugzilla bug 3219). -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
replication catch up after stop
I'm using replication with cyrus-imap on CentOS 5.4 rpm version 2.3.7. It works very well, but occasionally stops (many times my fault). When restarting everything, there are log files left in the /var/lib/imap/sync directory with the name log-# where # is most likely some process number. Many times if I just leave these alone, they disappear after a period of time. I'm assuming that they have processed and that everything is in sync and have verified this to be the case. Some times, however, they just remain in the directory. I've looked at the documentation, but it doesn't seem to go into any depth on this subject. Can someone explain how the log files created in this directory work and if there is a way to manually process these files? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
move default domain mailboxes
After setting up murder, is it possible to move mailboxes that use the default domain from one backend server to another? The new back-end server has a different default domain. After which the default domain on the original backend server will be changed. before: server 1 default domain: domainA.tld mailbox for user st...@domaina.tld is user.steve server 2 default domain: domainB.tld next: server 1 default domain: domainA.tld move mailbox for user st...@domaina.tld to server 2 server 2 default domain: domainB.tld mailbox becomes user.st...@domaina.tld finally: server 1 default domain: domainC.tld server 2 default domain: domainB.tld mailbox user st...@domaina.tld is still user.st...@domaina.tld Steve Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
duplicate default domain entries on murder backends
I'm wondering if it's possible to use the same default domain entries on various murder backends. I'm using a virtual domain setup, and, as I'm using the full email address for all domain users, I'm guessing that the default domain can be an unused or a fake domain. If so, can this fake domain be used across the various murder backends? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
duplicate default domain entries on murder backends
I'm wondering if it's possible to use the same default domain entries on various murder backends. I'm using a virtual domain setup, and, as I'm using the full email address for all domain users, I'm guessing that the default domain can be an unused or a fake domain. If so, can this fake domain be used across the various murder backends? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/