Re: ETOOMANYREFS related errors
On 11 Mar 2016, at 03:48, Luis Ugalde wrote: > > Hi, > > I'm starting to see, on a pretty standard Debian Jessie installation, some > error messages that are apparently related to the ETOOMANYREFS errno. > > Firstly, the mail log shows this: > dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many references: > cannot splice Apparently because Linux thinks the same fd has been passed around recursively too many times: http://lkml.iu.edu/hypermail/linux/kernel/1101.0/01917.html But Dovecot doesn't pass it recursively. It's only passed once from pop3-login to pop3 process. > Is this something that Dovecot should be able to handle, or is it strictly > Debian/libc/MySillyMistake related? > > #uname -a > Linux server 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 > (2016-01-17) x86_64 GNU/Linux I wonder if there's a new kernel change that started detecting the recursion wrong.
Re: sis deduplication broken from 2.2.16 upwards
> On 11 Mar 2016, at 02:37, Charles Marcus wrote: > > On 3/9/2016 9:02 PM, Timo Sirainen wrote: >> On 08 Mar 2016, at 01:50, Pavel Stano wrote: >>> >>> sis attachment deduplication is broken in 2.2.16 upwards. >>> It is caused by this commit. >>> https://github.com/dovecot/core/commit/664bf3e236c214aee86294483c379e4fa66c2e63 >>> >>> in src/lib-fs/fs-sis.c function fs_sis_try_link() is comparation of >>> inodes of hash files. >>> Because fs_stat() after that commit use fstat() on open fd of temporary >>> file instead of stat on filename. But that temporary file has differnt >>> inode. >>> >>> It not cause any corruption but it will not save any space. >>> Because every duplicate attachment will be in separate file. >> Thanks, fixed: >> https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c841584ea1 > > So, after the fix is applied, does dovecot silently delete the > duplicated files, or is there a command that needs to be run manually? You'd have to do it manually in some way. A script that does something like: Go through all attachment directories and for each file: - Sort files by filename - Identify that files A and B the same (beginning of the filename begins with same hash), but have a different inode - ln A B.tmp && mv B.tmp B
ETOOMANYREFS related errors
Hi, I'm starting to see, on a pretty standard Debian Jessie installation, some error messages that are apparently related to the ETOOMANYREFS errno. Firstly, the mail log shows this: dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many references: cannot splice And then the login process fails: dovecot: pop3-login: Internal login failure (pid=34388 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=rip, lip=lip, session= Is this something that Dovecot should be able to handle, or is it strictly Debian/libc/MySillyMistake related? #uname -a Linux server 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux #dovecot --version 2.2.13 #dpkg -l | grep -E 'dovecot-core|libc6|linux-image' ii dovecot-core 1:2.2.13-12~deb8u1amd64 ii libc6:amd642.19-18+deb8u3amd64 ii libc6-i386 2.19-18+deb8u3amd64 ii linux-image-3.16.0-4-amd64 3.16.7-ckt20-1+deb8u3 amd64 Best Regards, Luis
Re: Troubleshooting mailbox problems
On 3/10/2016 3:50 PM, Andrew McGlashan wrote: If they are using POP to download messages from any client, make sure the client does a leave on server for at least long enough for other client devices to download the messages. Cheers A. Yes, that's the whole problem. They SAY that it's set for leave on server, but it's not happening that way. Essentially, observed behavior does not match the values of settings in the mail clients. -- --- - Nick Bright- - Vice President of Technology - - Valnet -=- We Connect You -=- - - Tel 888-332-1616 x 315 / Fax 620-331-0789 - - Web http://www.valnet.net/ - --- - Are your files safe?- - Valnet Vault - Secure Cloud Backup - - More information & 30 day free trial at - - http://www.valnet.net/services/valnet-vault - --- This email message and any attachments are intended solely for the use of the addressees hereof. This message and any attachments may contain information that is confidential, privileged and exempt from disclosure under applicable law. If you are not the intended recipient of this message, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply E-mail and immediately delete this message from your system.
Re: Troubleshooting mailbox problems
If they are using POP to download messages from any client, make sure the client does a leave on server for at least long enough for other client devices to download the messages. Cheers A.
Re: Troubleshooting mailbox problems
On 3/10/2016 12:54 PM, Nick Bright wrote: Thanks for your reply Gordon, I've added the mail_log to my mail_plugins list in 10-mail.conf I'm seeing quite a bit of activity in the debug log file, but I'm not exactly sure how to interpret it all - specifically what I should look for when a POP3 or IMAP client is sending delete commands. Is there a guide on the wiki that I'm not seeing for how to interpret the logs to see this kind of activity? I must have done something wrong, as adding the mail_log to the mail_plugins list made all authentication timeout. -- --- - Nick Bright- - Vice President of Technology - - Valnet -=- We Connect You -=- - - Tel 888-332-1616 x 315 / Fax 620-331-0789 - - Web http://www.valnet.net/ - --- - Are your files safe?- - Valnet Vault - Secure Cloud Backup - - More information & 30 day free trial at - - http://www.valnet.net/services/valnet-vault - --- This email message and any attachments are intended solely for the use of the addressees hereof. This message and any attachments may contain information that is confidential, privileged and exempt from disclosure under applicable law. If you are not the intended recipient of this message, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply E-mail and immediately delete this message from your system.
Re: Troubleshooting mailbox problems
Thanks for your reply Gordon, I've added the mail_log to my mail_plugins list in 10-mail.conf I'm seeing quite a bit of activity in the debug log file, but I'm not exactly sure how to interpret it all - specifically what I should look for when a POP3 or IMAP client is sending delete commands. Is there a guide on the wiki that I'm not seeing for how to interpret the logs to see this kind of activity? -- --- - Nick Bright- - Vice President of Technology - - Valnet -=- We Connect You -=- - - Tel 888-332-1616 x 315 / Fax 620-331-0789 - - Web http://www.valnet.net/ - --- - Are your files safe?- - Valnet Vault - Secure Cloud Backup - - More information & 30 day free trial at - - http://www.valnet.net/services/valnet-vault - --- This email message and any attachments are intended solely for the use of the addressees hereof. This message and any attachments may contain information that is confidential, privileged and exempt from disclosure under applicable law. If you are not the intended recipient of this message, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply E-mail and immediately delete this message from your system.
Re: Troubleshooting mailbox problems
On 03/10/2016 04:55 PM, Nick Bright wrote: Greetings, I'm running Dovecot 2.0.9 on my CentOS 6 server, for several thousand mailboxes. Recently, I've had several reports of "my mailbox is suddenly empty, where'd my mail go?" I've enabled debug logging, but I'd like to make sure I have the best level of debug to see things like "delete message" commands? I've configured in logging: auth_debug = yes auth_debug_passwords = no mail_debug = yes mail_plugins= [...] mail_log [..] plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_group_events = yes mail_log_fields = uid box msgid size } Does this look like a good start? I really think it's a client side problem, but sometimes end users always want to blame the server; and proof must be shown e.g. "Look, here's the log, you're sending delete commands." These parameters should be sufficient to reach your goal. Best regards, Gordon
Troubleshooting mailbox problems
Greetings, I'm running Dovecot 2.0.9 on my CentOS 6 server, for several thousand mailboxes. Recently, I've had several reports of "my mailbox is suddenly empty, where'd my mail go?" I've enabled debug logging, but I'd like to make sure I have the best level of debug to see things like "delete message" commands? I've configured in logging: auth_debug = yes auth_debug_passwords = no mail_debug = yes plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_group_events = yes mail_log_fields = uid box msgid size } Does this look like a good start? I really think it's a client side problem, but sometimes end users always want to blame the server; and proof must be shown e.g. "Look, here's the log, you're sending delete commands." At which point, we can proceed with dealing with their software problem. Thanks, -- --- - Nick Bright- - Vice President of Technology - - Valnet -=- We Connect You -=- - - Tel 888-332-1616 x 315 / Fax 620-331-0789 - - Web http://www.valnet.net/ - --- - Are your files safe?- - Valnet Vault - Secure Cloud Backup - - More information & 30 day free trial at - - http://www.valnet.net/services/valnet-vault - --- This email message and any attachments are intended solely for the use of the addressees hereof. This message and any attachments may contain information that is confidential, privileged and exempt from disclosure under applicable law. If you are not the intended recipient of this message, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply E-mail and immediately delete this message from your system.
Re: Timout for LDAP connection
Hi Timo, On 01.03.2016 22:51, Timo Sirainen wrote: On 29 Feb 2016, at 17:18, Gordon Grubert wrote: Hi, we are using a round robin dns record for connections to our ldap system. This works fine for almost all cases. In particular, for dovecot does this mean, when an ldap server is stopped, dovecot instantly reconnects to another ldap server. But when the network connection to the active ldap server is broken, dovecot sticks to the failed ldap server. Is there any possibility to define a connection timeout? What should happen is that as long as new requests keep coming, Dovecot realizes after about 60 seconds that the LDAP server is hanging. It then reconnects and the reconnection should work. But... First of all, 60 seconds is likely a much too long timeout. But more importantly it looks like there's something weird now going on with OpenLDAP library. I added this somewhat recently and tested that it works: https://github.com/dovecot/core/commit/fb3178a1924dae52151d88c4d4ded879df43dd3f But now that I'm testing it, the timeout doesn't seem to be triggering. I don't know what happened to it that it suddenly doesn't work.. This also means that OpenLDAP seems to be internally stuck trying to connect to a server that isn't responding. Dovecot doesn't currently make the decisions on which LDAP server to connect to. It just passes through all the hosts to OpenLDAP library and lets it handle it. And it seems like OpenLDAP library can't right now do this failover. So maybe Dovecot should be responsible for that as well.. Anyway, for now you could set up haproxy to localhost and configure Dovecot LDAP to connect to haproxy and haproxy connect to the actual LDAP servers. today I've upgraded to 2.2.21-1~auto+171 on debian 8 and made a lot of "interruption tests". Your fix not really solved the problem. But I found another interesting fact: The openldap client on debian 8 can handle hard communication interrupts correctly. I've added NETWORK_TIMEOUT 5 TIMEOUT 5 to ldap.conf because man 5 ldap.conf says: NETWORK_TIMEOUT Specifies the timeout (in seconds) after which the poll(2)/select(2) following a connect(2) returns in case of no activity. TIMEOUT Specifies a timeout (in seconds) after which calls to synchronous LDAP APIs will abort if no response is received. Also used for any ldap_result(3) calls where a NULL timeout parameter is supplied. We are using the ISC DHCP server with dynamic ldap connections. This daemon uses - like dovecot - the LDAP API of the openldap client for access to the ldap server. The DHCP opens a persistent ldap connection to handle all dhcp requests (same behavior like dovecot). Here, the timeouts for connection loss are working. Therefore, my question: Why does this not work for dovecot, too, when dovecot uses the same API? Dovecot does not get a response from the LDAP server and has to reconnect, only. IMAP server world domination requires a reconnect in case of connection timeouts ;-) Best regards, Gordon -- Technischer Leiter & stellv. Direktor Universitätsrechenzentrum (URZ) E.-M.-Arndt-Universität Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86 1456 Fax. +49 3834 86 1401 smime.p7s Description: S/MIME Cryptographic Signature
Re: zlib plugin doesn't add "Z" flag to Maildir filename
On 2016-03-10 16:06, Alessio Cecchi wrote: Il 10/03/2016 15:33, Tom Sommer ha scritto: On 2016-03-10 15:00, Alessio Cecchi wrote: Il 10/03/2016 10:38, Tom Sommer ha scritto: I enabled zlib compression, so new mails are saved compressed. Now I want to convert the old mails to gzip with the find and gzip command, but now I don't know which files are already gzip compressed by dovecot. Simple check if the file is already compress (es. if gzip -t "${MAILFILE}" 2> /dev/null". Thanks. I worked around it with `file`. I suggest to use "gzip -t" for test if an email is already compress, with file I found some unrecognized compress email, a real example: # file 1424615175.M471013P18835.pop04.domain.com\,S\=3982\,W\=4097\:2\,S 1424615175.M471013P18835.pop04.domain.com,S=3982,W=4097:2,S: Minix filesystem, V3, 6762 zones # gzip -t 1424615175.M471013P18835.pop04.domain.com\,S\=3982\,W\=4097\:2\,S # echo $? 0 Duly noted, thanks :)
Re: zlib plugin doesn't add "Z" flag to Maildir filename
Il 10/03/2016 15:33, Tom Sommer ha scritto: On 2016-03-10 15:00, Alessio Cecchi wrote: Il 10/03/2016 10:38, Tom Sommer ha scritto: I enabled zlib compression, so new mails are saved compressed. Now I want to convert the old mails to gzip with the find and gzip command, but now I don't know which files are already gzip compressed by dovecot. Simple check if the file is already compress (es. if gzip -t "${MAILFILE}" 2> /dev/null". Thanks. I worked around it with `file`. I suggest to use "gzip -t" for test if an email is already compress, with file I found some unrecognized compress email, a real example: # file 1424615175.M471013P18835.pop04.domain.com\,S\=3982\,W\=4097\:2\,S 1424615175.M471013P18835.pop04.domain.com,S=3982,W=4097:2,S: Minix filesystem, V3, 6762 zones # gzip -t 1424615175.M471013P18835.pop04.domain.com\,S\=3982\,W\=4097\:2\,S # echo $? 0 -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
Re: v2.2.22 release candidate released
On 2016-03-04 16:49, Timo Sirainen wrote: On 04 Mar 2016, at 17:40, Miquel van Smoorenburg wrote: Question: some time ago you mentioned that you were going to work on caldav/carddav support. What is the status of that, and will the calendar/contacts database be available over JMAP as well? Initially JMAP will be email-only. The CalDAV/CardDAV plans have been postponed for now. Damn. I'd kill for a Dovecot-integrated/quality CalDAV-server.
Re: zlib plugin doesn't add "Z" flag to Maildir filename
On 2016-03-10 15:00, Alessio Cecchi wrote: Il 10/03/2016 10:38, Tom Sommer ha scritto: I enabled zlib compression, so new mails are saved compressed. Now I want to convert the old mails to gzip with the find and gzip command, but now I don't know which files are already gzip compressed by dovecot. Simple check if the file is already compress (es. if gzip -t "${MAILFILE}" 2> /dev/null". Thanks. I worked around it with `file`. Shouldn't zlib_save itself add Z to the Maildir flags? As suggested in the compress guide on http://wiki2.dovecot.org/Plugins/Zlib ? No, zlib plugins only compress email without chaning the name. I know, I'm suggesting it _should_ change the Maildir flag. I suggest to do not use script that rename email files because they can invalid the uidlist. I don't think adding a Maildir flag does that. If it does, then the wiki should be changed. // Tom
Re: zlib plugin doesn't add "Z" flag to Maildir filename
Il 10/03/2016 10:38, Tom Sommer ha scritto: I enabled zlib compression, so new mails are saved compressed. Now I want to convert the old mails to gzip with the find and gzip command, but now I don't know which files are already gzip compressed by dovecot. Simple check if the file is already compress (es. if gzip -t "${MAILFILE}" 2> /dev/null". Shouldn't zlib_save itself add Z to the Maildir flags? As suggested in the compress guide on http://wiki2.dovecot.org/Plugins/Zlib ? No, zlib plugins only compress email without chaning the name. I suggest to do not use script that rename email files because they can invalid the uidlist. -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
Re: sis deduplication broken from 2.2.16 upwards
On 3/9/2016 9:02 PM, Timo Sirainen wrote: > On 08 Mar 2016, at 01:50, Pavel Stano wrote: >> >> sis attachment deduplication is broken in 2.2.16 upwards. >> It is caused by this commit. >> https://github.com/dovecot/core/commit/664bf3e236c214aee86294483c379e4fa66c2e63 >> >> in src/lib-fs/fs-sis.c function fs_sis_try_link() is comparation of >> inodes of hash files. >> Because fs_stat() after that commit use fstat() on open fd of temporary >> file instead of stat on filename. But that temporary file has differnt >> inode. >> >> It not cause any corruption but it will not save any space. >> Because every duplicate attachment will be in separate file. > Thanks, fixed: > https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c841584ea1 So, after the fix is applied, does dovecot silently delete the duplicated files, or is there a command that needs to be run manually?
Re: Client-initiated secure renegotiation
On 10.03.2016 12:40, Osiris wrote: That's just the question of Florent: how to disable Secure Client-Initiated Renegotiation. Hi! There is no way to disable this in OpenSSL, and the CVE you refer to has been disputed. Please see http://cve.mitre.org/cgi-bin/cvename.cgi?name=cve-2011-1473 and https://www.openssl.org/docs/manmaster/ssl/SSL_CTX_set_options.html. Without altering OpenSSL sources, secure renegotiations will take place. --- Aki Tuomi Dovecot Oy
Re: Client-initiated secure renegotiation
On 10-03-16 11:21, Andrey Fesenko wrote: > On Thu, Mar 10, 2016 at 12:30 PM, Osiris wrote: >> On 09-03-16 13:14, djk wrote: >>> On 09/03/16 10:44, Florent B wrote: Hi, I don't see any SSL configuration option in Dovecot to disable "Client-initiated secure renegotiation". It is advised to disable it as it can cause DDoS (CVE-2011-1473). Is it possible to have this possibility through an SSL option or other ? Thank you. Florent >>> ssl_protocols = !SSLv3 !SSLv2 >>> >>> Is that enough? >> I'm afraid not. I've got SSLv2 and SSLv3 disabled and with `openssl >> s_client -connect $host:993` I still can successfully renegotiate by >> passing a single 'R'. > Are you use good ssl_cipher_list > (https://wiki.mozilla.org/Security/Server_Side_TLS)? > > My config > ## Service options > # 10-ssl > ssl = yes > ssl_cert = ssl_key = ssl_require_crl = no > ssl_ca = ssl_cipher_list = > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > ssl_parameters_regenerate = 72h > # The !TLSv1 are OK, without TLS not work imtest (cyrus test suit) > ssl_protocols = !SSLv2 !SSLv3 > # Prefer the server's order of ciphers over client's > # Only available on dovecot 2.2.6 and later:: > ssl_prefer_server_ciphers = yes > # Only available on dovecot 2.2.7 and later:: > ssl_dh_parameters_length = 2048 > > Work fine, but only testssl.sh scanner generate small warning "Secure > Client-Initiated Renegotiation VULNERABLE (NOT ok), DoS threat" > > openssl s_client -connect $host:993 -ssl2(3) and openssl s_client > -connect $host:143 -starttls imap -showcerts -state -crlf -ssl2(3) > break connection > That's just the question of Florent: how to disable Secure Client-Initiated Renegotiation.
Re: Client-initiated secure renegotiation
On Thu, Mar 10, 2016 at 12:30 PM, Osiris wrote: > On 09-03-16 13:14, djk wrote: >> On 09/03/16 10:44, Florent B wrote: >>> Hi, >>> >>> I don't see any SSL configuration option in Dovecot to disable >>> "Client-initiated secure renegotiation". >>> >>> It is advised to disable it as it can cause DDoS (CVE-2011-1473). >>> >>> Is it possible to have this possibility through an SSL option or other ? >>> >>> Thank you. >>> >>> Florent >> ssl_protocols = !SSLv3 !SSLv2 >> >> Is that enough? > > I'm afraid not. I've got SSLv2 and SSLv3 disabled and with `openssl > s_client -connect $host:993` I still can successfully renegotiate by > passing a single 'R'. Are you use good ssl_cipher_list (https://wiki.mozilla.org/Security/Server_Side_TLS)? My config ## Service options # 10-ssl ssl = yes ssl_cert =
zlib plugin doesn't add "Z" flag to Maildir filename
I enabled zlib compression, so new mails are saved compressed. Now I want to convert the old mails to gzip with the find and gzip command, but now I don't know which files are already gzip compressed by dovecot. Shouldn't zlib_save itself add Z to the Maildir flags? As suggested in the compress guide on http://wiki2.dovecot.org/Plugins/Zlib ? -- Tom
Re: Client-initiated secure renegotiation
On 09-03-16 13:14, djk wrote: > On 09/03/16 10:44, Florent B wrote: >> Hi, >> >> I don't see any SSL configuration option in Dovecot to disable >> "Client-initiated secure renegotiation". >> >> It is advised to disable it as it can cause DDoS (CVE-2011-1473). >> >> Is it possible to have this possibility through an SSL option or other ? >> >> Thank you. >> >> Florent > ssl_protocols = !SSLv3 !SSLv2 > > Is that enough? I'm afraid not. I've got SSLv2 and SSLv3 disabled and with `openssl s_client -connect $host:993` I still can successfully renegotiate by passing a single 'R'.
Re: Setting up public mailboxes - user not found
Am 09.03.2016 um 20:02 schrieb Marti Markov: Hi all, This is the first time I use the dovecot mail list so I’m sorry if I forget something. My problem is that for some reason I can get public mailboxes to work. I have setup the directory Public, the folders and the cur,tmp and new folder in them: Public | |- .office3 | |- cur |- tmp |- new |- dovecot-acl Here is my dovecot conf: namespace { list = yes location = maildir:/home/vmail/xxx.com/Public:INDEXPVT=~/Maildir/Public prefix = Public/ separator = / subscriptions = yes type = public } You can use a "normal" user who has a sieve script: require ... any filter if necessary fileinto "Public"; discard; ... "Public" is your prefix I have several subfolder so I use: fileinto "Public/subfolder1"; You would have /home/vmail/xxx.com/Public/subfolder1 with cur new tmp and a file "dovecot-acl" containing anyone lrs -- Burckhard Schmidt