Re: Logging the TLS cipher suite
Thank you very much!
Gedalya wrote on 12/03/2016 06:30:
Add %k to login_log_format_elements (in conf.d/10-logging.conf)
for example
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k
session=<%{session}>
On 03/12/2016 12:20 AM, Luigi Rosa wrote:
Hi,
could it be possible to log the TLS cipher suite as Postfix does?
This is a typical TLS Dovecot log line:
imap-login: Login: user=, method=DIGEST-MD5, rip=1.2.3.4,
lip=4.3.2.1, mpid=19671, TLS, session=
This is the Postfix equivalent
postfix/smtp[59723]: Anonymous TLS connection established to
mail.acmne.com[1.2.3.4]:25: TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)
--
Ciao,
luigi
/
+--[Luigi Rosa]--
\
Natalya Simonova: You destroy every vehicle you get into?
James Bond: Standard operating procedure. Boys with toys.
--"007 Goldeneye"
Re: Logging the TLS cipher suite
Forgot the important part, sorry
http://wiki.dovecot.org/Variables
On 03/12/2016 12:30 AM, Gedalya wrote:
Add %k to login_log_format_elements (in conf.d/10-logging.conf)
for example
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e
%c %k session=<%{session}>
On 03/12/2016 12:20 AM, Luigi Rosa wrote:
Hi,
could it be possible to log the TLS cipher suite as Postfix does?
This is a typical TLS Dovecot log line:
imap-login: Login: user=, method=DIGEST-MD5,
rip=1.2.3.4, lip=4.3.2.1, mpid=19671, TLS, session=
This is the Postfix equivalent
postfix/smtp[59723]: Anonymous TLS connection established to
mail.acmne.com[1.2.3.4]:25: TLSv1.2 with cipher AECDH-AES256-SHA
(256/256 bits)
Re: Logging the TLS cipher suite
Add %k to login_log_format_elements (in conf.d/10-logging.conf)
for example
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
%k session=<%{session}>
On 03/12/2016 12:20 AM, Luigi Rosa wrote:
Hi,
could it be possible to log the TLS cipher suite as Postfix does?
This is a typical TLS Dovecot log line:
imap-login: Login: user=, method=DIGEST-MD5,
rip=1.2.3.4, lip=4.3.2.1, mpid=19671, TLS, session=
This is the Postfix equivalent
postfix/smtp[59723]: Anonymous TLS connection established to
mail.acmne.com[1.2.3.4]:25: TLSv1.2 with cipher AECDH-AES256-SHA
(256/256 bits)
Logging the TLS cipher suite
Hi, could it be possible to log the TLS cipher suite as Postfix does? This is a typical TLS Dovecot log line: imap-login: Login: user=, method=DIGEST-MD5, rip=1.2.3.4, lip=4.3.2.1, mpid=19671, TLS, session= This is the Postfix equivalent postfix/smtp[59723]: Anonymous TLS connection established to mail.acmne.com[1.2.3.4]:25: TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits) -- Ciao, luigi / +--[Luigi Rosa]-- \ We're not surrounded, we're in a target-rich environment!
Re: sis deduplication broken from 2.2.16 upwards
On 3/10/2016 7:56 PM, Timo Sirainen wrote: >> On 11 Mar 2016, at 02:37, Charles Marcus wrote: >> >> On 3/9/2016 9:02 PM, Timo Sirainen wrote: >>> On 08 Mar 2016, at 01:50, Pavel Stano wrote: sis attachment deduplication is broken in 2.2.16 upwards. It is caused by this commit. https://github.com/dovecot/core/commit/664bf3e236c214aee86294483c379e4fa66c2e63 in src/lib-fs/fs-sis.c function fs_sis_try_link() is comparation of inodes of hash files. Because fs_stat() after that commit use fstat() on open fd of temporary file instead of stat on filename. But that temporary file has differnt inode. It not cause any corruption but it will not save any space. Because every duplicate attachment will be in separate file. >>> Thanks, fixed: >>> https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c841584ea1 >> So, after the fix is applied, does dovecot silently delete the >> duplicated files, or is there a command that needs to be run manually? > You'd have to do it manually in some way. A script that does something like: > > Go through all attachment directories and for each file: > - Sort files by filename > - Identify that files A and B the same (beginning of the filename begins > with same hash), but have a different inode > - ln A B.tmp && mv B.tmp B Ugh... ok thanks, but it seems like that would be much safer as a doveadm command...
Re: Timout for LDAP connection
On 11 Mar 2016, at 04:15, Gordon Grubert wrote: > > But I found another interesting fact: The openldap client on debian 8 > can handle hard communication interrupts correctly. I've added > > NETWORK_TIMEOUT 5 > TIMEOUT 5 > > to ldap.conf because man 5 ldap.conf says: > > NETWORK_TIMEOUT > Specifies the timeout (in seconds) after which the poll(2)/select(2) > following a connect(2) returns in case of no activity. > > TIMEOUT > Specifies a timeout (in seconds) after which calls to > synchronous LDAP APIs will abort if no response is received. Also > used for any ldap_result(3) calls where a NULL timeout parameter is > supplied. Dovecot doesn't use any synchronous openldap calls, so according to these manual pages the above settings are ignored by openldap library with Dovecot. > We are using the ISC DHCP server with dynamic ldap connections. This > daemon uses - like dovecot - the LDAP API of the openldap client for > access to the ldap server. The DHCP opens a persistent ldap connection > to handle all dhcp requests (same behavior like dovecot). Here, the > timeouts for connection loss are working. > > Therefore, my question: Why does this not work for dovecot, too, when > dovecot uses the same API? Dovecot does not get a response from the > LDAP server and has to reconnect, only. I bet ISC DHCP uses synchronous openldap calls. Dovecot can't also do the timeout handling internally, because it can only abort the entire openldap connect call. Dovecot can't tell openldap to connect to the next server. The only solution I can think of is that Dovecot doesn't let openldap do the multi-server connection handling, but instead have Dovecot create a separate openldap instance for each server and manage the connections + timeouts internally. But that's a lot of work.. Actually, a workaround might be to do synchronous binding. I'd rather not change Dovecot to do this by default, because it hangs the entire auth process while it's binding. But SASL authentication has no async API in openldap, so by enabling it you'll get synchronous authentication which will probably apply openldap's internally configured timeouts. Maybe you can get it working with: sasl_bind = yes sasl_mech = PLAIN
Re: Timout for LDAP connection
On 03/11/2016 03:30 PM, Gordon Grubert wrote: Of course, such a WORKAROUND could be used and I'm sure that this works. But Timo says, dovecot is using the LDAP API. The openldap client can handle network timeouts. Therefore, dovecot has to be able to use these timeouts, too, like described in ldap.conf(5). Sure sure, absolutely agreed.
Re: Looking for way to monitor dsync, confirm it is or isn't running
Rich Wales wrote: > I am syncing two Dovecot sites using the dsync function. If you are referring to replication ... > I would like to be able to run some sort of periodic health check to > confirm that dsync is (or is not) running properly between the two > sites, and alert me if dsync is failing or lagging excessively. > > Does anyone know of a tool to do this? No replication running: | mail> doveadm replicator status | Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: No such file or directory Replication running: | mail> doveadm replicator status | Queued 'sync' requests0 | Queued 'high' requests0 | Queued 'low' requests 0 | Queued 'failed' requests 0 | Queued 'full resync' requests 0 | Waiting 'failed' requests 0 If those numbers tend to become significantly larger than 0, then replication has issues. I do not use that for health checking by something like ... > (If possible, something I can set up to run periodically in Nagios?) … but used it once in a while when suspecting issues with replication. HTH, Michael
Looking for way to monitor dsync, confirm it is or isn't running
I am syncing two Dovecot sites using the dsync function. I would like to be able to run some sort of periodic health check to confirm that dsync is (or is not) running properly between the two sites, and alert me if dsync is failing or lagging excessively. Does anyone know of a tool to do this? (If possible, something I can set up to run periodically in Nagios?) Thanks for any suggestions. Rich Wales ri...@richw.org
Re: NetApp NFS vs. ZFS and NFS for Maildir
On Fri, 11 Mar 2016 11:58:00 -0300 Juan Bernhard wrote: > > El 11/03/2016 a las 11:22 a.m., Alessio Cecchi escribió: > > Hi, > > > > I'm evaluating to switch from NetApp to a ZFS appliance (like Qsan). Our > > setup is Dovecot, Maildir for email storage and NFS to share mailboxes > > (more than 30k users) across POP/IMAP and MX servers. > > > > NetApp NFS works fine also under high load but have some limitation for > > inode numbers per Volume and is expensive (but recently their prices > > have dropped). > > > > ZFS, I read, suggest to create many small Raid Group to increase IOPS, > > but this configuration (N Raid instead of one RAID-DP like NetApp) is > > more complex to manage, or not? > > > > Someone has experiences with ZFS and NFS(v3) in high load environments? > > > > Thanks > > Be careful to no do any synchronous writes under ZFS. Every sync write > can take up to 3 seconds of latency (under freebsd, I didnt test ZFS in > linux). Im using it in a 3k user environment and works great with a 4TB > raid 10, and dovecot cache files in a SSD disk. > > Saludos, Juan. zfs set sync=disabled ? -- Regards, Stephan
Re: NetApp NFS vs. ZFS and NFS for Maildir
El 11/03/2016 a las 12:04 p.m., Charles Marcus escribió: On 3/11/2016 9:58 AM, Juan Bernhard wrote: Be careful to no do any synchronous writes under ZFS. Every sync write can take up to 3 seconds of latency (under freebsd, I didnt test ZFS in linux). Im using it in a 3k user environment and works great with a 4TB raid 10, and dovecot cache files in a SSD disk. From what I've heard you should not use hardware based RAID (ie, RAID10) setups with ZFS, you should let ZFS handle it. Maybe that is the source of your latency issues? There are actually two ZFS raid1 but in the same zpool, and the SSD in local, not in the ZFS dtorage. The latency was because I tried to use a ZFS for datasore of a vmware undres NFS, and NFS under vmware is always synchronous. I improved 10 times the performance under iSCSI
Re: NetApp NFS vs. ZFS and NFS for Maildir
On 3/11/2016 9:58 AM, Juan Bernhard wrote: > Be careful to no do any synchronous writes under ZFS. Every sync write > can take up to 3 seconds of latency (under freebsd, I didnt test ZFS in > linux). Im using it in a 3k user environment and works great with a 4TB > raid 10, and dovecot cache files in a SSD disk. >From what I've heard you should not use hardware based RAID (ie, RAID10) setups with ZFS, you should let ZFS handle it. Maybe that is the source of your latency issues?
Re: NetApp NFS vs. ZFS and NFS for Maildir
El 11/03/2016 a las 11:22 a.m., Alessio Cecchi escribió: Hi, I'm evaluating to switch from NetApp to a ZFS appliance (like Qsan). Our setup is Dovecot, Maildir for email storage and NFS to share mailboxes (more than 30k users) across POP/IMAP and MX servers. NetApp NFS works fine also under high load but have some limitation for inode numbers per Volume and is expensive (but recently their prices have dropped). ZFS, I read, suggest to create many small Raid Group to increase IOPS, but this configuration (N Raid instead of one RAID-DP like NetApp) is more complex to manage, or not? Someone has experiences with ZFS and NFS(v3) in high load environments? Thanks Be careful to no do any synchronous writes under ZFS. Every sync write can take up to 3 seconds of latency (under freebsd, I didnt test ZFS in linux). Im using it in a 3k user environment and works great with a 4TB raid 10, and dovecot cache files in a SSD disk. Saludos, Juan.
Re: Timout for LDAP connection
On 11.03.2016 10:10, mj wrote: Hi, We're now running with ldap via haproxy, as was suggested in this thread by Timo. So far, so good: it seems to work very well. Of course, such a WORKAROUND could be used and I'm sure that this works. But Timo says, dovecot is using the LDAP API. The openldap client can handle network timeouts. Therefore, dovecot has to be able to use these timeouts, too, like described in ldap.conf(5). Best regards, Gordon
NetApp NFS vs. ZFS and NFS for Maildir
Hi, I'm evaluating to switch from NetApp to a ZFS appliance (like Qsan). Our setup is Dovecot, Maildir for email storage and NFS to share mailboxes (more than 30k users) across POP/IMAP and MX servers. NetApp NFS works fine also under high load but have some limitation for inode numbers per Volume and is expensive (but recently their prices have dropped). ZFS, I read, suggest to create many small Raid Group to increase IOPS, but this configuration (N Raid instead of one RAID-DP like NetApp) is more complex to manage, or not? Someone has experiences with ZFS and NFS(v3) in high load environments? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
ldap userdb/passwd connection timeout
Hello. I want to use second ldap server for failover. So, i add a second section for userdb/passdb. My question is: what connection timeout, after that dovecot will be try to connectd another ldap server? Can i change it? Thanks.
Re: sis deduplication broken from 2.2.16 upwards
Am 11.03.2016 um 01:56 schrieb Timo Sirainen: On 11 Mar 2016, at 02:37, Charles Marcus wrote: On 3/9/2016 9:02 PM, Timo Sirainen wrote: On 08 Mar 2016, at 01:50, Pavel Stano wrote: sis attachment deduplication is broken in 2.2.16 upwards. It is caused by this commit. https://github.com/dovecot/core/commit/664bf3e236c214aee86294483c379e4fa66c2e63 in src/lib-fs/fs-sis.c function fs_sis_try_link() is comparation of inodes of hash files. Because fs_stat() after that commit use fstat() on open fd of temporary file instead of stat on filename. But that temporary file has differnt inode. It not cause any corruption but it will not save any space. Because every duplicate attachment will be in separate file. Thanks, fixed: https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c841584ea1 So, after the fix is applied, does dovecot silently delete the duplicated files, or is there a command that needs to be run manually? You'd have to do it manually in some way. A script that does something like: Go through all attachment directories and for each file: - Sort files by filename - Identify that files A and B the same (beginning of the filename begins with same hash), but have a different inode - ln A B.tmp && mv B.tmp B This functionality is how it works in sis-queue correct? Wouldn't it be nice to adopted doveadm sis deduplicate to handle this? regards -- Harald Leithner ITronic Wiedner Hauptstraße 120/5.1, 1050 Wien, Austria Tel: +43-1-545 0 604 Mobil: +43-699-123 78 4 78 Mail: leith...@itronic.at | itronic.at
Re: Timout for LDAP connection
Hi, We're now running with ldap via haproxy, as was suggested in this thread by Timo. So far, so good: it seems to work very well. MJ On 03/10/2016 04:15 PM, Gordon Grubert wrote: Hi Timo, On 01.03.2016 22:51, Timo Sirainen wrote: On 29 Feb 2016, at 17:18, Gordon Grubert wrote: Hi, we are using a round robin dns record for connections to our ldap system. This works fine for almost all cases. In particular, for dovecot does this mean, when an ldap server is stopped, dovecot instantly reconnects to another ldap server. But when the network connection to the active ldap server is broken, dovecot sticks to the failed ldap server. Is there any possibility to define a connection timeout? What should happen is that as long as new requests keep coming, Dovecot realizes after about 60 seconds that the LDAP server is hanging. It then reconnects and the reconnection should work. But... First of all, 60 seconds is likely a much too long timeout. But more importantly it looks like there's something weird now going on with OpenLDAP library. I added this somewhat recently and tested that it works: https://github.com/dovecot/core/commit/fb3178a1924dae52151d88c4d4ded879df43dd3f But now that I'm testing it, the timeout doesn't seem to be triggering. I don't know what happened to it that it suddenly doesn't work.. This also means that OpenLDAP seems to be internally stuck trying to connect to a server that isn't responding. Dovecot doesn't currently make the decisions on which LDAP server to connect to. It just passes through all the hosts to OpenLDAP library and lets it handle it. And it seems like OpenLDAP library can't right now do this failover. So maybe Dovecot should be responsible for that as well.. Anyway, for now you could set up haproxy to localhost and configure Dovecot LDAP to connect to haproxy and haproxy connect to the actual LDAP servers. today I've upgraded to 2.2.21-1~auto+171 on debian 8 and made a lot of "interruption tests". Your fix not really solved the problem. But I found another interesting fact: The openldap client on debian 8 can handle hard communication interrupts correctly. I've added NETWORK_TIMEOUT 5 TIMEOUT 5 to ldap.conf because man 5 ldap.conf says: NETWORK_TIMEOUT Specifies the timeout (in seconds) after which the poll(2)/select(2) following a connect(2) returns in case of no activity. TIMEOUT Specifies a timeout (in seconds) after which calls to synchronous LDAP APIs will abort if no response is received. Also used for any ldap_result(3) calls where a NULL timeout parameter is supplied. We are using the ISC DHCP server with dynamic ldap connections. This daemon uses - like dovecot - the LDAP API of the openldap client for access to the ldap server. The DHCP opens a persistent ldap connection to handle all dhcp requests (same behavior like dovecot). Here, the timeouts for connection loss are working. Therefore, my question: Why does this not work for dovecot, too, when dovecot uses the same API? Dovecot does not get a response from the LDAP server and has to reconnect, only. IMAP server world domination requires a reconnect in case of connection timeouts ;-) Best regards, Gordon
Re: Troubleshooting mailbox problems
On 11.03.2016 01:18, Nick Bright wrote: On 3/10/2016 3:50 PM, Andrew McGlashan wrote: If they are using POP to download messages from any client, make sure the client does a leave on server for at least long enough for other client devices to download the messages. Cheers A. Yes, that's the whole problem. They SAY that it's set for leave on server, but it's not happening that way. Essentially, observed behavior does not match the values of settings in the mail clients. But with your logging settings above you can prove the client commands. Best regards, Gordon
Re: ETOOMANYREFS related errors
El vie., 11 mar. 2016 a las 2:08, Timo Sirainen () escribió: > On 11 Mar 2016, at 03:48, Luis Ugalde wrote: > > > > Hi, > > > > I'm starting to see, on a pretty standard Debian Jessie installation, > some > > error messages that are apparently related to the ETOOMANYREFS errno. > > > > Firstly, the mail log shows this: > > dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many > references: > > cannot splice > > Apparently because Linux thinks the same fd has been passed around > recursively too many times: > http://lkml.iu.edu/hypermail/linux/kernel/1101.0/01917.html > > But Dovecot doesn't pass it recursively. It's only passed once from > pop3-login to pop3 process. > > > Is this something that Dovecot should be able to handle, or is it > strictly > > Debian/libc/MySillyMistake related? > > > > #uname -a > > Linux server 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 > > (2016-01-17) x86_64 GNU/Linux > > I wonder if there's a new kernel change that started detecting the > recursion wrong. > > Yes, It's started to happen with the latest kernels from the Stable branch. This is the latest I have installed: 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 Older Debian kernels are not showing up anything. This one, for example: 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 Regards.
