Re: v2.3.9 released
On 2019-12-04 11:34, Aki Tuomi via dovecot wrote: + Add lmtp_add_received_header setting. It can be used to prevent LMTP from adding "Received:" headers. Thank you for this :) -- Tom
Re: About "received" header when using Dovecot proxy
On 2019-12-02 13:42, Riku via dovecot wrote: Hello. My name is Riku. Currently, I use Dovecot as a proxy for another SMTP server. However, this seems to cause the IP address of the "received" header to be that of the proxy server. Is it possible to change this so that the IP address of the sender is entered? The version of Dovecot is "2.3.8 (9df20d2db)". Sorry for the incomprehensible explanation. This has been discussed a few times on the list already, and I believe there is a fix coming at some point: https://github.com/dovecot/core/pull/74 Currently there is none -- Tom
Re: Dovecot v2.3.8 released
On 2019-10-20 11:30, Timo Sirainen via dovecot wrote: On 18 Oct 2019, at 13.43, Tom Sommer via dovecot wrote: Quite a lot of mail downloads for a single session. I wonder if the user really had that many new mails or if they were being redownloaded for some reason? Oct 18 13:40:56 imap()<7552>: Debug: Mailbox Junk: Mailbox opened because: autoexpunge Oct 18 13:40:56 imap()<7552>: Debug: Mailbox Junk E-mail: Mailbox opened because: autoexpunge Oct 18 13:40:56 imap()<7552>: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.542 + waiting input/output for 78.357 secs, 60 B in + 39221480+8192 B out, state=wait-output) in=290 out=39401283 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=94 body_bytes=39210315 state=wait-output means Dovecot was waiting for client to read the data it is sending. This made me think it might be a connection (TCP/IP) thing. My Director is running with 3 listen-IPs, I removed two of them and this appeared to have made the errors stop. Unsure if anything was changed in this regard between 2.3.7.2 and 2.3.8. --- Tom
Re: Dovecot v2.3.8 released
On 2019-10-20 11:30, Timo Sirainen via dovecot wrote: On 18 Oct 2019, at 13.43, Tom Sommer via dovecot wrote: I am seeing a lot of errors since the upgrade, on multiple client accounts: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.242 + waiting input/output for 108.816 secs, 60 B in + 24780576+8192 B out, state=wait-output) Using NFS storage (not running with the mail_nfs_index or mail_nfs_storage) Was something changed in terms of IO/Idle timeouts? We are also seeing different I/O patterns since the upgrade, more I/O is being used than normal. What was the previous version you were running? 2.3.7.2 This is mail_debug from one of the accounts in question: Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: Mailbox opened because: SELECT Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17854: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17854: Opened mail because: full mail .. Oct 18 13:39:48 imap()<7552>: Debug: Mailbox INBOX: UID 17947: Opened mail because: full mail Quite a lot of mail downloads for a single session. I wonder if the user really had that many new mails or if they were being redownloaded for some reason? They might redownload because of UID FETCH failing? Oct 18 13:40:56 imap()<7552>: Debug: Mailbox Junk: Mailbox opened because: autoexpunge Oct 18 13:40:56 imap()<7552>: Debug: Mailbox Junk E-mail: Mailbox opened because: autoexpunge Oct 18 13:40:56 imap()<7552>: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.542 + waiting input/output for 78.357 secs, 60 B in + 39221480+8192 B out, state=wait-output) in=290 out=39401283 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=94 body_bytes=39210315 state=wait-output means Dovecot was waiting for client to read the data it is sending. In v2.3.7 there was some changes related to this, but were you previously successfully running v2.3.7? In v2.3.8 I can't really think of such changes. Yes, we were successfully running 2.3.7.2 before, the issue started just after the upgrade It can't be related to changes in the indexes? Increasing I/O There were no input/output errors in the logs prior to 2.3.8 --- Tom
Re: Dovecot v2.3.8 released
On 2019-10-18 14:55, Aki Tuomi via dovecot wrote: On 18/10/2019 14:25 Tom Sommer via dovecot wrote: On 2019-10-08 13:18, Aki Tuomi via dovecot wrote: > https://dovecot.org/releases/2.3/dovecot-2.3.8.tar.gz > https://dovecot.org/releases/2.3/dovecot-2.3.8.tar.gz.sig > Binary packages in https://repo.dovecot.org/ > - imap: SETMETADATA with literal value would delete the metadata value > instead of updating it. > - imap: When client issues FETCH PREVIEW (LAZY=FUZZY) command, the > caching decisions should be updated so that newly saved mails will have > the preview cached. > - With mail_nfs_index=yes and/or mail_nfs_storage=yes setuid/setgid > permission bits in some files may have become dropped with some NFS > servers. Changed NFS flushing to now use chmod() instead of chown(). I am seeing a lot of errors since the upgrade, on multiple client accounts: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.242 + waiting input/output for 108.816 secs, 60 B in + 24780576+8192 B out, state=wait-output) This is not actually an error, it's indicating that the remote end disconnected mid-command. Right, but this a symptom of the increased IO and thus latency in 2.3.8, so the client disconnects while waiting for a response? --- Tom
Re: Dovecot v2.3.8 released
On 2019-10-18 13:25, Tom Sommer via dovecot wrote: I am seeing a lot of errors since the upgrade, on multiple client accounts: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.242 + waiting input/output for 108.816 secs, 60 B in + 24780576+8192 B out, state=wait-output) Using NFS storage (not running with the mail_nfs_index or mail_nfs_storage) Was something changed in terms of IO/Idle timeouts? We are also seeing different I/O patterns since the upgrade, more I/O is being used than normal. This is mail_debug from one of the accounts in question: Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: Mailbox opened because: SELECT Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17854: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17854: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17855: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17855: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17856: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17856: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17857: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17857: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17858: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17858: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17859: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17859: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17860: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17860: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17861: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17861: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17862: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17862: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17863: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17863: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17864: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17864: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17865: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17865: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17866: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17866: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17867: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17867: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17868: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17868: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17869: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17869: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17870: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17870: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17871: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17871: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17872: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17872: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17873: Opened mail because: prefetch Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17873: Opened mail because: full mail Oct 18 13:39:37 imap()<7552>: Debug: Mailbox INBOX: UID 17874: Opened mail because: pr
Re: Dovecot v2.3.8 released
On 2019-10-08 13:18, Aki Tuomi via dovecot wrote: https://dovecot.org/releases/2.3/dovecot-2.3.8.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.8.tar.gz.sig Binary packages in https://repo.dovecot.org/ - imap: SETMETADATA with literal value would delete the metadata value instead of updating it. - imap: When client issues FETCH PREVIEW (LAZY=FUZZY) command, the caching decisions should be updated so that newly saved mails will have the preview cached. - With mail_nfs_index=yes and/or mail_nfs_storage=yes setuid/setgid permission bits in some files may have become dropped with some NFS servers. Changed NFS flushing to now use chmod() instead of chown(). I am seeing a lot of errors since the upgrade, on multiple client accounts: Info: Connection closed: read(size=7902) failed: Connection reset by peer (UID FETCH running for 0.242 + waiting input/output for 108.816 secs, 60 B in + 24780576+8192 B out, state=wait-output) Using NFS storage (not running with the mail_nfs_index or mail_nfs_storage) Was something changed in terms of IO/Idle timeouts? Tried doing reindex/resync, but did not help. -- Tom
Re: fts-elastic plugin
On 2019-09-19 10:24, Filip Hanes via dovecot wrote: Hi all, I have recently worked on fts plugin for ElasticSearch. https://github.com/filiphanes/fts-elastic It is forked from fts-elasticsearch as you can see in PR https://github.com/atkinsj/fts-elasticsearch/pull/21 In my opinion fts-elastic is now functionally on level as currently recommended FTS plugin: fts-solr. I would like to implement proper rescan inspired by fts-lucene so it would become the most complete fts plugin for dovecot. -- Awesome. Thank you for this. --- Tom
Re: Force dovecot-uidlist reset
Nevermind, the indexes were not deleted correctly - the method described below works :) --- Tom On 2019-09-09 09:45, Tom Sommer via dovecot wrote: Is there a way to force Dovecot to rebuild dovecot-uidlist from zero? It seems deleting all indexes and dovecot-* files followed by "doveadm force-resync" is not enough? It just gets the same UIDs? Perhaps from Maildir filenames? But I would like to reset the uid of all mails. Thanks
Force dovecot-uidlist reset
Is there a way to force Dovecot to rebuild dovecot-uidlist from zero? It seems deleting all indexes and dovecot-* files followed by "doveadm force-resync" is not enough? It just gets the same UIDs? Perhaps from Maildir filenames? But I would like to reset the uid of all mails. Thanks -- Tom
Re: Feature wishlist: Allow to hide client IP/host in submission service
On 2019-08-28 14:07, Timo Sirainen via dovecot wrote: On 25 Aug 2019, at 21.51, Sebastian Krause via dovecot wrote: Hi, In many mail setups a required feature (for privacy reasons) is to hide the host and IP of clients (in the "Received" header) that use the authenticated submission over port 587. In Postfix that's possible (https://serverfault.com/q/413533/86332), but not very nice to configure especially if you only want want to strip the Received header for port 587 submissions, but not on port 25. As far as I can see this configuration is not possible at all in the Dovecot submission server because the function which adds the Received header with the client's IP address (smtp_server_transaction_write_trace_record) is always called in submission-commands.c. It would be very useful if the submission server could anonymize the client with a single configuration option, then all the Postfix configuration mess (and using SASL) could be skipped by simply using the Dovecot submission server instead. Yeah, it would be useful to hide the client's IP and do it by default. Actually I think there shouldn't even be an option to not hide it. Or would it be better or worse to just not have the Received header added at all? Better to just remove the Received header entirely. Make lmtp_add_received_headers work on submission as well, maybe?
Re: Feature wishlist: Allow to hide client IP/host in submission service
On 2019-08-25 20:51, Sebastian Krause via dovecot wrote: Hi, In many mail setups a required feature (for privacy reasons) is to hide the host and IP of clients (in the "Received" header) that use the authenticated submission over port 587. In Postfix that's possible (https://serverfault.com/q/413533/86332), but not very nice to configure especially if you only want want to strip the Received header for port 587 submissions, but not on port 25. As far as I can see this configuration is not possible at all in the Dovecot submission server because the function which adds the Received header with the client's IP address (smtp_server_transaction_write_trace_record) is always called in submission-commands.c. It would be very useful if the submission server could anonymize the client with a single configuration option, then all the Postfix configuration mess (and using SASL) could be skipped by simply using the Dovecot submission server instead. The anonymization would work by replacing the client's EHLO host with "submission" and the IP address with 127.0.0.1. In full the Received header would look something like this where the first line is always the same: Received: from submission (unknown [127.0.0.1]) by mail.example.com with ESMTPSA id 8bV9D+51Yl1FOwAA1ctoJQ (envelope-from ) for ; Sun, 25 Aug 2019 13:50:06 +0200 Check https://github.com/dovecot/core/pull/74 Unsure if it covers Submission though
Re: Dovecot v2.3.7.1 & Pigeonhole v0.5.7.1 released
On 2019-07-23 14:01, Timo Sirainen via dovecot wrote: https://dovecot.org/releases/2.3/dovecot-2.3.7.1.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.7.1.tar.gz.sig https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.1.tar.gz https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.1.tar.gz.sig Binary packages in https://repo.dovecot.org/ These releases fix the reported regressions in v2.3.7 & v0.5.7. Dovecot core: - Fix TCP_NODELAY errors being logged on non-Linux OSes - lmtp proxy: Fix assert-crash when client uses BODY=8BITMIME - Remove wrongly added checks in namespace prefix checking Pigeonhole: - dsync: Sieve script syncing failed if mailbox attributes weren't enabled. Thank you
Re: Dovecot release v2.3.7
On 2019-07-16 09:46, Timo Sirainen via dovecot wrote: > On 13 Jul 2019, at 14.44, Tom Sommer via dovecot wrote: > >> LMTP is broken on director: >> >> Jul 13 13:42:41 lmtp(34824): Panic: file smtp-params.c: line 685 >> (smtp_params_mail_add_body_to_event): assertion failed: ((caps & >> SMTP_CAPABILITY_8BITMIME) != 0) > > Thanks, fixed: > https://github.com/dovecot/core/commit/c4de81077c11d09eddf6a5c93676ee82350343a6 Thanks. Is there a new release?
Re: Dovecot release v2.3.7
On 2019-07-13 13:44, Tom Sommer via dovecot wrote: LMTP is broken on director: Jul 13 13:42:41 lmtp(34824): Panic: file smtp-params.c: line 685 (smtp_params_mail_add_body_to_event): assertion failed: ((caps & SMTP_CAPABILITY_8BITMIME) != 0) Jul 13 13:42:41 lmtp(34824): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xdf06a) [0x7f561203806a] -> /usr/lib64/dovecot/libdovecot.so.0(+0xdf0b1) [0x7f56120380b1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x40161) [0x7f5611f99161] -> /usr/lib64/dovecot/libdovecot.so.0(+0x44fc4) [0x7f5611f9dfc4] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_client_transaction_start+0x78) [0x7f5611fa7a58] -> dovecot/lmtp [172.17.165.6 MAIL FROM](lmtp_proxy_rcpt+0xa37) [0x558eaf181d67] -> dovecot/lmtp [172.17.165.6 MAIL FROM](client_default_cmd_rcpt+0x9e) [0x558eaf17eebe] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_server_cmd_rcpt+0x222) [0x7f5611fafd12] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_server_command_new+0x150) [0x7f5611fb58a0] -> /usr/lib64/dovecot/libdovecot.so.0(+0x606c8) [0x7f5611fb96c8] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f561204f275] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x95) [0x7f561204f3a5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f561204f588] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f5611fc68f3] -> dovecot/lmtp [172.17.165.6 MAIL FROM](main+0x289) [0x558eaf17dc19] -> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f5611be3d20] -> dovecot/lmtp [172.17.165.6 MAIL FROM](+0x5849) [0x558eaf17d849] Downgraded to 2.3.6 (from source, because it's gone from repo) - Works. -- Tom
Re: Dovecot release v2.3.7
Can you please put dovecot-2.3.6-2.x86_64 back in the repo so we can
downgrade?
---
Tom
On 2019-07-12 14:29, Aki Tuomi via dovecot wrote:
Hi!
We are pleased to release Dovecot release v2.3.7.
Tarball is available at
https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz.sig
Binary packages are available at https://repo.dovecot.org/
Changes
---
* fts-solr: Removed break-imap-search parameter
+ Added more events for the new statistics, see
https://doc.dovecot.org/admin_manual/list_of_events/
+ mail-lua: Add IMAP metadata accessors, see
https://doc.dovecot.org/admin_manual/lua/
+ Add event exporters that allow exporting raw events to log files and
external systems, see
https://doc.dovecot.org/configuration_manual/event_export/
+ SNIPPET is now PREVIEW and size has been increased to 200 characters.
+ Add body option to fts_enforced. This triggers building FTS index
only
on body search, and an error using FTS index fails the search rather
than reads through all the mails.
- Submission/LMTP: Fixed crash when domain argument is invalid in a
second EHLO/LHLO command.
- Copying/moving mails using Maildir format loses IMAP keywords in the
destination if the mail also has no system flags.
- mail_attachment_detection_options=add-flags-on-save caused email body
to be unnecessarily opened when FETCHing mail headers that were
already cached.
- mail attachment detection keywords not saved with maildir.
- dovecot.index.cache may have grown excessively large in some
situations. This happened especially when using autoexpunging with
lazy_expunge folders. Also with mdbox format in general the cache
file
wasn't recreated as often as it should have.
- Autoexpunged mails weren't immediately deleted from the disk.
Instead,
the deletion from disk happened the next time the folder was opened.
This could have caused unnecessary delays if the opening was done by
an interactive IMAP session.
- Dovecot's TCP connections sometimes add extra 40ms latency due to not
enabling TCP_NODELAY. HTTP and SMTP/LMTP connections weren't
affected, but everything else was. This delay wasn't always visible -
only in some situations with some message/packet sizes.
- imapc: Fix various crash conditions
- Dovecot builds were not always reproducible.
- login-proxy: With shutdown_clients=no after config reload the
existing connections could no longer be listed or kicked with
doveadm.
- "doveadm proxy kick" with -f parameter caused a crash in some
situations.
- Auth policy can cause segmentation fault crash during auth process
shutdown if all auth requests have not been finished.
- Fix various minor bugs leading into incorrect behaviour in mailbox
list index handling. These rarely caused noticeable problems.
- LDAP auth: Iteration accesses freed memory, possibly crashing
auth-worker
- local_name { .. } filter in dovecot.conf does not correctly support
multiple names and wildcards were matched incorrectly.
- replicator: dsync assert-crashes if it can't connect to remote TCP
server.
- config: Memory leak in config process when ssl_dh setting wasn't
set and there was no ssl-parameters.dat file.
This caused config process to die once in a while
with "out of memory".
---
Aki Tuomi
Open-Xchange oy
Re: Dovecot release v2.3.7
LMTP is broken on director:
Jul 13 13:42:41 lmtp(34824): Panic: file smtp-params.c: line 685
(smtp_params_mail_add_body_to_event): assertion failed: ((caps &
SMTP_CAPABILITY_8BITMIME) != 0)
Jul 13 13:42:41 lmtp(34824): Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(+0xdf06a) [0x7f561203806a] ->
/usr/lib64/dovecot/libdovecot.so.0(+0xdf0b1) [0x7f56120380b1] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x40161) [0x7f5611f99161] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x44fc4) [0x7f5611f9dfc4] ->
/usr/lib64/dovecot/libdovecot.so.0(smtp_client_transaction_start+0x78)
[0x7f5611fa7a58] -> dovecot/lmtp [172.17.165.6 MAIL
FROM](lmtp_proxy_rcpt+0xa37) [0x558eaf181d67] -> dovecot/lmtp
[172.17.165.6 MAIL FROM](client_default_cmd_rcpt+0x9e) [0x558eaf17eebe]
-> /usr/lib64/dovecot/libdovecot.so.0(smtp_server_cmd_rcpt+0x222)
[0x7f5611fafd12] ->
/usr/lib64/dovecot/libdovecot.so.0(smtp_server_command_new+0x150)
[0x7f5611fb58a0] -> /usr/lib64/dovecot/libdovecot.so.0(+0x606c8)
[0x7f5611fb96c8] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55)
[0x7f561204f275] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x95)
[0x7f561204f3a5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f561204f588] ->
/usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f5611fc68f3] -> dovecot/lmtp [172.17.165.6 MAIL FROM](main+0x289)
[0x558eaf17dc19] -> /lib64/libc.so.6(__libc_start_main+0x100)
[0x7f5611be3d20] -> dovecot/lmtp [172.17.165.6 MAIL FROM](+0x5849)
[0x558eaf17d849]
---
Tom
On 2019-07-12 14:29, Aki Tuomi via dovecot wrote:
Hi!
We are pleased to release Dovecot release v2.3.7.
Tarball is available at
https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz.sig
Binary packages are available at https://repo.dovecot.org/
Changes
---
* fts-solr: Removed break-imap-search parameter
+ Added more events for the new statistics, see
https://doc.dovecot.org/admin_manual/list_of_events/
+ mail-lua: Add IMAP metadata accessors, see
https://doc.dovecot.org/admin_manual/lua/
+ Add event exporters that allow exporting raw events to log files and
external systems, see
https://doc.dovecot.org/configuration_manual/event_export/
+ SNIPPET is now PREVIEW and size has been increased to 200 characters.
+ Add body option to fts_enforced. This triggers building FTS index
only
on body search, and an error using FTS index fails the search rather
than reads through all the mails.
- Submission/LMTP: Fixed crash when domain argument is invalid in a
second EHLO/LHLO command.
- Copying/moving mails using Maildir format loses IMAP keywords in the
destination if the mail also has no system flags.
- mail_attachment_detection_options=add-flags-on-save caused email body
to be unnecessarily opened when FETCHing mail headers that were
already cached.
- mail attachment detection keywords not saved with maildir.
- dovecot.index.cache may have grown excessively large in some
situations. This happened especially when using autoexpunging with
lazy_expunge folders. Also with mdbox format in general the cache
file
wasn't recreated as often as it should have.
- Autoexpunged mails weren't immediately deleted from the disk.
Instead,
the deletion from disk happened the next time the folder was opened.
This could have caused unnecessary delays if the opening was done by
an interactive IMAP session.
- Dovecot's TCP connections sometimes add extra 40ms latency due to not
enabling TCP_NODELAY. HTTP and SMTP/LMTP connections weren't
affected, but everything else was. This delay wasn't always visible -
only in some situations with some message/packet sizes.
- imapc: Fix various crash conditions
- Dovecot builds were not always reproducible.
- login-proxy: With shutdown_clients=no after config reload the
existing connections could no longer be listed or kicked with
doveadm.
- "doveadm proxy kick" with -f parameter caused a crash in some
situations.
- Auth policy can cause segmentation fault crash during auth process
shutdown if all auth requests have not been finished.
- Fix various minor bugs leading into incorrect behaviour in mailbox
list index handling. These rarely caused noticeable problems.
- LDAP auth: Iteration accesses freed memory, possibly crashing
auth-worker
- local_name { .. } filter in dovecot.conf does not correctly support
multiple names and wildcards were matched incorrectly.
- replicator: dsync assert-crashes if it can't connect to remote TCP
server.
- config: Memory leak in config process when ssl_dh setting wasn't
set and there was no ssl-parameters.dat file.
This caused config process to die once in a while
with "out of memory".
---
Aki Tuomi
Open-Xchange oy
Re: Orphaned processes after doveadm log reopen
On 2019-07-09 07:37, Aki Tuomi wrote: On 8.7.2019 14.54, Tom Sommer via dovecot wrote: On 2019-07-08 13:36, Tom Sommer via dovecot wrote: I rotate logs every night on my Director, running "doveadm log reopen" It happens on "/etc/init.d/dovecot restart", not "doveadm log reopen" - sorry So it happens when you run killproc on dovecot Do you happen to have shutdown_clients=no ? Nope. shutdown_clients = yes
Re: Orphaned processes after doveadm log reopen
On 2019-07-08 13:36, Tom Sommer via dovecot wrote: I rotate logs every night on my Director, running "doveadm log reopen" It happens on "/etc/init.d/dovecot restart", not "doveadm log reopen" - sorry So it happens when you run killproc on dovecot --- Tom
Re: Orphaned processes after doveadm log reopen
On 2019-07-08 13:36, Tom Sommer via dovecot wrote: I rotate logs every night on my Director, running "doveadm log reopen" I've noticed a problem with processes not closing correctly, they can be open for days and cause corrupt index files because they are connected to different backends (so it writes to the wrong server when the process disconnects) 23518 ?S140:53 dovecot/anvil [4 connections] 23519 ?S444:45 dovecot/log 23520 ?S497:19 dovecot/imap-login [8 pre-login + 27 proxies] 23531 ?S 0:27 dovecot/config 26201 ?S551:53 dovecot/imap-login [7 pre-login + 33 proxies] 22126 ?S960:15 dovecot/pop3-login [28 pre-login + 396 proxies] 59515 ?S218:32 dovecot/pop3-login [44 pre-login + 468 proxies] Is there any way to force these orphaned processes to terminate after a certain time? I ran "strace -f -p" on 59515 which caused the process to wake up and terminate? Very strange. I can send the dump from strace if you want, but since it contains addresses I would prefer it to be done off-list
Orphaned processes after doveadm log reopen
I rotate logs every night on my Director, running "doveadm log reopen" I've noticed a problem with processes not closing correctly, they can be open for days and cause corrupt index files because they are connected to different backends (so it writes to the wrong server when the process disconnects) 23518 ?S140:53 dovecot/anvil [4 connections] 23519 ?S444:45 dovecot/log 23520 ?S497:19 dovecot/imap-login [8 pre-login + 27 proxies] 23531 ?S 0:27 dovecot/config 26201 ?S551:53 dovecot/imap-login [7 pre-login + 33 proxies] 22126 ?S960:15 dovecot/pop3-login [28 pre-login + 396 proxies] 59515 ?S218:32 dovecot/pop3-login [44 pre-login + 468 proxies] Is there any way to force these orphaned processes to terminate after a certain time? -- Tom
Re: Frequent Out of Memory for service(config)
On 2019-05-13 21:56, Root Kev via dovecot wrote: Hello Group, We have dovecot deployed as solely a Pop3 service that is used by our applications to pass mail from one application to another internally. We have roughly 4 applications that connect to the Pop3 service every 2 seconds, to check for new messages and pop them for processing if they are present. Depending on the site, we have between 1024-2048MB of memory set for default_vsz_limit. In all systems we see the Out of memory alert several times a day. We previously did not see this at all when running on CentOS6, with less memory. I see this too on servers running quota-service (dunno if it is related). --- Tom
Re: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header
On 2019-04-19 15:26, Aki Tuomi via dovecot wrote: Unfortunately we have quite long list of things to do, so sometimes even trivial things can take a long time. Not to hijack the thread, but perhaps you could elaborate on what has changed within Dovecot? Timo seems to be put in the background, releases are less frequent and with less changes/additions. The days of "Oh, great idea - I added that, see this commit" seem gone. Is this because OX acquired Dovecot, so priorities have changed? Or what is going on? Mostly just curious. -- Tom
Re: quota-service with Director - A workaround
On 2019-03-21 10:28, Sami Ketola via dovecot wrote: On 20 Mar 2019, at 18.17, Tom Sommer via dovecot wrote: On 2019-03-20 16:40, Sami Ketola via dovecot wrote: On 20 Mar 2019, at 17.13, Tom Sommer via dovecot wrote: I realize quota-service on Director is not supported, which is a shame. As a workaround I'm thinking of setting up quota-service on one of my backend nodes, and have all my Postfix services ask this one node for the quota status. This sort of defeats the purpose of the Director (having per-user assigned hot nodes), since now this one node running the quota-service will access all mailboxes to check the status of all inbound mail. Is this a problem though? In terms of NFS locking etc. etc.? Might be. Wouldn't it be just easier to use the overquota-flag available since 2.2.16 and set up overquota flag in LDAP or userdb of choice and configure postfix to check that flag? I don't really want to involve LDAP in my setup :) So use what ever your shared userdb service is as you must have one if you are using multiple backends and directors. Does it work with mysql userdb? Is there an example to look at anywhere?
Re: quota-service with Director - A workaround
On 2019-03-20 16:40, Sami Ketola via dovecot wrote: On 20 Mar 2019, at 17.13, Tom Sommer via dovecot wrote: I realize quota-service on Director is not supported, which is a shame. As a workaround I'm thinking of setting up quota-service on one of my backend nodes, and have all my Postfix services ask this one node for the quota status. This sort of defeats the purpose of the Director (having per-user assigned hot nodes), since now this one node running the quota-service will access all mailboxes to check the status of all inbound mail. Is this a problem though? In terms of NFS locking etc. etc.? Might be. Wouldn't it be just easier to use the overquota-flag available since 2.2.16 and set up overquota flag in LDAP or userdb of choice and configure postfix to check that flag? I don't really want to involve LDAP in my setup :)
quota-service with Director - A workaround
I realize quota-service on Director is not supported, which is a shame. As a workaround I'm thinking of setting up quota-service on one of my backend nodes, and have all my Postfix services ask this one node for the quota status. This sort of defeats the purpose of the Director (having per-user assigned hot nodes), since now this one node running the quota-service will access all mailboxes to check the status of all inbound mail. Is this a problem though? In terms of NFS locking etc. etc.? -- Tom
Re: [ext] Dovecot Wiki: Please disable edit on double click
On 2019-03-20 12:02, Aki Tuomi via dovecot wrote: On 20.3.2019 12.59, Ralf Hildebrandt via dovecot wrote: * Michael Goth via dovecot : could you maybe disable the 'edit on doubleclick' feature on wiki2.dovecot.org? Everytime I try to select a word by double clicking on it, I end up in editing mode. It's just a minor thing, but maybe I'm not the only one who's annoyed by this ;) +1 Amen to that. I never bothered to ask, but it annoys the shit out of me! +10 It should be disabled now, unless you log into the wiki. Then you need to disable it from preferences. Happy day!
Director 2.3.5: openssl_iostream_handle_error
Thanks for 2.3.5 - it fixed a lot of stuff :) I see this Panic in my Director logs now, randomly and rarely: Mar 07 11:01:29 pop3-login: Panic: file iostream-openssl.c: line 586 (openssl_iostream_handle_error): assertion failed: (errno != 0) Mar 07 11:01:29 auth: Warning: auth client 13625 disconnected with 19 pending requests: EOF Mar 07 11:01:29 pop3-login: Fatal: master: service(pop3-login): child 13625 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2) Hoping the above is enough to debug -- Tom
