Re: v2.2.16 released
On 03/23/2015 12:10 AM, Gedalya wrote: This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. We never had such a crash on this server before. However, this too was a single incident. All accounts use Maildir. Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains///Maildir/dovecot.index: Append with UID 6684, but next_uid = 6685 Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error: /stor/mail/domains///Maildir/dovecot.index view syncing failed to apply changes Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344) [0x7f34cda5b344] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](client_output+0xe0) [0x7f34cda52910] -> /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f34cd28f88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f34cd28e918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f34cd2307d3] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](main+0x2b7) [0x7f34cda46997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0xdb09) [0x7f34cda46b09] Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Fatal: master: service(imap): child 30144 killed with signal 6 (core dumps disabled) And now it just happened, again, same user! The previous dovecot version before the upgrade was 2.2.12 Mar 28 07:51:17 imap1 dovecot: imap(@.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/.com//Maildir/dovecot.index: Append with UID 12794, but next_uid = 12795 Mar 28 07:51:17 imap1 dovecot: imap(@.com): Error: /stor/mail/domains/.com//Maildir/dovecot.index view syncing failed to apply changes Mar 28 07:51:17 imap1 dovecot: imap(@.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 28 07:51:17 imap1 dovecot: imap(@.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f136acdc57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f136acdc5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f136ac8b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f136b006391] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](imap_sync_more+0xce) [0x7f136b4bad2e] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](+0x22344) [0x7f136b4bb344] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](cmd_sync_delayed+0x1bf) [0x7f136b4bb67f] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](client_handle_input+0x1d5) [0x7f136b4b2725] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](client_input+0x75) [0x7f136b4b2a35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f136acee80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f136acef88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f136acee899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f136acee918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f136ac907d3] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](main+0x2b7) [0x7f136b4a6997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f136a8f4ead] -> dovecot/imap [@.com xx.xx.xx.xx UID FETCH](+0xdb09) [0x7f136b4a6b09] Mar 28 07:51:17 imap1 dovecot: imap(@.com): Fatal: master: service(imap): child 12444 killed with signal 6 (core dumps disabled)
Re: v2.2.16 released
This happened after upgrading to 2.2.16. Running on Debian wheezy amd64.
We never had such a crash on this server before. However, this too was a
single incident.
All accounts use Maildir.
Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error: Log
synchronization error at seq=0,offset=0 for
/stor/mail/domains///Maildir/dovecot.index: Append with UID
6684, but next_uid = 6685
Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error:
/stor/mail/domains///Maildir/dovecot.index view syncing failed
to apply changes
Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Panic: file
index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion
failed: (range->seq2 <= ctx->messages_count)
Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] ->
/usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] ->
/usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] ->
dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap
[@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344)
[0x7f34cda5b344] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH -
2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] ->
dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](client_output+0xe0) [0x7f34cda52910] ->
/usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb)
[0x7f34cd28f88b] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9)
[0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f34cd28e918] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f34cd2307d3] -> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH -
2373 bytes waiting](main+0x2b7) [0x7f34cda46997] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead]
-> dovecot/imap [@---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](+0xdb09) [0x7f34cda46b09]
Mar 21 09:32:02 imap1 dovecot: imap(@---.com): Fatal: master:
service(imap): child 30144 killed with signal 6 (core dumps disabled)
# doveconf -n
# 2.2.16: /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.7
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8
auth_master_user_separator = *
auth_mechanisms = plain login cram-md5
auth_verbose = yes
auth_verbose_passwords = plain
dict {
expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
login_greeting = Dovecot ready
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l
pip=%{real_rip} mpid=%e %c session=<%{session}>
mail_gid = vmail
mail_location = /nowhere
mail_plugins = quota expire listescape
mail_uid = vmail
managesieve_sieve_capability = fileinto envelope encoded-character
subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables mailbox date index ihave duplicate
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /etc/dovecot/master-users
driver = passwd-file
master = yes
pass = yes
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
antispam_backend = pipe
antispam_pipe_program = /usr/bin/ssh
antispam_pipe_program_args =
-l;spamd;-i;/etc/dovecot/sareport.key;mx1.x
antispam_pipe_program_notspam_arg = revoke
antispam_pipe_program_spam_arg = report
antispam_pipe_tmpdir = /tmp
antispam_spam = Junk
antispam_trash_pattern_ignorecase = trash;Deleted *
expire = Trash
expire2 = Trash/*
expire3 = Junk
expire_dict = proxy::expire
quota = dict:user::proxy::quota
quota_rule = *:storage=2G
quota_rule2 = Trash:storage=+250M
quota_rule3 = Junk:ignore
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=85%% quota-warning 85 %u
sieve = ~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve-global/fileinto-spam.sieve
sieve_dir = ~/sieve
sieve_extensions = -vacation -body -reject -enotify -environment
-virustest -spamtest
sieve_max_actions = 32
sieve_max_redirects = 4
sieve_max_script_size = 10K
sieve_quota_max_scripts = 100
sieve_quota_max_storage = 4M
}
pop3_no_flag_updates = yes
protocols = imap pop3 lmtp sieve
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 2448
unix_listener auth-userdb {
group
Re: v2.2.16 released
On 13 Mar 2015, at 12:34, Reindl Harald wrote: > > > Am 13.03.2015 um 11:23 schrieb Timo Sirainen: >> On 12 Mar 2015, at 21:09, Reindl Harald wrote: >>> >>> /usr/lib64/dovecot/stats/libstats_mail.so >>> >>> why in the world a new sub-directory containing just one so-file enforcing >>> pakcage buildsers to change SPEC files? >> >> So that external plugins can add more files in there and extend the >> available statistics > > but they can also go to /usr/lib64/dovecot/ > http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath These are plugins, not libraries. They don't need rpaths. The reason for the new directory's existence is exactly the same as for the already existing plugin directories: $lib/dovecot/auth/ $lib/dovecot/doveadm/ $lib/dovecot/settings/ $lib/dovecot/sieve/ So auth, doveadm, settings and sieve all work by loading all the plugins from inside the directories. If I changed the stats to work differently, e.g. load $lib/dovecot/libstats_*.so it would work differently than everything else, which would be confusing. For consistency it would be better then to get rid of the above directories as well. But changing that would break backwards compatibility with external plugins (e.g. with Sieve).
Re: v2.2.16 released
Am 13.03.2015 um 11:23 schrieb Timo Sirainen:
On 12 Mar 2015, at 21:09, Reindl Harald wrote:
/usr/lib64/dovecot/stats/libstats_mail.so
why in the world a new sub-directory containing just one so-file enforcing
pakcage buildsers to change SPEC files?
So that external plugins can add more files in there and extend the available
statistics
but they can also go to /usr/lib64/dovecot/
http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath
hence
cat /etc/ld.so.conf.d/dovecot-x86_64.conf
/usr/lib64/dovecot
i just "rm -rf" the folder and other stuff for private builds in
environments where dovecot is running only as proxy
rm -rf %{buildroot}%{_sysconfdir}/%{name}/README \
%{buildroot}%{_docdir}/%{name}-%{version} \
%{buildroot}%{_includedir}/%{name}/ \
%{buildroot}%{_mandir}/man1/ \
%{buildroot}%{_mandir}/man7/ \
%{buildroot}%{_datarootdir}/aclocal/ \
%{buildroot}%{_bindir}/dsync \
%{buildroot}%{_libdir}/%{name}/*_plugin.so \
%{buildroot}%{_libdir}/%{name}/doveadm/*_plugin.so \
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so \
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0 \
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0.0.0 \
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so \
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0 \
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0.0.0 \
%{buildroot}%{_libdir}/%{name}/stats \
%{buildroot}%{_libexecdir}/%{name}/%{name}-lda \
%{buildroot}%{_libexecdir}/%{name}/gdbhelper \
%{buildroot}%{_libexecdir}/%{name}/quota-status \
%{buildroot}%{_libexecdir}/%{name}/deliver \
%{buildroot}%{_libexecdir}/%{name}/lmtp
signature.asc
Description: OpenPGP digital signature
Re: v2.2.16 released
On 12 Mar 2015, at 21:09, Reindl Harald wrote: > > /usr/lib64/dovecot/stats/libstats_mail.so > > why in the world a new sub-directory containing just one so-file enforcing > pakcage buildsers to change SPEC files? So that external plugins can add more files in there and extend the available statistics.
Re: v2.2.16 released
/usr/lib64/dovecot/stats/libstats_mail.so why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? Am 12.03.2015 um 18:30 schrieb Timo Sirainen: http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig A few fixes and some imapc improvements since the release candidate. * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-] parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes. signature.asc Description: OpenPGP di
Re: v2.2.16 released
For this no man-page available. Am 12.03.2015 um 18:30 schrieb Timo Sirainen: + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve).
v2.2.16 released
http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig A few fixes and some imapc improvements since the release candidate. * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-] parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes.
