Hi, I have two Debian Jessie servers with Dovecot 2.2.13 TCP replication
on that have worked fine for years, but now one of them is running low
on disk space, so I wanted to try enabling zlib.
I crafted a script following the description given in
https://wiki.dovecot.org/Plugins/Zlib and xz'ed some inboxes on the
stand-by server, the one with low disk space. So every email in those
inboxes is xz'ed but the file name hasn't changed and contains the
original size.
This server is on stand-by so most of the email is replicated
unidirectionally to it. But administrative emails like cronjobs and
monitoring are delivered locally, so it replicates those to the hot server.
The issue appeared when this stand-by server receives such an email and
tries to replicate them to the other server.
I'm attaching the full snippet of the log from the hot server, because
it throws a longer backtrace. The short version is like this:
dovecot[8438]: imap(redac...@address.org): Error: Cached message size
larger than expected (478 > 289)
dovecot[8438]: imap(redac...@address.org): Error: Maildir filename has
wrong S value, renamed the file from
/srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=478:2,
to
/srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=289:2,
dovecot[8438]: imap(redac...@address.org): Error: Corrupted index cache
file /srv/email/address.org/redacted/Maildir/dovecot.index.cache: Broken
physical size for mail UID 45123
After this there's an error and the replication fails. The file is
there, it's gzipped and can be zcat'ed but it appears as a blank email
on clients.
I've recovered a backup but the issue persists. I also changed from xz
to gz because the Debian package docs only mention gz and bzip2, but the
issue is the same.
From what I understand and tested, the stand-by server is receiving the
email and compressing it but maintaining the original size on the file
name. So that's ok, but when the hot server receives the copy, it
believes the size is wrong and changes it to the compressed size. Then
for some reason the index gets corrupted.
I'm attaching the doveconf for both servers. They're mostly the same,
and the only changes introduced were the zlib plugin and its options.
Also the script(s) that I used to compress the inboxes.
Am I correct? Is it an issue of replicator not understanding the emails
are compressed? I couldn't find anything related to zlib with
replication. Maybe it's something fixed in newer versions and I should
go that rabbit hole?
Thanks! :)
dovecot[8438]: imap(redac...@address.org): Error: Cached message size larger than expected (478 > 289)
dovecot[8438]: imap(redac...@address.org): Error: Maildir filename has wrong S value, renamed the file from /srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=478:2, to /srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=289:2,
dovecot[8438]: imap(redac...@address.org): Error: Corrupted index cache file /srv/email/address.org/redacted/Maildir/dovecot.index.cache: Broken physical size for mail UID 45123
dovecot[8438]: imap(redac...@address.org): Panic: file istream.c: line 167 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip)
dovecot[8438]: imap(redac...@address.org): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6b6fe) [0x7f0471f756fe] -> /usr/lib/dovecot/libdovecot.so.0(+0x6b7ec) [0x7f0471f757ec] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0471f2c8fb] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x210) [0x7f0471f7e970] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f0471f7f07d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f0471f69f44] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x328) [0x7f0472283568] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x4b3f1) [0x7f047222f3f1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_stream+0x4d) [0x7f047225c2ed] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x4b1b7) [0x7f047222f1b7] -> dovecot/imap(+0x1c5b3) [0x560912d4f5b3] -> dovecot/imap(+0x1a85a) [0x560912d4d85a] -> dovecot/imap(imap_fetch_more+0x37) [0x560912d4e957] -> dovecot/imap(cmd_fetch+0x340) [0x560912d42070] -> dovecot/imap(command_exec+0x3c) [0x560912d4c09c] -> dovecot/imap(+0x17fb2) [0x560912d4afb2] -> dovecot/imap(+0x18066) [0x560912d4b066] -> dovecot/imap(client_handle_input+0x13d) [0x560912d4b39d] -> dovecot/imap(client_input+0x85) [0x560912d4b745] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x3f) [0x7f0471f86d0f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xf9) [0x7f0471f87d09] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f0471f86d79] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0471f86df8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f0471f31dc3] -> dovecot/imap(main+0x2c4)
