Re: [EXT] Re: Corrupted sizes using zlib plugin
Am 06.03.23 um 16:45 schrieb Aki Tuomi:
On 06/03/2023 16:52 EET Tim Evers wrote:
Am 06.03.23 um 15:29 schrieb Aki Tuomi:
On 06/03/2023 15:56 EET Tim Evers wrote:
Am 06.03.23 um 14:49 schrieb Aki Tuomi:
On 06/03/2023 15:45 EET Tim Evers wrote:
Am 06.03.23 um 14:00 schrieb Aki Tuomi:
On 06/03/2023 13:06 EET Tim Evers wrote:
Am 06.03.23 um 11:59 schrieb Aki Tuomi:
On 06/03/2023 12:44 EET Tim Evers wrote:
Am 06.03.23 um 11:42 schrieb Aki Tuomi:
On 06/03/2023 12:32 EET Tim Evers wrote:
Hi,
since I did not get any feedback on my bug report post regarding
corrupted sizes while using zlib
(https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I
would like to confirm that this bug report reached someone in charge.
Or if not - I would kindly ask for directions on how to post it in a way
that spawns some action.
Thx
Tim
Can you try erasing dovecot.index.cache and see if the problem fixes itself?
It's possible your cache contains invalid values from old errors.
Aki
Did that already several times (other dovecot.index.* files too). The
only thing that helps is actually deleting the dovecot-uidlist file but
the problem returns.
Tim
I think it's because the value on the filename is wrong. The filename contains
the virtual and physical size.
Can you try if https://github.com/dovecot/tools/blob/main/maildir-size-fix.pl
fixes your issue?
Aki
Quoting
https://doc.dovecot.org/configuration_manual/zlib_plugin/ :
"All mails must have ,|S=|in their filename where contains
the original uncompressed mail size, otherwise there will be problems
with quota calculation as well as other potential random failures."
And the script you send:
"# Check if the files are compressed. Use the uncompressed size for S=size."
Which it does in my example. S= contains the uncompressed size, not the
size on disk. Is this not correct? All Files are written exclusively by
dovecot, no other software is involved.
Tim
Can you send output from `doveconf -n`? I am having suspicion that zlib plugin
isn't always involved in your config, which causes the compressed file to be
cached wrongly. Especially it feels like your IMAP protocol does not have
compression on while LMTP does.
Aki
Actually it is the other way round (I removed zlib from lmtp to reduce
the impact of the problem):
# 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.20 (149edcf2)
# OS: Linux 4.15.0-204-generic x86_64 Ubuntu 18.04.6 LTS
# Hostname: node2.mst27.artfiles.de
auth_verbose = yes
auth_worker_max_count = 1000
default_client_limit = 5000
dict {
sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
import_environment = TZ CLUSTERNAME
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_location = maildir:~/Maildir
mail_plugins = " mail_log notify old_stats"
mailbox_idle_check_interval = 10 mins
maildir_broken_filename_sizes = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext editheader imapsieve vnd.dovecot.imapsieve
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix = INBOX.
separator = .
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
imapsieve_mailbox1_before = file:~/af_report-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_name = INBOX.Junk
imapsieve_mailbox2_before = file:~/af_report-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = INBOX.Junk
imapsieve_mailbox2_name = *
quota = dict:User quota::proxy::sqlquota
quota_rule = *:storage=0:messages=0
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_before = file:~/af_master.sieve
sieve_before2 = file:~/af_konto.sieve
sieve_before3 = file:~/af_domain.sieve
sieve_before4 = file:~/af_spamfilter.sieve
sieve_extensions = +editheader
sieve_filter_bin_dir = /usr/local/bin/sieve
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.filter
+vnd.dovecot.environment
sieve_max_redirects = 100
sieve_pipe_bin_dir = /usr/local/bin/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
stats_refresh = 30 secs
zlib_save = gz
zlib_save_level = 6
}
pop3_no_flag_upd
Re: Corrupted sizes using zlib plugin
Am 06.03.23 um 11:59 schrieb Aki Tuomi: On 06/03/2023 12:44 EET Tim Evers wrote: Am 06.03.23 um 11:42 schrieb Aki Tuomi: On 06/03/2023 12:32 EET Tim Evers wrote: Hi, since I did not get any feedback on my bug report post regarding corrupted sizes while using zlib (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I would like to confirm that this bug report reached someone in charge. Or if not - I would kindly ask for directions on how to post it in a way that spawns some action. Thx Tim Can you try erasing dovecot.index.cache and see if the problem fixes itself? It's possible your cache contains invalid values from old errors. Aki Did that already several times (other dovecot.index.* files too). The only thing that helps is actually deleting the dovecot-uidlist file but the problem returns. Tim I think it's because the value on the filename is wrong. The filename contains the virtual and physical size. Can you try if https://github.com/dovecot/tools/blob/main/maildir-size-fix.pl fixes your issue? Aki Quoting https://doc.dovecot.org/configuration_manual/zlib_plugin/ : "All mails must have ,|S=|in their filename where contains the original uncompressed mail size, otherwise there will be problems with quota calculation as well as other potential random failures." And the script you send: "# Check if the files are compressed. Use the uncompressed size for S=size." Which it does in my example. S= contains the uncompressed size, not the size on disk. Is this not correct? All Files are written exclusively by dovecot, no other software is involved. Tim
Re: Corrupted sizes using zlib plugin
Am 06.03.23 um 11:42 schrieb Aki Tuomi: On 06/03/2023 12:32 EET Tim Evers wrote: Hi, since I did not get any feedback on my bug report post regarding corrupted sizes while using zlib (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I would like to confirm that this bug report reached someone in charge. Or if not - I would kindly ask for directions on how to post it in a way that spawns some action. Thx Tim Can you try erasing dovecot.index.cache and see if the problem fixes itself? It's possible your cache contains invalid values from old errors. Aki Did that already several times (other dovecot.index.* files too). The only thing that helps is actually deleting the dovecot-uidlist file but the problem returns. Tim
Corrupted sizes using zlib plugin
Hi, since I did not get any feedback on my bug report post regarding corrupted sizes while using zlib (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I would like to confirm that this bug report reached someone in charge. Or if not - I would kindly ask for directions on how to post it in a way that spawns some action. Thx Tim
Re: Corrupted sizes in cache once again
Hi again,
so this is the actual bug report :)
I installed 2.3.20 from repo yet the errors persist.
I made the following observations:
For a mailbox producing the "broken physical size" messages the culprit
seems not to be the index.cache file but the dovecot-uidlist file.
Removing the cache file does nothing regarding the bug while removing
the dovecot-uidlist file immediately cleared the errors and all
subsequent access went through without error.
An example of an uidlist entry *with* errors:
70755 G1674471014.M447254P8688.node2 S937
:1674471014.M447254P8688.node2,S=1693,W=1730
Note: the S937 entry. This is definitely wrong.
Same mail *without* the errors (dovecot-uidlist auto-recreated after
deletion):
76040 :1674471014.M447254P8688.node2,S=1693,W=1730
Errors in log (domain anonymized):
Feb 15 21:02:57 node2 dovecot:
imap(local_u...@domain.com)<36929>: Error: Mailbox
INBOX.Trash: Deleting corrupted cache record uid=70755: UID 70755:
Broken physical size in mailbox INBOX.Trash:
read(compress(/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/cur/1674471014.M447254P8688.node2,S=1693,W=1730:2,))
failed: Cached message size smaller than expected (937 < 1693,
box=INBOX.Trash, UID=70755)
Feb 15 21:02:57 node2 dovecot:
imap(local_u...@domain.com)<36929>: Error: Mailbox
INBOX.Trash: UID=70755:
read(compress(/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/cur/1674471014.M447254P8688.node2,S=1693,W=1730:2,))
failed: Cached message size smaller than expected (937 < 1693,
box=INBOX.Trash, UID=70755) (read reason=)
Note: reported physical size from "cache", 937 is the compressed size on
disk. Decompressed size is 1693.
Excerpt from strace (full strace attached):
36929 openat(AT_FDCWD,
"/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/dovecot.index.cache",
O_RDWR) = -1 ENOENT (No such file or directory)
36929 openat(AT_FDCWD,
"/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/cur/1674471014.M447254P8688.node2,S=1693,W=1730:2,",
O_RDONLY) = 20
36929 fstat(20, {st_mode=S_IFREG|0600, st_size=937, ...}) = 0
36929 pread64(20,
"\37\213\10\0\0\0\0\0\0\3\255\224]O\3438\30\205\257\311\257\260\346\252hI\326\316g\223\231\2I\232\360!\272Ci\27\255\26\241\310M\334\326C\342tl\247|\374\372y\303\262\2\26\0207\233H\215\353\274\366y\217\237\243\\\260\222\361-\253\"\264\224m\203\32\312\353\266\323\266E\245^\362\232)\253bhpeS\214#\262\f\206\21]b;\212\354\360z\327\330Y\334?\326saclU\345F1\271e\262_q\313\365\0321\325\350\215B\203\371\331\214XN\224\245\343\343\254\230e\351\271\355\371\27\244(.fqq>\233\365O\230?\216a\272(\342lV\364\203\243t\22\301\23T\6\331\35o\220k\205\256e\303_^!\262"...,
8192, 0) = 937
36929 openat(AT_FDCWD,
"/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/dovecot.index.cache",
O_RDWR) = -1 ENOENT (No such file or directory)
36929 openat(AT_FDCWD,
"/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/dovecot.index.cache",
O_RDWR) = -1 ENOENT (No such file or directory)
36929 alarm(180) = 0
36929 fcntl(16, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0,
l_len=0}) = 0
36929 alarm(0) = 180
36929
stat("/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/dovecot.index.log",
{st_mode=S_IFREG|0600, st_size=144, ...}) = 0
36929 fstat(16, {st_mode=S_IFREG|0600, st_size=144, ...}) = 0
36929 write(16,
"\200\200\200\203\0\0\10\0208\0\0\0\200\200\200\207@\0\0\20\2\0\0\0002\315\321\\\0\0\0\0\4\0\4\0\1\0\0\0\200\200\200\204\0\2\0\20c\24\1\0\0\0\0\0",
56) = 56
36929 fcntl(16, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0,
l_len=0}) = 0
36929
stat("/home/mail/domains/v/a/domain.com/local_user/Maildir/.Trash/dovecot.index.log",
{st_mode=S_IFREG|0600, st_size=200, ...}) = 0
36929 fstat(16, {st_mode=S_IFREG|0600, st_size=200, ...}) = 0
36929 write(2, "\1\30436929 47
imap(local_u...@domain.com)<36929>: Mailbox
INBOX.Trash: Deleting corrupted cache record uid=70755: UID 70755:
Broken physical size in mailbox INBOX.Trash: read(compress(/home/ma"...,
389) = 389
No crash, no core, no backtrace.
That's what I found so far. I have no idea how the S937 made it into the
uidlist file. The email itself seems to be uploaded (not imap copied)
through Outlook, which mangled the received headers in the process.
Don't know if that is of any significance, but they are no longer in
chronologically descending order.
Please not that in the config zlib is off for lmtp to reduce the impact
of this issue. It was on when this mail was delivered.
I will be more than happy to provide further information.
Thanks
Tim
Am 02.02.23 um 16:23 schrieb Aki Tuomi:
On 02/02/2023 17:19 EET Stuart Henderson wrote:
On 2023-02-01, Tim Evers wrote:
I run a fairly large Dovecot Installation (around 100k mailboxes) on
sever
Re: Corrupted sizes in cache once again
Maybe I was a bit unclear: I have about 1000 error messages per day from random accounts (about 500 in total so far) on all clusters. These are transparent to the user, so it's more like background noise at the moment. No VM involved. All machines are baremetal DRBD two-node clusters. As far as I see it I can not nail it down to specific accounts, POP3 vs. IMAP, LMTP delivery vs. IMAP store or Sieve vs. non-Sieve etc. Tim Am 02.02.23 um 17:55 schrieb Christopher Wensink: Can you isolate the problem account on a separate VM to see if the problem follows the account or the original vm? Chris On 2/2/2023 9:58 AM, Tim Evers wrote: Good point - these are 8 diferrent DRBD clusters. I failed over one testing this theory. Problem persists. So I would rule out underlying issues. Especially since the "wrong" value is suspiciously often the on-disk size rather than a random value one would expect if there is corruption underneath. Tim Am 02.02.23 um 16:43 schrieb Christopher Wensink: Something to try, this all could be happening because of underlying disk failure on the array it is running on. If this is a VM, can you move the operation to another host or data store to rule out hardware issues? On 2/2/2023 9:19 AM, Stuart Henderson wrote: On 2023-02-01, Tim Evers wrote: I run a fairly large Dovecot Installation (around 100k mailboxes) on several servers. gzip compression is on. Every once in a while I get the dreaded "cache corruption" messages in the log: Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,)) failed: Cached message size smaller than expected (2877 < 8099, box=INBOX, UID=3868) Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,)) failed: Cached message size smaller than expected (5533 < 8192, box=INBOX, UID=3875) The first entry shows 2877 (size on disk) vs. 8099 (real size unzipped, also in the filename: S=8099). The second entry shows 5533 (size on disk) vs. 8192 - this is not correct in any way. Size on disk is 13907 as noted in the filename. Both mails were delivered trough LMTP and retrieved by the POP3 service. Anyone with an idea what might be happening here? I've read all available info in the doc and in the previous discussions / bug reports, but nothing seems to match my case. And where does that 8192 come from - it looks suspicious? Version is 2.3.7.2 (Ubuntu 20.04) 2.3.7.2 is rather old now. There were definitely fixes regarding compression around the 2.3.10-2.3.12 timeframe or thereabouts (I forget all the details but it took a release or two before some remaining issues were sorted out after changes in the area). I'd be looking to get it updated to a current version first.
Re: Corrupted sizes in cache once again
Good point - these are 8 diferrent DRBD clusters. I failed over one testing this theory. Problem persists. So I would rule out underlying issues. Especially since the "wrong" value is suspiciously often the on-disk size rather than a random value one would expect if there is corruption underneath. Tim Am 02.02.23 um 16:43 schrieb Christopher Wensink: Something to try, this all could be happening because of underlying disk failure on the array it is running on. If this is a VM, can you move the operation to another host or data store to rule out hardware issues? On 2/2/2023 9:19 AM, Stuart Henderson wrote: On 2023-02-01, Tim Evers wrote: I run a fairly large Dovecot Installation (around 100k mailboxes) on several servers. gzip compression is on. Every once in a while I get the dreaded "cache corruption" messages in the log: Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,)) failed: Cached message size smaller than expected (2877 < 8099, box=INBOX, UID=3868) Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,)) failed: Cached message size smaller than expected (5533 < 8192, box=INBOX, UID=3875) The first entry shows 2877 (size on disk) vs. 8099 (real size unzipped, also in the filename: S=8099). The second entry shows 5533 (size on disk) vs. 8192 - this is not correct in any way. Size on disk is 13907 as noted in the filename. Both mails were delivered trough LMTP and retrieved by the POP3 service. Anyone with an idea what might be happening here? I've read all available info in the doc and in the previous discussions / bug reports, but nothing seems to match my case. And where does that 8192 come from - it looks suspicious? Version is 2.3.7.2 (Ubuntu 20.04) 2.3.7.2 is rather old now. There were definitely fixes regarding compression around the 2.3.10-2.3.12 timeframe or thereabouts (I forget all the details but it took a release or two before some remaining issues were sorted out after changes in the area). I'd be looking to get it updated to a current version first.
Re: Corrupted sizes in cache once again
Am 02.02.23 um 16:23 schrieb Aki Tuomi: On 02/02/2023 17:19 EET Stuart Henderson wrote: On 2023-02-01, Tim Evers wrote: I run a fairly large Dovecot Installation (around 100k mailboxes) on several servers. gzip compression is on. Every once in a while I get the dreaded "cache corruption" messages in the log: Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,)) failed: Cached message size smaller than expected (2877 < 8099, box=INBOX, UID=3868) Error: Corrupted record in index cache file /[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical size in mailbox INBOX: read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,)) failed: Cached message size smaller than expected (5533 < 8192, box=INBOX, UID=3875) The first entry shows 2877 (size on disk) vs. 8099 (real size unzipped, also in the filename: S=8099). The second entry shows 5533 (size on disk) vs. 8192 - this is not correct in any way. Size on disk is 13907 as noted in the filename. Both mails were delivered trough LMTP and retrieved by the POP3 service. Anyone with an idea what might be happening here? I've read all available info in the doc and in the previous discussions / bug reports, but nothing seems to match my case. And where does that 8192 come from - it looks suspicious? Version is 2.3.7.2 (Ubuntu 20.04) 2.3.7.2 is rather old now. There were definitely fixes regarding compression around the 2.3.10-2.3.12 timeframe or thereabouts (I forget all the details but it took a release or two before some remaining issues were sorted out after changes in the area). I'd be looking to get it updated to a current version first. For bug reports, we do ask that you try to reproduce it with 2.3.20 (current latest), you can get packages from https://repo.dovecot.org/ and would be nice if you can provide steps to reproduce this issue. Also please see https://www.dovecot.org/bugreport-mail/ if you have not yet. Aki This is not a bug report (yet) - I am asking for ideas to narrow down the issue myself. I would not expect something this obvious and prevalent being a bug in a 10+ years old subsystem (zlib). Tim
Re: Corrupted sizes in cache once again
I would like to add some more observations:
I hav another mailbox with one mail's size being saved wrong in the
cache file:
doveadm dump:
RECORD: seq=40, uid=1202, flags=0x00
- ext 5 cache : 8504 (3821)
- ext 6 vsize : 145625 (d9380200)
: vsize = 5166845802713
: highest_uid = 0
: message_count = 8716
- cache offset=8504 size=212, prev_offset = 0
- hdr.From: 42: From: [redacted]
- hdr.Message-ID: 45: Message-ID: [redacted]
- hdr.Subject: 44: Subject: [redacted]
- flags: (2000)
RECORD: seq=41, uid=1203, flags=0x00
- ext 5 cache : 8716 (0c22)
- ext 6 vsize : 105141 (b59a0100)
: vsize = 105141
: highest_uid = 0
: message_count = 0
- cache offset=8716 size=280, prev_offset = 0
- hdr.From: 104: From: [redacted]
- hdr.Message-ID: 108: Message-ID: [redacted]
- hdr.Subject: 105: Subject: [redacted]
- flags: (2000)
The message with seq=40 is the mail dovecot complains about:
Error: Corrupted record in index cache file
/[redacted]/Maildir/dovecot.index.cache: UID 1202: Broken physical size
in mailbox INBOX:
read(zlib(/[redacted]/Maildir/new/1675176154.M829333P14495.node2,S=232816,W=236045))
failed: Cached message size smaller than expected (145082 < 146788,
box=INBOX, UID=1202)
Size on disk is 145082
Size after decompression is 232816 (as expected)
Message with seq=41 is correct with both vsize in cache and decompressed
size being equal.
Before the POP3 access that produced the error both mails were only ever
touched by the dovecot lmtp delivery process.
Am 01.02.23 um 14:13 schrieb Tim Evers:
Hi,
I run a fairly large Dovecot Installation (around 100k mailboxes) on
several servers.
gzip compression is on.
Every once in a while I get the dreaded "cache corruption" messages in
the log:
Error: Corrupted record in index cache file
/[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical
size in mailbox INBOX:
read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,))
failed: Cached message size smaller than expected (2877 < 8099,
box=INBOX, UID=3868)
Error: Corrupted record in index cache file
/[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical
size in mailbox INBOX:
read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,))
failed: Cached message size smaller than expected (5533 < 8192,
box=INBOX, UID=3875)
The first entry shows 2877 (size on disk) vs. 8099 (real size
unzipped, also in the filename: S=8099).
The second entry shows 5533 (size on disk) vs. 8192 - this is not
correct in any way. Size on disk is 13907 as noted in the filename.
Both mails were delivered trough LMTP and retrieved by the POP3 service.
Anyone with an idea what might be happening here? I've read all
available info in the doc and in the previous discussions / bug
reports, but nothing seems to match my case. And where does that 8192
come from - it looks suspicious?
Version is 2.3.7.2 (Ubuntu 20.04)
doveconf -n excerpt:
protocol lmtp {
mail_plugins = " old_stats mail_log notify zlib sieve quota"
}
protocol imap {
mail_max_userip_connections = 50
mail_plugins = " old_stats mail_log notify zlib imap_old_stats quota
imap_quota imap_sieve"
}
protocol pop3 {
mail_max_userip_connections = 50
mail_plugins = " old_stats mail_log notify zlib"
}
Tim
Corrupted sizes in cache once again
Hi,
I run a fairly large Dovecot Installation (around 100k mailboxes) on
several servers.
gzip compression is on.
Every once in a while I get the dreaded "cache corruption" messages in
the log:
Error: Corrupted record in index cache file
/[redacted]/Maildir/dovecot.index.cache: UID 3868: Broken physical size
in mailbox INBOX:
read(zlib(/[redacted]/Maildir/cur/1674129792.M797543P21755.node2,S=8099,W=8276:2,))
failed: Cached message size smaller than expected (2877 < 8099,
box=INBOX, UID=3868)
Error: Corrupted record in index cache file
/[redacted]/Maildir/dovecot.index.cache: UID 3875: Broken physical size
in mailbox INBOX:
read(zlib(/[redacted]/Maildir/cur/1674212201.M985809P29112.node2,S=13907,W=14121:2,))
failed: Cached message size smaller than expected (5533 < 8192,
box=INBOX, UID=3875)
The first entry shows 2877 (size on disk) vs. 8099 (real size unzipped,
also in the filename: S=8099).
The second entry shows 5533 (size on disk) vs. 8192 - this is not
correct in any way. Size on disk is 13907 as noted in the filename.
Both mails were delivered trough LMTP and retrieved by the POP3 service.
Anyone with an idea what might be happening here? I've read all
available info in the doc and in the previous discussions / bug reports,
but nothing seems to match my case. And where does that 8192 come from -
it looks suspicious?
Version is 2.3.7.2 (Ubuntu 20.04)
doveconf -n excerpt:
protocol lmtp {
mail_plugins = " old_stats mail_log notify zlib sieve quota"
}
protocol imap {
mail_max_userip_connections = 50
mail_plugins = " old_stats mail_log notify zlib imap_old_stats quota
imap_quota imap_sieve"
}
protocol pop3 {
mail_max_userip_connections = 50
mail_plugins = " old_stats mail_log notify zlib"
}
Tim
Environment Variable Expansion
System: Debian Jessie
I'm trying to put an environment variable in the sql config like:
password_query = SELECT userid AS user, crypt AS password, maildir as
userdb_home, 500 as userdb_uid, 500 as userdb_gid FROM local_account
WHERE userid = '%u' and mbox_host = '%{env:CLUSTERNAME}' \
and ( ( imap_aktiv='1' and '%s'='imap' ) or ( pop_aktiv='1' and
'%s'='pop3' ) or ( sieve_aktiv='1' and '%s'='sieve' ) ) \
and aktiv_abruf='1'
My variable is
%{env:CLUSTERNAME}
CLUSTERNAME is set through /etc/default/dovecot. I tried setting the
import_environment config variable to
TZ CLUSTERNAME
but that changed nothing.
When I execute this through a login attempt, I see the following in the
tcpflow output:
SELECT userid AS user, crypt AS password, maildir as userdb_home, 508 as
userdb_uid, 503 as userdb_gid FROM local_account WHERE userid = 'te' and
mbox_host = 'env:CLUSTERNAME}' and ( ( imap_aktiv='1' and 'pop3'='imap'
) or ( pop_aktiv='1' and 'pop3'='pop3' ) or ( sieve_aktiv='1' and
'pop3'='sieve' ) ) and aktiv_abruf='1'
So
%{env:CLUSTERNAME}
was parsed to
env:CLUSTERNAME}
which is not what the doc (http://wiki.dovecot.org/Variables) says, and
it also seems like the parser somehow sees this as a short variable
(removing %+{).
Any hints what might be happening here / how I can make it work.
Regards,
Tim
