Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from SOLVED
Hi, in our case, the problem is solved since dovecot 2.2.13. Best regards, Gordon
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 20:10, schrieb Timo Sirainen: Hi, > Well, the question is then.. Why were the corrupted in the first place? Based > on your previous error message it looked like the cache file contained the > compressed size, so maybe zlib plugin wasn't loaded for some Dovecot process > at that time? AFAIK zlib is always on: mailserver2:~# doveconf | grep plugins mail_plugins = quota acl mail_log notify zlib mail_plugins = quota acl mail_log notify zlib sieve mail_plugins = quota acl mail_log notify zlib imap_quota imap_acl mail_plugins = quota acl mail_log notify zlib I'll try to create and send you a test-case with an infected maildir. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 27.3.2013, at 18.41, Peer Heinlein wrote: > Am 27.03.2013 08:44, schrieb Timo Sirainen: > >> Oh, except I actually forgot to load zlib plugin in my previous test. > I can't reproduce with these steps.. and I don't really see why they > would cause it anyway. A broken cached size would cause that rename, but > not a missing cached size. > > Thats why I wrote from STEP ONE and from STEP TWO. > > I had the impression, that Dovecot first rebuilds his cache with WRONG > sizes and THEN starts in step two the renaming. Well, the question is then.. Why were the corrupted in the first place? Based on your previous error message it looked like the cache file contained the compressed size, so maybe zlib plugin wasn't loaded for some Dovecot process at that time? Anyway, yeah, I guess there are two potential improvements here: Either a) don't rename maildir file if S=size is different from cached size or b) rename the S=size to the correct decompressed size (=no renaming if it's correct). Not sure which one is easier to do, possibly b) and possibly it should be done anyway. In any case, I think this is a good addition: http://hg.dovecot.org/dovecot-2.2/rev/6d9444ea1c9a
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 17:51, schrieb Peer Heinlein: > Am 25.03.2013 15:27, schrieb Robert Schetterer: > > >> please reread the list archive ,solutions where massive posted >> and a new repair script was created > > We did that before, for sure. > > But there are NO working solutions for that and the problem still exists > and ist a massive problem, because a simple version upgrade doesn't work > and leads to a DOS of the infected systems. > > The repair script hasn't worked at all with our kind of Maildir-Filenames. > > If others run into the same problem: > > We used this simple piece of code (which is much easier to read and adapt): > > for FILE in * ; do > OLDNAME=$FILE > SIZE=`zcat $FILE | wc -c` > NEWNAME=`echo $FILE | sed "s/\(.*\)S=.*:\(.*\)/\1S=$SIZE:\2/g"` > > if [ ! $OLDNAME = $NEWNAME ] ; then > echo mv "$OLDNAME""$NEWNAME" > fi > done > > > Peer > > Hi Peer , as talked to Gordon, this was a total upgrade from 2.0.x to 2.1.x and you converted all maildir to compressed before, right ? posting some conf parameters might be helpfull , did you investigated broken maildirs for mixing compressed and uncompressed mails exist, as i understand Gordon there should be only compressed ? Have you checked about double compressed mails in broken maildirs ? As wrote before, at my bug time after repair with script and upgrading dovecot, failures had gone and never returned, but my setup may be different from yours and failure did happend sometime upgrade dove 2.1.x not at migrate from dove 2.0.x, did you changed something other too ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 25.03.2013 15:27, schrieb Robert Schetterer: > please reread the list archive ,solutions where massive posted > and a new repair script was created We did that before, for sure. But there are NO working solutions for that and the problem still exists and ist a massive problem, because a simple version upgrade doesn't work and leads to a DOS of the infected systems. The repair script hasn't worked at all with our kind of Maildir-Filenames. If others run into the same problem: We used this simple piece of code (which is much easier to read and adapt): for FILE in * ; do OLDNAME=$FILE SIZE=`zcat $FILE | wc -c` NEWNAME=`echo $FILE | sed "s/\(.*\)S=.*:\(.*\)/\1S=$SIZE:\2/g"` if [ ! $OLDNAME = $NEWNAME ] ; then echo mv "$OLDNAME""$NEWNAME" fi done Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 11:19, schrieb Gordon Grubert: >>> We made a simple downgrade to 2.0.21 and now everything's working >>> perfect. >> >> maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But >> yeah, looks like there's a bug. > > No, it does not. In Timo's first mail he wrote maildir_broken_fileNAME_sizes and we used that (which didn't helped at all). mailserver2:~/dovecot.neu.2-1# grep -r maildir_broken * conf.d/10-mail.conf:maildir_broken_filename_sizes = yes mailserver2:~/dovecot.neu.2-1# But maybe it's working with "maildir_broken_file_sizes" :-), we can test that on our test-system. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 08:44, schrieb Timo Sirainen: > Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size. Thats why I wrote from STEP ONE and from STEP TWO. I had the impression, that Dovecot first rebuilds his cache with WRONG sizes and THEN starts in step two the renaming. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 11:55, schrieb Gordon Grubert: > On 03/27/2013 11:48 AM, Robert Schetterer wrote: >> agree, when i first run at that problem >> maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs >> manual by script, upgraded dovecot 2.1.x to newer version, that problem >> never came back again, just for interest what dovecot source did you >> use, did you you compile modifications by your own ? > > I used the debian binary package for dovecot 2.1.15 from > xi.rename-it.nl did you changed anything in your config too, using or changing other features too , while upgrade ? Did you modifications to the sources ( debian rules etc ) and recompile i.e integrate lucene etc ? > > Best regards, > Gordon > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 03/27/2013 11:48 AM, Robert Schetterer wrote: agree, when i first run at that problem maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs manual by script, upgraded dovecot 2.1.x to newer version, that problem never came back again, just for interest what dovecot source did you use, did you you compile modifications by your own ? I used the debian binary package for dovecot 2.1.15 from xi.rename-it.nl Best regards, Gordon
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 11:19, schrieb Gordon Grubert: > On 03/27/2013 08:38 AM, Timo Sirainen wrote: >> On 27.3.2013, at 9.34, Peer Heinlein >> wrote: >> >>> Am 26.03.2013 23:15, schrieb Peer Heinlein: >>> If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files. >>> >>> More and more users complained this morning about broken mailboxes and >>> our logfile was full of errors. >>> >>> We made a simple downgrade to 2.0.21 and now everything's working >>> perfect. >> >> maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But >> yeah, looks like there's a bug. > > No, it does not. > > Best regards, > Gordon > > agree, when i first run at that problem maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs manual by script, upgraded dovecot 2.1.x to newer version, that problem never came back again, just for interest what dovecot source did you use, did you you compile modifications by your own ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 03/27/2013 08:38 AM, Timo Sirainen wrote: On 27.3.2013, at 9.34, Peer Heinlein wrote: Am 26.03.2013 23:15, schrieb Peer Heinlein: If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files. More and more users complained this morning about broken mailboxes and our logfile was full of errors. We made a simple downgrade to 2.0.21 and now everything's working perfect. maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug. No, it does not. Best regards, Gordon smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 27.03.2013 08:44, schrieb Timo Sirainen: > On 27.3.2013, at 0.15, Peer Heinlein wrote: > >> Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached >> message size smaller than expe >> cted (1467 < 3780) >> >> Error: Maildir filename has wrong S value, r >> enamed the file from >> /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m >> ailserver2,S=3780,W=3860:2, to >> /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 >> 13P23361.mailserver2,S=1856:2, >> >> >> HOW TO REPRODUCE: >> >> *) Create a Maildir-Store with zip enabled guess you mean zlib ? >> *) Deliver Mails into it. Everything's working fine, the filenames are right >> *) Delete dovecot.index* in fact i did this 2 weeks ago , no errors came up with 2.1.15, maildirs/mailboxes got work again >> *) STEP ONE: Dovecot's complaining about broken index-files >> *) STEP TWO: Dovecot's renaming the files > > Oh, except I actually forgot to load zlib plugin in my previous test. I can't > reproduce with these steps.. and I don't really see why they would cause it > anyway. A broken cached size would cause that rename, but not a missing > cached size. > my problem was ,i couldnt find out why i needed to delete index* to get 2 Mailboxes work again,for more magic, no problem in the logs and mailboxes worked in thunderbird linux but not in thunderbird windows ( clean new setups ),i speculated to some problem with massive pop3 and imap in parallel from different ip at same time to the same mailbox via loadbalancers crashing something, but sadly couldnt reproduce it yet Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 27.3.2013, at 0.15, Peer Heinlein wrote: > Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached > message size smaller than expe > cted (1467 < 3780) > > Error: Maildir filename has wrong S value, r > enamed the file from > /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m > ailserver2,S=3780,W=3860:2, to > /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 > 13P23361.mailserver2,S=1856:2, > > > HOW TO REPRODUCE: > > *) Create a Maildir-Store with zip enabled > *) Deliver Mails into it. Everything's working fine, the filenames are right > *) Delete dovecot.index* > *) STEP ONE: Dovecot's complaining about broken index-files > *) STEP TWO: Dovecot's renaming the files Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 27.3.2013, at 9.34, Peer Heinlein wrote: > Am 26.03.2013 23:15, schrieb Peer Heinlein: > >> If we're right, this could be grow to a real problem. Every Server with >> zipped Maildirs can be completly ruined just by deleting his >> index-cache-files. > > More and more users complained this morning about broken mailboxes and > our logfile was full of errors. > > We made a simple downgrade to 2.0.21 and now everything's working perfect. maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 26.03.2013 23:15, schrieb Peer Heinlein: > If we're right, this could be grow to a real problem. Every Server with > zipped Maildirs can be completly ruined just by deleting his > index-cache-files. More and more users complained this morning about broken mailboxes and our logfile was full of errors. We made a simple downgrade to 2.0.21 and now everything's working perfect. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 25.03.2013 15:38, schrieb Timo Sirainen: Hi, >> The same problem was reported by Ralf Hildebrandt one year ago. The bug >> should be fixed with revision 3599790da3d7 but it seems to be there >> again. > > The Dovecot bug was fixed, but the real reason for this is that the S=values > are wrong in your maildir. You can either run the fixing script or set > maildir_broken_filename_sizes=yes. Looks like this (or a related) bug still exist. If you have a Maildir-Storage with gzip compression enabled, everything's working fine if the user receives mail by LMTP. The mail is saved in his Maildir-Storage, having the right (uncompressed) size in the filename. vmail vmail 1.9K Mar 26 22:17 1364332643.M527513P23361.mailserver2,S=3780,W=3860:2 But: If the dovecot.index is broken, corrupt or deleted, Dovecot isn't able to rebuild his index-files. In Step ONE dovecot creates his index-files, but looks like Dovecot's using the (smaller) FILEsize instead of the (larger) real size. In Step TWO Dovecot's realizing that the cached size and the stored file size in the filename doesn't fit together. But Dovecot doesn't fix his index file; instead Dovecot's renaming the Maildir-Files, storing the (small) file size in the filename. Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached message size smaller than expe cted (1467 < 3780) Error: Maildir filename has wrong S value, r enamed the file from /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m ailserver2,S=3780,W=3860:2, to /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 13P23361.mailserver2,S=1856:2, HOW TO REPRODUCE: *) Create a Maildir-Store with zip enabled *) Deliver Mails into it. Everything's working fine, the filenames are right *) Delete dovecot.index* *) STEP ONE: Dovecot's complaining about broken index-files *) STEP TWO: Dovecot's renaming the files I haven't seen any way to find to workaround or repair this broken Maildir-Storage. Even if I rename all files and set sizes in the filenames, Dovecot's complaining about the mismatch in his cache and starts his (broken) repair action. If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 25.3.2013, at 16.22, Gordon Grubert wrote: > Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=, > method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session= > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size > smaller than expected (2252 < 4821) > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has > wrong S value, renamed the file from > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS > to > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS .. > The same problem was reported by Ralf Hildebrandt one year ago. The bug > should be fixed with revision 3599790da3d7 but it seems to be there > again. The Dovecot bug was fixed, but the real reason for this is that the S=values are wrong in your maildir. You can either run the fixing script or set maildir_broken_filename_sizes=yes. > Input/output errors of the file system are improbable because all > files are accessible and can be read with cat and less. That's just Dovecot's internal way of passing a failure to other parts of the code.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Am 25.03.2013 15:22, schrieb Gordon Grubert: > Dear list, > > we're using dovecot 2.1.15 (debian binary package). The following > error can be found in the mail log files: > > Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=, > method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session= > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message > size smaller than expected (2252 < 4821) > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename > has wrong S value, renamed the file from > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS > to > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS > > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index > cache file > /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken > physical size for mail UID 25250 > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message > size smaller than expected (2252 < 4821) > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename > has wrong S value, renamed the file from > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS > to > /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS > > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index > cache file > /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken > physical size for mail UID 25250 > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: > read(/var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS) > failed: Input/output error (FETCH for mailbox INBOX UID 25250) > Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Disconnected: Internal > error occurred. Refer to server log for more information. [2013-03-25 > 15:08:46] in=246 out=953 > > The same problem was reported by Ralf Hildebrandt one year ago. The bug > should be fixed with revision 3599790da3d7 but it seems to be there > again. > > Input/output errors of the file system are improbable because all > files are accessible and can be read with cat and less. > > Any ideas? > > Best regards, > Gordon please reread the list archive ,solutions where massive posted and a new repair script was created Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Dear list, we're using dovecot 2.1.15 (debian binary package). The following error can be found in the mail log files: Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=, method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session= Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: read(/var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS) failed: Input/output error (FETCH for mailbox INBOX UID 25250) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Disconnected: Internal error occurred. Refer to server log for more information. [2013-03-25 15:08:46] in=246 out=953 The same problem was reported by Ralf Hildebrandt one year ago. The bug should be fixed with revision 3599790da3d7 but it seems to be there again. Input/output errors of the file system are improbable because all files are accessible and can be read with cat and less. Any ideas? Best regards, Gordon -- Leiter AG Technische Infrastruktur und Basisdienste Universitaetsrechenzentrum (URZ) E.-M.-Arndt-Universitaet Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86-1456 Fax. +49 3834 86-1401 smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Timo Sirainen : > > Right now the whole system is down because nobody can acces his/her > > mails due to this. > > All of your mails are compressed and have wrong S=size in the filename? You > can disable the check with the attached patch, but I'm not sure if there are > other places where it fails. At least quota calculations won't be correct. That patch totally saved my ass. I rolled it out today and the Mar 22 09:33:00 postamt dovecot: imap(stoffelm): Error: Maildir filename has wrong S value, renamed the file from /home/s/t/stoffelm/Maildir/.Deleted Messages/cur/1331891533.M93099P19536.postamt.charite.de,S=1860:2,Scd to /home/s/t/stoffelm/Maildir/.Deleted Messages/cur/1331891533.M93099P19536.postamt.charite.de,S=1860:2,Scd errors subsided. At the same time the users CAN access the affected folder. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Quoting Timo Sirainen :
On 21.3.2012, at 17.52, Ralf Hildebrandt wrote:
* Timo Sirainen :
It's renaming itself to itself again?
Hmm. Yeah, this is a bit problematic for compressed mails. If the
S=size isn't correct, Dovecot fixes it by stat()ing the file and using
it as the size. And that's of course wrong. Also Dovecot can't simply
remove the S=size, because the current Maildir code assumes that it
always exists for compressed mails. There's no easy and efficient way
to fix this.. Maybe you could just manually rename the files to have
correct S=size? :) zcat file | wc should give the right size.
Right now the whole system is down because nobody can acces his/her
mails due to this.
All of your mails are compressed and have wrong S=size in the
filename? You can disable the check with the attached patch, but I'm
not sure if there are other places where it fails. At least quota
calculations won't be correct.
The issue only started happening since I upgraded to 2.1.1, it didn't
exist before then, I have check my system, and files before the date
of upgrade are fine, only files/emails moved after upgrading to 2.1.1
have lost the S= value.
I have made something that can pretty easily fix the issue, but it
only stays fixed till another email gets moved and looses it's S= value.
Sorry, I haven't had time to test out 2.1.3 yet.
This will print out the commands needed to fix the files though.
find . -name '*hostname:*' -exec 'gzip' '-l' '{}' ';' | awk
'/hostname/ {for(x=4;xsplit(fn,a,":"); print "mv \"" fn "\" \"" a[1] ",S=" $2 ":" a[2] "\"";}'
Just change hostname to dovecot is using in your files.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 21.3.2012, at 17.52, Ralf Hildebrandt wrote: > * Timo Sirainen : > >>> It's renaming itself to itself again? >> >> Hmm. Yeah, this is a bit problematic for compressed mails. If the >> S=size isn't correct, Dovecot fixes it by stat()ing the file and using >> it as the size. And that's of course wrong. Also Dovecot can't simply >> remove the S=size, because the current Maildir code assumes that it >> always exists for compressed mails. There's no easy and efficient way >> to fix this.. Maybe you could just manually rename the files to have >> correct S=size? :) zcat file | wc should give the right size. > > Right now the whole system is down because nobody can acces his/her > mails due to this. All of your mails are compressed and have wrong S=size in the filename? You can disable the check with the attached patch, but I'm not sure if there are other places where it fails. At least quota calculations won't be correct. diff Description: Binary data
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Timo Sirainen : > > It's renaming itself to itself again? > > Hmm. Yeah, this is a bit problematic for compressed mails. If the > S=size isn't correct, Dovecot fixes it by stat()ing the file and using > it as the size. And that's of course wrong. Also Dovecot can't simply > remove the S=size, because the current Maildir code assumes that it > always exists for compressed mails. There's no easy and efficient way > to fix this.. Maybe you could just manually rename the files to have > correct S=size? :) zcat file | wc should give the right size. Right now the whole system is down because nobody can acces his/her mails due to this. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 21.3.2012, at 16.33, Ralf Hildebrandt wrote: >>> but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S >>> and W tags. >>> >>> 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa >>> >>> This is happening for all folder moves. >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7 > > That doesn't seem to work: It fixed only the duplicate S= and W= values. > Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: Maildir filename > has wrong S value, renamed the file from > /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S > to > /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S > Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: > read(/home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S) > failed: Input/output error (uid=5270) > > It's renaming itself to itself again? Hmm. Yeah, this is a bit problematic for compressed mails. If the S=size isn't correct, Dovecot fixes it by stat()ing the file and using it as the size. And that's of course wrong. Also Dovecot can't simply remove the S=size, because the current Maildir code assumes that it always exists for compressed mails. There's no easy and efficient way to fix this.. Maybe you could just manually rename the files to have correct S=size? :) zcat file | wc should give the right size.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Timo Sirainen : > On 20.3.2012, at 16.55, Patrick Domack wrote: > > > but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S > > and W tags. > > > > 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa > > > > This is happening for all folder moves. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7 That doesn't seem to work: Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: Maildir filename has wrong S value, renamed the file from /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S to /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: read(/home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S) failed: Input/output error (uid=5270) It's renaming itself to itself again? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Thanks, applied it to 2.1.3 and going to test. You didn't even give me enough time to look at the source myself to find the issue. Quoting Timo Sirainen : On 20.3.2012, at 16.55, Patrick Domack wrote: but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S and W tags. 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa This is happening for all folder moves. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 20.3.2012, at 16.55, Patrick Domack wrote: > but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S and > W tags. > > 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa > > This is happening for all folder moves. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Patrick Domack : > And found two more users with this issue, but while looking at it, I > see another related issue, but it's not throwing an error. > > all email in the INBOX/new and /cur are correct > > but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have > double S and W tags. > > 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa > > This is happening for all folder moves. Yes indeed: postamt:/home/h/a/happel/Maildir/.Trash/cur# ll total 16 -rw--- 1 happel users 7541 Mar 20 15:23 1332253428.M342974P5666.postamt.charite.de,S=37641,W=38197,S=37641,W=38197:2,Se -rw--- 1 happel users 6378 Mar 20 15:42 1332254568.M9552P591.postamt.charite.de,S=27486,W=28188,S=27486,W=28188:2,Se -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
And found two more users with this issue, but while looking at it, I see another related issue, but it's not throwing an error. all email in the INBOX/new and /cur are correct but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S and W tags. 1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa This is happening for all folder moves. the Sent folder isn't affected, but I assume cause an email wasn't moved in that case. Quoting Ralf Hildebrandt : * Patrick Domack : I'm having this problem also, with a very very few users. But in my case the email isn't double gzip, just single like normal. Error: read(.../.Deleted Messages/cur/1331840112.M186676P27974.5013:2,) failed: Input/output error (uid=250) All I have to do is rename the file to add back the lost S= part and all is fine. This has happened in the inbox, deleted, and trash folders so far. and always after a change, the S= exists for new emails. It's like it's loosing it on adding the read flag, and mailbox moves Yes, I'm also seeing it now with mailboxes where no mail is doubly gzipped. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Patrick Domack : > I'm having this problem also, with a very very few users. > > But in my case the email isn't double gzip, just single like normal. > > Error: read(.../.Deleted > Messages/cur/1331840112.M186676P27974.5013:2,) failed: Input/output > error (uid=250) > > All I have to do is rename the file to add back the lost S= part and > all is fine. > This has happened in the inbox, deleted, and trash folders so far. > and always after a change, the S= exists for new emails. It's like > it's loosing it on adding the read flag, and mailbox moves Yes, I'm also seeing it now with mailboxes where no mail is doubly gzipped. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
I'm having this problem also, with a very very few users. But in my case the email isn't double gzip, just single like normal. Error: read(.../.Deleted Messages/cur/1331840112.M186676P27974.5013:2,) failed: Input/output error (uid=250) All I have to do is rename the file to add back the lost S= part and all is fine. This has happened in the inbox, deleted, and trash folders so far. and always after a change, the S= exists for new emails. It's like it's loosing it on adding the read flag, and mailbox moves But out of millions of emails, only a very few are like this, that I know of, around 6 emails. I manually fixed them, will be looking to see if this issue comes back. Quoting Timo Sirainen : On 2.3.2012, at 12.43, Ralf Hildebrandt wrote: Alternatively you can just tell Dovecot not to care about it: maildir_broken_filename_sizes=yes. Although you probably can't do that if you have compressed mails. In the case above that mail was gzipped twice :( Yes, looks like Dovecot can't correctly fix the wrong S size for gzipped mails. I don't know if I should bother fixing it, especially since in your case the doubly-gzipped mails will look corrupted to user..
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 2.3.2012, at 12.43, Ralf Hildebrandt wrote: >> Alternatively you can just tell Dovecot not to care about it: >> maildir_broken_filename_sizes=yes. Although you probably can't do that if >> you have compressed mails. > > In the case above that mail was gzipped twice :( Yes, looks like Dovecot can't correctly fix the wrong S size for gzipped mails. I don't know if I should bother fixing it, especially since in your case the doubly-gzipped mails will look corrupted to user..
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
* Timo Sirainen : > On 2.3.2012, at 12.25, Ralf Hildebrandt wrote: > > > Starting with 2.1.1 we suddely encounter quite a lot of these messages: > > > > Mar 2 11:09:28 postamt dovecot: imap(username): Error: Maildir filename > > has wrong S value, renamed the file from > > /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden > > etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S to > > /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden > > etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S > .. > > While this has (assumedly) been working with 2.0.18. > > Dovecot v2.0 didn't detect these problems, and might have truncated some > mails in some situations. COuld be! > > Another issue > > with this: This fixes ONE file, and throws an error. Repeatedly > > accessing this folder fixes more files, until at some point all files > > were fixed. > > Right, because after it notices a problem it disconnects the client since it > can't really do anything else. Running doveadm fetch for all the mails should > fix all of them. Ah yes, good idea Mar 2 11:39:39 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=141.42.206.38, lip=141.42.206.36, mpid=28959, secured Mar 2 11:39:41 postamt dovecot: imap(user): Error: Cached message size smaller than expected (168202 < 170440) Mar 2 11:39:41 postamt dovecot: imap(user): Error: Maildir filename has wrong S value, renamed the file from /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ to /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ Mar 2 11:39:41 postamt dovecot: imap(user): Error: Corrupted index cache file /home/g/z/user/Maildir/.Partys/dovecot.index.cache: Broken physical size for mail UID 81 Mar 2 11:39:41 postamt dovecot: imap(user): Error: read(/home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ) failed: Input/output error (uid=81) Mar 2 11:39:41 postamt dovecot: imap(user): Disconnected: Internal error occurred. Refer to server log for more information. [2012-03-02 11:39:41] in=735 out=5258 Look at that renaming operation: It simply reused the same name: from /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ to /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ > Alternatively you can just tell Dovecot not to care about it: > maildir_broken_filename_sizes=yes. Although you probably can't do that if you > have compressed mails. In the case above that mail was gzipped twice :( -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
On 2.3.2012, at 12.25, Ralf Hildebrandt wrote: > Starting with 2.1.1 we suddely encounter quite a lot of these messages: > > Mar 2 11:09:28 postamt dovecot: imap(username): Error: Maildir filename has > wrong S value, renamed the file from > /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden > etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S to > /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden > etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S .. > While this has (assumedly) been working with 2.0.18. Dovecot v2.0 didn't detect these problems, and might have truncated some mails in some situations. > Another issue > with this: This fixes ONE file, and throws an error. Repeatedly > accessing this folder fixes more files, until at some point all files > were fixed. Right, because after it notices a problem it disconnects the client since it can't really do anything else. Running doveadm fetch for all the mails should fix all of them. Alternatively you can just tell Dovecot not to care about it: maildir_broken_filename_sizes=yes. Although you probably can't do that if you have compressed mails.
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
> Hello, same problem here after upgrading from 2.0.18 to 2.1.0, > apparently it happens only on servers with qmail, not on servers with > exim or dovecot as lda: I'm using the dovecot LDA, but then it's not clear if the messages affected are REALLY old and thus might predate the use of the dovecot LDA... > > Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Cached message size > smaller than expected (59998 < 60150) > Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Maildir filename > has wrong S value, renamed the file from > /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2, > to > /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=60150:2, > Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Corrupted index > cache file /home/vpopmail/domains/2/root/Maildir/dovecot.index.cache: > Broken physical size for mail UID 40669 > Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: > read(/home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2,) > failed: Input/output error (uid=40669) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] 2.1: Error: Maildir filename has wrong S value, renamed the file from
Il 02/03/2012 11:25, Ralf Hildebrandt ha scritto: Hi! Starting with 2.1.1 we suddely encounter quite a lot of these messages: Mar 2 11:09:28 postamt dovecot: imap(username): Error: Maildir filename has wrong S value, renamed the file from /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S to /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S Mar 2 11:09:28 postamt dovecot: imap(username): Error: Maildir filename has wrong S value, renamed the file from /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S to /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S Mar 2 11:09:28 postamt dovecot: imap(username): Error: read(/home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S) failed: Input/output error (uid=69) While this has (assumedly) been working with 2.0.18. Another issue with this: This fixes ONE file, and throws an error. Repeatedly accessing this folder fixes more files, until at some point all files were fixed. Hello, same problem here after upgrading from 2.0.18 to 2.1.0, apparently it happens only on servers with qmail, not on servers with exim or dovecot as lda: Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Cached message size smaller than expected (59998 < 60150) Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Maildir filename has wrong S value, renamed the file from /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2, to /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=60150:2, Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Corrupted index cache file /home/vpopmail/domains/2/root/Maildir/dovecot.index.cache: Broken physical size for mail UID 40669 Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: read(/home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2,) failed: Input/output error (uid=40669)
