Re: [Dovecot] zlib plugin maildir broken ?/ Cached message size smaller than expected / with 2.1.9 patch ?

[Robert Schetterer]

Am 03.08.2012 15:56, schrieb Timo Sirainen:
> On 2.8.2012, at 10.09, Robert Schetterer wrote:
> 
>> Hi Timo, most of the Problems are fixed now
>> many thx for your script
>>
>> , but how to avoid that in future ( Maildir filename has wrong S value )
> 
> You'd need to find out why it started happening in the first place. 

the problem is/was getting maildirs fixed, no time to debug on
production system

in deed i have no idea why this happend, i found out some broken
were old maildirs, so for them i bet coruption by migration 2.0 -2.1
other cuted out attachments imap online with serveral clients on = user
fault in my meaning
left the cases which i reproduced myself, or some brand new mailboxes
it was about compressed mail and nocompressed with wrong message size in
one folder

What software wrote those filenames? Are new filenames still creating
with wrong S value?
Does your dovecot.conf use zlib plugin globally (instead of only in some
protocol sections, which might cause this kind of trouble)?


10-mail.conf:mail_plugins = $mail_plugins zlib expire virtual fts
fts_lucene mail_log autocreate notify acl listescape
20-imap.conf:  mail_plugins = imap_acl acl fts fts_lucene mail_log
autocreate notify expire virtual quota imap_quota zlib imap_zlib listescape
90-plugin.conf:zlib_save_level = 6
90-plugin.conf:zlib_save = gz

should be ok ?

> 
>> you released a patch
>>
>> diff -r 3599790da3d7 src/lib-storage/index/istream-mail.c
>> --- a/src/lib-storage/index/istream-mail.c   Wed Mar 21 14:25:12 2012 +0200
>> +++ b/src/lib-storage/index/istream-mail.c   Wed Mar 21 17:58:00 2012 +0200
>> @@ -20,6 +20,7 @@
>>  struct mail *mail = mstream->mail;
>>  enum mail_lookup_abort orig_lookup_abort;
>>
>> +return FALSE;
>>  if (mstream->expected_size != (uoff_t)-1)
>>  return TRUE;
>>
>> would it be a good idea to include it in 2.1.9 for me ?
>> as it looks like its not in the public code ?
> 
> It's an ugly workaround for the wrong S=size problem when zlib is used. 
> Preferably you wouldn't ever need to use it.

ok , i fixed all mailboxes with your script and one from ralf with
compressed maildirs, updated to 2.1.9, using lucene now, disabled stats
( for kernel bugs ), and for now all problems are gone

if i cant find any better reason, i go for shit happens, and learned how
to fix it, however it were only 6 maildirs from 200 which got broken
going trough lot of updates in years , so this is ok in my mind

good time for users learn to build some mail archive from time to time
however no mail got lost

> 


-- 
Best Regards
MfG Robert Schetterer

Re: [Dovecot] zlib plugin maildir broken ?/ Cached message size smaller than expected / with 2.1.9 patch ?

[Timo Sirainen]

On 2.8.2012, at 10.09, Robert Schetterer wrote:

> Hi Timo, most of the Problems are fixed now
> many thx for your script
> 
> , but how to avoid that in future ( Maildir filename has wrong S value )

You'd need to find out why it started happening in the first place. What 
software wrote those filenames? Are new filenames still creating with wrong S 
value? Does your dovecot.conf use zlib plugin globally (instead of only in some 
protocol sections, which might cause this kind of trouble)?

> you released a patch
> 
> diff -r 3599790da3d7 src/lib-storage/index/istream-mail.c
> --- a/src/lib-storage/index/istream-mail.cWed Mar 21 14:25:12 2012 +0200
> +++ b/src/lib-storage/index/istream-mail.cWed Mar 21 17:58:00 2012 +0200
> @@ -20,6 +20,7 @@
>   struct mail *mail = mstream->mail;
>   enum mail_lookup_abort orig_lookup_abort;
> 
> + return FALSE;
>   if (mstream->expected_size != (uoff_t)-1)
>   return TRUE;
> 
> would it be a good idea to include it in 2.1.9 for me ?
> as it looks like its not in the public code ?

It's an ugly workaround for the wrong S=size problem when zlib is used. 
Preferably you wouldn't ever need to use it.

Re: [Dovecot] zlib plugin maildir broken ?/ Cached message size smaller than expected / with 2.1.9 patch ?

[Robert Schetterer]

Am 01.08.2012 14:58, schrieb Timo Sirainen:
> On 1.8.2012, at 15.43, Robert Schetterer wrote:
> 
 If you don't mind the messages' IMAP UIDs changing, you could give the -f 
 parameter to fix those.
>>>
>>> i will try things couldn not get more worst
>>
>> ./maildir-size-check.sh -f maildir is an unknown option...?
> 
> Oh, looks like I forgot to re-upload it after adding that option. It's there 
> now.
> 

Hi Timo, most of the Problems are fixed now
many thx for your script

, but how to avoid that in future ( Maildir filename has wrong S value )

with Ralf

you released a patch

diff -r 3599790da3d7 src/lib-storage/index/istream-mail.c
--- a/src/lib-storage/index/istream-mail.c  Wed Mar 21 14:25:12 2012 +0200
+++ b/src/lib-storage/index/istream-mail.c  Wed Mar 21 17:58:00 2012 +0200
@@ -20,6 +20,7 @@
struct mail *mail = mstream->mail;
enum mail_lookup_abort orig_lookup_abort;

+   return FALSE;
if (mstream->expected_size != (uoff_t)-1)
return TRUE;

would it be a good idea to include it in 2.1.9 for me ?
as it looks like its not in the public code ?

-- 
Best Regards
MfG Robert Schetterer