Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-24 Thread Stan Hoeppner
On 12/23/2011 11:08 AM, hydra wrote: Selinux wasn't running. I can reproduce the problem with this setup: - restore the maildir with the broken index.cache - run `/usr/bin/doveadm -D expunge -A mailbox Trash savedbefore 1w` in an infinite loop - use horde webmail to access the mail with

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-23 Thread hydra
Yes, it is reproducible. About the file, I have tarred up the original maildir before I deleted the dovecot.index.cache, so when I try to bring in the original file, a 'cat' does no harm. Moreover, I've tried to do this will all files on the virtual machine, same results - no harm. On Thu, Dec

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-23 Thread Stan Hoeppner
On 12/23/2011 4:12 AM, hydra wrote: Yes, it is reproducible. About the file, I have tarred up the original maildir before I deleted the dovecot.index.cache, so when I try to bring in the original file, a 'cat' does no harm. Moreover, I've tried to do this will all files on the virtual

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-23 Thread hydra
Selinux wasn't running. I can reproduce the problem with this setup: - restore the maildir with the broken index.cache - run `/usr/bin/doveadm -D expunge -A mailbox Trash savedbefore 1w` in an infinite loop - use horde webmail to access the mail with IMAP - randomly display messages I could

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-22 Thread hydra
Hello Timo, thank you for the reply. I was suspecting the same. However: - the machine runs under Vmware, - I've tried 3 different kernel versions, - I've tried 3 different SCSI controllers. All same results. On Wed, Dec 21, 2011 at 6:16 PM, Timo Sirainen t...@iki.fi wrote: On 21.12.2011, at

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-22 Thread Stan Hoeppner
On 12/22/2011 8:08 AM, hydra wrote: Hello Timo, thank you for the reply. I was suspecting the same. However: - the machine runs under Vmware, - I've tried 3 different kernel versions, - I've tried 3 different SCSI controllers. All same results. dmesg output? Log errors? Is your EXT4

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-22 Thread hydra
On Thu, Dec 22, 2011 at 3:49 PM, Stan Hoeppner s...@hardwarefreak.comwrote: On 12/22/2011 8:08 AM, hydra wrote: Hello Timo, thank you for the reply. I was suspecting the same. However: - the machine runs under Vmware, - I've tried 3 different kernel versions, - I've tried 3 different

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-22 Thread Timo Sirainen
On 22.12.2011, at 18.06, hydra wrote: After the virtual machine reboot, the CPU usage is normal again, but just until doveadm is launched again (it was run from cron). Sorry for the partition/drive terminology mess up. So it's reproducible? You could see if you can reproduce it by simply

[Dovecot] Corrupted index cache file and high CPU usage

2011-12-21 Thread hydra
I suspect, that dovecot v2.0.16 under Linux (tested on Gentoo 3.0.4 Hardened, Gentoo 3.0.6, Gentoo 3.1.4) causes 100% cpu utilization when index.cache is broken and doveadm is lanched. Dovecot uses LDAP as the userdb and provides IMAP and LDA. Because each user has a quota, the doveadm's expunge

Re: [Dovecot] Corrupted index cache file and high CPU usage

2011-12-21 Thread Timo Sirainen
On 21.12.2011, at 18.38, hydra wrote: I suspect, that dovecot v2.0.16 under Linux (tested on Gentoo 3.0.4 Hardened, Gentoo 3.0.6, Gentoo 3.1.4) causes 100% cpu utilization when index.cache is broken and doveadm is lanched. Dovecot uses LDAP as the userdb and provides IMAP and LDA. Because

Re: [Dovecot] Corrupted index cache file

2011-05-09 Thread Timo Sirainen
On Fri, 2011-04-15 at 08:32 +0200, Frank Bonnet wrote: Apr 14 19:24:37 imap(xxx): Error: FETCH [] for mailbox Drafts (Hugues at ESIEE) UID 17836 got too little data: 16384 vs 18012 Apr 14 19:24:37 imap(xxx): Error: Corrupted index cache file

Re: [Dovecot] Corrupted index cache file

2011-05-09 Thread Maarten Bezemer
For what it's worth.. I've seen the FETCH error with got too little data also with Maildir setup. So, not only limited to mbox. This was quite some time ago with (I think) 1.0.15. Back then, I just removed the dovecot.index and cache files (leaving only the uidlist file) for that maildir

[Dovecot] Corrupted index cache file

2011-04-15 Thread Frank Bonnet
Hello I got this error message yesterday evening for a user I've checked LDA error log it happened 6 times in 3 days for 5 differents users, not much but I wonder what's going on as actually the server is moderately loaded ( Easter holidays ) we have currently 250/300 IMAP processes , in normal

Re: [Dovecot] Corrupted index cache file

2011-04-15 Thread Charles Marcus
On 2011-04-15 2:32 AM, Frank Bonnet wrote: Dovecot or hardware problem ??? Thanks a lot for any ideas/infos Sorry, crystal ball is broken, rather than making us guess, maybe you could provide your config details (dovecot -n output for starters)? -- Best regards, Charles

Re: [Dovecot] Corrupted index cache file

2011-04-15 Thread Frank Bonnet
On 04/15/2011 01:38 PM, Charles Marcus wrote: On 2011-04-15 2:32 AM, Frank Bonnet wrote: Dovecot or hardware problem ??? Thanks a lot for any ideas/infos Sorry, crystal ball is broken, rather than making us guess, maybe you could provide your config details (dovecot -n output for

Re: [Dovecot] Corrupted index cache file and Maildir filename has wrong W value

2010-05-25 Thread Barry Scott
On Tuesday 25 May 2010 16:26:15 Timo Sirainen wrote: On Tue, 2010-05-04 at 18:21 +0100, Barry Scott wrote: May 4 15:25:03 f12barry dovecot: IMAP(bscott): Maildir filename has wrong W value:

Re: [Dovecot] Corrupted index cache file and Maildir filename has wrong W value

2010-05-25 Thread Barry Scott
On Tuesday 25 May 2010 16:26:15 Timo Sirainen wrote: On Tue, 2010-05-04 at 18:21 +0100, Barry Scott wrote: May 4 15:25:03 f12barry dovecot: IMAP(bscott): Maildir filename has wrong W value: /home/bscott/Maildir/._NewMail-com/cur/1271683097.M615087P3538.f12barry.o

[Dovecot] Corrupted index cache file and Maildir filename has wrong W value

2010-05-04 Thread Barry Scott
kmail stopped working and when I check the dovecot log file I see: May 4 11:40:38 f12barry dovecot: IMAP(bscott): Corrupted index cache file /home/bscott/Maildir/._NewMail-com/dovecot.index.cache: Corrupted virtual size for uid=234: 633278 != 633251 May 4 11:40:38 f12barry dovecot:

[Dovecot] corrupted index cache file, physcial size, in migration to snow leopard

2010-04-21 Thread Norman Packard
Hi, I am in the middle of migrating a server from Apple's leopard to snow leopard. Having big problems with migrating mail to dovecot. The version is: # dovecotd --version 1.1.17apple0.5 and I use the migration script provide by Apple, /usr/libexec/dovecot/migrate_mail_data.pl The

Re: [Dovecot] Corrupted index cache file still occurs with Dovecot 1.2.9

2010-01-20 Thread Stuart Rowan
Timo Sirainen wrote, on 19/01/10 13:09: On Mon, 2010-01-11 at 11:07 +, Stuart Rowan wrote: Timo Sirainen wrote, on 28/07/09 19:24: On Tue, 2009-07-28 at 19:17 +0100, Stuart Rowan wrote: dovecot: 2009-07-28 10:23:16 Error: IMAP(strr): Corrupted index cache file

Re: [Dovecot] Corrupted index cache file still occurs with Dovecot 1.2.9

2010-01-20 Thread Timo Sirainen
On Wed, 2010-01-20 at 10:33 +, Stuart Rowan wrote: dovecot: 2009-07-28 10:23:16 Error: IMAP(strr): Corrupted index cache file /home/local/strr/Maildir/dovecot.index.cache: record continues outside its allocated size dovecot: 2009-07-28 10:23:56 Error: IMAP(strr): Corrupted index

Re: [Dovecot] Corrupted index cache file still occurs with Dovecot 1.2.9

2010-01-19 Thread Timo Sirainen
On Mon, 2010-01-11 at 11:07 +, Stuart Rowan wrote: Timo Sirainen wrote, on 28/07/09 19:24: On Tue, 2009-07-28 at 19:17 +0100, Stuart Rowan wrote: dovecot: 2009-07-28 10:23:16 Error: IMAP(strr): Corrupted index cache file /home/local/strr/Maildir/dovecot.index.cache: record continues

[Dovecot] Corrupted index cache file still occurs with Dovecot 1.2.9

2010-01-11 Thread Stuart Rowan
Timo Sirainen wrote, on 28/07/09 19:24: On Tue, 2009-07-28 at 19:17 +0100, Stuart Rowan wrote: dovecot: 2009-07-28 10:23:16 Error: IMAP(strr): Corrupted index cache file /home/local/strr/Maildir/dovecot.index.cache: record continues outside its allocated size dovecot: 2009-07-28 10:23:56

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-10 Thread Max Ivanov
Dovecot never modifies the S= field, even when it notices that it's wrong. So it's your LDA that writes them wrong. Is it ok to rename all files deleting S= and W= fields if no keywords was used?

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-09 Thread Timo Sirainen
On Jun 8, 2009, at 7:06 AM, Brent Bloxam wrote: Hello, I'm running Dovecot 1.1.13 on FreeBSD 7.0-RELEASE and I'm receiving a large number of errors much like the following: dovecot: Jun 08 09:27:04 Error: IMAP(x...@x.x): Corrupted index cache file /nfs/x.x/x/Maildir/dovecot.index.cache:

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-09 Thread Brent Bloxam
Timo Sirainen wrote: On Jun 8, 2009, at 7:06 AM, Brent Bloxam wrote: Hello, I'm running Dovecot 1.1.13 on FreeBSD 7.0-RELEASE and I'm receiving a large number of errors much like the following: dovecot: Jun 08 09:27:04 Error: IMAP(x...@x.x): Corrupted index cache file

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-09 Thread Timo Sirainen
On Jun 9, 2009, at 11:49 AM, Brent Bloxam wrote: Timo Sirainen wrote: On Jun 8, 2009, at 7:06 AM, Brent Bloxam wrote: Hello, I'm running Dovecot 1.1.13 on FreeBSD 7.0-RELEASE and I'm receiving a large number of errors much like the following: dovecot: Jun 08 09:27:04 Error:

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-09 Thread Brent Bloxam
Timo Sirainen wrote: On Jun 9, 2009, at 11:49 AM, Brent Bloxam wrote: Timo Sirainen wrote: On Jun 8, 2009, at 7:06 AM, Brent Bloxam wrote: Hello, I'm running Dovecot 1.1.13 on FreeBSD 7.0-RELEASE and I'm receiving a large number of errors much like the following: dovecot: Jun 08 09:27:04

Re: [Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-09 Thread Timo Sirainen
On Jun 9, 2009, at 12:26 PM, Brent Bloxam wrote: Thanks. The errors I've checked are occurring for recent files, looks like the S= field doesn't match the real size, for example: 33568 Jun 8 23:46 1235105170.42208.abc.xyz.com,S=33498:2,RS That's the problem then. Looking at older e-mails

[Dovecot] Corrupted index cache file issues (Corrupted physical size)

2009-06-08 Thread Brent Bloxam
Hello, I'm running Dovecot 1.1.13 on FreeBSD 7.0-RELEASE and I'm receiving a large number of errors much like the following: dovecot: Jun 08 09:27:04 Error: IMAP(x...@x.x): Corrupted index cache file /nfs/x.x/x/Maildir/dovecot.index.cache: Corrupted physical size for uid=2284: 5312 != 5217

[Dovecot] Corrupted Index Cache

2009-04-23 Thread Jeff Grossman
I am starting to use Squirrelmail a little bit more to read my mail during the day and I seem to be getting this error more frequently while using Squirrelmail. Just a coincidence or do you think using Squirrelmail might have something to do with it? Corrupted index cache file

[Dovecot] Corrupted index cache file

2009-02-13 Thread Proskurin Kirill
Hello all. I run on a problems? dovecot: Feb 13 08:36:13 Error: IMAP(logis...@fxclub.org): Maildir filename has wrong W value: /var/spool/dovecot/domains/fxclub.org/logistic/.fxclub/1135926892.Re1a4ad89Q1436.newmail.fxclub.org,S=24630,W=2 4849:2, dovecot: Feb 13 08:36:13 Error:

Re: [Dovecot] Corrupted index cache file

2009-02-13 Thread Diego Liziero
On Fri, Feb 13, 2009 at 12:32 AM, Timo Sirainen t...@iki.fi wrote: On Fri, 2009-02-13 at 00:10 +0100, Diego Liziero wrote: dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 .. When this error appears, the last mail attachment

Re: [Dovecot] Corrupted index cache file

2009-02-13 Thread Jose Celestino
Words by Timo Sirainen [Fri, Feb 13, 2009 at 12:27:07PM -0500]: On Fri, 2009-02-13 at 11:46 +0300, Proskurin Kirill wrote: Hello all. I run on a problems? dovecot: Feb 13 08:36:13 Error: IMAP(logis...@fxclub.org): Maildir filename has wrong W value: The problem is what it says.

Re: [Dovecot] Corrupted index cache file

2009-02-13 Thread Timo Sirainen
On Fri, 2009-02-13 at 17:47 +, Jose Celestino wrote: Words by Timo Sirainen [Fri, Feb 13, 2009 at 12:27:07PM -0500]: On Fri, 2009-02-13 at 11:46 +0300, Proskurin Kirill wrote: Hello all. I run on a problems? dovecot: Feb 13 08:36:13 Error: IMAP(logis...@fxclub.org): Maildir

[Dovecot] Corrupted index cache file

2009-02-12 Thread Frank Bonnet
Hello I got this error yesterday dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): Corrupted index cache file /user/dumontj/.imap/Sent/dovecot.index.cache: Broken virtual size for

Re: [Dovecot] Corrupted index cache file

2009-02-12 Thread Timo Sirainen
On Thu, 2009-02-12 at 09:31 +0100, Frank Bonnet wrote: Hello I got this error yesterday dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): Corrupted index cache file

Re: [Dovecot] Corrupted index cache file

2009-02-12 Thread Charles Marcus
On 2/12/2009 3:31 AM, Frank Bonnet wrote: dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): Corrupted index cache file /user/dumontj/.imap/Sent/dovecot.index.cache: Broken virtual

Re: [Dovecot] Corrupted index cache file

2009-02-12 Thread Diego Liziero
On Thu, Feb 12, 2009 at 5:39 PM, Charles Marcus cmar...@media-brokers.com wrote: On 2/12/2009 3:31 AM, Frank Bonnet wrote: dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): Corrupted

Re: [Dovecot] Corrupted index cache file

2009-02-12 Thread Timo Sirainen
On Fri, 2009-02-13 at 00:10 +0100, Diego Liziero wrote: dovecot: Feb 11 16:07:27 Error: IMAP(dumontj): FETCH for mailbox Sent UID 7139 got too little data: 2 vs 11160 .. It's really _rare_ and difficult to reproduce. Usually, it happens when a thunderbird user tells dovecot to store a

Re: [Dovecot] Corrupted index cache files

2008-08-14 Thread Charles Marcus
On 8/13/2008, Mike Abbott ([EMAIL PROTECTED]) wrote: Post your dovecot -n output? Here's some of it. Not very enlightening. Why just some of it? Its not like its that much, and what you may *think* is irrelevant, may in fact be *very* relevant... -- Best regards, Charles

Re: [Dovecot] Corrupted index cache files

2008-08-14 Thread Mike Abbott
I re-ran imaptest on an empty mail store, single client, multiple users, using your dovecot-crlf input file, for a couple hours. Here's the distribution of errors that imaptest reports: 100 Error: user%d[%d]: ...: Header DELIVERED-TO changed 167 Error: user%d[%d]: ...: Header CC changed

Re: [Dovecot] Corrupted index cache files

2008-08-14 Thread Timo Sirainen
On Aug 14, 2008, at 9:56 AM, Mike Abbott wrote: I re-ran imaptest on an empty mail store, single client, multiple users, using your dovecot-crlf input file, for a couple hours. Here's the distribution of errors that imaptest reports: 100 Error: user%d[%d]: ...: Header DELIVERED-TO changed

Re: [Dovecot] Corrupted index cache files

2008-08-13 Thread Mike Abbott
One or more users? Many different users. Post your dovecot -n output? Here's some of it. Not very enlightening. # 1.1.2: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ verbose_proctitle: yes first_valid_uid: 6 last_valid_uid: 5 first_valid_gid: 6 last_valid_gid: 5

Re: [Dovecot] Corrupted index cache files

2008-08-13 Thread Timo Sirainen
On Aug 13, 2008, at 6:09 PM, Mike Abbott wrote: mail_access_groups: mail mail_privileged_group: mail You probably won't need either of these. And there's no point in setting them to the same value. Can you reproduce these easily with my imaptest tool? http://imapwiki.org/ImapTest Some

Re: [Dovecot] Corrupted index cache files

2008-08-13 Thread Timo Sirainen
On Aug 13, 2008, at 6:09 PM, Mike Abbott wrote: Can you reproduce these easily with my imaptest tool? http://imapwiki.org/ImapTest Some of them. When running imaptest I see these dovecot errors: Corrupted index cache file %s: record continues outside its allocated size Corrupted index

Re: [Dovecot] Corrupted index cache files

2008-08-13 Thread Mike Abbott
How soon? With what kind of imaptest parameters? I can't reproduce this on my Macbook (OS X 10.5.4, HFS+). I ran imaptest for just a few minutes and saw all these errors in that time. Default imaptest parameters except for user/host names etc. Error: UIVALIDITY changed: %d - %d Did you

Re: [Dovecot] Corrupted index cache files

2008-08-13 Thread Timo Sirainen
On Aug 13, 2008, at 7:12 PM, Mike Abbott wrote: Error: UIVALIDITY changed: %d - %d Did you test with a single user or multiple users? imaptest's support for testing multiple users is currently a bit broken, but this shouldn't happen with a single user.. This error happens with

Re: [Dovecot] Corrupted index cache files

2008-08-12 Thread Timo Sirainen
On Aug 8, 2008, at 9:59 PM, Mike Abbott wrote: I see these errors more often than I'd like from Dovecot-1.1.2 on Mac OS X 10.5.4 (names and numbers elided): One or more users? Corrupted index cache file %s: Corrupted virtual size for uid=%d: %d != %d Corrupted index cache file %s: Broken

Re: [Dovecot] Corrupted index cache files

2008-08-09 Thread Charles Marcus
On 8/8/2008 9:59 PM, Mike Abbott wrote: I see these errors more often than I'd like from Dovecot-1.1.2 on Mac OS X 10.5.4 (names and numbers elided): Corrupted index cache file %s: Corrupted virtual size for uid=%d: %d != %d Corrupted index cache file %s: Broken virtual size for mail UID %d

[Dovecot] Corrupted index cache files

2008-08-08 Thread Mike Abbott
I see these errors more often than I'd like from Dovecot-1.1.2 on Mac OS X 10.5.4 (names and numbers elided): Corrupted index cache file %s: Corrupted virtual size for uid=%d: %d ! = %d Corrupted index cache file %s: Broken virtual size for mail UID %d Corrupted index cache file %s:

Re: [Dovecot] Dovecot corrupted index cache

2008-06-28 Thread Timo Sirainen
On Wed, 2008-06-25 at 09:40 -0700, Michael D. Godfrey wrote: A guess would be that this is likely due to the endianess of the multiple architectures that the index is being accessed with. We have the same issue here across i686/x86_64/sparc. I'm about to post to an older email thread

Re: [Dovecot] Dovecot corrupted index cache

2008-06-28 Thread Michael D Godfrey
On Wed, 2008-06-25 at 09:40 -0700, Michael D. Godfrey wrote: A guess would be that this is likely due to the endianess of the multiple architectures that the index is being accessed with. We have the same issue here across i686/x86_64/sparc. I'm about to post to an older email

Re: [Dovecot] Dovecot corrupted index cache

2008-06-28 Thread Timo Sirainen
On Sat, 2008-06-28 at 10:00 -0700, Michael D Godfrey wrote: On Wed, 2008-06-25 at 09:40 -0700, Michael D. Godfrey wrote: A guess would be that this is likely due to the endianess of the multiple architectures that the index is being accessed with. We have the same issue here

Re: [Dovecot] Dovecot corrupted index cache

2008-06-25 Thread David Halik
A guess would be that this is likely due to the endianess of the multiple architectures that the index is being accessed with. We have the same issue here across i686/x86_64/sparc. I'm about to post to an older email thread about this as well. Michael D. Godfrey wrote: System: Linux

Re: [Dovecot] Dovecot corrupted index cache

2008-06-25 Thread Michael D. Godfrey
A guess would be that this is likely due to the endianess of the multiple architectures that the index is being accessed with. We have the same issue here across i686/x86_64/sparc. I'm about to post to an older email thread about this as well. This is a good guess. We use a mixture of i386

[Dovecot] Dovecot corrupted index cache

2008-06-24 Thread Michael D. Godfrey
System: Linux 2.6.25.6-55.fc9.x86_64 #1 SMP Tue Jun 10 16:05:21 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux Dovecot: dovecot-1.0.14-8.fc9.x86_64 We have used Dovecot for several years. The error message below appears in the system log quite frequently. I have run a test which shows that this

[Dovecot] Corrupted index cache file error (Dovecot 1.0.3)

2007-10-05 Thread Mike Brudenell
Greetings - Now that users are beginning to pile up more on our new Dovecot-based IMAP service I'm seeing a small number of entries like this in the logfiles: Corrupted index cache file /mailstore/index/o/ozw100/.INBOX/ dovecot.index.cache : invalid record size We are using Dovecot

Re: [Dovecot] Corrupted index cache file error (Dovecot 1.0.3)

2007-10-05 Thread Chris Wakelin
Mike Brudenell wrote: Greetings - Now that users are beginning to pile up more on our new Dovecot-based IMAP service I'm seeing a small number of entries like this in the logfiles: Corrupted index cache file /mailstore/index/o/ozw100/.INBOX/dovecot.index.cache : invalid record size

Re: [Dovecot] Corrupted index cache file error (Dovecot 1.0.3)

2007-10-05 Thread Timo Sirainen
On Fri, 2007-10-05 at 17:40 +0100, Mike Brudenell wrote: Corrupted index cache file /mailstore/index/o/ozw100/.INBOX/ dovecot.index.cache : invalid record size All cache file errors are completely transparent to clients. The cache file gets deleted and Dovecot creates the wanted data from