Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Kind reminder :-) D. On Thu 13/02/2014 15:52, Dimos Alevizos wrote: > At last, I was finally able to find time and run a backtrace, I've > attached the file. > If you don't remember (and hey, you can blame you after all this > time) the problem is only with a couple (~10) of users among the > hundreds of thousands currently using the service. > > Please let me know if I should provide any other info, logs or whatever. > > Dimos Alevizos > > Original Message ---- > Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with > 'FFrom' or 'FrFrom' > From: Timo Sirainen > To: Dimos Alevizos > CC: Dimitris Paouris , Dovecot Mailing List > > Date: 02/11/2013 01:16 μμ > > >Well, if that patch didn’t work, then the problem is elsewhere. There aren’t > >many other good possibilities left though.. How about adding this patch, it > >should be even safer than the previous one: > > > >http://hg.dovecot.org/dovecot-2.2/rev/d3062d066593 > > > >On 30.10.2013, at 12.42, Dimos Alevizos wrote: > > > >>I'm afraid it doesn't seem to be working. > >>I've compiled a patched 2.2.6 dovecot with the patch you sent and installed > >>it in a production server (had to be 2.2.6 cause we've upgraded all the > >>rest since I begun this thread months ago) and although we still have mbox > >>corruptions (rarely as before) the server isn't crashing : > >> > >>Oct 30 11:15:19 pop04 dovecot: pop3-login: Login: user=, > >>method=PLAIN, rip=85.72.232.35, lip=83.235.66.43, mpid=24419, secured, > >>session=<+0ywxfHpIQBVSOgj> > >>Oct 30 11:15:20 pop04 dovecot: pop3(arto...@otenet.gr): Disconnected: > >>Logged out top=0/0, retr=0/0, del=0/1336, size=471029518 > >>Oct 30 11:19:12 pop04 dovecot: lmtp(2863, arto...@otenet.gr): > >>r7U3KnyhcFIvCwAAckDtvw: > >>msgid=: > >>size=17823 saved mail to INBOX > >>Oct 30 11:33:12 pop04 dovecot: pop3-login: Login: user=, > >>method=PLAIN, rip=85.72.224.94, lip=83.235.66.43, mpid=600, secured, > >>session= > >>Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Syncing > >>INBOX failed: Mailbox isn't a valid mbox file > >>Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Couldn't > >>init INBOX: Mailbox isn't a valid mbox file > >>Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Mailbox init failed > >>top=0/0, retr=0/0, del=0/0, size=0 > >>Oct 30 11:33:33 pop04 dovecot: lmtp(16314, arto...@otenet.gr): > >>Au4vIMqucFK6PwAAckDtvw: msgid=<004401ced552$bb5ecd70$321c6850$@planet.nl>: > >>size=7975817 save failed to INBOX: Mailbox isn't a valid mbox file > >> > >>Perhaps the patch is only valid for 2.1.16 and needs to be modified for > >>2.2.6 ? > >> > >>Thank you for your time > >>Dimos Alevizos > >> > >> > >> Original Message > >>Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or > >>'FrFrom' > >>From: Timo Sirainen > >>To: Dimos Alevizos > >>CC: dovecot@dovecot.org, Dimitris Paouris > >>Date: 26/06/2013 06:59 μμ > >> > >>>It crashes one specific IMAP/POP3 session, so others are unaffected. The > >>>potential problems: > >>> > >>> * It might cause the user's mbox to become crashing constantly. so first > >>> crash -> client reconnects -> client attempts the same operation -> crash > >>> again. Then again, this might not happen, it depends. > >>> > >>> * The mbox file would probably become slightly more corrupted than > >>> normally, because it doesn't finish moving data around. No data should > >>> get actually lost, but some parts could become duplicated (e.g. some > >>> headers or even mails, possibly causing UID renumbering = redownloading). > >>> > >>>So not ideal in production, but shouldn't be too bad either, especially if > >>>you just wait for the first crash and then immediately switch to the old > >>>unpatched version. > >>> > >>>On 26.6.2013, at 10.21, Dimos Alevizos wrote: > >>> > >>>>Hi, > >>>> > >>>>I haven't had the time to compile it yet, but a question just occurred. > >>>>Given that it's so rare and we can't reproduce it on a dev server, how > >>>>saf
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
At last, I was finally able to find time and run a backtrace, I've attached the file. If you don't remember (and hey, you can blame you after all this time) the problem is only with a couple (~10) of users among the hundreds of thousands currently using the service. Please let me know if I should provide any other info, logs or whatever. Dimos Alevizos Original Message Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: Timo Sirainen To: Dimos Alevizos CC: Dimitris Paouris , Dovecot Mailing List Date: 02/11/2013 01:16 μμ Well, if that patch didn’t work, then the problem is elsewhere. There aren’t many other good possibilities left though.. How about adding this patch, it should be even safer than the previous one: http://hg.dovecot.org/dovecot-2.2/rev/d3062d066593 On 30.10.2013, at 12.42, Dimos Alevizos wrote: I'm afraid it doesn't seem to be working. I've compiled a patched 2.2.6 dovecot with the patch you sent and installed it in a production server (had to be 2.2.6 cause we've upgraded all the rest since I begun this thread months ago) and although we still have mbox corruptions (rarely as before) the server isn't crashing : Oct 30 11:15:19 pop04 dovecot: pop3-login: Login: user=, method=PLAIN, rip=85.72.232.35, lip=83.235.66.43, mpid=24419, secured, session=<+0ywxfHpIQBVSOgj> Oct 30 11:15:20 pop04 dovecot: pop3(arto...@otenet.gr): Disconnected: Logged out top=0/0, retr=0/0, del=0/1336, size=471029518 Oct 30 11:19:12 pop04 dovecot: lmtp(2863, arto...@otenet.gr): r7U3KnyhcFIvCwAAckDtvw: msgid=: size=17823 saved mail to INBOX Oct 30 11:33:12 pop04 dovecot: pop3-login: Login: user=, method=PLAIN, rip=85.72.224.94, lip=83.235.66.43, mpid=600, secured, session= Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Syncing INBOX failed: Mailbox isn't a valid mbox file Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Couldn't init INBOX: Mailbox isn't a valid mbox file Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 Oct 30 11:33:33 pop04 dovecot: lmtp(16314, arto...@otenet.gr): Au4vIMqucFK6PwAAckDtvw: msgid=<004401ced552$bb5ecd70$321c6850$@planet.nl>: size=7975817 save failed to INBOX: Mailbox isn't a valid mbox file Perhaps the patch is only valid for 2.1.16 and needs to be modified for 2.2.6 ? Thank you for your time Dimos Alevizos ---- Original Message ---- Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: Timo Sirainen To: Dimos Alevizos CC: dovecot@dovecot.org, Dimitris Paouris Date: 26/06/2013 06:59 μμ It crashes one specific IMAP/POP3 session, so others are unaffected. The potential problems: * It might cause the user's mbox to become crashing constantly. so first crash -> client reconnects -> client attempts the same operation -> crash again. Then again, this might not happen, it depends. * The mbox file would probably become slightly more corrupted than normally, because it doesn't finish moving data around. No data should get actually lost, but some parts could become duplicated (e.g. some headers or even mails, possibly causing UID renumbering = redownloading). So not ideal in production, but shouldn't be too bad either, especially if you just wait for the first crash and then immediately switch to the old unpatched version. On 26.6.2013, at 10.21, Dimos Alevizos wrote: Hi, I haven't had the time to compile it yet, but a question just occurred. Given that it's so rare and we can't reproduce it on a dev server, how safe is this to use on a production server ? When you say "crash" you mean the whole dovecot server or that specific client's child ? D. Original Message Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: dalevizo To: Timo Sirainen CC: dovecot@dovecot.org, Dimitris Paouris Date: 24/06/2013 01:41 μμ Thanx I'll try the patch as soon as possible and I'll let you know. It is indeed very rare. We're only seeing 4-5 corruptions in about 13 million logins per day. I've been trying to convince our design team that we should move to maildir, but the truth is that it's quite a change, and we're way too busy to deal with everything else AND a migration from mbox to maildir. D. On Mon 24/06/2013 13:16, Timo Sirainen wrote: On 19.6.2013, at 16.00, Dimos Alevizos wrote: we're having some problems with our dovecot setup. I've seen similar problems in the mailing list some years ago but alas wasn't able to find a solution. Our setup is as follows : An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) which proxies pop3/imap/lmtp t
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Well, if that patch didn’t work, then the problem is elsewhere. There aren’t many other good possibilities left though.. How about adding this patch, it should be even safer than the previous one: http://hg.dovecot.org/dovecot-2.2/rev/d3062d066593 On 30.10.2013, at 12.42, Dimos Alevizos wrote: > I'm afraid it doesn't seem to be working. > I've compiled a patched 2.2.6 dovecot with the patch you sent and installed > it in a production server (had to be 2.2.6 cause we've upgraded all the rest > since I begun this thread months ago) and although we still have mbox > corruptions (rarely as before) the server isn't crashing : > > Oct 30 11:15:19 pop04 dovecot: pop3-login: Login: user=, > method=PLAIN, rip=85.72.232.35, lip=83.235.66.43, mpid=24419, secured, > session=<+0ywxfHpIQBVSOgj> > Oct 30 11:15:20 pop04 dovecot: pop3(arto...@otenet.gr): Disconnected: Logged > out top=0/0, retr=0/0, del=0/1336, size=471029518 > Oct 30 11:19:12 pop04 dovecot: lmtp(2863, arto...@otenet.gr): > r7U3KnyhcFIvCwAAckDtvw: > msgid=: size=17823 > saved mail to INBOX > Oct 30 11:33:12 pop04 dovecot: pop3-login: Login: user=, > method=PLAIN, rip=85.72.224.94, lip=83.235.66.43, mpid=600, secured, > session= > Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Syncing INBOX > failed: Mailbox isn't a valid mbox file > Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Couldn't init > INBOX: Mailbox isn't a valid mbox file > Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Mailbox init failed > top=0/0, retr=0/0, del=0/0, size=0 > Oct 30 11:33:33 pop04 dovecot: lmtp(16314, arto...@otenet.gr): > Au4vIMqucFK6PwAAckDtvw: msgid=<004401ced552$bb5ecd70$321c6850$@planet.nl>: > size=7975817 save failed to INBOX: Mailbox isn't a valid mbox file > > Perhaps the patch is only valid for 2.1.16 and needs to be modified for 2.2.6 > ? > > Thank you for your time > Dimos Alevizos > > > Original Message > Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or > 'FrFrom' > From: Timo Sirainen > To: Dimos Alevizos > CC: dovecot@dovecot.org, Dimitris Paouris > Date: 26/06/2013 06:59 μμ > >> It crashes one specific IMAP/POP3 session, so others are unaffected. The >> potential problems: >> >> * It might cause the user's mbox to become crashing constantly. so first >> crash -> client reconnects -> client attempts the same operation -> crash >> again. Then again, this might not happen, it depends. >> >> * The mbox file would probably become slightly more corrupted than >> normally, because it doesn't finish moving data around. No data should get >> actually lost, but some parts could become duplicated (e.g. some headers or >> even mails, possibly causing UID renumbering = redownloading). >> >> So not ideal in production, but shouldn't be too bad either, especially if >> you just wait for the first crash and then immediately switch to the old >> unpatched version. >> >> On 26.6.2013, at 10.21, Dimos Alevizos wrote: >> >>> Hi, >>> >>> I haven't had the time to compile it yet, but a question just occurred. >>> Given that it's so rare and we can't reproduce it on a dev server, how safe >>> is this to use on a production server ? >>> When you say "crash" you mean the whole dovecot server or that specific >>> client's child ? >>> >>> D. >>> >>> Original Message >>> Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or >>> 'FrFrom' >>> From: dalevizo >>> To: Timo Sirainen >>> CC: dovecot@dovecot.org, Dimitris Paouris >>> Date: 24/06/2013 01:41 μμ >>> >>>> Thanx I'll try the patch as soon as possible and I'll let you know. >>>> It is indeed very rare. We're only seeing 4-5 corruptions in about 13 >>>> million logins per day. >>>> I've been trying to convince our design team that we should move to >>>> maildir, but the truth is that it's quite a change, and we're way too >>>> busy to deal with everything else AND a migration from mbox to maildir. >>>> >>>> D. >>>> >>>> On Mon 24/06/2013 13:16, Timo Sirainen wrote: >>>>> On 19.6.2013, at 16.00, Dimos Alevizos wrote: >>>>> >>>>>> we're having some problems with our dovecot setup. >>>>>> I've see
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
I'm afraid it doesn't seem to be working. I've compiled a patched 2.2.6 dovecot with the patch you sent and installed it in a production server (had to be 2.2.6 cause we've upgraded all the rest since I begun this thread months ago) and although we still have mbox corruptions (rarely as before) the server isn't crashing : Oct 30 11:15:19 pop04 dovecot: pop3-login: Login: user=, method=PLAIN, rip=85.72.232.35, lip=83.235.66.43, mpid=24419, secured, session=<+0ywxfHpIQBVSOgj> Oct 30 11:15:20 pop04 dovecot: pop3(arto...@otenet.gr): Disconnected: Logged out top=0/0, retr=0/0, del=0/1336, size=471029518 Oct 30 11:19:12 pop04 dovecot: lmtp(2863, arto...@otenet.gr): r7U3KnyhcFIvCwAAckDtvw: msgid=: size=17823 saved mail to INBOX Oct 30 11:33:12 pop04 dovecot: pop3-login: Login: user=, method=PLAIN, rip=85.72.224.94, lip=83.235.66.43, mpid=600, secured, session= Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Syncing INBOX failed: Mailbox isn't a valid mbox file Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Error: Couldn't init INBOX: Mailbox isn't a valid mbox file Oct 30 11:33:12 pop04 dovecot: pop3(arto...@otenet.gr): Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 Oct 30 11:33:33 pop04 dovecot: lmtp(16314, arto...@otenet.gr): Au4vIMqucFK6PwAAckDtvw: msgid=<004401ced552$bb5ecd70$321c6850$@planet.nl>: size=7975817 save failed to INBOX: Mailbox isn't a valid mbox file Perhaps the patch is only valid for 2.1.16 and needs to be modified for 2.2.6 ? Thank you for your time Dimos Alevizos ---- Original Message Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: Timo Sirainen To: Dimos Alevizos CC: dovecot@dovecot.org, Dimitris Paouris Date: 26/06/2013 06:59 μμ It crashes one specific IMAP/POP3 session, so others are unaffected. The potential problems: * It might cause the user's mbox to become crashing constantly. so first crash -> client reconnects -> client attempts the same operation -> crash again. Then again, this might not happen, it depends. * The mbox file would probably become slightly more corrupted than normally, because it doesn't finish moving data around. No data should get actually lost, but some parts could become duplicated (e.g. some headers or even mails, possibly causing UID renumbering = redownloading). So not ideal in production, but shouldn't be too bad either, especially if you just wait for the first crash and then immediately switch to the old unpatched version. On 26.6.2013, at 10.21, Dimos Alevizos wrote: Hi, I haven't had the time to compile it yet, but a question just occurred. Given that it's so rare and we can't reproduce it on a dev server, how safe is this to use on a production server ? When you say "crash" you mean the whole dovecot server or that specific client's child ? D. Original Message Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: dalevizo To: Timo Sirainen CC: dovecot@dovecot.org, Dimitris Paouris Date: 24/06/2013 01:41 μμ Thanx I'll try the patch as soon as possible and I'll let you know. It is indeed very rare. We're only seeing 4-5 corruptions in about 13 million logins per day. I've been trying to convince our design team that we should move to maildir, but the truth is that it's quite a change, and we're way too busy to deal with everything else AND a migration from mbox to maildir. D. On Mon 24/06/2013 13:16, Timo Sirainen wrote: On 19.6.2013, at 16.00, Dimos Alevizos wrote: we're having some problems with our dovecot setup. I've seen similar problems in the mailing list some years ago but alas wasn't able to find a solution. Our setup is as follows : An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot 2.1.16). All mailboxes and indexes are on NFS and all servers are Centos. The problem is that at times we see mailboxes (all of them are in mbox format) beginning with FFrom or FrFrom and of course dovecot says it's not a valid mbox file. This is quite an old bug, but it happens rarely enough that I haven't been able to reproduce and fix it. Actually people hadn't complained about it for a long time now, so I had assumed it had somehow gotten fixed already. With the attached debug patch it should crash instead of (completely) corrupting the mbox file. Debugging the resulting core file with gdb could be useful in figuring this out. Although I wouldn't recommend mbox format for any big installation anyway..
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
It crashes one specific IMAP/POP3 session, so others are unaffected. The potential problems: * It might cause the user's mbox to become crashing constantly. so first crash -> client reconnects -> client attempts the same operation -> crash again. Then again, this might not happen, it depends. * The mbox file would probably become slightly more corrupted than normally, because it doesn't finish moving data around. No data should get actually lost, but some parts could become duplicated (e.g. some headers or even mails, possibly causing UID renumbering = redownloading). So not ideal in production, but shouldn't be too bad either, especially if you just wait for the first crash and then immediately switch to the old unpatched version. On 26.6.2013, at 10.21, Dimos Alevizos wrote: > Hi, > > I haven't had the time to compile it yet, but a question just occurred. > Given that it's so rare and we can't reproduce it on a dev server, how safe > is this to use on a production server ? > When you say "crash" you mean the whole dovecot server or that specific > client's child ? > > D. > > ---- Original Message ---- > Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or > 'FrFrom' > From: dalevizo > To: Timo Sirainen > CC: dovecot@dovecot.org, Dimitris Paouris > Date: 24/06/2013 01:41 μμ > >> Thanx I'll try the patch as soon as possible and I'll let you know. >> It is indeed very rare. We're only seeing 4-5 corruptions in about 13 >> million logins per day. >> I've been trying to convince our design team that we should move to >> maildir, but the truth is that it's quite a change, and we're way too >> busy to deal with everything else AND a migration from mbox to maildir. >> >> D. >> >> On Mon 24/06/2013 13:16, Timo Sirainen wrote: >>> On 19.6.2013, at 16.00, Dimos Alevizos wrote: >>> >>>> we're having some problems with our dovecot setup. >>>> I've seen similar problems in the mailing list some years ago but alas >>>> wasn't able to find a solution. >>>> >>>> Our setup is as follows : >>>> An MX farm (postfix) sends mails via LMTP to a director farm (dovecot >>>> 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot >>>> 2.1.16). >>>> All mailboxes and indexes are on NFS and all servers are Centos. >>>> >>>> The problem is that at times we see mailboxes (all of them are in mbox >>>> format) beginning with FFrom or FrFrom and of course dovecot says it's not >>>> a valid mbox file. >>> >>> This is quite an old bug, but it happens rarely enough that I haven't been >>> able to reproduce and fix it. Actually people hadn't complained about it >>> for a long time now, so I had assumed it had somehow gotten fixed already. >>> >>> With the attached debug patch it should crash instead of (completely) >>> corrupting the mbox file. Debugging the resulting core file with gdb could >>> be useful in figuring this out. >>> >>> Although I wouldn't recommend mbox format for any big installation anyway.. >>> >> >> >>> >>> >> >
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Hi, I haven't had the time to compile it yet, but a question just occurred. Given that it's so rare and we can't reproduce it on a dev server, how safe is this to use on a production server ? When you say "crash" you mean the whole dovecot server or that specific client's child ? D. Original Message ---- Subject: Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom' From: dalevizo To: Timo Sirainen CC: dovecot@dovecot.org, Dimitris Paouris Date: 24/06/2013 01:41 μμ Thanx I'll try the patch as soon as possible and I'll let you know. It is indeed very rare. We're only seeing 4-5 corruptions in about 13 million logins per day. I've been trying to convince our design team that we should move to maildir, but the truth is that it's quite a change, and we're way too busy to deal with everything else AND a migration from mbox to maildir. D. On Mon 24/06/2013 13:16, Timo Sirainen wrote: On 19.6.2013, at 16.00, Dimos Alevizos wrote: we're having some problems with our dovecot setup. I've seen similar problems in the mailing list some years ago but alas wasn't able to find a solution. Our setup is as follows : An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot 2.1.16). All mailboxes and indexes are on NFS and all servers are Centos. The problem is that at times we see mailboxes (all of them are in mbox format) beginning with FFrom or FrFrom and of course dovecot says it's not a valid mbox file. This is quite an old bug, but it happens rarely enough that I haven't been able to reproduce and fix it. Actually people hadn't complained about it for a long time now, so I had assumed it had somehow gotten fixed already. With the attached debug patch it should crash instead of (completely) corrupting the mbox file. Debugging the resulting core file with gdb could be useful in figuring this out. Although I wouldn't recommend mbox format for any big installation anyway..
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Thanx I'll try the patch as soon as possible and I'll let you know. It is indeed very rare. We're only seeing 4-5 corruptions in about 13 million logins per day. I've been trying to convince our design team that we should move to maildir, but the truth is that it's quite a change, and we're way too busy to deal with everything else AND a migration from mbox to maildir. D. On Mon 24/06/2013 13:16, Timo Sirainen wrote: > On 19.6.2013, at 16.00, Dimos Alevizos wrote: > > > we're having some problems with our dovecot setup. > > I've seen similar problems in the mailing list some years ago but alas > > wasn't able to find a solution. > > > > Our setup is as follows : > > An MX farm (postfix) sends mails via LMTP to a director farm (dovecot > > 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot > > 2.1.16). > > All mailboxes and indexes are on NFS and all servers are Centos. > > > > The problem is that at times we see mailboxes (all of them are in mbox > > format) beginning with FFrom or FrFrom and of course dovecot says it's not > > a valid mbox file. > > This is quite an old bug, but it happens rarely enough that I haven't been > able to reproduce and fix it. Actually people hadn't complained about it for > a long time now, so I had assumed it had somehow gotten fixed already. > > With the attached debug patch it should crash instead of (completely) > corrupting the mbox file. Debugging the resulting core file with gdb could be > useful in figuring this out. > > Although I wouldn't recommend mbox format for any big installation anyway.. > > >
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On 19.6.2013, at 16.00, Dimos Alevizos wrote: > we're having some problems with our dovecot setup. > I've seen similar problems in the mailing list some years ago but alas wasn't > able to find a solution. > > Our setup is as follows : > An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) > which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot 2.1.16). > All mailboxes and indexes are on NFS and all servers are Centos. > > The problem is that at times we see mailboxes (all of them are in mbox > format) beginning with FFrom or FrFrom and of course dovecot says it's not a > valid mbox file. This is quite an old bug, but it happens rarely enough that I haven't been able to reproduce and fix it. Actually people hadn't complained about it for a long time now, so I had assumed it had somehow gotten fixed already. With the attached debug patch it should crash instead of (completely) corrupting the mbox file. Debugging the resulting core file with gdb could be useful in figuring this out. Although I wouldn't recommend mbox format for any big installation anyway.. diff Description: Binary data
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Yeah that's because by mistake we put quota both in 10-mail.conf and
20-imap.conf, however I don't see how that could be the cause of the
problem.
D.
On Thu 20/06/2013 10:23, Benny Pedersen wrote:
> Dimos Alevizos skrev den 2013-06-19 15:00:
>
> >protocol imap {
> > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
> > mail_max_userip_connections = 100
> > mail_plugins = quota imap_quota quota notify mail_log
> >}
>
> quota listed 2 times
>
> --
> senders that put my email into body content will deliver it to my
> own trashcan, so if you like to get reply, dont do it
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Dimos Alevizos skrev den 2013-06-19 15:00:
protocol imap {
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
mail_max_userip_connections = 100
mail_plugins = quota imap_quota quota notify mail_log
}
quota listed 2 times
--
senders that put my email into body content will deliver it to my own
trashcan, so if you like to get reply, dont do it
[Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Hello, we're having some problems with our dovecot setup. I've seen similar problems in the mailing list some years ago but alas wasn't able to find a solution. Our setup is as follows : An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot 2.1.16). All mailboxes and indexes are on NFS and all servers are Centos. The problem is that at times we see mailboxes (all of them are in mbox format) beginning with FFrom or FrFrom and of course dovecot says it's not a valid mbox file. If we manually remove the offending extra characters from the beginning of the file everything is working again, but often the same user will show the problem again tomorrow. However it's very rare and so far we haven't been able to reproduce the problem. After activating the mail_log plugin hoping it'll help us locate the problem we run across this case : The user (let's call him u...@domain.gr) logs in via pop3 and deletes some mails but one of them fails: Jun 19 11:09:18 pop01 dovecot: pop3-login: Login: user=, method=PLAIN, rip=83.235.173.26, lip=83.235.66.40, mpid=1389, secured, session= Jun 19 11:09:18 director5 dovecot: pop3-login: proxy(u...@domain.gr): started proxying to 83.235.66.40:110: user=, method=PLAIN, rip=83.235.173.26, lip=83.235.66.65, session= Jun 19 11:09:24 pop01 dovecot: pop3(u...@domain.gr): expunge: box=INBOX, uid=20670, msgid=<9f5b1a20a9428ed31a9e7e42ce4411f08d0.20130612070...@mail84.us2.rsgsv.net>, size=36526 Jun 19 11:09:24 pop01 dovecot: pop3(u...@domain.gr): expunge: box=INBOX, uid=20671, msgid=, size=4637 Jun 19 11:09:24 pop01 dovecot: pop3(u...@domain.gr): expunge: box=INBOX, uid=20672, msgid=<1113670779560.910791405.1470.2.32032...@scheduler.constantcontact.com>, size=38655 Jun 19 11:09:24 pop01 dovecot: pop3(u...@domain.gr): expunge: box=INBOX, uid=20673, msgid=<83600a274c1407c5aa4c263e2592c...@debop.gr>, size=1181688 Jun 19 11:09:24 pop01 dovecot: pop3(u...@domain.gr): expunge: box=INBOX, uid=20674, msgid=, size=61116 Jun 19 11:09:26 pop01 dovecot: pop3(u...@domain.gr): Error: Next message unexpectedly corrupted in mbox file /var/mail/K/Y/V/domain_gr_user_007 at 1 Jun 19 11:09:26 pop01 dovecot: pop3(u...@domain.gr): Error: Cached message offset 1 is invalid for mbox file /var/mail/K/Y/V/domain_gr_user_007 Jun 19 11:09:27 pop01 dovecot: pop3(u...@domain.gr): Disconnected: Logged out top=0/0, retr=2/1502843, del=122/359, size=137448432 Jun 19 11:09:27 pop01 dovecot: pop3(u...@domain.gr): Warning: Our dotlock file /var/mail/K/Y/V/domain_gr_user_007.lock was deleted (locked 9 secs ago, touched 9 secs ago) Jun 19 11:09:27 pop01 dovecot: pop3(u...@domain.gr): Error: file_dotlock_delete() failed with mbox file /var/mail/K/Y/V/domain_gr_user_007: No such file or directory Jun 19 11:09:27 director5 dovecot: pop3-login: proxy(u...@domain.gr): disconnecting 83.235.173.26 (Disconnected by server): user=, method=PLAIN, rip=83.235.173.26, lip=83.235.66.65, session= However he's still able to log in again : Jun 19 11:11:36 pop01 dovecot: pop3-login: Login: user=, method=PLAIN, rip=85.74.231.10, lip=83.235.66.40, mpid=10094, secured, session= Jun 19 11:11:36 director3 dovecot: pop3-login: proxy(u...@domain.gr): started proxying to 83.235.66.40:110: user=, method=PLAIN, rip=85.74.231.10, lip=83.235.66.63, session= Jun 19 11:11:51 pop01 dovecot: pop3(u...@domain.gr): Disconnected: Logged out top=0/0, retr=0/0, del=0/237, size=75036938 Jun 19 11:11:51 director3 dovecot: pop3-login: proxy(u...@domain.gr): disconnecting 85.74.231.10 (Disconnected by server): user=, method=PLAIN, rip=85.74.231.10, lip=83.235.66.63, session= A few minutes later he receives another mail which is delivered with no problem : Jun 19 11:30:39 corvus amavis[546]: (00546-22) Passed CLEAN, [91.190.168.40] [91.190.168.40] -> , Message-ID: <58175-bWFyaW5hQGJlcm5pZXItZWxpYWR lcy5ncg==@free.splio.com>, mail_id: J1P-0WpmQksJ, Hits: 1.412, size: 9025, queued_as: 02BA4AE00414, Tests: [DKIM_VALID=-0.1,DKIM_VERIFIED=-0.1,FREEMAIL_FROM=0.001,HTML_IMAGE_RATIO_04=0.61,INVALID_MSGID=1,URIBL_BLOCKED=0.001], autolearn=disabled, 1533 ms Jun 19 11:30:39 corvus smtp/smtpd[1283]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 02BA4AE00414; from= to= proto=ESMTP helo= Jun 19 11:30:39 pop01 dovecot: lmtp(4099, u...@domain.gr): 77O4IHlGwVEDEAAAYg/qxw: msgid=58175-bWFyaW5hQGJlcm5pZXItZWxpYWRlcy5ncg==@free.splio.com: from=residence.sudde...@gmx.com size=9610 saved mail to INBOX Jun 19 11:30:39 corvus postfix/lmtp[1350]: 02BA4AE00414: to=, relay=lmtp.otenet.gr[62.103.147.209]:24, delay=0.12, delays=0.01/0/0/0.11, dsn=2.0.0, status=sent (250 2.0.0 77O4IHlGwVEDEAAAYg/qxw Saved) And then RIGHT after the successful delivery, the mbox somehow ends up corrupted with an FFrom in the beginning : Jun 19 11:31:27 pop01 dovecot: pop3-login: Login: user=, met
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Tue, 2010-07-27 at 09:32 -0400, Erik Kratzenberg wrote: > > If you delete all messages from mailbox, INTERNAL DATA message doesn't > > show up there? The mbox file size becomes zero? If so, it doesn't sound > > like you're really using Dovecot v1.x.. > > It looks like if I'm starting without the folder existing (whether it's > a new delivery rule or you delete the folder from IMAP), deliver doesn't > create the FOLDER INTERNAL DATA so it will just start appending messages > to it. Right, that's how it should work. > Once that first message is received, IMAP won't create the FOLDER > INTERNAL DATA message until I delete all the messages from the folder. Same here. > When I delete all the messages from the folder, they are not actuall > removed until I restart my mail client. If new messages are received to > that folder before I restart my mail client, then it also won't create > the FOLDER INTERNAL DATA message. I guess this means the messages are marked as \Deleted, but client hasn't sent EXPUNGE yet. > I'm not sure if that's directly related to this issue, but I know the > corrupted From line is always the first message in a mailbox and is > never the FOLDER INTERNAL DATA message. What I find strange is that you said this happens to POP3 users who delete all of the mails. Once all mails are deleted, the FOLDER INTERNAL DATA message should be written. After it is written, it should never get deleted afterwards. So how is it possible that POP3 users who regularly delete all of their mails still don't have the FOLDER INTERNAL DATA message?
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On 07/27/10 10:09, Stan Hoeppner wrote: I've only had this problem once, many months ago. I solved it by making sure the user was not logged in, editing the mbox file with vi, removing the leading "F", and saving the file. IIRC I then deleted the Dovecot index files to be on the safe side (not sure if it was necessary) and let them auto rebuild. Yeah, we can easily fix them when they come up, but unfortunately with thousands of mailboxes spread across many servers we're having it occur pretty frequently. A handful of mailboxes have had it happen more than once, but usually it's a new mailbox each time.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Erik Kratzenberg put forth on 7/27/2010 8:32 AM: >> If you delete all messages from mailbox, INTERNAL DATA message doesn't >> show up there? The mbox file size becomes zero? If so, it doesn't sound >> like you're really using Dovecot v1.x.. > > It looks like if I'm starting without the folder existing (whether it's > a new delivery rule or you delete the folder from IMAP), deliver doesn't > create the FOLDER INTERNAL DATA so it will just start appending messages > to it. > > Once that first message is received, IMAP won't create the FOLDER > INTERNAL DATA message until I delete all the messages from the folder. > > When I delete all the messages from the folder, they are not actuall > removed until I restart my mail client. If new messages are received to > that folder before I restart my mail client, then it also won't create > the FOLDER INTERNAL DATA message. > > I'm not sure if that's directly related to this issue, but I know the > corrupted From line is always the first message in a mailbox and is > never the FOLDER INTERNAL DATA message. I've only had this problem once, many months ago. I solved it by making sure the user was not logged in, editing the mbox file with vi, removing the leading "F", and saving the file. IIRC I then deleted the Dovecot index files to be on the safe side (not sure if it was necessary) and let them auto rebuild. -- Stan
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
If you delete all messages from mailbox, INTERNAL DATA message doesn't show up there? The mbox file size becomes zero? If so, it doesn't sound like you're really using Dovecot v1.x.. It looks like if I'm starting without the folder existing (whether it's a new delivery rule or you delete the folder from IMAP), deliver doesn't create the FOLDER INTERNAL DATA so it will just start appending messages to it. Once that first message is received, IMAP won't create the FOLDER INTERNAL DATA message until I delete all the messages from the folder. When I delete all the messages from the folder, they are not actuall removed until I restart my mail client. If new messages are received to that folder before I restart my mail client, then it also won't create the FOLDER INTERNAL DATA message. I'm not sure if that's directly related to this issue, but I know the corrupted From line is always the first message in a mailbox and is never the FOLDER INTERNAL DATA message.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Mon, 2010-07-26 at 12:47 -0400, Erik Kratzenberg wrote: > > On Tue, 2010-07-20 at 08:42 -0400, Erik Kratzenberg wrote: > >> I experienced this today with my own IMAP folder, so maybe being able to > >> offer a more specific order of events will help. Also not sure if I > >> previously mentioned that this is occasionally happening with IMAP > >> folders even though it's much more common under POP3. > >> > >> 1) I had just cleaned out my Junk folder. > >> 2) Two messages were received without error. > > > > So basically what POP3 clients usually do, delete everything. Does the > > mailbox have the "FOLDER INTERNAL DATA" message at the beginning? > > It did not and still doesn't, after having more messages received. If you delete all messages from mailbox, INTERNAL DATA message doesn't show up there? The mbox file size becomes zero? If so, it doesn't sound like you're really using Dovecot v1.x..
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Tue, 2010-07-20 at 08:42 -0400, Erik Kratzenberg wrote: I experienced this today with my own IMAP folder, so maybe being able to offer a more specific order of events will help. Also not sure if I previously mentioned that this is occasionally happening with IMAP folders even though it's much more common under POP3. 1) I had just cleaned out my Junk folder. 2) Two messages were received without error. So basically what POP3 clients usually do, delete everything. Does the mailbox have the "FOLDER INTERNAL DATA" message at the beginning? It did not and still doesn't, after having more messages received. 3) I marked the folder as read in Thunderbird. 4) I restarted Thunderbird 5) The next delivery attempt to that mailbox (using dovecot deliver) failed: save failed to Junk: Mailbox isn't a valid mbox file The odd thing is that the delivery *did* succeed because that message is in there. Or maybe you just got two of those same messages? :) I don't think it could have written it after giving this error. Well, it had the exact same Message ID and also had several deliveries afterwards that succeeded before I tried to access the mailbox again with IMAP. The very first message header is the one that was corrupted which always seems to be the case. Does "deliver" read/write the entire mbox file or just append the new message to the end? deliver first checks that the mbox file is valid by reading and verifying the From_-line at the beginning of the file. If it's ok, then it appends to end of file. The corruption most likely happened during the expunge, not in deliver. Hmm. See what it logs with attached patch? I'll get that patch added and let you know. Thanks.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Tue, 2010-07-20 at 08:42 -0400, Erik Kratzenberg wrote:
> I experienced this today with my own IMAP folder, so maybe being able to
> offer a more specific order of events will help. Also not sure if I
> previously mentioned that this is occasionally happening with IMAP
> folders even though it's much more common under POP3.
>
> 1) I had just cleaned out my Junk folder.
> 2) Two messages were received without error.
So basically what POP3 clients usually do, delete everything. Does the
mailbox have the "FOLDER INTERNAL DATA" message at the beginning?
> 3) I marked the folder as read in Thunderbird.
> 4) I restarted Thunderbird
> 5) The next delivery attempt to that mailbox (using dovecot deliver)
> failed:
>
> save failed to Junk: Mailbox isn't a valid mbox file
>
> The odd thing is that the delivery *did* succeed because that message is
> in there.
Or maybe you just got two of those same messages? :) I don't think it
could have written it after giving this error.
> The very first message header is the one that was corrupted
> which always seems to be the case. Does "deliver" read/write the entire
> mbox file or just append the new message to the end?
deliver first checks that the mbox file is valid by reading and
verifying the From_-line at the beginning of the file. If it's ok, then
it appends to end of file. The corruption most likely happened during
the expunge, not in deliver.
Hmm. See what it logs with attached patch?
diff -r f8c14abf52b4 src/lib-storage/index/mbox/mbox-sync.c
--- a/src/lib-storage/index/mbox/mbox-sync.c Wed Jul 14 13:53:34 2010 +0100
+++ b/src/lib-storage/index/mbox/mbox-sync.c Tue Jul 20 17:42:47 2010 +0100
@@ -1487,6 +1487,21 @@
sync_ctx->errors = FALSE;
}
+static void mbox_verify(struct mbox_sync_context *sync_ctx, int state)
+{
+ char buf[100];
+ int ret;
+
+ if ((ret = pread(sync_ctx->write_fd, buf, sizeof(buf), 0)) < 0) {
+ mbox_set_syscall_error(sync_ctx->mbox, "pread()");
+ return;
+ }
+ if (ret == 0)
+ return;
+ if (ret < 5 || strncmp(buf, "From ", 5) != 0)
+ i_error("mbox got corrupted, state=%d", state);
+}
+
static int mbox_sync_do(struct mbox_sync_context *sync_ctx,
enum mbox_sync_flags flags)
{
@@ -1496,6 +1511,8 @@
unsigned int i;
int ret, partial;
+ mbox_verify(sync_ctx, 0);
+
st = i_stream_stat(sync_ctx->file_input, FALSE);
if (st == NULL) {
mbox_set_syscall_error(sync_ctx->mbox, "i_stream_stat()");
@@ -1558,8 +1575,10 @@
partial = FALSE;
}
+ mbox_verify(sync_ctx, 1);
if (mbox_sync_handle_eof_updates(sync_ctx, &mail_ctx) < 0)
return -1;
+ mbox_verify(sync_ctx, 2);
/* only syncs left should be just appends (and their updates)
which weren't synced yet for some reason (crash). we'll just
@@ -1572,11 +1591,13 @@
/* Rewrite uid_last in X-IMAPbase header if we've seen it
(ie. the file isn't empty) */
ret = mbox_rewrite_base_uid_last(sync_ctx);
+ mbox_verify(sync_ctx, 3);
}
if (mbox_sync_update_index_header(sync_ctx) < 0)
return -1;
+ mbox_verify(sync_ctx, 4);
return 0;
}
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
I experienced this today with my own IMAP folder, so maybe being able to offer a more specific order of events will help. Also not sure if I previously mentioned that this is occasionally happening with IMAP folders even though it's much more common under POP3. 1) I had just cleaned out my Junk folder. 2) Two messages were received without error. 3) I marked the folder as read in Thunderbird. 4) I restarted Thunderbird 5) The next delivery attempt to that mailbox (using dovecot deliver) failed: save failed to Junk: Mailbox isn't a valid mbox file The odd thing is that the delivery *did* succeed because that message is in there. The very first message header is the one that was corrupted which always seems to be the case. Does "deliver" read/write the entire mbox file or just append the new message to the end? On 06/08/10 11:24, Erik Kratzenberg wrote: We've had situations where the same user will have their mailbox corrupted 2 or 3 times a day. We're not using filesystem quotas and nothing but Dovecot is writing to these mailboxes. We've been pushing people to switch to IMAP which seems to be the only solution so far. That's decreased the number of reports we're getting, but they're definitely still coming in. On 05/25/10 15:10, Timo Sirainen wrote: On Fri, 2010-05-07 at 15:03 -0400, Erik Kratzenberg wrote: -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > > Couldn't init INBOX: Mailbox isn't a valid mbox > > -The mailbox start with either 'FFrom or 'FrFrom' It's happening several times a day, sometimes to the same mailbox. How often does the same user get the error? We also noticed some other storage and assertion errors around the same time the corruption occurs which may or may not be related: Mar 26 14:49:36 dovecot: Panic: POP3(u...@home.com): file mbox-sync.c: line 1314 (mbox_sync_handle_eof_updates): assertion failed: (offset == 0|| offset> 31) There's a good chance this is related. Mar 24 15:21:59 dovecot: POP3(u...@home.com): Cached message offset 2 is invalid for mbox file /usr/boxes/mailbox Are you using filesystem quota for users? That might explain this. Otherwise, the above shouldn't be happening if Dovecot is the only software writing to the mbox files.. I'm not sure if "mail_debug=yes" will give us any additional info, but I'm turning that on now. No, that won't log anything useful.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
We've had situations where the same user will have their mailbox corrupted 2 or 3 times a day. We're not using filesystem quotas and nothing but Dovecot is writing to these mailboxes. We've been pushing people to switch to IMAP which seems to be the only solution so far. That's decreased the number of reports we're getting, but they're definitely still coming in. On 05/25/10 15:10, Timo Sirainen wrote: On Fri, 2010-05-07 at 15:03 -0400, Erik Kratzenberg wrote: -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > > Couldn't init INBOX: Mailbox isn't a valid mbox > > -The mailbox start with either 'FFrom or 'FrFrom' It's happening several times a day, sometimes to the same mailbox. How often does the same user get the error? We also noticed some other storage and assertion errors around the same time the corruption occurs which may or may not be related: Mar 26 14:49:36 dovecot: Panic: POP3(u...@home.com): file mbox-sync.c: line 1314 (mbox_sync_handle_eof_updates): assertion failed: (offset == 0|| offset> 31) There's a good chance this is related. Mar 24 15:21:59 dovecot: POP3(u...@home.com): Cached message offset 2 is invalid for mbox file /usr/boxes/mailbox Are you using filesystem quota for users? That might explain this. Otherwise, the above shouldn't be happening if Dovecot is the only software writing to the mbox files.. I'm not sure if "mail_debug=yes" will give us any additional info, but I'm turning that on now. No, that won't log anything useful.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Fri, 2010-05-07 at 15:03 -0400, Erik Kratzenberg wrote: > -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > >> > > Couldn't init INBOX: Mailbox isn't a valid mbox > >> > > -The mailbox start with either 'FFrom or 'FrFrom' > It's happening several times a day, sometimes to the same mailbox. How often does the same user get the error? > We also noticed some other storage and assertion errors around the same > time the corruption occurs which may or may not be related: > > Mar 26 14:49:36 dovecot: Panic: POP3(u...@home.com): > file mbox-sync.c: line 1314 (mbox_sync_handle_eof_updates): assertion > failed: (offset == 0|| offset > 31) There's a good chance this is related. > Mar 24 15:21:59 dovecot: POP3(u...@home.com): Cached > message offset 2 is invalid for mbox file > /usr/boxes/mailbox Are you using filesystem quota for users? That might explain this. Otherwise, the above shouldn't be happening if Dovecot is the only software writing to the mbox files.. > I'm not sure if "mail_debug=yes" will give us any additional info, but > I'm turning that on now. No, that won't log anything useful.
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On 04/29/10 13:52, Timo Sirainen wrote: On Wed, 2010-04-28 at 10:46 -0400, Erik Kratzenberg wrote: -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > > Couldn't init INBOX: Mailbox isn't a valid mbox > > -The mailbox start with either 'FFrom or 'FrFrom' > We're having the same issues on our servers here and it oddly started at exactly the same time (March 20th or so) with no changes made by us. How often do they happen? If it really is related to time, the only thing I can think of is that daylight saving time changed, which increased/decreased some timestamp string and broke some length calculation.. It's happening several times a day, sometimes to the same mailbox. I'm not sure if that's a reflection of how busy the mailbox is or not. It's difficult to say how many mailboxes we have being accessed via POP3 since it only seems to affect those, so I can't give you a percentage. We've upgraded to the latest Dovecot 1.2.11 and are using Dovecot 'deliver' exclusively. Tried setting fsync_disable=no, but the problem persists. You could try mbox_lazy_writes=no and mbox_dirty_syncs=no, although disabling lazy writing makes the performance worse. Let me know if those fix it.. Unfortunately, these didn't help. We definitely did notice some locking issues with those turned off though so we turned them back on once we were sure they didn't help with the corruption. We also noticed some other storage and assertion errors around the same time the corruption occurs which may or may not be related: --- Mar 28 11:24:20 dovecot: POP3(u...@home.com): Disconnected: Storage error during logout. top=0/0, retr=0/0, del=1/35, size=1167453 Mar 26 14:49:36 dovecot: Panic: POP3(u...@home.com): file mbox-sync.c: line 1314 (mbox_sync_handle_eof_updates): assertion failed: (offset == 0|| offset > 31) Mar 24 15:21:59 dovecot: POP3(u...@home.com): Cached message offset 2 is invalid for mbox file /usr/boxes/mailbox Mar 24 15:21:59 dovecot: POP3(u...@home.com): Disconnected: Storage error during logout. top=0/0, retr=0/0, del=1/232, size=10622872 --- I'm not sure if "mail_debug=yes" will give us any additional info, but I'm turning that on now. Thanks, Erik
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Wed, 2010-04-28 at 10:46 -0400, Erik Kratzenberg wrote: > > > -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > > >Couldn't init INBOX: Mailbox isn't a valid mbox > > > -The mailbox start with either 'FFrom or 'FrFrom' > > > We're having the same issues on our servers here and it > oddly started at exactly the same time (March 20th or so) > with no changes made by us. How often do they happen? If it really is related to time, the only thing I can think of is that daylight saving time changed, which increased/decreased some timestamp string and broke some length calculation.. > We've upgraded to the latest Dovecot 1.2.11 and are using > Dovecot 'deliver' exclusively. Tried setting > fsync_disable=no, but the problem persists. You could try mbox_lazy_writes=no and mbox_dirty_syncs=no, although disabling lazy writing makes the performance worse. Let me know if those fix it.. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
> > -Dovecot.log : Mar 29 11:00:06 Error: POP3(user_login): > >Couldn't init INBOX: Mailbox isn't a valid mbox > > -The mailbox start with either 'FFrom or 'FrFrom' > > I haven't seen these since v0.99. Are you using also Dovecot > deliver? Is anything else than Dovecot modifying the mbox > files? I don't really have any great ideas, other than some > locking problem if something besides Dovecot does changes.. We're having the same issues on our servers here and it oddly started at exactly the same time (March 20th or so) with no changes made by us. We've upgraded to the latest Dovecot 1.2.11 and are using Dovecot 'deliver' exclusively. Tried setting fsync_disable=no, but the problem persists. It seems to only occur for mailboxes that are being accessed via POP3 and leaving mail on the server. Any further insights? I'm surprised more people aren't experiencing this problem. I don't think we're doing anything particularly special. Thanks, Erik # 1.2.11: /etc/postfix/dovecot.conf # OS: FreeBSD 7.2-STABLE i386 base_dir: /var/run/dovecot protocols: imap imaps pop3 pop3s ssl_cert_file: sslcert.pem ssl_key_file: sslcert.pem ssl_cipher_list: ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM:!SSLv2 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login mbox_read_locks: flock mbox_write_locks: flock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_no_flag_updates: yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: auth_socket_path: /var/run/dovecot/auth-master postmaster_address: postmaster mail_plugins: quota sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login master_user_separator: * passdb: driver: passwd-file args: virtual.passwd passdb: driver: passwd-file args: username_format...@%l virtual.ip.passwd passdb: driver: passwd-file args: master.passwd passdb: driver: passwd-file args: master.user pass: yes master: yes userdb: driver: passwd-file args: virtual.passwd userdb: driver: passwd-file args: master.passwd socket: type: listen master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: users
Re: [Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
On Tue, 2010-03-30 at 14:56 +0200, Henri Khou wrote: > -Dovecot.log : ar 29 11:00:06 Error: POP3(user_login): Couldn't init > INBOX: Mailbox isn't a valid mbox > -The mailbox start with either 'FFrom or 'FrFrom' I haven't seen these since v0.99. Are you using also Dovecot deliver? Is anything else than Dovecot modifying the mbox files? I don't really have any great ideas, other than some locking problem if something besides Dovecot does changes.. signature.asc Description: This is a digitally signed message part
[Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
Hi, Since the 23rd of March, we encounter mailbox corruption for some users (15 out of 5000) using POP mainly via Thunderbird. Symptoms are : -users can login but fail to receive new mails and fail to read their inbox -Dovecot.log : ar 29 11:00:06 Error: POP3(user_login): Couldn't init INBOX: Mailbox isn't a valid mbox -The mailbox start with either 'FFrom or 'FrFrom' We can temporarily fix the issue by manually editing the mailbox so that it begins with a proper 'From'. For some users, the problem reoccurred after we apply this workaround. I have to mention that this server has been running fine since almost a year with the current hardware and software configuration. Does anybody has an idea regarding this issue? Many thanks Henri = System :openSUSE 11.1 (x86_64) Local FS : ReiserFS dovecot -n # 1.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.27.37-0.1-default x86_64 openSUSE 11.1 (x86_64) log_path: /var/log/dovecot.log syslog_facility: local7 protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 ssl_cert_file: /etc/ssl/certs/cert_742088754.pem ssl_key_file: /etc/ssl/private/imap.ehess.fr.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_greeting_capability(managesieve): no mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 mail_max_userip_connections(managesieve): 10 mail_uid: mail mail_gid: mail mail_location: mbox:~/:INBOX=/var/mail/%u:INDEX=/var/run/dovecot/indexes/%n mbox_lazy_writes: no mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota trash expire mail_plugins(imap): quota imap_quota trash expire mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib64/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib64/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib64/dovecot/modules/managesieve imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_reuse_xuidl(default): no pop3_reuse_xuidl(imap): no pop3_reuse_xuidl(pop3): yes pop3_reuse_xuidl(managesieve): no pop3_lock_session(default): no pop3_lock_session(imap): no pop3_lock_session(pop3): yes pop3_lock_session(managesieve): no pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): dict_db_config: /etc/dovecot/dovecot-db.conf sieve_storage(default): sieve_storage(imap): sieve_storage(pop3): sieve_storage(managesieve): ~/sieve sieve(default): sieve(imap): sieve(pop3): sieve(managesieve): ~/.dovecot.sieve auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: pam args: dovecot userdb: driver: passwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: fs:User quota:user:mount=/var/spool/mail quota_rule: *:storage=40M quota_rule2: INB0X:storage=250M quota_rule3: Trash:storage=20M quota_rule4: Spam:ignore quota_rule4: Sent:storage=20M trash: /etc/dovecot/dovecot-trash.conf expire: Trash 7 Spam 1 expire_dict: proxy::expire dict: expire: db:/var/lib/dovecot/expire.db
