Re: Fatal error: Exists wrong 513 512 483 17057
Hi, Bron. Thanks for your reply.
On 5/2/19 17:21, Bron Gondwana wrote:
> You need to run cyr_expire occasionally to clean up deleted messages!
Checking the syslog, apparently it ran automatically this morning:
--
# grep cyr_expire /var/log/syslog /var/log/syslog.1
/var/log/syslog.1:Feb 5 04:01:03 mail cyrus/cyr_expire[6111]: Expired 0
and expunged 0 out of 53555 messages from 80 mailboxes
/var/log/syslog.1:Feb 5 04:01:03 mail cyrus/cyr_expire[6111]:
duplicate_prune: pruning back 3.00 days
/var/log/syslog.1:Feb 5 04:03:16 mail cyrus/cyr_expire[6111]:
duplicate_prune: purged 562 out of 1756 entries
/var/log/syslog.1:Feb 5 04:30:00 mail cyrus/cyr_expire[6585]: Expired 0
and expunged 0 out of 53558 messages from 80 mailboxes
/var/log/syslog.1:Feb 5 04:30:00 mail cyrus/cyr_expire[6585]: Removed 0
deleted mailboxes
/var/log/syslog.1:Feb 5 04:30:00 mail cyrus/cyr_expire[6585]:
duplicate_prune: pruning back 4.00 days
/var/log/syslog.1:Feb 5 04:30:00 mail cyrus/cyr_expire[6585]:
duplicate_prune: purged 0 out of 1200 entries
/var/log/syslog.1:Feb 5 04:45:00 mail cyrus/cyr_expire[6849]: Expired 0
and expunged 0 out of 53558 messages from 80 mailboxes
/var/log/syslog.1:Feb 5 04:45:00 mail cyrus/cyr_expire[6849]:
duplicate_prune: pruning back 4.00 days
/var/log/syslog.1:Feb 5 04:45:00 mail cyrus/cyr_expire[6849]:
duplicate_prune: purged 0 out of 1200 entries
--
But still I see those files from 2016 to the present.
Checking /etc/cyrus.conf I found this configuration as default in Stretch:
--
EVENTS {
(...)
# this is only necessary if using duplicate delivery suppression
delprunecmd="/usr/sbin/cyrus expire -E 3" at=0401
(...)
# Expire data older than 28 days.
deleteprune cmd="/usr/sbin/cyrus expire -E 4 -D 28" at=0430
expungeprunecmd="/usr/sbin/cyrus expire -E 4 -X 28" at=0445
(...)
--
Am I missing something?
Thanks for your time.
Kind regards,
Daniel
signature.asc
Description: OpenPGP digital signature
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Fatal error: Exists wrong 513 512 483 17057
You need to run cyr_expire occasionally to clean up deleted messages! Bron. On Wed, Feb 6, 2019, at 04:45, Daniel Bareiro wrote: > Hi, Bron. > > On 4/2/19 14:02, Bron Gondwana wrote: > > > Awesome - yes, the IOERROR messages are because files it expected to > > find weren't there. It's frustrating that reconstruct isn't robust > > enough to bring a slightly bogus cyrus.index back from the dead, but the > > end result is a working mailbox with a fully correct cyrus.index file :) > > > > The only thing that may have happened is that some emails which were > > previously deleted in the TareasCron folder came back from the dead - > > and of course all the flags on those messages will have gone away too. > > But the mailbox will work correctly now. > > Maybe that explains why I came across about 16,000 messages in the > folder :) But everything works correctly now. > > I deleted the very old messages and only left the last ones, although in > the 'TasksCron' directory I am seeing messages on filesystem > (. files) of 2016 still. Could it be that for some reason > Cyrus keeps these files? In fact, in the 'Trash' directory I'm also > seeing files from 2016, even though I deleted everything I had there > using Thunderbird. > > Thanks for your reply. > > Kind regards, > Daniel > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > *Attachments:* > * signature.asc -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Fatal error: Exists wrong 513 512 483 17057
Hi, Bron. On 4/2/19 14:02, Bron Gondwana wrote: > Awesome - yes, the IOERROR messages are because files it expected to > find weren't there. It's frustrating that reconstruct isn't robust > enough to bring a slightly bogus cyrus.index back from the dead, but the > end result is a working mailbox with a fully correct cyrus.index file :) > > The only thing that may have happened is that some emails which were > previously deleted in the TareasCron folder came back from the dead - > and of course all the flags on those messages will have gone away too. > But the mailbox will work correctly now. Maybe that explains why I came across about 16,000 messages in the folder :) But everything works correctly now. I deleted the very old messages and only left the last ones, although in the 'TasksCron' directory I am seeing messages on filesystem (. files) of 2016 still. Could it be that for some reason Cyrus keeps these files? In fact, in the 'Trash' directory I'm also seeing files from 2016, even though I deleted everything I had there using Thunderbird. Thanks for your reply. Kind regards, Daniel signature.asc Description: OpenPGP digital signature Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Fatal error: Exists wrong 513 512 483 17057
Awesome - yes, the IOERROR messages are because files it expected to find weren't there. It's frustrating that reconstruct isn't robust enough to bring a slightly bogus cyrus.index back from the dead, but the end result is a working mailbox with a fully correct cyrus.index file :) The only thing that may have happened is that some emails which were previously deleted in the TareasCron folder came back from the dead - and of course all the flags on those messages will have gone away too. But the mailbox will work correctly now. Cheers, Bron. On Tue, Feb 5, 2019, at 03:26, Daniel Bareiro wrote: > Hi, Bron. Thanks for your reply. > > On 4/2/19 05:45, Bron Gondwana wrote: > > > Oh how frustrating... > > >> I tried using 'reconstruct' as you suggested and this was the result: > >> > >> -- > >> # /usr/lib/cyrus/bin/reconstruct -r user.admin > >> user.admin > >> user.admin.Backups > >> user.admin.Drafts > >> user.admin.Sent > >> user.admin.TareasCron: record corrupted 13 (maybe uid 9836) > >> user.admin.TareasCron: Mailbox format corruption detected Failed to > >> reconstruct mailbox > >> user.admin.Trash > >> -- > > > This clearly isn't syslogging enough! Can you check the permissions on > > the underlying filesystem! Also, try running reconstruct with -G, > > though it shouldn't make a difference. > > The previous thing was the output that I was obtaining in the console > when executing 'reconstruct'. This is what I get in the syslog when > executing the command with the '-G' flag: > > -- > Feb 4 11:48:29 mail cyrus/reconstruct[22727]: reconstructing user.admin > Feb 4 11:48:47 mail cyrus/reconstruct[22727]: mailbox: longlock > user.admin for 17.8 seconds > Feb 4 11:48:47 mail cyrus/reconstruct[22727]: Repacking mailbox > user.admin version 13 > Feb 4 11:48:48 mail cyrus/reconstruct[22727]: reconstructing > user.admin.Backups > Feb 4 11:48:50 mail cyrus/reconstruct[22727]: mailbox: longlock > user.admin.Backups for 2.5 seconds > Feb 4 11:48:50 mail cyrus/reconstruct[22727]: Repacking mailbox > user.admin.Backups version 13 > Feb 4 11:48:50 mail cyrus/reconstruct[22727]: reconstructing > user.admin.Drafts > Feb 4 11:48:51 mail cyrus/reconstruct[22727]: reconstructing > user.admin.Sent > Feb 4 11:48:51 mail cyrus/reconstruct[22727]: reconstructing > user.admin.TareasCron > -- > > But in essence the console shows the same output as without adding '-G': > > -- > # /usr/lib/cyrus/bin/reconstruct -G -r user.admin > user.admin > user.admin.Backups > user.admin.Drafts > user.admin.Sent > user.admin.TareasCron: record corrupted 13 (maybe uid 9836) > user.admin.TareasCron: Mailbox format corruption detected Failed to > reconstruct mailbox > user.admin.Trash > -- > > >> Just 'TasksCron' is the folder that I can not open. What do you suggest > >> to do in this case? > > > I'd take a backup, and then delete the cyrus.index file in that folder > > and run reconstruct again... That's a pretty extreme approach, but it > > should work. > > Let's try the pretty extreme approach... :-D > > -- > Feb 4 12:04:43 mail cyrus/reconstruct[22990]: reconstructing user.admin > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing > user.admin.Backups > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing > user.admin.Drafts > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing > user.admin.Sent > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing > user.admin.TareasCron > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: IOERROR: opening index > user.admin.TareasCron: System I/O error > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: create new mailbox > user.admin.TareasCron > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: failed to read index > header for user.admin.TareasCron > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9837 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9838 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9839 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9840 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9841 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9842 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9843 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9844 found - adding > Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 9845 found - adding > [...] > Feb 4 12:07:21 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 27841 found - adding > Feb 4 12:07:21 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid > 27842 found - adding > Feb 4 12:07:21 mail cyrus/reconstruct[22990]: mailbox: longlock > user.admin.TareasCron for 157.3 seconds > Feb
Re: Fatal error: Exists wrong 513 512 483 17057
Hi, Bron. Thanks for your reply. On 4/2/19 05:45, Bron Gondwana wrote: > Oh how frustrating... >> I tried using 'reconstruct' as you suggested and this was the result: >> >> -- >> # /usr/lib/cyrus/bin/reconstruct -r user.admin >> user.admin >> user.admin.Backups >> user.admin.Drafts >> user.admin.Sent >> user.admin.TareasCron: record corrupted 13 (maybe uid 9836) >> user.admin.TareasCron: Mailbox format corruption detected Failed to >> reconstruct mailbox >> user.admin.Trash >> -- > This clearly isn't syslogging enough! Can you check the permissions on > the underlying filesystem! Also, try running reconstruct with -G, > though it shouldn't make a difference. The previous thing was the output that I was obtaining in the console when executing 'reconstruct'. This is what I get in the syslog when executing the command with the '-G' flag: -- Feb 4 11:48:29 mail cyrus/reconstruct[22727]: reconstructing user.admin Feb 4 11:48:47 mail cyrus/reconstruct[22727]: mailbox: longlock user.admin for 17.8 seconds Feb 4 11:48:47 mail cyrus/reconstruct[22727]: Repacking mailbox user.admin version 13 Feb 4 11:48:48 mail cyrus/reconstruct[22727]: reconstructing user.admin.Backups Feb 4 11:48:50 mail cyrus/reconstruct[22727]: mailbox: longlock user.admin.Backups for 2.5 seconds Feb 4 11:48:50 mail cyrus/reconstruct[22727]: Repacking mailbox user.admin.Backups version 13 Feb 4 11:48:50 mail cyrus/reconstruct[22727]: reconstructing user.admin.Drafts Feb 4 11:48:51 mail cyrus/reconstruct[22727]: reconstructing user.admin.Sent Feb 4 11:48:51 mail cyrus/reconstruct[22727]: reconstructing user.admin.TareasCron -- But in essence the console shows the same output as without adding '-G': -- # /usr/lib/cyrus/bin/reconstruct -G -r user.admin user.admin user.admin.Backups user.admin.Drafts user.admin.Sent user.admin.TareasCron: record corrupted 13 (maybe uid 9836) user.admin.TareasCron: Mailbox format corruption detected Failed to reconstruct mailbox user.admin.Trash -- >> Just 'TasksCron' is the folder that I can not open. What do you suggest >> to do in this case? > I'd take a backup, and then delete the cyrus.index file in that folder > and run reconstruct again... That's a pretty extreme approach, but it > should work. Let's try the pretty extreme approach... :-D -- Feb 4 12:04:43 mail cyrus/reconstruct[22990]: reconstructing user.admin Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing user.admin.Backups Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing user.admin.Drafts Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing user.admin.Sent Feb 4 12:04:44 mail cyrus/reconstruct[22990]: reconstructing user.admin.TareasCron Feb 4 12:04:44 mail cyrus/reconstruct[22990]: IOERROR: opening index user.admin.TareasCron: System I/O error Feb 4 12:04:44 mail cyrus/reconstruct[22990]: create new mailbox user.admin.TareasCron Feb 4 12:04:44 mail cyrus/reconstruct[22990]: failed to read index header for user.admin.TareasCron Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9837 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9838 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9839 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9840 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9841 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9842 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9843 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9844 found - adding Feb 4 12:04:44 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 9845 found - adding [...] Feb 4 12:07:21 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 27841 found - adding Feb 4 12:07:21 mail cyrus/reconstruct[22990]: user.admin.TareasCron uid 27842 found - adding Feb 4 12:07:21 mail cyrus/reconstruct[22990]: mailbox: longlock user.admin.TareasCron for 157.3 seconds Feb 4 12:07:21 mail cyrus/reconstruct[22990]: reconstructing user.admin.Trash -- I guess the 'IOERROR' is because the cyrus.index file can not be found. When afterwards it says 'create new mailbox user.admin.TareasCron' what it is creating is the index? Because, if so, I don't understand why later it says 'failed to read index header for user.admin.TareasCron'. In any case, I am now able to access the 'TareasCron' folder without problems. Thank you so much! >> I'm using Cyrus 2.5.10-3 on Debian Stretch. A few days ago I upgraded >> the entire operating system with Debian Jessie and Cyrus 2.4.17. > That upgrade should be fine, and 2.5.10 should be fine as well. I just > poked around in the code there to see how the reconstruct might be > failing, and there's not enough information to
Re: Fatal error: Exists wrong 513 512 483 17057
Oh how frustrating... On Mon, Feb 4, 2019, at 08:48, Daniel Bareiro wrote: > Hi, Bron. Thanks for your reply. > > On 3/2/19 08:33, Bron Gondwana wrote: > > > Wow. A reconstruct should fix that. What version are you running? > > I tried using 'reconstruct' as you suggested and this was the result: > > -- > # /usr/lib/cyrus/bin/reconstruct -r user.admin > user.admin > user.admin.Backups > user.admin.Drafts > user.admin.Sent > user.admin.TareasCron: record corrupted 13 (maybe uid 9836) > user.admin.TareasCron: Mailbox format corruption detected Failed to > reconstruct mailbox This clearly isn't syslogging enough! Can you check the permissions on the underlying filesystem! Also, try running reconstruct with -G, though it shouldn't make a difference. > user.admin.Trash > -- > > Just 'TasksCron' is the folder that I can not open. What do you suggest > to do in this case? I'd take a backup, and then delete the cyrus.index file in that folder and run reconstruct again... That's a pretty extreme approach, but it should work. > I'm using Cyrus 2.5.10-3 on Debian Stretch. A few days ago I upgraded > the entire operating system with Debian Jessie and Cyrus 2.4.17. That upgrade should be fine, and 2.5.10 should be fine as well. I just poked around in the code there to see how the reconstruct might be failing, and there's not enough information to know what's happening. Bron. -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Fatal error: Exists wrong 513 512 483 17057
Hi, Bron. Thanks for your reply. On 3/2/19 08:33, Bron Gondwana wrote: > Wow. A reconstruct should fix that. What version are you running? I tried using 'reconstruct' as you suggested and this was the result: -- # /usr/lib/cyrus/bin/reconstruct -r user.admin user.admin user.admin.Backups user.admin.Drafts user.admin.Sent user.admin.TareasCron: record corrupted 13 (maybe uid 9836) user.admin.TareasCron: Mailbox format corruption detected Failed to reconstruct mailbox user.admin.Trash -- Just 'TasksCron' is the folder that I can not open. What do you suggest to do in this case? I'm using Cyrus 2.5.10-3 on Debian Stretch. A few days ago I upgraded the entire operating system with Debian Jessie and Cyrus 2.4.17. Kind regards, Daniel signature.asc Description: OpenPGP digital signature Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Fatal error: Exists wrong 513 512 483 17057
Wow. A reconstruct should fix that. What version are you running?
On Sun, Feb 3, 2019, at 02:11, Daniel Bareiro wrote:
> Hi!
>
> Several days ago I am trying to solve this issue but without success.
> When opening a folder inside a mailbox through Thunderbird, I get this
> message: "Server [servername] has disconnected. The server may have gone
> down or there may be a network problem".
>
> In that access attempt the log shows an entry like this:
>
> --
> Feb 2 21:55:38 mail cyrus/imaps[6510]: Fatal error: Exists wrong 513
> 512 483 17057
> --
>
> Maybe the index is corrupt and has some duplicate message?
>
> Investigating on the web I found this reference [1]:
>
> --
> /* make sure we don't overflow the memory we mapped */
> if (msgno >= state->mapsize) {
> char buf[2048];
> sprintf(buf, "Exists wrong %u %u %u %u", msgno,
> state->mapsize, mailbox->i.exists, mailbox->i.num_records);
> fatal(buf, EC_IOERR);
> }
> --
>
> I would like to know what you think. Is it possible that this is a
> problem with the index for this mailbox? If so, do you recommend some
> way to regenerate it?
>
>
> Thanks in advance for your reply.
>
> Kind regards,
> Daniel
>
> [1]
> https://forge.bluemind.net/stash/projects/BM/repos/bm-cyrus-imapd/browse/imap/index.c?at=c196236d937608270283c681ae27f1bb00265b62&raw
>
>
>
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
> *Attachments:*
> * signature.asc
--
Bron Gondwana
br...@fastmail.fm
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
