On Tue, 2010-07-20 at 08:42 -0400, Erik Kratzenberg wrote:
> I experienced this today with my own IMAP folder, so maybe being able to
> offer a more specific order of events will help. Also not sure if I
> previously mentioned that this is occasionally happening with IMAP
> folders even though it's much more common under POP3.
>
> 1) I had just cleaned out my Junk folder.
> 2) Two messages were received without error.
So basically what POP3 clients usually do, delete everything. Does the
mailbox have the "FOLDER INTERNAL DATA" message at the beginning?
> 3) I marked the folder as read in Thunderbird.
> 4) I restarted Thunderbird
> 5) The next delivery attempt to that mailbox (using dovecot deliver)
> failed:
>
> save failed to Junk: Mailbox isn't a valid mbox file
>
> The odd thing is that the delivery *did* succeed because that message is
> in there.
Or maybe you just got two of those same messages? :) I don't think it
could have written it after giving this error.
> The very first message header is the one that was corrupted
> which always seems to be the case. Does "deliver" read/write the entire
> mbox file or just append the new message to the end?
deliver first checks that the mbox file is valid by reading and
verifying the From_-line at the beginning of the file. If it's ok, then
it appends to end of file. The corruption most likely happened during
the expunge, not in deliver.
Hmm. See what it logs with attached patch?
diff -r f8c14abf52b4 src/lib-storage/index/mbox/mbox-sync.c
--- a/src/lib-storage/index/mbox/mbox-sync.c Wed Jul 14 13:53:34 2010 +0100
+++ b/src/lib-storage/index/mbox/mbox-sync.c Tue Jul 20 17:42:47 2010 +0100
@@ -1487,6 +1487,21 @@
sync_ctx->errors = FALSE;
}
+static void mbox_verify(struct mbox_sync_context *sync_ctx, int state)
+{
+ char buf[100];
+ int ret;
+
+ if ((ret = pread(sync_ctx->write_fd, buf, sizeof(buf), 0)) < 0) {
+ mbox_set_syscall_error(sync_ctx->mbox, "pread()");
+ return;
+ }
+ if (ret == 0)
+ return;
+ if (ret < 5 || strncmp(buf, "From ", 5) != 0)
+ i_error("mbox got corrupted, state=%d", state);
+}
+
static int mbox_sync_do(struct mbox_sync_context *sync_ctx,
enum mbox_sync_flags flags)
{
@@ -1496,6 +1511,8 @@
unsigned int i;
int ret, partial;
+ mbox_verify(sync_ctx, 0);
+
st = i_stream_stat(sync_ctx->file_input, FALSE);
if (st == NULL) {
mbox_set_syscall_error(sync_ctx->mbox, "i_stream_stat()");
@@ -1558,8 +1575,10 @@
partial = FALSE;
}
+ mbox_verify(sync_ctx, 1);
if (mbox_sync_handle_eof_updates(sync_ctx, &mail_ctx) < 0)
return -1;
+ mbox_verify(sync_ctx, 2);
/* only syncs left should be just appends (and their updates)
which weren't synced yet for some reason (crash). we'll just
@@ -1572,11 +1591,13 @@
/* Rewrite uid_last in X-IMAPbase header if we've seen it
(ie. the file isn't empty) */
ret = mbox_rewrite_base_uid_last(sync_ctx);
+ mbox_verify(sync_ctx, 3);
}
if (mbox_sync_update_index_header(sync_ctx) < 0)
return -1;
+ mbox_verify(sync_ctx, 4);
return 0;
}
