Re: [Dovecot] Maildir files with mtime in the future
* Timo Sirainen [2009-04-03 12:44]: > On Thu, 2009-04-02 at 19:36 +0200, Holger Weiss wrote: > > However, if the client does so, Dovecot will set the mtime of the > > Maildir file in question to the date specified by the client even if > > it's in the future. Since files with an mtime in the future can cause > > all sorts of trouble (e.g., they might get backed up repeatedly by > > incremental backups), it would be nice if Dovecot would (optionally?) > > use the current time if the client specifies a time in the future. For > > example, we use the following patch: > > Committed the patch, but forgot to add your name to the commit message, > sorry. :( No problem. > I also changed it so that it allows times 2 hours into the future (I'm > not sure why I chose exactly 2 hours). Sounds good. Thank you, Holger
Re: [Dovecot] Maildir files with mtime in the future
On Thu, 2009-04-02 at 19:36 +0200, Holger Weiss wrote: > However, if the client does so, Dovecot will set the mtime of the > Maildir file in question to the date specified by the client even if > it's in the future. Since files with an mtime in the future can cause > all sorts of trouble (e.g., they might get backed up repeatedly by > incremental backups), it would be nice if Dovecot would (optionally?) > use the current time if the client specifies a time in the future. For > example, we use the following patch: Committed the patch, but forgot to add your name to the commit message, sorry. :( I also changed it so that it allows times 2 hours into the future (I'm not sure why I chose exactly 2 hours). signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Maildir files with mtime in the future
* Ben Winslow [2009-04-03 10:59]: > On Fri, 3 Apr 2009 01:14:47 +0200 > Holger Weiss wrote: > > I'd guess most backup software will include files with an mtime newer > > than the time of the previous backup in incremental backups. At > > least, Bacula[*] and Veritas NetBackup do it that way. > > Aren't IMAP messages supposed to be immutable? If the actual message > is changed, it should reappear as a new file/message, and therefore be > backed up because it didn't exist in the previous backup run. The problem with an mtime in the future is that the file in question will be backed up again and again with every incremental backup run even though the file is unchanged. Holger
Re: [Dovecot] Maildir files with mtime in the future
On Fri, 3 Apr 2009 01:14:47 +0200 Holger Weiss wrote: > I'd guess most backup software will include files with an mtime newer > than the time of the previous backup in incremental backups. At > least, Bacula[*] and Veritas NetBackup do it that way. Aren't IMAP messages supposed to be immutable? If the actual message is changed, it should reappear as a new file/message, and therefore be backed up because it didn't exist in the previous backup run. -- Ben Winslow
Re: [Dovecot] Maildir files with mtime in the future
* Timo Sirainen [2009-04-02 19:35]: > On Fri, 2009-04-03 at 01:14 +0200, Holger Weiss wrote: > > > This is a problem only with Maildir, so if the code is added it should > > > go to maildir-save.c. > > > > The mtime-problem is Maildir-specific, but an INTERNALDATE in the future > > is obviously incorrect in any case. > > Not necessarily. Although I guess it's always some kind of a > misconfiguration in that case. Like if server's time isn't correct. Yes, but I guess the code should simply assume the server's time is correct (or things may go south). > Or maybe client wants to live in future. :) True, you never know ... :-) > I'll ask in imap list what others think about it. Thanks a lot! Holger
Re: [Dovecot] Maildir files with mtime in the future
On Fri, 2009-04-03 at 01:14 +0200, Holger Weiss wrote: > > This is a problem only with Maildir, so if the code is added it should > > go to maildir-save.c. > > The mtime-problem is Maildir-specific, but an INTERNALDATE in the future > is obviously incorrect in any case. Not necessarily. Although I guess it's always some kind of a misconfiguration in that case. Like if server's time isn't correct. Or maybe client wants to live in future. :) I'll ask in imap list what others think about it. If no one can give any good reasons for future timestamps I guess I'll just apply your patch. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Maildir files with mtime in the future
* Timo Sirainen [2009-04-02 18:44]: > On Thu, 2009-04-02 at 19:36 +0200, Holger Weiss wrote: > > | If a date-time is specified, the internal date SHOULD be set in the > > | resulting message; otherwise, the internal date of the resulting > > | message is set to the current date and time by default. > > > > [ ftp://ftp.fu-berlin.de/doc/rfc/rfc3501.txt (6.3.11. APPEND Command) ] > > > > However, if the client does so, Dovecot will set the mtime of the > > Maildir file in question to the date specified by the client even if > > it's in the future. Since files with an mtime in the future can cause > > all sorts of trouble (e.g., they might get backed up repeatedly by > > incremental backups), it would be nice if Dovecot would (optionally?) > > use the current time if the client specifies a time in the future. > > Hmm. I don't really like violating a SHOULD. Maybe the reason why this is not defined as a MUST is that the server needn't accept obviously incorrect dates :-) > I wonder how big of a problem this actually is. Are people often backing > up based on mtime? I'd guess most backup software will include files with an mtime newer than the time of the previous backup in incremental backups. At least, Bacula[*] and Veritas NetBackup do it that way. > > For example, we use the following patch: > > This is a problem only with Maildir, so if the code is added it should > go to maildir-save.c. The mtime-problem is Maildir-specific, but an INTERNALDATE in the future is obviously incorrect in any case. Holger [*] See: http://www.bacula.org/manuals/en/install/install/Configuring_Director.html
Re: [Dovecot] Maildir files with mtime in the future
On Thu, 2009-04-02 at 19:36 +0200, Holger Weiss wrote: > | If a date-time is specified, the internal date SHOULD be set in the > | resulting message; otherwise, the internal date of the resulting > | message is set to the current date and time by default. > > [ ftp://ftp.fu-berlin.de/doc/rfc/rfc3501.txt (6.3.11. APPEND Command) ] > > However, if the client does so, Dovecot will set the mtime of the > Maildir file in question to the date specified by the client even if > it's in the future. Since files with an mtime in the future can cause > all sorts of trouble (e.g., they might get backed up repeatedly by > incremental backups), it would be nice if Dovecot would (optionally?) > use the current time if the client specifies a time in the future. Hmm. I don't really like violating a SHOULD. And I don't like adding a setting to do it either. There are already too many settings. I wonder how big of a problem this actually is. Are people often backing up based on mtime? > For example, we use the following patch: This is a problem only with Maildir, so if the code is added it should go to maildir-save.c. signature.asc Description: This is a digitally signed message part
[Dovecot] Maildir files with mtime in the future
If Maildir storage is used, the mtime of a given Maildir file is set to
the message's INTERNALDATE. Now, when a client APPENDs a message to an
IMAP mailbox, the client may optionally specify the INTERNALDATE:
| If a date-time is specified, the internal date SHOULD be set in the
| resulting message; otherwise, the internal date of the resulting
| message is set to the current date and time by default.
[ ftp://ftp.fu-berlin.de/doc/rfc/rfc3501.txt (6.3.11. APPEND Command) ]
However, if the client does so, Dovecot will set the mtime of the
Maildir file in question to the date specified by the client even if
it's in the future. Since files with an mtime in the future can cause
all sorts of trouble (e.g., they might get backed up repeatedly by
incremental backups), it would be nice if Dovecot would (optionally?)
use the current time if the client specifies a time in the future. For
example, we use the following patch:
-- 8< 8< --
diff --git a/src/imap/cmd-append.c b/src/imap/cmd-append.c
index 0b22fd5..b7e42b9 100644
--- a/src/imap/cmd-append.c
+++ b/src/imap/cmd-append.c
@@ -319,6 +319,12 @@ static bool cmd_append_continue_parsing(struct
client_command_context *cmd)
return cmd_append_cancel(ctx, nonsync);
}
+ if (internal_date != (time_t)-1 && internal_date > time(NULL)) {
+ /* the client specified a time in the future, set it to now. */
+ internal_date = (time_t)-1;
+ timezone_offset = 0;
+ }
+
if (ctx->msg_size == 0) {
/* no message data, abort */
client_send_tagline(cmd, "NO Can't save a zero byte message.");
-- 8< 8< --
Holger
