That's what sieve is for -- do it in the server and you won't have to rely on a particular client doing it for you.The mail provider (MX) for my domain, fastmail.fm, runs cyrus. I use Eudora (for Mac, v5.2), mostly in POP mode, but I use some IMAP features too. In particular, some of my filters copy incoming (POP) messages to an IMAP mailbox at fastmail.fm. That's where the problems start.
Some of these incoming messages contain NULs or bare CR or LF. Yes, the sender is broken as far as RFC2822 is concerned, but the messages get through anyway. And the messages are valid RFC822/STD0011 format.
When Eudora tries to copy these (APPEND them) to the IMAP mailbox,
cyrus (at fastmail.fm) returns an error. I could live with an
occasional copy failure, but the worst part is that when Eudora gets
the server error, it thinks it's a terrible problem and throws up a
dialog box and ceases all processing. Since I (like many people) depend
on Eudora cleaning up my mailbox and doing other things with incoming
mail automatically when I'm not at my desk, this gets to be a serious
problem.
I disagree - it sounds like it would be defensible if Cyrus supported storing such messages even though it is clearly recommended against by the standard. If Eudora insists on storing such messages, it should be prepared to deal with a server that is unwilling to do so.So I started reading RFC3501 to find the reason. I assumed that I'd find a good reason that I could quote to Eudora support, telling them why Eudora has to clean up the message before storing it in an IMAP mailbox. But I didn't find that. What I found -- under the APPEND command (section 6.3.11) -- is
: The APPEND command appends the literal argument as a new message : to the end of the specified destination mailbox. This argument SHOULD : be in the format of an [RFC-2822] message.
Note well: that's "SHOULD", not "MUST". This is important. RFC2119 gives the meaning of SHOULD:
: This word [...] mean[s] that there may exist valid reasons in : particular circumstances to ignore a particular item [...]
So based on my reading of the RFC, it's the client's choice: it should normally append RFC2822 messages, but if it has a valid reason, it's allowed to append something that's not RFC2822. Now, IMAP mailboxes are intended for email -- "Internet message format" or "Internet text messages" in the RFC language -- and so it would be hard to make a case for storing anything that's not such a message. But RFC822 messages are still rampant on the Internet. In fact, as I understand it, although RFC2822 has obsoleted RFC822, STD0011 (which is identical to RFC822) is still a standard and has not yet been superseded.
And it certainly seems to me that making a copy of an existing message is a "valid reason" for copying it intact, without the modifications needed to force it to conform to the stricter format of RFC2822. Since RFC3501 leaves this decision up to the client, it follows that cyrus is broken when it refuses the message. If RFC3501 said "MUST", then I'd say it's Eudora's responsibility to fix the message before attempting an APPEND. But the RFC says "SHOULD".
Is there any good argument for cyrus' action? If there is, I'd be happy
to take it to Eudora and push them to fix Eudora. Eudora's not exactly
known for its stellar IMAP support, and I'd like them to improve this.
I've shoved the RFCs in their face plenty of times in the past. But in
this case, my reading of the RFCs is that Eudora's APPEND action is
defensible and cyrus' action is incorrect.
If you allow the IMAP server to store arbitrary data, it makes the other functions much more difficult -- if it can't assume there is a message-id that is globally unique, it has to create its own unique key for a message. Searching based on header fields is more problematic since you can't assume there is even a header or that if there is one that it's value corresponds to anything you might expect (ie, character set, line length, and other issues).
Allowing null characters in particular is problematic for any code that uses null-terminated strings for messages or parts of messages, and would require changing the code everywhere to use and pass the length of all the strings instead. I don't know if there is a technical reason behind not supporting bare newlines or not.
As far as STD0011 not being obsoleted, there are plenty of RFCs etc. that are not obsoleted by something but are still not best current practice. Clearly if the RFC it has been based on has been obsoleted, the STD should be updated as well.
-- John A. Tamplin Unix System Administrator Emory University, School of Public Health +1 404/727-9931
