[ On Tuesday, May 31, 2005 at 10:12:10 (+0200), Patrik Henningsson wrote: ] > Subject: Re: Double Carriage return breaks header .. > > So now we know that Cyrus presumably is doing wrong, any brave soul > contribute with a patch?
I may try my hand at a patch, though not today and not likely this week. BTW, here's some more detail, and yet another problem, about all this EOL business. The big worry in SMTP is, and always has been, the recognition of a bare <LF> as a line terminator (and thus "<LF>.<LF>" as end of message), _not_ bare <CR> -- so far as I know there has never been any useful SMTP implementation that recognized bare <CR> as a line terminator. The problem with recognizing bare <LF> as EOL being that the client-SMTP cannot know whether or not to do dot-stuffing to protect the dot in a "<LF>.<LF>" sequence it wants to send. If it guesses wrong about the server it's talking to then it either risks message truncation or message corruption (adding an extra, unwanted, "."). Despite the fact that RFC 2821 has explicitly forbidden new conforming implementations from recognizing anything but <CRLF> as a line terminator the "Right(tm)" solution to this issue is of course to always MIME encode any message part containing the "<LF>.<LF>" sequence in such a way that the sequence no longer appears that way in the SMTP data, since of course one can no more predict the RFC 2821 conformance of a server-SMTP than once can predict it's use of <LF>. I.e. RFC 2821's rule is completely useless here -- too many servers will likely always recognize "<LF>.<LF>" as the end of a message -- MIME encoding is the only true solution. Presumably though Cyrus wouldn't have to worry about any of this if its LMTP server only recognized <CRLF> as the line terminator, as per RFC 2821 (though RFC 2033 is only "just" updated to point to RFC 2821 courtesy RFC 3848), and thus only allowed "<CRLF>.<CRLF>" as the end of the message. Unfortunately this is yet another problem with Cyrus (or another manifestation of the same problem, or maybe just a clearer illustration of the same problem) -- its LMTP engine not only stuffs newlines before bare <CR>s in the headers (thus corrupting message headers as we've been discussing in this thread), but it also recognizes the "<LF>.<LF>" sequence as the end of the message (thus also risking message truncation if the client-LMTP doesn't do dot-stuffing): # echo -e "LHLO building.weird.com\r\nMAIL FROM:<[EMAIL PROTECTED]>\r\nRCPT TO:<[EMAIL PROTECTED]>\r\nDATA\r\nTo: gwoods\r\nSubject: TEST\r\r\nX-header: bar\r\n\r\n\n.\ncrline\r\r\nfoo\r\n\r\n.\r\nQUIT\r" | nc -U /var/imap/socket/lmtp 220 building LMTP Cyrus v2.2.10 ready 250-building 250-8BITMIME 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-SIZE 4000000 250-AUTH EXTERNAL 250 IGNOREQUOTA 250 2.1.0 ok 250 2.1.5 ok 354 go ahead 250 2.1.5 Ok 500 5.5.2 Syntax error 500 5.5.2 Syntax error 500 5.5.2 Syntax error 500 5.5.2 Syntax error 221 2.0.0 bye # od -c /var/spool/imap/w/user/woods/44. 0000000 R e t u r n - P a t h : < w o 0000020 o d s @ w e i r d . c o m > \r \n 0000040 R e c e i v e d : f r o m b 0000060 u i l d i n g . w e i r d . c o 0000100 m ( [ u n i x s o c k e t ] 0000120 ) \r \n \t b y b u i l d i n g 0000140 ( C y r u s v 2 . 2 . 1 0 ) 0000160 w i t h L M T P A ; \r \n \t 0000200 T u e , 3 1 M a y 2 0 0 5 0000220 1 7 : 1 2 : 4 2 - 0 4 0 0 \r 0000240 \n X - S i e v e : C M U S i 0000260 e v e 2 . 2 \r \n T o : g w o 0000300 o d s \r \n S u b j e c t : T E 0000320 S T \r \n M e s s a g e - I D : 0000340 < c m u - l m t p d - 2 2 9 7 2 0000360 - 1 1 1 7 5 7 3 9 6 2 - 0 @ b u 0000400 i l d i n g > \r \n D a t e : T 0000420 u e , 3 1 M a y 2 0 0 5 0000440 1 7 : 1 2 : 4 2 - 0 4 0 0 \r \n 0000460 \r \n X - h e a d e r : b a r \r 0000500 \n \r \n \r \n 0000505 Note there are no lines "crline" or "foo" in the body, and presumably they're the cause of the "syntax error" responses during the transaction. And also note the problem of the headers ending before the "X-header:", finally clearly reproducing the problem that was reported at the start of this thread. Removing the "\n.\n" from the data stream still reproduces the split headers problem, but also reveals that the bare <CR> botch inserts a newline in the headers, not in the body -- "crline" is not followed by a blank line, but it is missing the extra <CR> that should be there: # echo -e "LHLO building.weird.com\r\nMAIL FROM:<[EMAIL PROTECTED]>\r\nRCPT TO:<[EMAIL PROTECTED]>\r\nDATA\r\nTo: gwoods\r\nSubject: TEST\r\r\nX-header: bar\r\n\r\ncrline\r\r\nfoo\r\n\r\n.\r\nQUIT\r" | nc -U /var/imap/socket/lmtp 220 building LMTP Cyrus v2.2.10 ready 250-building 250-8BITMIME 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-SIZE 4000000 250-AUTH EXTERNAL 250 IGNOREQUOTA 250 2.1.0 ok 250 2.1.5 ok 354 go ahead 250 2.1.5 Ok 221 2.0.0 bye # od -c /var/spool/imap/w/user/woods/45. 0000000 R e t u r n - P a t h : < w o 0000020 o d s @ w e i r d . c o m > \r \n 0000040 R e c e i v e d : f r o m b 0000060 u i l d i n g . w e i r d . c o 0000100 m ( [ u n i x s o c k e t ] 0000120 ) \r \n \t b y b u i l d i n g 0000140 ( C y r u s v 2 . 2 . 1 0 ) 0000160 w i t h L M T P A ; \r \n \t 0000200 T u e , 3 1 M a y 2 0 0 5 0000220 1 7 : 2 3 : 0 8 - 0 4 0 0 \r 0000240 \n X - S i e v e : C M U S i 0000260 e v e 2 . 2 \r \n T o : g w o 0000300 o d s \r \n S u b j e c t : T E 0000320 S T \r \n M e s s a g e - I D : 0000340 < c m u - l m t p d - 2 3 0 4 4 0000360 - 1 1 1 7 5 7 4 5 8 8 - 0 @ b u 0000400 i l d i n g > \r \n D a t e : T 0000420 u e , 3 1 M a y 2 0 0 5 0000440 1 7 : 2 3 : 0 8 - 0 4 0 0 \r \n 0000460 \r \n X - h e a d e r : b a r \r 0000500 \n \r \n c r l i n e \r \n f o o \r \n 0000520 \r \n 0000522 At least Cyrus LMTP doesn't accept <CR> as EOL anywhere but in the headers. Small consolation, but something.... I guess the question "we" need to ask is whether there are any existing client-LMTP implementations which might send just <LF> as their end-of-line terminator. I should hope not.... I'm going to guess that humans don't test LMTP by hand nearly so often as they do SMTP, so the need for recognizing bare <LF> as a line terminator, for use with telnet for example, should be nearly nil. -- Greg A. Woods H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <[EMAIL PROTECTED]> Planix, Inc. <[EMAIL PROTECTED]> Secrets of the Weird <[EMAIL PROTECTED]> --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html