I'm going to go and get Cyrus and play with their lmtp server and see what differs. Postfix seems to be strict about a few little things... it's possible that they'd only tested with Cyrus or a small handful of others and therefore have some extra assumptions about what an LMTP session is supposed to look like. RFC's and Real Life Don't Always Match (TM).
So, for example, when I was talking smtp directly with Postfix, the MAIL FROM: and RCPT TO: commands were only accepted with the colons, ':', and otherwise generated an error. IMHO, the RFC suggests that little "errors" like that should be accepted anyhow. Aaron Ilja Booij <[EMAIL PROTECTED]> said: > Hmm, I still don't really understand the problem. I've done some > ngrep-ing, to see what goes over the network. > > This is a session that's OK: > > T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 220 test01 DBMail LMTP service ready to rock.. > > ## > T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > LHLO test01.office.fastxs.net.. > > ## > T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE.. > > # > T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > MAIL FROM:<[EMAIL PROTECTED]> SIZE=866..RCPT > TO:<[EMAIL PROTECTED]>..DATA.. > # > T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Sender <[EMAIL PROTECTED]> OK.. > > ## > T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Recipient <[EMAIL PROTECTED]> OK..354 Start mail > input; end with <CRLF>.<CRLF>.. > ## > T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > Received: from earthquake.office.fastxs.net > (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n > et (Postfix) with ESMTP id 394DD19F30A...for > <[EMAIL PROTECTED]>; Fri, 5 Mar 2004 13:45:46 +0100 (C > ET)..Received: from earthquake.office.fastxs.net (localhost > [127.0.0.1])...by earthquake.office.fastxs.net (Postfi > x) with ESMTP id 405D4375FE...for <[EMAIL PROTECTED]>; > Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Receiv > ed: (from [EMAIL PROTECTED])...by earthquake.office.fastxs.net > (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@ > test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Date: > Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From: > Ilja Booij <[EMAIL PROTECTED]>..Message-Id: > <[EMAIL PROTECTED] > net>..To: [EMAIL PROTECTED]: 5-3 > 27......27..... > ## > T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Message received OK.. > > ## > T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Recipient <[EMAIL PROTECTED]> OK.. > > > > If another message is sent (over the same connection): > > > ## > T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > RSET.. > > ## > T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > MAIL FROM:<[EMAIL PROTECTED]> SIZE=858..RCPT > TO:<[EMAIL PROTECTED]>..DATA.. > ## > T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 OK.. > > ## > T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Sender <[EMAIL PROTECTED]> OK.. > > ## > T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 250 Recipient <[EMAIL PROTECTED]> OK.. > > ## > T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 354 Start mail input; end with <CRLF>.<CRLF>.. > > ## > T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP] > RSET..QUIT.. > > ### > T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP] > 500 Error reading header... > > # > > You can see that the client (postfix) starts with an RSET command, and > starts sending MAIL_FROM, RCPT and DATA. > The server responds with 250 OK to the reset, and 250 Sender OK and 250 > Recipient OK, and lets the client start sending (354 Start mail input). > > The problem is that Postfix detects that it should not send data, > because the message is supposedly bounced by dbmail.. But why? > > Ilja > > Ilja Booij wrote: > > > Hi, > > > > Robert, thanks for your suggestion. It seems to work perfectly here. > > Messages now get delivered to users without a problem. > > > > I'll take a look at lmtp.c to see if I can spot the difference between > > the server getting a QUIT from the client and a reconnection on a new > > message, and a cached connection. > > > > Ilja > > > > > > Robert Theisen wrote: > > > >> New to the list here. But I read over the archives regarding this > >> problem and I may have a little to add. > >> > >> I was able to get postfix to consistently deliver via lmtp to dbmail > >> (even two consecutive messages to the same user) by setting the > >> postfix directive lmtp_cache_connection to false instead of true. > >> This would termintate the lmtp connection after each delivery. I > >> assume, with lmtp_cache_connection turned on, the connection would die > >> after lmtp_timeout was reached and a new connection would be > >> established and the mail would get sent. This may have been a "neat > >> feature" that works with Cyrus (postfix's preferred lmtp client) or it > >> may be a specification of the lmtp rfc (I have not read it). But it > >> seems like a simple enough fix to make it work. It would probably > >> just involve having the conversation status fall back to a state the > >> postfix expects after message delivery and waiting in that state until > >> either the connection terminates or the conversation picks up again. > >> I took a cursory glance at the lmtp.c and lmtpd.c code but did not > >> have enough time to really dig into it. > >> Good Luck. I am anxiously awaiting the 2.0 release. :) > >> > >> Robert Theisen > >> > >> Aaron Stone wrote: > >> > >>> Sure thing, I've got about an hour of time this morning. Looks like > >>> you're > >>> burning the midnight oil over there! Yikes! > >>> > >>> Aaron > >>> > >>> > >>> Ilja Booij <[EMAIL PROTECTED]> said: > >>> > >>> > >>> > >>>> Hi, > >>>> > >>>> I haven't been able to find the cause of the problem yet. I found > >>>> some more info in the logs though: > >>>> > >>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A: > >>>> to=<[EMAIL PROTECTED]>, > >>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host > >>>> localhost.fastxs.net[127.0.0.1] said: 250 Recipient > >>>> <[EMAIL PROTECTED]> OK) > >>>> > >>>> apparently, postfix thinks we want to bounce the message. But why? A > >>>> previous message to the same user arrives correctly.. > >>>> > >>>> strange, looking further. Aaron, can you also take a look at this? > >>>> > >>>> Ilja > >>>> > >>>> Ilja Booij wrote: > >>>> > >>>> > >>>> > >>>>> OK, I've found something: > >>>>> > >>>>> It seems that Postfix sends a RSET command, when we expect to get > >>>>> the header. > >>>>> > >>>>> readheader() then waits for postfix to send more, while postfix > >>>>> waits for a '250 OK' Message from dbmail-lmtp. > >>>>> > >>>>> So, there's probably something wrong in our LMTP-logic. > >>>>> I'll take a look at it, after I've done some small scripting job > >>>>> here for a customer. > >>>>> > >>>>> Ilja > >>>>> > >>>>> Ilja Booij wrote: > >>>>> > >>>>> > >>>>> > >>>>>> OK, > >>>>>> > >>>>>> this seems to have tackled the problem of the double delivery :) > >>>>>> > >>>>>> There still is another problem though: > >>>>>> It still seems to hang for a while on every second delivery > >>>>>> attempt to the LMTP daemon. > >>>>>> > >>>>>> Can you try the following scenario: > >>>>>> > >>>>>> * configure MTA to use dbmail-lmtp for delivery > >>>>>> * send message to a recipient > >>>>>> * verify that the message is received using dbmail-lmtp > >>>>>> * send a second message. > >>>>>> > >>>>>> What I observe here is that the second message takes a lot longer > >>>>>> to be delivered than the first one. The second one is only > >>>>>> delivered after minutes. > >>>>>> > >>>>>> I have the feeling that something is not right here.. > >>>>>> > >>>>>> Ilja > >>>>>> > >>>>>> Aaron Stone wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>>> New versions checked in, though not extensively tested yet. > >>>>>>> > >>>>>>> Ilja Booij <[EMAIL PROTECTED]> said: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> sounds ok to me :) > >>>>>>>> > >>>>>>>> Ilja > >>>>>>>> > >>>>>>>> Aaron Stone wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> Working on it as we speak! Catch you in about an hour? > >>>>>>>>> > >>>>>>>>> Aaron > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Ilja Booij <[EMAIL PROTECTED]> said: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> Aaron Stone wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is > >>>>>>>>>>> because the old > >>>>>>>>>>> insert_messages() function used to take lists of users to > >>>>>>>>>>> directly deliver. > >>>>>>>>>>> The new insert_messages() function takes a list of dsnuser > >>>>>>>>>>> structures, and > >>>>>>>>>>> expects that *either* the useridnr field is filled (for > >>>>>>>>>>> direct-to-useridnr > >>>>>>>>>>> delivery, which isn't supported by any of the frontends, but > >>>>>>>>>>> is supported in > >>>>>>>>>>> the lower layers ;-) or the address field, which is first > >>>>>>>>>>> checked as a > >>>>>>>>>>> username and then as an alias (this allows usernames in the > >>>>>>>>>>> form of > >>>>>>>>>>> '[EMAIL PROTECTED]' to operate without requiring an alias that > >>>>>>>>>>> says > >>>>>>>>>>> '[EMAIL PROTECTED] > >>>>>>>>>>> delivers to [EMAIL PROTECTED]'). > >>>>>>>>>>> > >>>>>>>>>>> Some refactoring might be necessary, because we need to > >>>>>>>>>>> inform the MTA > >>>>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> whether > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>>>>> or not its envelope recipients were valid before it will send > >>>>>>>>>>> us the message. > >>>>>>>>>>> This means users need to be validated before read_header(), > >>>>>>>>>>> which is a > >>>>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> problem > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>>>>> because at the moment this vaildation happens in > >>>>>>>>>>> insert_messages(). > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> OK, that's clear > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the message > >>>>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> regardless of > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>>>>> the disposition of the recipients, and only at the very end > >>>>>>>>>>> are error > >>>>>>>>>>> conditions returned as exit codes... in LMTP land we need to > >>>>>>>>>>> know ahead of > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> time. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> I think what I'll do is move the user validation code from > >>>>>>>>>>> insert_messages() > >>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or > >>>>>>>>>>> dsn_prepare_deliveries(). > >>>>>>>>>>> Then, we'll have this order: > >>>>>>>>>>> > >>>>>>>>>>> For command-line and envelope-recipient deliveries: > >>>>>>>>>>> - receive addresses and store them into the dsnusers list. > >>>>>>>>>>> - dsn_prepare_deliveries() > >>>>>>>>>>> - if no deliveries are valid, return an error. > >>>>>>>>>>> > >>>>>>>>>>> - read_header() > >>>>>>>>>>> > >>>>>>>>>>> For deliver-to header deliveries: > >>>>>>>>>>> - mime_readheader() > >>>>>>>>>>> - mail_adr_list() > >>>>>>>>>>> - dsn_prepare_deliveries() > >>>>>>>>>>> - if no deliveries are valid, return an error. > >>>>>>>>>>> > >>>>>>>>>>> - insert_messages() > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> I like the part where you say 'What I'll do' :) > >>>>>>>>>> Do you think this work will take long? I think we must really > >>>>>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really > >>>>>>>>>> has to work if we want to release. > >>>>>>>>>> > >>>>>>>>>> Ilja > >>>>>>>>>> > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> Dbmail-dev mailing list > >>>>>>>>>> Dbmail-dev@dbmail.org > >>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> _______________________________________________ > >>>>>>>> Dbmail-dev mailing list > >>>>>>>> Dbmail-dev@dbmail.org > >>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> Dbmail-dev mailing list > >>>>>> Dbmail-dev@dbmail.org > >>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > >>>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> Dbmail-dev mailing list > >>>>> Dbmail-dev@dbmail.org > >>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > >>>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> Dbmail-dev mailing list > >>>> Dbmail-dev@dbmail.org > >>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > >>>> > >>>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> > >> _______________________________________________ > >> Dbmail-dev mailing list > >> Dbmail-dev@dbmail.org > >> http://twister.fastxs.net/mailman/listinfo/dbmail-dev > > > > _______________________________________________ > > Dbmail-dev mailing list > > Dbmail-dev@dbmail.org > > http://twister.fastxs.net/mailman/listinfo/dbmail-dev > _______________________________________________ > Dbmail-dev mailing list > Dbmail-dev@dbmail.org > http://twister.fastxs.net/mailman/listinfo/dbmail-dev > --
