[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Dear Viktor, dear Wietse, thanks again for your vigilant eyes. On 2024-01-05 19:31:35 +0100, Viktor Dukhovni via Postfix-users wrote: On Fri, Jan 05, 2024 at 06:46:01PM +0100, Peter Wienemann via Postfix-users wrote: RFC 2033 says: "The LMTP protocol is identical to the SMTP protocol [SMTP] [HOST-REQ] with its service extensions [ESMTP], except as modified by this document." But, in fact LMTP MX records are only for SMTP relay, and to not apply to either submission or LMTP. I do not find any exceptions mentioned in RFC 2033 concerning the choice of target hosts. Therefore - to my understanding - the same target selection rules apply for SMTP and LMTP. If this is a misunderstanding on my part, please correct me. They do not. MX records specify the location of the inbound SMTP servers for a domain, and there is no reason to expect to find the associated LMTP servers at the same set of servers. I completely overlooked that part. Since the RFC failed to note this distinction, the problem is with the RFC. It seems that I am in good company with my mistake. Obviously the same happened to the author of the Postfix documentation, too. ;-) Here is an excerpt from smtp(8)/lmtp(8): --- The SMTP+LMTP client looks up a list of mail exchanger addresses for the destination host, sorts the list by preference, and connects to each listed address until it finds a server that responds. --- Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Peter Wienemann via Postfix-users: > Dear Wietse, > > thanks for your careful review. > > On 2024-01-05 16:11:56 +0100, Wietse Venema via Postfix-users wrote: > > Peter Wienemann via Postfix-users: > >> smtp(8): > >> > >> The Postfix SMTP+LMTP client supports multiple destinations separated > >> by comma or whitespace (Postfix 3.5 and later). Delivery is tried in the > >> specified order. For each individual destination the delivery rules of > >> RFC 5321, Section 5.1 are applied. > >> > > > > Unfortunately this says that RFC 5321 applies to LMTP deliveries, > > RFC 2033 says: "The LMTP protocol is identical to the SMTP protocol > [SMTP] [HOST-REQ] with its service extensions [ESMTP], except as > modified by this document." Indeed. The PROTOCOL FORMAT is similar to SMTP with small differences. However the USAGE is completely different. 1 - RFC 2033 says that LMTP [...] MUST NOT be used on the TCP service port 25. 2 - MX records specify where a domain receives email on the TCP service port 25. Therefore, MX records MUST NOT be used to locate an LMTP service. I am not going into the rest of the responses. I will only state that Postfix documentation is NOT A TUTORIAL for how email works. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
On Fri, Jan 05, 2024 at 06:46:01PM +0100, Peter Wienemann via Postfix-users wrote: > > Unfortunately this says that RFC 5321 applies to LMTP deliveries, > > RFC 2033 says: "The LMTP protocol is identical to the SMTP protocol [SMTP] > [HOST-REQ] with its service extensions [ESMTP], except as modified by this > document." But, in fact LMTP MX records are only for SMTP relay, and to not apply to either submission or LMTP. > I do not find any exceptions mentioned in RFC 2033 concerning the choice of > target hosts. Therefore - to my understanding - the same target selection > rules apply for SMTP and LMTP. If this is a misunderstanding on my part, > please correct me. They do not. MX records specify the location of the inbound SMTP servers for a domain, and there is no reason to expect to find the associated LMTP servers at the same set of servers. In any case, LMTP delivery is a local matter (replacing ad-hoc local delivery IPC mechanisms), and there is no reason for a sending domain domain to connect to the LMTP servers of an unrelated domain, so the resolution mechanism is unspecified. Indeed, how exactly would you specify unix-domain sockets in MX records? This is why LMTP nexthops are just transport end-points, not logical destinations subject to MX lookup. Since the RFC failed to note this distinction, the problem is with the RFC. -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Dear Wietse, thanks for your careful review. On 2024-01-05 16:11:56 +0100, Wietse Venema via Postfix-users wrote: Peter Wienemann via Postfix-users: smtp(8): The Postfix SMTP+LMTP client supports multiple destinations separated by comma or whitespace (Postfix 3.5 and later). Delivery is tried in the specified order. For each individual destination the delivery rules of RFC 5321, Section 5.1 are applied. Unfortunately this says that RFC 5321 applies to LMTP deliveries, RFC 2033 says: "The LMTP protocol is identical to the SMTP protocol [SMTP] [HOST-REQ] with its service extensions [ESMTP], except as modified by this document." I do not find any exceptions mentioned in RFC 2033 concerning the choice of target hosts. Therefore - to my understanding - the same target selection rules apply for SMTP and LMTP. If this is a misunderstanding on my part, please correct me. and it ignores the possibility that SRV record lookups have been enabled. I will keep the text that delivery is tried in the specified order. Yes, this is a valid point which I missed. Thanks for bringing this up. transport(5): This tries to deliver to bar.example (following the rules of RFC 5321, Section 5.1) before trying to deliver to foo.example. Here I think that the original text is more clear than the updated text, which again implies that RFC 5321 applies to LMTP, In this case delivery via LMTP is explicitly excluded since this refers to the example: example.com smtp:bar.example, foo.example and ignores the possibility of SRV record lookups or [] MX overrides. Yes, SRV record lookups would still be an option. [] MX overrides are incompatible with the given example. Clearly, it is not desirable to enumerate all the SMTP client's email delivery strategies here. Those strategies are orthogonal to the issue of multiple destinations, and they do not belong here. I like your comment. This gives rise to another option: Just refer to the delivery strategy described in smtp(8). This would solve all the problems mentioned above. Although after re-reading smtp(8) in the given context, I think that the cases a) There are no MX records b) MX lookup is switched off c) SRV lookups are enabled are not covered in the "description" section either. At least cases b) and c) can be figured out by looking through subsequent sections. Case a) is also missing in the explanation of the "domainname:port" syntax in section "SMTP destination syntax". Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Peter Wienemann via Postfix-users: > Hi Viktor, > > On 2024-01-02 18:13:22 +0100, Viktor Dukhovni via Postfix-users wrote: > > That said, indeed the documentation is not explicit on this point, one > > has to read "between the lines". If your technical writing skills are > > adequate, perhaps you could suggest some concise and clear text > > explaining the semantics. > > > > This would need to be added to smtp(8), transport(5) and postconf(5), > > with multiple affected parameters in the last case. > > here are my suggestions (I hope I provided enough existing context to > make it clear where to insert the new text): > > smtp(8): > > The Postfix SMTP+LMTP client supports multiple destinations separated > by comma or whitespace (Postfix 3.5 and later). Delivery is tried in the > specified order. For each individual destination the delivery rules of > RFC 5321, Section 5.1 are applied. > Unfortunately this says that RFC 5321 applies to LMTP deliveries, and it ignores the possibility that SRV record lookups have been enabled. I will keep the text that delivery is tried in the specified order. > transport(5): > > This tries to deliver to bar.example (following the rules of RFC 5321, > Section 5.1) before trying to deliver to foo.example. > Here I think that the original text is more clear than the updated text, which again implies that RFC 5321 applies to LMTP, and ignores the possibility of SRV record lookups or [] MX overrides. Clearly, it is not desirable to enumerate all the SMTP client's email delivery strategies here. Those strategies are orthogonal to the issue of multiple destinations, and they do not belong here. > postconf(5): > > Multiple destinations are supported in Postfix 3.5 and later. In this > case delivery is tried in the specified order. For each individual > destination the rules of RFC 5321, Section 5.1 are applied. > Same problem. The text "delivery is tried in the specified order" is correct, the thext that follows is incomplete or misleading. > What I do not like about the above suggestions is the fact that the MX > lookup part of RFC 5321 is skipped if the [] notation is used. Thus > strictly speaking the rules of RFC 5321 are only partially followed in > that case. But since the user explicitly requested Postfix to skip MX > lookup in those cases, I hope that the above additions introduce more > clarity than confusion. There are many cases where MX lookup is not desirable. The most common cases are the submiossion submoissions/smtps services; the less common case is SRV lookups. Wietse > Of course it would be even better to explain the algorithm described in > RFC 5321 with a few words directly in the Postfix documentation rather > than refer to the RFC, but this would be really tough to achieve. > > Best regards, > > Peter > ___ > Postfix-users mailing list -- postfix-users@postfix.org > To unsubscribe send an email to postfix-users-le...@postfix.org > ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Hi Viktor, On 2024-01-02 18:13:22 +0100, Viktor Dukhovni via Postfix-users wrote: That said, indeed the documentation is not explicit on this point, one has to read "between the lines". If your technical writing skills are adequate, perhaps you could suggest some concise and clear text explaining the semantics. This would need to be added to smtp(8), transport(5) and postconf(5), with multiple affected parameters in the last case. here are my suggestions (I hope I provided enough existing context to make it clear where to insert the new text): smtp(8): The Postfix SMTP+LMTP client supports multiple destinations separated by comma or whitespace (Postfix 3.5 and later). Delivery is tried in the specified order. For each individual destination the delivery rules of RFC 5321, Section 5.1 are applied. transport(5): This tries to deliver to bar.example (following the rules of RFC 5321, Section 5.1) before trying to deliver to foo.example. postconf(5): Multiple destinations are supported in Postfix 3.5 and later. In this case delivery is tried in the specified order. For each individual destination the rules of RFC 5321, Section 5.1 are applied. What I do not like about the above suggestions is the fact that the MX lookup part of RFC 5321 is skipped if the [] notation is used. Thus strictly speaking the rules of RFC 5321 are only partially followed in that case. But since the user explicitly requested Postfix to skip MX lookup in those cases, I hope that the above additions introduce more clarity than confusion. Of course it would be even better to explain the algorithm described in RFC 5321 with a few words directly in the Postfix documentation rather than refer to the RFC, but this would be really tough to achieve. Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
On Tue, Jan 02, 2024 at 11:12:28AM +0100, Peter Wienemann via Postfix-users wrote: > To avoid a potential misunderstanding: I do not see any reason to cast doubt > on the RFC compliance of Postfix. I think the issue discussed in this thread > rather goes beyond what is specified in RFCs. It basically boils down to the > question whether the RFC imposed ranking based on DNS record types is > stronger or weaker than the ranking imposed by the order of the specified > relay hosts. I neither found an answer to that question in RFCs nor in the > Postfix documentation (I hope I have not missed anything). Wietse kindly > answered that the RFC imposed ranking is implemented on the level of > individual entries of the relay host list rather than on the ensemble of > relay host entries. Maybe it is worth clarifying this point in the Postfix > documentation. FWIW, it seems clear to me that Postfix should try each destination in turn, with all subsequent destinations as *fallback*, rather than concurrent alteratives. For concurrent alternatives, one would define a single destination with suitable priority MX records. One important use case is to shunt mail that fails the first delivery attempt to a fallback queue that processes "problem" mail. That said, indeed the documentation is not explicit on this point, one has to read "between the lines". If your technical writing skills are adequate, perhaps you could suggest some concise and clear text explaining the semantics. This would need to be added to smtp(8), transport(5) and postconf(5), with multiple affected parameters in the last case. -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Hi Viktor, On 2023-12-29 19:59:42 +0100, Viktor Dukhovni via Postfix-users wrote: RFCs 5321, 2821 and 821 have been around long enough to expect compliance with core elements of the SMTP specification from any mainstream MTA, and particularly Postfix, which appears to be the most widely deployed MTA by some measures. To avoid a potential misunderstanding: I do not see any reason to cast doubt on the RFC compliance of Postfix. I think the issue discussed in this thread rather goes beyond what is specified in RFCs. It basically boils down to the question whether the RFC imposed ranking based on DNS record types is stronger or weaker than the ranking imposed by the order of the specified relay hosts. I neither found an answer to that question in RFCs nor in the Postfix documentation (I hope I have not missed anything). Wietse kindly answered that the RFC imposed ranking is implemented on the level of individual entries of the relay host list rather than on the ensemble of relay host entries. Maybe it is worth clarifying this point in the Postfix documentation. Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
On Fri, Dec 29, 2023 at 07:45:45PM +0100, Peter Wienemann via Postfix-users wrote: > > And then shows some examples that deminstarte that the using > > MX records is mutually exclusive with using address (A or ) records. > > I think what bears the potential for confusion is what you mean by "trying > MX records". In my opinion the example showed that MX lookups are always > tried, even if MX records are not present That's of course unavoidable to determine *whether* they are present. > In that sense MX and A/ (DNS) trials are not mutually exclusive. > What is mutually exclusive is the subsequent delivery attempt. Either > destinations specified by MX records are tried if there are any or > otherwise delivery is attempted to hosts specified by A/ records. As already noted, delivery is mutually exclusive, to either the IP addresses of hosts found in MX records, or to the addresses of the domain if MX lookups return NODATA. RFCs 5321, 2821 and 821 have been around long enough to expect compliance with core elements of the SMTP specification from any mainstream MTA, and particularly Postfix, which appears to be the most widely deployed MTA by some measures. -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Hi Wietse, On 2023-12-29 18:36:59 +0100, Wietse Venema via Postfix-users wrote: Peter Wienemann via Postfix-users: On 2023-12-15 22:17:08 +0100, Wietse Venema via Postfix-users wrote: There is no such thing as falling back to A or records after trying MX records. The two are mutually exclusive. I am confused by the last two sentences. Let us consider a relay host And then shows some examples that deminstarte that the using MX records is mutually exclusive with using address (A or ) records. I think what bears the potential for confusion is what you mean by "trying MX records". In my opinion the example showed that MX lookups are always tried, even if MX records are not present and thus finally destinations specified by A or records are used for delivery attempts. In that sense MX and A/ (DNS) trials are not mutually exclusive. What is mutually exclusive is the subsequent delivery attempt. Either destinations specified by MX records are tried if there are any or otherwise delivery is attempted to hosts specified by A/ records. There is no way that Postfix will deliver to d2 (example2.com) before d1 (example.com). Thanks for this clarification. I think all my questions are answered now. Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Peter Wienemann via Postfix-users: > Dear Wietse, > > On 2023-12-15 22:17:08 +0100, Wietse Venema via Postfix-users wrote: > > Peter Wienemann via Postfix-users: > >> Thanks Wietse! Your pseudo-code clarifies the approach chosen by > >> Postfix. What still remains unclear to me is the order in which > >> destinations are tried. Let us again consider the following example: > >> > >> relayhost = mail1.example.com, mail2.example.com > >> > >> I think RFC 5321 requires that one starts with MX records before falling > >> back to A or records (unless [] notation is used). So my guess is > >> that the trial sequence starts in the following way: > > > > No. Given destinations (d1, d2), the pesudocode is: > > > > for each destination d in (d1, d2): > > > > try to deliver the remaining recipients to destination d > > > > if no recipients remain, stop. > > > > defer any remaining recipients > > > > The "program then "executes" the following steps: > > > > try to deliver the remaining recipients to destination d1 > > > > if no recipients remain, stop. > > > > try to deliver the remaining recipients to destination d2 > > > > if no recipients remain, stop. > > > > defer any remaining recipients. > > > >> (falling back to A/ records after trying MX records) > > > > There is no such thing as falling back to A or records after > > trying MX records. The two are mutually exclusive. > > I am confused by the last two sentences. Let us consider a relay host And then shows some examples that deminstarte that the using MX records is mutually exclusive with using address (A or ) records. > If this is the case, I wonder what happens in the following situation: > > relayhost = example1.com, example2.com > > with the following DNS entries: > > example1.com: > A: ip4 > > example2.com: > MX: host5 (with ip5) > A: ip6 > > Assuming all IPs are accessible, will Postfix deliver emails to ip4 > (because example1.com is first in the relayhost list) or ip5 (because MX > records "win")? WTF. Given two destinations (d1, d2) were: d1 = example1.com d2 = example2.com The pesudocode is: for each destination d in (d1, d2): try to deliver the remaining recipients to destination d if no recipients remain, stop. defer any remaining recipients stop The "program" then "executes" the following steps: try to deliver the remaining recipients to destination d1 if no recipients remain, stop. try to deliver the remaining recipients to destination d2 if no recipients remain, stop. defer any remaining recipients. stop There is no way that Postfix will deliver to d2 (example2.com) before d1 (example.com). If these abstractions are too difficult, then perhaps you need to find a different line of bnusiness. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Dear Wietse, On 2023-12-15 22:17:08 +0100, Wietse Venema via Postfix-users wrote: Peter Wienemann via Postfix-users: Thanks Wietse! Your pseudo-code clarifies the approach chosen by Postfix. What still remains unclear to me is the order in which destinations are tried. Let us again consider the following example: relayhost = mail1.example.com, mail2.example.com I think RFC 5321 requires that one starts with MX records before falling back to A or records (unless [] notation is used). So my guess is that the trial sequence starts in the following way: No. Given destinations (d1, d2), the pesudocode is: for each destination d in (d1, d2): try to deliver the remaining recipients to destination d if no recipients remain, stop. defer any remaining recipients The "program then "executes" the following steps: try to deliver the remaining recipients to destination d1 if no recipients remain, stop. try to deliver the remaining recipients to destination d2 if no recipients remain, stop. defer any remaining recipients. (falling back to A/ records after trying MX records) There is no such thing as falling back to A or records after trying MX records. The two are mutually exclusive. I am confused by the last two sentences. Let us consider a relay host entry "example.com". Case 1: "example.com" has the following DNS entries: MX: host1 (with ip1) A: ip2 : ip3 My understanding of RFC 5321, Section 5.1 is that delivery has to be attempted to ip1. If ip1 is unavailable, retry later until ip1 becomes available or give-up time has been reached. In that case A and records will never be used. If the relayhost entry is relayhost = [example.com] either ip2 or ip3 is tried for delivery. I suppose that in that case it depends on the "smtp_address_preference" setting whether ip2 or ip3 will "win". Case 2: "example.com" has the following DNS entries: A: ip2 : ip3 An MX record is checked for example.com and returns no entry. In that case RFC 5321 foresees (at least as I understand it) to fall back to A or entries ("implicit MX" rule). That means in that case either ip2 or ip3 is tried for delivery (similar to the [example.com] case). Does the above summarize Postfix's behaviour correctly? If this is the case, I wonder what happens in the following situation: relayhost = example1.com, example2.com with the following DNS entries: example1.com: A: ip4 example2.com: MX: host5 (with ip5) A: ip6 Assuming all IPs are accessible, will Postfix deliver emails to ip4 (because example1.com is first in the relayhost list) or ip5 (because MX records "win")? Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Peter Wienemann via Postfix-users: > On 2023-12-12 15:51:58 +0100, Wietse Venema via Postfix-users wrote: > > Peter Wienemann via Postfix-users: > >> Dear Postfix experts, > >> > >> checking the documentation for the relayhost parameter [0] I find no > >> indication how Postfix behaves in case of multiple relay hosts with > >> multiple DNS entries. Let us assume the following setting: > > > > for each destination d in relayhost: > > > > try to deliver the remaining recipients to destination d > > > > if no recipients remain, stop. > > > > defer any remaining recipients > > > > Perhaps surprisingly, Postfix delivers mail to "destination d" for > > all supported forms of "d", whether that is a domain name, [hostname], > > [ipaddress], and with or without an explicit :port. > > Thanks Wietse! Your pseudo-code clarifies the approach chosen by > Postfix. What still remains unclear to me is the order in which > destinations are tried. Let us again consider the following example: > > relayhost = mail1.example.com, mail2.example.com > > I think RFC 5321 requires that one starts with MX records before falling > back to A or records (unless [] notation is used). So my guess is > that the trial sequence starts in the following way: No. Given destinations (d1, d2), the pesudocode is: for each destination d in (d1, d2): try to deliver the remaining recipients to destination d if no recipients remain, stop. defer any remaining recipients The "program then "executes" the following steps: try to deliver the remaining recipients to destination d1 if no recipients remain, stop. try to deliver the remaining recipients to destination d2 if no recipients remain, stop. defer any remaining recipients. > (falling back to A/ records after trying MX records) There is no such thing as falling back to A or records after trying MX records. The two are mutually exclusive. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
On 2023-12-12 15:51:58 +0100, Wietse Venema via Postfix-users wrote: Peter Wienemann via Postfix-users: Dear Postfix experts, checking the documentation for the relayhost parameter [0] I find no indication how Postfix behaves in case of multiple relay hosts with multiple DNS entries. Let us assume the following setting: for each destination d in relayhost: try to deliver the remaining recipients to destination d if no recipients remain, stop. defer any remaining recipients Perhaps surprisingly, Postfix delivers mail to "destination d" for all supported forms of "d", whether that is a domain name, [hostname], [ipaddress], and with or without an explicit :port. Thanks Wietse! Your pseudo-code clarifies the approach chosen by Postfix. What still remains unclear to me is the order in which destinations are tried. Let us again consider the following example: relayhost = mail1.example.com, mail2.example.com I think RFC 5321 requires that one starts with MX records before falling back to A or records (unless [] notation is used). So my guess is that the trial sequence starts in the following way: 1. Try all MX records for mail1.example.com. 2. What comes next? Try all MX records for mail2.example.com? Or try A records for mail1.example.com? Or try records for mail1.example.com? Or is some randomness involved? Best regards, Peter ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Behaviour in case of multiple relay hosts with multiple DNS records
Peter Wienemann via Postfix-users: > Dear Postfix experts, > > checking the documentation for the relayhost parameter [0] I find no > indication how Postfix behaves in case of multiple relay hosts with > multiple DNS entries. Let us assume the following setting: for each destination d in relayhost: try to deliver the remaining recipients to destination d if no recipients remain, stop. defer any remaining recipients Perhaps surprisingly, Postfix delivers mail to "destination d" for all supported forms of "d", whether that is a domain name, [hostname], [ipaddress], and with or without an explicit :port. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org