Re: Larning about relays
On 2012-06-28 email builder wrote: Do not accept mail that you know you can't deliver. Either use $relay_recipient_maps (building the list and pushing it to the frontend server can be implemented with a simple script), use LDAP lookups (in case your recipients are listed in an LDAP directory), or use the reject_unverified_recipient restriction [1]. So either way, the relay server ends up with a list of valid destination addresses right? Looks like there's just no other correct way to do it. Right? There may be other ways, but I'm not aware of any. OK. To be clear, if one were to keep the recipient list out of the hands of the relay server, the only reasonable thing to do in that scenario is to silently discard all bounces from the internal MTA, right? The reasonable thing to do in a situation like that would be to re-evaluate the reasoning that led to the decision to not have the relay know about valid recipients. This avoids backscatter but no one would get the helpful bounces in the case of address typos, user being over quota, etc. Yes. Regards Ansgar Wiechers -- Abstractions save us time working, but they don't save us time learning. --Joel Spolsky
Re: Larning about relays
Thanks for the help everyone. Does anyone have experience or comments about this: The only other solution I can see is to use something like Nginx to just proxy the SMTP conversation through the relay to the internal MTA. I haven't tried this before, but if Nginx can perform fast enough and can also be DNS load balanced in connecting to internal MTAs, would it work? If the relay is not near the internal MTAs, will network latency make this a dubious choice? As I understand it, the advantage to that setup would be that the internal MTAs could be configured as if they were public facing themselves. Yes?
Re: Larning about relays
email builder: Thanks for the help everyone.? Does anyone have experience or comments about this: The only other solution I can see is to use something like Nginx to just proxy the SMTP conversation through the relay to the internal MTA.? I haven't tried this before, but if Nginx can perform fast enough and can also be DNS load balanced in connecting to internal MTAs, would it work?? If the relay is not near the internal MTAs, will network latency make this a dubious choice? As I understand it, the advantage to that setup would be that the internal MTAs could be configured as if they were public facing themselves.? Yes? Postfix has the hooks to work with nginx or haproxy. Postfix 2.9 adds supports nxginx's built-in SASL authentication, and Postfix 2.10 adds support for haproxy. I have no experience with such set-ups, and have no information about whether they will help under conditions of Postfix overload, or whether they will make the problem worse. Wietse
Re: Larning about relays
Specific questions I had were if I can use the standard DNS load balancing (multiple MX records, same priority, possibly multiple IPs resolving to one A record) setup *behind* a relay server (those MTAs behind the relay only being available via the relay and never directly). Well, it looks like I could do relay_domains = example.com transport table: example.com relay:other.com I have to use other.com in the transport because I need to use DNS-based load balancing of multiple SMTP servers on the backend, but the relay is listed as the primary MX for example.com (my understanding is that I can't do this: example.com relay:internal.smtp.example.com because mail would loop, right? So will using another (essentially junk) domain work for this? Also need to make sure I know how address verification works when ideally the relay doesn't have access to the list of (virtual) users. It looks like with relay_recipient_maps empty, all mail just gets sent through the relay to the main SMTP server, which is fine. But when the main SMTP server gets mail to invalid recipients, how does it reject it? Am I correct to assume that because the relay server already did the SMTP conversation with the client, the main SMTP server can't do any real time address checks? This seems less than ideal. Also, it looks like any bounces or anything else that the main SMTP server needs to send back go through the relay if I specify it in either the relayhost setting or if I put it in the transport table with: * smtp:relay.example.com Is there a difference between that and relayhost? Which is better? Do both methods force postfix to send anything outgoing (bounces OR outgoing user email OR anything else) through that given host? And assuming I can run postscreen on the relay and not need to do much connection, client and RBL filtering on the main MTAs (except address related since the relay won't have access to local addresses). I still need to learn about this more, I don't know how much of my recipient/client/helo (etc.) restrictions I can put on the relay and how the main SMTP server will handle those restrictions that the relay can't handle when it is being relayed to indirectly.
Re: Larning about relays
On 2012-06-28 email builder wrote: Specific questions I had were if I can use the standard DNS load balancing (multiple MX records, same priority, possibly multiple IPs resolving to one A record) setup *behind* a relay server (those MTAs behind the relay only being available via the relay and never directly). Well, it looks like I could do relay_domains = example.com transport table: example.com relay:other.com I have to use other.com in the transport because I need to use DNS-based load balancing of multiple SMTP servers on the backend, but the relay is listed as the primary MX for example.com (my understanding is that I can't do this: example.com relay:internal.smtp.example.com because mail would loop, right? So will using another (essentially junk) domain work for this? As long as internal.smtp.example.com IS the final destination for example.com and the relaying server is NOT the final destination for example.com, the mail should not loop. Also need to make sure I know how address verification works when ideally the relay doesn't have access to the list of (virtual) users. It looks like with relay_recipient_maps empty, all mail just gets sent through the relay to the main SMTP server, which is fine. No, that ain't fine, because ... But when the main SMTP server gets mail to invalid recipients, how does it reject it? It doesn't reject it. Instead it generates bounces, which - in case of spam - will hit innocent bystanders whose addresses were spoofed. Do not accept mail that you know you can't deliver. Either use $relay_recipient_maps (building the list and pushing it to the frontend server can be implemented with a simple script), use LDAP lookups (in case your recipients are listed in an LDAP directory), or use the reject_unverified_recipient restriction [1]. Am I correct to assume that because the relay server already did the SMTP conversation with the client, the main SMTP server can't do any real time address checks? This seems less than ideal. The main server will do real time address checks. For the transaction between frontend and backend, that is. However, since SMTP defines a store-and-forward architecture, mail transactions aren't end-to-end, but hop-to-hop. Also, it looks like any bounces or anything else that the main SMTP server needs to send back go through the relay if I specify it in either the relayhost setting or if I put it in the transport table with: * smtp:relay.example.com Is there a difference between that and relayhost? Which is better? Do both methods force postfix to send anything outgoing (bounces OR outgoing user email OR anything else) through that given host? If I understand man 5 transport correctly, both settings should do the same. I would prefer $relayhost for better readability, though. And assuming I can run postscreen on the relay and not need to do much connection, client and RBL filtering on the main MTAs (except address related since the relay won't have access to local addresses). I still need to learn about this more, I don't know how much of my recipient/client/helo (etc.) restrictions I can put on the relay and how the main SMTP server will handle those restrictions that the relay can't handle when it is being relayed to indirectly. All of your restrictions should be implemented on the frontend server, and the backend server should accept mail only from trusted locations. [1] http://www.postfix.org/postconf.5.html#reject_unverified_recipient Regards Ansgar Wiechers -- Abstractions save us time working, but they don't save us time learning. --Joel Spolsky
Re: Larning about relays
Specific questions I had were if I can use the standard DNS load balancing (multiple MX records, same priority, possibly multiple IPs resolving to one A record) setup *behind* a relay server (those MTAs behind the relay only being available via the relay and never directly). Well, it looks like I could do relay_domains = example.com transport table: example.com relay:other.com I have to use other.com in the transport because I need to use DNS-based load balancing of multiple SMTP servers on the backend, but the relay is listed as the primary MX for example.com (my understanding is that I can't do this: example.com relay:internal.smtp.example.com because mail would loop, right? So will using another (essentially junk) domain work for this? As long as internal.smtp.example.com IS the final destination for example.com and the relaying server is NOT the final destination for example.com, the mail should not loop. I'm not sure whether or not you understood me or if we are in agreement. smtp.example.com is will be the relay server, which is the primary MX. However, multiple SMTP servers behind it need to be DNS-load balanced. So if I do this, mail should loop because the current (relay) server is, as far as DNS MX records are concerned, the correct destination server: example.com relay:internal.smtp.example.com I can't use brackets [] to disable MX lookups since I need the DNS load balancing. Thus I thought to use a different junk domain name example.com relay:other.tld Also need to make sure I know how address verification works when ideally the relay doesn't have access to the list of (virtual) users. It looks like with relay_recipient_maps empty, all mail just gets sent through the relay to the main SMTP server, which is fine. No, that ain't fine, because ... But when the main SMTP server gets mail to invalid recipients, how does it reject it? It doesn't reject it. Instead it generates bounces, which - in case of spam - will hit innocent bystanders whose addresses were spoofed. Do not accept mail that you know you can't deliver. Either use $relay_recipient_maps (building the list and pushing it to the frontend server can be implemented with a simple script), use LDAP lookups (in case your recipients are listed in an LDAP directory), or use the reject_unverified_recipient restriction [1]. So either way, the relay server ends up with a list of valid destination addresses right? Looks like there's just no other correct way to do it. Right? Am I correct to assume that because the relay server already did the SMTP conversation with the client, the main SMTP server can't do any real time address checks? This seems less than ideal. The main server will do real time address checks. For the transaction between frontend and backend, that is. However, since SMTP defines a store-and-forward architecture, mail transactions aren't end-to-end, but hop-to-hop. OK, so I'll have to ensure that postscreen on the relay can do as much as possible. In this situation, the main/real SMTP server on the backend SHOULD NOT do any of those kinds of checks, because they'd be checking the relay instead of the real sending client. It seems like the real backend SMTP server basically shouldn't be doing any verification at all (maybe just internal anti spam/virus content filtering)??? . and certainly should not be bouncing anything that fails its tests (because of backscatter concerns)
Re: Larning about relays
On 2012-06-28 email builder wrote: Well, it looks like I could do relay_domains = example.com transport table: example.com relay:other.com I have to use other.com in the transport because I need to use DNS-based load balancing of multiple SMTP servers on the backend, but the relay is listed as the primary MX for example.com (my understanding is that I can't do this: example.com relay:internal.smtp.example.com because mail would loop, right? So will using another (essentially junk) domain work for this? As long as internal.smtp.example.com IS the final destination for example.com and the relaying server is NOT the final destination for example.com, the mail should not loop. I'm not sure whether or not you understood me or if we are in agreement. There is a third option. smtp.example.com is will be the relay server, which is the primary MX. However, multiple SMTP servers behind it need to be DNS-load balanced. So if I do this, mail should loop because the current (relay) server is, as far as DNS MX records are concerned, the correct destination server: example.com relay:internal.smtp.example.com You're confusing DNS and MTA configuration. MX records tell the world where to send mail for example.com. However, the MX record does NOT magically make an MTA running on that host the final destination for example.com SMTP-wise. An MTA becomes the final destination for example.com when the domain is configured in either $mydestination, $virtual_alias_domains or $virtual_mailbox_domains. It is NOT the final destination for any domain configured in $relay_domains. There's also nothing wrong with suppressing MX lookups. You don't need MX records for load-balancing mail sent to backend servers. Example: 8 $ORIGIN example.com @ IN MX 10 external1 @ IN MX 10 external2 external1 IN A 10.42.23.11 external2 IN A 10.42.23.12 internal IN A 10.42.23.101 internal IN A 10.42.23.102 8 The above snippet defines two MX records external1.example.com and external2.example.com. These are the hosts that the world should send mail for example.com to. Since both records are defined with the same preference, inbound mail will be (DNS-)load-balanced over the two servers. Both servers run Postfix with $relay_domains = example.com and a transport map example.com relay:[internal.example.com] This will cause the two frontend servers to relay all mail for example.com to internal.example.com. No MX lookup is required for this, because you already know (and control) the destination host. And resolution of the A record internal.example.com will automatically load-balance the delivery, because the A record is configured with more than one address (round-robin DNS). Finally, the two backend servers 10.42.23.101 and 10.42.23.102 also run Postfix, but with a configuration like $mydestination = example.com $relay_domains = or $virtual_mailbox_domains = example.com $relay_domains = making them the final destination for SMTP delivery. I can't use brackets [] to disable MX lookups since I need the DNS load balancing. Thus I thought to use a different junk domain name example.com relay:other.tld As explained above: you don't need MX lookups for DNS load-balancing in the backend. [...] But when the main SMTP server gets mail to invalid recipients, how does it reject it? It doesn't reject it. Instead it generates bounces, which - in case of spam - will hit innocent bystanders whose addresses were spoofed. Do not accept mail that you know you can't deliver. Either use $relay_recipient_maps (building the list and pushing it to the frontend server can be implemented with a simple script), use LDAP lookups (in case your recipients are listed in an LDAP directory), or use the reject_unverified_recipient restriction [1]. So either way, the relay server ends up with a list of valid destination addresses right? Looks like there's just no other correct way to do it. Right? There may be other ways, but I'm not aware of any. Regards Ansgar Wiechers -- Abstractions save us time working, but they don't save us time learning. --Joel Spolsky
Re: Larning about relays
Ansgar, thank you for your patience: Well, it looks like I could do relay_domains = example.com transport table: example.com relay:other.com I have to use other.com in the transport because I need to use DNS-based load balancing of multiple SMTP servers on the backend, but the relay is listed as the primary MX for example.com (my understanding is that I can't do this: example.com relay:internal.smtp.example.com because mail would loop, right? So will using another (essentially junk) domain work for this? As long as internal.smtp.example.com IS the final destination for example.com and the relaying server is NOT the final destination for example.com, the mail should not loop. I'm not sure whether or not you understood me or if we are in agreement. There is a third option. smtp.example.com is will be the relay server, which is the primary MX. However, multiple SMTP servers behind it need to be DNS-load balanced. So if I do this, mail should loop because the current (relay) server is, as far as DNS MX records are concerned, the correct destination server: example.com relay:internal.smtp.example.com You're confusing DNS and MTA configuration. MX records tell the world where to send mail for example.com. However, the MX record does NOT magically make an MTA running on that host the final destination for example.com SMTP-wise. An MTA becomes the final destination for example.com when the domain is configured in either $mydestination, $virtual_alias_domains or $virtual_mailbox_domains. It is NOT the final destination for any domain configured in $relay_domains. No, this I understand. I based my claim above on the understanding that this is what happens with a transport such as this: example.com relay:internal.smtp.example.com * mail for example.com arrives at the relay because it is the highest priority MX record for example.com * the relay is told by its configuration (see above) that internal.smtp.example.com is the next hop for mail to example.com * so the relay looks up the MX for mail going to example.com addresses via internal.smtp.example.com and finds again that the MX for example.com is itself, creating the loop I am probably misunderstanding something (like, maybe I can set the MX for internal.smtp.example.com differently and it will be looked up instead of just example.com? or maybe internal.smtp.example.com being the next hop overrides any MX lookup ???) There's also nothing wrong with suppressing MX lookups. You don't need MX records for load-balancing mail sent to backend servers. Example: 8 $ORIGIN example.com @ IN MX 10 external1 @ IN MX 10 external2 external1 IN A 10.42.23.11 external2 IN A 10.42.23.12 internal IN A 10.42.23.101 internal IN A 10.42.23.102 8 The above snippet defines two MX records external1.example.com and external2.example.com. These are the hosts that the world should send mail for example.com to. Since both records are defined with the same preference, inbound mail will be (DNS-)load-balanced over the two servers. Both servers run Postfix with $relay_domains = example.com and a transport map example.com relay:[internal.example.com] This will cause the two frontend servers to relay all mail for example.com to internal.example.com. No MX lookup is required for this, because you already know (and control) the destination host. And resolution of the A record internal.example.com will automatically load-balance the delivery, because the A record is configured with more than one address (round-robin DNS). OK, I didn't think of that - DNS load balancing with multiple A records instead of multiple MX records (for the internal hosts). I was stuck on how it's usually done for multiple public-facing hosts. Thanks for clarifying that tip for me. I think there has been some discussion that some mail servers like to have multiple MX records and some prefer multiple A records, thus a safe solution (for public facing MTAs) is to use both. In this case, are we assured that Postfix will always spread the load round robin style to multiple A records? If that's true, this seems like the best solution But when the main SMTP server gets mail to invalid recipients, how does it reject it? It doesn't reject it. Instead it generates bounces, which - in case of spam - will hit innocent bystanders whose addresses were spoofed. Do not accept mail that you know you can't deliver. Either use $relay_recipient_maps (building the list and pushing it to the frontend server can be implemented with a simple script), use LDAP lookups (in case your recipients are listed in an LDAP directory), or use the reject_unverified_recipient restriction [1]. So either way, the relay server ends up with a list of valid destination addresses right? Looks like there's just no other
Re: Larning about relays
On Thu, Jun 28, 2012 at 07:30:02PM -0700, email builder wrote: example.com relay:internal.smtp.example.com * mail for example.com arrives at the relay because it is the highest priority MX record for example.com Yes, but don't use the word relay here, it is too easily confused with the transport name, calle it the border MTA or the SMTP gateway, ... * the relay is told by its configuration (see above) that internal.smtp.example.com is the next hop for mail to example.com Yes, the nexthop *domain* is internal.smtp.example.com. * so the relay looks up the MX for mail going to example.com addresses via internal.smtp.example.com and finds again that the MX for example.com is itself, creating the loop NO. The MX lookup is for the nexthop domain, internal.smtp.example.com, that's where the mail goes, don't confuse recipient domains with nexthop relays. If there is no explicit MX for this, the A record(s) will be used instead. -- Viktor.
Re: Larning about relays
* mail for example.com arrives at the relay because it is the highest priority MX record for example.com Yes, but don't use the word relay here, it is too easily confused with the transport name, calle it the border MTA or the SMTP gateway, OK thanks for the language tips * the relay is told by its configuration (see above) that internal.smtp.example.com is the next hop for mail to example.com Yes, the nexthop *domain* is internal.smtp.example.com. * so the relay looks up the MX for mail going to example.com addresses via internal.smtp.example.com and finds again that the MX for example.com is itself, creating the loop NO. The MX lookup is for the nexthop domain, internal.smtp.example.com, that's where the mail goes, don't confuse recipient domains with nexthop relays. If there is no explicit MX for this, the A record(s) will be used instead. I see. So the MX lookup will be specific to the exact domain with hostname, etc. So MX load balancing would in fact work here. Although the suggestion to disable MX lookups and use A-record load balancing for the internal SMTP servers looks like a good idea Thank you
