Re: AW: forcing MX lookups
On 21/02/2012 19:26, Wietse Venema wrote: Ed W: As the OP suggested, a desirable solution would be for the MTA to only check the various maps to decide a domain is local *after* having done a DNS check to see if the MX record points to this machine. ie the end goal is if the MX record points to some other machine, then we deliver to that machine, even if it's listed in our maps as being local... You can use transport_maps=tcp:host:port etc. to ask an external program for the MX lookup, and to have that external program decide if it should reply with a local mail delivery transport if the MX record points to your machine. This will drop your mail delivery performance quite a lot, as there is only one queue manager, and each mail delivery request will be waiting for the previous transport map lookup to complete. A better alternative may be check_recipient_mx_access at RCPT TO time. smtpd_recipient_restrictions = # This example assumes that we are not providing mail relay service # or mail submission service. ... reject_unauth_destination check_recipient_mx_access cidr:/etc/postfix/mxnetworks ... /etc/postfix/mxnetworks: # Assuming this is your network with MX servers. 1.2.3.4/24 dunno # If the MX did not match, reject the request. 0.0.0.0/0 reject This rejects mail when a recipient domain is claimed by one of your customers, but the MX host for the domain is not local. This is better than doing it at queue manager time, because multiple smtpd processes can do these MX lookups in parallel. Wietse Thanks this seems like the better solution. It would seem that an incremental tweak could be to combine both solutions, rather than reject, redirect to some transport map which forwards to some instance without local maps - this would cause the mail to be forwarded to the MX destination as defined by DNS. (Why: Most users should be fine with reject, but for my requirements it's proved better to bounce invalid emails from *authorised users* vs rejecting at submission time. Mac Mail for example doesn't seem to correctly show reject error messages. Also we have users behind VERY slow dialup connections and bouncing works better for their submission process also.) Thanks for the solution! Ed W
Re: AW: forcing MX lookups
On 16/02/2012 23:07, Tom Hendrikx wrote: On 16-02-12 23:52, Dipl.-Ing. Juergen Ladstaetter wrote: Thank you both very much. That input was very good and I might rethink the strategy we're aiming at. Probably active DNS checks and periodic re-checks are better to ensure some security. Thanks guys Checking DNS at input time would still suffice. You simply require that domains entered have their MXen pointing to a predefined set of hosts (your cluster). They might change their own MX records later on (which will only harm the customer), but ibm.com will never point to your MXen to your cluster, so no customer can ever enter it. As long as you don't allow changing the domain itself without a re-check, no customer will ever be able to configure a domain that has MX records not controlled by that same customer. Shops that do hosted exchange etc (google, outlook.com) ask you to (temporarily) add some unique key/identifier to your DNS zone on order to prove that you actually own the zone (and the MX records). Same principle, but a bit more work for the customer. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von /dev/rob0 Gesendet: Thursday, February 16, 2012 3:38 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups On Thu, Feb 16, 2012 at 03:20:30PM -0500, Michael Orlitzky wrote: On 02/16/2012 12:13 PM, Dipl.-Ing. Juergen Ladstaetter wrote: yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? Even with two instances you could have problems. For example, your users might have aliases that get expanded on the incoming instance, where the maps are controlled by customers. If one of your customers sets up example.com, and has u...@example.com aliased to u...@example.net hosted elsewhere, they could be open to another customer stealing the example.net mail. If there is a way to force all alias expansion to go through the clean instance, this might work. Only thing I can think of is to append a domain component to all such names as used in aliasing, stripping it off on the way out. Then if it's valid, the clean relayhost would pass it right back. u...@example.comu...@example.net.Juergen Maybe either generic(5) maps on the dirty instance, or canonical(5) on the clean one, could strip this out and send it properly. One instance per customer is /probably/ safe, but I wouldn't swear to it without some more thought. At least in that case they'd only have themselves to blame. :) I would also consider periodic automated DNS checks which would disable any domain where DNS points elsewhere. (Or at least alert administrators to check on it.) The alias expansion suggestion above also highlights how/why this is quite a challenging problem if you can only rely on polling DNS entries that you don't control to see if some twit changed them on you... Also the dual instance fix is an imperfect solution... Ideally we would have support from the MTA itself.. I think this is a general problem for any mail admin who needs to allow (untrusted) customers to dynamically update the list of domains hosted on a given machine (ie where the customer controls the DNS, not us). I have the same problem and don't have a clean solution. I suspect that the folks arguing in favour of trying to make this solely a web application problem don't have such a customer base? As the OP suggested, a desirable solution would be for the MTA to only check the various maps to decide a domain is local *after* having done a DNS check to see if the MX record points to this machine. ie the end goal is if the MX record points to some other machine, then we deliver to that machine, even if it's listed in our maps as being local... This does seem problematic to implement though. My first thought would be a special case for the transport map where the next hop can be short circuited to be a no-op if the DNS result actually points back to us? (Perhaps could be implemented using a daemon providing some tcp tables based service?). The challenge seems to be how to identify if a given MX result actually means me, especially in the case where there could be a cluster of machines, multiple A records, hostile reverse DNS or even exotica such as anycast dns...? I guess if you know all the IPs of your machines you can do something (fragile)? Or if you can enforce that the MX record points to a specific named A record? However, seems pretty fragile and easy to make yourself a mail loop? Some kind of custom tcp tables seems like the only workable solution here? Have all the customer (untrusted) mail domains have a dynamic transport map, then you can have some kind of checking heuristics to test the MX records point to our server(s). Perhaps look for specific A record names, followed by a connection to the
Re: AW: forcing MX lookups
Ed W: As the OP suggested, a desirable solution would be for the MTA to only check the various maps to decide a domain is local *after* having done a DNS check to see if the MX record points to this machine. ie the end goal is if the MX record points to some other machine, then we deliver to that machine, even if it's listed in our maps as being local... You can use transport_maps=tcp:host:port etc. to ask an external program for the MX lookup, and to have that external program decide if it should reply with a local mail delivery transport if the MX record points to your machine. This will drop your mail delivery performance quite a lot, as there is only one queue manager, and each mail delivery request will be waiting for the previous transport map lookup to complete. A better alternative may be check_recipient_mx_access at RCPT TO time. smtpd_recipient_restrictions = # This example assumes that we are not providing mail relay service # or mail submission service. ... reject_unauth_destination check_recipient_mx_access cidr:/etc/postfix/mxnetworks ... /etc/postfix/mxnetworks: # Assuming this is your network with MX servers. 1.2.3.4/24 dunno # If the MX did not match, reject the request. 0.0.0.0/0 reject This rejects mail when a recipient domain is claimed by one of your customers, but the MX host for the domain is not local. This is better than doing it at queue manager time, because multiple smtpd processes can do these MX lookups in parallel. Wietse
Re: AW: forcing MX lookups
On 21-02-12 20:06, Ed W wrote: On 16/02/2012 23:07, Tom Hendrikx wrote: On 16-02-12 23:52, Dipl.-Ing. Juergen Ladstaetter wrote: Thank you both very much. That input was very good and I might rethink the strategy we're aiming at. Probably active DNS checks and periodic re-checks are better to ensure some security. Thanks guys Checking DNS at input time would still suffice. You simply require that domains entered have their MXen pointing to a predefined set of hosts (your cluster). They might change their own MX records later on (which will only harm the customer), but ibm.com will never point to your MXen to your cluster, so no customer can ever enter it. As long as you don't allow changing the domain itself without a re-check, no customer will ever be able to configure a domain that has MX records not controlled by that same customer. Shops that do hosted exchange etc (google, outlook.com) ask you to (temporarily) add some unique key/identifier to your DNS zone on order to prove that you actually own the zone (and the MX records). Same principle, but a bit more work for the customer. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von /dev/rob0 Gesendet: Thursday, February 16, 2012 3:38 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups On Thu, Feb 16, 2012 at 03:20:30PM -0500, Michael Orlitzky wrote: On 02/16/2012 12:13 PM, Dipl.-Ing. Juergen Ladstaetter wrote: yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? Even with two instances you could have problems. For example, your users might have aliases that get expanded on the incoming instance, where the maps are controlled by customers. If one of your customers sets up example.com, and has u...@example.com aliased to u...@example.net hosted elsewhere, they could be open to another customer stealing the example.net mail. If there is a way to force all alias expansion to go through the clean instance, this might work. Only thing I can think of is to append a domain component to all such names as used in aliasing, stripping it off on the way out. Then if it's valid, the clean relayhost would pass it right back. u...@example.comu...@example.net.Juergen Maybe either generic(5) maps on the dirty instance, or canonical(5) on the clean one, could strip this out and send it properly. One instance per customer is /probably/ safe, but I wouldn't swear to it without some more thought. At least in that case they'd only have themselves to blame. :) I would also consider periodic automated DNS checks which would disable any domain where DNS points elsewhere. (Or at least alert administrators to check on it.) The alias expansion suggestion above also highlights how/why this is quite a challenging problem if you can only rely on polling DNS entries that you don't control to see if some twit changed them on you... Also the dual instance fix is an imperfect solution... Ideally we would have support from the MTA itself.. I think this is a general problem for any mail admin who needs to allow (untrusted) customers to dynamically update the list of domains hosted on a given machine (ie where the customer controls the DNS, not us). I have the same problem and don't have a clean solution. I suspect that the folks arguing in favour of trying to make this solely a web application problem don't have such a customer base? I don't run a setup like this, but deal with the DNS verification part of it on a daily basis. I fail to see the real issue here: if a customer actually owns the domain, and he changes the MX records to somewhere else, he'll only scatter his mail delivery (i.e. local mail on your platform, other mail on the new MXen), but this does not harm you, only the customer. Whether the customer is fraudulent or not, he can only influence deliverability of domains he actually added by himself, or that he owns in DNS. The single challenge is that you need to verify that the customer can only add a domain that he actually owns, in order to make sure that no customer can steal mail for remote destinations that aren't his. This last part is relatively easy to solve by creating a procedure to verify that the customer is the owner of the domain in DNS, by requiring him to add a specific key to the zone for ownership verification. Simply said: Google [1] and many others have already solved this issue, I don't see why your situation differs from theirs. Also I fail to see how a fraudulent customer would be able to steal mail for example.net as described in the alias example, if he's not able to prove that he owns the example.net zone in DNS (thus cannot add the example.net zone to your localdomains). Please educate me. [1] http://support.google.com/a/bin/answer.py?hl=enanswer=183895 --
AW: forcing MX lookups
The configuration for domains etc. is stored in mysql tables but that has nothing to do with the initial problem described in my other email. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 12:20 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups Am 16.02.2012 18:13, schrieb Dipl.-Ing. Juergen Ladstaetter: We're currently developing a project where customers can add their own domains to our mailsystem. The biggest problem would be that a customer adds a domain he doesn't own or isn't represented by our mail cluster. For example a customer adds ibm.com - a manual validation through one of our employees isn't possible and an automated validation makes no sense since the MX records could be changed at any time, which would force a re-check. My thought was this: when sending the mail, configure postfix that he does a MX lookup and sends the mail to the IP given by the lookup. In this case the customer could add ibm.com, but he wouldn't be able to grab the mails sent from our cluster. I tried a few things but haven't come to a clean solution yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? put your postfix-configuration in mysql-tables and create a limited user for query if input is valid and allowed from the moment on you are dealing with websites AND configurations you should have as much as possible of your config in databases
AW: forcing MX lookups
I know that it doesn't need a MX record. I just want to know if there is some way of configuring postfix to make DNS (A or MX) lookups for every mail sent. Since a two instance configuration would work, I just wanted to know if there is a way to configure one instance to do this. No need to get impolite... -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 12:54 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups what is this for a strange day you are the second one in a few hours not understand that a domain does not need any MX-RECORD to be a valid maildomain your other things are also not job of postfix if you have a application where users can input data you are responsible to verify the input inside your application before proceed make DNS requests and whatever verifications in your app and leave postfix in peace - postfix is a MTA and should only be used as MTA Am 16.02.2012 18:45, schrieb Dipl.-Ing. Juergen Ladstaetter: The configuration for domains etc. is stored in mysql tables but that has nothing to do with the initial problem described in my other email. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 12:20 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups Am 16.02.2012 18:13, schrieb Dipl.-Ing. Juergen Ladstaetter: We're currently developing a project where customers can add their own domains to our mailsystem. The biggest problem would be that a customer adds a domain he doesn't own or isn't represented by our mail cluster. For example a customer adds ibm.com - a manual validation through one of our employees isn't possible and an automated validation makes no sense since the MX records could be changed at any time, which would force a re-check. My thought was this: when sending the mail, configure postfix that he does a MX lookup and sends the mail to the IP given by the lookup. In this case the customer could add ibm.com, but he wouldn't be able to grab the mails sent from our cluster. I tried a few things but haven't come to a clean solution yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? put your postfix-configuration in mysql-tables and create a limited user for query if input is valid and allowed from the moment on you are dealing with websites AND configurations you should have as much as possible of your config in databases
AW: forcing MX lookups
Alright then let me try to make it more clear for you: if you have a local-domain it will not make a lookup if it would the mail could not be delivered local That's the point. Even though it's configured as local-domain I would want it to look up any records (MX, A) and try to sent the mail to the underlying mailserver. why should it do this and what should happen relay the message? Why: because there is probably a configuration variable that allows me this configuration. If not then I have to do it differently What should happen: Email - lookup - connect - deliver Normally with local-domains it would be Email - local deliver. I would want to add 'lookup - connect' as it's done with non-local-domains why if it is as local domain configured? To ensure that mails are delivered to the right server since a 100% dynamic system sadly is open for fraudulent entries. again: it is generally a complete misdesign to use a mailserver with whatever tricks to do strange things even if it would work - such solutions are the root cause for most of our problems these days Not really. Open relays cause problems. Non-standard designs that are well maintained and secured don't. in other words: this all does not make any sense Neither do parts of your email. I, and I think many others, would appreciate it if you reply in a more polite way since we're only here to share information and furthermore it would be amazing if you re-read your emails before sending them since they are full of weird sentences, typos and wrong grammar. Thanks -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 1:22 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups how should anybody imagine lookups for every mail if you have a local-domain it will not make a lookup if it would the mail could not be delivered local so no, you can have not in one instance local inboxes for example.com and force postfix to do any dns-lookups for example.com why should it do this and what should happen relay the message? why if it is as local domain configured? again: it is generally a complete misdesign to use a mailserver with whatever tricks to do strange things even if it would work - such solutions are the root cause for most of our problems these days in other words: this all does not make any sense Am 16.02.2012 19:05, schrieb Dipl.-Ing. Juergen Ladstaetter: I know that it doesn't need a MX record. I just want to know if there is some way of configuring postfix to make DNS (A or MX) lookups for every mail sent. Since a two instance configuration would work, I just wanted to know if there is a way to configure one instance to do this. No need to get impolite... -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 12:54 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups what is this for a strange day you are the second one in a few hours not understand that a domain does not need any MX-RECORD to be a valid maildomain your other things are also not job of postfix if you have a application where users can input data you are responsible to verify the input inside your application before proceed make DNS requests and whatever verifications in your app and leave postfix in peace - postfix is a MTA and should only be used as MTA Am 16.02.2012 18:45, schrieb Dipl.-Ing. Juergen Ladstaetter: The configuration for domains etc. is stored in mysql tables but that has nothing to do with the initial problem described in my other email. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von Reindl Harald Gesendet: Thursday, February 16, 2012 12:20 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups Am 16.02.2012 18:13, schrieb Dipl.-Ing. Juergen Ladstaetter: We're currently developing a project where customers can add their own domains to our mailsystem. The biggest problem would be that a customer adds a domain he doesn't own or isn't represented by our mail cluster. For example a customer adds ibm.com - a manual validation through one of our employees isn't possible and an automated validation makes no sense since the MX records could be changed at any time, which would force a re-check. My thought was this: when sending the mail, configure postfix that he does a MX lookup and sends the mail to the IP given by the lookup. In this case the customer could add ibm.com, but he wouldn't be able to grab the mails sent from our cluster. I tried a few things but haven't come to a clean solution yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance
AW: forcing MX lookups
Thank you both very much. That input was very good and I might rethink the strategy we're aiming at. Probably active DNS checks and periodic re-checks are better to ensure some security. Thanks guys -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von /dev/rob0 Gesendet: Thursday, February 16, 2012 3:38 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups On Thu, Feb 16, 2012 at 03:20:30PM -0500, Michael Orlitzky wrote: On 02/16/2012 12:13 PM, Dipl.-Ing. Juergen Ladstaetter wrote: yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? Even with two instances you could have problems. For example, your users might have aliases that get expanded on the incoming instance, where the maps are controlled by customers. If one of your customers sets up example.com, and has u...@example.com aliased to u...@example.net hosted elsewhere, they could be open to another customer stealing the example.net mail. If there is a way to force all alias expansion to go through the clean instance, this might work. Only thing I can think of is to append a domain component to all such names as used in aliasing, stripping it off on the way out. Then if it's valid, the clean relayhost would pass it right back. u...@example.comu...@example.net.Juergen Maybe either generic(5) maps on the dirty instance, or canonical(5) on the clean one, could strip this out and send it properly. One instance per customer is /probably/ safe, but I wouldn't swear to it without some more thought. At least in that case they'd only have themselves to blame. :) I would also consider periodic automated DNS checks which would disable any domain where DNS points elsewhere. (Or at least alert administrators to check on it.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: AW: forcing MX lookups
On 16-02-12 23:52, Dipl.-Ing. Juergen Ladstaetter wrote: Thank you both very much. That input was very good and I might rethink the strategy we're aiming at. Probably active DNS checks and periodic re-checks are better to ensure some security. Thanks guys Checking DNS at input time would still suffice. You simply require that domains entered have their MXen pointing to a predefined set of hosts (your cluster). They might change their own MX records later on (which will only harm the customer), but ibm.com will never point to your MXen to your cluster, so no customer can ever enter it. As long as you don't allow changing the domain itself without a re-check, no customer will ever be able to configure a domain that has MX records not controlled by that same customer. Shops that do hosted exchange etc (google, outlook.com) ask you to (temporarily) add some unique key/identifier to your DNS zone on order to prove that you actually own the zone (and the MX records). Same principle, but a bit more work for the customer. -Ursprüngliche Nachricht- Von: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] Im Auftrag von /dev/rob0 Gesendet: Thursday, February 16, 2012 3:38 PM An: postfix-users@postfix.org Betreff: Re: forcing MX lookups On Thu, Feb 16, 2012 at 03:20:30PM -0500, Michael Orlitzky wrote: On 02/16/2012 12:13 PM, Dipl.-Ing. Juergen Ladstaetter wrote: yet. Is there any way to configure postfix to always make MX record DNS lookups, or is the only way through a second postfix instance that has no localdomains specified? Even with two instances you could have problems. For example, your users might have aliases that get expanded on the incoming instance, where the maps are controlled by customers. If one of your customers sets up example.com, and has u...@example.com aliased to u...@example.net hosted elsewhere, they could be open to another customer stealing the example.net mail. If there is a way to force all alias expansion to go through the clean instance, this might work. Only thing I can think of is to append a domain component to all such names as used in aliasing, stripping it off on the way out. Then if it's valid, the clean relayhost would pass it right back. u...@example.com u...@example.net.Juergen Maybe either generic(5) maps on the dirty instance, or canonical(5) on the clean one, could strip this out and send it properly. One instance per customer is /probably/ safe, but I wouldn't swear to it without some more thought. At least in that case they'd only have themselves to blame. :) I would also consider periodic automated DNS checks which would disable any domain where DNS points elsewhere. (Or at least alert administrators to check on it.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
