On Sun, Feb 7, 2016 at 2:29 PM, Gilles Chehade <[email protected]> wrote:
> - many MX do not accept that From / To / Cc headers don't have domains; > - and when they find one, they WILL append their own domain; > > So if you're not appending domain yourself, then you have a chance that the > next node to receive your message will append its own down, and since there > is a chance that it will forward the mail elsewhere, the destination gets a > mail with completely bogus headers. > > If some intermediate (not source or destination) MTA appends then yes, it's probably 100% wrong, however if the source appends it has pretty high chance of getting it wrong, it's going to be 100% wrong for everyone in the recipients except the ones that happen to be on the same domain. If you flip this the other way around and assume the intermediate MTA doesn't mangle the message then only the recipient MTA might mangle it and this is likely to be better because even if it does mangle it it would only be as bad as the source MTA mangling it (it would have destination MTA domain appended, not some arbitrary intermediate one), but it would still be limited to those "broken" MTA's, while the email might have recipients behind a dozen different MTA's and domain, everybody else getting "correct" headers. Also I'm not sure how often there really are any "intermediate" MTA's in the same sense that there are intermediate routers for IP traffic. Either they are intermediate because configured so by the sender side or the recipient side, so more or less under the control of the sender or recipient, and thus not truly intermediate. Unless you use some complex logic you're likely to get it wrong, so whether it's the source, intermediate(s) or destination MTA, they should probably leave the headers (DATA) alone, of course for the BCC the logic is pretty simple, I can't think of any valid reason why you'd want that to be included. > > Here's an example of how this can happen: > > - imagine you have a mail server at mx1.opensmtpd.org and imagines that > all > it does is accept mail that gets ~/.forward-ed > - now imagine [email protected] and [email protected] are exchanging mail, > but > they both use Outlook and the MTA at mx1.opensmtpd.org doesn't do > append, > whereas the destinations for @faurot.net and @poolp.org do. > > 1- Eric sends a mail to gilles: > From: eric > To: gilles > 2- mx1.opensmtpd.org receives and doesn't touch the header: > 3- mx1.poolp.org receives and appends domains: > From: [email protected] > To: [email protected] > > Now gilles replies: > From: gilles > To: eric > > 2- mx1.opensmtpd.org receives and doesn't touch the header: > 3- mx1.faurot.net receives and appends domains: > From: [email protected] > To: [email protected] > Assume the same, but now at step two mx1.opensmtpd.org does touch the headers (like it does). case 1 result: From: [email protected] To: [email protected] case 2 result: From: [email protected] To: [email protected] I don't think that's any better. Also, if opensmtd MTA would not touch them, and all three used it, then you'd get the correct results: case1: From: eric To: gilles case 2: From: gilles To: eric Again, if some complex logic was used, it might be possible to add the domains in some cases, but I'm not really sure if that's useful. Like comparing the sender and recipients to the SMTP protocol info and adjusting the headers accordingly, this might pretty quickly get pretty messy. I'm not sure what other MTA's out there do though. I guess if you are the originating MTA (the one that received the email from the MUA), you could probably match the SMTP info to the headers and add correct domains or addresses. I'm not 100% sure if it's possibly to always know if the MTA is the originating MTA, haven't thought of all the weird cases. Also I'm not sure if the ordering in the SMTP recipient list is the same as in the DATA header, so matching the addresses from the SMTP info to the DATA header recipients might really be impossible, in which case isn't guessing they're all @local.domain.tld just as bad? Worth noting that in the example I gave in my first email, there's no address, just a human readable name in quotes ("Person Name"), adding a domain to that would be wrong in any case. In that instance you would need to add full address resulting in ("Person Name" <[email protected]>), not just domain resulting in ("Person Name"@originating.mta.tld). > > If the MTA checks that user-parts for local domains exist, you have a > chance > that mail will get rejected at the destination. And if there is a gilles@ > at > faurot.net, eric will assume the mail came from the local user, when it > came > from a different gilles ... > > Doesn't the check use the SMTP protocol info, not the DATA headers? And isn't that info always correct? > There are many other weird cases like this if you involve backup MX, hubs > or > throw mixes of MX that do and don't append domains. If you intend to send > to > remote hosts, appending local domains (including if it is incorrect) is > just > a better choice than what will result if you don't. > I don't know all the details and real world implementations, but I would think doing the correct thing is better and if there's something broken in the wild, they should fix that instead. Adding a "random" domain to the end of every name doesn't seem like a good solution. Do you know how other MTA's handle this? The few I tested seemed to (mostly) not touch the DATA header, which to me seems the most correct thing to do. > > Maybe we should add a method to enable/disable this per admin-request but > it > is not going to make it into next release as we're way too close to it now. > > I think that would be useful, and it might come with some granularity, as in: - Fix From - Fix To / Cc - Fix Bcc (not sure if useful, since I can't think of any reason why you'd want them included, but maybe for standards compliance?) I'm not sure what the standards say about fixing the "Bcc", it seems Exchange does not remove them, but does reformat them, for example if there's multiple Bcc rows they get combined, but not removed. Fixing the "From" might be easy, since there can probably only be one sender, so you can copy it from the SMTP info; though I'm not sure what happens if there's a different "Sender" and "From", where an email is sent on behalf of someone else. Fixing the "To" and "Cc" seems a bit harder, and I can't think of any way you really can fix those, assuming the MUA sends the recipients in different order in the SMTP vs the DATA header. Do you know if Outlook is the only one leaving out the email addresses from headers? And also why does it leave them out? I mean, how are others supposed to be able to reply to emails if the addresses are left out of the actual message? > > > PS. Regarding the quote from the article, I didn't fully understand why > the > > "From" header would ever incorrectly have the local domain.. I mean, if > > OpenSMTPD prior to the fix mentioned in the article did not mangle the > DATA > > section, then how could the "From" header which is in the DATA section > > accidentally have the local domain appended? > > > > Well simple: > > 1- setup a mail server for your friends and use a MTA that doesn't append > 2- have all your friends setup a ~/.forward file > 3- chances are your friends are running Sendmail or Postfix with append > 4- whenever a friends mail another and there's no domain in headers, the > recipient will see his local domain for all addresses (same as above). > So their local MTA which was not opensmtpd appended the local domain? That makes sense, I was under the impression that it was an issue in opensmtpd, not other MTA's, so I couldn't figure out why it was happening. > Yes, the work done to implement the filters layer was very invasive and > took more than 3 OpenBSD release cycles, so it was worked off-tree. The > version in OpenBSD is 5.4.4 + all security/reliability fixes backported > just not the new features. > > We have managed to merge back all changes to OpenBSD so next release is > going to ship with latest version again. > Ah, ok. It took me a while to figure out why the "rcpt-to" wasn't working, and I was accidentally looking at the openbsd.org manuals which by default used the current and not OpenBSD 5.8, so manuals said it was there but it wasn't =)
