Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
I am ready to embrace NXDOMAIN as the preferred test, based on two assumptions which I am hoping the group can endorse. Scope The PSD case is a special case of not-valid-for-mail names, and I was interested in trying to address more than that one case. A complete list seems to be: - Fictitious TLDs - PSD names, because they do not send mail - Unregistered and therefore fictitious organization domains, and their descendants - Fictitious additions to the set of names used for an organization domain - Use of names that are only used for non-mail purposes - Use of names that are used for SMTP but not for RFC5322.From A first question is whether the domain owner is the only arbiter of what names are real or fictitious within his space. This is necessary because DMARC Fail occurs on messages that I want and allow, even though I know that they did not originate from the domain owner. Some US Government web sites and some email security services are important enough to break the rules and get away with it. However, these "acceptable impersonations" are always using an actual email address. Similarly, mailing list messages may produce DMARC Fail, but the subscriber is a valid member of a valid domain. So my first assumption is, "Yes, the domain owner has official control over the definition of valid names within his organization" Not all RFC5322.From domains exist in DNS, and not all DNS names are valid as RFC5322.From domains. "Valid" becomes a nebulous concept based on the intent of the domain owner. While NXDOMAIN is certainly a solution to the PSD problem, it is not obvious that it is a sufficient solution to the general problem. My second assumption is that as an evaluator, I am unlikely to gain any benefit from checking for invalid names underneath an organization domain. If a domain owner ensures that both mail-enabled and non-mail names are protected using SP=REJECT, then an attacker gains no advantage from choosing an invalid name over a valid one. Similarly, if the domain owner leaves both mail-enabled and non-mail names unprotected using SP=NONE, then the attacker has no advantage from choosing an invalid name over a valid one. To the contrary, it seems advantageous for the attacker to use a valid name when one is available.Using an invalid name becomes advantageous only if the domain owner protects all mail-enabled names with P=REJECT, but leaves non-mail names unprotected using SP=NONE. This configuration seems bizarre to me, and therefore unimportant. Nonetheless, it still might be useful if all non-mail names could be determined with confidence, but this requires near-perfect information to be useful, and experience shows that this is unlikely. If the second assumption is accepted, then the last three scope topics disappear. For the remaining three - For unregistered and therefore fictitious organization domains. NXDOMAIN is the simplest and most appropriate test. We block based on NP=REJECT in the PSD's DMARC policy when a TXT (or any) name lookup returns NXDOMAIN. Organizations within that PSD can avoid false positives easily: either publish their own DMARC policy without NP=REJECT, or ensure that all of their RFC5322.From domains do exist in DNS, even if the entry is as trivial as Type=TXT, DATA="I Exist". - For PSDs, we block if the PSD has a DMARC record with the PSD flag. If that test is considered insufficient, we block by consulting the publicsuffix.org list and block names from that list that the evaluator considers applicable. - For fictitious TLDS, we have an undiscussed topic. If all valid TLDs now participate in DNS SEC, then a DNS lookup on valid TLDs will return DATA or NOERROR, while fictitious TLDS will always return NXDOMAIN.If the PSD flag becomes ubiquitous, then a tree walk which does not produce a DMARC policy could also be evidence of an invalid TLD. If PSD Flag usage is not universal, then a DNS query on the TLD name should be used as a follow-up to the tree walk. If some valid TLDs do not participate in DNS SEC and do not publish the PSD flag, then we are back to consulting a list. Doug On Wed, Dec 22, 2021 at 2:29 AM Murray S. Kucherawy wrote: > On Tue, Dec 21, 2021 at 11:32 AM John Levine wrote: > >> The DNS has had a formal definition of non-existence for over 30 >> years. You look up a name, if it returns records or NOERROR it exists, >> if it returns NXDOMAIN it doesn't. There is no reason for us to invent >> something new and incompatible. >> >> >I don't remember exactly why we settled on A/ / MX, but the lack of >> a clear, actionable definition is why we included one. >> >> See above. I don't remember where the text in A.4 came from, but it is >> wrong. >> If we are telling people to test whether a domain exists, they should do >> it >> the way the DNS does it. The correct test happens to be cheaper than A.4, >> one query rather than three. >> > > We're talking about two different things here, I think.
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Tue, Dec 21, 2021 at 11:32 AM John Levine wrote: > The DNS has had a formal definition of non-existence for over 30 > years. You look up a name, if it returns records or NOERROR it exists, > if it returns NXDOMAIN it doesn't. There is no reason for us to invent > something new and incompatible. > > >I don't remember exactly why we settled on A/ / MX, but the lack of a > clear, actionable definition is why we included one. > > See above. I don't remember where the text in A.4 came from, but it is > wrong. > If we are telling people to test whether a domain exists, they should do it > the way the DNS does it. The correct test happens to be cheaper than A.4, > one query rather than three. > We're talking about two different things here, I think. The DNS definition of "nonexistent" is as cited above while the DMARC definition matches the well established SMTP algorithm that figures out where the next hop for a particular recipient is. If there is no next hop, then for email purposes, the domain doesn't "exist". The logic goes something like: If this message fails DMARC, and a bounce has nowhere to go, then I'm pretty sure I don't want to deliver it. We're free to change our minds about what test is appropriate here, but that was the genesis of A.4. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Tue, Dec 21, 2021 at 4:31 AM Scott Kitterman wrote: > I don't remember exactly why we settled on A/ / MX, but the lack of a > clear, actionable definition is why we included one. Lack of DNS records > related to email authentication only means lack of email authentication, > which is in no way required. Given the way most systems are typically > architected, by the time you are doing email authentication, A or and > MX will already be in the local cache, so this is a pretty inexpensive > thing to check. > It comes from SMTP itself. RFC 5321 and its antecedent(s) specify that to identify the next hop for a message needing routing, you look up the MX for the recipient's domain. If that's missing, you try the A/ for that same name. If that's also missing, the message is undeliverable. Since that test has worked well for SMTP for rather a long time, DMARC adopted it. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On 12/18/2021 10:17 AM, Phillip Hallam-Baker wrote: Mailing lists are not well supported by SMTP and never will be. Any discussion of how to make mailing lists work better has to begin with the acceptance that they will never work very well which is what most people have been arguing in this thread. SMTP is a transfer protocol. It does not know about mailing lists and doesn't need to. (There is text in the SMTP spec about mailing lists, but really the text has nothing to do with SMTP, per se, and not a lot to do with the operation of mailing lists. Mailing lists are user-level processes that sit at a layer above SMTP. Rather than seeing mailing lists as they work today as the pinnacle of evolution, we should see them for what they are, a rather disgusting hack that we never got round to fixing. Can't say I've ever noticed anyone suggesting that mailing lists are the pinnacle of anything. On the other hand, efforts to produce more interesting capabilities that aren't centralized have not gotten much traction. One keeps hoping, but field experience is not encouraging. Why is it assumed that the input to a mailing list is an SMTP email? That seems to me to be a rather narrow assumption. Not everyone makes that assumption. That said, where is the spec for the alternative and who supports that spec? Once we recognize that the inputs and outputs from 'mailing lists' can be through other transports than SMTP, all arguments about preserving 'original' headers collapse. This is not an interaction between an SMTP sender and receiver, the mailing list itself is a principal. When playing in the sandbox of a particular set of specifications, then it's fine -- actually it's essential -- to specify requirements such as information preservation. If the point is that mail can transit heterogeneous environments, with different semantics, then the fact of that difference ensures information loss. Absent that assurance, why shouldn't the information be preserved, no matter how it is transported? That is, after all, one of the benefits of distinguish the message object specification from the message transport specification. Apologies. I've probably entirely missed your point. d/ -- Dave Crocker dcroc...@gmail.com 408.329.0791 Volunteer, Silicon Valley Chapter Information & Planning Coordinator American Red Cross dave.crock...@redcross.org ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Tuesday, December 21, 2021 2:32:12 PM EST John Levine wrote: > It appears that Scott Kitterman said: > >>> What definition are you wondering why we didn't stick to? > >> > >>Real non-existence. I'm not sure how to define it formally, ... > > The DNS has had a formal definition of non-existence for over 30 > years. You look up a name, if it returns records or NOERROR it exists, > if it returns NXDOMAIN it doesn't. There is no reason for us to invent > something new and incompatible. > > >I don't remember exactly why we settled on A/ / MX, but the lack of a > >clear, actionable definition is why we included one. > See above. I don't remember where the text in A.4 came from, but it is > wrong. If we are telling people to test whether a domain exists, they > should do it the way the DNS does it. The correct test happens to be > cheaper than A.4, one query rather than three. Fair enough. I can't remember why we thought RFC 8020[1] wasn't adequate when we did RFC 9091. I think we should modify the DMARC text to just refer to it. Scott K [1] https://www.rfc-editor.org/rfc/rfc8020.txt ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
It appears that Scott Kitterman said: >>> What definition are you wondering why we didn't stick to? >> >>Real non-existence. I'm not sure how to define it formally, ... The DNS has had a formal definition of non-existence for over 30 years. You look up a name, if it returns records or NOERROR it exists, if it returns NXDOMAIN it doesn't. There is no reason for us to invent something new and incompatible. >I don't remember exactly why we settled on A/ / MX, but the lack of a >clear, actionable definition is why we included one. See above. I don't remember where the text in A.4 came from, but it is wrong. If we are telling people to test whether a domain exists, they should do it the way the DNS does it. The correct test happens to be cheaper than A.4, one query rather than three. R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On December 21, 2021 12:05:35 PM UTC, Alessandro Vesely wrote: >On Mon 20/Dec/2021 18:41:25 +0100 Scott Kitterman wrote: >> On Monday, December 20, 2021 12:39:15 PM EST Alessandro Vesely wrote: >>> On Sun 19/Dec/2021 21:42:16 +0100 Scott Kitterman wrote: >>> > If the domain owner has suggested that you reject mail from a sub-domain >>> > that has none of A, , or MX records, why would you not do that? >>> Are we sure that's what the domain owner meant to suggest? >>> >>> An organization can use sp= on its root domain record, or it can define a >>> DMARC record at some subdomains. So far so good. >>> >>> Then it turns out that one can also define DMARC record at some non-existing >>> sub domains, possibly as an alternative to using np=... Now this begins to >>> look puzzling. >>> >>> The reason to introduce np= was to select domains that really don't exist. >>> Why don't we stick to that definition? >> >> What definition are you wondering why we didn't stick to? > > >Real non-existence. I'm not sure how to define it formally, because it would >be exorbitant to require to try everything. On the other hand an application >probably tried to authenticate SPF, DKIM, and DMARC already. If any of those >record was found, then it's silly to even lookup A/ / MX for the purpose >to >determine if the domain exists. I don't remember exactly why we settled on A/ / MX, but the lack of a clear, actionable definition is why we included one. Lack of DNS records related to email authentication only means lack of email authentication, which is in no way required. Given the way most systems are typically architected, by the time you are doing email authentication, A or and MX will already be in the local cache, so this is a pretty inexpensive thing to check. Scott K ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Mon 20/Dec/2021 18:41:25 +0100 Scott Kitterman wrote: On Monday, December 20, 2021 12:39:15 PM EST Alessandro Vesely wrote: On Sun 19/Dec/2021 21:42:16 +0100 Scott Kitterman wrote: > If the domain owner has suggested that you reject mail from a sub-domain > that has none of A, , or MX records, why would you not do that? Are we sure that's what the domain owner meant to suggest? An organization can use sp= on its root domain record, or it can define a DMARC record at some subdomains. So far so good. Then it turns out that one can also define DMARC record at some non-existing sub domains, possibly as an alternative to using np=... Now this begins to look puzzling. The reason to introduce np= was to select domains that really don't exist. Why don't we stick to that definition? What definition are you wondering why we didn't stick to? Real non-existence. I'm not sure how to define it formally, because it would be exorbitant to require to try everything. On the other hand an application probably tried to authenticate SPF, DKIM, and DMARC already. If any of those record was found, then it's silly to even lookup A/ / MX for the purpose to determine if the domain exists. Best Ale -- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
It appears that Alessandro Vesely said: >On Sun 19/Dec/2021 21:42:16 +0100 Scott Kitterman wrote: >> If the domain owner has suggested that you reject mail from a sub-domain >> that has none of A, , or MX records, why would you not do that? >Then it turns out that one can also define DMARC record at some non-existing >sub domains, possibly as an alternative to using np=... Now this begins to >look puzzling. Only if you misunderstand the way the DNS works. If there is a txt record for _dmarc.foo.bar, that means the name foo.bar exists. It may not have any records, but it exists. That's the difference between a DNS NXDOMAIN response for a non-existent domain and NOERROR for one with no records. The difference is not an accident, not a mistake, and is not going to change. See for example RFC 8020. >The reason to introduce np= was to select domains that really don't exist. >Why >don't we stick to that definition? Yes indeed, why don't we, and end this pointless argument? R's John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Monday, December 20, 2021 12:39:15 PM EST Alessandro Vesely wrote: > On Sun 19/Dec/2021 21:42:16 +0100 Scott Kitterman wrote: > > If the domain owner has suggested that you reject mail from a sub-domain > > that has none of A, , or MX records, why would you not do that? > Are we sure that's what the domain owner meant to suggest? > > An organization can use sp= on its root domain record, or it can define a > DMARC record at some subdomains. So far so good. > > Then it turns out that one can also define DMARC record at some non-existing > sub domains, possibly as an alternative to using np=... Now this begins to > look puzzling. > > The reason to introduce np= was to select domains that really don't exist. > Why don't we stick to that definition? What definition are you wondering why we didn't stick to? Scott K ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On Sun 19/Dec/2021 21:42:16 +0100 Scott Kitterman wrote: If the domain owner has suggested that you reject mail from a sub-domain that has none of A, , or MX records, why would you not do that? Are we sure that's what the domain owner meant to suggest? An organization can use sp= on its root domain record, or it can define a DMARC record at some subdomains. So far so good. Then it turns out that one can also define DMARC record at some non-existing sub domains, possibly as an alternative to using np=... Now this begins to look puzzling. The reason to introduce np= was to select domains that really don't exist. Why don't we stick to that definition? Best Ale -- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
In an organization of non-trivial size the chance that the person making the decision about what to put in DNS can reliably answer the question "used for RC5322.From addressing" is vanishingly small. They can, however, be very confident about if they have set up A//MX records. At best you are trading one kind of uncertainty for another. Scott K On Monday, December 20, 2021 7:12:34 AM EST Douglas Foster wrote: > Scott asked some questions along the lines of "since no test is perfect, > why don't we just run with the easily-understood MX/A/AAA test. I had not > responded to that part of the question previously. > > I have two major problems with the MX/A/ test: > > - If we are only testing for "Does this domain send out SMTP mail?", then > SPF must be included in the criteria. SPF speaks to what the domain sends, > while MX speaks to what the domain receives, and A/ speaks to what the > domain might receive. If we are testing for send behavior, we need to > examine the send authentication attribute. Any test which is based on > SMTP and other things should include SPF for the same reason. > > - Since we are not testing exclusively for SMTP behavior, we need to think > about how to avoid false positives, and how a domain owner changes a test > result from Fail to Pass. I have been calling this action "compliance > measures", hoping that the term was clear.For the MX/A/ test, with > or without SPF, the only way to indicate compliance is to publish false > information about SMTP configurations that do not apply to this domain. > Such false information has the potential for changing the behavior of SMTP > participants, and cannot be acceptable in a standards-track document. > > I have suggested feature-specific DNS flags, to indicate that a particular > domain does or does not participate in FROM addressing, or does not > participate in SMTP addressing. These have the advantage of being very > specific, but they depend on domain owners publishing new information. > A test which minimizes publishing new data has a better chance of being > usable, with confidence, by evaluators. So I suggested an exact-match > DMARC policy as an indicator to mean "used for RC5322.From addressing." > I am not entirely happy with that workaround, as it may have unwanted side > effects about how reporting occurs back to the domain owner. > > DF > > > > On Sun, Dec 19, 2021 at 3:42 PM Scott Kitterman > > wrote: > > If the domain owner has suggested that you reject mail from a sub-domain > > that has none of A, , or MX records, why would you not do that? Just > > as with any DMARC policy it's on the sender to ensure authorized email > > conforms to the policy. My impression is that you think that rejecting > > mail from such sub-domains is inherently risky somehow? > > > > My view is that it's substantially less risky than for more usual > > sub-domains. Note that I don't claim it's risk free. No filtering > > decision is risk free, so I don't find suggestions that it's not totally > > free of uncertainty particularly useful. > > > > My sense is that you are still searching for something that the np= tag > > was never meant to be. It might be more fruitful to try and specify what > > problem you are trying to solve and how we might go about it independent > > of > > the non-existent domain definition. Maybe you can propose a totally new > > tag that addresses the issues you are concerned about. If this new tag > > gets support from the group then we could look at if np= is still needed > > or > > if it's redundant. > > > > Scott K > > > > On December 19, 2021 7:35:30 PM UTC, Douglas Foster < > > > > dougfoster.emailstanda...@gmail.com> wrote: > > >What I said was that reception of NDRmessages is only a requirement for > > > > the > > > > >SMTP From address, so they are only required for the RFC5322.From address > > >when the two From addresses match. For msiling list messages, tbe two > > >do > > >not match. > > > > > >My topic was about the ability or inability to detect a never-valid > > > > RFC5322 > > > > >>From address. I am not engaged in any effort to change mailing lists. > > >> > > > NP=reject MUST never reject mailing list traffic. If we cannot do that, > > > > > >NP is useless. > > > > > >But we can meet that requirement, if we construct the right test. I can > > >support several different variations of the test, which differences in > > >strictness and complexity. I just cannot support the MX-A- test. > > > > > >Doug > > > > > >Doug > > > > > >On Sat, Dec 18, 2021, 12:15 PM John Levine wrote: > > >> It appears that Jeremy Harris said: > > >> >On 18/12/2021 03:47, Douglas Foster wrote: > > >> >> MX checks are a valid tool for assessing SMTP MailFrom addresses, > > > > since > > > > >> the > > >> > > >> >> sender is supposed to be ready to accept non-delivery reports and > > > > other > > > > >> >> automated messages. Of course,
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
Scott asked some questions along the lines of "since no test is perfect, why don't we just run with the easily-understood MX/A/AAA test. I had not responded to that part of the question previously. I have two major problems with the MX/A/ test: - If we are only testing for "Does this domain send out SMTP mail?", then SPF must be included in the criteria. SPF speaks to what the domain sends, while MX speaks to what the domain receives, and A/ speaks to what the domain might receive. If we are testing for send behavior, we need to examine the send authentication attribute. Any test which is based on SMTP and other things should include SPF for the same reason. - Since we are not testing exclusively for SMTP behavior, we need to think about how to avoid false positives, and how a domain owner changes a test result from Fail to Pass. I have been calling this action "compliance measures", hoping that the term was clear.For the MX/A/ test, with or without SPF, the only way to indicate compliance is to publish false information about SMTP configurations that do not apply to this domain. Such false information has the potential for changing the behavior of SMTP participants, and cannot be acceptable in a standards-track document. I have suggested feature-specific DNS flags, to indicate that a particular domain does or does not participate in FROM addressing, or does not participate in SMTP addressing. These have the advantage of being very specific, but they depend on domain owners publishing new information. A test which minimizes publishing new data has a better chance of being usable, with confidence, by evaluators. So I suggested an exact-match DMARC policy as an indicator to mean "used for RC5322.From addressing." I am not entirely happy with that workaround, as it may have unwanted side effects about how reporting occurs back to the domain owner. DF On Sun, Dec 19, 2021 at 3:42 PM Scott Kitterman wrote: > If the domain owner has suggested that you reject mail from a sub-domain > that has none of A, , or MX records, why would you not do that? Just > as with any DMARC policy it's on the sender to ensure authorized email > conforms to the policy. My impression is that you think that rejecting > mail from such sub-domains is inherently risky somehow? > > My view is that it's substantially less risky than for more usual > sub-domains. Note that I don't claim it's risk free. No filtering > decision is risk free, so I don't find suggestions that it's not totally > free of uncertainty particularly useful. > > My sense is that you are still searching for something that the np= tag > was never meant to be. It might be more fruitful to try and specify what > problem you are trying to solve and how we might go about it independent of > the non-existent domain definition. Maybe you can propose a totally new > tag that addresses the issues you are concerned about. If this new tag > gets support from the group then we could look at if np= is still needed or > if it's redundant. > > Scott K > > On December 19, 2021 7:35:30 PM UTC, Douglas Foster < > dougfoster.emailstanda...@gmail.com> wrote: > >What I said was that reception of NDRmessages is only a requirement for > the > >SMTP From address, so they are only required for the RFC5322.From address > >when the two From addresses match. For msiling list messages, tbe two do > >not match. > > > >My topic was about the ability or inability to detect a never-valid > RFC5322 > >>From address. I am not engaged in any effort to change mailing lists. > > NP=reject MUST never reject mailing list traffic. If we cannot do that, > >NP is useless. > > > >But we can meet that requirement, if we construct the right test. I can > >support several different variations of the test, which differences in > >strictness and complexity. I just cannot support the MX-A- test. > > > >Doug > > > >Doug > > > >On Sat, Dec 18, 2021, 12:15 PM John Levine wrote: > > > >> It appears that Jeremy Harris said: > >> >On 18/12/2021 03:47, Douglas Foster wrote: > >> >> MX checks are a valid tool for assessing SMTP MailFrom addresses, > since > >> the > >> >> sender is supposed to be ready to accept non-delivery reports and > other > >> >> automated messages. Of course, this has applicability if (but only > if) > >> >> the RFC5322.From domain is the same as the RFC5321.MailFrom domain. > >> > > >> >I disagree. It is well-established practice for a mailing list manager > >> >to accept and process NDRs accepted on the 5321.mailfrom (which differs > >> >from the 5322.from). > >> > >> Jeremy is right. Mailing lists always, and I mean always, put their > >> own 5321 bounce address on the messages so they can do bounce > >> management. If you look at the mail from this list, the bounce address > >> is dmarc-boun...@ietf.org. > >> > >> I have to say I am dismayed that we are spending time dealing with such > >> utterly basic misconceptions here. > >>
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
Yes, "Acceptable" is in the eye of the evaluator. I was not taking a position about what should be acceptable. I grudgingly accept messages that violate my DMARC policy, from some sources. Their messages are simply too important. My point is that RFC 7960 issues will ONLY occur on actual user accounts, not for accounts on non-mail domain names. A domain name that has never been authorized or used for email purposes will never have users and will never be "acceptable". "Never" means that I have opportunity to eliminate ambiguity. This is the mail that I want to block. Doug On Sun, Dec 19, 2021 at 8:56 PM tjw ietf wrote: > I have seen many cases of an actual user generating an “acceptable > iMpersonation”, but the Organization wanted those ruled Invalid because it > was Not approved. > > Also please be more precise when using the term “domain” around DNS > people. To us every label separated by a ‘.’ is a domain. > > Tim > > Sent from my iPhone > > On Dec 19, 2021, at 20:18, Douglas Foster < > dougfoster.emailstanda...@gmail.com> wrote: > > > Yes, I am looking for a high degree of certainty, comparable to what I get > with DMARC PASS. (For example, the DKIM private key might have been > compromised, but I choose not to worry about that possibility.) > > As I have said several times recently, DMARC FAIL is an ambiguous result, > and it is desirable to partition the set of all failures to see if > certainty can be obtained for some of them. ARC is an attempt to solve > uncertainty for a subset of messages by providing a new authentication > method which says ARC-satisfactory messages are definitely not malicious > impersonations. NP is, at least to my mind, is an attempt to solve > uncertainty in the opposite direction by identifying a subset of failures > that can only be malicious impersonations. > > When acceptable impersonation occurs, it is always on behalf of an actual > user. If there is no actual user, impersonation can only be malicious. > For simplicity, and of necessity, we ignore risks associated with the > local-part and focus on the domain name. We ask whether an entire domain > can be categorized as never-valid, so that messages to that domain can be > moved from unverified-with-uncertain-risk to unverified-with-confirmed-risk. > > At minimum, to satisfy the PSD goal, we need to be able to say that > "DoesNotExist.tld" is invalid because the organization is not registered > with a PSD. For that to work, we also need to not block messages from > valid names. The NXDOMAIN test is sufficient to solve the PSD problem, so > the only issue for registered domains is to ensure that valid domains do > not return NXDOMAIN. A more complex rule can block more non-mail names > from impersonation attacks, but they increase the compliance issues. > > DF > > > > On Sun, Dec 19, 2021 at 3:42 PM Scott Kitterman > wrote: > >> If the domain owner has suggested that you reject mail from a sub-domain >> that has none of A, , or MX records, why would you not do that? Just >> as with any DMARC policy it's on the sender to ensure authorized email >> conforms to the policy. My impression is that you think that rejecting >> mail from such sub-domains is inherently risky somehow? >> >> My view is that it's substantially less risky than for more usual >> sub-domains. Note that I don't claim it's risk free. No filtering >> decision is risk free, so I don't find suggestions that it's not totally >> free of uncertainty particularly useful. >> >> My sense is that you are still searching for something that the np= tag >> was never meant to be. It might be more fruitful to try and specify what >> problem you are trying to solve and how we might go about it independent of >> the non-existent domain definition. Maybe you can propose a totally new >> tag that addresses the issues you are concerned about. If this new tag >> gets support from the group then we could look at if np= is still needed or >> if it's redundant. >> >> Scott K >> >> On December 19, 2021 7:35:30 PM UTC, Douglas Foster < >> dougfoster.emailstanda...@gmail.com> wrote: >> >What I said was that reception of NDRmessages is only a requirement for >> the >> >SMTP From address, so they are only required for the RFC5322.From address >> >when the two From addresses match. For msiling list messages, tbe two >> do >> >not match. >> > >> >My topic was about the ability or inability to detect a never-valid >> RFC5322 >> >>From address. I am not engaged in any effort to change mailing lists. >> > NP=reject MUST never reject mailing list traffic. If we cannot do that, >> >NP is useless. >> > >> >But we can meet that requirement, if we construct the right test. I can >> >support several different variations of the test, which differences in >> >strictness and complexity. I just cannot support the MX-A- test. >> > >> >Doug >> > >> >Doug >> > >> >On Sat, Dec 18, 2021, 12:15 PM John Levine wrote: >> > >> >> It appears
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
It appears that Scott Kitterman said: >If the domain owner has suggested that you reject mail from a sub-domain that >has none of A, , or MX records, why would you not do that? Just as with >any DMARC policy it's on >the sender to ensure authorized email conforms to the policy. My impression >is that you think that rejecting mail from such sub-domains is inherently >risky somehow? I may not have been clear enough -- I agree that sp= and np= let you say things about non-existent subdomains of a domain that does exist. But if a domain does not exist, and is not a subdomain of one that publishes a sp= or np= policy, DMARC has nothing to say about it. As I said, there may be other reasons to reject the mail, but they're not DMARC. R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
If the domain owner has suggested that you reject mail from a sub-domain that has none of A, , or MX records, why would you not do that? Just as with any DMARC policy it's on the sender to ensure authorized email conforms to the policy. My impression is that you think that rejecting mail from such sub-domains is inherently risky somehow? My view is that it's substantially less risky than for more usual sub-domains. Note that I don't claim it's risk free. No filtering decision is risk free, so I don't find suggestions that it's not totally free of uncertainty particularly useful. My sense is that you are still searching for something that the np= tag was never meant to be. It might be more fruitful to try and specify what problem you are trying to solve and how we might go about it independent of the non-existent domain definition. Maybe you can propose a totally new tag that addresses the issues you are concerned about. If this new tag gets support from the group then we could look at if np= is still needed or if it's redundant. Scott K On December 19, 2021 7:35:30 PM UTC, Douglas Foster wrote: >What I said was that reception of NDRmessages is only a requirement for the >SMTP From address, so they are only required for the RFC5322.From address >when the two From addresses match. For msiling list messages, tbe two do >not match. > >My topic was about the ability or inability to detect a never-valid RFC5322 >>From address. I am not engaged in any effort to change mailing lists. > NP=reject MUST never reject mailing list traffic. If we cannot do that, >NP is useless. > >But we can meet that requirement, if we construct the right test. I can >support several different variations of the test, which differences in >strictness and complexity. I just cannot support the MX-A- test. > >Doug > >Doug > >On Sat, Dec 18, 2021, 12:15 PM John Levine wrote: > >> It appears that Jeremy Harris said: >> >On 18/12/2021 03:47, Douglas Foster wrote: >> >> MX checks are a valid tool for assessing SMTP MailFrom addresses, since >> the >> >> sender is supposed to be ready to accept non-delivery reports and other >> >> automated messages. Of course, this has applicability if (but only if) >> >> the RFC5322.From domain is the same as the RFC5321.MailFrom domain. >> > >> >I disagree. It is well-established practice for a mailing list manager >> >to accept and process NDRs accepted on the 5321.mailfrom (which differs >> >from the 5322.from). >> >> Jeremy is right. Mailing lists always, and I mean always, put their >> own 5321 bounce address on the messages so they can do bounce >> management. If you look at the mail from this list, the bounce address >> is dmarc-boun...@ietf.org. >> >> I have to say I am dismayed that we are spending time dealing with such >> utterly basic misconceptions here. >> >> R's, >> John >> >> ___ >> dmarc mailing list >> dmarc@ietf.org >> https://www.ietf.org/mailman/listinfo/dmarc >> ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
What I said was that reception of NDRmessages is only a requirement for the SMTP From address, so they are only required for the RFC5322.From address when the two From addresses match. For msiling list messages, tbe two do not match. My topic was about the ability or inability to detect a never-valid RFC5322 >From address. I am not engaged in any effort to change mailing lists. NP=reject MUST never reject mailing list traffic. If we cannot do that, NP is useless. But we can meet that requirement, if we construct the right test. I can support several different variations of the test, which differences in strictness and complexity. I just cannot support the MX-A- test. Doug Doug On Sat, Dec 18, 2021, 12:15 PM John Levine wrote: > It appears that Jeremy Harris said: > >On 18/12/2021 03:47, Douglas Foster wrote: > >> MX checks are a valid tool for assessing SMTP MailFrom addresses, since > the > >> sender is supposed to be ready to accept non-delivery reports and other > >> automated messages. Of course, this has applicability if (but only if) > >> the RFC5322.From domain is the same as the RFC5321.MailFrom domain. > > > >I disagree. It is well-established practice for a mailing list manager > >to accept and process NDRs accepted on the 5321.mailfrom (which differs > >from the 5322.from). > > Jeremy is right. Mailing lists always, and I mean always, put their > own 5321 bounce address on the messages so they can do bounce > management. If you look at the mail from this list, the bounce address > is dmarc-boun...@ietf.org. > > I have to say I am dismayed that we are spending time dealing with such > utterly basic misconceptions here. > > R's, > John > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
My analysis is rather different but comes to a stronger statement of John's point. Mailing lists are not well supported by SMTP and never will be. Any discussion of how to make mailing lists work better has to begin with the acceptance that they will never work very well which is what most people have been arguing in this thread. That does not mean that we should not attempt to make changes but it does mean that any changes should be considered through two lenses, not just one. First 'will this make mailing lists a bit more tolerable', Second, 'will this help a successor technology work better'. Rather than seeing mailing lists as they work today as the pinnacle of evolution, we should see them for what they are, a rather disgusting hack that we never got round to fixing. Why is it assumed that the input to a mailing list is an SMTP email? That seems to me to be a rather narrow assumption. I am typing this into Gmail, why assume that my HTTP transaction must be mediated by an SMTP transaction to reach a mailing list user? Why should 'mailing list' necessarily involve SMTP at all? Once we recognize that the inputs and outputs from 'mailing lists' can be through other transports than SMTP, all arguments about preserving 'original' headers collapse. This is not an interaction between an SMTP sender and receiver, the mailing list itself is a principal. I am of course aware that you can read IETF lists via IMAP, NNTP etc. In the short term, those efforts are being hobbled by the lack of decent clients. If the Web 4.0 folk get somewhere with a new generation of user centric social media, that might change. On Sat, Dec 18, 2021 at 12:15 PM John Levine wrote: > It appears that Jeremy Harris said: > >On 18/12/2021 03:47, Douglas Foster wrote: > >> MX checks are a valid tool for assessing SMTP MailFrom addresses, since > the > >> sender is supposed to be ready to accept non-delivery reports and other > >> automated messages. Of course, this has applicability if (but only if) > >> the RFC5322.From domain is the same as the RFC5321.MailFrom domain. > > > >I disagree. It is well-established practice for a mailing list manager > >to accept and process NDRs accepted on the 5321.mailfrom (which differs > >from the 5322.from). > > Jeremy is right. Mailing lists always, and I mean always, put their > own 5321 bounce address on the messages so they can do bounce > management. If you look at the mail from this list, the bounce address > is dmarc-boun...@ietf.org. > > I have to say I am dismayed that we are spending time dealing with such > utterly basic misconceptions here. > > R's, > John > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
It appears that Jeremy Harris said: >On 18/12/2021 03:47, Douglas Foster wrote: >> MX checks are a valid tool for assessing SMTP MailFrom addresses, since the >> sender is supposed to be ready to accept non-delivery reports and other >> automated messages. Of course, this has applicability if (but only if) >> the RFC5322.From domain is the same as the RFC5321.MailFrom domain. > >I disagree. It is well-established practice for a mailing list manager >to accept and process NDRs accepted on the 5321.mailfrom (which differs >from the 5322.from). Jeremy is right. Mailing lists always, and I mean always, put their own 5321 bounce address on the messages so they can do bounce management. If you look at the mail from this list, the bounce address is dmarc-boun...@ietf.org. I have to say I am dismayed that we are spending time dealing with such utterly basic misconceptions here. R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On 18/12/2021 03:47, Douglas Foster wrote: MX checks are a valid tool for assessing SMTP MailFrom addresses, since the sender is supposed to be ready to accept non-delivery reports and other automated messages. Of course, this has applicability if (but only if) the RFC5322.From domain is the same as the RFC5321.MailFrom domain. I disagree. It is well-established practice for a mailing list manager to accept and process NDRs accepted on the 5321.mailfrom (which differs from the 5322.from). -- Cheers, Jeremy ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
This is a useful point for discussion. MX checks are a valid tool for assessing SMTP MailFrom addresses, since the sender is supposed to be ready to accept non-delivery reports and other automated messages. Of course, this has applicability if (but only if) the RFC5322.From domain is the same as the RFC5321.MailFrom domain. As far as I know, DMARC is the only established tool for evaluating the RFC5322.From address when it differs from the RFC5321.MailFrom address. So to my thinking, the NP test is new territory without precedent. I am looking for a test that says decisively, "The domain owner has never authorized the use of this name for any mail-related purposes." There are many reasons why an acceptable message may fail DMARC verification. One of those reasons is that the message is an impersonation, but the impersonation is acceptable, typically because it is sent for the benefit of an account user. But acceptable messages will always use a domain name that is already in use by the domain owner, never one that they invented. The use of a never-authorized name will always be an unacceptable form of impersonation, and should be blocked. I thought that the whole WG had consensus on this point. The question then becomes, "How do we construct a decisive test?" We need a test that has very little possibility of a false positive that would cause a message to be blocked. My primary target is the educational segment, where everyone assumes that they will always use P/SP=NONE. because they never want their legitimate messages, including mailing list messages to be blocked. They should be willing to use NP=Reject if they are confident that NP will never block a legitimate message. A weak algorithm is acceptable if it blocks some but not all messages from never-used domains; false negatives are undesirable but tolerable. A result of DMARC FAIL with NP=FALSE will leave the evaluator in the same risk posture as if the NP test had not been performed. The only method we have for evaluating NP is the DNS, so the weakest possible rule is to block on NXDOMAIN. Unfortunately, even that test returns false positives. They occurs when messages sent by email service providers use the provider's SMTP address, and this is the norm. For these mailings, the only constraints on the RFC5322.From address are the ones imposed by the provider (you will not impersonate my other clients) and by the client itself (via DMARC). With relaxed alignment, the client can invent and use domain names that are not in DNS.Consequently, any NP test will likely require a participating organization to implement some compliance measures, such as creating DNS entries for those that do not exist at present. Those measures should be as simple and error-free as possible, or the risk-averse organization will simply not participate. To implement the NP tests, we devise DNS queries which indicate that the name exists. When a DATA result is not obtained by any of those queries, we return NP=TRUE.To minimize false positives, we want to use an expansive set of DNS queries. MX+A+ gives one set of in-use names. Adding SPF will add another set of in-use names. Adding exact-match DMARC and DKIM will add another set of in-use names. The only reason to omit these tests is if you can demonstrate that one of them is always superfluous because it will only include names obtained from other sources, or because it misleads. A+ may belong because it adds to the inclusive list, but it weakens the rule a lot. So I ask whether an acceptable result can be produced without it, a direct challenge to my previous sentence. This is a tricky one, and best left for another stage in the analysis. Doug On Fri, Dec 17, 2021 at 7:01 PM Scott Kitterman wrote: > > > On December 17, 2021 11:26:38 PM UTC, Douglas Foster < > dougfoster.emailstanda...@gmail.com> wrote: > .. > >A year after raising my concerns, I am still trying to get a collaborative > >discussion started about what the optimal test looks like. In a > >collaborative discussion, those who are happy with the status quo convince > >the skeptics to come on board, listen to their concerns, acknowledge them, > >and do what they can to accommodate those concerns so that consensus can > be > >achieved.I am willing to be convinced, but I am skeptical and I am > >looking for some collaboration. > > > It may be that this is a cultural issue then. In the IETF, where there is > an established consensus (rough or not), the burden is on those seeking to > overturn the consensus to convince people. If you think about it, if a > working group were obligated to rehash things whenever new people show up, > it would be difficult to accomplish anything in an open environment where > new people can show up anytime. > > I suspect you might have more luck if you first consulted Chesterton's > Fence. I think you'd have more luck with questions about why things are > the way
[dmarc-ietf] IETF Process/Culture was Re: 3.2.6 The meaning of non-existence
On December 17, 2021 11:26:38 PM UTC, Douglas Foster wrote: .. >A year after raising my concerns, I am still trying to get a collaborative >discussion started about what the optimal test looks like. In a >collaborative discussion, those who are happy with the status quo convince >the skeptics to come on board, listen to their concerns, acknowledge them, >and do what they can to accommodate those concerns so that consensus can be >achieved.I am willing to be convinced, but I am skeptical and I am >looking for some collaboration. > It may be that this is a cultural issue then. In the IETF, where there is an established consensus (rough or not), the burden is on those seeking to overturn the consensus to convince people. If you think about it, if a working group were obligated to rehash things whenever new people show up, it would be difficult to accomplish anything in an open environment where new people can show up anytime. I suspect you might have more luck if you first consulted Chesterton's Fence. I think you'd have more luck with questions about why things are the way they are than immediate assertions that they are wrong. Scott K P.S. At least as I understand it. I'm relatively new too. ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
