Re: [dmarc-ietf] Some Proposed Language for a New pct Tag Defintion
On Mon, Aug 2, 2021 at 12:49 PM Todd Herr wrote: > And this: > > > 6.7.4. History of the "pct" Tag > > [...] > I suggest making this an appendix instead of leaving it up in the normative area, with an appropriate forward reference. And +1 to Michael's point about "experimental". -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Some Proposed Language for a New pct Tag Defintion
It appears that Todd Herr said: >> I like simple, but I also like the idea of a separate section that >discusses the history of the pct tag and why the old values won't work any >longer. OK except: > remains the default, and "0". The value of "0" took on unintended > significance during the experimental stage as a value used by some > intermediaries and mailbox providers as an indicator to either > deviate from standard handling of the message and/or to alter the > substance of reports generated, ... Alter the reports? Huh? I was under the impression that the policy didn't affect the reports, much less the pct. R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Some Proposed Language for a New pct Tag Defintion
On Mon, Aug 2, 2021 at 3:49 PM Todd Herr wrote: > > On Sat, Jul 31, 2021 at 4:38 PM John Levine wrote: > >> I'd make it a lot simpler: >> >>pct: (plain-text integer; OPTIONAL; default is 100). For the >> RFC5322.From domain to which the DMARC record applies, the "pct" >> tag describes what receivers are requested to to do with unaligned >> messages. >> This parameter does not affect the generation of DMARC reports. >> Possible values are as follows: >> >> 0: A request to not apply the policy, but for message forwarders >> to do whatever message rewriting they do that is intended to >> avoid >> sending DMARC unaligned mail. >> >> 100: The default, a request to apply the policy as specified. >> >> Any other value: results are undefined. >> >> >> I like simple, but I also like the idea of a separate section that > discusses the history of the pct tag and why the old values won't work any > longer. > > So, how about this: > >pct: (plain-text integer; OPTIONAL; default is 100). For the > > RFC5322.From domain to which the DMARC record applies, the "pct" > > tag serves as a signal to the actor performing DMARC validation > > checks as to whether or not the domain owner wishes the assessment > > policy declared in the "p=" tag to actually be applied. This > > parameter does not affect the generation of DMARC reports. > > Possible values are as follows: > > > 0: A request that the actor performing the DMARC validation check > > not apply the policy, but instead apply any special handling > > rules it might have in place. See Section 6.7.4 for further > > details. > > > 100: The default, a request to apply the policy as specified to > > any message that produces a DMARC "fail" result. > > > And this: > > > 6.7.4. History of the "pct" Tag > > >An earlier experimental version of the DMARC protocol [RFC7489] > >specified all integers from 0 to 100 inclusive as valid values for > >the pct tag. The intent of the tag was to provide domain owners with > >a method to gradually change their preferred assessment policy (the > >p= tag) from 'none' to 'quarantine' or from 'quarantine' to 'reject' > >by requesting the stricter treatment for just a percentage of > >messages that produced DMARC results of "fail". > > >Operational experience showed that the pct tag did not function as > >intended due to many factors, and so this version of the DMARC > >protocol has eliminated all possible values save for "100", which > >remains the default, and "0". The value of "0" took on unintended > >significance during the experimental stage as a value used by some > >intermediaries and mailbox providers as an indicator to either > >deviate from standard handling of the message and/or to alter the > >substance of reports generated, and these "custom" actions proved too > >valuable to the email community to remove from this version of the > >protocol. > "6.7.4. History of the "pct" Tag An earlier experimental version of the DMARC protocol [RFC7489]..." RFC7489 is Informational, not experimental. In IETF nomenclature, experimental status has a specific meaning. Michael Hammer ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Some Proposed Language for a New pct Tag Defintion
On Sat, Jul 31, 2021 at 4:38 PM John Levine wrote: > I'd make it a lot simpler: > >pct: (plain-text integer; OPTIONAL; default is 100). For the > RFC5322.From domain to which the DMARC record applies, the "pct" > tag describes what receivers are requested to to do with unaligned > messages. > This parameter does not affect the generation of DMARC reports. > Possible values are as follows: > > 0: A request to not apply the policy, but for message forwarders > to do whatever message rewriting they do that is intended to > avoid > sending DMARC unaligned mail. > > 100: The default, a request to apply the policy as specified. > > Any other value: results are undefined. > > > I like simple, but I also like the idea of a separate section that discusses the history of the pct tag and why the old values won't work any longer. So, how about this: pct: (plain-text integer; OPTIONAL; default is 100). For the RFC5322.From domain to which the DMARC record applies, the "pct" tag serves as a signal to the actor performing DMARC validation checks as to whether or not the domain owner wishes the assessment policy declared in the "p=" tag to actually be applied. This parameter does not affect the generation of DMARC reports. Possible values are as follows: 0: A request that the actor performing the DMARC validation check not apply the policy, but instead apply any special handling rules it might have in place. See Section 6.7.4 for further details. 100: The default, a request to apply the policy as specified to any message that produces a DMARC "fail" result. And this: 6.7.4. History of the "pct" Tag An earlier experimental version of the DMARC protocol [RFC7489] specified all integers from 0 to 100 inclusive as valid values for the pct tag. The intent of the tag was to provide domain owners with a method to gradually change their preferred assessment policy (the p= tag) from 'none' to 'quarantine' or from 'quarantine' to 'reject' by requesting the stricter treatment for just a percentage of messages that produced DMARC results of "fail". Operational experience showed that the pct tag did not function as intended due to many factors, and so this version of the DMARC protocol has eliminated all possible values save for "100", which remains the default, and "0". The value of "0" took on unintended significance during the experimental stage as a value used by some intermediaries and mailbox providers as an indicator to either deviate from standard handling of the message and/or to alter the substance of reports generated, and these "custom" actions proved too valuable to the email community to remove from this version of the protocol. -- *Todd Herr* | Technical Director, Standards and Ecosystem *e:* todd.h...@valimail.com *m:* 703.220.4153 This email and all data transmitted with it contains confidential and/or proprietary information intended solely for the use of individual(s) authorized to receive it. If you are not an intended and authorized recipient you are hereby notified of any use, disclosure, copying or distribution of the information included in this transmission is prohibited and may be unlawful. Please immediately notify the sender by replying to this email and then delete it from your system. ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Some Proposed Language for a New pct Tag Defintion
On Sun 01/Aug/2021 20:56:55 +0200 Douglas Foster wrote:
Ale, I tried to explain my objections in the original post. However, it is a
very important question, so I am happy to revise and extend my points.
Forgive me for being long-winded , I am trying to be thorough because I see
problems at many levels.
You're adding useless complications.
Random Guessing can increase the volume of wrong decisions.
The basic math does not work. Assume that a message sequence has a
probability P of being unwanted, and a probability of Q = 1-P of being
wanted. Does it make sense to use a random number based on P to discard messages?
Probability of outcomes:
·P*P – unwanted messages, correctly blocked
·P*Q – unwanted messages, incorrectly accepted
·Q*P – wanted messages, incorrectly blocked
·Q*Q – wanted messages correctly accepted
Total error rate is 2*P*Q. We have exchanged a one-sided error (allowing P
unwanted messages) for a two-sided error distribution? Does it improve the
overall error rate. Specifically, when is 2*P*Q < P ?
Cancelling P from both sides (P>0) yields 2*Q < 1 and Q < 0.5
If the message stream is more than 50% unwanted, then random guessing might
produce fewer total errors than allow-all. If the message stream at least 50%
wanted, then random guessing produces inferior results.
In plain English, if you have more spam than ham, then blocking at random is
correct in most cases. That's an obvious statement which adds nothing to the
discussion.
Other filtering stages will raise Q and lower P
Since the specific issue is failed DMARC Authentication, we also need to
consider how this task fits into the evaluation process. I believe my
process is typical:
·First, messages from known-bad senders are blocked.
·Second, sender authentication is performed, at which point some messages may
be discarded.
·Third, content filtering is applied, and suspicious content is blocked.
·Fourth, end-user activity occurs, where some messages are ignored or discarded.
One effect of the first stage is that it lowers P and raises Q. During sender
authentication, Q is likely to be above 50% even if the initial mail stream has
a Q below 50%.
The purpose of authentication is to recognize senders by name rather than by IP
number. Thus, according to the sense of "known-bad senders", authentication
can be considered a prerequisite of the first stage. If authentication fails,
you don't know who the sender is, therefore you don't know if it's good or bad.
If a false negative occurs during sender authentication, causing an unwanted
message to be allowed, the message may be blocked during content filtering or
it may be ignored by the user. Consequently, if the probability P is
applicable during sender authentication, the probability of a threat being
successful is less than P.
No. A successful authentication of a spammy message is not a false negative.
The fact that a message is unwanted has nothing to do with DMARC.
Random guessing will increase the volume of unrecoverable errors.
If a false positive occurs during sender authentication, causing a wanted
message to be blocked, there is no opportunity for recovery.
Actually, an opportunity of recovery exists. The sender can have feedback
mechanisms, such as 5yz SMTP replies, delivery notifications, return receipts
or other web-based actions. It can use feedback to recover from authentication
errors. Such errors happen in an apparent random fashion too. For example,
when the word "from" followed by a space appears in the beginning of a line,
some agent insert a greater-than sign ('>') before it, thereby breaking a DKIM
signature. As soon as the sender recognizes that delivery failed, it can
repeat sending the same message several times until, by chance, it gets a toss
greater than its pct. Phishers, OTOH, are known for not retrying.
The above is a use case for pct!= 0 and pct!=100.
Therefore, false positives are a greater problem than false negatives, and
the random guessing algorithm has the effect of replacing false negatives
with false positives.
Replacing what...?
Sender’s probability has no relation to Evaluator’s probability
For any single domain, incoming messages can be broken into three categories:
·Legitimately-sourced messages which arrive with valid credentials.
·Legitimately-sourced messages which arrive with failed credentials.
·Impersonation messages which arrive with failed credentials.
For simplicity, assume that sender and receiver interests are aligned – the
receiver wants to accept all legitimately-sourced messages from the domain.
Since the sender is moving toward P=REJECT and the recipient wants to enforce
P=REJECT, we will also assume that mailing lists are not part of the mail stream.
Neither sender nor receiver know the volume of unwanted impersonating
messages. This means that the denominator is unknown, but would be determined
by the
