Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Douglas Foster]

If a mailing list can perform 100% sender authentication, it makes sense to
delegate that responsibility to the list  -- giving it the same treatment
as I am currently giving to the best-known ESPs.Getting there requires
a list that:
- wants one of my users as a subscriber,
- can perform 100% sender authentication,
- has activated a mechanism to notify me that it can perform delegated
authentication
- and provides ARC data to provide correlating evidence for whatever can be
correlated.

We do not currently have the policy communication mechanism, so the list
cannot give me the notification that I would want.

Doug


On Mon, Jul 24, 2023 at 11:57 AM Neil Anuskiewicz 
wrote:

>
>
> On Jul 23, 2023, at 9:39 PM, Douglas Foster <
> dougfoster.emailstanda...@gmail.com> wrote:
>
> 
> Neil, this is about whether to block first and ask questions later:
>  quarantining first is safer, but not initially feasible.
>
> My working assumption is that essentially all evaluators release
> unauthenticated traffic to their users every day.   To fix the mailing list
> problem, we are proposing to ask them to do so on a slightly greater scale.
>
> If you start on a path toward 100% authentication, the assumption is that
> you cannot quarantine every unauthenticated message because you will not
> have the staffing resources to work the quarantine queue in a timely
> manner.   So you have to do triage.
>
> Authentication moves:
>
>- Start by updating your configuration to authenticate based on the
>DMARC concept instead of RFC 7489 and the sender policy.   Don't waste
>precious resources checking for impersonation on messages that already have
>verifiable authentication.   Don't allow the absence of someone else's
>DMARC policy record to become a reason to put your network at risk.
>
>- Also consider that authentication comes in multiple forms.   I
>decided that a few major ESPs could be trusted to enforce client
>authentication during account setup and account use, so I did not have to
>repeat the process on every message from them.   That does not mean that I
>accept every message, because I don't.   Some of them still send a lot of
>unwanted traffic.  But it does mean that I accept the From address as valid
>without requiring an aligned DKIM signature.   This move also improved my
>authentication percentage.
>
>- Collectively, these moves will decrease the false positives in your
>pool of unauthenticated messages.
>
> Audit moves:
>
>- Initially, you can start with after-the-fact audits on the most
>obvious suspects.   For example, find the messages that fail both sender
>authentication and content filtering.   These messages have a high
>probability of malice, so verify the assessment and then block the sender
>completely.   You can actually start anywhere, because any form of sampling
>will work.  You have a dual goal:   find the wanted-but-unauthenticated
>message senders, and give them an authentication rule, while also finding
>the unwanted message senders and giving them a block rule.  Every
>investigation moves a message source from unauthenticated to authenticated
>or from unauthenticated to blocked.  Since you have a finite number of
>message sources, you have a finite number of investigations to complete.
>
>- Gradually, send an increasing volume of messages to quarantine,
>based on perceived probability of fraud.   Assume that this queue will
>still contain wanted messages, so the quarantine volume has to be throttled
>by your capacity to review all of its traffic.  Wanted messages still need
>to be released to users in something approximating a timely manner.
>
>- Over time, this process is guaranteed to reduce the volume of
>unauthenticated traffic.   When my SPF non-PASS rate was down to a few
>messages per day, I decided that I could quarantine any SPF result other
>than PASS.   Some of my business partners had no SPF policy at all, so they
>were quarantined and then equipped with an alternate authentication rule.
> By now, virtually all of the authentication-related quarantine is unwanted
>traffic.I haven't yet enforced quarantine on From verification failure,
>but my FROM PASS rate is about 97%, and the remaining 3% is mostly ESPs
>that have not been given special treatment.   So I am focusing on content
>filtering.   Most of the malicious traffic comes from mailbox provider
>accounts used for malicious purposes, which was sort of the goal of the
>100% authentication effort.   I have to depend on content filtering to flag
>those suspicious actors, and the confirmed attackers will be given a block
>rule.
>
> Doug Foster
>
> On Sun, Jul 23, 2023 at 9:30 AM Neil Anuskiewicz 
> wrote:
>
>> Doug, you’ve done a fine job of explaining as I groked what you said. If
>> I get I think most people here got it. I’ve been busy with work 

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Neil Anuskiewicz]

> On Jul 19, 2023, at 3:21 PM, Douglas Foster 
>  wrote:
> 
> 
> Perhaps you can clarify what you think DMARC is.
> 
> Apparently a significant number of evaluators think that "DMARC Fail with 
> p=reject always means unwanted mail".   Or to use Michael Hammer's language, 
> "DMARC Fail with p=reject means the domain owner wants it rejected so I will 
> reject it."These are exactly the attitudes that cause us so much trouble 
> because (a) DMARC Fail with p=reject does not mean that rejection is always 
> the correct response, and (b) DMARC Fail with p=none is an important piece of 
> information.   

Doug, if what you’re saying is true then the root cause might be our fault. If 
we’ve not communicated clearly enough DMARC’s purpose. A communication problem 
likely needs to be solved with more and better communication via the standards 
doc itself and other channels.

N
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] What happens when the DMARC record contains two identical tags?

[Mark Alley]

On 7/24/2023 11:13 AM, OLIVIER HUREAU wrote:



> If you want more than just the ABNF to defend that position, have a 
look at the DKIM RFC, from which this syntax was cloned; it says:


Then wouldn't it make sense to add this specification to DMARC-bis?


+1

I'm aware it's already technically implied through the existing ABNF(s), 
but given the simplicity and conciseness of the language Murray provided 
from the DKIM RFC, I concur it may be worth at least stating it 
explicitly for clarity in bis.


- Mark Alley
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] What happens when the DMARC record contains two identical tags?

[Murray S. Kucherawy]

On Mon, Jul 24, 2023 at 9:13 AM OLIVIER HUREAU <
olivier.hur...@univ-grenoble-alpes.fr> wrote:

> > Correct, the ABNF doesn't allow this construction, so it's a syntax
> error.
>
> DMARCbis ABNF is not as restrictive as RFC 7489 :
>
>
> dmarc-record  = dmarc-version *(dmarc-sep dmarc-tag) [dmarc-sep] *WSP
>
>
> > If you want more than just the ABNF to defend that position, have a
> look at the DKIM RFC, from which this syntax was cloned; it says:
>
> Then wouldn't it make sense to add this specification to DMARC-bis?
>

Can't hurt.

-MSK
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] What happens when the DMARC record contains two identical tags?

[OLIVIER HUREAU]

> Correct, the ABNF doesn't allow this construction, so it's a syntax error. 

DMARCbis ABNF is not as restrictive as RFC 7489 : 

dmarc-record  = dmarc-version *(dmarc-sep dmarc-tag) [dmarc-sep] *WSP 

> If you want more than just the ABNF to defend that position, have a look at 
> the DKIM RFC, from which this syntax was cloned; it says: 

Then wouldn't it make sense to add this specification to DMARC-bis? 

Olivier 


De: "Murray S. Kucherawy"  
À: "dmarc"  
Envoyé: Lundi 24 Juillet 2023 17:57:43 
Objet: Re: [dmarc-ietf] What happens when the DMARC record contains two 
identical tags? 

On Mon, Jul 24, 2023 at 8:37 AM Mark Alley mailto:40tekmarc@dmarc.ietf.org | 40tekmarc@dmarc.ietf.org ] > wrote: 





>From my understanding, that would most likely result in a permerror code for 
>evaluation, given that's not syntactically valid per the ABNF for the 
>dmarc-record, see below where it only shows each tag once from [ 
>https://datatracker.ietf.org/doc/html/rfc7489#section-6.4 | section
6.4 ] in 7489. 


In DMARCbis, I don't see this particular table in [ 
https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-dmarcbis-28#section-5.4 
| section
5.4 ] , but given the tags are all mentioned explicitly once in both 
ABNFs, it would imply they're expected only once in the record, otherwise, 
permerror; But I'll defer to the authors if that's correct. 



Correct, the ABNF doesn't allow this construction, so it's a syntax error. 

If you want more than just the ABNF to defend that position, have a look at the 
DKIM RFC, from which this syntax was cloned; it says: 
Tags with duplicate names MUST NOT occur within a single tag-list; if
   a tag name does occur more than once, the entire tag-list is invalid. 
-MSK 

___ 
dmarc mailing list 
dmarc@ietf.org 
https://www.ietf.org/mailman/listinfo/dmarc 
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] What happens when the DMARC record contains two identical tags?

[Murray S. Kucherawy]

On Mon, Jul 24, 2023 at 8:37 AM Mark Alley  wrote:

> From my understanding, that would most likely result in a permerror code
> for evaluation, given that's not syntactically valid per the ABNF for the
> dmarc-record, see below where it only shows each tag once from section 6.4
>  in 7489.
>
> In DMARCbis, I don't see this particular table in section 5.4
> ,
> but given the tags are all mentioned explicitly once in both ABNFs, it
> would imply they're expected only once in the record, otherwise, permerror;
> But I'll defer to the authors if that's correct.
>
Correct, the ABNF doesn't allow this construction, so it's a syntax error.

If you want more than just the ABNF to defend that position, have a look at
the DKIM RFC, from which this syntax was cloned; it says:

   Tags with duplicate names MUST NOT occur within a single tag-list; if
   a tag name does occur more than once, the entire tag-list is invalid.

-MSK
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Neil Anuskiewicz]

On Jul 23, 2023, at 9:39 PM, Douglas Foster  wrote:Neil, this is about whether to block first and ask questions later:   quarantining first is safer, but not initially feasible.My working assumption is that essentially all evaluators release unauthenticated traffic to their users every day.   To fix the mailing list problem, we are proposing to ask them to do so on a slightly greater scale.If you start on a path toward 100% authentication, the assumption is that you cannot quarantine every unauthenticated message because you will not have the staffing resources to work the quarantine queue in a timely manner.   So you have to do triage.   Authentication moves:Start by updating your configuration to authenticate based on the DMARC concept instead of RFC 7489 and the sender policy.   Don't waste precious resources checking for impersonation on messages that already have verifiable authentication.   Don't allow the absence of someone else's DMARC policy record to become a reason to put your network at risk.Also consider that authentication comes in multiple forms.   I decided that a few major ESPs could be trusted to enforce client authentication during account setup and account use, so I did not have to repeat the process on every message from them.   That does not mean that I accept every message, because I don't.   Some of them still send a lot of unwanted traffic.  But it does mean that I accept the From address as valid without requiring an aligned DKIM signature.   This move also improved my authentication percentage.Collectively, these moves will decrease the false positives in your pool of unauthenticated messages.Audit moves:Initially, you can start with after-the-fact audits on the most obvious suspects.   For example, find the messages that fail both sender authentication and content filtering.   These messages have a high probability of malice, so verify the assessment and then block the sender completely.   You can actually start anywhere, because any form of sampling will work.  You have a dual goal:   find the wanted-but-unauthenticated message senders, and give them an authentication rule, while also finding the unwanted message senders and giving them a block rule.  Every investigation moves a message source from unauthenticated to authenticated or from unauthenticated to blocked.  Since you have a finite number of message sources, you have a finite number of investigations to complete.Gradually, send an increasing volume of messages to quarantine, based on perceived probability of fraud.   Assume that this queue will still contain wanted messages, so the quarantine volume has to be throttled by your capacity to review all of its traffic.  Wanted messages still need to be released to users in something approximating a timely manner.Over time, this process is guaranteed to reduce the volume of unauthenticated traffic.   When my SPF non-PASS rate was down to a few messages per day, I decided that I could quarantine any SPF result other than PASS.   Some of my business partners had no SPF policy at all, so they were quarantined and then equipped with an alternate authentication rule.   By now, virtually all of the authentication-related quarantine is unwanted traffic.    I haven't yet enforced quarantine on From verification failure, but my FROM PASS rate is about 97%, and the remaining 3% is mostly ESPs that have not been given special treatment.   So I am focusing on content filtering.   Most of the malicious traffic comes from mailbox provider accounts used for malicious purposes, which was sort of the goal of the 100% authentication effort.   I have to depend on content filtering to flag those suspicious actors, and the confirmed attackers will be given a block rule.Doug FosterOn Sun, Jul 23, 2023 at 9:30 AM Neil Anuskiewicz  wrote:Doug, you’ve done a fine job of explaining as I groked what you said. If I get I think most people here got it. I’ve been busy with work and personal life so haven’t had as much time to lurk here. I’m curious what sparked your concern now? Also, isn’t it best to block first and ask questions later to mitigate damage while you investigate? I guess I’m saying the two ideas aren’t mutually exclusive.

Neil

> On Jul 15, 2023, at 4:27 AM, Douglas Foster  wrote:
> 
> 
> Murray recently observed, correctly, that something went horribly wrong with the DMARC rollout, because it caused (continues to cause) many safe and wanted messages to be blocked.
> 
> My assessment was in a recent post:
> 
> Something about the language of RFC 7489 caused implementers to focus on blocking individual messages, when the appropriate use of DMARC is to identify and investigate potentially malicious sources.
> 
> The "message blocking" approach violates the interests of the users it is intended to protect:
> 
> - An attacker sends 10 messages that maliciously impersonates a big bank.  With help from DMARC p=reject, the evaluator b

Re: [dmarc-ietf] What happens when the DMARC record contains two identical tags?

[Mark Alley]

From my understanding, that would most likely result in a permerror 
code for evaluation, given that's not syntactically valid per the ABNF 
for the dmarc-record, see below where it only shows each tag once from 
section 6.4  
in 7489.


In DMARCbis, I don't see this particular table in section 5.4 
, 
but given the tags are all mentioned explicitly once in both ABNFs, it 
would imply they're expected only once in the record, otherwise, 
permerror; But I'll defer to the authors if that's correct.


dmarc-record    = dmarc-version dmarc-sep
   [dmarc-request]
   [dmarc-sep dmarc-srequest]
   [dmarc-sep dmarc-auri]
   [dmarc-sep dmarc-furi]
   [dmarc-sep dmarc-adkim]
   [dmarc-sep dmarc-aspf]
   [dmarc-sep dmarc-ainterval]
   [dmarc-sep dmarc-fo]
   [dmarc-sep dmarc-rfmt]
   [dmarc-sep dmarc-percent]
   [dmarc-sep]
   ; components other than dmarc-version and
   ; dmarc-request may appear in any order

- Mark Alley

On 7/24/2023 10:05 AM, OLIVIER HUREAU wrote:
I am wondering how a parser should behave when the record contains two 
identical tags.


i.e: 'v=DMARC1; p=none; rua=mailto:t...@example.org; 
rua=mailto:t...@example.com;'


While RFC 7489 and DMARC-bis state that any unknown tags must be 
ignored, I have not found any specifications about identical tags.


Should we take into consideration the first tag? The second? Both? or 
ignore both?


Best,
Olivier

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

[dmarc-ietf] What happens when the DMARC record contains two identical tags?

[OLIVIER HUREAU]

I am wondering how a parser should behave when the record contains two 
identical tags. 

i.e: 'v=DMARC1; p=none; rua=mailto:t...@example.org; 
rua=mailto:t...@example.com;' 

While RFC 7489 and DMARC-bis state that any unknown tags must be ignored, I 
have not found any specifications about identical tags. 

Should we take into consideration the first tag? The second? Both? or ignore 
both? 

Best, 
Olivier 
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Another p=reject text proposal

[Douglas Foster]

I have stumbled into the same position as Barry, or maybe a more extreme
one.   I send no bounce messages, and very few 5xx responses.

In my environment, one of the more common causes of non-delivery is a
terminated employee.   It has not seemed like automated messages sources
pay much attention to NDR reports, so sending them seemed like a waste of
time.  I tried reject during SMTP based on sender verification, but it went
badly because of a software bug, and a manager silently lost subscription
to a critical service.   This leaves me reluctant to try again.  I also see
clear evidence of spammers trying to do directory harvesting by guessing
account names, and I don't want to help them.

More generally, there is the need to perform both Sender Authentication and
Sender Reputation checking when trying to decide if any particular sender
is worthy of a notification.   It just became easier to block silently.

Doug


On Mon, Jul 24, 2023 at 4:10 AM Alessandro Vesely  wrote:

> On Sun 23/Jul/2023 22:12:55 +0200 Barry Leiba wrote:
> >> Without bounces the sender is in the dark.
> >
> > Yes, if the sender is a human.
> >
> > Not so, if the sender is a mailing list and that sender will then
> > unsubscribe the intended recipient.
> > Also not so, if the sender is a malfeasant who may use the bounce
> > message for bad purposes.
> >
> > It's very clear to me that if I think a bounce message will be
> > harmful, I will not send one.  I will happily prefer silent discard
> > over a bounce when I think that's a better approach for that
> > situation.  Bouncing a legitimate mailing list message is just bad.
> > If you have reason to believe you're going to do that... don't.
> > Either deliver the message or silently discard it.  But don't bounce
> > it.
>
>
> Living aside the malfeasant case for a moment, do you think the worthiness
> of
> bounces can be stated depending on the type of sending?  Always?
>
> The only meaningful signal a mailing list can get out of a 5xx response is
> to
> deduce that that mailbox doesn't exist any more, so it must be
> unsubscribed.
>
> For alias expansions, there is the case that the author can appreciate to
> learn
> that her correspondent's mailbox is full, or that her writings was
> considered
> spam.  But again, consider that friends most likely write directly to the
> target mailbox, while newsletters deserve the same treatment as mailing
> lists.
>
> Is List-Unsubscribe: an indicator of drop vs. reject?
>
>
> Best
> Ale
> --
>
>
>
>
>
> ___
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc
>
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Why does DKIM fail when SPF succeeds (was: DMARC2 & SPF Dependency Removal)

[OLIVIER HUREAU]

Hi, 

> c) There is a pattern of similar looking reports, which omit the  
> element in the  altogether and always report 
> fail in the policy result. I suspect a product, which makes 
> it a bit too easy to enable DMARC validation without also enabling DKIM 
> verification, but I wasn't able to identify the product yet. 

I have also discovered that some report sender does not send valid aggregate 
reports because the DKIM and SPF auth Result type are not in the right 
position. 

According to the RFC 7489 XSD, Auth Result type is as follows: 

 
 
 
 
 
 
 
 

According to XML definitions, the position cannot be swapped and the 
DKIMAuthResultType (if there is one) must appear before the SPFAuthResultType. 
However, some reporter does not follow this implementation. 

E.g: the no longer maintained Linkedin dmarc-sys : 
[ https://github.com/LinkedInAttic/dmarc-msys/blob/master/dmarc_report.py#L240 
| https://github.com/LinkedInAttic/dmarc-msys/blob/master/dmarc_report.py#L240 
] where SPFAuthResultType appears before DKIMAuthResultType . 

Are you talking about the same error? 

Best, 
Olivier 



De: "Matthäus Wander"  
À: "dmarc"  
Envoyé: Dimanche 23 Juillet 2023 22:05:44 
Objet: [dmarc-ietf] Why does DKIM fail when SPF succeeds (was: DMARC2 & SPF 
Dependency Removal) 

Barry Leiba wrote on 2023-06-10 01:50: 
> That's interesting and disturbing if it remains consistent. 
> Theoretically, DKIM should *never* fail when SPF succeeds, so if 
> that's happening it means there is: 
> 1. bad signing software, 
> 2. bad verifying software, 
> 3. misconfiguration somewhere, 
> ...or a combination of those three. 
> 
> I would *really* like to see a current study of this, because I think 
> it's critical for the future viability of DMARC, whether or not we 
> accept the proposal to remove SPF. 
Not a study, but some data points I've observed: 

a) Signing with 3072-bit RSA leads to DKIM verification failures, 
because a popular mail gateway product (Cisco ESA) does not support RSA 
key lengths larger than 2048 bit. 

b) Messages are generated by an automated system without a Date header 
and signed by a central MTA. An outgoing mail gateway then adds the 
missing Date header (Postfix option 'always_add_missing_headers'), thus 
invalidating the DKIM signature. 

Such misconfigurations go unnoticed for years until someone checks the 
DMARC reports. While aggregate reports are incredibly helpful, it is 
still difficult to identify the cause of subtle DKIM failures. I'd wish 
that the  field would be filled by reporting software with 
the DKIM verification error message ('body hash did not verify', etc.) 
to aid with troubleshooting. 

Contacting the report  or postmaster address has never worked for 
me: if they don't bounce, nobody replies. 

c) There is a pattern of similar looking reports, which omit the  
element in the  altogether and always report 
fail in the policy result. I suspect a product, which makes 
it a bit too easy to enable DMARC validation without also enabling DKIM 
verification, but I wasn't able to identify the product yet. 

Regards, 
Matt 

___ 
dmarc mailing list 
dmarc@ietf.org 
https://www.ietf.org/mailman/listinfo/dmarc 
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Another p=reject text proposal

[Alessandro Vesely]

On Sun 23/Jul/2023 22:12:55 +0200 Barry Leiba wrote:

Without bounces the sender is in the dark.


Yes, if the sender is a human.

Not so, if the sender is a mailing list and that sender will then
unsubscribe the intended recipient.
Also not so, if the sender is a malfeasant who may use the bounce
message for bad purposes.

It's very clear to me that if I think a bounce message will be
harmful, I will not send one.  I will happily prefer silent discard
over a bounce when I think that's a better approach for that
situation.  Bouncing a legitimate mailing list message is just bad.
If you have reason to believe you're going to do that... don't.
Either deliver the message or silently discard it.  But don't bounce
it.



Living aside the malfeasant case for a moment, do you think the worthiness of 
bounces can be stated depending on the type of sending?  Always?


The only meaningful signal a mailing list can get out of a 5xx response is to 
deduce that that mailbox doesn't exist any more, so it must be unsubscribed.


For alias expansions, there is the case that the author can appreciate to learn 
that her correspondent's mailbox is full, or that her writings was considered 
spam.  But again, consider that friends most likely write directly to the 
target mailbox, while newsletters deserve the same treatment as mailing lists.


Is List-Unsubscribe: an indicator of drop vs. reject?


Best
Ale
--





___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc