Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal

[Neil Anuskiewicz]

> On Jun 8, 2023, at 4:25 PM, Scott Kitterman  wrote:
> 
> The data I have seen (and it sounds like Mike is saying the same thing) 
> shows DKIM verification results are less than 100%, consistently, for direct 
> connections.  It was always lower than the SPF pass rate for hosts listed in 
> a domain's SPF record.  I understand that in theory, it shouldn't matter, but 
> in practice it is.
> 
> Software engineering isn't a perfect science.  In general, a more complex 
> protocol will suffer more defects.  If you want to design things that only 
> work when software is perfect, I'm not interested.
> 
> In terms of utility for direct connections, I think having both helps and I 
> don't find claims the SPF can never pass when DKIM fails to be credible.  If 
> someone has data that shows that's true now, I'd be interested to see it (my 
> experience is a few years ago, so things may have changed).
> 
> Ultimately, I think SPF and DKIM both suffer from what I'll genetically call 
> data hygiene problems.  SPF is mostly adding third party providers who are 
> insufficiently careful (might not even care, I don't know) generating SPF 
> pass results for "bad" mail.  DKIM is mostly about replay attacks.  Neither 
> of these are protocol problems and I don't think support dumping one or the 
> other out of DMARC.
> 
> One could make the opposite argument too, and I think it would be equally 
> valid:
> 
> The only value DKIM brings for DMARC is for indirect mail flows.  For any 
> direct connections, SPF is sufficient.  All the proposed DKIM changes to 
> solve the DKIM replay problem are likely to break indirect mail flows anyway, 
> so there's no longer a point to keep DKIM.  It's much more complicated and it 
> looks like the benefit of it is going away, let's just simplify the protocol 
> and get rid of it.
> 
> Now, I think that's a bad argument, but I don't think it's any worse than the 
> argument being presented to get rid of SPF.

I’ve observed the pattern of a high dmarc pass rate on DKiM alone, over 99% in 
most cases, yet as cogently stated by others, SPF will take a 99.5% pass rate 
to a rock solid 100%.

These might be an acceptable amount of blocked mail in some situations but 
there are many other scenarios where these lost emails, which add up fast, cost 
the company money and potentially strain relationships.  Email is perhaps the 
most important tool for business communication between businesses. An account 
manager sends a contract to an important prospect who doesn’t get it. That’s 
the cost. It might not be your first day at 99.9% but that number is on a 
rendezvous with its destiny to cost the business and individuals in various 
ways.

The effects of what seem to be small but aren’t when it comes to communication 
in business. 

Sure, we narrow the scope of our spf as much as we possibly can but we don’t 
toss that 0.5% traffic away without a reasonably good reason. 

If someone can cogently explain the assumption that the costs of spf 
implantation done judiciously (You don’t put an ESPs entire include in your org 
domain’s spf but you don’t just punt, declining the benefits of well implanted 
SPF. That’s an unforced error. Sometimes you have to throw SPF overboard but 
that’s not optimal. I get the feeling some here might make spf walk the plank 
right away. I ask that you reconsider that notion.
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Douglas Foster]

Neil, this is about whether to block first and ask questions later:
 quarantining first is safer, but not initially feasible.

My working assumption is that essentially all evaluators release
unauthenticated traffic to their users every day.   To fix the mailing list
problem, we are proposing to ask them to do so on a slightly greater scale.

If you start on a path toward 100% authentication, the assumption is that
you cannot quarantine every unauthenticated message because you will not
have the staffing resources to work the quarantine queue in a timely
manner.   So you have to do triage.

Authentication moves:

   - Start by updating your configuration to authenticate based on the
   DMARC concept instead of RFC 7489 and the sender policy.   Don't waste
   precious resources checking for impersonation on messages that already have
   verifiable authentication.   Don't allow the absence of someone else's
   DMARC policy record to become a reason to put your network at risk.

   - Also consider that authentication comes in multiple forms.   I
   decided that a few major ESPs could be trusted to enforce client
   authentication during account setup and account use, so I did not have to
   repeat the process on every message from them.   That does not mean that I
   accept every message, because I don't.   Some of them still send a lot of
   unwanted traffic.  But it does mean that I accept the From address as valid
   without requiring an aligned DKIM signature.   This move also improved my
   authentication percentage.

   - Collectively, these moves will decrease the false positives in your
   pool of unauthenticated messages.

Audit moves:

   - Initially, you can start with after-the-fact audits on the most
   obvious suspects.   For example, find the messages that fail both sender
   authentication and content filtering.   These messages have a high
   probability of malice, so verify the assessment and then block the sender
   completely.   You can actually start anywhere, because any form of sampling
   will work.  You have a dual goal:   find the wanted-but-unauthenticated
   message senders, and give them an authentication rule, while also finding
   the unwanted message senders and giving them a block rule.  Every
   investigation moves a message source from unauthenticated to authenticated
   or from unauthenticated to blocked.  Since you have a finite number of
   message sources, you have a finite number of investigations to complete.

   - Gradually, send an increasing volume of messages to quarantine, based
   on perceived probability of fraud.   Assume that this queue will still
   contain wanted messages, so the quarantine volume has to be throttled by
   your capacity to review all of its traffic.  Wanted messages still need to
   be released to users in something approximating a timely manner.

   - Over time, this process is guaranteed to reduce the volume of
   unauthenticated traffic.   When my SPF non-PASS rate was down to a few
   messages per day, I decided that I could quarantine any SPF result other
   than PASS.   Some of my business partners had no SPF policy at all, so they
   were quarantined and then equipped with an alternate authentication rule.
By now, virtually all of the authentication-related quarantine is unwanted
   traffic.I haven't yet enforced quarantine on From verification failure,
   but my FROM PASS rate is about 97%, and the remaining 3% is mostly ESPs
   that have not been given special treatment.   So I am focusing on content
   filtering.   Most of the malicious traffic comes from mailbox provider
   accounts used for malicious purposes, which was sort of the goal of the
   100% authentication effort.   I have to depend on content filtering to flag
   those suspicious actors, and the confirmed attackers will be given a block
   rule.

Doug Foster

On Sun, Jul 23, 2023 at 9:30 AM Neil Anuskiewicz 
wrote:

> Doug, you’ve done a fine job of explaining as I groked what you said. If I
> get I think most people here got it. I’ve been busy with work and personal
> life so haven’t had as much time to lurk here. I’m curious what sparked
> your concern now? Also, isn’t it best to block first and ask questions
> later to mitigate damage while you investigate? I guess I’m saying the two
> ideas aren’t mutually exclusive.
>
> Neil
>
> > On Jul 15, 2023, at 4:27 AM, Douglas Foster <
> dougfoster.emailstanda...@gmail.com> wrote:
> >
> > 
> > Murray recently observed, correctly, that something went horribly wrong
> with the DMARC rollout, because it caused (continues to cause) many safe
> and wanted messages to be blocked.
> >
> > My assessment was in a recent post:
> >
> > Something about the language of RFC 7489 caused implementers to focus on
> blocking individual messages, when the appropriate use of DMARC is to
> identify and investigate potentially malicious sources.
> >
> > The "message blocking" approach violates the interests of the users it
> is intended

Re: [dmarc-ietf] Protocols All The Way Down (long - sorry)

[Neil Anuskiewicz]

Michael,As someone who did work for an ESP, your bit about the conflict between short term cash flow that a short term customer with some money to spend for a few months and the optimizing for customer lifetime value. That war, or shall I say tension, was always there. I’d imagine the little ESP I worked for was not unusual. That tension must play out many places as the roles and wildly different incentives bake tension right into the cake.On Jul 12, 2023, at 6:33 AM, Dotzero  wrote:On Wed, Jul 12, 2023 at 6:47 AM Scott Kitterman  wrote:It's not news that there's a risk for SPF associated with shared IP addresses.  
That's why it's in RFC 7208 (11.4) and was in RFC 4408 similarly before.+1 

I understand your view and agree on the problem.  I also sympathize with the 
need to technically address conditions as they exist in operations.  I'm not 
convinced that we should bake the solutions into protocol, however.

As I've said before, I think this is primarily and economics/incentives 
problem, not a technical problem.  If you increase the incentives for third 
party providers to support DKIM in place of SPF, I don't think you've actually 
solved anything.The reality is that many ESPs have asked "how does DMARC fit into my business model?". My response has always been that it is not up to the community to figure out your business model. 

As fas as I've seen, the low cost (for a third party to sign using their 
customer's domain name) approach is for the third party provider to publish 
DKIM key records based on their own private keys and then tell their customer 
to put a CNAME in their DNS that points to it.  In the case of a shared 
server, the third party provider then needs to keep track of which customer is 
authorized to sign for which d= domain, but they can use their own keys/
infrastructure for doing so. That is one approach. We used to delegate subdomains and contractually require key rotation, etc. We did that after 2007 (due to Storm Worm), well before DMARC was published. There were ESPs that balked at signing with our keys and they were not considered as potential vendors. There were others that worked with us and also provided dedicated IPs that were shared among our various brands (for SPF). Many folks lag behind because they perceive the "cost" of security as higher than the cost(s) associated with the consequences of poor practices.A slightly more complicated approach would be for the domain to provide private signing keys to the 3rd party. 

The internal management function of keeping track of which customers can use 
which signing domains is essentially the same as the internal management 
function of keeping track of which customers can use which Mail From addresses 
with SPF.

My speculation (and it is that), is that the most cost sensitive third party 
providers currently use SPF only and are less likely to keep effective control 
of which customer can send from which identity precisely because it's less 
expensive.  If you change the deployment incentives so that DKIM is now more 
necessary for some of their customers, they'll no doubt deploy it, but without 
better internal controls, you'll end up with the same problems with these 
providers expressed through a different technology.In some ways, receivers/mailbox providers enable the bad practices by accepting problematic email because that requires less work in dealing with the ISP Relations people from those mailers. As a sidenote, one need only look at the ratio of deliverability people to compliance people at some of these organizations to understand priorities. A harder line would force those mailers to reevaluate their practices and offerings. This is an operations/implementation problem and not so much a protocol/standards problem. Just saying.Michael Hammer
___dmarc mailing listdmarc@ietf.orghttps://www.ietf.org/mailman/listinfo/dmarc___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Why does DKIM fail when SPF succeeds (was: DMARC2 & SPF Dependency Removal)

[Matthäus Wander]

Murray S. Kucherawy wrote on 2023-07-24 00:10:
On Sun, Jul 23, 2023 at 1:06 PM Matthäus Wander 
> wrote:


b) Messages are generated by an automated system without a Date header
and signed by a central MTA. An outgoing mail gateway then adds the
missing Date header (Postfix option 'always_add_missing_headers'), thus
invalidating the DKIM signature.


Why is the signer claiming to sign a header field ("Date", in this case) 
that isn't there?  This seems like a bug.


The signer uses a fixed set of header fields to sign, which usually 
exist or should be oversigned if nonexistent (one size fits most). The 
signer is not tailored towards this specific mail source. But yes, it's 
a bug in the system.


Regards,
Matt

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Why does DKIM fail when SPF succeeds (was: DMARC2 & SPF Dependency Removal)

[Murray S. Kucherawy]

On Sun, Jul 23, 2023 at 1:06 PM Matthäus Wander  wrote:

> b) Messages are generated by an automated system without a Date header
> and signed by a central MTA. An outgoing mail gateway then adds the
> missing Date header (Postfix option 'always_add_missing_headers'), thus
> invalidating the DKIM signature.
>

Why is the signer claiming to sign a header field ("Date", in this case)
that isn't there?  This seems like a bug.

-MSK
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Another p=reject text proposal

[Neil Anuskiewicz]

Yes sir, I’m convinced. Some of my more conservative customers who take 
security deadly serious won’t even bounce a DMARC failure with a helpful 
message. It’s discard. I respect the person I’m thinking of who hates bouncing. 
He’s appropriately paranoid for a InfoSec manager.

> On Jul 23, 2023, at 1:13 PM, Barry Leiba  wrote:
> 
> 
>> 
>> Without bounces the sender is in the dark.
> 
> Yes, if the sender is a human.
> 
> Not so, if the sender is a mailing list and that sender will then
> unsubscribe the intended recipient.
> Also not so, if the sender is a malfeasant who may use the bounce
> message for bad purposes.
> 
> It's very clear to me that if I think a bounce message will be
> harmful, I will not send one.  I will happily prefer silent discard
> over a bounce when I think that's a better approach for that
> situation.  Bouncing a legitimate mailing list message is just bad.
> If you have reason to believe you're going to do that... don't.
> Either deliver the message or silently discard it.  But don't bounce
> it.
> 
> Barry

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] DMARC and Data Hygiene

[Neil Anuskiewicz]

 At a fundamental authentication property level, there are additional differences too.  IPs tend to be shared more than private keys. Proving an IP based validation to a third party is harder than with a digital signature.  (This is one the issues of trusting the SPF results in ARC-Authentication-Result, whereas with DKIM once can try to validate the signature yourself).  On the other hand it's only fair to make the observation that SPF is easier to set up than DKIM and we want to capture those benefits as well.Is dkim really harder than SPF? I think part of the reason some of mega ESPs only support dkim alignment is because a lot can be automated. And you’re addiding something, not changing something (I.e., the envelope from). In most cases these days implementing dkim on the user side involves copying and pasting a txt or, increasingly common, a CNAME. It’s easier for everyone.I think DKIM’s simple for the end user at the mega ESPs. The incentives push them to make it as easy possible, while punting or shoving aligned spf aside.No, SPF has its place unless I’ve been a way from this list for so long I didn’t get the memo on that. So yes not deprecated until we come up with something a bit better or it becomes more of a liability than an asset.We're definitely not saying SPF should be deprecated, and in fact we want more deployment of SPF as it's highly complementary to DKIM both from a deployment perspective but as a spam fighting signal.  We just want to prevent its use in From header spoofing.  [1] https://www.sysnet.ucsd.edu/~voelker/pubs/forwarding-eurosp23.pdf 

I'll lay out examples to demonstrate:

Case 1 - First Party Only

An organization only authorizes email to be sent from MTAs it directly 
controls (I know this mostly doesn't exist, but it's useful to show where the 
problems are).  The organization does not send email for any third parties.

In this case, as long as the organization can limit sending to authorized 
users, the quality of both SPF and DKIM pass results should be well aligned 
with the nature of the mail the organization sends.

Case 2 - Sender For Others, Own Domain

An domain uses it's own domain to send mail for others (like all webmail 
providers).  The domain's email is sent using it's own infrastructure, so it 
doesn't need to worry directly about third parties.

In this case, there's still no real risk of external forgery, so an SPF or 
DKIM pass means it was sent by the domain.  The risk is to the domain's 
reputation if users sign up and use the service to send "bad" mail.

If the domain fails to prevent 'bad' messages from being sent, then these 'bad 
messages get a DMARC pass.  The mail is sent through the authorized email 
server, so it gets SPF pass and a valid DKIM signature.

This is a particular threat for DKIM, because once this succeeds for a single 
message, the user can replay the message on third party infrastructure to send 
it to large numbers of recipients.  This is the core issue the DKIM working 
group was resurrected to address.  It is not, however, clear that there's a 
protocol solution to this problem and the group has been pretty quiet 
recently.Just pointing out we're prototyping one of the drafts and there's other work behind the scene. 
Case 3 - Sender For Others, Their Domain

When organizations authorized third parties to send on their behalf (by 
publishing an SPF record or a DKIM public key record in DNS), they are 
trusting their domain's reputation to those third parties.

In particular, they are at risk of those third parties doing poor inbound 
validation and other customers of the authorized third parties injecting 'bad' 
mail authorized by their domain.

This is where I think the focus of recent discussion has been.

In these cases, email is emitted via third parties and passes SPF (may also be 
DKIM signed, but I suspect it's less common because replay is easier), so it's 
authorized, but unwanted.

Case 2 and Case 3 are both real problems, but only because people are trying 
to make DMARC pass a positive assertion.  In order to do that reliably, 
organizations need to be more like Case 1 and be very careful vetting third 
parties that they authorize for.  I don't think that's a protocol problem.I'm not sure.  Much as Case 1 provides safety, the email Internet today has a large part that's based on Case 2 and 3 and it's up to us to frame standards that provide safety for our users by solving the issues you mentioned.  (Or if those needs are ignored, then likely those users will move to walled gardens that can claim more safety)  For case 2, we (and others) put drafts that provide a technical solution.  For case 3, we posed some ideas that can hopefully help.-Wei

Scott K




___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

___dmarc mailing listdmarc@ietf.orghttps://www.ietf.org/mailman/listinfo/dmarc__

Re: [dmarc-ietf] DMARC and Data Hygiene

[Neil Anuskiewicz]

On Jun 20, 2023, at 11:42 PM, Wei Chuang  wrote:On Tue, Jun 20, 2023 at 8:18 AM Scott Kitterman  wrote:I am starting a separate thread, because this isn't just about SPF.

I think the criticism of the reliability of SPF data is valid, but I think 
DKIM is similarly problematic.  Once you get rid of SPF, you'll find you 
haven't really solved much.  The next logical step will be to get rid of DKIM.

DKIM has man wonderful features, but the bottom line for DMARC is a valid DKIM 
signature indicates that the mail was sent (at some point) from an MTA that 
was authorized by the signing domain.  This is what a SPF pass result means 
too.  DKIM has broader utility in DMARC because it can persist through some 
indirect mail flows, but either way, an SPF pass and a valid DKIM signature 
both mean the message was authorized by the domain in the relevant identity.

The particular SPF problem that's being the focus of some much attention is 
addressed in the RFC 7208 security considerations (See section 11.4).

DKIM has a similar, but different problem.  The DKIM replay problem is bad 
enough that the IETF has chartered a working group to try to address it:

https://datatracker.ietf.org/doc/charter-ietf-dkim/I would argue we can see a difference in the SPF and DKIM attacks with respect to spoofing and that makes a difference for phishing.  With the SPF upgrade attack, the adversary can spoof an arbitrary identity in the victim domain and have it SPF authenticated.  See [1] for examples.  With DKIM replay, the adversary needs access to an account in the victim domain to obtain a signed message, but can't arbitrarily spoof some other identity in the victim domain.  At a fundamental authentication property level, there are additional differences too.  IPs tend to be shared more than private keys. Proving an IP based validation to a third party is harder than with a digital signature.  (This is one the issues of trusting the SPF results in ARC-Authentication-Result, whereas with DKIM once can try to validate the signature yourself).  On the other hand it's only fair to make the observation that SPF is easier to set up than DKIM and we want to capture those benefits as well.We're definitely not saying SPF should be deprecated, and in fact we want more deployment of SPF as it's highly complementary to DKIM both from a deployment perspective but as a spam fighting signal.  We just want to prevent its use in From header spoofing.  [1] https://www.sysnet.ucsd.edu/~voelker/pubs/forwarding-eurosp23.pdf 

I'll lay out examples to demonstrate:

Case 1 - First Party Only

An organization only authorizes email to be sent from MTAs it directly 
controls (I know this mostly doesn't exist, but it's useful to show where the 
problems are).  The organization does not send email for any third parties.

In this case, as long as the organization can limit sending to authorized 
users, the quality of both SPF and DKIM pass results should be well aligned 
with the nature of the mail the organization sends.

Case 2 - Sender For Others, Own Domain

An domain uses it's own domain to send mail for others (like all webmail 
providers).  The domain's email is sent using it's own infrastructure, so it 
doesn't need to worry directly about third parties.

In this case, there's still no real risk of external forgery, so an SPF or 
DKIM pass means it was sent by the domain.  The risk is to the domain's 
reputation if users sign up and use the service to send "bad" mail.

If the domain fails to prevent 'bad' messages from being sent, then these 'bad 
messages get a DMARC pass.  The mail is sent through the authorized email 
server, so it gets SPF pass and a valid DKIM signature.

This is a particular threat for DKIM, because once this succeeds for a single 
message, the user can replay the message on third party infrastructure to send 
it to large numbers of recipients.  This is the core issue the DKIM working 
group was resurrected to address.  It is not, however, clear that there's a 
protocol solution to this problem and the group has been pretty quiet 
recently.Just pointing out we're prototyping one of the drafts and there's other work behind the scene. 
Case 3 - Sender For Others, Their Domain

When organizations authorized third parties to send on their behalf (by 
publishing an SPF record or a DKIM public key record in DNS), they are 
trusting their domain's reputation to those third parties.

In particular, they are at risk of those third parties doing poor inbound 
validation and other customers of the authorized third parties injecting 'bad' 
mail authorized by their domain.

This is where I think the focus of recent discussion has been.

In these cases, email is emitted via third parties and passes SPF (may also be 
DKIM signed, but I suspect it's less common because replay is easier), so it's 
authorized, but unwanted.

Case 2 and Case 3 are both real problems, but only because people are trying 
to make DMARC pass a

Re: [dmarc-ietf] Idle Musings - Why Is It DMARC and not DMARD?

[Neil Anuskiewicz]

> On Jun 30, 2023, at 11:33 AM, Dave Crocker  wrote:
> 
> On 6/30/2023 11:22 AM, Todd Herr wrote:
>> Why is the mechanism called "Domain-based Message Authentication, Reporting, 
>> and Conformance" and not "Domain-based Message Authentication, Reporting, 
>> and Disposition"?
> 
> Say DMARC out loud.  Now say DMARD out loud.

I like the way you think, Mr. Crocker. What a difference a letter makes. Dmarc 
sounds important, serious. DMARD sounds like something a high college friend 
might have made up to describe something stupid. 
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Another p=reject text proposal

[Barry Leiba]

> Without bounces the sender is in the dark.

Yes, if the sender is a human.

Not so, if the sender is a mailing list and that sender will then
unsubscribe the intended recipient.
Also not so, if the sender is a malfeasant who may use the bounce
message for bad purposes.

It's very clear to me that if I think a bounce message will be
harmful, I will not send one.  I will happily prefer silent discard
over a bounce when I think that's a better approach for that
situation.  Bouncing a legitimate mailing list message is just bad.
If you have reason to believe you're going to do that... don't.
Either deliver the message or silently discard it.  But don't bounce
it.

Barry

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] I-D Action: draft-ietf-dmarc-dmarcbis-28.txt

[Neil Anuskiewicz]

I hated that concept until I tried it out with a client, a law firm, and they 
loved it and I saved them massive headaches. The general purpose we used was a 
subdomain just for lists and it’s worked perfectly so far.

I’m now in favor of advocating general purpose domains. Let’s get ESR to put 
that term in the Hacker’s Dictionary and we’ll be set. ☺️

> On Jul 7, 2023, at 12:18 AM, Alessandro Vesely  wrote:
> 
> On Thu 06/Jul/2023 21:01:48 +0200 internet-drafts wrote:
>> A New Internet-Draft is available from the on-line Internet-Drafts
>> directories. This Internet-Draft is a work item of the Domain-based Message
>> Authentication, Reporting & Conformance (DMARC) WG of the IETF.
>>Title   : Domain-based Message Authentication, Reporting, and 
>> Conformance (DMARC)
>>Authors : Todd M. Herr
>>  John Levine
>>Filename: draft-ietf-dmarc-dmarcbis-28.txt
>>Pages   : 72
>>Date: 2023-07-06
> 
> 
> Section 7.6, Expansion of Domain Owner Actions Section, final paragraph:
> 
>In particular, this document makes explicit that domains for general-
>purpose email MUST NOT deploy a DMARC policy of p=reject.
> 
> Wasn't this rejected already?
> 
> 
> Best
> Ale
> -- 
> 
> 
> 
> 
>
> 
> ___
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

[dmarc-ietf] Why does DKIM fail when SPF succeeds (was: DMARC2 & SPF Dependency Removal)

[Matthäus Wander]

Barry Leiba wrote on 2023-06-10 01:50:

That's interesting and disturbing if it remains consistent.
Theoretically, DKIM should *never* fail when SPF succeeds, so if
that's happening it means there is:
1. bad signing software,
2. bad verifying software,
3. misconfiguration somewhere,
...or a combination of those three.

I would *really* like to see a current study of this, because I think
it's critical for the future viability of DMARC, whether or not we
accept the proposal to remove SPF.

Not a study, but some data points I've observed:

a) Signing with 3072-bit RSA leads to DKIM verification failures, 
because a popular mail gateway product (Cisco ESA) does not support RSA 
key lengths larger than 2048 bit.


b) Messages are generated by an automated system without a Date header 
and signed by a central MTA. An outgoing mail gateway then adds the 
missing Date header (Postfix option 'always_add_missing_headers'), thus 
invalidating the DKIM signature.


Such misconfigurations go unnoticed for years until someone checks the 
DMARC reports. While aggregate reports are incredibly helpful, it is 
still difficult to identify the cause of subtle DKIM failures. I'd wish 
that the  field would be filled by reporting software with 
the DKIM verification error message ('body hash did not verify', etc.) 
to aid with troubleshooting.


Contacting the report  or postmaster address has never worked for 
me: if they don't bounce, nobody replies.


c) There is a pattern of similar looking reports, which omit the  
element in the  altogether and always report 
fail in the policy result. I suspect a product, which makes 
it a bit too easy to enable DMARC validation without also enabling DKIM 
verification, but I wasn't able to identify the product yet.


Regards,
Matt

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Protocols All The Way Down (long - sorry)

[Neil Anuskiewicz]

> On Jul 12, 2023, at 3:47 AM, Scott Kitterman  wrote:
> 
> It's not news that there's a risk for SPF associated with shared IP 
> addresses.  
> That's why it's in RFC 7208 (11.4) and was in RFC 4408 similarly before.
> 
> I understand your view and agree on the problem.  I also sympathize with the 
> need to technically address conditions as they exist in operations.  I'm not 
> convinced that we should bake the solutions into protocol, however.
> 
> As I've said before, I think this is primarily and economics/incentives 
> problem, not a technical problem.  If you increase the incentives for third 
> party providers to support DKIM in place of SPF, I don't think you've 
> actually 
> solved anything.

Perhaps not but sans any additional incentives the big ESPs, especially in the 
SME niche, are already doing the CNAME efficient system and that efficiency 
creates its own incentives. SPF is hard to scale especially in the SME focused 
ESP with a massive customer base. From the ESP point of view it’s a win, I 
think, so this trend will likely continue.

Perhaps the value neutral discussion of SPF OR DKIM logic could be less 
dispassionate, remembering that the IETF WG has the technical soft leadership. 
We can see where things are going and think accordingly. I think that standards 
are your ostensible mission but really you also need to be a top shelf sales 
organization.

Maybe in some ways DKIM and SPF need to be put in different cribs so the 
emphcise isn’t on the OR. Perhaps it’s on the relative value of SPF and dkim 
even though in theory they could get you to the goal the same. It’s ok to 
decide dkim is sold and spf is a real nice to have add on like a real nice 
stereo. No need to be heavy handed if you sell effectively.  
> 
> As fas as I've seen, the low cost (for a third party to sign using their 
> customer's domain name) approach is for the third party provider to publish 
> DKIM key records based on their own private keys and then tell their customer 
> to put a CNAME in their DNS that points to it.  In the case of a shared 
> server, the third party provider then needs to keep track of which customer is
> authorized to sign for which d= domain, but they can use their own keys/
> infrastructure for doing so.
> 
> The internal management function of keeping track of which customers can use 
> which signing domains is essentially the same as the internal management 
> function of keeping track of which customers can use which Mail From 
> addresses 
> with SPF.
> 
> My speculation (and it is that), is that the most cost sensitive third party 
> providers currently use SPF only and are less likely to keep effective 
> control 
> of which customer can send from which identity precisely because it's less 
> expensive.  If you change the deployment incentives so that DKIM is now more 
> necessary for some of their customers, they'll no doubt deploy it, but without
> better internal controls, you'll end up with the same problems with these 
> providers expressed through a different technology.
> 
> Scott K
> 
>> On Tuesday, July 11, 2023 8:49:28 PM EDT Wei Chuang wrote:
>> The data we presented June 20th (archive
>> )
>> also suggests that it is premature to drop SPF from DMARC.  However I
>> differ in believing that we should accept the spoofing risk demonstrated
>> recently via SPF, and that we shouldn't strive to reduce it by making
>> improvements to DMARC and possibly SPF.  One approach is to enable
>> different senders to distinguish their differing levels of risk and
>> infrastructure by letting them specify an "auth=" flag proposed on the 20th
>> .
>> For example a mom-and-pop sender on their own infrastructure with their own
>> IPs will have a different profile than a risky, enterprise sender on shared
>> IPs.  The former would be reasonably safe using SPF and would benefit most
>> from using it.  The latter should use DKIM authentication only due to the
>> risk of SPF upgrade spoofing and the phishing exposure.  I describe some
>> additional ideas about how to use "auth=" flag with different risk and
>> infrastructure stratification on the DKIM list on July 3rd (archive
>> > /> ).
>> 
>> And FWIW I don't think mom-and-pop senders will find it that hard to adopt
>> DKIM if sufficiently motivated.  For example for BIMI on Gmail, we now
>> require DKIM-only authentication.  Indeed a new BIMI sender that I perceive
>> to be running literally a "mom-and-pop" produce shop was using SPF
>> authentication and was confused as to why their logo wasn't showing.  After
>> explaining the requirement they were able to move to DKIM authentication in
>> one day.  Yes, it did take explaining twice what was happening, but they
>> were able to deploy DKIM on their own and got their

Re: [dmarc-ietf] Another p=reject text proposal

[Neil Anuskiewicz]

> On Jul 7, 2023, at 8:48 AM, John Levine  wrote:
> 
> It appears that Barry Leiba   said:
>> I, too, prefer MUST to SHOULD there, but it was clear to me that we
>> will not reach rough consensus on MUST, but that we can reach rough
>> consensus on SHOULD.
>> 
>> I do like your suggestion of silent discard rather than bounce, and I
>> would want to see that change made -- perhaps with a note that
>> aggregate reports will still include information about those discards.
> 
> Silent discard breaks RFC 5321 and some people feel very strongly
> about it. For reasons I am sure I don't need to remind you of, don't
> go there.
> 
> If you get so many bounces that you find it annoying, that is nature's
> way of reminding you that you should do something about it.  Personally,
> I use sorting rules to put the bounces in a place where they don't
> clutter up my main inbox.

Without bounces the sender is in the dark. This is a tech ecosystem. Every such 
community I’ve been involved with in any way has had the “we are all in this 
together.” I’ve never experienced more generosity than from fellow techies. My 
second techie job I just hinted I was struggling with something and 15 minutes 
later he at my office door pointing in the direction of solutions at like 7 pm 
on a work day. No all nighter needed. He stayed to watch me take his guidance 
then we called it a day.

I help others whenever I possibly can as that’s the ethos engrained in me by 
mentors such as him. He didn’t just teach me techie stuff. He taught me the 
ethos. That ethos must continue.

We help each other in all kinds of ways including bouncing email to help the 
admin do the job. We wouldn’t be able to sleep if we made someone’s life harder 
because of a fuck up. That’s who we are.
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Another p=reject text proposal

[Neil Anuskiewicz]

> On Jul 6, 2023, at 12:04 PM, Barry Leiba  wrote:
> 
> As I've said before, I think we should specify what we think is right,
> and allow implementers to make their decisions.  We can't and won't
> police them.

No way. It wouldn’t be possible even if we all made it our life’s work to be 
standards cops. I agree with stay on target, on scope with pointers as a 
courtesy when appropriate. I’ve not visited in a while (busy) but I feel the 
energy is more focused, with people prepared to swap away distractions, time 
sinks, and wild goose chases. You all are an impressively disciplined group.
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Neil Anuskiewicz]

>> 
>> 
> 
> I think it is a mistake to consider technologies such as SPF, DKIM, and DMARC 
> as anti-spam.  They are a component of spam mitigation, but authentication of 
> an identifier isn't a solution for spam.  Meng Wong (for those of you that 
> weren't around, he's the one that synthesized SPF out of previous proposals 
> and was the early leader of the SPF project) used to say that SPF is not 
> anti-spam in the same way that flour is not food.  I think that extends to 
> DKIM and DMARC as well.
> 
> The challenge with the argument that this is brand protection, so the brand 
> owner should decide is that it's primarily the receiver that needs to invest 
> in integrating these technologies into their systems and accept the 
> inevitable support burden that comes with them.  The brand owner wants 
> something from the receiver, so it needs to be simple and reliable enough to 
> be worthwhile.
> 
> SPF includes a policy mechanism so that receivers can reject mail that is not 
> authorized by SPF.  Outside of small sites with a lot of control over their 
> mail stream, it's almost never used .  It was tried and for large providers 
> with a heterogeneous user base it wasn't cost effective to support due to the 
> number of false positives and the associated tech support costs.
> 
> DMARC seems to be heading the same way due to domains using p=reject that (at 
> least in my opinion) shouldn't.
> 
> Since there are no internet police, deployment of these technologies is a 
> matter of incentives.  We do protocol design, not economics, so it's a tough 
> problem in the IETF context, but we need to keep it in mind.  Getting the 
> incentives right is probably the most important, but least tractable part of 
> https://www.ietf.org/mailman/listinfo/dmarc

I agree that none of this primarily anti spam. It’s pro brand rep and anti 
fraud. 

I saw what happened with SPF and how it seemed completely random who would 
decide to enforce on hardfail. I recall a client losing mail as they used a 
hard fail and suddenly one of the more well known hosting companies started 
honoring hard fails which was a fiasco for my client who had just copied and 
pasted the -all in without thought to the implications. The outcomes were bad 
in a random way. It wasn’t hard to persuade him back to soft fail.

I feel with dmarc people come to recognize dmarc as the policy layer. SPF had 
too much responsibility. When one goes to p=reject one’s palms get sweaty. It’s 
clearly the policy layer unlike -all which was a bit too subtle way to 
communicate policy intent. Dmarc has the intuitive grok advantage.

I hope you’re wrong or that there’s a way to mitigate this problem you speak 
of. As I remember that as a time of confusion. Confusion and chaos are our foes.
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Scott Kitterman]

On July 21, 2023 6:53:03 PM UTC, "Jan Dušátko" 
 wrote:
>
>
>Dne 21. 7. 2023 v 16:34 Murray S. Kucherawy napsal(a):
>> On Wed, Jul 19, 2023 at 6:31 PM Douglas Foster <
>> dougfoster.emailstanda...@gmail.com> wrote:
>> 
>>> I don't take DMARC as a certain result to be used in isolation, but
>>> clearly a quorum evaluators do, and hence the mailing list problem that has
>>> caused such consternation.
>>> 
>>> If we want to diminish their numbers, we have to communicate very
>>> differently than RFC 7489.
>>> 
>>> My problem with your favorite line is that the domain owner's preference
>>> is of no interest to my filtering decision, but the DMARC result is.
>>> 
>> Since even before SPF, people have been looking for a silver bullet to stop
>> spam and phishing.  I'm not surprised to hear that there are products out
>> there that promise or implement such, despite the specifications not
>> actually saying this is a good idea, or even (in DKIM's case, I believe)
>> being rather explicit that it isn't.
>> 
>> I don't think anyone is disputing that the DMARC result by itself is not a
>> clear answer about what one should do upon receiving a message.  The only
>> time you really know something is when DMARC passes, but even that isn't a
>> strong signal about the content of the message.  All other answers are
>> muddy, and should be treated that way.  If DMARCbis needs to make this more
>> explicit, I don't see a problem with doing so.
>> 
>> I think a DMARC-aware product that's reject-on-fail by default has made a
>> questionable choice, and not making that configurable is doubly so.
>> 
>> However, I don't think an evaluator should be looking at the "p=" value and
>> then trying to infer anything about the sending domain solely from that.
>> This, to me, is crystal ball territory.  We should omit it from our
>> calculus.
>> 
>> -MSK, participating
>> 
>Although SPF, DKIM, DMARC, and ARC are all intended to protect against SPAM, 
>it is not possible, as a matter of principle, to protect against such kind of 
>communications. They will only allow, to a certain extent, protection against 
>fake emails. But SPAM is junk mail, which includes official, albeit 
>unsolicited, communications from organizations that meet the requirements of 
>the technology. It is unethical, but technically acceptable.
>Moreover, the assessment of junk mail is highly subjective, for the reason I 
>prefer the term accepted censorship. Because censorship is an intervention 
>that, as a rule, cannot be influenced by the user. For the reason given, it is 
>not possible to protect against SPAM in its entirety. These technologies only 
>allow protection against fake mail.
>This leads me to the next step, which you may not agree with. If this 
>technology serves to protect against mail being counterfeited, it can be 
>argued, with a degree of simplification, that it provides some form of brand 
>protection. And the methods of brand protection, including the hardness of the 
>methods used, should be decided by the owner. And the enforcement of those 
>rules should be ensured by the administrator. Am I thinking correctly?
>

I think it is a mistake to consider technologies such as SPF, DKIM, and DMARC 
as anti-spam.  They are a component of spam mitigation, but authentication of 
an identifier isn't a solution for spam.  Meng Wong (for those of you that 
weren't around, he's the one that synthesized SPF out of previous proposals and 
was the early leader of the SPF project) used to say that SPF is not anti-spam 
in the same way that flour is not food.  I think that extends to DKIM and DMARC 
as well.

The challenge with the argument that this is brand protection, so the brand 
owner should decide is that it's primarily the receiver that needs to invest in 
integrating these technologies into their systems and accept the inevitable 
support burden that comes with them.  The brand owner wants something from the 
receiver, so it needs to be simple and reliable enough to be worthwhile.

SPF includes a policy mechanism so that receivers can reject mail that is not 
authorized by SPF.  Outside of small sites with a lot of control over their 
mail stream, it's almost never used .  It was tried and for large providers 
with a heterogeneous user base it wasn't cost effective to support due to the 
number of false positives and the associated tech support costs.

DMARC seems to be heading the same way due to domains using p=reject that (at 
least in my opinion) shouldn't.

Since there are no internet police, deployment of these technologies is a 
matter of incentives.  We do protocol design, not economics, so it's a tough 
problem in the IETF context, but we need to keep it in mind.  Getting the 
incentives right is probably the most important, but least tractable part of 
this.

Scott K

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Neil Anuskiewicz]

Doug, you’ve done a fine job of explaining as I groked what you said. If I get 
I think most people here got it. I’ve been busy with work and personal life so 
haven’t had as much time to lurk here. I’m curious what sparked your concern 
now? Also, isn’t it best to block first and ask questions later to mitigate 
damage while you investigate? I guess I’m saying the two ideas aren’t mutually 
exclusive.

Neil

> On Jul 15, 2023, at 4:27 AM, Douglas Foster 
>  wrote:
> 
> 
> Murray recently observed, correctly, that something went horribly wrong with 
> the DMARC rollout, because it caused (continues to cause) many safe and 
> wanted messages to be blocked.
> 
> My assessment was in a recent post:
> 
> Something about the language of RFC 7489 caused implementers to focus on 
> blocking individual messages, when the appropriate use of DMARC is to 
> identify and investigate potentially malicious sources.
> 
> The "message blocking" approach violates the interests of the users it is 
> intended to protect:
> 
> - An attacker sends 10 messages that maliciously impersonates a big bank.  
> With help from DMARC p=reject, the evaluator blocks them all.  The attacker 
> follows up with 10 messages that maliciously impersonate a major university.  
>  The stupid evaluator says, "p=none means no problem here".   The message is 
> accepted and the user is harmed because the evaluator learned nothing from 
> blocking the successful attack.
> 
> Consider a variant of the above:   The attacker never impersonates a big bank 
> with p=reject, it only impersonates big universities with p=none.   The 
> foolish evaluator blocks nothing because it only acts on domains with 
> p=reject.  Again, the user is harmed.
> 
> And we know the flip side:   Safe and wanted messages get blocked because 
> p=reject is enforced without investigation against mailing lists and other 
> traffic.
> 
> Security theory says that ANY unauthenticated message COULD be a malicious 
> impersonation, and worthy of investigation.   Experience says that malicious 
> impersonations are a small percentage of all unauthenticated messages.  As a 
> result, the appropriate response to an authentication failure is to 
> investigate, not to block.
> 
> I don't know exactly how to communicate the difference between message 
> blocking and sender investigation in DMARCbis, but I sure hope that we will 
> try.
> 
> Doug Foster
>  
> ___
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] How did DMARC go wrong, and how does our document fix it?

[Alessandro Vesely]

On Fri 21/Jul/2023 20:53:03 +0200 Jan Dušátko wrote:

Dne 21. 7. 2023 v 16:34 Murray S. Kucherawy napsal(a):

On Wed, Jul 19, 2023 at 6:31 PM Douglas Foster 
 wrote:

My problem with your favorite line is that the domain owner's preference 
is of no interest to my filtering decision, but the DMARC result is.


Since even before SPF, people have been looking for a silver bullet to stop 
spam and phishing.  [...]


Although SPF, DKIM, DMARC, and ARC are all intended to protect against SPAM, it 
is not possible, as a matter of principle, to protect against such kind of 
communications. They will only allow, to a certain extent, protection against 
fake emails. But SPAM is junk mail, which includes official, albeit 
unsolicited, communications from organizations that meet the requirements of 
the technology.



Exactly.  The silver bullet goal was exactly to force spammers to send their 
junk with their own name and credentials.  At that point, the narration goes, 
it will be easy to obtain domain reputation or age.


That trend is already ongoing.  There are even specialized registrars.  For 
example NameSilo offers discounts for customers who buy 5000+ (privacy 
protected) domains[*].


To be precise, SPF, DKIM, DMARC, and ARC are intended to protect against spoof, 
not against spam.  Thereafter, the focus moves to registration data and 
reputation, balancing the right to anonymity with domain responsibility.  IMHO, 
anonymity has to be granted by someone who is not anonymous in turn.



Best
Ale
--

[*] https://www.namesilo.com/pricing












___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

[dmarc-ietf] Messages from the dmarc list for the week ending Sun Jul 23 06:00:04 2023

[John Levine]

   Count|  Bytes |  Who
++---
 50 ( 100%) | 589562 ( 100%) | Total
  9 (18.0%) | 155955 (26.5%) | Douglas Foster 

  8 (16.0%) |  49794 ( 8.4%) | Alessandro Vesely 
  4 ( 8.0%) | 109702 (18.6%) | OLIVIER HUREAU 

  4 ( 8.0%) |  35411 ( 6.0%) | Murray S. Kucherawy 
  4 ( 8.0%) |  27631 ( 4.7%) | Scott Kitterman 
  4 ( 8.0%) |  19667 ( 3.3%) | John Levine 
  3 ( 6.0%) |  15678 ( 2.7%) | Baptiste Carvello 

  2 ( 4.0%) |  43790 ( 7.4%) | Dotzero 
  2 ( 4.0%) |  38595 ( 6.5%) | Mark Alley 
  2 ( 4.0%) |  16716 ( 2.8%) | Jan Dušátko 
  2 ( 4.0%) |  14587 ( 2.5%) | Tero Kivinen 
  2 ( 4.0%) |  10385 ( 1.8%) | Barry Leiba 
  1 ( 2.0%) |  21004 ( 3.6%) | Wei Chuang 
  1 ( 2.0%) |  18820 ( 3.2%) | Emanuel Schorsch 
  1 ( 2.0%) |   6364 ( 1.1%) | Matthäus Wander 
  1 ( 2.0%) |   5463 ( 0.9%) | Benny Pedersen 

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc