Re: [dmarc-ietf] ARC questions

[Michael Thomas]

On 11/25/20 4:14 PM, Murray S. Kucherawy wrote:
On Wed, Nov 25, 2020 at 11:03 AM Michael Thomas > wrote:



That's been known for over 15 years. I'm still trying to
understand the assertion that DKIM signatures are a "bad fit". I
just looked at a random message off of this thread and they look
identical except for the i= field. another field could trivially
be added to DKIM and auth-res -- since unknown fields are supposed
to be ignored -- if this binding property is absolutely needed,
which I remain unconvinced by as well.

I'm not sure about "bad fit".  The original plan was to have an 
Authentication-Results (AR) clone called 
Original-Authentication-Results (OAR) which was specifically the AR 
content generated at the first non-submission MTA. There was some 
friction (mostly from me, as I recall) about having two header fields 
with identical syntax and nearly identical semantics.  There was 
further complication in the fact that one ADMD could apply multiple 
ARs on a single message (for multiple methods, or multiple instances 
of the same method, or a combination of those), or they could be 
reordered, etc.  To make it more resilient, things like instance 
numbers were added.  The work was eventually generalized to be a 
"chain of custody" sort of model, and in doing that I think the 
decision was made to make a new name to ensure ARC and DKIM would be 
processed differently.



But what about DKIM? And why do they need to be processed differently? 
When I first saw this, I looked at the ARC-Signature which looks 
identical to a DKIM signature (i didn't notice the i= at the time), and 
am like what is this? The i= counter could be trivially be grafted onto 
DKIM if it were needed, which I'm still sort of dubious (though it is a 
nice to have feature, I admit). That way you only need a single DKIM 
signature with the OAR's signed. As far as I can tell, that would fall 
back gracefully for non-implementing DKIM verifiers. Do you disagree?





If you ask me, part of the experiment should be able to show use
cases that DKIM + auth-res (or maybe old-auth-res if needed)
CANNOT address that ARC can, and most especially what percentage
of email traffic we're actually talking here. As far as I can see
ARC doesn't bring anything especially new for the mailing list
kind of use cases since it really easy to see who the
intermediary was that broke the signature. I have been told there
are some vague other kinds of use cases but is unclear whether
those use cases actually happen before or after the message
arrives at the front door of the final receiving domain. Yes, i
read section 7, but it doesn't tell me why this can't be handled
with current technology.


Obviously it's too late to include this in RFC 8617, but those would 
indeed be interesting data to have.



Yeah, quantifying the problems kinda seems like the first order of 
business if you ask me. I mean, how many of these scenarios are in 
reality goofy self-inflicted wounds? I can't say but there seems to be a 
lot more "this can happen" than "this is how often this happens" from 
what I've see thus far on this thread.




When you say "easy to see", do you mean for software or for humans?



Software. Only software can pry apart that ball of header spaghetti. But 
I think with the simple a mailing list it is pretty easy to determine, 
which now that I think about it I actually did back in the day when I 
was experimenting with recovering mailing list modifications. It didn't 
occur to me that that was supposed to be hard.




It seems to me that the lynchpin is whether I trust the domain
that broke the signature and resigned. That's either a previously
solved or unsolved problem depending on your perspective. If I
trust the intermediary domain to not send me spam via reputation,
I forward it along to be delivered and ignore the DMARC record.
Why do I care whether it originally validated from the sending
domain or not? If you ask me, that's the intermediary domain's
problem since they have an incentive to keep their reputation and
not send spam through.

Suppose you're sending to a list that I'm on, I enforce DMARC, I trust 
that MLM not to send spam via reputation, you have a good reputation, 
and you publish a DMARC "reject" policy.  That means if a bad actor 
sends a message to the list as you but doesn't sign it at all, I'll 
still accept it because I trust that MLM even though according to your 
policy request, I should bounce it.  ARC provides the missing data I 
need to enforce your request.


Sure, but the easier answer: to use my mailing list, you must have 
either DKIM, SPF or both. Sounds like a good way to not be essentially 
an open relay. Don't mailing lists have those sorts of policies these 
days? I would say that ones that don't probably shouldn't have good 
reputations since they are, in effect... 

Re: [dmarc-ietf] ARC questions

[Murray S. Kucherawy]

On Wed, Nov 25, 2020 at 11:03 AM Michael Thomas  wrote:

> On 11/24/20 8:19 PM, Murray S. Kucherawy wrote:
>
> On Tue, Nov 24, 2020 at 7:27 PM Douglas Foster <
> dougfoster.emailstanda...@gmail.com> wrote:
>
>> Michael, I think the purpose is stated well enough:   Mailing lists want
>> to keep adding their content to messages, without being blocked by
>> recipients.   This means that they have to provide recipients with enough
>> information for them to accept the forwarded content.   Google and
>> Microsoft seem to be on-board with this project, so it seems pretty
>> successful already.   This train is not easily stopped.
>>
>
> That sounds about right.  Put another way: DMARC's success is at least in
> part stymied by what MLMs do that invalidates DKIM; ARC is an attempt to
> carry forward from the MLM, in a credible way, an indication of what the
> MLM saw in terms of DKIM results when the message got to the MLM.  So then,
> although an author signature will fail post-MLM, the MLM signature will
> pass, and the ARC data will tell you that the author signature was good
> when the MLM got it.  If you trust the chain, then that can be an
> alternative to the DKIM input when the final recipient performs a DMARC
> evaluation.
>
> That's been known for over 15 years. I'm still trying to understand the
> assertion that DKIM signatures are a "bad fit". I just looked at a random
> message off of this thread and they look identical except for the i= field.
> another field could trivially be added to DKIM and auth-res -- since
> unknown fields are supposed to be ignored -- if this binding property is
> absolutely needed, which I remain unconvinced by as well.
>
I'm not sure about "bad fit".  The original plan was to have an
Authentication-Results (AR) clone called Original-Authentication-Results
(OAR) which was specifically the AR content generated at the first
non-submission MTA.  There was some friction (mostly from me, as I recall)
about having two header fields with identical syntax and nearly identical
semantics.  There was further complication in the fact that one ADMD could
apply multiple ARs on a single message (for multiple methods, or multiple
instances of the same method, or a combination of those), or they could be
reordered, etc.  To make it more resilient, things like instance numbers
were added.  The work was eventually generalized to be a "chain of custody"
sort of model, and in doing that I think the decision was made to make a
new name to ensure ARC and DKIM would be processed differently.

I imagine it could've been done as an Applicability Statement over DKIM and
AR, but I'm not sure now whether that would've been simpler.

> If you ask me, part of the experiment should be able to show use cases
> that DKIM + auth-res (or maybe old-auth-res if needed) CANNOT address that
> ARC can, and most especially what percentage of email traffic we're
> actually talking here. As far as I can see ARC doesn't bring anything
> especially new for the mailing list kind of use cases since it really easy
> to see who the intermediary was that broke the signature. I have been told
> there are some vague other kinds of use cases but is unclear whether those
> use cases actually happen before or after the message arrives at the front
> door of the final receiving domain. Yes, i read section 7, but it doesn't
> tell me why this can't be handled with current technology.
>
> Obviously it's too late to include this in RFC 8617, but those would
indeed be interesting data to have.

When you say "easy to see", do you mean for software or for humans?

> It seems to me that the lynchpin is whether I trust the domain that broke
> the signature and resigned. That's either a previously solved or unsolved
> problem depending on your perspective. If I trust the intermediary domain
> to not send me spam via reputation, I forward it along to be delivered and
> ignore the DMARC record. Why do I care whether it originally validated from
> the sending domain or not? If you ask me, that's the intermediary domain's
> problem since they have an incentive to keep their reputation and not send
> spam through.
>
Suppose you're sending to a list that I'm on, I enforce DMARC, I trust that
MLM not to send spam via reputation, you have a good reputation, and you
publish a DMARC "reject" policy.  That means if a bad actor sends a message
to the list as you but doesn't sign it at all, I'll still accept it because
I trust that MLM even though according to your policy request, I should
bounce it.  ARC provides the missing data I need to enforce your request.

If the MLM enforces DMARC, that's slightly better, but I think ARC was
meant to be an incremental drop-in for MLM operators that don't (or won't)
do DMARC.

-MSK
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[John Levine]

In article <695b8714-b174-e3d6-d6c0-1a1d535fb...@mtcc.com> you write:
>Not everything is service provider. We were investigating this from an 
>enterprise standpoint.
>
>And if you can't trust mailing traffic from providers what is the point 
>of ARC?

Um, please see the previous umpteen messages describing how ARC works
and how it addresses this particular problem.

R's,
John

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Michael Thomas]

On 11/25/20 12:31 PM, John Levine wrote:

In article ,
Michael Thomas   wrote:

When I was at Cisco, with l= and some subject line heuristics I could
get probably like 90+% verification rate across the entire company, a
company that uses external mailing lists a lot. Definitely not 100% though.

I think you will find that at very large mail systems like gmail and
Microsoft and Yahoo, 90% might as well be 0%. The volume of errors is
just too high and the number of complaints would be impossible.

While I almost never see the sort of spam leakage through mailing lists
that Brandon reports, I believe him when he says it's enough of a problem
that Gmail can't just whitelist traffic from mailing lists.



Not everything is service provider. We were investigating this from an 
enterprise standpoint.


And if you can't trust mailing traffic from providers what is the point 
of ARC?


Mike

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[John Levine]

In article ,
Michael Thomas   wrote:
>When I was at Cisco, with l= and some subject line heuristics I could 
>get probably like 90+% verification rate across the entire company, a 
>company that uses external mailing lists a lot. Definitely not 100% though.

I think you will find that at very large mail systems like gmail and
Microsoft and Yahoo, 90% might as well be 0%. The volume of errors is
just too high and the number of complaints would be impossible.

While I almost never see the sort of spam leakage through mailing lists 
that Brandon reports, I believe him when he says it's enough of a problem
that Gmail can't just whitelist traffic from mailing lists.

R's,
John
-- 
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Michael Thomas]

On 11/25/20 11:11 AM, Alessandro Vesely wrote:

Hi,

On 25/11/2020 19:24, Jesse Thompson wrote:

On 11/25/20 11:30 AM, Alessandro Vesely wrote:
Without resorting to ARC, it is still possible to validate author 
domain's signatures directly if the MLM just adds a subject tag and 
a footer, like, for example, this list does.   While ARC solves 
"deep" forwarding problems, which may arise in the context of email 
address portability, MLM transformation reversion solves the simpler 
mailing list problem, including reverting munged From:'s.


I agree that ARC isn't really needed to do this (trust the last hop 
from the MLM and determine the original authenticity from the MLM's 
perspective)



I didn't mean to trust the MLM.  I meant remove the subject tag and 
the footer, then the original DKIM signature verifies.  See:

https://datatracker.ietf.org/doc/draft-vesely-dmarc-mlm-transform/



When I was at Cisco, with l= and some subject line heuristics I could 
get probably like 90+% verification rate across the entire company, a 
company that uses external mailing lists a lot. Definitely not 100% though.


Mike

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Alessandro Vesely]

Hi,

On 25/11/2020 19:24, Jesse Thompson wrote:

On 11/25/20 11:30 AM, Alessandro Vesely wrote:

Without resorting to ARC, it is still possible to validate author domain's signatures 
directly if the MLM just adds a subject tag and a footer, like, for example, this list 
does.   While ARC solves "deep" forwarding problems, which may arise in the 
context of email address portability, MLM transformation reversion solves the simpler 
mailing list problem, including reverting munged From:'s.


I agree that ARC isn't really needed to do this (trust the last hop from the 
MLM and determine the original authenticity from the MLM's perspective)



I didn't mean to trust the MLM.  I meant remove the subject tag and the footer, 
then the original DKIM signature verifies.  See:

https://datatracker.ietf.org/doc/draft-vesely-dmarc-mlm-transform/



Plus, if it eventually solves the "deep" forwarding issue, then ARC is 
certainly better than trying to follow received header chains, etc.



IMHO, that's where the real value or ARC lies.  Large mailbox providers forward 
lots of messages to one another, as set up by users, and they seem to prefer to 
forward messages anyway rather than filter before forwarding.  That's what John 
reported in:

https://mailarchive.ietf.org/arch/msg/dmarc/OmTzwzP9GuE1oF5m1TvUZVA799c



Anecdotally, after much debate, our team is leaning more towards *not* 
reverting munged From:'s from our own MLM

1. Until ARC has a reputation model that is commonly adopted, header munging 
isn't going to subside.  I still find MLM operators who are just now realizing 
that they have to munge messages.  We need to tell users that this is the new, 
growing, reality.



Yup.



2. If we only unmunge for our own domains' users' authoring messages to our own MLM, it 
has limited overall effect, and it distorts the user-reality story from point #1.  We 
would have to unmunge for all domains' authors sending to all "trusted" MLMs in 
order to give the users what they expect from their prior reality.

3. Since we can only unmunge for our own recipients, it just creates an 
inconsistent experience on top of the already inconsistent experience of the 
conditional munging most MLMs do based on the authors' DMARC policies.



If the original signature verifies, each MDA can restore the unmunged From: 
right before committing to local storage.  That way, the rewritten From: 
becomes a transfer artifact, not seen by users.



Best
Ale
--





















___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] ARC questions

[Michael Thomas]

On 11/24/20 7:27 PM, Douglas Foster wrote:



In my opinion, ARC does leave a lot of unanswered questions about how 
you use the data that ARC provides.   Again, the big organizations 
have the brain power at their disposal to figure that out for 
themselves, later.




They've had that data for over a decade from what I can tell. The exact 
same data that ARC provides, but perhaps a little hard to tease apart in 
some edge cases.


Mike

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] ARC questions

[Michael Thomas]

On 11/24/20 8:19 PM, Murray S. Kucherawy wrote:
On Tue, Nov 24, 2020 at 7:27 PM Douglas Foster 
> wrote:


Michael, I think the purpose is stated well enough:   Mailing
lists want to keep adding their content to messages, without being
blocked by recipients.   This means that they have to provide
recipients with enough information for them to accept the
forwarded content.  Google and Microsoft seem to be on-board with
this project, so it seems pretty successful already.   This train
is not easily stopped.


That sounds about right.  Put another way: DMARC's success is at least 
in part stymied by what MLMs do that invalidates DKIM; ARC is an 
attempt to carry forward from the MLM, in a credible way, an 
indication of what the MLM saw in terms of DKIM results when the 
message got to the MLM.  So then, although an author signature will 
fail post-MLM, the MLM signature will pass, and the ARC data will tell 
you that the author signature was good when the MLM got it.  If you 
trust the chain, then that can be an alternative to the DKIM input 
when the final recipient performs a DMARC evaluation.


That's been known for over 15 years. I'm still trying to understand the 
assertion that DKIM signatures are a "bad fit". I just looked at a 
random message off of this thread and they look identical except for the 
i= field. another field could trivially be added to DKIM and auth-res -- 
since unknown fields are supposed to be ignored -- if this binding 
property is absolutely needed, which I remain unconvinced by as well.





Sections 1 and 7.2.1 of RFC 8617 do say all of this, though perhaps 
not as concretely as one might like.


In my opinion, ARC does leave a lot of unanswered questions about
how you use the data that ARC provides.   Again, the big
organizations have the brain power at their disposal to figure
that out for themselves, later.


This is why it was published as Experimental.  Its efficacy is not 
(yet) known, nor are any side effects. Although, now that you have me 
thinking about it: It's been a year; have we any meaningful data about 
this yet?


If you ask me, part of the experiment should be able to show use cases 
that DKIM + auth-res (or maybe old-auth-res if needed) CANNOT address 
that ARC can, and most especially what percentage of email traffic we're 
actually talking here. As far as I can see ARC doesn't bring anything 
especially new for the mailing list kind of use cases since it really 
easy to see who the intermediary was that broke the signature. I have 
been told there are some vague other kinds of use cases but is unclear 
whether those use cases actually happen before or after the message 
arrives at the front door of the final receiving domain. Yes, i read 
section 7, but it doesn't tell me why this can't be handled with current 
technology.


It seems to me that the lynchpin is whether I trust the domain that 
broke the signature and resigned. That's either a previously solved or 
unsolved problem depending on your perspective. If I trust the 
intermediary domain to not send me spam via reputation, I forward it 
along to be delivered and ignore the DMARC record. Why do I care whether 
it originally validated from the sending domain or not? If you ask me, 
that's the intermediary domain's problem since they have an incentive to 
keep their reputation and not send spam through.


Mike

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Jesse Thompson]

On 11/25/20 11:30 AM, Alessandro Vesely wrote:
> Without resorting to ARC, it is still possible to validate author domain's 
> signatures directly if the MLM just adds a subject tag and a footer, like, 
> for example, this list does.   While ARC solves "deep" forwarding problems, 
> which may arise in the context of email address portability, MLM 
> transformation reversion solves the simpler mailing list problem, including 
> reverting munged From:'s.

I agree that ARC isn't really needed to do this (trust the last hop from the 
MLM and determine the original authenticity from the MLM's perspective), 
although I think that ARC somewhat standardizes what headers/value to look for 
(I may be wrong).  Plus, if it eventually solves the "deep" forwarding issue, 
then ARC is certainly better than trying to follow received header chains, etc.

Anecdotally, after much debate, our team is leaning more towards *not* 
reverting munged From:'s from our own MLM

1. Until ARC has a reputation model that is commonly adopted, header munging 
isn't going to subside.  I still find MLM operators who are just now realizing 
that they have to munge messages.  We need to tell users that this is the new, 
growing, reality.

2. If we only unmunge for our own domains' users' authoring messages to our own 
MLM, it has limited overall effect, and it distorts the user-reality story from 
point #1.  We would have to unmunge for all domains' authors sending to all 
"trusted" MLMs in order to give the users what they expect from their prior 
reality.

3. Since we can only unmunge for our own recipients, it just creates an 
inconsistent experience on top of the already inconsistent experience of the 
conditional munging most MLMs do based on the authors' DMARC policies.  

We would rather see:

3a) MLMs that munge for all messages regardless of the author's domain's DMARC 
policy 

3b) MLMs that allow operators to configure recipient destinations that they 
know are going to trust non-munged messages (via ARC or whatever)

Jesse

P.S. If anyone knows how to trick Google Groups into doing 3a and/or 3b, please 
ping me (off list is OK)

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Douglas E. Foster]

Indirect mail flows are difficult to detect.   SMTP address rewrite is already 
common practice for forwarding.More to the point, John's interest is finding 
ways to increase the trust level for forwarded mail, while your idea says that 
direct mail is more trusted than indirect maill, which is the problem he is 
trying to overcome.We need to be able to evaluate indirect mail based on both 
the submitter MTA. and the originator MTA.   ARC gets us started in that 
direction.   I think more filtering data is needed and am working on a proposal 
to that effect.Doug

 Original message 
From: Alessandro Vesely  Date: 
11/25/20  6:28 AM  (GMT-05:00) To: dmarc-ietf  
Subject: [dmarc-ietf] A policy for direct mail flows only, was ARC 
questions 
On Mon 23/Nov/2020 22:27:41 +0100 John Levine wrote:
> ARC deals with the problem that most list software forwards everything
> with a subscriber's address on the From: line and does a lousy job of
> spam filtering. The question is if the entity sending the message to
> the list was who it purported to be.
>
> For example, if a message from a list fails DMARC alignment, but ARC
> says it was aligned on the way in, it's likely a real message from a
> subscriber. If it was unaligned on the way in, it's likely spam.


I publish p=none in order to avoid spurious rejections due to casual message
modifications that happen in transit.  However, I'm quite confident that SPF or
DKIM verify, since users submit messages through the right mail server.

Couldn't I address direct flows only?  Doing so would prevent a casual spammer
from abusing mailing lists I'm subscribed to by simply faking From:.

A direct flow is one were SPF credentials (helo name or return address) are
aligned with From:.  That includes some simple forwarding, but not mailing list
traffic.  Direct policy could be expressed as dp=.  Authenticate as usual,
either SPF or DKIM.  On failure, discard only if direct flow.  For example:
   v=DMARC1; p=none; dp=reject;

Makes sense?

Best
Ale
--






















___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc


___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

[dmarc-ietf] A policy for direct mail flows only, was ARC questions

[Alessandro Vesely]

On Mon 23/Nov/2020 22:27:41 +0100 John Levine wrote:

ARC deals with the problem that most list software forwards everything
with a subscriber's address on the From: line and does a lousy job of
spam filtering. The question is if the entity sending the message to
the list was who it purported to be. 


For example, if a message from a list fails DMARC alignment, but ARC
says it was aligned on the way in, it's likely a real message from a
subscriber. If it was unaligned on the way in, it's likely spam.



I publish p=none in order to avoid spurious rejections due to casual message 
modifications that happen in transit.  However, I'm quite confident that SPF or 
DKIM verify, since users submit messages through the right mail server.


Couldn't I address direct flows only?  Doing so would prevent a casual spammer 
from abusing mailing lists I'm subscribed to by simply faking From:.


A direct flow is one were SPF credentials (helo name or return address) are 
aligned with From:.  That includes some simple forwarding, but not mailing list 
traffic.  Direct policy could be expressed as dp=.  Authenticate as usual, 
either SPF or DKIM.  On failure, discard only if direct flow.  For example:


   v=DMARC1; p=none; dp=reject;

Makes sense?

Best
Ale
--






















___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Doing a tree walk rather than PSL lookup

[Alessandro Vesely]

On Tue 24/Nov/2020 20:29:11 +0100 John R Levine wrote:



"Holy Roman Empire"


Organizations, typically universities, where the nominal organization tree and 
the actual control are different.  The PSL isn't useful because the party that 
controls their Org domain often doesn't control lower parts of the DNS tree.



The PSL takes care of particular cases, listing suffixes like cloudfront.net or 
various flavors of amazonaws.com, which officially look like any other 2nd 
level domain, but actually host independent organizations.  Those entries are 
commented with the names and email addresses of the principal who submitted 
them.  In their own words:


In addition, owners of privately-registered domains who themselves issue
subdomains to mutually-untrusting parties may wish to be added to the
PRIVATE section of the list.
https://github.com/publicsuffix/list/wiki/Guidelines

Best
Ale
--


















___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc