Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
I entirely agree that a confirmed identity is useful because it allows us to whiteliist safely. I became alienated from my commercial products because they did not understand that whitelisting should require confirmed identity. I think they should understand security principles if they are in that business.My commercial products also choke when the blacklists grow large. Once I found a customizable filter, I moved my rules into database tables. I use segment matching, instead of ends-with matching, to ensure indexed queries. It no longer matters if a blacklist has 500 entries or 15,000.IP blocking is the best way to prevent domain churning, but domain blocking helps prevrnt IP churning. I try to blacklist both at the same time.Most of my spam arrives either as first-party mail (SPF Pass with domain alignment) or as a client of an ESP, mostly Sendgrid, (with SPF Pass and no DMARC policy.) Commercial products that ignore Message From are unable to handle ESP-based spamming. ESP-based spamming also evades some URL filtering, because all of the URLs point back to the ESP.My vendors say that malicious Office365 accounts are the new attack vector, but I have not seen it yet. When it happens, my filter rules change to full address blacklists instead of domain blacklists.DFSent from my Verizon, Samsung Galaxy smartphone Original message From: "Murray S. Kucherawy" Date: 9/8/20 4:31 PM (GMT-05:00) To: Doug Foster Cc: IETF DMARC WG Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists On Tue, Sep 8, 2020 at 5:09 AM Doug Foster wrote: > However, I disagree about negative reputation.Content filtering alone > is insufficient and even more error prone. In the last year, I have had > spam campaigns about LED lighting, stand-up desks, touchless thermometers, > and knife sharpeners. I cannot anticipate all the ways that spammers will > hide their dirty deeds. But I know it is spam, not merely unwanted > advertising, because of receiving many similar messages from many different > domains using many different servers. Third-party RBLs help but are > insufficient. I am gradually building my own reputation database based on > the traffic that I am receiving. By blocking the problem sources, I have > been able to get the spam problem under something approaching good control. > Content filtering is a useful tool for day-zero detection of a new spam > source. Once a source is detected, it needs to be blocked. > > > > Whether a message passes SPF and DMARC criteria is part of my search > critieria for unwanted traffic, but definitely not the only one. As has > been observed, actual spoofing of the From address is not a huge part of my > problem at present. This is largely because spammers have easy enough > tools in Friendly Name spoofing and corporate logo misuse. But I also > attribute that low volume to the existence of SPF and DMARC. > Suppose I'm one of your touchless thermometer spammers. Your system identifies me and the DKIM signing domain I'm using. I notice, through well-established means, that my spam is no longer getting through to you. So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or whatever a few smashes of the keyboard yields, and start signing with that instead of whatever domain I was using before. For a couple of bucks, I have now escaped my negative reputation in your system. Maybe I bounce it through a botnet too, so that you can't catch me with an IP reputation either. Negative reputations are trivially shed. It follows that it's not terribly useful to track them, at least not long-term. You end up with records of spammy domains that you'll notice only sent mail for the shortest of time ranges, long enough to get in undetected or under the guise of "too new to block", and then abandoned when they stop working. Blocking domains you've never heard of before is often disruptive when, say, you join a loyalty program for some vendor you've never dealt with before and actually do want their mail, so you're between a rock and a hard place. Instead, positive reputations are the things on which you can reliably act, giving such messages preferential treatment. It's generally a much higher bar, plus the namespace of domains that manage to earn positive reputations is small, and they tend to be well-behaved over longer periods of time. Content filtering is a different matter. It's focused on what's in the message, irrespective of where it came from. But that's a whole new game to play, and definitely not anything in which DMARC is interested. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
On Tue, Sep 8, 2020 at 1:30 PM Murray S. Kucherawy wrote: > On Tue, Sep 8, 2020 at 5:09 AM Doug Foster 40bayviewphysicians@dmarc.ietf.org> wrote: > >> However, I disagree about negative reputation.Content filtering alone >> is insufficient and even more error prone. In the last year, I have had >> spam campaigns about LED lighting, stand-up desks, touchless thermometers, >> and knife sharpeners. I cannot anticipate all the ways that spammers will >> hide their dirty deeds. But I know it is spam, not merely unwanted >> advertising, because of receiving many similar messages from many different >> domains using many different servers. Third-party RBLs help but are >> insufficient. I am gradually building my own reputation database based on >> the traffic that I am receiving. By blocking the problem sources, I have >> been able to get the spam problem under something approaching good control. >> Content filtering is a useful tool for day-zero detection of a new spam >> source. Once a source is detected, it needs to be blocked. >> >> >> >> Whether a message passes SPF and DMARC criteria is part of my search >> critieria for unwanted traffic, but definitely not the only one. As has >> been observed, actual spoofing of the From address is not a huge part of my >> problem at present. This is largely because spammers have easy enough >> tools in Friendly Name spoofing and corporate logo misuse. But I also >> attribute that low volume to the existence of SPF and DMARC. >> > > Suppose I'm one of your touchless thermometer spammers. Your system > identifies me and the DKIM signing domain I'm using. I notice, through > well-established means, that my spam is no longer getting through to you. > So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or > whatever a few smashes of the keyboard yields, and start signing with that > instead of whatever domain I was using before. For a couple of bucks, I > have now escaped my negative reputation in your system. Maybe I bounce it > through a botnet too, so that you can't catch me with an IP reputation > either. > > Negative reputations are trivially shed. It follows that it's not > terribly useful to track them, at least not long-term. You end up with > records of spammy domains that you'll notice only sent mail for the > shortest of time ranges, long enough to get in undetected or under the > guise of "too new to block", and then abandoned when they stop working. > Blocking domains you've never heard of before is often disruptive when, > say, you join a loyalty program for some vendor you've never dealt with > before and actually do want their mail, so you're between a rock and a hard > place. > > Instead, positive reputations are the things on which you can reliably > act, giving such messages preferential treatment. It's generally a much > higher bar, plus the namespace of domains that manage to earn positive > reputations is small, and they tend to be well-behaved over longer periods > of time. > I disagree, we track reputations both good and bad, and they are incorporated in spam rules across the ladder. A surprising number of negative reputations are not shed, even at very-low... and sure, we do sunset reputations that go unused. At the very least, there is a time lag before the spammer notices the effect and switches. I mean, a blacklist is ultimately a determination that a reputation is so low as to block completely, and that seems to be the main way that anti-spam information is distributed and used by most medium to small providers. That set of botnet IPs definitely will get a low reputation themselves and end up on blacklists. And forcing the spammers to spend money on things like new domain names is part of the benefit. OTOH, we also don't believe in "too new to block", unknown reputation is a great reason to apply throttling at the least. Maybe some of this is large system stuff, where you can expect to see more traffic and things don't tend to be unique... but of course we also get complaints from very small volume folks as well. Brandon ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
On Tue, Sep 8, 2020 at 5:09 AM Doug Foster wrote: > However, I disagree about negative reputation.Content filtering alone > is insufficient and even more error prone. In the last year, I have had > spam campaigns about LED lighting, stand-up desks, touchless thermometers, > and knife sharpeners. I cannot anticipate all the ways that spammers will > hide their dirty deeds. But I know it is spam, not merely unwanted > advertising, because of receiving many similar messages from many different > domains using many different servers. Third-party RBLs help but are > insufficient. I am gradually building my own reputation database based on > the traffic that I am receiving. By blocking the problem sources, I have > been able to get the spam problem under something approaching good control. > Content filtering is a useful tool for day-zero detection of a new spam > source. Once a source is detected, it needs to be blocked. > > > > Whether a message passes SPF and DMARC criteria is part of my search > critieria for unwanted traffic, but definitely not the only one. As has > been observed, actual spoofing of the From address is not a huge part of my > problem at present. This is largely because spammers have easy enough > tools in Friendly Name spoofing and corporate logo misuse. But I also > attribute that low volume to the existence of SPF and DMARC. > Suppose I'm one of your touchless thermometer spammers. Your system identifies me and the DKIM signing domain I'm using. I notice, through well-established means, that my spam is no longer getting through to you. So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or whatever a few smashes of the keyboard yields, and start signing with that instead of whatever domain I was using before. For a couple of bucks, I have now escaped my negative reputation in your system. Maybe I bounce it through a botnet too, so that you can't catch me with an IP reputation either. Negative reputations are trivially shed. It follows that it's not terribly useful to track them, at least not long-term. You end up with records of spammy domains that you'll notice only sent mail for the shortest of time ranges, long enough to get in undetected or under the guise of "too new to block", and then abandoned when they stop working. Blocking domains you've never heard of before is often disruptive when, say, you join a loyalty program for some vendor you've never dealt with before and actually do want their mail, so you're between a rock and a hard place. Instead, positive reputations are the things on which you can reliably act, giving such messages preferential treatment. It's generally a much higher bar, plus the namespace of domains that manage to earn positive reputations is small, and they tend to be well-behaved over longer periods of time. Content filtering is a different matter. It's focused on what's in the message, irrespective of where it came from. But that's a whole new game to play, and definitely not anything in which DMARC is interested. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
SPF/DMARC are not optimal tools for detecting malice.In my experience, the abundance of sender configuration errors are the limiting factor. However, I disagree about negative reputation.Content filtering alone is insufficient and even more error prone. In the last year, I have had spam campaigns about LED lighting, stand-up desks, touchless thermometers, and knife sharpeners. I cannot anticipate all the ways that spammers will hide their dirty deeds. But I know it is spam, not merely unwanted advertising, because of receiving many similar messages from many different domains using many different servers. Third-party RBLs help but are insufficient. I am gradually building my own reputation database based on the traffic that I am receiving. By blocking the problem sources, I have been able to get the spam problem under something approaching good control. Content filtering is a useful tool for day-zero detection of a new spam source. Once a source is detected, it needs to be blocked. Whether a message passes SPF and DMARC criteria is part of my search critieria for unwanted traffic, but definitely not the only one. As has been observed, actual spoofing of the From address is not a huge part of my problem at present. This is largely because spammers have easy enough tools in Friendly Name spoofing and corporate logo misuse. But I also attribute that low volume to the existence of SPF and DMARC. Doug Foster From: Murray S. Kucherawy [mailto:superu...@gmail.com] Sent: Monday, September 07, 2020 4:30 PM To: Doug Foster Cc: IETF DMARC WG Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists Historically, I've found that a negative source reputation is easy to dodge. It's trivial to come from an unknown IP address or register a new domain name, so an actor with a negative reputation can trivially move to a neutral one. Thus, a receiver/verifier seeking to make a meaningful judgement can only really focus on positive reputations when making meaningful filtering decisions; everyone else is basically the same in terms of value. Another way to look at this: DKIM (and I believe SPF) only really tells you something interesting when it passes. That means (for DKIM) the content was unmodified, and the signature is validated by a key that is verifiably present in some domain's DNS data. That means the domain announcing that key implicitly "takes some responsibility" for the content just verified. So the only variable here is the value, or reputation, of the verified name. All of this is orthogonal to DMARC though, which doesn't care about reputation. It only cares about alignment, and specifically alignment that is under complete control of the sender. -MSK On Sat, Sep 5, 2020 at 10:55 AM Douglas E. Foster wrote: What I am trying to accomplish is different than what can be accomplished with the canned-modifications indicator.To explain, I need to digress into my theory of operation for spam filters: 1) Source identification allows assignment of a Source reputation. Source reputation is more important than content filtering, because hostile content always comes from a source that should not be trusted. Content filtering always produces false positives and false negatives, and the workarounds to those errors are always dependent on source identification 2) Impersonation is always attractive to an attacker because it allows him to exploit the reputation of the impersonated identity. Therefore impersonation is an inherently untrusted activity. 3) Spam filtering will assign sources to three reputation groups: negative reputation (rejected), neutral reputation (acceptance depends on content filtering), and positive reputation (some or all content filtering bypassed.) SPF and DMARC are designed to block impersonation, and mailing lists look like impersonated traffic, so the message moves from neutral to negative reputation. How to overcome that? One solution is to use out-of-band information to justify overlooking the negative clues, then implementing local policy based on that informatoin, so that traffic is moved from negative reputation to positive reputation. ARC and canned-modification tagging are approaches to providing in-message data intended to support application of that local policy.But we have found no way to ensure the out-of-band information flow needed to justify the local policy, for all of the mediators that need that status. We have also identified no method for the recipient to notify the mediator that the local policy is established, although this could also be handed out-of-band. However, these techniques have the benefit of depending on the mediator and the recipient, and not on the sender. A second solution depends on explicit sender authorization to eliminate the apparent imp
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
In article , Murray S. Kucherawy wrote: >Another way to look at this: DKIM (and I believe SPF) only really tells you >something interesting when it passes. That means (for DKIM) the content >was unmodified, and the signature is validated by a key that is verifiably >present in some domain's DNS data. ... Yes, exactly. With SPF, no sensible person rejects mail on SPF -ALL other than the special case of plain -ALL that means it sends no mail whatsoever. Other than that, neither DKIM nor SPF can distinguish between "this is fake" and "this was sent by a route that SPF/DKIM can't describe." -- Regards, John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
uthor, and an automated > spam filter has no ability to identify changes of intent. So is there a > group of mediators who only require the ability to add a subject prefix, > subject suffix, body header, or body footer? > > - The better spam filters offer URL rewrite, which alters original > content.The weaker spam filters may only use these four features, but > they are the ones that are least likely to add an exotic new feature like > dual authorship detection. So I reluctantly conclude that there is no > significant opportunity for using this approach on the "spam filter with > auto-forward" problem. > > - But is there is a group of mailing lists that only need these four > capabilities? I was hoping so. > > DF > > -- > *From*: Alessandro Vesely > *Sent*: 9/5/20 5:36 AM > *To*: dmarc@ietf.org > *Subject*: Re: [dmarc-ietf] AutoForward problems - Change log benefits to > mailing lists > > On Fri 04/Sep/2020 04:05:24 +0200 Douglas E. Foster wrote: > > > > Of the three types of content changes that I proposed, the easiest to > specify > > and get implemented is the first type, where the mediator only adds > content, > > adds a change log indicating the additions, and signs the result. I am > hoping > > and assuming that if mailing lists have freedom to add their branding to > the > > subject and body, most lists would not need to make more complex changes. > > > The change log must not be a generic patch, but rather a stylized list of > pre-canned modifications, much like envisaged in the dkim-transform draft. > This limitation can reduce the attack surface, although it cannot prevent > malicious URLs in the footer. > > > > The signed change log would allow participating recipients to identify > the > > signed additions added by the list or other mediator, while also > identifying > > the signed original after the list additions are virtually removed. > > > I don't think the change log has to be signed. If undoing the changes > leads to > a verifiable signature, then add a dkim=pass for the original signer. Else > dkim=fail. Signing the change log doesn't hurt, but it doesn't help either. > > If verification succeeds, Authentication-Results: can report enough > transformation details to allow the MDA to restore the original From:, in > case > the MLM rewrote it. > > > > Once the additions and the original are reliably identified to a source > > domain, suspicion of spoofing is no longer a concern. Each chunk of > content > > can be evaluated based on the reputation of the verified source domain > and > > the specifics of the content. > > > Nested additions are possible.Each new signature adds an entry to a > > verification stack. Any change can be removed, virtually or actually, > by > > reversing the change at each level, working backwards from last to first. > > > I beg to disagree. On the one hand, we already have ARC to unwind a chain > of > message handlers. The "defect" of ARC is that it needs a full domain > reputation system in order to work reliably. Where the reputation of > "intermediate" mediators is needed, ARC is the right tool. > > On the other hand, a deterministic tool should only be interested in who > is the > actual author of a message and what has the domain owner to say about > attributing such authorship. This can be done without assessing reputation. > > IMHO, the original author domain deserves an aggregate report mentioning > the > result of evaluating DKIM transformations, even if From: was rewritten. So > does the last From: rewriter. Intermediate mediators don't. > > > Best > Ale > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
What I am trying to accomplish is different than what can be accomplished with the canned-modifications indicator.To explain, I need to digress into my theory of operation for spam filters: 1) Source identification allows assignment of a Source reputation. Source reputation is more important than content filtering, because hostile content always comes from a source that should not be trusted. Content filtering always produces false positives and false negatives, and the workarounds to those errors are always dependent on source identification 2) Impersonation is always attractive to an attacker because it allows him to exploit the reputation of the impersonated identity. Therefore impersonation is an inherently untrusted activity. 3) Spam filtering will assign sources to three reputation groups: negative reputation (rejected), neutral reputation (acceptance depends on content filtering), and positive reputation (some or all content filtering bypassed.) SPF and DMARC are designed to block impersonation, and mailing lists look like impersonated traffic, so the message moves from neutral to negative reputation. How to overcome that? One solution is to use out-of-band information to justify overlooking the negative clues, then implementing local policy based on that informatoin, so that traffic is moved from negative reputation to positive reputation. ARC and canned-modification tagging are approaches to providing in-message data intended to support application of that local policy.But we have found no way to ensure the out-of-band information flow needed to justify the local policy, for all of the mediators that need that status. We have also identified no method for the recipient to notify the mediator that the local policy is established, although this could also be handed out-of-band. However, these techniques have the benefit of depending on the mediator and the recipient, and not on the sender. A second solution depends on explicit sender authorization to eliminate the apparent impersonation. This category encompasses conditional signatures, ATSP, RHSWL, DKIM delegation, and SPF inclusion. These approaches require the involvement of sender, list, and recipient. We have concluded that these approaches are victims of unlikely participation by senders, and further limited because the list does not know if a recipient will recognize the sender's authorization.Finally, I observed that this solution cannot help with the problem created by spam filter tagging prior to an auto-forward, so it cannot solve the whole problem. A third solution is to abandon DMARC and allow impersonation to be unrestricted. I am suggesting a fourth approach. This one seeks to address the impersonation problem by clearly identifying each part of the message to its source, so that impersonation is not an issue, and each source's contribution is evaluated based on that source's reputation and that source's content. The goal is to move the imputed source reputation from negative to neutral, rather than from negative to positive. If the source reputation can be defaulted to neutral, the approach can be used by any mediator without prior registration with recipients and without any prior authorization by senders. But on continued reflection, I realize that this approach requires complete isolation between the content added by a mediator and the content provided by the originator. Any other changes to the original content could maliciously alter the original intent of the author, and an automated spam filter has no ability to identify changes of intent. So is there a group of mediators who only require the ability to add a subject prefix, subject suffix, body header, or body footer? - The better spam filters offer URL rewrite, which alters original content. The weaker spam filters may only use these four features, but they are the ones that are least likely to add an exotic new feature like dual authorship detection. So I reluctantly conclude that there is no significant opportunity for using this approach on the "spam filter with auto-forward" problem. - But is there is a group of mailing lists that only need these four capabilities? I was hoping so. DF From: Alessandro Vesely Sent: 9/5/20 5:36 AM To: dmarc@ietf.org Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists On Fri 04/Sep/2020 04:05:24 +0200 Douglas E. Foster wrote: > > Of the three types of content changes that I proposed, the easiest to specify > and get implemented is the first type, where the mediator only adds content, > adds a change log indicating the additions, and signs the result. I am > hoping > and assuming that if mailing lists have freedom to add their branding to the > subject and body, most lists would not need to make more comple
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
On Fri 04/Sep/2020 04:05:24 +0200 Douglas E. Foster wrote: Of the three types of content changes that I proposed, the easiest to specify and get implemented is the first type, where the mediator only adds content, adds a change log indicating the additions, and signs the result. I am hoping and assuming that if mailing lists have freedom to add their branding to the subject and body, most lists would not need to make more complex changes. The change log must not be a generic patch, but rather a stylized list of pre-canned modifications, much like envisaged in the dkim-transform draft. This limitation can reduce the attack surface, although it cannot prevent malicious URLs in the footer. The signed change log would allow participating recipients to identify the signed additions added by the list or other mediator, while also identifying the signed original after the list additions are virtually removed. I don't think the change log has to be signed. If undoing the changes leads to a verifiable signature, then add a dkim=pass for the original signer. Else dkim=fail. Signing the change log doesn't hurt, but it doesn't help either. If verification succeeds, Authentication-Results: can report enough transformation details to allow the MDA to restore the original From:, in case the MLM rewrote it. Once the additions and the original are reliably identified to a source domain, suspicion of spoofing is no longer a concern. Each chunk of content can be evaluated based on the reputation of the verified source domain and the specifics of the content. > Nested additions are possible. Each new signature adds an entry to a verification stack. Any change can be removed, virtually or actually, by reversing the change at each level, working backwards from last to first. I beg to disagree. On the one hand, we already have ARC to unwind a chain of message handlers. The "defect" of ARC is that it needs a full domain reputation system in order to work reliably. Where the reputation of "intermediate" mediators is needed, ARC is the right tool. On the other hand, a deterministic tool should only be interested in who is the actual author of a message and what has the domain owner to say about attributing such authorship. This can be done without assessing reputation. IMHO, the original author domain deserves an aggregate report mentioning the result of evaluating DKIM transformations, even if From: was rewritten. So does the last From: rewriter. Intermediate mediators don't. Best Ale -- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
Although I re-raised the change log issue as part of the auto-forward problem, I am hoping that it will have significant benefit to the mailing list community also. Of the three types of content changes that I proposed, the easiest to specify and get implemented is the first type, where the mediator only adds content, adds a change log indicating the additions, and signs the result. I am hoping and assuming that if mailing lists have freedom to add their branding to the subject and body, most lists would not need to make more complex changes. The signed change log would allow participating recipients to identify the signed additions added by the list or other mediator, while also identifying the signed original after the list additions are virtually removed. Once the additions and the original are reliably identified to a source domain, suspicion of spoofing is no longer a concern. Each chunk of content can be evaluated based on the reputation of the verified source domain and the specifics of the content. Nested additions are possible.Each new signature adds an entry to a verification stack. Any change can be removed, virtually or actually, by reversing the change at each level, working backwards from last to first. In practice, the stack is expected to be short. To prevent malicious misuse, the specification should put a small limit on the number of layers in the stack. Just as importantly, I believe it solves the problem of letting the list know whether From rewriting is necessary or not. I argue that there is no information-leakage risk associated with a domain publishing a DNS entry that says "This domain understands multi-author documents of type 1".This type of flag says nothing about what the domain will do with any particular message with multiple authorship. The recipient domain will have the option of passing the entire message, stripping the wrapper and passing the original, or blocking the whole thing. But it will not be blocking a message because it does not understand the message's actual identity, which is the cause of our current problem. Once a list knows that its identity will not be misjudged, it gains the freedom to assume that its content will be judge fairly, so "From" rewriting should not be necessary when sending to domains that have published this flag. Because this approach adds confidence rather than risk, I am hopeful that even AOL would be willing to implement support for it. The more complex changes, edits and deletions, require a more complex specification and more complex code. That level of specification will be needed to provide a solution to spam filters that do URL rewriting. But that complexity can wait for another day while we try to get a level 1 change log accepted. John, I am hoping that you find some opportunity here. Do you? Doug Foster From: "John Levine" Sent: 9/3/20 6:44 PM To: dmarc@ietf.org Subject: Re: [dmarc-ietf] AutoForward problems In article <767e2dcc-e87c-1e90-2f86-486e51a3c...@wisc.edu>, Jesse Thompson wrote: >I realize that John's message in the other thread probably wasn't referencing >auto-forwarding, but I think his point >dovetails to the auto-forwarding issue: > >> As always, as I hope we all remember DMARC alignment doesn't mean not spam, >> and you still do all of the stuff you do to sort your mail. ... As you say, it's the same problem, it's what people see as the same message but with changes that fail with current authentication schemes. >a) It assumes that the domain owner has the ability and desire to authorize >every potential forwarder, even though >auto-forwarding is a user-level decision. It's not the domain owner, it's more likely the MTA deciding what signatures to apply. >c) Selectively allowing forwarding at the user level is difficult because >users are gonna do what they wanna do (you >try telling faculty they can't forward). It's not the case that all >enterprises want to prohibit all of their users >from auto-forwarding (even though that's the answer you may get if you survey >the CISO community) Yeah. The likely damage from allowing wisc.edu to forward seems pretty low, the damage from outlook.com or gmail.com considerably more. R's, John -- Regards, John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems
In article <767e2dcc-e87c-1e90-2f86-486e51a3c...@wisc.edu>, Jesse Thompson wrote: >I realize that John's message in the other thread probably wasn't referencing >auto-forwarding, but I think his point >dovetails to the auto-forwarding issue: > >> As always, as I hope we all remember DMARC alignment doesn't mean not spam, >> and you still do all of the stuff you do to sort your mail. ... As you say, it's the same problem, it's what people see as the same message but with changes that fail with current authentication schemes. >a) It assumes that the domain owner has the ability and desire to authorize >every potential forwarder, even though >auto-forwarding is a user-level decision. It's not the domain owner, it's more likely the MTA deciding what signatures to apply. >c) Selectively allowing forwarding at the user level is difficult because >users are gonna do what they wanna do (you >try telling faculty they can't forward). It's not the case that all >enterprises want to prohibit all of their users >from auto-forwarding (even though that's the answer you may get if you survey >the CISO community) Yeah. The likely damage from allowing wisc.edu to forward seems pretty low, the damage from outlook.com or gmail.com considerably more. R's, John -- Regards, John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems
stem communicate to the user that some new policy is in effect short of sending an email or stopping the mail flow and hoping they notice and call for support? With the fetch model, at least there are more opportunities to interact with the user where they are (i.e. during the authorization flow during re-authentication after a session timeout). Of course, all of that depends on the fetching side's UI for handling re-authentication, so it's not a slam dunk alternative to SMTP forwarding. Jesse > > Doug Foster > > > -Original Message- > From: dmarc [mailto:dmarc-boun...@ietf.org] On Behalf Of Jesse Thompson > Sent: Thursday, September 03, 2020 8:42 AM > To: dmarc@ietf.org > Subject: Re: [dmarc-ietf] AutoForward problems > > On 9/2/20 6:33 AM, Douglas E. Foster wrote: >> For mailing lists, we have pushed the limits of authorization. But there > is another class of problems where sender authorization is not feasible: > mail which is auto-forwarded after a spam-filter has made content-altering > changes. > > Yes, this is an increasing problem, as exemplified by the number of > enterprises that have started tagging subjects and bodies with [External] > warnings. Coupled with DMARC adoption, the ability for users to > auto-forward successfully is shrinking. > > I realize that John's message in the other thread probably wasn't > referencing auto-forwarding, but I think his point dovetails to the > auto-forwarding issue: > >> As always, as I hope we all remember DMARC alignment doesn't mean not >> spam, and you still do all of the stuff you do to sort your mail. >> This scheme depends on the forwarders you authorize being >> well-behaved. That's why I am concerned that senders need to be >> selective about who they allow to forward. > > I am concerned with: > > a) It assumes that the domain owner has the ability and desire to authorize > every potential forwarder, even though auto-forwarding is a user-level > decision. > > b) It assumes that all potential forwarders check for authorization before > forwarding -- essentially the same problem with the way most ESPs will use a > domain in the From header without checking for authorization via DMARC. > > c) Selectively allowing forwarding at the user level is difficult because > users are gonna do what they wanna do (you try telling faculty they can't > forward). It's not the case that all enterprises want to prohibit all of > their users from auto-forwarding (even though that's the answer you may get > if you survey the CISO community) > > d) Selectively forwarding based on the knowledge of whether the forwarded > message will reach the destination is challenging because the intermediary > doesn't really know if it will succeed, and selectively forwarding in this > way will lead to the user only seeing a subset of their messages being > forwarded. This is essentially what's happening today (see my first > sentence in this message), and I know users who are still bound and > determined to auto-forward and they have just resigned themselves to the > fact that they have to periodically check the intermediary mailbox for all > of the messages that didn't survive the forward. > > >> In the absence of such techniques, the forwarding system (or the outbound > gateway) needs to be aware that a message has been forwarded after > content-altering changes. When this is the case, the message either > requires a known exception at the receiving system or a From address rewrite > so the message will pass DMARC. At minimum, our DMARC specification should > make this clear. > > I agree with this, however, I'm increasingly of the mindset that since this > is a user-level issue, we should be thinking about solving it via OAuth > mechanisms. Modern email services do support the ability to fetch from a > mailbox, authorized via OAuth, so it might be easier to convince the > downstream mailbox providers to enhance their fetching mechanisms to support > the features that people expect of inbound mail (i.e. apply inbound > filtering rules). The mailbox provider would still need a way to determine > the original DMARC results, spam analysis, etc, but wouldn't that be easier > to implement since the message is "frozen in time" and wasn't subjected to > the altering changes implied by forwarding? > > Jesse > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > > > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems
OAUTH vs. More careful forwarding Pursuing both techniques makes the most sense.Some users may be unwilling (or not allowed) to store credentials in the target server, while being unwilling to do a manual operation to obtain their new messages. Since OAUTH is a pull operation instead of a push, it provides several benefits: - Certainty that the forwarding target actually wants the message. - Near-certainty that the message will be accepted by the client system, since the pull bypasses the SMTP filtering process. This includes both sender authentication issues like DMARC as well as other content filtering applied by the forwarding target's domain. You point out the organizational politics will affect the process: - For consumer-focused services like Gmail and Hotmail, the user owns the data and has primary control over its disposition. - For centralized organizations (most large companies), the organization owns the data and can control its disposition. - For decentralized organizations like your university, control is effectively shared and negotiated. Forwarding Control Mechanism -- With any of the above political structures, I argue that the user should not have sole control over the forwarding process. Assume that UserA@DomainA wants to forward to UserB@DomainB: DomainA interests include: - If DomainB objects to the messages for any reason, it may blacklist DomainA and cause disruption to traffic other than this one user's messages. - The forwarding action may violate company confidentiality or organizational obligations to protect privacy of others (GDPR, PCI DSS, HIPPA, etc.) - DnmainA may want to assess whether the forwarding request is in good faith, and that it is not a mule account for exfiltration of data. DomainB and UserB@DomainB interests include: - Does UserB actually want these messages? - What if UserB is the victim of a typo? - What if the auto-forward is being implemented as an attack vector on UserB? - What if the auto-forward is going to a non-existent account because of a typo? DomainA would benefit from registering the forwarding configuration with DomainB: - So that if DomainB has more aggressive spam filtering, the blocked traffic will be blamed on the source and not on DomainA. - So that DMARC verification failures can be overlooked. - More generally, so that DomainB does not perceive UserA@DomainA as an open relay or other threat vector. - If the account is overquota or closed, DomainA has an interest in receiving notification of this event. Consequently, I think the protocol should be: - UserA@DomainA requests forwarding from his domain owner. - DomainA requires a confirming email from UserB@DomainB, which should be a trivial effort for UserA@DomainA to make happen. - DomainA decides how much to engage the domain owner of DomainB for purposes of registration and whitelisting. - Assuming that no veto is received, DomainA activates the forward. Outbound gateways can / should be used to enforce a domain's controls on the forwarding process. This is very different from what I have seen implemented in mail store products. Many systems allow any user to forward anywhere. Some systems allow the domain administrator to disable all forwarding from any account. I do not think I have seen admin-controlled selective forwarding, but others have more product experience than I do. Doug Foster -Original Message- From: dmarc [mailto:dmarc-boun...@ietf.org] On Behalf Of Jesse Thompson Sent: Thursday, September 03, 2020 8:42 AM To: dmarc@ietf.org Subject: Re: [dmarc-ietf] AutoForward problems On 9/2/20 6:33 AM, Douglas E. Foster wrote: > For mailing lists, we have pushed the limits of authorization. But there is another class of problems where sender authorization is not feasible: mail which is auto-forwarded after a spam-filter has made content-altering changes. Yes, this is an increasing problem, as exemplified by the number of enterprises that have started tagging subjects and bodies with [External] warnings. Coupled with DMARC adoption, the ability for users to auto-forward successfully is shrinking. I realize that John's message in the other thread probably wasn't referencing auto-forwarding, but I think his point dovetails to the auto-forwarding issue: > As always, as I hope we all remember DMARC alignment doesn't mean not > spam, and you still do all of the stuff you do to sort your mail. > This scheme depends on the forwarders you authorize being > well-behaved. That's why I am concerned that senders need to be > selective about who they allow to forward. I am concerned with: a) It assumes that the domain owner has the ability and desire to authorize every potential forwarder, even though auto-forwarding is a user-level decision. b) It assumes that all potential forwarders check for authorization before forwarding -- essentiall
Re: [dmarc-ietf] AutoForward problems
On 9/2/20 6:33 AM, Douglas E. Foster wrote: > For mailing lists, we have pushed the limits of authorization. But there is > another class of problems where sender authorization is not feasible: mail > which is auto-forwarded after a spam-filter has made content-altering changes. Yes, this is an increasing problem, as exemplified by the number of enterprises that have started tagging subjects and bodies with [External] warnings. Coupled with DMARC adoption, the ability for users to auto-forward successfully is shrinking. I realize that John's message in the other thread probably wasn't referencing auto-forwarding, but I think his point dovetails to the auto-forwarding issue: > As always, as I hope we all remember DMARC alignment doesn't mean not spam, > and you still do all of the stuff you do to sort your mail. This scheme > depends on the forwarders you authorize being well-behaved. That's why I > am concerned that senders need to be selective about who they allow to > forward. I am concerned with: a) It assumes that the domain owner has the ability and desire to authorize every potential forwarder, even though auto-forwarding is a user-level decision. b) It assumes that all potential forwarders check for authorization before forwarding -- essentially the same problem with the way most ESPs will use a domain in the From header without checking for authorization via DMARC. c) Selectively allowing forwarding at the user level is difficult because users are gonna do what they wanna do (you try telling faculty they can't forward). It's not the case that all enterprises want to prohibit all of their users from auto-forwarding (even though that's the answer you may get if you survey the CISO community) d) Selectively forwarding based on the knowledge of whether the forwarded message will reach the destination is challenging because the intermediary doesn't really know if it will succeed, and selectively forwarding in this way will lead to the user only seeing a subset of their messages being forwarded. This is essentially what's happening today (see my first sentence in this message), and I know users who are still bound and determined to auto-forward and they have just resigned themselves to the fact that they have to periodically check the intermediary mailbox for all of the messages that didn't survive the forward. > In the absence of such techniques, the forwarding system (or the outbound > gateway) needs to be aware that a message has been forwarded after > content-altering changes. When this is the case, the message either > requires a known exception at the receiving system or a From address rewrite > so the message will pass DMARC. At minimum, our DMARC specification should > make this clear. I agree with this, however, I'm increasingly of the mindset that since this is a user-level issue, we should be thinking about solving it via OAuth mechanisms. Modern email services do support the ability to fetch from a mailbox, authorized via OAuth, so it might be easier to convince the downstream mailbox providers to enhance their fetching mechanisms to support the features that people expect of inbound mail (i.e. apply inbound filtering rules). The mailbox provider would still need a way to determine the original DMARC results, spam analysis, etc, but wouldn't that be easier to implement since the message is "frozen in time" and wasn't subjected to the altering changes implied by forwarding? Jesse ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] AutoForward problems
For mailing lists, we have pushed the limits of authorization. But there is another class of problems where sender authorization is not feasible: mail which is auto-forwarded after a spam-filter has made content-altering changes. In some respects, the problem is similar to a mailing list. This mailing list adds a prefix to the subject, adds a footer to the body, and converts content to text. The first two changes are potentially reversible, while the last one is not. Commonly used spam filters may add a prefix to the subject, a header or footer to the body, and rewrite embedded URLs to provide time-of-click protection.Again, the first two changes are potentially reversible, while the last is more difficult. Because an auto-forward has the potential for senders from any source, third-party authorization of the forwarding domain is not possible. Instead the mechanism requires either that the recipient domain provides a DMARC exception for the forwarding stream, or the forwarding domain must remove the spam-filter's changes so that signature integrity is restored. In general, the changes provided by a spam filter are intended for the subscribed client, so it is an interesting licensing question whether those protections should extend to external non-subscriber addresses that are served via auto-forwarding. But for our purposes, removing those edits would solve the DMARC problem by restoring signature integrity. So I think we need to re-examine rollback options I think it is useful to categorize the types of edits that may occur: - A simple insertion of new text. A change log which documents the start point and length of insertion would be sufficient to identify the added text separately from the prior text. If the document is signed after the changes are applied, an MUA could potentially separate the contents of each author, with author identification. This may actually enhance efforts to use spam-filter additions as a trust indicator. An MTA could evaluate the original content against the original signature, or could remove the added content to restore the original message. - A replacement or removal of text, where the original content can be represented in a message header.In this case, the change log would include the starting point of the change, the length of the new text, and the entirety of the original text.An MTA could reverse the changes or evaluate the original signature after applying a virtual reversal of the changes. An MUA would probably not be able to identify the changed regions to the user, especially if the changes were in the non-visible portion of an HTML component. - A replacement or removal of text, where the original content is too complex to be stored in the message header. In this case, the original content can only be recovered if the original content is retained external to the message in a proprietary data store.Since many organizations use the same vendor for both inbound and outbound gateways, external data stores are plausible. In the absence of such techniques, the forwarding system (or the outbound gateway) needs to be aware that a message has been forwarded after content-altering changes. When this is the case, the message either requires a known exception at the receiving system or a From address rewrite so the message will pass DMARC. At minimum, our DMARC specification should make this clear. Doug Foster ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
