Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
I entirely agree that a confirmed identity is useful because it allows us to whiteliist safely. I became alienated from my commercial products because they did not understand that whitelisting should require confirmed identity. I think they should understand security principles if they are in that business.My commercial products also choke when the blacklists grow large. Once I found a customizable filter, I moved my rules into database tables. I use segment matching, instead of ends-with matching, to ensure indexed queries. It no longer matters if a blacklist has 500 entries or 15,000.IP blocking is the best way to prevent domain churning, but domain blocking helps prevrnt IP churning. I try to blacklist both at the same time.Most of my spam arrives either as first-party mail (SPF Pass with domain alignment) or as a client of an ESP, mostly Sendgrid, (with SPF Pass and no DMARC policy.) Commercial products that ignore Message From are unable to handle ESP-based spamming. ESP-based spamming also evades some URL filtering, because all of the URLs point back to the ESP.My vendors say that malicious Office365 accounts are the new attack vector, but I have not seen it yet. When it happens, my filter rules change to full address blacklists instead of domain blacklists.DFSent from my Verizon, Samsung Galaxy smartphone Original message From: "Murray S. Kucherawy" Date: 9/8/20 4:31 PM (GMT-05:00) To: Doug Foster Cc: IETF DMARC WG Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists On Tue, Sep 8, 2020 at 5:09 AM Doug Foster wrote: > However, I disagree about negative reputation.Content filtering alone > is insufficient and even more error prone. In the last year, I have had > spam campaigns about LED lighting, stand-up desks, touchless thermometers, > and knife sharpeners. I cannot anticipate all the ways that spammers will > hide their dirty deeds. But I know it is spam, not merely unwanted > advertising, because of receiving many similar messages from many different > domains using many different servers. Third-party RBLs help but are > insufficient. I am gradually building my own reputation database based on > the traffic that I am receiving. By blocking the problem sources, I have > been able to get the spam problem under something approaching good control. > Content filtering is a useful tool for day-zero detection of a new spam > source. Once a source is detected, it needs to be blocked. > > > > Whether a message passes SPF and DMARC criteria is part of my search > critieria for unwanted traffic, but definitely not the only one. As has > been observed, actual spoofing of the From address is not a huge part of my > problem at present. This is largely because spammers have easy enough > tools in Friendly Name spoofing and corporate logo misuse. But I also > attribute that low volume to the existence of SPF and DMARC. > Suppose I'm one of your touchless thermometer spammers. Your system identifies me and the DKIM signing domain I'm using. I notice, through well-established means, that my spam is no longer getting through to you. So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or whatever a few smashes of the keyboard yields, and start signing with that instead of whatever domain I was using before. For a couple of bucks, I have now escaped my negative reputation in your system. Maybe I bounce it through a botnet too, so that you can't catch me with an IP reputation either. Negative reputations are trivially shed. It follows that it's not terribly useful to track them, at least not long-term. You end up with records of spammy domains that you'll notice only sent mail for the shortest of time ranges, long enough to get in undetected or under the guise of "too new to block", and then abandoned when they stop working. Blocking domains you've never heard of before is often disruptive when, say, you join a loyalty program for some vendor you've never dealt with before and actually do want their mail, so you're between a rock and a hard place. Instead, positive reputations are the things on which you can reliably act, giving such messages preferential treatment. It's generally a much higher bar, plus the namespace of domains that manage to earn positive reputations is small, and they tend to be well-behaved over longer periods of time. Content filtering is a different matter. It's focused on what's in the message, irrespective of where it came from. But that's a whole new game to play, and definitely not anything in which DMARC is interested. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
On Tue, Sep 8, 2020 at 1:30 PM Murray S. Kucherawy wrote: > On Tue, Sep 8, 2020 at 5:09 AM Doug Foster 40bayviewphysicians@dmarc.ietf.org> wrote: > >> However, I disagree about negative reputation.Content filtering alone >> is insufficient and even more error prone. In the last year, I have had >> spam campaigns about LED lighting, stand-up desks, touchless thermometers, >> and knife sharpeners. I cannot anticipate all the ways that spammers will >> hide their dirty deeds. But I know it is spam, not merely unwanted >> advertising, because of receiving many similar messages from many different >> domains using many different servers. Third-party RBLs help but are >> insufficient. I am gradually building my own reputation database based on >> the traffic that I am receiving. By blocking the problem sources, I have >> been able to get the spam problem under something approaching good control. >> Content filtering is a useful tool for day-zero detection of a new spam >> source. Once a source is detected, it needs to be blocked. >> >> >> >> Whether a message passes SPF and DMARC criteria is part of my search >> critieria for unwanted traffic, but definitely not the only one. As has >> been observed, actual spoofing of the From address is not a huge part of my >> problem at present. This is largely because spammers have easy enough >> tools in Friendly Name spoofing and corporate logo misuse. But I also >> attribute that low volume to the existence of SPF and DMARC. >> > > Suppose I'm one of your touchless thermometer spammers. Your system > identifies me and the DKIM signing domain I'm using. I notice, through > well-established means, that my spam is no longer getting through to you. > So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or > whatever a few smashes of the keyboard yields, and start signing with that > instead of whatever domain I was using before. For a couple of bucks, I > have now escaped my negative reputation in your system. Maybe I bounce it > through a botnet too, so that you can't catch me with an IP reputation > either. > > Negative reputations are trivially shed. It follows that it's not > terribly useful to track them, at least not long-term. You end up with > records of spammy domains that you'll notice only sent mail for the > shortest of time ranges, long enough to get in undetected or under the > guise of "too new to block", and then abandoned when they stop working. > Blocking domains you've never heard of before is often disruptive when, > say, you join a loyalty program for some vendor you've never dealt with > before and actually do want their mail, so you're between a rock and a hard > place. > > Instead, positive reputations are the things on which you can reliably > act, giving such messages preferential treatment. It's generally a much > higher bar, plus the namespace of domains that manage to earn positive > reputations is small, and they tend to be well-behaved over longer periods > of time. > I disagree, we track reputations both good and bad, and they are incorporated in spam rules across the ladder. A surprising number of negative reputations are not shed, even at very-low... and sure, we do sunset reputations that go unused. At the very least, there is a time lag before the spammer notices the effect and switches. I mean, a blacklist is ultimately a determination that a reputation is so low as to block completely, and that seems to be the main way that anti-spam information is distributed and used by most medium to small providers. That set of botnet IPs definitely will get a low reputation themselves and end up on blacklists. And forcing the spammers to spend money on things like new domain names is part of the benefit. OTOH, we also don't believe in "too new to block", unknown reputation is a great reason to apply throttling at the least. Maybe some of this is large system stuff, where you can expect to see more traffic and things don't tend to be unique... but of course we also get complaints from very small volume folks as well. Brandon ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
On Tue, Sep 8, 2020 at 5:09 AM Doug Foster wrote: > However, I disagree about negative reputation.Content filtering alone > is insufficient and even more error prone. In the last year, I have had > spam campaigns about LED lighting, stand-up desks, touchless thermometers, > and knife sharpeners. I cannot anticipate all the ways that spammers will > hide their dirty deeds. But I know it is spam, not merely unwanted > advertising, because of receiving many similar messages from many different > domains using many different servers. Third-party RBLs help but are > insufficient. I am gradually building my own reputation database based on > the traffic that I am receiving. By blocking the problem sources, I have > been able to get the spam problem under something approaching good control. > Content filtering is a useful tool for day-zero detection of a new spam > source. Once a source is detected, it needs to be blocked. > > > > Whether a message passes SPF and DMARC criteria is part of my search > critieria for unwanted traffic, but definitely not the only one. As has > been observed, actual spoofing of the From address is not a huge part of my > problem at present. This is largely because spammers have easy enough > tools in Friendly Name spoofing and corporate logo misuse. But I also > attribute that low volume to the existence of SPF and DMARC. > Suppose I'm one of your touchless thermometer spammers. Your system identifies me and the DKIM signing domain I'm using. I notice, through well-established means, that my spam is no longer getting through to you. So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or whatever a few smashes of the keyboard yields, and start signing with that instead of whatever domain I was using before. For a couple of bucks, I have now escaped my negative reputation in your system. Maybe I bounce it through a botnet too, so that you can't catch me with an IP reputation either. Negative reputations are trivially shed. It follows that it's not terribly useful to track them, at least not long-term. You end up with records of spammy domains that you'll notice only sent mail for the shortest of time ranges, long enough to get in undetected or under the guise of "too new to block", and then abandoned when they stop working. Blocking domains you've never heard of before is often disruptive when, say, you join a loyalty program for some vendor you've never dealt with before and actually do want their mail, so you're between a rock and a hard place. Instead, positive reputations are the things on which you can reliably act, giving such messages preferential treatment. It's generally a much higher bar, plus the namespace of domains that manage to earn positive reputations is small, and they tend to be well-behaved over longer periods of time. Content filtering is a different matter. It's focused on what's in the message, irrespective of where it came from. But that's a whole new game to play, and definitely not anything in which DMARC is interested. -MSK ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists
SPF/DMARC are not optimal tools for detecting malice.In my experience, the abundance of sender configuration errors are the limiting factor. However, I disagree about negative reputation.Content filtering alone is insufficient and even more error prone. In the last year, I have had spam campaigns about LED lighting, stand-up desks, touchless thermometers, and knife sharpeners. I cannot anticipate all the ways that spammers will hide their dirty deeds. But I know it is spam, not merely unwanted advertising, because of receiving many similar messages from many different domains using many different servers. Third-party RBLs help but are insufficient. I am gradually building my own reputation database based on the traffic that I am receiving. By blocking the problem sources, I have been able to get the spam problem under something approaching good control. Content filtering is a useful tool for day-zero detection of a new spam source. Once a source is detected, it needs to be blocked. Whether a message passes SPF and DMARC criteria is part of my search critieria for unwanted traffic, but definitely not the only one. As has been observed, actual spoofing of the From address is not a huge part of my problem at present. This is largely because spammers have easy enough tools in Friendly Name spoofing and corporate logo misuse. But I also attribute that low volume to the existence of SPF and DMARC. Doug Foster From: Murray S. Kucherawy [mailto:superu...@gmail.com] Sent: Monday, September 07, 2020 4:30 PM To: Doug Foster Cc: IETF DMARC WG Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists Historically, I've found that a negative source reputation is easy to dodge. It's trivial to come from an unknown IP address or register a new domain name, so an actor with a negative reputation can trivially move to a neutral one. Thus, a receiver/verifier seeking to make a meaningful judgement can only really focus on positive reputations when making meaningful filtering decisions; everyone else is basically the same in terms of value. Another way to look at this: DKIM (and I believe SPF) only really tells you something interesting when it passes. That means (for DKIM) the content was unmodified, and the signature is validated by a key that is verifiably present in some domain's DNS data. That means the domain announcing that key implicitly "takes some responsibility" for the content just verified. So the only variable here is the value, or reputation, of the verified name. All of this is orthogonal to DMARC though, which doesn't care about reputation. It only cares about alignment, and specifically alignment that is under complete control of the sender. -MSK On Sat, Sep 5, 2020 at 10:55 AM Douglas E. Foster wrote: What I am trying to accomplish is different than what can be accomplished with the canned-modifications indicator.To explain, I need to digress into my theory of operation for spam filters: 1) Source identification allows assignment of a Source reputation. Source reputation is more important than content filtering, because hostile content always comes from a source that should not be trusted. Content filtering always produces false positives and false negatives, and the workarounds to those errors are always dependent on source identification 2) Impersonation is always attractive to an attacker because it allows him to exploit the reputation of the impersonated identity. Therefore impersonation is an inherently untrusted activity. 3) Spam filtering will assign sources to three reputation groups: negative reputation (rejected), neutral reputation (acceptance depends on content filtering), and positive reputation (some or all content filtering bypassed.) SPF and DMARC are designed to block impersonation, and mailing lists look like impersonated traffic, so the message moves from neutral to negative reputation. How to overcome that? One solution is to use out-of-band information to justify overlooking the negative clues, then implementing local policy based on that informatoin, so that traffic is moved from negative reputation to positive reputation. ARC and canned-modification tagging are approaches to providing in-message data intended to support application of that local policy.But we have found no way to ensure the out-of-band information flow needed to justify the local policy, for all of the mediators that need that status. We have also identified no method for the recipient to notify the mediator that the local policy is established, although this could also be handed out-of-band. However, these techniques have the benefit of depending on the mediator and the recipient, and not on the sender. A second solution depends on explicit sender authorization to eliminate the apparent impersonation. This
