Re: Penalty for no/bad SPF
I've noticed anecdotally that many times when a legitimate source has no PTR on the mail host, or has a PTR with no matching A record, the domain does have a SPF record authorizing the host. This carelessness is probably born from ignorance or overwork. The PTR pass could be used to balance out the PTR fail. I have not had a chance yet to test this out in real mail flow to see how close it comes to being something good enough to reject mail. Joseph Brennan
Re: Penalty for no/bad SPF
On 01/25/2018 02:19 PM, Reindl Harald wrote: Am 25.01.2018 um 20:45 schrieb David Jones: On 01/25/2018 01:19 PM, David Jones wrote: On 01/25/2018 12:59 PM, Reindl Harald wrote: Am 25.01.2018 um 19:48 schrieb David Jones: Since very few sites can reject on SPF fails because SPF failures are so prevalent on legit email, I don't think this is happening in the real world. says who? check_recipient_access proxy:hash:/etc/postfix/skip_spf_check.cf permit_dnswl_client dnswl-aggregate.thelounge.net=127.0.0.5 permit_dnswl_client wl.mailspike.net=127.0.0.[19;20] permit_dnswl_client list.dnswl.org=127.0.[0..255].[2;3] check_policy_service unix:private/spf-policy You are excluding a ton of clients from SPF checks with that config. How many total IPs are covered in your local dnswl-aggregate.thelounge.net whitelist? My policyd-spf runs from the Postfix master.cf to add headers to all email for SA to examine. If you are excluding hundred's of thousands of IPs in those 4 Postfix config lines, then that's not a legitimate claim to be rejecting SPF fails. that's called a smart setup which prevents false-positives and using SPF in the mid-stage where the spf-polidcyd for SPF_PAS does a "permit" and hands over to the milters to save most of the generic stuff You usually are very good with providing numbers so show us how many rejects are happening from SPF failure out of the total volume of email not much, but that's not because the whitelists but because postscreen filters out the majority long before smptd even get a connection SPF_NONE which maskes it to the milters get a 0.5 penalty current month Connections: 215161 Postscreen WL: 20214 (9.39 %) Delivered: 38684 Blocked: 176477 Invalid User: 858 Disallowed User: 16 Reject Postscreen: 90180 Reject Postfix: 6403 Reject Milter: 2188 Reject Temporary: 497 Greylisted: 1768 Blacklist: 89565 Pregreet: 75250 Hangup: 124674 Protocol Error: 118 Illegal Syntax: 2 SpamAssassin: 2184 Virus (Milter): 4 Virus (SA): 292 Helo: 263 Subject: 96 From: 20 Attachment: 1 Header Length: 5 Sender Regex: 1 Sender Blocked: 163 Sender Verify: 42 Sender Invalid: 61 Sender Spoofed: 46 Sender Parked: 0 Spam-TLD: 82 PTR Missing: 142 PTR Generic: 425 SPF: 155 It sounds like to a good/solid setup but I am still thinking there are a lot of potential SPF failures being excluded by those 4 Postfix lines. How many mailboxes do you filter for? Those numbers seem rather low so maybe a few hundred mailboxes? -- David Jones
Re: Penalty for no/bad SPF
On 01/25/2018 01:19 PM, David Jones wrote: On 01/25/2018 12:59 PM, Reindl Harald wrote: Am 25.01.2018 um 19:48 schrieb David Jones: Since very few sites can reject on SPF fails because SPF failures are so prevalent on legit email, I don't think this is happening in the real world. says who? check_recipient_access proxy:hash:/etc/postfix/skip_spf_check.cf permit_dnswl_client dnswl-aggregate.thelounge.net=127.0.0.5 permit_dnswl_client wl.mailspike.net=127.0.0.[19;20] permit_dnswl_client list.dnswl.org=127.0.[0..255].[2;3] check_policy_service unix:private/spf-policy You are excluding a ton of clients from SPF checks with that config. How many total IPs are covered in your local dnswl-aggregate.thelounge.net whitelist? My policyd-spf runs from the Postfix master.cf to add headers to all email for SA to examine. If you are excluding hundred's of thousands of IPs in those 4 Postfix config lines, then that's not a legitimate claim to be rejecting SPF fails. You usually are very good with providing numbers so show us how many rejects are happening from SPF failure out of the total volume of email. -- David Jones
Re: Penalty for no/bad SPF
On 01/25/2018 12:59 PM, Reindl Harald wrote: Am 25.01.2018 um 19:48 schrieb David Jones: Since very few sites can reject on SPF fails because SPF failures are so prevalent on legit email, I don't think this is happening in the real world. says who? check_recipient_access proxy:hash:/etc/postfix/skip_spf_check.cf permit_dnswl_client dnswl-aggregate.thelounge.net=127.0.0.5 permit_dnswl_client wl.mailspike.net=127.0.0.[19;20] permit_dnswl_client list.dnswl.org=127.0.[0..255].[2;3] check_policy_service unix:private/spf-policy You are excluding a ton of clients from SPF checks with that config. How many total IPs are covered in your local dnswl-aggregate.thelounge.net whitelist? My policyd-spf runs from the Postfix master.cf to add headers to all email for SA to examine. -- David Jones
Re: Penalty for no/bad SPF
On 01/25/2018 12:27 PM, RW wrote: On Thu, 25 Jan 2018 09:53:12 -0600 David Jones wrote: On 01/25/2018 09:34 AM, RW wrote: There is nothing wrong with stopping a soft fail if that is what they want to do. In fact, most people should stop at soft fail unless they really know what they are doing or they are a major brand with a high risk spoofing. There's more to it than that. All of the above use DMARC and if you use -all in combination with DMARC you are allowing the SPF result (which is only one component of DMARC) and SPF's legacy policy mechanism to overide both the DMARC result and the DMARC policy. The DMARC RFC has a warning about this. My understanding based on real world results and the link below says that for DMARC to pass you have to have SPF pass and envelope-from domain alignment _OR_ DKIM pass and header From: domain alignment. If you have both then it's even better. https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/ SPF_PASS can hit with either "~all" or "-all" so it doesn't make a difference to DMARC pass. From RFC 7489 .10.1. Issues Specific to SPF ... Some receiver architectures might implement SPF in advance of any DMARC operations. This means that a "-" prefix on a sender's SPF mechanism, such as "-all", could cause that rejection to go into effect early in handling, causing message rejection before any DMARC processing takes place. Operators choosing to use "-all" should be aware of this. Since very few sites can reject on SPF fails because SPF failures are so prevalent on legit email, I don't think this is happening in the real world. This is my main point that some large force like Google or SA needs to take steps to make SPF failures worth something. This would start with Microsoft stop recommending "-all" to their Office 365 customers who don't know what they are doing. Google correctly recommends "~all" to their customers as a default. Then if you know what you are doing and have run DMARC reporting for several months to get your SPF record complete, then you switch to "-all". -- David Jones
Re: Penalty for no/bad SPF
On Thu, 25 Jan 2018 09:53:12 -0600 David Jones wrote: > On 01/25/2018 09:34 AM, RW wrote: > >> There is nothing wrong with stopping a soft fail if that is what > >> they want to do. In fact, most people should stop at soft fail > >> unless they really know what they are doing or they are a major > >> brand with a high risk spoofing. > > > > There's more to it than that. > > > > All of the above use DMARC and if you use -all in combination with > > DMARC you are allowing the SPF result (which is only one component > > of DMARC) and SPF's legacy policy mechanism to overide both the > > DMARC result and the DMARC policy. The DMARC RFC has a warning > > about this. > > My understanding based on real world results and the link below says > that for DMARC to pass you have to have SPF pass and envelope-from > domain alignment _OR_ DKIM pass and header From: domain alignment. > If you have both then it's even better. > > https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/ > > SPF_PASS can hit with either "~all" or "-all" so it doesn't make a > difference to DMARC pass. From RFC 7489 .10.1. Issues Specific to SPF ... Some receiver architectures might implement SPF in advance of any DMARC operations. This means that a "-" prefix on a sender's SPF mechanism, such as "-all", could cause that rejection to go into effect early in handling, causing message rejection before any DMARC processing takes place. Operators choosing to use "-all" should be aware of this.
Re: Penalty for no/bad SPF
On 01/25/2018 11:43 AM, Pedro David Marco wrote: Do not forget that accounts in valid servers are hacked oftenly... --- PedroD I have stated multiple times, I am not recommending whitelist_* entries for domains with user mailboxes that can be compromised unless there is a very specific reason to. For example, this would not be wise or recommended: whitelist_auth *@nytimes.com but this would be: whitelist_auth *@*.nytimes.com As more an more bulk senders use subdomains with good SPF, DKIM, and DMARC, this allows us to safelist them if they have a good reputation, handle abuse reports, have proper opt-out, etc. -- David Jones
Re: Penalty for no/bad SPF
Do not forget that accounts in valid servers are hacked oftenly... ---PedroD
Re: Penalty for no/bad SPF
On 2018-01-24 21:45, Joseph Brennan wrote: > David Joneswrote: > >> SA could be the large force that helps improve the mail standards like >> DMARC -- SPF + DKIM with a little extra on top. > > DMARC is not a standard according to RFC 7489, "Status of This Memo". > It's just informational, for those who want to play the game. DMARC is > destroying forwarding and mailing lists, and I'm sorry to see the > elephants in the email room implementing it-- though Gmail still does > not always reject based on DMARC reject, as if they use that plus some > internal system to make the call. > DMARC is not destroying anything if forwarding and mailing lists are configured properly (like this one). The whole point of DKIM/DMARC was to authenticate forwarded e-mail, which is broken by design in SPF. If we could make all mailing lists operators fix the DKIM breaking features like title modification and adding footers we could just reject literally everything that fails DKIM. Then the spoofing problem would be fixed once and for all and SPF would be just a fail-safe in case something went wrong. Gmail implementing DMARC is probably the best anti phishing/spoofing decision made in the last few years. I am sure you would agree if you were administering paypal's or banks mail servers. Karol -- Karol Augustin ka...@augustin.pl http://karolaugustin.pl/ +353 85 775 5312
Re: Penalty for no/bad SPF
On 01/25/2018 09:34 AM, RW wrote: On Wed, 24 Jan 2018 16:26:58 -0600 David Jones wrote: On 01/24/2018 04:00 PM, Vincent Fox wrote: However, look at all the major providers with messed up records and neutral or soft fail. They should have the most resources to accomplish this and the most incentives to list all their netblocks and set to hard fail. Google is soft fail. Hotmail is soft fail. And Yahoo has the strongest DMARC policy and the weakest SPF policy. There is nothing wrong with stopping a soft fail if that is what they want to do. In fact, most people should stop at soft fail unless they really know what they are doing or they are a major brand with a high risk spoofing. There's more to it than that. All of the above use DMARC and if you use -all in combination with DMARC you are allowing the SPF result (which is only one component of DMARC) and SPF's legacy policy mechanism to overide both the DMARC result and the DMARC policy. The DMARC RFC has a warning about this. My understanding based on real world results and the link below says that for DMARC to pass you have to have SPF pass and envelope-from domain alignment _OR_ DKIM pass and header From: domain alignment. If you have both then it's even better. https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/ SPF_PASS can hit with either "~all" or "-all" so it doesn't make a difference to DMARC pass. -- David Jones
Re: Penalty for no/bad SPF
On 01/25/2018 04:19 AM, Bill Shirley wrote: I'm all for tightening up standards compliance with email, but what I would see if this would happen is a request from my customers saying: Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist him please Bob's email admin won't ever know that his SPF is failing. Bill I am trying to help with this problem by sending emails nor more than per day like this one back to the sender: https://pastebin.com/DC2EA34s This way the sender knows that they just sent the email and gets this response immediately. If they can read the first sentence and forward it to their mail admins, then there's a small chance this will help. I am sure that some users are setting up an Inbox rule to delete this email. I have had a couple of responses wanting help with fixing their SPF record so there is some good coming from this. This email response is completely automated. I am using swatch to watch for SPF_FAIL in my mail logs then it launches a script that generates that form email filling some details. Swatch is limited to replying only once per every 24 hours per sender address. -- David Jones
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018 16:26:58 -0600 David Jones wrote: > On 01/24/2018 04:00 PM, Vincent Fox wrote: > > However, look at all the major providers with messed up records and > > neutral or soft fail. They should have the most resources to > > accomplish this and the most incentives to list all their > > netblocks and set to hard fail. > > > > > > Google is soft fail. > > Hotmail is soft fail. And Yahoo has the strongest DMARC policy and the weakest SPF policy. > There is nothing wrong with stopping a soft fail if that is what they > want to do. In fact, most people should stop at soft fail unless > they really know what they are doing or they are a major brand with a > high risk spoofing. There's more to it than that. All of the above use DMARC and if you use -all in combination with DMARC you are allowing the SPF result (which is only one component of DMARC) and SPF's legacy policy mechanism to overide both the DMARC result and the DMARC policy. The DMARC RFC has a warning about this.
Re: Penalty for no/bad SPF
On Thu, 25 Jan 2018 05:19:38 -0500 Bill Shirleywrote: > I'm all for tightening up standards compliance with email, but what I > would see if this would happen is a request from my customers saying: > Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist > him please Bob's email admin won't ever know that his SPF is failing. This matches our experience. We perform a very mild form of SPF enforcement: If a message fails or softfails SPF, then we ignore any sender or domain whitelist. One of our top support questions is how to allow the whitelist anyway. :( Regards, Dianne.
Re: Penalty for no/bad SPF
On 01/25/2018 04:49 AM, Giovanni Bechis wrote: On 01/25/18 11:19, Bill Shirley wrote: I'm all for tightening up standards compliance with email, but what I would see if this would happen is a request from my customers saying: Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist him please Bob's email admin won't ever know that his SPF is failing. IMHO when the majority (including some big providers) will put Bob's email into spam folders, his admin will take a look at his spf records; nobody knows if it will work or not but I think it is worth trying. Nobody cared about https till last year, now even very simple web sites uses it only because web browsers are giving warnings, not because they do care about security. Giovanni Exactly. If we partnered with Google (maybe Google Summer of Code) to add DMARC and ARC support to SA. Then we announce that we want to move these email standards forward over the next two years. This would help in the fight against spoofing and improve our options to whitelist good senders to better target the spammers. -- David Jones
Re: Penalty for no/bad SPF
On 01/25/2018 02:13 AM, Matus UHLAR - fantomas wrote: On 01/24/2018 03:45 PM, Joseph Brennan wrote: The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. On 24.01.18 16:04, David Jones wrote: The key point here is the bulk nytimes.com email that is system generated, i.e. not humans with real mailboxes that could be compromised, is from subdomains so this entry would be safe since they do have good SPF records on subdomains: whitelist_auth *@*.nytimes.com this only applies when SPF succeeds so it won't fix their broken SPF :-) But this encourages them to make sure their SPF is not broken if we do this in the main SA ruleset for everyone running sa-update regularly. -- David Jones
Re: Penalty for no/bad SPF
On 01/25/18 11:19, Bill Shirley wrote: > I'm all for tightening up standards compliance with email, but what I would > see > if this would happen is a request from my customers saying: > Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist him > please > Bob's email admin won't ever know that his SPF is failing. > IMHO when the majority (including some big providers) will put Bob's email into spam folders, his admin will take a look at his spf records; nobody knows if it will work or not but I think it is worth trying. Nobody cared about https till last year, now even very simple web sites uses it only because web browsers are giving warnings, not because they do care about security. Giovanni > Bill > > On 1/24/2018 2:59 PM, David Jones wrote: >> On 01/24/2018 01:33 PM, Bill Cole wrote: >>> On 24 Jan 2018, at 9:12, David Jones wrote: >>> What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? >>> >>> -1 >>> >>> In every real mailstream I've worked with in the lifetime of SPF, lack of >>> SPF has *always* had a correlation with ham, not spam. >>> >> >> I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. >> >>> >>> SPF hard failures are a more complicated case because the sort of spam that >>> hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a >>> few minutes, making it hard for a site using DNSBLs to know how much of it >>> there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam >>> where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. >>> Most SPF_FAIL spam scores well into double digits here. >> >> I am proposing that if SPF were more accurately deployed then SPF_FAIL would >> be worth something. We could whitelist_auth more trusted senders and then >> be able to turn up the scores for the rest of the mail flow. >> >> If the huge SA community around the world were to help push SPF adoption and >> accurate deployments, then we could move on to DKIM too. Right now, the >> best option we have is to get DMARC properly deployed as much as possible >> where p=reject actually rejects the message unlike SPF_FAIL that we can't >> trust. >> >
Re: Penalty for no/bad SPF
I'm all for tightening up standards compliance with email, but what I would see if this would happen is a request from my customers saying: Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist him please Bob's email admin won't ever know that his SPF is failing. Bill On 1/24/2018 2:59 PM, David Jones wrote: On 01/24/2018 01:33 PM, Bill Cole wrote: On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here. I am proposing that if SPF were more accurately deployed then SPF_FAIL would be worth something. We could whitelist_auth more trusted senders and then be able to turn up the scores for the rest of the mail flow. If the huge SA community around the world were to help push SPF adoption and accurate deployments, then we could move on to DKIM too. Right now, the best option we have is to get DMARC properly deployed as much as possible where p=reject actually rejects the message unlike SPF_FAIL that we can't trust.
Re: Penalty for no/bad SPF
On 01/24/2018 03:45 PM, Joseph Brennan wrote: The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. On 24.01.18 16:04, David Jones wrote: The key point here is the bulk nytimes.com email that is system generated, i.e. not humans with real mailboxes that could be compromised, is from subdomains so this entry would be safe since they do have good SPF records on subdomains: whitelist_auth *@*.nytimes.com this only applies when SPF succeeds so it won't fix their broken SPF :-) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows 2000: 640 MB ought to be enough for anybody
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Bill Cole wrote: On 24 Jan 2018, at 17:20 (-0500), John Hardin wrote: On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? There are entities that still hold fast to their legacy Class A networks and expose them to some degree to the world. Those who have tried to change policy from inside such an organization might argue that a multiple-B SPF authorization is neither malicious nor messed up in itself, but rather merely an admission of a reality which i arguably messed up but not at all malicious. Somebody legitimately that big could be whitelisted (SPF Malice score = 0). -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- To be civilized is to restrain the ability to commit mayhem. To be incapable of committing mayhem is not the mark of the civilized, merely the domesticated.-- Trefor Thomas --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 17:20 (-0500), John Hardin wrote: On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? There are entities that still hold fast to their legacy Class A networks and expose them to some degree to the world. Those who have tried to change policy from inside such an organization might argue that a multiple-B SPF authorization is neither malicious nor messed up in itself, but rather merely an admission of a reality which i arguably messed up but not at all malicious. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Dianne Skoll wrote: On Wed, 24 Jan 2018 14:20:57 -0800 (PST) John Hardinwrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? +all is malicious for sure. More than a Class-B might just be bad planning AKA Microsoft's outbound IP address list. I was thinking more the case where subrange assignments were used to avoid explicitly using "+all" as a way to avoid naive malice checks, e.g. doing "+ip4:0.0.0.0/1 +ip4:255.0.0.0/1". For reliability the threshold size might need to be larger than a class-B (that was off-the-cuff), or it might need some explicit whitelisting for broken-yet-legit domains (AKA msft). However, a malicious actor can use the "exists:" mechanism to simulate +all in a way that can't easily be proven by an SPF evaluator. :( I would like to see the exists: mechanism tossed. So we add a point for using "exists:" and for defining multiple subranges that add up to > class-B (or whatever threshold seems reasonable) and for any other constructs that are abused by spammers to define "SPF pass wherever my bots send from". It's sounding like we need "SPF cluebat" and "SPF malice" scoring plugins... :) -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...intellectuals have no interest in what _creates_ wealth, and what _inhibits_ the creation of wealth. They are very concerned about the _distribution_ of it, but they act as if wealth just exists somehow. It's like manna from heaven, it's only a question of how we split it up.-- Thomas Sowell --- 3 days until the 51st anniversary of the loss of Apollo 1
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018 14:20:57 -0800 (PST) John Hardinwrote: > > At this point, I would be willing to penalize sites with bad SPF > > records (syntactically invalid; more than one different SPF record > > attached to the same domain, etc.) Those people really deserve > > penalties because they've messed up. > Does that include "+all" or authorizing more than a class-b space > through any method, which I'd characterize as "malicious" rather than > "messed up"? +all is malicious for sure. More than a Class-B might just be bad planning AKA Microsoft's outbound IP address list. However, a malicious actor can use the "exists:" mechanism to simulate +all in a way that can't easily be proven by an SPF evaluator. :( I would like to see the exists: mechanism tossed. Regards, Dianne.
Re: Penalty for no/bad SPF
On 2018-01-24 18:10, Bill Cole wrote: > 1. Mail with an envelope sender domain that has no SPF record is more > likely to be spam than the overall mail stream. > > 2. Mail whose envelope sender domain has a published SPF record which > repudiates the sending IP is more likely to be spam than the overall > mail stream. > > I don't see evidence that either of those are true now, that they have > ever been true, or that they are becoming closer to true over time. I am not taking sides in this dispute, but I'd like to point out that there is a phenomenon called "self-fulfilling prophesy". As in all affairs that involve human behavior and persuasion. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. To reply privately _only_ on Usenet, fetch the TXT record for the domain.
Re: Penalty for no/bad SPF
SPF is designed for authentication, not spam filtering. Using a crowbar as a hammer. We apply a small score mainly so we see the elements reported. If the "majors" are using in their hygiene stack, for evalation like you are, I haven't seen much evidence of that. Of course it's hard to test, since we don't have log access or often intelligible header diagnostics. But from years of blackbox practical experience working cases: Cust: My Spam...err bulkmail is going to Junk, can you hook me up w/ SPF! Tech : OK we hooked you up. Cust: My Spamerr bulkmail is still going to Junk Tech: OK now diagnosed your ACTUAL problem (URL, PTR, spammy content. etc). We need to do X, Y, and Z. Cust: Great that fixed it! I can't recall a case in the last 5 years where SPF, or DKIM for that matter, tipped anyone from Junk to not. I apply an increased score for RDNS_NONE. Because I think its an ABOMINATION that so many operations think it OK to skip DNS plumbing. But I do recognize it seems a hopeless battle, trying to clean up the internet. YMMV. From: Rupert Gallagher <r...@protonmail.com> Sent: Wednesday, January 24, 2018 3:00:37 PM To: David Jones; 'users@spamassassin.apache.org' Subject: Re: Penalty for no/bad SPF If all those smarties who speak against spf would simply shut-up and implement it correctly, with dkim and dmarc, and read the dmarc reports, then they would know how valuable it is. On raising the score: done, long ago, happy with the outcome, and strongly recommend it.
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 14:59 (-0500), David Jones wrote: On 01/24/2018 01:33 PM, Bill Cole wrote: On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. I did not think or say that you were. You are proposing that SA as a project should *act as if* these correlations exist and are steadily becoming stronger: 1. Mail with an envelope sender domain that has no SPF record is more likely to be spam than the overall mail stream. 2. Mail whose envelope sender domain has a published SPF record which repudiates the sending IP is more likely to be spam than the overall mail stream. I don't see evidence that either of those are true now, that they have ever been true, or that they are becoming closer to true over time. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here. I am proposing that if SPF were more accurately deployed then SPF_FAIL would be worth something. That is a truism but what you are proposing is that SA should behave as if SPF deployment is becoming broader and more accurate steadily over time, as a sort of sympathetic magic to drive that improvement. I am saying that history implies that this would not work as hoped. Also, it is a bad precedent for project policy.Slowly rising SA scores don't exert any pressure on a sender with broken SPF until they actually cause mail to be dropped or rejected. We should not be intentionally increasing false positives no matter how noble the end goal. Individual sites obviously can make that choice on their own, now. For example, on my own personal system I would very likely see some FPs due to my unjustifiably high fixed score for SPF_FAIL, were it not for other bespoke tweaks which make actually effective FPs extremely unlikely. We could whitelist_auth more trusted senders and then be able to turn up the scores for the rest of the mail flow. If the huge SA community around the world were to help push SPF adoption and accurate deployments, then we could move on to DKIM too. Right now, the best option we have is to get DMARC properly deployed as much as possible where p=reject actually rejects the message unlike SPF_FAIL that we can't trust. SA is not the vehicle for such pressure. What has been driving careful DMARC deployment is not the long tail of sites with <10k users using SA, it is the handful of behemoth providers plus some key commercial and government systems that honor p=reject. This was also the strongest driver for SPF adoption, until those behemoths recognized how flawed SPF & DKIM alone were and came up with DMARC as a "solution" with the unsurprising side effect of damaging traditional discussion mailing lists. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Penalty for no/bad SPF
If all those smarties who speak against spf would simply shut-up and implement it correctly, with dkim and dmarc, and read the dmarc reports, then they would know how valuable it is. On raising the score: done, long ago, happy with the outcome, and strongly recommend it.
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 14:24 -0800, John Hardin wrote: > I think he was referring to MTA-side forwarding, not forwarding an > email you received (which forward comes *from you*). > I was wondering if this could be related to Joseph's comment that "DMARC is destroying forwarding and mailing lists" as some sort of circumvention, thats's all. > > and the more so because this is not an optional feature. > > *that* is unjustified. > Agreed: I feel a Bugzilla request coming on. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 04:00 PM, Vincent Fox wrote: so there's this argument that goes: "well we won't really see the benefits until it's FULLY and RIGIDLY implemented." However, look at all the major providers with messed up records and neutral or soft fail. They should have the most resources to accomplish this and the most incentives to list all their netblocks and set to hard fail. Google is soft fail. Hotmail is soft fail. (etc etc ad nauseum) I rest my case. There is nothing wrong with stopping a soft fail if that is what they want to do. In fact, most people should stop at soft fail unless they really know what they are doing or they are a major brand with a high risk spoofing. People are blindly following Microsoft's DNS entries for Office 365 setup with "-all" when they don't know what they are doing. Microsoft should be telling people to do "~all" in their setup instructions. Then Microsoft should be checking their customer's SPF records for them and showing them when it's broken in the the Admin Center. 1. We need SPF_FAIL hits to mean something and they don't. 2. We can use subdomains with SPF_PASS to safelist trusted senders that are targets of spoofing. After 14+ years we are still having this ridiculous argument about how in 14 MORE years when we finally fully implement this flawed technology, it'll do something useful. Meanwhile i see it as being more risk than benefit. With a big force like SA or Google, we could do this in a couple of years slowly and easily then start doing the same for DKIM. Frankly I'd rather these manhours be used on having correct A & PTR records, which seems to be beyond the pale for some bulkmail vendors. We could do the same thing for RDNS_NONE hits. Good idea. -- David Jones
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Martin Gregorie wrote: On Wed, 2018-01-24 at 16:45 -0500, Joseph Brennan wrote: DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, Could this be why recent releases of the Evolution MUA now send forwarded messages as attachments appended to a new message? I think he was referring to MTA-side forwarding, not forwarding an email you received (which forward comes *from you*). Its annoying, particularly if you're intending to intersperse comments in the original message as part of a detailed discussion, Reply? and the more so because this is not an optional feature. *that* is unjustified. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista is at best mildly annoying and at worst makes you want to rush to Redmond, Wash. and rip somebody's liver out. -- Forbes --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista is at best mildly annoying and at worst makes you want to rush to Redmond, Wash. and rip somebody's liver out. -- Forbes --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 16:45 -0500, Joseph Brennan wrote: > DMARC is not a standard according to RFC 7489, "Status of This Memo". > It's just informational, for those who want to play the game. DMARC > is destroying forwarding and mailing lists, > Could this be why recent releases of the Evolution MUA now send forwarded messages as attachments appended to a new message? Its annoying, particularly if you're intending to intersperse comments in the original message as part of a detailed discussion, and the more so because this is not an optional feature. Apologies if this is a bit OTT, but I don't understand why anybody would think its a good way to handle forwarded messages. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 03:45 PM, Joseph Brennan wrote: David Joneswrote: SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, and I'm sorry to see the elephants in the email room implementing it-- though Gmail still does not always reject based on DMARC reject, as if they use that plus some internal system to make the call. The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. Joseph Brennan Columbia University IT The key point here is the bulk nytimes.com email that is system generated, i.e. not humans with real mailboxes that could be compromised, is from subdomains so this entry would be safe since they do have good SPF records on subdomains: whitelist_auth *@*.nytimes.com In fact, I have put this in the 60_whitelist_auth.cf: def_whitelist_auth *@*.nytimes.com ... so bulk emails from them should be scoring pretty low for everyone running sa-update. This should prove my main point of this thread. SA actually allows for more than 10 DNS lookups so it's more forgiving than the actual spec and would likely hit SPF_PASS still as long as the emails came from a covered source. I have a recent email from a human-looking email address @nytimes.com that SA shows as SPF_PASS from a Google mail server. -- David Jones
Re: Penalty for no/bad SPF
so there's this argument that goes: "well we won't really see the benefits until it's FULLY and RIGIDLY implemented." However, look at all the major providers with messed up records and neutral or soft fail. They should have the most resources to accomplish this and the most incentives to list all their netblocks and set to hard fail. Google is soft fail. Hotmail is soft fail. (etc etc ad nauseum) I rest my case. After 14+ years we are still having this ridiculous argument about how in 14 MORE years when we finally fully implement this flawed technology, it'll do something useful. Meanwhile i see it as being more risk than benefit. Frankly I'd rather these manhours be used on having correct A & PTR records, which seems to be beyond the pale for some bulkmail vendors. From: David Jones <djo...@ena.com> Sent: Wednesday, January 24, 2018 12:12:56 PM To: users@spamassassin.apache.org Subject: Re: Penalty for no/bad SPF On 01/24/2018 01:58 PM, Vincent Fox wrote: > I'd rather not think about the manhours I've wasted this year on SPF. > > > The guy at Evotec.com, among others, who thinks rejecting > > for SOFTFAIL is a perfectly valid anti-spoofing strategy and > > doesn't blink when pointed to RFC 4408 sec 2.5.5. > > > Vendors who's first response is: > > "Our LEGIT spamerrr bulkmail is ending in your Junk. Response > > #1 in our binder is you MUST list us in your SPF record." > > Dig, dig, dig maillogs. All emails using Envelope From properly > > so SPF is a waste of everyone's time. > The Internet is very slow to change. It takes a large force like Google to improve things slowly over time. They are doing good work in the TLS and browser encryption area. SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. > > Records we included to ours, where the vendor makes a typo in > > THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. > > Either way our record suddenly returning PERMERROR and we > > have to get someone in, and boot vendor off the island on a Sunday. > I have a script that checks all of our customer's SPF records for syntax problems and too many DNS lookups based on pyspf just like http://www.kitterman.com/spf/validate.html does so I can correct it or notify them immediately. > > Endless hours explaining to campus clients, what SPF is and why > If SA all around the world says the same thing you are telling them then they will have to listen and fix their problem or remove their SPF record which is better than having an incorrect one. > it is not a good primary strategy to solve Junk mail issues > > The only good thing I have to say about SPF, is it seems to > > be a permanent employment program for people who are > > otherwise useless. > -- David Jones
Re: Penalty for no/bad SPF
David Joneswrote: SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, and I'm sorry to see the elephants in the email room implementing it-- though Gmail still does not always reject based on DMARC reject, as if they use that plus some internal system to make the call. The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. Joseph Brennan Columbia University IT
Re: Penalty for no/bad SPF
At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. I would not be willing to penalize sites with *no* SPF at all just yet. Maybe eventually. Regards, Dianne.
Re: Penalty for no/bad SPF
On 01/24/2018 01:58 PM, Vincent Fox wrote: I'd rather not think about the manhours I've wasted this year on SPF. The guy at Evotec.com, among others, who thinks rejecting for SOFTFAIL is a perfectly valid anti-spoofing strategy and doesn't blink when pointed to RFC 4408 sec 2.5.5. Vendors who's first response is: "Our LEGIT spamerrr bulkmail is ending in your Junk. Response #1 in our binder is you MUST list us in your SPF record." Dig, dig, dig maillogs. All emails using Envelope From properly so SPF is a waste of everyone's time. The Internet is very slow to change. It takes a large force like Google to improve things slowly over time. They are doing good work in the TLS and browser encryption area. SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. Records we included to ours, where the vendor makes a typo in THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. Either way our record suddenly returning PERMERROR and we have to get someone in, and boot vendor off the island on a Sunday. I have a script that checks all of our customer's SPF records for syntax problems and too many DNS lookups based on pyspf just like http://www.kitterman.com/spf/validate.html does so I can correct it or notify them immediately. Endless hours explaining to campus clients, what SPF is and why If SA all around the world says the same thing you are telling them then they will have to listen and fix their problem or remove their SPF record which is better than having an incorrect one. it is not a good primary strategy to solve Junk mail issues The only good thing I have to say about SPF, is it seems to be a permanent employment program for people who are otherwise useless. -- David Jones
Re: Penalty for no/bad SPF
I'd rather not think about the manhours I've wasted this year on SPF. The guy at Evotec.com, among others, who thinks rejecting for SOFTFAIL is a perfectly valid anti-spoofing strategy and doesn't blink when pointed to RFC 4408 sec 2.5.5. Vendors who's first response is: "Our LEGIT spamerrr bulkmail is ending in your Junk. Response #1 in our binder is you MUST list us in your SPF record." Dig, dig, dig maillogs. All emails using Envelope From properly so SPF is a waste of everyone's time. Records we included to ours, where the vendor makes a typo in THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. Either way our record suddenly returning PERMERROR and we have to get someone in, and boot vendor off the island on a Sunday. Endless hours explaining to campus clients, what SPF is and why it is not a good primary strategy to solve Junk mail issues. The only good thing I have to say about SPF, is it seems to be a permanent employment program for people who are otherwise useless. From: Martin Gregorie <mar...@gregorie.org> Sent: Wednesday, January 24, 2018 11:32:27 AM To: users@spamassassin.apache.org Subject: Re: Penalty for no/bad SPF On Wed, 2018-01-24 at 19:01 +, Vincent Fox wrote: > SPF is a zombie legacy that someone should shoot in > the head. > SPF is still good for what I've always thought was its main use: detecting spam delivered by backscatter. Given that its dirt cheap to implement, and easy too verify now that there are good checking tools there's no reason or necessity to kill it. > Maybe then we could design something that is useful for what we all > desire, which is properly authenticating senders. > ... provided that you can persuade all legit senders to buy into using it while preventing all spammers and bots from hijacking it. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 01:33 PM, Bill Cole wrote: On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here. I am proposing that if SPF were more accurately deployed then SPF_FAIL would be worth something. We could whitelist_auth more trusted senders and then be able to turn up the scores for the rest of the mail flow. If the huge SA community around the world were to help push SPF adoption and accurate deployments, then we could move on to DKIM too. Right now, the best option we have is to get DMARC properly deployed as much as possible where p=reject actually rejects the message unlike SPF_FAIL that we can't trust. -- David Jones
Re: Penalty for no/bad SPF
On 01/24/2018 01:01 PM, Vincent Fox wrote: SPF is designed for whitelisting, not blacklist. Not true. We are supposed to reject email that doesn't pass SPF checks if the domain has told us to with "-all". That is a form of blacklisting controlled by the domain itself to protect it's own brand/reputation from spoofing. If you don't know what you are doing and create an SPF record with "-all", then that's your own fault. Google has started putting many of these emails with SPF_FAIL in their Spam folder which puts more importance to get your SPF record correct and complete. Remember when "shields" appeared in mail clients, and how fast that feature disappeared? Far too many people clicking on phish that seemed "authentic". With the explosion of cheap domains and registrars, there's really no snowshoe Black Hat operation that can't comply. Compliance is 99.% in every phish I've investigated last year, the outlier I can recall was a simple typo in 1 server in the sender's network. SPF is authorization not authentication. There is a difference and complete security requires both. You use SPF to tell the Internet authorized mail servers for your domain. That doesn't have a direct tie to spam but it does allow a level of whitelisting (I like to call it safelisting) for senders you trust. SPF is a zombie legacy that someone should shoot in the head. Maybe then we could design something that is useful for what we all desire, which is properly authenticating senders. If keep in mind that SPF is authorization and DKIM is authentication, you really need both. When a sender with DMARC p=reject aligns perfectly in both SPF and DKIM then all you know is that email is legit and not spoofed. It doesn't say anything about spam or ham in the content. If you whitelist trusted senders then you segment them out of the way which allows fine tuning on the rest of the mail flow. *From:* David Jones <djo...@ena.com> *Sent:* Wednesday, January 24, 2018 6:12:19 AM *To:* 'users@spamassassin.apache.org' *Subject:* Penalty for no/bad SPF Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones -- David Jones
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here.
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 19:01 +, Vincent Fox wrote: > SPF is a zombie legacy that someone should shoot in > the head. > SPF is still good for what I've always thought was its main use: detecting spam delivered by backscatter. Given that its dirt cheap to implement, and easy too verify now that there are good checking tools there's no reason or necessity to kill it. > Maybe then we could design something that is useful for what we all > desire, which is properly authenticating senders. > ... provided that you can persuade all legit senders to buy into using it while preventing all spammers and bots from hijacking it. Martin
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018 19:01:28 + Vincent Foxwrote: > SPF is a zombie legacy that someone should shoot in > the head. +1 > Maybe then we could design something that > is useful for what we all desire, which is properly > authenticating senders. We cannot authenticate senders and keep SMTP as useful as it is now. The whole beauty of email is that some stranger can contact you out of the blue. And that mailing lists can send mail on your behalf. We can work on people not being able to spoof "well-known" senders; for that, I'd much prefer DKIM+DMARC over SPF. SPF was a mistake and we should all admit that. Regards, Dianne.
Re: Penalty for no/bad SPF
SPF is designed for whitelisting, not blacklist. Remember when "shields" appeared in mail clients, and how fast that feature disappeared? Far too many people clicking on phish that seemed "authentic". With the explosion of cheap domains and registrars, there's really no snowshoe Black Hat operation that can't comply. Compliance is 99.% in every phish I've investigated last year, the outlier I can recall was a simple typo in 1 server in the sender's network. SPF is a zombie legacy that someone should shoot in the head. Maybe then we could design something that is useful for what we all desire, which is properly authenticating senders. From: David Jones <djo...@ena.com> Sent: Wednesday, January 24, 2018 6:12:19 AM To: 'users@spamassassin.apache.org' Subject: Penalty for no/bad SPF Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones
Re: Penalty for no/bad SPF
On Wed, Jan 24, 2018 at 08:12:19AM -0600, David Jones wrote: > Google Chrome and other browsers have been slowly penalizing sites not > using encryption to the point that soon they will be alerting users of > plain HTTP sites. This along with letsencrypt.org has been moving the > HTTPS bar forward to improve web security and privacy. > > I think it's time for the SA community to help move the bar forward with > SPF. The problem is many sysadmins that don't understand SPF have been > implementing SPF incorrectly (thank you Microsoft Office 365) and > incompletely without understanding they are shooting themselves in the foot. > > I decided about a month ago to start sending feedback emails to senders > with SPF PERMERR and SPF FAIL in an attempt to help them help themselves > improve _their_ mail delivery. If you setup your SPF record like > Microsoft recommends with a "-all" and it's not completely covering all > legit sources of email, it's completely useless for any MTAs and mail > filters to take SPF_FAIL hits seriously. We should have rejected the > email per that sending domain's own wishes but we all know they didn't > intend for us to really block it so what good is it? > > What does everyone think about slowly increasing the score for SPF_NONE > and SPF_FAIL over time in the SA rulesets to force the awareness and > importance of proper SPF? This may need to have an official > announcement of what the plans/timelines would be so we could get the > word out. > it sounds like a plan, I am all for that. Giovanni > These days with DMARC reporting, it's not impossible to figure out a > good SPF record like it was 10 years ago. The real problem with SMTP in > general is there is no reliable way to get feedback to mail admins > without sending confusing technical emails to regular users. > > -- > David Jones signature.asc Description: PGP signature
Penalty for no/bad SPF
Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones