Re: [mailop] Many SPF failures lately
You appear to be making the naive assumption that every SPF record is correct, or worse, that whatever the SPF record must be correct even if it's not what the system manager intended, or it doesn't describe the domain's actual mail. In reality, nearly every SPF record is wrong, because SPF's simple model of mail transport cannot describe all the ways that people send mail. A competent mail system manager will deal with that, rather than imagining that he can force the rest of the world to change. As I have said many times, my goal is to deliver the mail that my users want. SPF, DKIM, and DMARC are all useful, but I do not make unilateral decisions based on what any of them say. Having talked to the people who run many of the world's largest mail systems, I can assure you that they don't either. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
John, I have an incredible amount of respect for you and your work, so I hope our disagreement is only because we're talking past each other. The receiving MTA has many tools to choose from when they apply spam filtering to incoming email. They may filter on sending IP, key words, score the message based on various elements of the message and other contextual data. There is no silver bullet and no regulatory mandate on exactly how it is to be done, and the receiving MTA's chief accountability is to its own customer, the intended recipient of the message. All that said, one of the ways for a sending domain to communicate to the receiving MTA how a message from their domain is to be treated is SPF. The SPF standard has described a reasonable range of actions, and when the sending domain selects "-all" they are communicating in the strongest terms possible what they want the receiving MTA to do with the message. Again, the receiving MTA doesn't have to do SPF checking and they don't have to even respect what the sending domain specified in that SPF record, but what I don't understand is how the receiving MTA retains the primary burden to still deliver message when the sending domain specifies "-all". To ignore the sender's explicit request is to claim that the receiving MTA knows better than the sending domain what do with the message. Now, it is possible, as has been laid out, to use the same tools that one regularly uses to assess a message and then decide to ignore the action items specified in the sending domain's SPF. If the receiving MTA does that, and does it well, the recipient wins. Do it wrong, and the sending domain's (best) intentions were frustrated and receiving MTA's customers were done a disservice (hopefully minor, but potentially more major if it was a phish). In regards to DMARC, do you feel so strongly about DMARC that you believe any mail operator that doesn't support DMARC processing on message receipt is doing a poor job? Frank -Original Message- From: John R Levine [mailto:jo...@taugh.com] Sent: Saturday, May 20, 2017 11:59 AM To: frnk...@iname.com Cc: mailop Subject: RE: [mailop] Many SPF failures lately On Sat, 20 May 2017, frnk...@iname.com wrote: > Are you saying that checking the box on our commercial spam filtering > system’s “check SPF” feature, which quarantines messages that have SPF > failures (-all), was a poor decision on my part? If it does that on a simple SPF failure with no other indication that a message is spam, yes.* I expect that's the sort of thing Neil was referring to when he mentioned firing offenses. > I don’t understand what DMARC has to do with this – a sender who > implements an SPF record should not the assume the receiver has also > implemented DMARC checking. Now I must say that I am really, really glad that I am not one of your mail users. Just for starters, why do you think that DMARC checks both SPF and DKIM and applies the policy only if they both fail? R's, John * - disregarding the special case of an SPF record that contains only -all, meaning that a domain sends no mail at all. But I don't think that's what we're talking about here. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Sat, 20 May 2017, frnk...@iname.com wrote: Are you saying that checking the box on our commercial spam filtering system’s “check SPF” feature, which quarantines messages that have SPF failures (-all), was a poor decision on my part? If it does that on a simple SPF failure with no other indication that a message is spam, yes.* I expect that's the sort of thing Neil was referring to when he mentioned firing offenses. I don’t understand what DMARC has to do with this – a sender who implements an SPF record should not the assume the receiver has also implemented DMARC checking. Now I must say that I am really, really glad that I am not one of your mail users. Just for starters, why do you think that DMARC checks both SPF and DKIM and applies the policy only if they both fail? R's, John * - disregarding the special case of an SPF record that contains only -all, meaning that a domain sends no mail at all. But I don't think that's what we're talking about here.___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Kinda have to agree with Frank on this one. What is the point of having SPF records if you don't want mail receivers to abide by what you are asking of them (via an spf record)? Brielle Sent from my iPhone > On May 19, 2017, at 4:47 PM, wrote: > > Yet the senders, via their SPF records with a "-all", told me to reject those > messages. As MTA's, we're doing what the send told us to do. > > Frank > > -Original Message- > From: John Levine [mailto:jo...@taugh.com] > Sent: Friday, May 19, 2017 9:56 AM > To: mailop@mailop.org > Cc: frnk...@iname.com > Subject: Re: [mailop] Many SPF failures lately > > In article <002401d2d07c$de401730$9ac04590$@iname.com> you write: >> I turned on SPF checking on our incoming email server about two or three >> months and notified >> domain holders who were sending legitimate email from bad IPs, and there, >> too, some fixed up >> their SPF records, but the majority didn't do anything. So we keep >> rejecting those emails. Most >> of them tend to be from auto-notify systems (bank statements, receipts for >> purchases from online >> stores, etc). The recipients don't complain to the sender because they're >> not aware they were >> supposed to get an email, and since a human didn't send it, there's no one >> on the sending side >> chasing it down. Most well-known cuplprit is Travelocity and their flight >> change notifications. >> Too bad the travelers aren't getting notified. > > I must say I'm glad that I'm not one of your mail users. > > For my users, I have the quaint idea that I should try and deliver the > mail that they obviously want. > > R's, > John > > > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Neil, Thanks for sharing with ExactTarget. Are you saying that checking the box on our commercial spam filtering system’s “check SPF” feature, which quarantines messages that have SPF failures (-all), was a poor decision on my part? I don’t understand what DMARC has to do with this – a sender who implements an SPF record should not the assume the receiver has also implemented DMARC checking. Let me remind everyone again – a message was sent to us from an IP address that was outside the range of the SPF record for that sending email address’s domain, and the SPF record told us to discard the message. I really don’t understand why I’m being blamed for not delivering the message. If the sender wanted a different behavior they should have used a “~all”. I feel I already went above and beyond the call of duty by contacting dozens and dozens of senders who had incomplete SPF records. It just turns out that I didn’t have a contact at Travelocity. Regards, Frank From: Neil Schwartzman [mailto:spamfighter...@icloud.com] Sent: Saturday, May 20, 2017 10:58 AM To: frnk...@iname.com Cc: Brandon Long ; mailop ; John Levine Subject: Re: [mailop] Many SPF failures lately Yeah. I did let exact target know. I work supporting a userbase probably a few hundred million the size of yours, and I can tell you, in my world. knowingly, blithely dropping legitimate email is likely a firing offense. I suggest you may wish to avail yourself of deep knowledge of DMARC technologies so you can actual insight into what senders intend you to do in light of their declarations. -- Neil Schwartzman spamfigh...@gmail.com <mailto:spamfigh...@gmail.com> Tel.: +1 (514) 629-6345 On May 20, 2017, at 11:31, mailto:frnk...@iname.com> > mailto:frnk...@iname.com> > wrote: I guess it depends on how our customers forward to the email account provided by us. I’m sure that there are some messages that we do block due to forwarding, but when I manually examined four weeks of SPF-based blocks, I don’t recall seeing one example. You’re very much right that waiting for feedback from end-users is very much incomplete. We do not do policy enforcement purely based on SPF unless it is a ”-all”. For all others it’s part of the spam analysis mix. If someone does know the mail operator/group for Travelocity, perhaps they can be alerted to the issue I raised. Frank From: Brandon Long [mailto:bl...@google.com] Sent: Saturday, May 20, 2017 1:56 AM To: Frank Bulk mailto:frnk...@iname.com> > Cc: John Levine mailto:jo...@taugh.com> >; mailop mailto:mailop@mailop.org> > Subject: Re: [mailop] Many SPF failures lately Is forwarding mail something your users never do? Or do you think the sender should be able to specify that the mail can't be forwarded? With the exception of a pure -all record, policy enforcement based purely on spf is a poor choice. Maybe, depending on your users, it won't raise the fp rate that much. OTOH, if you just reject without letting in a fraction, how do you even know what your fp rate is? Waiting for feedback from your users that they're missing messages they may not even know they should have gotten is a poor way to measure effectiveness. Brandon On May 19, 2017 9:34 PM, mailto:frnk...@iname.com> > wrote: John, I'm a bit bewildered -- these aren't random strangers, they're the actual sender. Am I supposed to second-guess the sender's instructions? If I have to second-guess every sender's "-all" then I have to have another layer of subjective analysis -- currently manual, in my situation. Frank -Original Message- From: John R Levine [mailto:jo...@taugh.com <mailto:jo...@taugh.com> ] Sent: Friday, May 19, 2017 7:22 PM To: frnk...@iname.com <mailto:frnk...@iname.com> Cc: mailop@mailop.org <mailto:mailop@mailop.org> Subject: RE: [mailop] Many SPF failures lately > Yet the senders, via their SPF records with a "-all", told me to reject those messages. As MTA's, we're doing what the send told us to do. I don't know about you, but I do not blindly follow instructions from random strangers. It rarely leads to good outcomes. > For my users, I have the quaint idea that I should try and deliver the > mail that they obviously want. Regards, John Levine, jo...@taugh.com <mailto:jo...@taugh.com> , Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly ___ mailop mailing list mailop@mailop.org <mailto:mailop@mailop.org> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org <mailto:mailop@mailop.org> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
I guess it depends on how our customers forward to the email account provided by us. I’m sure that there are some messages that we do block due to forwarding, but when I manually examined four weeks of SPF-based blocks, I don’t recall seeing one example. You’re very much right that waiting for feedback from end-users is very much incomplete. We do not do policy enforcement purely based on SPF unless it is a ”-all”. For all others it’s part of the spam analysis mix. If someone does know the mail operator/group for Travelocity, perhaps they can be alerted to the issue I raised. Frank From: Brandon Long [mailto:bl...@google.com] Sent: Saturday, May 20, 2017 1:56 AM To: Frank Bulk Cc: John Levine ; mailop Subject: Re: [mailop] Many SPF failures lately Is forwarding mail something your users never do? Or do you think the sender should be able to specify that the mail can't be forwarded? With the exception of a pure -all record, policy enforcement based purely on spf is a poor choice. Maybe, depending on your users, it won't raise the fp rate that much. OTOH, if you just reject without letting in a fraction, how do you even know what your fp rate is? Waiting for feedback from your users that they're missing messages they may not even know they should have gotten is a poor way to measure effectiveness. Brandon On May 19, 2017 9:34 PM, mailto:frnk...@iname.com> > wrote: John, I'm a bit bewildered -- these aren't random strangers, they're the actual sender. Am I supposed to second-guess the sender's instructions? If I have to second-guess every sender's "-all" then I have to have another layer of subjective analysis -- currently manual, in my situation. Frank -Original Message- From: John R Levine [mailto:jo...@taugh.com <mailto:jo...@taugh.com> ] Sent: Friday, May 19, 2017 7:22 PM To: frnk...@iname.com <mailto:frnk...@iname.com> Cc: mailop@mailop.org <mailto:mailop@mailop.org> Subject: RE: [mailop] Many SPF failures lately > Yet the senders, via their SPF records with a "-all", told me to reject those messages. As MTA's, we're doing what the send told us to do. I don't know about you, but I do not blindly follow instructions from random strangers. It rarely leads to good outcomes. > For my users, I have the quaint idea that I should try and deliver the > mail that they obviously want. Regards, John Levine, jo...@taugh.com <mailto:jo...@taugh.com> , Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly ___ mailop mailing list mailop@mailop.org <mailto:mailop@mailop.org> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Is forwarding mail something your users never do? Or do you think the sender should be able to specify that the mail can't be forwarded? With the exception of a pure -all record, policy enforcement based purely on spf is a poor choice. Maybe, depending on your users, it won't raise the fp rate that much. OTOH, if you just reject without letting in a fraction, how do you even know what your fp rate is? Waiting for feedback from your users that they're missing messages they may not even know they should have gotten is a poor way to measure effectiveness. Brandon On May 19, 2017 9:34 PM, wrote: > John, > > I'm a bit bewildered -- these aren't random strangers, they're the actual > sender. Am I supposed to second-guess the sender's instructions? If I > have > to second-guess every sender's "-all" then I have to have another layer of > subjective analysis -- currently manual, in my situation. > > Frank > > > -Original Message- > From: John R Levine [mailto:jo...@taugh.com] > Sent: Friday, May 19, 2017 7:22 PM > To: frnk...@iname.com > Cc: mailop@mailop.org > Subject: RE: [mailop] Many SPF failures lately > > > Yet the senders, via their SPF records with a "-all", told me to reject > those messages. As MTA's, we're doing what the send told us to do. > > I don't know about you, but I do not blindly follow instructions from > random strangers. It rarely leads to good outcomes. > > > For my users, I have the quaint idea that I should try and deliver the > > mail that they obviously want. > > Regards, > John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY > Please consider the environment before reading this e-mail. https://jl.ly > > > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
John, I'm a bit bewildered -- these aren't random strangers, they're the actual sender. Am I supposed to second-guess the sender's instructions? If I have to second-guess every sender's "-all" then I have to have another layer of subjective analysis -- currently manual, in my situation. Frank -Original Message- From: John R Levine [mailto:jo...@taugh.com] Sent: Friday, May 19, 2017 7:22 PM To: frnk...@iname.com Cc: mailop@mailop.org Subject: RE: [mailop] Many SPF failures lately > Yet the senders, via their SPF records with a "-all", told me to reject those messages. As MTA's, we're doing what the send told us to do. I don't know about you, but I do not blindly follow instructions from random strangers. It rarely leads to good outcomes. > For my users, I have the quaint idea that I should try and deliver the > mail that they obviously want. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Yet the senders, via their SPF records with a "-all", told me to reject those messages. As MTA's, we're doing what the send told us to do. I don't know about you, but I do not blindly follow instructions from random strangers. It rarely leads to good outcomes. For my users, I have the quaint idea that I should try and deliver the mail that they obviously want. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
I looked at the last week of blocked email from Travelocity.com and found just one blocked message. It was a flight change email from traveloc...@e.travelocity.com with a source IP of 66.244.67.50. fbulk@frankb-PC:/mnt/c/Users/fbulk$ dig TXT e.travelocity.com +short "spf2.0/pra include:cust-senderid.exacttarget.com -all" "v=spf1 include:cust-spf.exacttarget.com -all" fbulk@frankb-PC:/mnt/c/Users/fbulk$ dig TXT cust-spf.exacttarget.com +short "v=spf1 ip4:64.132.92.0/24 ip4:64.132.88.0/23 ip4:66.231.80.0/20 ip4:68.232.192.0/20 ip4:199.122.120.0/21 ip4:207.67.38.0/24 " "ip4:207.67.98.192/27 ip4:207.250.68.0/24 ip4:209.43.22.0/28 ip4:198.245.80.0/20 ip4:136.147.128.0/20 ip4:136.147.176.0/20 ip4:13.111.0.0/18 -all" fbulk@frankb-PC:/mnt/c/Users/fbulk$ Besides cust-spf-exacttarget.com having some extra quotes in their SPF record, you can see that 66.244.67.50 is not in the above SPF record(s). Frank -Original Message- From: mailop [mailto:mailop-boun...@mailop.org] On Behalf Of Carl Byington Sent: Friday, May 19, 2017 11:55 AM To: mailop@mailop.org Subject: Re: [mailop] Many SPF failures lately -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On Fri, 2017-05-19 at 03:49 -0500, frnk...@iname.com wrote: > Most well-known cuplprit is Travelocity and their flight change > notifications. The only travelocity mail I see here is from traveloc...@ac.travelocity.com via 192.161.140.0/24. Are the flight change notifications from some other system? ac.travelocity.com CNAME -> travelocity.neolane.net travelocity.neolane.net TXT -> redirect p140.neolane.net p140.neolane.net TXT "v=spf1 ip4:192.161.140.0/24 -all" Even if spf fails, we would accept those based on the DKIM signature by ac.travelocity.com which is listed in our local policy database. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEAREKAAYFAlkfI0oACgkQL6j7milTFsF0QgCfU/e06B6EOZ9sOLGOUX+HBtpV X1UAnjCwr/FwQXA3jbew/nHT1IVC2apB =Iv5/ -END PGP SIGNATURE- ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Yet the senders, via their SPF records with a "-all", told me to reject those messages. As MTA's, we're doing what the send told us to do. Frank -Original Message- From: John Levine [mailto:jo...@taugh.com] Sent: Friday, May 19, 2017 9:56 AM To: mailop@mailop.org Cc: frnk...@iname.com Subject: Re: [mailop] Many SPF failures lately In article <002401d2d07c$de401730$9ac04590$@iname.com> you write: >I turned on SPF checking on our incoming email server about two or three >months and notified >domain holders who were sending legitimate email from bad IPs, and there, too, >some fixed up >their SPF records, but the majority didn't do anything. So we keep rejecting >those emails. Most >of them tend to be from auto-notify systems (bank statements, receipts for >purchases from online >stores, etc). The recipients don't complain to the sender because they're not >aware they were >supposed to get an email, and since a human didn't send it, there's no one on >the sending side >chasing it down. Most well-known cuplprit is Travelocity and their flight >change notifications. >Too bad the travelers aren't getting notified. I must say I'm glad that I'm not one of your mail users. For my users, I have the quaint idea that I should try and deliver the mail that they obviously want. R's, John ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On Fri, 2017-05-19 at 03:49 -0500, frnk...@iname.com wrote: > Most well-known cuplprit is Travelocity and their flight change > notifications. The only travelocity mail I see here is from traveloc...@ac.travelocity.com via 192.161.140.0/24. Are the flight change notifications from some other system? ac.travelocity.com CNAME -> travelocity.neolane.net travelocity.neolane.net TXT -> redirect p140.neolane.net p140.neolane.net TXT "v=spf1 ip4:192.161.140.0/24 -all" Even if spf fails, we would accept those based on the DKIM signature by ac.travelocity.com which is listed in our local policy database. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEAREKAAYFAlkfI0oACgkQL6j7milTFsF0QgCfU/e06B6EOZ9sOLGOUX+HBtpV X1UAnjCwr/FwQXA3jbew/nHT1IVC2apB =Iv5/ -END PGP SIGNATURE- ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Fri, 19 May 2017, Luis E. Muñoz wrote: Well, it's not unheard of to see TOSes that contain provisions for spam/malware/illegal content filtering. Considering that from the 1st paragraph of RFC-7208 it's clear that the intent is to "authorize", I would think the shoe would fit. If I were looking for an excuse to play BOFH and throw away mail, that's as good an excuse as any. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On 19 May 2017, at 8:52, John Levine wrote: In article you write: It might be obvious in this particular case but it isn't in general if your users asked or agreed to reject SPF-Fails. I would be pretty impressed to find a mail system where the users even knew what SPF fails were, much less agreeing to lose real mail because of them. Well, it's not unheard of to see TOSes that contain provisions for spam/malware/illegal content filtering. Considering that from the 1st paragraph of RFC-7208 it's clear that the intent is to "authorize", I would think the shoe would fit. SPF can be a useful tool, but it's really tiring that people keep trying to make it a FUSSP. Because it isn't. I don't think anybody has claimed SPF to be a FUSSP on this discussion. As you say, it's a useful tool and some are trying to make the best use out of it. Best regards -lem ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
In article you write: >It might be obvious in this particular case but it isn't in general if >your users asked or agreed to reject SPF-Fails. I would be pretty impressed to find a mail system where the users even knew what SPF fails were, much less agreeing to lose real mail because of them. SPF can be a useful tool, but it's really tiring that people keep trying to make it a FUSSP. Because it isn't. R's, John ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Fri, 19 May 2017, at 14:56, John Levine wrote: > In article <002401d2d07c$de401730$9ac04590$@iname.com> you write: > >notified domain holders who were sending legitimate email from bad > >IPs (...) > >Most well-known cuplprit is Travelocity and their flight change > >notifications. Too bad the travelers aren't getting notified. > > I must say I'm glad that I'm not one of your mail users. > > For my users, I have the quaint idea that I should try and deliver > the mail that they obviously want. It might be obvious in this particular case but it isn't in general if your users asked or agreed to reject SPF-Fails. -- -- Andreas :-) ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
In article <002401d2d07c$de401730$9ac04590$@iname.com> you write: >I turned on SPF checking on our incoming email server about two or three >months and notified >domain holders who were sending legitimate email from bad IPs, and there, too, >some fixed up >their SPF records, but the majority didn't do anything. So we keep rejecting >those emails. Most >of them tend to be from auto-notify systems (bank statements, receipts for >purchases from online >stores, etc). The recipients don't complain to the sender because they're not >aware they were >supposed to get an email, and since a human didn't send it, there's no one on >the sending side >chasing it down. Most well-known cuplprit is Travelocity and their flight >change notifications. >Too bad the travelers aren't getting notified. I must say I'm glad that I'm not one of your mail users. For my users, I have the quaint idea that I should try and deliver the mail that they obviously want. R's, John ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
We have an automated SPF checking system in place for clients/partners/vendors and auto-notify them of invalid/malformed SPF records every three weeks. The responsive ones got them fixed up, but I still have three die-hards that haven't made any changes. Their domains are low-volume, so they probably haven't had a palpable issue. I turned on SPF checking on our incoming email server about two or three months and notified domain holders who were sending legitimate email from bad IPs, and there, too, some fixed up their SPF records, but the majority didn't do anything. So we keep rejecting those emails. Most of them tend to be from auto-notify systems (bank statements, receipts for purchases from online stores, etc). The recipients don't complain to the sender because they're not aware they were supposed to get an email, and since a human didn't send it, there's no one on the sending side chasing it down. Most well-known cuplprit is Travelocity and their flight change notifications. Too bad the travelers aren't getting notified. Frank -Original Message- From: mailop [mailto:mailop-boun...@mailop.org] On Behalf Of Michael Orlitzky Sent: Tuesday, May 16, 2017 8:20 AM To: mailop@mailop.org Subject: Re: [mailop] Many SPF failures lately On 05/15/2017 12:34 PM, D'Arcy Cain wrote: > > My personal preference is to just bounce it and make them fix their > records but it is becoming a support problem because the senders are not > reading the bounce message which explains the problem and has a link to > a page with more detail. They simply contact our users saying that it > must be our problem. > I usually respond with something like "the administrator of the sending system told us to reject this message, you'll have to take it up with him." Then if you ever hear from that guy, tell him to delete the SPF record completely. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
It's good practice to read the standard before implementing it. For permerror, 550 with 5.5.2 status code must be used, it's RFC 7208 requirement for one, who decides to block delivery based on SPF error. 451 with 4.4.3 status code should only be used for temperror, such as DNS timeout . Using 4xx for permerror is not good for sender, because he will not be aware e-mail is not delivered before some timeout and there is little chance for administrator to notice the problem in reasonable time. For temperror 4xx is used, because there is a high chance the problem is general (e.g. connectivity issues) and will be resolved. 18.05.2017 19:16, Dominique Rousseau пишет: > Le Mon, May 15, 2017 at 12:34:30PM -0400, D'Arcy Cain [da...@vex.net] a écrit: > [... broken SPF ...] >> My personal preference is to just bounce it and make them fix their >> records but it is becoming a support problem because the senders are >> not reading the bounce message which explains the problem and has a >> link to a page with more detail. They simply contact our users >> saying that it must be our problem. > You could soft-bounce with 4xx error code stating the problem > encountered. The "good" sending administrators would then view their > outgoing queues growing, have a clue about the problem, correct it, and > let the spool catch-up. > For the "bad" ones, their queues would pile up anf finally bounce to > their users. > > Else, I would consider "broken SPF" as "no SPF" rather than fail. > -- Vladimir Dubrovin @Mail.Ru ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
Le Mon, May 15, 2017 at 12:34:30PM -0400, D'Arcy Cain [da...@vex.net] a écrit: [... broken SPF ...] > > My personal preference is to just bounce it and make them fix their > records but it is becoming a support problem because the senders are > not reading the bounce message which explains the problem and has a > link to a page with more detail. They simply contact our users > saying that it must be our problem. You could soft-bounce with 4xx error code stating the problem encountered. The "good" sending administrators would then view their outgoing queues growing, have a clue about the problem, correct it, and let the spool catch-up. For the "bad" ones, their queues would pile up anf finally bounce to their users. Else, I would consider "broken SPF" as "no SPF" rather than fail. -- Dominique Rousseau Neuronnexion, Prestataire Internet & Intranet 21 rue Frédéric Petit - 8 Amiens tel: 03 22 71 61 90 - fax: 03 22 71 61 99 - http://www.neuronnexion.coop ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
In article you write: >I hope that bouncing messages from servers not listed in a valid SPF >record isn't controversial. Controversial? Naah. Foolish and self-defeating? Definitely. R's, John ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
According to the standard, invlid SPF record results in spf=permerror, not in spf=fail. It's up to you to reject the message in this case, but it's definitely not what system administrator of the sending system told you. 16.05.2017 16:20, Michael Orlitzky пишет: > On 05/15/2017 12:34 PM, D'Arcy Cain wrote: >> My personal preference is to just bounce it and make them fix their >> records but it is becoming a support problem because the senders are not >> reading the bounce message which explains the problem and has a link to >> a page with more detail. They simply contact our users saying that it >> must be our problem. >> > I usually respond with something like "the administrator of the sending > system told us to reject this message, you'll have to take it up with > him." Then if you ever hear from that guy, tell him to delete the SPF > record completely. > > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop -- Vladimir Dubrovin @Mail.Ru ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On 05/15/2017 12:34 PM, D'Arcy Cain wrote: > > My personal preference is to just bounce it and make them fix their > records but it is becoming a support problem because the senders are not > reading the bounce message which explains the problem and has a link to > a page with more detail. They simply contact our users saying that it > must be our problem. > I usually respond with something like "the administrator of the sending system told us to reject this message, you'll have to take it up with him." Then if you ever hear from that guy, tell him to delete the SPF record completely. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Mon, May 15, 2017 at 11:02 AM, D'Arcy Cain wrote: > On 2017-05-15 01:20 PM, Ken O'Driscoll via mailop wrote: > >> >> On Mon, 2017-05-15 at 12:34 -0400, D'Arcy Cain wrote: >> [...snip...] >> >>> Are admins getting dumber or is the software (py-policyd in our case) >>> getting tougher? >>> >> >> You'd be surprised how many people who would identify as being technical >> or >> worse, mail admins, still don't fully understand how SPF works. That >> > > I would not be surprised at all. After thirty five years in various areas > of the computer industry I know exactly what to expect from the "experts." > > What do others think is best practice? Should we treat broken SPF >>> records as if there was no record and just not check the sending server? >>> >> >> This approach is taken by many mailbox providers. I don't think treating >> an >> invalid SPF as the equivalent of none being present is a controversial >> position anymore. >> > > Yah, I already changed this while waiting for this discussion. Looks like > it should be permanent. Maybe I can add something to the message to let > users know that the sender is suspicious. Probably something in the > headers that can be examined by Spamassassin and add points. I would do > the same thing for broken records as well as non-existent ones. > > I have an ISP client with the same challenge. They don't accept email from >> sources not listed in the SPF (if present). If someone opens a ticket they >> > > I hope that bouncing messages from servers not listed in a valid SPF > record isn't controversial. I mean, you can, but there's this thing called forwarding, and there are good reasons to not rewrite the envelope sender when forwarding... and sometimes good reasons to do so. Brandon ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Mon, 2017-05-15 at 14:02 -0400, D'Arcy Cain wrote: > Sounds like manual labour. My client has the scale to be able to have customer service do that. But, you could automate it fairly easily. Even going so far as to notify the DNS RP with a different message than the sender. #rainy-day-project Ken. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On 2017-05-15 01:20 PM, Ken O'Driscoll via mailop wrote: On Mon, 2017-05-15 at 12:34 -0400, D'Arcy Cain wrote: [...snip...] Are admins getting dumber or is the software (py-policyd in our case) getting tougher? You'd be surprised how many people who would identify as being technical or worse, mail admins, still don't fully understand how SPF works. That I would not be surprised at all. After thirty five years in various areas of the computer industry I know exactly what to expect from the "experts." What do others think is best practice? Should we treat broken SPF records as if there was no record and just not check the sending server? This approach is taken by many mailbox providers. I don't think treating an invalid SPF as the equivalent of none being present is a controversial position anymore. Yah, I already changed this while waiting for this discussion. Looks like it should be permanent. Maybe I can add something to the message to let users know that the sender is suspicious. Probably something in the headers that can be examined by Spamassassin and add points. I would do the same thing for broken records as well as non-existent ones. I have an ISP client with the same challenge. They don't accept email from sources not listed in the SPF (if present). If someone opens a ticket they I hope that bouncing messages from servers not listed in a valid SPF record isn't controversial. use a 24h white-list which lets their user receive the mail and gives the sender time to resolve their SPF. It's worked for the last few years without issue. Sounds like manual labour. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:da...@vex.net VoIP: sip:da...@vex.net ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Mon, May 15, 2017 at 10:20 AM, Ken O'Driscoll via mailop < mailop@mailop.org> wrote: > > On Mon, 2017-05-15 at 12:34 -0400, D'Arcy Cain wrote: > [...snip...] > > Are admins getting dumber or is the software (py-policyd in our case) > > getting tougher? > > You'd be surprised how many people who would identify as being technical or > worse, mail admins, still don't fully understand how SPF works. That > includes not keeping their SPF tidy and falling foul of the ten lookup > limit. > We've found, for larger entities, the ten item lookup is easily exceeded, especially since folks (like us) require multiple lookups (ie, GSuite's _ spf.google.com is 4 lookups in total). I know there are some folks who then write scripts or use a service to combine records for all of their senders into a small enough number to make it work (with monitoring to keep abreast of changes to the original records), but that's quite a bit of work. We've chosen to be more lenient in terms of the number of lookups we allow, as getting the information on authentication is more useful to us than the marginal increase in dns requests we make (and acknowledging that our cache hit rate is probably higher than smaller providers). It's true that relying on more than 10 working widely is not a great strategy. Brandon > > Add to that some major ESPs still recommending that their customers create > / replace their SPF with "v=spf1 +include:spf.esp.com ?all" > > Plus, some mailbox providers recommending that their customers create / > replace their SPF with "v=spf1 +include:spf.provider.com -all" > > And, at least one major DNS provider still letting their users create > depreciated "SPF" type records. > > The all to common "cut and paste" admin is not well served. > > > What do others think is best practice? Should we treat broken SPF > > records as if there was no record and just not check the sending server? > > This approach is taken by many mailbox providers. I don't think treating an > invalid SPF as the equivalent of none being present is a controversial > position anymore. > > [...snip...] > > They simply contact our users saying that it must be our problem. > > I have an ISP client with the same challenge. They don't accept email from > sources not listed in the SPF (if present). If someone opens a ticket they > use a 24h white-list which lets their user receive the mail and gives the > sender time to resolve their SPF. It's worked for the last few years > without issue. > > > Ken. > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
On Mon, 2017-05-15 at 12:34 -0400, D'Arcy Cain wrote: [...snip...] > Are admins getting dumber or is the software (py-policyd in our case) > getting tougher? You'd be surprised how many people who would identify as being technical or worse, mail admins, still don't fully understand how SPF works. That includes not keeping their SPF tidy and falling foul of the ten lookup limit. Add to that some major ESPs still recommending that their customers create / replace their SPF with "v=spf1 +include:spf.esp.com ?all" Plus, some mailbox providers recommending that their customers create / replace their SPF with "v=spf1 +include:spf.provider.com -all" And, at least one major DNS provider still letting their users create depreciated "SPF" type records. The all to common "cut and paste" admin is not well served. > What do others think is best practice? Should we treat broken SPF > records as if there was no record and just not check the sending server? This approach is taken by many mailbox providers. I don't think treating an invalid SPF as the equivalent of none being present is a controversial position anymore. [...snip...] > They simply contact our users saying that it must be our problem. I have an ISP client with the same challenge. They don't accept email from sources not listed in the SPF (if present). If someone opens a ticket they use a 24h white-list which lets their user receive the mail and gives the sender time to resolve their SPF. It's worked for the last few years without issue. Ken. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Many SPF failures lately
> On May 15, 2017, at 9:34 AM, D'Arcy Cain wrote: > > I tried sending this a few days ago but it doesn't appear to be arriving. > Trying again. Apologies if you see this twice. > > I am finding that lately there are a lot of reports of failures sending to us > due to SPF failures. Is anyone else seeing this? When I investigate I can > see the obvious errors in the record. Are admins getting dumber or is the > software (py-policyd in our case) getting tougher? What do others think is > best practice? Should we treat broken SPF records as if there was no record > and just not check the sending server? Not sure how to do that but hopefully > there is a switch for that in the configuration. > > My personal preference is to just bounce it and make them fix their records > but it is becoming a support problem because the senders are not reading the > bounce message which explains the problem and has a link to a page with more > detail. They simply contact our users saying that it must be our problem. Rejecting mail solely for SPF misconfiguration or failure is probably something you shouldn't do unless ideological purity is more important to you than the happiness of your users. Treating an invalid SPF record as no SPF record is about as strict as I'd want to get. Cheers, Steve ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
[mailop] Many SPF failures lately
I tried sending this a few days ago but it doesn't appear to be arriving. Trying again. Apologies if you see this twice. I am finding that lately there are a lot of reports of failures sending to us due to SPF failures. Is anyone else seeing this? When I investigate I can see the obvious errors in the record. Are admins getting dumber or is the software (py-policyd in our case) getting tougher? What do others think is best practice? Should we treat broken SPF records as if there was no record and just not check the sending server? Not sure how to do that but hopefully there is a switch for that in the configuration. My personal preference is to just bounce it and make them fix their records but it is becoming a support problem because the senders are not reading the bounce message which explains the problem and has a link to a page with more detail. They simply contact our users saying that it must be our problem. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:da...@vex.net VoIP: sip:da...@vybenetworks.com -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:da...@vex.net VoIP: sip:da...@vex.net ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
