New whitelisting trick using from and spf
Well, new to me. Maybe others have thought of this. Many domains send nothing but good email and if you whitelist them based on FCRDNS all is good. Been doing that. But ... Many domains send nothing but good email and they send through reputable email sender services which are mostly good by not perfect. So can't just whitelist that. What I'm doing now is whitelisting the domains that are good, but doing SPF checks on the from address. If the from address is whitelisted AND the SPF of the from address is good - I pass the email. I'm still experimenting with this but I think I'm onto something. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: New whitelisting trick using from and spf
>From: Marc Perkel >Sent: Sunday, March 5, 2017 12:38 PM >To: users@spamassassin.apache.org >Subject: New whitelisting trick using from and spf >Well, new to me. Maybe others have thought of this. Been doing this for a couple of years now. >What I'm doing now is whitelisting the domains that are good, but doing >SPF checks on the from address. whitelist_auth does this against SPF_PASS and DKIM_VALID_AU >If the from address is whitelisted AND the SPF of the from address is >good - I pass the email. I sent an email to the list about a month ago trying to explain this and see if there was any interest in setting up something to share these lists via saupdate just like paypal.com is done in the 60_whitelist_dkim.cf. I have over 3,000 whitelist_auth entries now and growing weekly based on low scores during the past week and some other characteristics. Seems like we could setup a clearinghouse much like the masscheck where entries could be submitted and then evaluated for approval. Dave
Re: New whitelisting trick using from and spf
On 2017-03-05 (18:59 MST), David Jones wrote: > > whitelist_auth does this against SPF_PASS and DKIM_VALID_AU I tired to do something along these lines at some point in the past by adding some lines to my local.cf like these: blacklist_from *@amazon.com whitelist_auth *@amazon.com blacklist_from *@paypal.com whitelist_auth *@paypal.com It didn’t have the desired effect and simply blacklisted all PayPal mail. While *I* was ok with blacklisting PayPal, others not so much... -- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Re: New whitelisting trick using from and spf
>From: @lbutlr >Sent: Monday, March 6, 2017 5:24 AM >To: users@spamassassin.apache.org >Subject: Re: New whitelisting trick using from and spf >On 2017-03-05 (18:59 MST), David Jones wrote: >> >> whitelist_auth does this against SPF_PASS and DKIM_VALID_AU >I tired to do something along these lines at some point in the past by >adding some lines to my local.cf like these: >blacklist_from *@amazon.com >whitelist_auth *@amazon.com >blacklist_from *@paypal.com >whitelist_auth *@paypal.com >It didn’t have the desired effect and simply blacklisted all PayPal mail. >While *I* was ok with blacklisting PayPal, others not so much... Spam/phishing emails pretending to be from Paypal won't have an envelope-from of *@paypal.com which is why you didn't get the desired effect. You rarely use the blacklist_from only when there is very dumb senders that you want to block. A multi-level approach will give you the results you expect: Level 1: RBLs, other DNS checks, postscreen, greylisting, etc. Level 2: SA bayes, ClamAV w extra sigs, meta rules, RBL scores, etc. Level 1 above is very low resource and fast. Level 2 is more resource intensive. By safelisting with whitelist_auth entries for trusted senders, you can turn up the sensitivity (increase scores) a bit more. Level 1 will knock down the majority of junk before SA sees it. For this reason, you have to whitelist certain sending IPs from Level 1 using postwhite so they get to Level 2. Dave
Re: New whitelisting trick using from and spf
>From: Reindl Harald >Sent: Monday, March 6, 2017 5:58 AM >To: David Jones; @; users@spamassassin.apache.org >Subject: Re: New whitelisting trick using from and spf >Am 06.03.2017 um 12:45 schrieb David Jones: >>> From: @lbutlr >>> Sent: Monday, March 6, 2017 5:24 AM >>> To: users@spamassassin.apache.org >>> Subject: Re: New whitelisting trick using from and spf >> >>> On 2017-03-05 (18:59 MST), David Jones wrote: >>>> >>>> whitelist_auth does this against SPF_PASS and DKIM_VALID_AU >> >>> I tired to do something along these lines at some point in the past by >>> adding some lines to my local.cf like these: >> >>> blacklist_from *@amazon.com >>> whitelist_auth *@amazon.com >>> blacklist_from *@paypal.com >>> whitelist_auth *@paypal.com >> >>> It didn’t have the desired effect and simply blacklisted all PayPal mail. >>> While *I* was ok with blacklisting PayPal, others not so much... >> >> Spam/phishing emails pretending to be from Paypal won't have an >> envelope-from of *@paypal.com which is why you didn't get the >> desired effect. You rarely use the blacklist_from only when there >> is very dumb senders that you want to block >that don't matter - "blacklist_from" also bpocks from-header Good to know but the From: header is not a reliably way to block email unless there is a very dumb/naive sender. >problem is that lines like above also block legit paypal traffic which >has a enevelope outside that domain but get blocked by "blacklist_from" >and "whitelist_auth" won#t trigger as long as you don't add all the >possible underlying envelope-senders (which are in case of newsletters >large, shared senders where you don't want that) >the concept above does not work in real life because it is a very naive >approach by lacking real expierience - took me less than 24 hours to >stop testing that and find out it does more harm than good Like I said, you very rarely should use blacklist_from. RBLs and other DNS checks should take care of most of the need for blacklist_from entries.
Re: New whitelisting trick using from and spf
On 05.03.17 10:38, Marc Perkel wrote: Well, new to me. Maybe others have thought of this. Many domains send nothing but good email and if you whitelist them based on FCRDNS all is good. Been doing that. But ... Many domains send nothing but good email and they send through reputable email sender services which are mostly good by not perfect. So can't just whitelist that. What I'm doing now is whitelisting the domains that are good, but doing SPF checks on the from address. do you mean the header From: address? because anyone doing SPF does spf checks does what you describe on the envelope from: addres. If the from address is whitelisted AND the SPF of the from address is good - I pass the email. or do you do this on MTA-level (which means it's off-topic)? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I intend to live forever - so far so good.
Re: New whitelisting trick using from and spf
Spam/phishing emails pretending to be from Paypal won't have an envelope-from of *@paypal.com which is why you didn't get the desired effect. You rarely use the blacklist_from only when there is very dumb senders that you want to block that don't matter - "blacklist_from" also bpocks from-header On 06.03.17 12:09, David Jones wrote: Good to know but the From: header is not a reliably way to block email unless there is a very dumb/naive sender. yes, it's very bad when your friend fakes up paypal From: address and he gets blocked :-) Imho, blocking this is really non-issue. problem is that lines like above also block legit paypal traffic which has a enevelope outside that domain but get blocked by "blacklist_from" and "whitelist_auth" won#t trigger as long as you don't add all the possible underlying envelope-senders (which are in case of newsletters large, shared senders where you don't want that) the concept above does not work in real life because it is a very naive approach by lacking real expierience - took me less than 24 hours to stop testing that and find out it does more harm than good Like I said, you very rarely should use blacklist_from. RBLs and other DNS checks should take care of most of the need for blacklist_from entries. using blacklist_from for idiots that spam you repeatedly and they can't be contacted or don't react is find. The only problem is that this kind of blacklists tends to gro up over time - that's why we bave SpamAssassin -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. If Barbie is so popular, why do you have to buy her friends?
Re: New whitelisting trick using from and spf
On 2017-03-06 (04:45 MST), David Jones wrote:
>
>> From: @lbutlr
>> Sent: Monday, March 6, 2017 5:24 AM
>> To: users@spamassassin.apache.org
>> Subject: Re: New whitelisting trick using from and spf
>
>> On 2017-03-05 (18:59 MST), David Jones wrote:
>>>
>>> whitelist_auth does this against SPF_PASS and DKIM_VALID_AU
>
>> I tired to do something along these lines at some point in the past by
>> adding some lines to my local.cf like these:
>
>> blacklist_from *@amazon.com
>> whitelist_auth *@amazon.com
>> blacklist_from *@paypal.com
>> whitelist_auth *@paypal.com
>
>> It didn’t have the desired effect and simply blacklisted all PayPal mail.
>> While *I* was ok with blacklisting PayPal, others not so much...
>
> Spam/phishing emails pretending to be from Paypal won't have an
> envelope-from of *@paypal.com which is why you didn't get the
> desired effect. You rarely use the blacklist_from only when there
> is very dumb senders that you want to block.
>
> A multi-level approach will give you the results you expect:
> Level 1: RBLs, other DNS checks, postscreen, greylisting, etc.
> Level 2: SA bayes, ClamAV w extra sigs, meta rules, RBL scores, etc.
Do all of that and fake PayPal/amazon/apple/{random bank} emails are received
every day.
It seems it should be easy to setup “If mail claims to be From: PayPal.com and
is not from PayPal, score +100” but it is not.
--
Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Re: New whitelisting trick using from and spf
On Sun, 5 Mar 2017 10:38:09 -0800 Marc Perkel wrote: > If the from address is whitelisted AND the SPF of the from address is > good - I pass the email. And that's exactly how SPF is supposed to work. You shouldn't whitelist domains willy-nilly because they can be spoofed, and you shouldn't allow SPF "pass" messages through without spam-scanning because spammers set up SPF records, but the combination of a domain you trust and an SPF "pass" is a definite ham indicator. We also ignore end-user-created whitelists on a domain or sender if there's an SPF "fail" or "softfail". Regards, DIanne.
Re: New whitelisting trick using from and spf
> It seems it should be easy to setup “If mail claims to be From: PayPal.com > and is not from PayPal, score +100” but it is not. This is what DMARC is for. Run opendmarc as a milter and reject failures. Or score later on DMARC failure, even if just selectively for highly phished domains. PayPal publishes p=reject, on paypal.com at least, if not their other domains.
Re: New whitelisting trick using from and spf
On Mon, 6 Mar 2017, Alan Hodgson wrote:
It seems it should be easy to setup “If mail claims to be From: PayPal.com
and is not from PayPal, score +100” but it is not.
This is what DMARC is for.
Run opendmarc as a milter and reject failures. Or score later on DMARC
failure, even if just selectively for highly phished domains.
PayPal publishes p=reject, on paypal.com at least, if not their other domains.
But that won't help you when the scammers set the user visible from as
"acco...@paypai.com" or some other variant (with the actual address part as
or something else.
user-agents (such as OutHouse) by default only show the "comment" part of the
address and hide the actual <> address part, making it easy for scammers to fool
the non-tech savvy users.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: New whitelisting trick using from and spf
On Monday 06 March 2017 11:58:25 David B Funk wrote: > On Mon, 6 Mar 2017, Alan Hodgson wrote: > >> It seems it should be easy to setup “If mail claims to be From: > >> PayPal.com > >> and is not from PayPal, score +100” but it is not. > > > > This is what DMARC is for. > > > > Run opendmarc as a milter and reject failures. Or score later on DMARC > > failure, even if just selectively for highly phished domains. > > > > PayPal publishes p=reject, on paypal.com at least, if not their other > > domains. > But that won't help you when the scammers set the user visible from as > "acco...@paypai.com" or some other variant (with the actual address part as > or something else. > > user-agents (such as OutHouse) by default only show the "comment" part of > the address and hide the actual <> address part, making it easy for > scammers to fool the non-tech savvy users. Well, sure. And they can use any variant of paypal.whatever that they own, too, to show in better email clients. But you do what you can. Personally I've been flagging anything with paypal or pay pal anywhere in the From: that doesn't have a whitelisted PayPal domain's DKIM signature on it, but I don't know how well that scales.
Re: New whitelisting trick using from and spf
On Mon, 6 Mar 2017 11:58:25 -0600 (CST)
David B Funk wrote:
> But that won't help you when the scammers set the user visible from
> as "acco...@paypai.com" or some other variant (with the actual
> address part as or something else.
I recall someone bringing this up on the DMARC discussion list. :)
http://lists.dmarc.org/pipermail/dmarc-discuss/2012-February/000206.html
> user-agents (such as OutHouse) by default only show the "comment"
> part of the address and hide the actual <> address part, making it
> easy for scammers to fool the non-tech savvy users.
Way back, I proposed recommending that MUAs that show DMARC results also
show the domain name in brackes, similar to what slashdot.org does
for URLS.
So if you have an email with this header:
From: "acco...@paypal.com"
The MUA should display the From: field as:
acco...@paypal.com [example.org]
That suggestion was deemed out of scope for the DMARC standard, so... bleh.
I really don't know a good way around this. Maybe if an email passes
DMARC but has a comment field that looks like an email address, the
comment fieldd should be suppressed? ("Looks like" should probably
be as generic as "contains an @ sign")
Regards,
Dianne.
Re: New whitelisting trick using from and spf
> On Mar 6, 2017, at 12:58 PM, David B Funk
> wrote:
>
> On Mon, 6 Mar 2017, Alan Hodgson wrote:
>
>>> It seems it should be easy to setup “If mail claims to be From: PayPal.com
>>> and is not from PayPal, score +100” but it is not.
>>
>> This is what DMARC is for.
>>
>> Run opendmarc as a milter and reject failures. Or score later on DMARC
>> failure, even if just selectively for highly phished domains.
>>
>> PayPal publishes p=reject, on paypal.com at least, if not their other
>> domains.
>
> But that won't help you when the scammers set the user visible from as
> "acco...@paypai.com" or some other variant (with the actual address part as
> or something else.
>
> user-agents (such as OutHouse) by default only show the "comment" part of the
> address and hide the actual <> address part, making it easy for scammers to
> fool the non-tech savvy users.
And OS-X Mail.app in some configurations, and iOS Mail.
They all fail not just for making phishing so much easier, but get on the phone
with a novice user using any of these email clients and ask them to give you
the actual email address of a sender, especially when they have for example,
two people name “John Smith” emailing them… It’s a terrible, terrible idea to
hide things to make email easier.
Charles
>
> --
> Dave Funk University of Iowa
> College of Engineering
> 319/335-5751 FAX: 319/384-0549 1256 Seamans Center
> Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
> #include
> Better is not better, 'standard' is better. B{
Re: New whitelisting trick using from and spf
On 03/06/17 04:19, Matus UHLAR - fantomas wrote: On 05.03.17 10:38, Marc Perkel wrote: Well, new to me. Maybe others have thought of this. Many domains send nothing but good email and if you whitelist them based on FCRDNS all is good. Been doing that. But ... Many domains send nothing but good email and they send through reputable email sender services which are mostly good by not perfect. So can't just whitelist that. What I'm doing now is whitelisting the domains that are good, but doing SPF checks on the from address. do you mean the header From: address? because anyone doing SPF does spf checks does what you describe on the envelope from: addres. Yes - I'm using the headers From: address. If the from address is whitelisted AND the SPF of the from address is good - I pass the email. or do you do this on MTA-level (which means it's off-topic)? I do it at the MTA level - but it's not off topic because the concept can be applied to spamassassin. Also - I have almost 100,000 domains in my hostkarma.junkemailfilter.com (127.0.0.1) rbl. So I'm passing a lot of good email with this trick. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: New whitelisting trick using from and spf
>From: Marc Perkel >Sent: Monday, March 6, 2017 11:05 AM >To: users@spamassassin.apache.org >Subject: Re: New whitelisting trick using from and spf >> do you mean the header From: address? >> >> because anyone doing SPF does spf checks does what you describe on the >> envelope from: addres. >Yes - I'm using the headers From: address. Not good. SPF should be checked against the envelope-from address which is more trustworthy. The From: header can be spoofed trivially with no validation/authentication if DMARC is not enabled. Most email is not enabled for actual DMARC checking. Most have SPF enabled. Some have DKIM enabled. But DMARC can go one step further to check the From: header and most don't do it unless they are a major target of spoofing like Paypal, eBay, etc. Dave
Re: New whitelisting trick using from and spf
On Mon, 6 Mar 2017 23:22:00 + David Jones wrote: [...] > Not good. SPF should be checked against the envelope-from > address which is more trustworthy. Er... well. The envelope-from is not any more trustworthy than the header From:. But it *is* the thing the SPF spec say to check, and *not* the header From:. Regards, Dianne.
Re: New whitelisting trick using from and spf
>From: Dianne Skoll >Sent: Monday, March 6, 2017 5:40 PM >To: users@spamassassin.apache.org >Subject: Re: New whitelisting trick using from and spf >On Mon, 6 Mar 2017 23:22:00 + >David Jones wrote: >> Not good. SPF should be checked against the envelope-from >> address which is more trustworthy. >Er... well. The envelope-from is not any more trustworthy than >the header From:. But it *is* the thing the SPF spec say to check, >and *not* the header From:. It should be way more trustworthy since it is where bounces go. Many MTAs can do DNS checks (make sure it exists in DNS) plus DBL checks against the envelope domain. Regular user mailboxes where compromised accounts come from usually don't/can't spoof the envelope-from. It's definitely more reliable which is why the SPF spec chose to use it. Later the DMARC standard came along to help protect the From: header from being spoofed but that is a much harder problem to solve.
Re: New whitelisting trick using from and spf
On 03/06/17 15:22, David Jones wrote: From: Marc Perkel Sent: Monday, March 6, 2017 11:05 AM To: users@spamassassin.apache.org Subject: Re: New whitelisting trick using from and spf do you mean the header From: address? because anyone doing SPF does spf checks does what you describe on the envelope from: addres. Yes - I'm using the headers From: address. Not good. SPF should be checked against the envelope-from address which is more trustworthy. The From: header can be spoofed trivially with no validation/authentication if DMARC is not enabled. Most email is not enabled for actual DMARC checking. Most have SPF enabled. Some have DKIM enabled. But DMARC can go one step further to check the From: header and most don't do it unless they are a major target of spoofing like Paypal, eBay, etc. Dave Yes - I'm doing something different - and possibly more effective. And it's working really well. Those who spoof would fail the test and not get while listed. The fact that From is easier to spoof makes it more effective - not less. So if the from is @paypal.com and the sending host is not SPF compatible then it doesn't get white listed. Seems to be working very well. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: New whitelisting trick using from and spf
On Tue, 7 Mar 2017 00:04:59 + David Jones wrote: > >Er... well. The envelope-from is not any more trustworthy than > >the header From:. But it *is* the thing the SPF spec say to check, > >and *not* the header From:. > It should be way more trustworthy since it is where bounces go. You assume that someone who is willing to forge a sender address (whether envelope or header) is going to be fastidious about receiving bounces? :) > Many MTAs can do DNS checks (make sure it exists in DNS) plus > DBL checks against the envelope domain. Regular user mailboxes > where compromised accounts come from usually don't/can't spoof > the envelope-from. It's definitely more reliable which is why the > SPF spec chose to use it. No, that's not true. It's no more "reliable" than anything else. In fact, in the entire SMTP transaction, there's only one set of email addresses that are reliable, and those are the RCPT To: addresses. SPF chose to use envelope sender not because it's more reliable, but (I suspect) so as not to break mailing lists. Anyway: We see millions of spams per day. Tons of them have spoofed envelope sender addresses and tons have spoofed From: header addresses. Regards, Dianne.
Re: New whitelisting trick using from and spf
On Tue, 7 Mar 2017 09:12:00 -0500 Dianne Skoll wrote: > SPF chose to use envelope sender not because it's more reliable, but > (I suspect) so as not to break mailing lists. More likely because the original intent was to reject as early as possible.
