Re: Hints needed for spf rule

[RW]

On Thu, 4 Oct 2018 01:29:11 -0400
Adam Katz wrote:


> The ptr mechanism in SPF is officially “do not use” right in the spec
> ; PTR records aren’t
> vetted (any network operator can assign literally any rDNS to their
> IPs), so it trivializes forgery that would elicit an SPF pass.

It marked as "do not use" because it slow, and it's more sensitive to
packet loss, not because it can be forged. The implementation  is
required to check that the DNS is full-circle by performing  A-record
look-ups on the rDNS result(s).

Re: Hints needed for spf rule

[Adam Katz]

(Please ignore my last message. My phone hit “send” randomly.)

On Sep 28, 2018, at 9:48 AM EDT, bOnK wrote:
> A better idea might be testing if SPF for a external domain would pass on 
> your own server.
> This is what milter greylist does.
> http://hcpnet.free.fr/milter-greylist/

That’s interesting! We’d definitely need to ensure external relays for such a 
rule in SA, though of course this’d also require some plugin dev work. Does 
anybody have stats on that?

> Though probably exceptional, according to the RFC +all *can be* restrictive...
> https://tools.ietf.org/html/rfc7208#appendix-A.4
> 
>> A.4.  Multiple Requirements Example
>> 
>>Say that your sender policy requires both that the IP address is
>>within a certain range and that the reverse DNS for the IP matches.
>>This can be done several ways, including the following:
>> 
>>example.com.   SPF  ( "v=spf1 "
>>  "-include:ip4._spf.%{d} "
>>  "-include:ptr._spf.%{d} "
>>  "+all" )
>>ip4._spf.example.com.  SPF  "v=spf1 -ip4:192.0.2.0/24 +all"
>>ptr._spf.example.com.  SPF  "v=spf1 -ptr +all"
>> 
>>This example shows how the "-include" mechanism can be useful, how an
>>SPF record that ends in "+all" can be very restrictive, and the use
>>of De Morgan's Law.

This is amazing. And disgusting.

And the only remotely legitimate usage of either the ptr mechanism or 
(separately) inanity like invoking De Morgan’s Law, and therefore also +all.

The ptr mechanism in SPF is officially “do not use” right in the spec 
; PTR records aren’t vetted 
(any network operator can assign literally any rDNS to their IPs), so it 
trivializes forgery that would elicit an SPF pass.

Using De Morgan to intersect ptr with an un-forgeable requirement alleviates 
the issues of ptr but it’s much less complicated to merely bless each one in 
the SPF record.

Any (non-spammer) senders large enough to have issues fitting individual IPs in 
the max size of a record should definitely not delegate control of SPF to rDNS. 
They should instead better allocate their IP space for proper control by CIDR 
or else give up and use an Email Service Provider that actually knows what it’s 
doing 

Re: Hints needed for spf rule

[Adam Katz]

On Sep 28, 2018, at 9:48 AM, bOnK wrote:
A better idea might be testing if SPF for a external domain would pass on your 
own server.
> 
> This is what milter greylist does.
> http://hcpnet.free.fr/milter-greylist/
> 
> Though probably exceptional, according to the RFC +all *can be* restrictive...
> https://tools.ietf.org/html/rfc7208
> 
> A.4.  Multiple Requirements Example
> 
>Say that your sender policy requires both that the IP address is
>within a certain range and that the reverse DNS for the IP matches.
>This can be done several ways, including the following:
> 
>example.com.   SPF  ( "v=spf1 "
>  "-include:ip4._spf.%{d} "
>  "-include:ptr._spf.%{d} "
>  "+all" )
>ip4._spf.example.com.  SPF  "v=spf1 -ip4:192.0.2.0/24 +all"
>ptr._spf.example.com.  SPF  "v=spf1 -ptr +all"
> 
>This example shows how the "-include" mechanism can be useful, how an
>SPF record that ends in "+all" can be very restrictive, and the use
>of De Morgan's Law.
> 
> -- 
> b.

Re: Hints needed for spf rule

[bOnK]

On 24-9-2018 17:13, Adam Katz wrote:
These SPF records are all effectively equivalent (the fourth is Sender 
ID , we'll get to #5 later):

v=spf1 +all
v=spf1 all
v=spf1 all 192.0.2.0/24
v=spf2.0/mfrom +all
v=spf1 1.2.3.0/1 128.4.5.0/2 192.6.7.8/3 -all

So therefore I propose regexps like |/^v=spf[12].*[\s+]all\b/| and 
|/^v=spf[12].*\s\?all\b/| (the latter should be very rare and a better 
indication of a clueless admin than a spammer).


The fifth item above permits 0.0.0.0 to 223.255.255.255 and therefore 
only multicast and the reserved Class E network are banned. To address 
this, consider |/^v=spf[12].*[0-9]\/[0-7]\b/|. I haven't observed this 
sort of workaround (yet), but it's the attackers' logical escalation 
in response to this. (I conservatively chose a max mask of /7, though 
I don't think there's any legitimate use of /8, even by the remaining 
Class A holders 
 
like AT, HP, and the US DoD—nobody /should/ have an email network 
even approaching a /16 let alone a /8, though note that Google 
currently includes three /16s. I'm not sure where to draw a similar 
"too large" threshold for IPv6; perhaps /32?)


-Adam (still here, sometimes)




A better idea might be testing if SPF for a external domain would pass 
on your own server.

This is what milter greylist does.
http://hcpnet.free.fr/milter-greylist/

Though probably exceptional, according to the RFC +all *can be* 
restrictive...

https://tools.ietf.org/html/rfc7208

A.4.  Multiple Requirements Example

   Say that your sender policy requires both that the IP address is
   within a certain range and that the reverse DNS for the IP matches.
   This can be done several ways, including the following:

   example.com.   SPF  ( "v=spf1 "
 "-include:ip4._spf.%{d} "
 "-include:ptr._spf.%{d} "
 "+all" )
   ip4._spf.example.com.  SPF  "v=spf1 -ip4:192.0.2.0/24 +all"
   ptr._spf.example.com.  SPF  "v=spf1 -ptr +all"

   This example shows how the "-include" mechanism can be useful, how an
   SPF record that ends in "+all" can be very restrictive, and the use
   of De Morgan's Law.

--
b.

Re: Hints needed for spf rule

[Adam Katz]

 

On 2018-09-22 10:33 am, Kevin A. McGrail wrote: 

> On 9/22/2018 10:29 AM, Matus UHLAR - fantomas wrote:
> 
>> remove those ?'s: /^v=spf1 .*?all/ and /^v=spf1 .*+all/
> 
> Updated. I was trying to stop a greedy regex if someone was doing a
> weird spf but I am overthinking.

These SPF records are all effectively equivalent (the fourth is Sender
ID [1], we'll get to #5 later): 

v=spf1 +all
v=spf1 all
v=spf1 all 192.0.2.0/24
v=spf2.0/mfrom +all
v=spf1 1.2.3.0/1 128.4.5.0/2 192.6.7.8/3 -all

So therefore I propose regexps like /^v=spf[12].*[s+]allb/ and
/^v=spf[12].*s?allb/ (the latter should be very rare and a better
indication of a clueless admin than a spammer).

The fifth item above permits 0.0.0.0 to 223.255.255.255 and therefore
only multicast and the reserved Class E network are banned. To address
this, consider /^v=spf[12].*[0-9]/[0-7]b/. I haven't observed this sort
of workaround (yet), but it's the attackers' logical escalation in
response to this. (I conservatively chose a max mask of /7, though I
don't think there's any legitimate use of /8, even by the remaining
Class A holders [2] like AT, HP, and the US DoD--nobody _should_ have
an email network even approaching a /16 let alone a /8, though note that
Google currently includes three /16s. I'm not sure where to draw a
similar "too large" threshold for IPv6; perhaps /32?)

-Adam (still here, sometimes) 

Links:
--
[1] https://en.wikipedia.org/wiki/Sender_ID
[2]
https://en.wikipedia.org/wiki/List_of_assigned_/8_IPv4_address_blocks#List_of_assigned_/8_blocks

Re: Hints needed for spf rule

[Kevin A. McGrail]

On 9/22/2018 10:29 AM, Matus UHLAR - fantomas wrote:
>
> remove those ?'s:
>
> /^v=spf1 .*\?all/
> and
> /^v=spf1 .*\+all/ 

Updated.  I was trying to stop a greedy regex if someone was doing a
weird spf but I am overthinking.

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Hints needed for spf rule

[Matus UHLAR - fantomas]

On 9/22/2018 9:55 AM, RW wrote:

  /^v=spf1 .+(\?|\+)all$/


I believe [?+] would do the same easy to read, parse and maybe even to
process (I have no idea how perl RE optimizer works)


.+ should be .* or it wont match

  'v=spf1 +all'

I would remove the '$' as it doesn't appear do anything useful and could
prevent matches on weird spf records.

It may be worth splitting them into two rules for '?' and '+', there's
no dns overhead and they seem like significantly different cases.


hypotetically - masschecks should prove that.

On 22.09.18 09:57, Kevin A. McGrail wrote:

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY -
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL _SENDERDOMAIN_ TXT /^v=spf1 .*?\?all/
  describe JMQ_SPF_NEUTRAL SPF set to ?all
  score    JMQ_SPF_NEUTRAL 0.5

  askdns   JMQ_SPF_ALL _SENDERDOMAIN_ TXT /^v=spf1 .*?\+all/
  describe JMQ_SPF_ALL SPF set to +all!
  score    JMQ_SPF_ALL 0.5
endif


remove those ?'s:

/^v=spf1 .*\?all/
and
/^v=spf1 .*\+all/

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows found: (R)emove, (E)rase, (D)elete

Re: Hints needed for spf rule

[Kevin A. McGrail]

On 9/22/2018 9:55 AM, RW wrote:
> On Sat, 22 Sep 2018 08:48:43 -0400
> Kevin A. McGrail wrote:
>
>> On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
>>> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY  
 ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
  score    JMQ_SPF_NEUTRAL_ALL 0.5
 endif  
>>> do you not check for "+all" by a reason?  
>> Would you believe I thought we were?  I misread the regex like a year
>> ago.  Updating now.
>   /^v=spf1 .+(\?|\+)all$/
>
> .+ should be .* or it wont match
>
>   'v=spf1 +all'
>
> I would remove the '$' as it doesn't appear do anything useful and could
> prevent matches on weird spf records.
>
> It may be worth splitting them into two rules for '?' and '+', there's
> no dns overhead and they seem like significantly different cases.
>
# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY -
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL _SENDERDOMAIN_ TXT /^v=spf1 .*?\?all/
  describe JMQ_SPF_NEUTRAL SPF set to ?all
  score    JMQ_SPF_NEUTRAL 0.5

  askdns   JMQ_SPF_ALL _SENDERDOMAIN_ TXT /^v=spf1 .*?\+all/
  describe JMQ_SPF_ALL SPF set to +all!
  score    JMQ_SPF_ALL 0.5
endif

Better?

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Hints needed for spf rule

[RW]

On Sat, 22 Sep 2018 08:48:43 -0400
Kevin A. McGrail wrote:

> On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
> > # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY  
> >> ifplugin Mail::SpamAssassin::Plugin::AskDNS
> >>  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
> >>  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
> >>  score    JMQ_SPF_NEUTRAL_ALL 0.5
> >> endif  
> >
> > do you not check for "+all" by a reason?  
> 
> Would you believe I thought we were?  I misread the regex like a year
> ago.  Updating now.

  /^v=spf1 .+(\?|\+)all$/

.+ should be .* or it wont match

  'v=spf1 +all'

I would remove the '$' as it doesn't appear do anything useful and could
prevent matches on weird spf records.

It may be worth splitting them into two rules for '?' and '+', there's
no dns overhead and they seem like significantly different cases.

Re: Hints needed for spf rule

[Kevin A. McGrail]

On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
>> ifplugin Mail::SpamAssassin::Plugin::AskDNS
>>  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
>>  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
>>  score    JMQ_SPF_NEUTRAL_ALL 0.5
>> endif
>
> do you not check for "+all" by a reason?

Would you believe I thought we were?  I misread the regex like a year
ago.  Updating now.

Re: Hints needed for spf rule

[Matus UHLAR - fantomas]

On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis  wrote:

I noticed that Google servers started blocking emails with "suspicious spf
records" like for example:
"v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".

Any idea on how to write a rule to catch something like that ?


On 18.09.18 13:01, Kevin A. McGrail wrote:

It's in KAM.cf, I believe:

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
 askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
 describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
 scoreJMQ_SPF_NEUTRAL_ALL 0.5
endif


do you not check for "+all" by a reason?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Posli tento mail 100 svojim znamim - nech vidia aky si idiot
Send this email to 100 your friends - let them see what an idiot you are

Re: Hints needed for spf rule

[Giovanni Bechis]

I forgot about KAM.cf, just grepped in rulesrc and found nothing.
 Giovanni

On 09/18/18 19:01, Kevin A. McGrail wrote:
> It's in KAM.cf, I believe:
> 
> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
> ifplugin Mail::SpamAssassin::Plugin::AskDNS
>   askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
>   describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
>   score    JMQ_SPF_NEUTRAL_ALL 0.5
> endif
> 
> --
> Kevin A. McGrail
> VP Fundraising, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171
> 
> 
> On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis  > wrote:
> 
> Hi,
> I noticed that Google servers started blocking emails with "suspicious 
> spf records" like for example:
> "v=spf1 include:musvc.com  include:turbo-smtp.com 
>  mx a +all".
> 
> Any idea on how to write a rule to catch something like that ?
> 
>  Thanks & Cheers
>   Giovanni
> 

Re: Hints needed for spf rule

[Kevin A. McGrail]

It's in KAM.cf, I believe:

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
  scoreJMQ_SPF_NEUTRAL_ALL 0.5
endif

--
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis  wrote:

> Hi,
> I noticed that Google servers started blocking emails with "suspicious spf
> records" like for example:
> "v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".
>
> Any idea on how to write a rule to catch something like that ?
>
>  Thanks & Cheers
>   Giovanni
>

Hints needed for spf rule

[Giovanni Bechis]

Hi,
I noticed that Google servers started blocking emails with "suspicious spf 
records" like for example:
"v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".

Any idea on how to write a rule to catch something like that ?

 Thanks & Cheers
  Giovanni