Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB

[John Hardin]

On Tue, 30 May 2017, Robert Kudyba wrote:


I note that message hit BAYES_00. If content like that is getting a
"strong ham" Bayes score, you should review your training processes and
Bayes corpora - you *do* keep copies of messages you train Bayes with,
right? :)


Yes just re-synced.


Did you do any review before re-training? Re-training with 
misclassifications in the corpora will not correct the problem.



But: fixing your Bayes and getting a non-forwarding DNS server for your
mail system so that you're not hitting RBL query limits are the biggest
things you need to do to address this.


It’s enabled and looks like it’s working based on this and that use_bayes 1 in 
local.cf
sa-learn --dump magic
0.000  0  3  0  non-token data: bayes db version
0.000  0688  0  non-token data: nspam
0.000  0  80012  0  non-token data: nham


That seems somewhat out-of-balance, and might lead to FNs due to Bayes. 
You should try to get more spam to train.



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  People think they're trading chaos for order [by ceding more and
  more power to the Government], but they're just trading normal
  human evil for the really dangerous organized kind of evil, the
  kind that simply does not give a shit. Only bureaucrats can give
  you true evil. -- Larry Correia
---
 7 days until the 73rd anniversary of D-Day

Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB

[Robert Kudyba]

> For the past few days lots of missed spam has been getting through, running
>>> SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with
>>> URIBL_RHS_DOB, i.e.,  domains registered in the last five days. Since we
>>> are not running our own DNS server (yet--need permission from our CISO)
>>> URIBL_BLOCKED is also being triggered. Is there a way to update this?
> 
>> Update what how?

You answered below…thanks.

> 
>> I note that message hit BAYES_00. If content like that is getting a 
>> "strong ham" Bayes score, you should review your training processes and 
>> Bayes corpora - you *do* keep copies of messages you train Bayes with, 
>> right? :)

Yes just re-synced.


> If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score 
>> of URIBL_RHS_DOB in your local rules file.
> 
>> If you'd prefer a more-focused solution, use a meta rule; perhaps:
> 
>>meta  LCL_DOB_FROM_INFO   __FROM_DOM_INFO && URIBL_RHS_DOB
>>score LCL_DOB_FROM_INFO   2.500  # or whatever you're comfortable with


Great trying this now.
> 
>> But: fixing your Bayes and getting a non-forwarding DNS server for your 
>> mail system so that you're not hitting RBL query limits are the biggest 
>> things you need to do to address this.

It’s enabled and looks like it’s working based on this and that use_bayes 1 in 
local.cf
sa-learn --dump magic
0.000  0  3  0  non-token data: bayes db version
0.000  0688  0  non-token data: nspam
0.000  0  80012  0  non-token data: nham
0.000  0 164827  0  non-token data: ntokens
0.000  0 1485101489  0  non-token data: oldest atime
0.000  0 1496149547  0  non-token data: newest atime
0.000  0  0  0  non-token data: last journal sync atime
0.000  0 1496152035  0  non-token data: last expiry atime
0.000  0   11059200  0  non-token data: last expire atime delta
0.000  0  99547  0  non-token data: last expire reduction 
count

> 
>>> I have't seen an update in sa-update since 03-May-2017 01:52:05:
> 
>> Masscheck and updates are *almost* back.

Great I’ll keep an eye out.

> 
>>> Here's a typical mail header & message content:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_Rw1S7mWe=DwIFAw=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY=bpKADIzstZa5G-g1qsGBa7gWKq4zTcrA_-E0jGYOsdo=_uJa-KDGfZ2CN8vjSlDNEmfotigbWHyD9TZaKnJwzNM=
>>>  
>>> 
>>>  
> 
>> Thanks for that.


Looks like the IP is being picked up on a few RBLs now.

> 
> Do you have any RBLs setup in sendmail?  You need
> to use bb.barracudacentral.org  and 
> zen.spamhaus.org 
> at a minimum.  Hopefully your DNS server situation
> can get fixed soon so you can use BLs successfully.
> 
Indeed we do plus spamcop:
FEATURE(`dnsbl', `b.barracudacentral.org', `', `"550 Mail from " 
$&{client_addr} " refused. Rejected for bad WHOIS info on IP of your SMTP 
server " in http://www.barracudacentral.org/lookups "')dnl
FEATURE(`dnsbl',`zen.spamhaus.org')dnl
FEATURE(`enhdnsbl', `bl.spamcop.net', `"Spam blocked see: 
http://spamcop.net/bl.shtml?"$&{client_addr}', `t')dnl

> If you switched to Postfix, there are many benefits
> to using Postscreen with weighted RBLs.  I have over
> 20 RBLs working together for best accuracy and low
> false positives.

We have several mailing lists and users past & present and the transition would 
be a bit painful.


> SpamAssassin is primarily going to be a content filter
> with some reputation checks.  Setup the MTA to be
> primarily reputation checks with DNS (i.e. make sure
> the sending IP has a PTR record [RDNS_NONE]) and
> RBL lookups.
> 
> The MTA should be blocking the majority of spam
> before it gets to SpamAssassin.

That’s what I thought, and we have even more filters in place, including the 
suggestion in 
https://www.autonarcosis.com/2015/10/14/vanity-top-level-domains-how-to-block-them-using-sendmail/
 

 to use the access file to block all of those vanity top level domains. I even 
have a regex to block anysubdomain.anydomain.us|info. And we have 
clamavjunofficial-sigs from extremeshok enabled.

Anything else to check?

Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB

[David Jones]

>From: John Hardin 
    
>On Mon, 29 May 2017, Robert Kudyba wrote:

>> For the past few days lots of missed spam has been getting through, running
>> SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with
>> URIBL_RHS_DOB, i.e.,  domains registered in the last five days. Since we
>> are not running our own DNS server (yet--need permission from our CISO)
>> URIBL_BLOCKED is also being triggered. Is there a way to update this?

>Update what how?

>I note that message hit BAYES_00. If content like that is getting a 
>"strong ham" Bayes score, you should review your training processes and 
>Bayes corpora - you *do* keep copies of messages you train Bayes with, 
>right? :)

>If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score 
>of URIBL_RHS_DOB in your local rules file.

>If you'd prefer a more-focused solution, use a meta rule; perhaps:

>   meta  LCL_DOB_FROM_INFO   __FROM_DOM_INFO && URIBL_RHS_DOB
>   score LCL_DOB_FROM_INFO   2.500  # or whatever you're comfortable with

>But: fixing your Bayes and getting a non-forwarding DNS server for your 
>mail system so that you're not hitting RBL query limits are the biggest 
>things you need to do to address this.

>> I have't seen an update in sa-update since 03-May-2017 01:52:05:

>Masscheck and updates are *almost* back.

>> Here's a typical mail header & message content:
>> https://pastebin.com/Rw1S7mWe

>Thanks for that.

Do you have any RBLs setup in sendmail?  You need
to use bb.barracudacentral.org and zen.spamhaus.org
at a minimum.  Hopefully your DNS server situation
can get fixed soon so you can use BLs successfully.

score.senderscore.com reputation is 0 out of 100

http://multirbl.valli.org/lookup/208.110.91.112.html

If you switched to Postfix, there are many benefits
to using Postscreen with weighted RBLs.  I have over
20 RBLs working together for best accuracy and low
false positives.

SpamAssassin is primarily going to be a content filter
with some reputation checks.  Setup the MTA to be
primarily reputation checks with DNS (i.e. make sure
the sending IP has a PTR record [RDNS_NONE]) and
RBL lookups.

The MTA should be blocking the majority of spam
before it gets to SpamAssassin.

Dave

Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB

[John Hardin]

On Mon, 29 May 2017, Robert Kudyba wrote:


For the past few days lots of missed spam has been getting through, running
SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with
URIBL_RHS_DOB, i.e.,  domains registered in the last five days. Since we
are not running our own DNS server (yet--need permission from our CISO)
URIBL_BLOCKED is also being triggered. Is there a way to update this?


Update what how?

I note that message hit BAYES_00. If content like that is getting a 
"strong ham" Bayes score, you should review your training processes and 
Bayes corpora - you *do* keep copies of messages you train Bayes with, 
right? :)


If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score 
of URIBL_RHS_DOB in your local rules file.


If you'd prefer a more-focused solution, use a meta rule; perhaps:

  meta  LCL_DOB_FROM_INFO   __FROM_DOM_INFO && URIBL_RHS_DOB
  score LCL_DOB_FROM_INFO   2.500  # or whatever you're comfortable with

But: fixing your Bayes and getting a non-forwarding DNS server for your 
mail system so that you're not hitting RBL query limits are the biggest 
things you need to do to address this.



I have't seen an update in sa-update since 03-May-2017 01:52:05:


Masscheck and updates are *almost* back.


Here's a typical mail header & message content:
https://pastebin.com/Rw1S7mWe


Thanks for that.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  USMC Rules of Gunfighting #2: Anything worth shooting
  is worth shooting twice. Ammo is cheap. Your life is expensive.
---
 Today: Memorial Day - honor those who sacrificed for our liberty

lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB

[Robert Kudyba]

For the past few days lots of missed spam has been getting through, running
SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with
URIBL_RHS_DOB, i.e.,  domains registered in the last five days. Since we
are not running our own DNS server (yet--need permission from our CISO)
URIBL_BLOCKED is also being triggered. Is there a way to update this? I
have't seen an update in sa-update since 03-May-2017 01:52:05:
SpamAssassin: Update processed successfully. Here's a typical mail header &
message content:
https://pastebin.com/Rw1S7mWe