Re: Bayes not auto-learning?

[Amir Caspi]

On Feb 23, 2018, at 11:47 PM, David B Funk  wrote:
> It could have 20 points from a whole bunch of body rules but if it only hit 2
> points via header rules it still will not auto-learn.

Gotcha. The spam in question that triggered this hit a lot of rules, but hard 
for me to tell on cursory inspection whether it satisfies sufficient header and 
body points.  But it LOOKS like there should be at least 3 points from header 
(MISSING_HEADERS, FREEMAIL_FORGED_REPLYTO, among others) and certainly 3 body 
(MONEY_FRAUD_3 at the very least).  The actual spam report is this:

*  0.0 FSL_CTYPE_WIN1251 Content-Type only seen in 419 spam
*  0.0 NSL_RCVD_FROM_USER Received from User
*  1.0 MISSING_HEADERS Missing To: header
*  0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
*  [score: 0.5004]
*  1.1 DCC_CHECK Detected as bulk mail by DCC (dcc-servers.net)
*  0.0 FROM_MISSP_MSFT From misspaced + supposed Microsoft tool
*  0.0 FSL_NEW_HELO_USER Spam's using Helo and User
*  2.6 MSOE_MID_WRONG_CASE No description available.
*  0.0 FROM_MISSP_USER From misspaced, from "User"
*  1.0 RDNS_DYNAMIC Delivered to internal network by host with
*  dynamic-looking rDNS
*  0.0 LOTS_OF_MONEY Huge... sums of money
*  0.0 FROM_MISSP_XPRIO Misspaced FROM + X-Priority
*  1.6 REPLYTO_WITHOUT_TO_CC No description available.
*  0.0 AXB_XMAILER_MIMEOLE_OL_024C2 Yet another X header trait
*  0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay
*  0.0 FSL_BULK_SIG Bulk signature with no Unsubscribe
*  2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From
*  1.0 FREEMAIL_REPLYTO Reply-To/From or Reply-To/body contain different
*  freemails
*  0.0 TO_NO_BRKTS_FROM_MSSP Multiple header formatting problems
*  1.9 FORGED_MUA_OUTLOOK Forged mail pretending to be from MS Outlook
*  1.6 TO_NO_BRKTS_DYNIP To: lacks brackets and dynamic rDNS
*  0.0 FILL_THIS_FORM Fill in a form with personal information
*  2.0 TO_NO_BRKTS_MSFT To: lacks brackets and supposed Microsoft tool
*  2.0 FILL_THIS_FORM_LONG Fill in a form with personal information
*  3.1 FROM_MISSP_FREEMAIL From misspaced + freemail provider
*  3.0 MONEY_FRAUD_3 Lots of money and several fraud phrases

But, it still didn't autolearn.

(I can post the entire spample if the above seems like it should have 
autolearned.)

> Another possible factor, if you have "bayes_auto_learn_on_error" enabled, 
> then autolearn will be skipped if Bayes already agrees with the condition of 
> the message. IE: if the message is already classifed as BAYES_99 then it 
> won't bother auto-learning it as yet another high-ranking spam.

I do not have that enabled.  Also, as you can see from above, this hit BAYES_50.

Does the above provide an indication as to why it didn't autolearn?

Thanks!

--- Amir

Re: Bayes not auto-learning?

[Ian Zimmerman]

On 2018-02-23 22:32, Amir Caspi wrote:

> So, I've been trying to tweak my setup and noticed that VERY few of my
> emails are being autolearned as spam, even when their spam threshold
> is far above the autolearn threshold.  The threshold is set to 12; I
> just saw a spam with score >25 not being autolearned.

Sigh.  This really is a FAQ, and I did ask it myself (maybe more than
once).

Read the fine documentation.  Shortned: the score that is compared to
the threshold for autolearning is _not_ the normal score that determines
spam/ham.

Despite the fact that is is documented, I find the algorithm to be too
opaque to feel in control.

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.

Re: Bayes not auto-learning?

[David B Funk]

On Fri, 23 Feb 2018, Amir Caspi wrote:


Hi all,

So, I've been trying to tweak my setup and noticed that VERY few of my 
emails are being autolearned as spam, even when their spam threshold is far above 
the autolearn threshold.  The threshold is set to 12; I just saw a spam with score 
>25 not being autolearned.

Are there rules that prevent autolearning?  If so, why?  If a spam 
scores really high because it hits (let's say) 10 or more rules, but just one 
of those rules is enough to prevent autolearning, that seems overly 
restrictive, no?

For example, for one of my users, out of about 650 spams received in 
the last month, only 10 have been autolearned.  For another user, only 12 of 
nearly 1400.  That seems like a very low percentage, and clearly some 
high-scoring spams are not being auto-learned.

Any explanation is appreciated!

Thanks!

--- Amir


If you read the spamassassin documentation about Bayes auto-learning you will 
see that there are several conditions that must be satisfied.


For example, there are some types of rules which aren't considered at all when 
computing the auto-learning threshold score (such as white/black list scores or 
rules tagged with the noautolearn tflag or the actual Bayes score itself).


Of the types of rules which are allowed, at least 3 of those points must come 
from header type rules and at least 3 of those points must come from body type 
rules.


So a spam can have 100 points from a blacklist and not auto-learn.

It could have 20 points from a whole bunch of body rules but if it only hit 2
points via header rules it still will not auto-learn.

Another possible factor, if you have "bayes_auto_learn_on_error" enabled, then 
autolearn will be skipped if Bayes already agrees with the condition of the 
message. IE: if the message is already classifed as BAYES_99 then it won't 
bother auto-learning it as yet another high-ranking spam.


What I usually see in auto-learned spam is that they hit a number of network RBL 
rules (spamhaus, SORBS, etc) and a number of body rules such as RAZOR, URIBLS, 
etc.



--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Bayes not auto-learning?

[Amir Caspi]

Hi all,

So, I've been trying to tweak my setup and noticed that VERY few of my 
emails are being autolearned as spam, even when their spam threshold is far 
above the autolearn threshold.  The threshold is set to 12; I just saw a spam 
with score >25 not being autolearned.

Are there rules that prevent autolearning?  If so, why?  If a spam 
scores really high because it hits (let's say) 10 or more rules, but just one 
of those rules is enough to prevent autolearning, that seems overly 
restrictive, no?

For example, for one of my users, out of about 650 spams received in 
the last month, only 10 have been autolearned.  For another user, only 12 of 
nearly 1400.  That seems like a very low percentage, and clearly some 
high-scoring spams are not being auto-learned.

Any explanation is appreciated!

Thanks!

--- Amir

Re: Custom rule don't match without empty line before the string!

[@lbutlr]

On 2018-02-23 (02:15 MST), saqariden  wrote:
> 
> our mailing service is not for external use, So the users are not supposed to 
> send or receive B64 encoded mails.

I've never seen anyone *intentionally* sent base64 mails (I mean, people, not 
spammers). That is a decision made by the MUA. Sounds like you may be trying to 
solve the wrong problem.

-- 
I've got a sonic screwdriver!
Yeah? I've got a chair!
 ...
Chairs *are* useful.

Re: Whitelist IP for SBL check

[shridhar shetty]

Yes, I missed it.

On Sat, Feb 24, 2018 at 12:49 AM, RW  wrote:

> On Sat, 24 Feb 2018 00:36:56 +0530
> shridhar shetty wrote:
>
>
> > 'Your local "fix" won't stop a URIBL_SBL hit at the other, rcpt's
> > end.' In such case we relay our mails through an external server
> > which has clean reputation. That way our mails are delivered to the
> > recipient.
>
> That will help with RCVD_IN_SBL, but URIBL_SBL is based on URI domains.
>

Re: Whitelist IP for SBL check

[RW]

On Sat, 24 Feb 2018 00:36:56 +0530
shridhar shetty wrote:


> 'Your local "fix" won't stop a URIBL_SBL hit at the other, rcpt's
> end.' In such case we relay our mails through an external server
> which has clean reputation. That way our mails are delivered to the
> recipient.

That will help with RCVD_IN_SBL, but URIBL_SBL is based on URI domains.

Re: Whitelist IP for SBL check

[shridhar shetty]

Hello Axb,

Below are the response to your queries.

Why not fix the SBL issue instead of trying to work around it?
Fixing the SBL issue is the first thing we do. But it takes some time so we
do not want our outbound mail service to be affected due to this.

'Your local "fix" won't stop a URIBL_SBL hit at the other, rcpt's end.'
In such case we relay our mails through an external server which has clean
reputation. That way our mails are delivered to the recipient.

Give us the SBL number and we may be able to help you out.
Do you mean the response code from zen.spamhaus? the response code is
127.0.0.2



On Fri, Feb 23, 2018 at 10:35 PM, Axb  wrote:

>
> On 02/23/2018 03:26 PM, shridhar shetty wrote:
>
>> Hello,
>>
>> In our infra we use spamassassin to scan our **outgoing** mails too. This
>> is to prevent spammers using our infra to send mails and get our IP's
>> blacklisted. We perform various DNSBL tests on the mail body.
>>
>> One of our IPs got listed in Spamhaus SBL for some reason, so now our
>> outgoing mails are getting detected as spam if the email body contains our
>> local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).
>> We have hundreds of domainnames mapped to an single IP.
>>
>
>
> Why not fix the SBL issue instead of trying to work around it?
> Your local "fix" won't stop a URIBL_SBL hit at the other, rcpt's end.
> Give us the SBL number and we may be able to help you out.
>
>
>

RCVD_OFFICE365 was ENCRYPTED_MESSAGE rule

[David Jones]

On 02/22/2018 06:10 PM, John Hardin wrote:

I was just referring to the OFFICE365 subrule, as there isn't one like 
that yet - hotmail, sure, outlook, sure, but not office365. We should 
have added that back when O365 started up.




I had already added a generic rule for this in my sandbox so you can see 
it at http://ruleqa.spamassassin.org now:


__RCVD_OFFICE365

Hotmail and Office 365 tenants come from this so it's not a direct 
relationship to spam but can be used in meta rules to amplify other 
spammy rules.


__RCVD_OFFICE365_PROXY

This is interesting because often when there is a true compromised 
account on O365, spammers will use authenticated SMTP to blast out spam 
not using the Outlook Web interface or an Outlook client.  This will hit 
on normal mail clients like Thunderbird or Apple Mail so it too is not a 
direct indication of spam.


My local __RCVD_OFFICE365 rule that combines sources of freemail like 
O365 in with FREEMAIL_* rules is already working well the past 24 hours. 
 I am offsetting my BAYES_00 score of -3.2 by adding back 2.0 when for 
email from O365.  It has helped to blocked a bogus file sharing email 
using a URL shortener that would have scored just below the MailScanner 
default threshold of 6.0.


--
David Jones

Re: Whitelist IP for SBL check

[Markus Clardy]

Considering the issue, couldn't you in theory just add "uridnsbl_skip_domain
ip.on.blk.lst"?

I mean, according to URIBL_SBL, it would be if the IP itself is on the
blacklist, so wouldn't skipping the "domain" of a specific IP skip
detection?

On Fri, Feb 23, 2018 at 4:55 PM, David Jones  wrote:

> On 02/23/2018 10:46 AM, Axb wrote:
>
>> On 02/23/2018 04:33 PM, David Jones wrote:
>>
>>> On 02/23/2018 08:26 AM, shridhar shetty wrote:
>>>
 Hello,

 In our infra we use spamassassin to scan our **outgoing** mails too.
 This is to prevent spammers using our infra to send mails and get our IP's
 blacklisted. We perform various DNSBL tests on the mail body.


>>> We also scan outbound aggressively to keep our own IPs clean.  I monitor
>>> for our own IPs getting listed in major RBLs every 15 minutes and hourly I
>>> have a script that checks my own IPs in all RBLs listed at
>>> http://multirbl.valli.org/.  You need to make sure you have a good
>>> abuse@ contact setup for your IP ranges based on a WHOIS lookup of the
>>> IPs.  You must setup feedback loops with all of the major platforms out
>>> there like Yahoo, AOL, Comcast, etc.
>>>
>>> We send out millions of spammy looking emails every week from from
>>> student management systems that don't have an opt-out method to lots of
>>> parents on freemail platforms.  We very rarely get listed on RBLs and have
>>> excellent delivery rates mainly because of compromised account detection
>>> and blocking of outbound mail from the single sender quickly when this is
>>> triggered.  Most sane RBLs will allow for a little junk outbound as long as
>>> you stop it quickly because compromised accounts happen.
>>>
>>>
>>> One of our IPs got listed in Spamhaus SBL for some reason, so now our
 outgoing mails are getting detected as spam if the email body contains our
 local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).
 We have hundreds of domainnames mapped to an single IP.

 Is there a way to exclude local IP from DNSBL checks. For eg: if there
 is a local domainname xyz.org  present in the mail
 body, then spamassassin should not mark it as spam even if A or NS record
 for xyz.org  is listed in SBL.


>>> Setup a quick meta rule that subtracts the same points that the local IP
>>> on Spamhaus adds until you can find a better way to handle this.
>>>
>>> header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
>>> meta SPAMHAUS_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && RCVD_IN_XBL
>>> score SPAMHAUS_LOCAL_IP_OFFSET -1.0
>>>
>>> You will need to adjust the header rule to match your Received header
>>> format of your particular MTA and also match the actual Spamhaus rule that
>>> is getting hit.  I just guessed it was RCVD_IN_XBL.
>>>
>>>
>> you are aware that your recommendation doesn't apply to a
>> uridnssub  URIBL_SBLzen.spamhaus.org.   A   127.0.0.2
>> hit ?
>>
>>
>>
>>
> I was in a hurry, sorry.  My last paragraph had a disclaimer that 2 things
> would need to be adjusted.  Here is 1 of them corrected so the OP will only
> have to make sure the header rule matches his MTA's format:
>
> header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
> meta URIBL_SBL_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && URIBL_SBL
> score URIBL_SBL_LOCAL_IP_OFFSET -1.0
>
> --
> David Jones
>



-- 
 - Markus

Re: Whitelist IP for SBL check

[Axb]

On 02/23/2018 03:26 PM, shridhar shetty wrote:

Hello,

In our infra we use spamassassin to scan our **outgoing** mails too. This
is to prevent spammers using our infra to send mails and get our IP's
blacklisted. We perform various DNSBL tests on the mail body.

One of our IPs got listed in Spamhaus SBL for some reason, so now our
outgoing mails are getting detected as spam if the email body contains our
local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).
We have hundreds of domainnames mapped to an single IP.



Why not fix the SBL issue instead of trying to work around it?
Your local "fix" won't stop a URIBL_SBL hit at the other, rcpt's end.
Give us the SBL number and we may be able to help you out.

Re: Whitelist IP for SBL check

[David Jones]

On 02/23/2018 10:46 AM, Axb wrote:

On 02/23/2018 04:33 PM, David Jones wrote:

On 02/23/2018 08:26 AM, shridhar shetty wrote:

Hello,

In our infra we use spamassassin to scan our **outgoing** mails too. 
This is to prevent spammers using our infra to send mails and get our 
IP's blacklisted. We perform various DNSBL tests on the mail body.




We also scan outbound aggressively to keep our own IPs clean.  I 
monitor for our own IPs getting listed in major RBLs every 15 minutes 
and hourly I have a script that checks my own IPs in all RBLs listed 
at http://multirbl.valli.org/.  You need to make sure you have a good 
abuse@ contact setup for your IP ranges based on a WHOIS lookup of the 
IPs.  You must setup feedback loops with all of the major platforms 
out there like Yahoo, AOL, Comcast, etc.


We send out millions of spammy looking emails every week from from 
student management systems that don't have an opt-out method to lots 
of parents on freemail platforms.  We very rarely get listed on RBLs 
and have excellent delivery rates mainly because of compromised 
account detection and blocking of outbound mail from the single sender 
quickly when this is triggered.  Most sane RBLs will allow for a 
little junk outbound as long as you stop it quickly because 
compromised accounts happen.



One of our IPs got listed in Spamhaus SBL for some reason, so now our 
outgoing mails are getting detected as spam if the email body 
contains our local domainname whose IP is listed in SBL(hitting 
URIBL_SBL rule).

We have hundreds of domainnames mapped to an single IP.

Is there a way to exclude local IP from DNSBL checks. For eg: if 
there is a local domainname xyz.org  present in the 
mail body, then spamassassin should not mark it as spam even if A or 
NS record for xyz.org  is listed in SBL.




Setup a quick meta rule that subtracts the same points that the local 
IP on Spamhaus adds until you can find a better way to handle this.


header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
meta SPAMHAUS_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && RCVD_IN_XBL
score SPAMHAUS_LOCAL_IP_OFFSET -1.0

You will need to adjust the header rule to match your Received header 
format of your particular MTA and also match the actual Spamhaus rule 
that is getting hit.  I just guessed it was RCVD_IN_XBL.




you are aware that your recommendation doesn't apply to a
uridnssub  URIBL_SBL    zen.spamhaus.org.   A   127.0.0.2
hit ?





I was in a hurry, sorry.  My last paragraph had a disclaimer that 2 
things would need to be adjusted.  Here is 1 of them corrected so the OP 
will only have to make sure the header rule matches his MTA's format:


header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
meta URIBL_SBL_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && URIBL_SBL
score URIBL_SBL_LOCAL_IP_OFFSET -1.0

--
David Jones

Re: Whitelist IP for SBL check

[Axb]

On 02/23/2018 04:33 PM, David Jones wrote:

On 02/23/2018 08:26 AM, shridhar shetty wrote:

Hello,

In our infra we use spamassassin to scan our **outgoing** mails too. 
This is to prevent spammers using our infra to send mails and get our 
IP's blacklisted. We perform various DNSBL tests on the mail body.




We also scan outbound aggressively to keep our own IPs clean.  I monitor 
for our own IPs getting listed in major RBLs every 15 minutes and hourly 
I have a script that checks my own IPs in all RBLs listed at 
http://multirbl.valli.org/.  You need to make sure you have a good 
abuse@ contact setup for your IP ranges based on a WHOIS lookup of the 
IPs.  You must setup feedback loops with all of the major platforms out 
there like Yahoo, AOL, Comcast, etc.


We send out millions of spammy looking emails every week from from 
student management systems that don't have an opt-out method to lots of 
parents on freemail platforms.  We very rarely get listed on RBLs and 
have excellent delivery rates mainly because of compromised account 
detection and blocking of outbound mail from the single sender quickly 
when this is triggered.  Most sane RBLs will allow for a little junk 
outbound as long as you stop it quickly because compromised accounts 
happen.



One of our IPs got listed in Spamhaus SBL for some reason, so now our 
outgoing mails are getting detected as spam if the email body contains 
our local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).

We have hundreds of domainnames mapped to an single IP.

Is there a way to exclude local IP from DNSBL checks. For eg: if there 
is a local domainname xyz.org  present in the mail 
body, then spamassassin should not mark it as spam even if A or NS 
record for xyz.org  is listed in SBL.




Setup a quick meta rule that subtracts the same points that the local IP 
on Spamhaus adds until you can find a better way to handle this.


header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
meta SPAMHAUS_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && RCVD_IN_XBL
score SPAMHAUS_LOCAL_IP_OFFSET -1.0

You will need to adjust the header rule to match your Received header 
format of your particular MTA and also match the actual Spamhaus rule 
that is getting hit.  I just guessed it was RCVD_IN_XBL.




you are aware that your recommendation doesn't apply to a
uridnssub  URIBL_SBLzen.spamhaus.org.   A   127.0.0.2
hit ?

Re: Whitelist IP for SBL check

[David Jones]

On 02/23/2018 08:26 AM, shridhar shetty wrote:

Hello,

In our infra we use spamassassin to scan our **outgoing** mails too. 
This is to prevent spammers using our infra to send mails and get our 
IP's blacklisted. We perform various DNSBL tests on the mail body.




We also scan outbound aggressively to keep our own IPs clean.  I monitor 
for our own IPs getting listed in major RBLs every 15 minutes and hourly 
I have a script that checks my own IPs in all RBLs listed at 
http://multirbl.valli.org/.  You need to make sure you have a good 
abuse@ contact setup for your IP ranges based on a WHOIS lookup of the 
IPs.  You must setup feedback loops with all of the major platforms out 
there like Yahoo, AOL, Comcast, etc.


We send out millions of spammy looking emails every week from from 
student management systems that don't have an opt-out method to lots of 
parents on freemail platforms.  We very rarely get listed on RBLs and 
have excellent delivery rates mainly because of compromised account 
detection and blocking of outbound mail from the single sender quickly 
when this is triggered.  Most sane RBLs will allow for a little junk 
outbound as long as you stop it quickly because compromised accounts happen.



One of our IPs got listed in Spamhaus SBL for some reason, so now our 
outgoing mails are getting detected as spam if the email body contains 
our local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).

We have hundreds of domainnames mapped to an single IP.

Is there a way to exclude local IP from DNSBL checks. For eg: if there 
is a local domainname xyz.org  present in the mail body, 
then spamassassin should not mark it as spam even if A or NS record for 
xyz.org  is listed in SBL.




Setup a quick meta rule that subtracts the same points that the local IP 
on Spamhaus adds until you can find a better way to handle this.


header __RCVD_LOCAL_IP Received =~ /\[xx\.xx\.xx\.xx\]/
meta SPAMHAUS_LOCAL_IP_OFFSET __RCVD_LOCAL_IP && RCVD_IN_XBL
score SPAMHAUS_LOCAL_IP_OFFSET -1.0

You will need to adjust the header rule to match your Received header 
format of your particular MTA and also match the actual Spamhaus rule 
that is getting hit.  I just guessed it was RCVD_IN_XBL.


--
David Jones

Whitelist IP for SBL check

[shridhar shetty]

Hello,

In our infra we use spamassassin to scan our **outgoing** mails too. This
is to prevent spammers using our infra to send mails and get our IP's
blacklisted. We perform various DNSBL tests on the mail body.

One of our IPs got listed in Spamhaus SBL for some reason, so now our
outgoing mails are getting detected as spam if the email body contains our
local domainname whose IP is listed in SBL(hitting URIBL_SBL rule).
We have hundreds of domainnames mapped to an single IP.

Is there a way to exclude local IP from DNSBL checks. For eg: if there is a
local domainname xyz.org present in the mail body, then spamassassin should
not mark it as spam even if A or NS record for xyz.org is listed in SBL.

I tried the following things which did not work.
1. Adding the local IP in "trusted_network" and "internal_network" in
local.cf
2. Using uridnsbl_skip_domain "" directives in local.cf works.
But adding hundreds of local domains doesn't seem like a solution.

Some details:
SpamAssassin Server version 3.4.1
Spamassassin rule which matched:

uridnssub   URIBL_SBLzen.spamhaus.org.   A   127.0.0.2
bodyURIBL_SBLeval:check_uridnsbl('URIBL_SBL')
describeURIBL_SBLContains an URL's NS IP listed in the
Spamhaus SBL blocklist
tflags  URIBL_SBLnet
reuse   URIBL_SBL



Thanks,
Shridhar

Re: oxy/diabetes/cbd/big pharma spam

[Joseph Brennan]

>> headerBOGUS_MIME_VERSION

So the secret is out. We are blocking as many as 40,000 a day. I
tested it for a few days, at a million messages a day, and nothing
else matches that error. It's a killer rule here.

The spam itself is very low scoring otherwise. Score for /shark.tank/i
matches a lot of this spam but not all. The domain names used are
domains of small companies that have nothing to do with the spam. The
spammer has been evading spamhaus honeypots remarkably well.

The source is not a botnet of end user hosts. I don't know what to
call this method. The spammer gets use of about two dozen servers from
a hosting company and blasts from them for a few days, and then jumps
to another hosting company. Blocking by IP is not effective for long
although the IP blocks that have been used are probably a nice
collection of easily abused providers. Since January 23 we have seen
hosts in these blocks, below. Yesterday was 23.95.197 and 104.234.218.

Joseph Brennan
Columbia University I T



23.94.138
23.94.165
23.95.197
23.95.200
45.65.16
46.102.117
46.166.186
63.143.38
64.186.14
66.70.254
67.214.188
69.195.136
74.63.251
74.80.147
76.164.198
84.247.12
85.17.31
104.160.179
104.234.218
107.175
128.201.32
128.201.33
128.201.34
149.56.7
158.69.128
173.198.192
173.199.178
192.140.20
192.140.21
192.140.23
198.23.197
209.240.101
209.240.99
216.245.210

Re: ENCRYPTED_MESSAGE rule

[RW]

On Fri, 23 Feb 2018 07:18:52 -0600
David Jones wrote:

> On 02/23/2018 06:29 AM, RW wrote:
> > On Thu, 22 Feb 2018 19:33:29 -0700
> > @lbutlr wrote:
> >   
> >> On 2018-02-22 (17:39 MST), RW  wrote:  
> >>>
> >>> Is it genuinely encrypted though? I'm wondering if it's just
> >>> base64 encoded, and possibly signed.  
> >>
> >> application/pkcs7-mime is S/MIME  
> > 
> > I know, but does that mean it's necessarily encrypted and not simply
> > signed?
> >   
> 
> Outlook Web says across the top of the message:
> 
> This message has a digital signature, but it wasn't verified because
> the S/MIME control isn't currently supported for your browser or
> platform.
> 
> Outlook client on a Mac says it was an encrypted email.
> 
> https://pastebin.com/Kf9KJKyh
> 

It's just signed, if you change the type to text/plain you can see the
raw mime message.


It has:

Content-Type: application/pkcs7-mime; smime-type=signed-data;
name="smime.p7m"

From a quick look at rfc5751 it looks like a purely encrypted email
would have "smime-type=enveloped-data", but I doubt that's common. With
a signed and encrypted email the two types are nested in either order.

It comes  down to usage, if the norm is for emails to be signed and
then encrypted, then this sort of email can easily be excluded from
ENCRYPTED_MESSAGE, but the other way around requires support for
S/MIME. 

Re: ENCRYPTED_MESSAGE rule

[David Jones]

On 02/23/2018 06:29 AM, RW wrote:

On Thu, 22 Feb 2018 19:33:29 -0700
@lbutlr wrote:


On 2018-02-22 (17:39 MST), RW  wrote:


Is it genuinely encrypted though? I'm wondering if it's just base64
encoded, and possibly signed.


application/pkcs7-mime is S/MIME


I know, but does that mean it's necessarily encrypted and not simply
signed?



Outlook Web says across the top of the message:

This message has a digital signature, but it wasn't verified because the 
S/MIME control isn't currently supported for your browser or platform.


Outlook client on a Mac says it was an encrypted email.

https://pastebin.com/Kf9KJKyh

The spample above now has this sender as blacklisted so you may want to 
run it through your own spamassassin configs to see what it hits.


--
David Jones

Re: ENCRYPTED_MESSAGE rule

[RW]

On Thu, 22 Feb 2018 19:33:29 -0700
@lbutlr wrote:

> On 2018-02-22 (17:39 MST), RW  wrote:
> > 
> > Is it genuinely encrypted though? I'm wondering if it's just base64
> > encoded, and possibly signed.  
> 
> application/pkcs7-mime is S/MIME

I know, but does that mean it's necessarily encrypted and not simply
signed?

Re: Custom rule don't match without empty line before the string!

[saqariden]

On 22/02/2018 17:48, RW wrote:

On Thu, 22 Feb 2018 10:35:48 -0600 (CST)
David B Funk wrote:


On Thu, 22 Feb 2018, RW wrote:


On Thu, 22 Feb 2018 15:54:45 +0100
saqariden wrote:
  

Hello guys,

I have the following SA rule which is supposed to block base64
encoded mails:

This may be dangerous.  If someone doesn't wish to use 8bit text
then base64 encoding of UTF-8 is a sensible choice; QP is very
inefficient unless the text is almost completely ASCII.

  

bodyEN_BASE64_B/(Content-Transfer-Encoding:
base64\sContent-Type: text\/(plain|html);
charset="?utf-8"?)|(mimeheader: text\/(plain|html);
charset="?utf-8"?\sContent-Transfer-Encoding: base64)/i
describe EN_BASE64_BTEXT OR HTML B64 ENCODED
score EN_BASE64_B5

this is the mail that i want to stop:


the rule don't match for this mail, but it match when i had an
empty line like this:
..
How can i do to match the both, with the empty line and without
it?

body rules check only the text that's visible in a mail client
(including the subject text). This rule only works at all if you
make the mime unparsable.

For mime you need "full" instead of "body". You then need an
explicit \n between lines.

I agree with RW about the risk of FPs from that approach,
particularly if you have international correspondents.

However if you really want to do that, you need to use the
"mimeheader" kind of rule. It works like a regular message 'header'
type of rule but processes mime headers within the message contents.

For example, to catch messages with a particular mime attachment file
name I have a rule:

mimeheader L_BANK_PHISH1Content-Disposition =~ m!attachment;
filename="[\w\s\d._-]{1,30}verification\.html?"$!i



mimeheader rules wont work in this case because you need to
check both Content-Type and Content-Transfer-Encoding in the same  mime
section.




Thank you for the answers, it help us so much.
our mailing service is not for external use, So the users are not 
supposed to send or receive B64 encoded mails.
There is no other solution except "full"? I avoid it because it 
requires additional resources. (1M mails/day to scan)


Regards.