Re: Random word spams and wiki spams

[Bill Cole]

On 7 Jul 2017, at 13:04, Alex wrote:


I'm interested in how your system would have (or currently does)
handle this email I received some days ago:
https://pastebin.com/innRFvZt

Its IP (106.186.119.240) is still not listed with spamhaus, sorbs or
hostkarma, and has an 83 rating with senderscore.


This never would have made it to SA on most systems I have recently 
managed:


1. Null sender with From & Subject both inconsistent with DSN or other 
legit null-sender mail.
2. That MIME structure is pathological. It merits a specific hard 
rejection with a derisive text part. Anything generating FPs (never seen 
one...) needs spanking.
3. Horrifically bad Received-SPF header, but I guess probably that's 
generated by something broken in *your* system, so isn't relevant.
4. Lots of example.com in headers but again, I guess that's you munging 
stuff and it's not stuff other  sites would see.
5. For my own system and some I manage, AS2516 is intrinsically suspect 
and that particular /18 can't talk to port 25 at all.



My personal SA would have rejected it because:

1. I don't trust BAYES_00 as much as masscheck because a lot of my ham 
describes or includes spam.
2. I have FROM_EXCESS_BASE64 pegged to 2, originally because it was too 
high and had FPs, now because masscheck scores it too low.
3. I have a local rule catching the same header as RCVD_DOUBLE_IP_SPAM 
catches in that one with a higher score because it has a perfect record.

4. Other proprietary local rules would add 1.7 to the score.
5. For my own system (but not most sites I have managed) any From header 
with a domain part directly under .cn scores so high that its message 
MUST be sent to an address with a special treatment (i.e. 
more_spam_to/all_spam_to or totally SA-exempt).
6. I reject at 4.5. I quarantine nothing because quarantining is an 
intrinsically bad idea. This message appears to have been quarantined, 
but should have been rejected.



It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.

I'm also interested in other solutions - are those of you with
MIMEDefang or other systems blocking these?


Some of my pre-SA blocking is in MIMEDefang, which is also what I use to 
hook in SA. If you run a milter-capable MTA and are comfortable writing 
small-scale Perl, MD is an ideal tool for hooking in SA and whatever AV 
you feel compelled to use. I have absolutely no critique of amavisd-new, 
which I gather is quite good, but I came from the Sendmail world where 
MD was dominant and I chose to stick with it when I switched to Postfix 
as my preferred MTA. If the idea of writing a little Perl disturbs you, 
MIMEDefang is probably not for you.

Re: Random word spams and wiki spams

[Alex]

Hi,

> Without that rule it might have flown below my sa-radar.
> Got some scoring on it by using this plugin:
> https://github.com/eilandert/Botnet.pm

Be careful with the botnet plugin - it's terribly out of date and very
prone to false-positives. It's just not effective anymore.

Re: Random word spams and wiki spams

[Alex]

HI,

>> __HAS_LIST_ID  exists:exists:List-Id
>
> typo ?

It also already exists:

# grep __HAS_LIST_ID *
10_hasbase.cf:header  __HAS_LIST_ID   exists:List-Id

> imho it should be exists:headername
>
>> HAS_LIST_UNSUB exists:List-Unsubscribe

So does this one:

72_active.cf:header __DOS_HAS_LIST_ID   exists:List-ID

> but check spamasassin own rules if that is not already defined, else you
> really redefine it

good call.



>
>> metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI &&
>> !__HAS_LIST_ID && !HAS_LIST_UNSUB)
>
>
> ok, renema HAS_LIST_UNSUB to match above

Re: Random word spams and wiki spams

[Alex]

Hi,

> Ummm.  Well.  I don't have any hits on that RHSBL rule in the past 2 weeks
> so maybe that is not a valid rule.  Ignore that one.  I think I will take it
> out of my ivm.cf file.
>
> To all, please don't setup these rules and flood the IVM DNS servers with
> requests.  IVM is a private RBL feed (not very expensive) so you should have
> a local rbldnsd instance with the DNS servers that the mail filtering
> servers point to serving invaluement.com authoritatively.
>
> Sorry Rob if we cause problems with your DNS servers accidentally by posting
> these rules.

Yes, apologies; the real hosts are private and designated for
individual subscribers.

sa-learn won't read db created via MSTOR

[Jerry Malcolm]

My client mail repository is in a sql db and is not an option for 
sa-learn to read directly.  That's fine.  I wrote a utility that reads 
all the mail out of the uncaught-spam folder from my db and creates an 
mbox folder using the mstor java package.  The mbox file gets created 
with no problem.  When I run sa-learn, it says 0 messages were 
examined.  The mbox folder has about 2500 spam messages in it.  I've 
seen lots of discussion on the forums about whether or not sa-learn will 
'process' a message based on whether it's processed it before, etc.  I 
understand that.   But this is the very first time I've ever tried to 
run sa-learn.   And this error implies that it is not even finding any 
messages to process.


Here's the command and response: (Win server 2008)

[C:\Program Files\JAM Software\SpamAssassin in a Box]sa-learn 
--spam --mbox --showdots c:\imaputil\temp\uncaughtspam.mstor\temp


Learned tokens from 0 message(s) (0 message(s) examined)

I've used the mstor package before and have had zero problems with it.  
So I have no reason to assume it's creating a corrupted mbox folder 
file.  The mbox folder is present and is being found (I tried renaming 
it and got a 'not found' error from sa-learn). I've opened it in an 
editor, and to the extent I can tell, it looks like an mbox file.  There 
is about a 10-15 sec time lapse while sa-learn is 'running' before it 
displays the message.  So it appears that it's reading the mbox file.  
But for some reason it thinks there are no messages inside it.


I'm at a loss right now.  Is there anyway to get additional information 
on why it thinks there are no messages in the mbox file?  I can post the 
mbox file if necessary.  If there are any debug flags that will help me 
figure out what is wrong, I can do debug as well.


Thanks.

Jerry

Re: Random word spams and wiki spams

[Benny Pedersen]

Tobi skrev den 2017-07-07 19:40:


https://pastebin.com/innRFvZt



__HAS_LIST_ID  exists:exists:List-Id


typo ?

imho it should be exists:headername


HAS_LIST_UNSUB exists:List-Unsubscribe


that would score 1.0, intended ?

if not change to __HAS_LIST_UNSUB

but check spamasassin own rules if that is not already defined, else you 
really redefine it


metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI 
&& !__HAS_LIST_ID && !HAS_LIST_UNSUB)


ok, renema HAS_LIST_UNSUB to match above

Re: [SOLVED] I'm an idiot

[Bill Cole]

On 7 Jul 2017, at 12:15, jdow wrote:

> On the other hand, FireFox reports:
> This site can’t be reached
>
> updates.spamassassin.org’s server DNS address could not be found.

Which is simultaneously:

1. True
2. Normal
3. Neither a cause nor symptom of any operational problem.

Re: Random word spams and wiki spams

[Tobi]

Am 07.07.2017 um 19:04 schrieb Alex:
>
> I'm interested in how your system would have (or currently does)
> handle this email I received some days ago:
> https://pastebin.com/innRFvZt
>
that one triggers one of my redpill meta rules and scores at 24.1 :-)

__HAS_LIST_ID  exists:exists:List-Id
HAS_LIST_UNSUB exists:List-Unsubscribe
metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI
&& !__HAS_LIST_ID && !HAS_LIST_UNSUB)

Without that rule it might have flown below my sa-radar.
Got some scoring on it by using this plugin:
https://github.com/eilandert/Botnet.pm

and with the built in rules MIME_BASE64_TEXT and FROM_EXCESS_BASE64. As
well RCVD_DOUBLE_IP_SPAM hit on that sample

Regards

tobi

Re: Random word spams and wiki spams

[David Jones]

On 07/07/2017 05:39 PM, Alex wrote:

Hi,


urirhssub   URIBL_IVMRHSBL  uri.invaluement.com.   A127.0.0.2
tflags  URIBL_IVMRHSBL  net
score   URIBL_IVMRHSBL  3.2


I did not have this one or the reuse line. Is that "right-hand-side"?
Do you have one such example?


header  RCVD_IN_IVMBL
eval:check_rbl('ivmbl-lastexternal','sip.invaluement.com')
tflags  RCVD_IN_IVMBL   net
score   RCVD_IN_IVMBL   4.2

header  RCVD_IN_IVM24BL
eval:check_rbl('ivm24bl-lastexternal','sip24.invaluement.com')
tflags  RCVD_IN_IVM24BL net
score   RCVD_IN_IVM24BL 3.2


I also had both of these as
check_rbl('ivmSIP-lastexternal','sip.invaluement.com') and
check_rbl('ivmSIP-lastexternal",'sip24.invaluement.com')
(first argument same for both)



Ummm.  Well.  I don't have any hits on that RHSBL rule in the past 2 
weeks so maybe that is not a valid rule.  Ignore that one.  I think I 
will take it out of my ivm.cf file.


To all, please don't setup these rules and flood the IVM DNS servers 
with requests.  IVM is a private RBL feed (not very expensive) so you 
should have a local rbldnsd instance with the DNS servers that the mail 
filtering servers point to serving invaluement.com authoritatively.


Sorry Rob if we cause problems with your DNS servers accidentally by 
posting these rules.


--
David Jones

Re: Random word spams and wiki spams

[Alex]

Hi,

> urirhssub   URIBL_IVMRHSBL  uri.invaluement.com.   A127.0.0.2
> tflags  URIBL_IVMRHSBL  net
> score   URIBL_IVMRHSBL  3.2

I did not have this one or the reuse line. Is that "right-hand-side"?
Do you have one such example?

> header  RCVD_IN_IVMBL
> eval:check_rbl('ivmbl-lastexternal','sip.invaluement.com')
> tflags  RCVD_IN_IVMBL   net
> score   RCVD_IN_IVMBL   4.2
>
> header  RCVD_IN_IVM24BL
> eval:check_rbl('ivm24bl-lastexternal','sip24.invaluement.com')
> tflags  RCVD_IN_IVM24BL net
> score   RCVD_IN_IVM24BL 3.2

I also had both of these as
check_rbl('ivmSIP-lastexternal','sip.invaluement.com') and
check_rbl('ivmSIP-lastexternal",'sip24.invaluement.com')
(first argument same for both)

Re: Random word spams and wiki spams

[David Jones]

On 07/07/2017 03:08 PM, Alex wrote:

Hi,

On Fri, Jul 7, 2017 at 3:45 PM, John Hardin  wrote:

On Fri, 7 Jul 2017, Alex wrote:


It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.


Defense in depth. For that sort of thing you also need dynamic blocking of
the malware hosts (as much as is possible) in either your site web proxy (if
you have one) or your firewall rules.


Yes, absolutely. We have scripts that can be used to populate a local
RBLs that extract the from, IPs, etc, and provide the ability to drop
them into a postfix client_access blocklist. It's easy to stop them
after the fact.

The problem (in this case) was that they were received over the course
of a few days during the 4th holiday, then we got burnt when everyone
came back to the office. Generally, though, there could be ten
malicious emails received, a handful will actually click, while others
report them, which is enough to tarnish reputation.

When there's a small handful of malicious emails that make it through,
among hundreds of thousands received per day, it's just not possible
to go through them. A more automated or assisted method is necessary,
or better protection to begin with...



Alex,

Since you have Invaluement feed, do you have the Invaluement URIBL rules 
setup? (Not just the sip.invaluement.com RBL.)  These catch a lot of 
malicious URLs:


##{ URIBL_IVMURI ifplugin Mail::SpamAssassin::Plugin::URIDNSBL

ifplugin Mail::SpamAssassin::Plugin::URIDNSBL

urirhssub   URIBL_IVMURIuri.invaluement.com. A 2
bodyURIBL_IVMURIeval:check_uridnsbl('URIBL_IVMURI')
describeURIBL_IVMURIlisted on ivmSIP/24 found at invaluement.com
tflags  URIBL_IVMURInet
score   URIBL_IVMURI8.2

urirhssub   URIBL_IVMRHSBL  uri.invaluement.com.   A127.0.0.2
tflags  URIBL_IVMRHSBL  net
score   URIBL_IVMRHSBL  3.2

endif
##} URIBL_IVMURI ifplugin Mail::SpamAssassin::Plugin::URIDNSBL

ifplugin Mail::SpamAssassin::Plugin::URIDNSBL
reuse   URIBL_IVMURI
endif
##} ifplugin Mail::SpamAssassin::Plugin::URIDNSBL


ifplugin Mail::SpamAssassin::Plugin::DNSEval

header  RCVD_IN_IVMBL 
eval:check_rbl('ivmbl-lastexternal','sip.invaluement.com')

tflags  RCVD_IN_IVMBL   net
score   RCVD_IN_IVMBL   4.2

header  RCVD_IN_IVM24BL 
eval:check_rbl('ivm24bl-lastexternal','sip24.invaluement.com')

tflags  RCVD_IN_IVM24BL net
score   RCVD_IN_IVM24BL 3.2

endif


--
Dave

Re: Random word spams and wiki spams

[John Hardin]

On Fri, 7 Jul 2017, Alex wrote:


On Fri, Jul 7, 2017 at 3:45 PM, John Hardin  wrote:

On Fri, 7 Jul 2017, Alex wrote:


It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.


Defense in depth. For that sort of thing you also need dynamic blocking of
the malware hosts (as much as is possible) in either your site web proxy (if
you have one) or your firewall rules.


Yes, absolutely. We have scripts that can be used to populate a local
RBLs that extract the from, IPs, etc, and provide the ability to drop
them into a postfix client_access blocklist. It's easy to stop them
after the fact.


I'm not referring to email, I'm referring to the web clients that will try 
to visit a malware hosting URL. Block malware downloads as much as 
possible on the *outbound* (retrieval) side as well as on the inbound 
(bait) side.


There are third-party sources for such information (e.g. 
malwaredomains.com) that provide IP and domain name lists that you could 
use to automate such filters proactively, rather than relying solely on 
identified messages in your mail stream.



The problem (in this case) was that they were received over the course
of a few days during the 4th holiday, then we got burnt when everyone
came back to the office.


Automated download blocking via malwaredomains and other such sources 
might have mitigated that - the emails would still go through, but anybody 
who fell for it and clicked on the download link might have been blocked 
(malwaredomains et. al. are, after all, reactive and imperfect, but they 
are helpful).


Generally, though, there could be ten malicious emails received, a 
handful will actually click, while others report them, which is enough 
to tarnish reputation.


Right.


When there's a small handful of malicious emails that make it through,
among hundreds of thousands received per day, it's just not possible
to go through them. A more automated or assisted method is necessary,
or better protection to begin with...




--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Our government should bear in mind the fact that the American
  Revolution was touched off by the then-current government
  attempting to confiscate firearms from the people.
---
 Today: Robert Heinlein's 110th birthday

Re: Random word spams and wiki spams

[jahlives]

Am 07.07.2017 um 19:04 schrieb Alex:
>
> I'm interested in how your system would have (or currently does)
> handle this email I received some days ago:
> https://pastebin.com/innRFvZt
>
that one triggers one of my redpill meta rules and scores at 24.1

__HAS_LIST_ID  exists:exists:List-Id
HAS_LIST_UNSUB exists:List-Unsubscribe
metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI
&& !__HAS_LIST_ID && !HAS_LIST_UNSUB)

Without that rule it might have flown below my sa-radar.
Got some scoring on it by using this plugin:
https://github.com/eilandert/Botnet.pm

and with the built in rules MIME_BASE64_TEXT and FROM_EXCESS_BASE64. As
well RCVD_DOUBLE_IP_SPAM hit on that sample

Regards

tobi

Re: Random word spams and wiki spams

[Alex]

Hi,

On Fri, Jul 7, 2017 at 3:45 PM, John Hardin  wrote:
> On Fri, 7 Jul 2017, Alex wrote:
>
>> It's just a short body with a URI which downloads malware. We got hit
>> by this pretty hard. This is where the real threats are. Receive one
>> of these to an Exchange distribution list and your reputation with the
>> customer suffers badly.
>
> Defense in depth. For that sort of thing you also need dynamic blocking of
> the malware hosts (as much as is possible) in either your site web proxy (if
> you have one) or your firewall rules.

Yes, absolutely. We have scripts that can be used to populate a local
RBLs that extract the from, IPs, etc, and provide the ability to drop
them into a postfix client_access blocklist. It's easy to stop them
after the fact.

The problem (in this case) was that they were received over the course
of a few days during the 4th holiday, then we got burnt when everyone
came back to the office. Generally, though, there could be ten
malicious emails received, a handful will actually click, while others
report them, which is enough to tarnish reputation.

When there's a small handful of malicious emails that make it through,
among hundreds of thousands received per day, it's just not possible
to go through them. A more automated or assisted method is necessary,
or better protection to begin with...

RE: Random word spams and wiki spams

[Charles Amstutz]

Mostly autolearn ham and train some spam, have found that one account needed 
ham though. 

Most user accounts in question are at least 200/200, most are well over a few 
thousand each (I believe) 

>> I need to read up bayes a bit, I was surprised to learn that after 
>> using sa-learn --spam, then bayes only tagged it at Bayes_50 instead 
>> of Bayes_99, Unless I did something incorrect.

>There is a minimum level of both spam *and ham* that Bayes must be trained 
>with before it will start providing scoreable analysis.

>How much have you trained it with?

RE: Random word spams and wiki spams

[John Hardin]

On Fri, 7 Jul 2017, Charles Amstutz wrote:

I need to read up bayes a bit, I was surprised to learn that after using 
sa-learn --spam, then bayes only tagged it at Bayes_50 instead of 
Bayes_99, Unless I did something incorrect.


There is a minimum level of both spam *and ham* that Bayes must be trained 
with before it will start providing scoreable analysis.


How much have you trained it with?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  It is not the place of government to make right every tragedy and
  woe that befalls every resident of the nation.
---
 Today: Robert Heinlein's 110th birthday

Re: Random word spams and wiki spams

[John Hardin]

On Fri, 7 Jul 2017, Alex wrote:


It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.


Defense in depth. For that sort of thing you also need dynamic blocking of 
the malware hosts (as much as is possible) in either your site web proxy 
(if you have one) or your firewall rules.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  It is not the place of government to make right every tragedy and
  woe that befalls every resident of the nation.
---
 Today: Robert Heinlein's 110th birthday

RE: Random word spams and wiki spams

[Charles Amstutz]

>> I find many don't contribute (despite it being open source) for fear of 
>> spammers using these ideas against us, but the project suffers as a result.

I think others don't due to IP rights. I'm glad people do though.

Re: Random word spams and wiki spams

[Alex]

Hi,

On Fri, Jul 7, 2017 at 2:30 PM, David Jones  wrote:
> On 07/07/2017 12:04 PM, Alex wrote:
>>
>> Hi,
>>
>> On Fri, Jul 7, 2017 at 12:14 PM, David Jones  wrote:
>>>
>>> On 07/07/2017 11:04 AM, Charles Amstutz wrote:


 Thank you everyone for the suggestions, I will look into it. One thing
 I've noticed is that sometimes it takes a day for any *BL's to pick up
 some
 of the spam, and by that time, the run could be done. Greylisting isn't
 an
 option. It sometimes feels like always reactive vs pro-active in
 filtering.
 For example, I try to block the old runs of "Ford Warranties", write a
 few
 rules, then never receive them again :)

 This is a slight over exaggeration, but close.

>>>
>>> No. I completely understand.  A couple of years ago I was doing the same
>>> thing always reacting to new spam campaigns.  It took a lot of my time
>>> and I
>>> never felt like I was winning those one-day battles.
>>>
>>> Now I have tuned my MTA (Postfix with postscreen) to reject the majority
>>> of
>>> junk before it ever reaches SA.  See the archives for these Postscreen
>>> weighted RBLs if you are running Postfix.  With about 24 RBLs including
>>> invaluement, I am able to be aggressive with many RBLs adding up to a
>>> block
>>> threshold of 8 in postscreen.
>>
>>
>> I also have postfix, invaluement, of course Kevin's KAM rules, and
>> many (all?) of the other RBLs you use, including senderscore at the
>> postfix and spamassassin level.
>>
>> I'm interested in how your system would have (or currently does)
>> handle this email I received some days ago:
>> https://pastebin.com/innRFvZt
>>
>> Its IP (106.186.119.240) is still not listed with spamhaus, sorbs or
>> hostkarma, and has an 83 rating with senderscore.
>>
>> It's just a short body with a URI which downloads malware. We got hit
>> by this pretty hard. This is where the real threats are. Receive one
>> of these to an Exchange distribution list and your reputation with the
>> customer suffers badly.
>>
>> I'm also interested in other solutions - are those of you with
>> MIMEDefang or other systems blocking these?
>>
>
> I ran that message through one of my filters manually:

One of your filters?

> -0.2 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
> trust
> [106.186.119.240 listed in list.dnswl.org]
> -0.0 SPF_PASS   SPF: sender matches SPF record
>  0.0 ENA_RELAY_JP   Relayed through Japan
>  2.2 ENA_RELAY_NOT_US   Relayed through country outside of the US

Can't do this - email is received from every country :-(

I do have the relaycountry plugin, but score is set low, and usually
used in metas.

>  1.8 RCVD_DOUBLE_IP_SPAMBulk email fingerprint (double IP) found

I'm noticing the ones that were quarantined were quarantined because
of this rule. Unfortunately I don't have the ones that were relayed
because it was too long ago.

> I guess I need to setup a wiki page or something similar with all of my
> tweaks and tuning to document it all in one place.

This is kind of a policy thing, no? In other words, I find many don't
contribute (despite it being open source) for fear of spammers using
these ideas against us, but the project suffers as a result.

We also have a few local rules, but not sure how helpful they would be
to others, and spammers more specifically. These days I can't imagine
using anything other than postfix, however.

RE: Random word spams and wiki spams

[Charles Amstutz]

I need to read  up bayes a bit, I was surprised to learn that after using 
sa-learn --spam, then bayes only tagged it at Bayes_50  instead of Bayes_99, 
Unless I did something incorrect.

Note: I do not use bayes files in user profiles, I use it in mysql database

Re: Random word spams and wiki spams

[David Jones]

On 07/07/2017 12:04 PM, Alex wrote:

Hi,

On Fri, Jul 7, 2017 at 12:14 PM, David Jones  wrote:

On 07/07/2017 11:04 AM, Charles Amstutz wrote:


Thank you everyone for the suggestions, I will look into it. One thing
I've noticed is that sometimes it takes a day for any *BL's to pick up some
of the spam, and by that time, the run could be done. Greylisting isn't an
option. It sometimes feels like always reactive vs pro-active in filtering.
For example, I try to block the old runs of "Ford Warranties", write a few
rules, then never receive them again :)

This is a slight over exaggeration, but close.



No. I completely understand.  A couple of years ago I was doing the same
thing always reacting to new spam campaigns.  It took a lot of my time and I
never felt like I was winning those one-day battles.

Now I have tuned my MTA (Postfix with postscreen) to reject the majority of
junk before it ever reaches SA.  See the archives for these Postscreen
weighted RBLs if you are running Postfix.  With about 24 RBLs including
invaluement, I am able to be aggressive with many RBLs adding up to a block
threshold of 8 in postscreen.


I also have postfix, invaluement, of course Kevin's KAM rules, and
many (all?) of the other RBLs you use, including senderscore at the
postfix and spamassassin level.

I'm interested in how your system would have (or currently does)
handle this email I received some days ago:
https://pastebin.com/innRFvZt

Its IP (106.186.119.240) is still not listed with spamhaus, sorbs or
hostkarma, and has an 83 rating with senderscore.

It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.

I'm also interested in other solutions - are those of you with
MIMEDefang or other systems blocking these?



I ran that message through one of my filters manually:


Content analysis details:   (7.1 points, 5.0 required)

 pts rule name  description
 -- 
--

-0.2 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[106.186.119.240 listed in list.dnswl.org]
-0.0 SPF_PASS   SPF: sender matches SPF record
 0.0 ENA_RELAY_JP   Relayed through Japan
 2.2 ENA_RELAY_NOT_US   Relayed through country outside of the US
 0.0 OS_UNKNOWN Relay runs on unknown OS
 0.8 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5007]
 1.7 MIME_BASE64_TEXT   RAW: Message text disguised using base64 
encoding

 1.0 FROM_EXCESS_BASE64 From: base64 encoded unnecessarily
-0.2 RCVD_IN_SENDERSCORE_80_89 Senderscore.org score of 80 to 89
 1.8 RCVD_DOUBLE_IP_SPAMBulk email fingerprint (double IP) found


I guess I didn't mention that setting up the RelayCountry plugin and 
then various rules based on country codes that are not normal for your 
location:


header  ENA_RELAY_NOT_USX-Relay-Countries =~ 
/\b[ABCDEFGHIJKLMNOPQRTVWYZ]{2}\b/
describeENA_RELAY_NOT_USRelayed through country outside 
of the US

score   ENA_RELAY_NOT_US2.2

header  ENA_RELAY_CNX-Relay-Countries =~ /CN/
describeENA_RELAY_CNRelayed through China
score   ENA_RELAY_CN2.2

header  ENA_RELAY_KRX-Relay-Countries =~ /KR/
describeENA_RELAY_KRRelayed through Korea
score   ENA_RELAY_KR6.2

header  ENA_RELAY_COX-Relay-Countries =~ /CO/
describeENA_RELAY_CORelayed through Columbia
score   ENA_RELAY_CO4.2

header  ENA_RELAY_RUX-Relay-Countries =~ /RU/
describeENA_RELAY_RURelayed through Russia
score   ENA_RELAY_RU6.2

I guess I need to setup a wiki page or something similar with all of my 
tweaks and tuning to document it all in one place.


--
Dave

Re: Random word spams and wiki spams

[Alex]

Hi,

On Fri, Jul 7, 2017 at 12:14 PM, David Jones  wrote:
> On 07/07/2017 11:04 AM, Charles Amstutz wrote:
>>
>> Thank you everyone for the suggestions, I will look into it. One thing
>> I've noticed is that sometimes it takes a day for any *BL's to pick up some
>> of the spam, and by that time, the run could be done. Greylisting isn't an
>> option. It sometimes feels like always reactive vs pro-active in filtering.
>> For example, I try to block the old runs of "Ford Warranties", write a few
>> rules, then never receive them again :)
>>
>> This is a slight over exaggeration, but close.
>>
>
> No. I completely understand.  A couple of years ago I was doing the same
> thing always reacting to new spam campaigns.  It took a lot of my time and I
> never felt like I was winning those one-day battles.
>
> Now I have tuned my MTA (Postfix with postscreen) to reject the majority of
> junk before it ever reaches SA.  See the archives for these Postscreen
> weighted RBLs if you are running Postfix.  With about 24 RBLs including
> invaluement, I am able to be aggressive with many RBLs adding up to a block
> threshold of 8 in postscreen.

I also have postfix, invaluement, of course Kevin's KAM rules, and
many (all?) of the other RBLs you use, including senderscore at the
postfix and spamassassin level.

I'm interested in how your system would have (or currently does)
handle this email I received some days ago:
https://pastebin.com/innRFvZt

Its IP (106.186.119.240) is still not listed with spamhaus, sorbs or
hostkarma, and has an 83 rating with senderscore.

It's just a short body with a URI which downloads malware. We got hit
by this pretty hard. This is where the real threats are. Receive one
of these to an Exchange distribution list and your reputation with the
customer suffers badly.

I'm also interested in other solutions - are those of you with
MIMEDefang or other systems blocking these?

RE: Random word spams and wiki spams

[Charles Amstutz]

Has anyone ever got something like machine learning (I get that is what bayes 
kind of is) or R working with spam assassin? I’ve seen Books on this and maybe 
was refering to Bayes, but not sure.

Re: Random word spams and wiki spams

[Pedro David Marco]

>Also, setup the KAM.cf rules and extra signatures for ClamAV from 
>Sanesecurity.  These often help with new spam campaigns.  I can post 
>which signature DBs I am using if that would be helpful.
>-- 
>Dave
Hi Dave...
i have had problems in the past with the script to download Sanesecurity 
DBs...what scripts are you using?
Thanks!

PedroD



   

RE: Random word spams and wiki spams

[Charles Amstutz]

I setup spamdyke to block .top and many other TLDs where mostly spam came from. 
Unfortunately, I had to remove them, and now have to rely on content analysis 
with the use of *BL's. 

With setting up pattern matching, in efforts to future proof blocking, it will 
catch legit email that use characters to form tables (happens occasionally). 

The only thing I could think of was to set individual scores lower, but high 
meta scores.  I appreciate the options for postfix, but I do not run that on 
incoming mail servers.


    Infinite Systems
    Charles Amstutz | Systems Administrator
    charl...@infinitesys.com 402.477.2474
    134 S 13th Street, Suite 302 | Lincoln, NE 68508
 


-Original Message-
From: David Jones [mailto:djo...@ena.com] 
Sent: Friday, July 7, 2017 11:15 AM
To: Charles Amstutz ; 'users@spamassassin.apache.org' 

Subject: Re: Random word spams and wiki spams

On 07/07/2017 11:04 AM, Charles Amstutz wrote:
> Thank you everyone for the suggestions, I will look into it. One thing 
> I've noticed is that sometimes it takes a day for any *BL's to pick up 
> some of the spam, and by that time, the run could be done. Greylisting 
> isn't an option. It sometimes feels like always reactive vs pro-active 
> in filtering.  For example, I try to block the old runs of "Ford 
> Warranties", write a few rules, then never receive them again :)
> 
> This is a slight over exaggeration, but close.
> 

No. I completely understand.  A couple of years ago I was doing the same thing 
always reacting to new spam campaigns.  It took a lot of my time and I never 
felt like I was winning those one-day battles.

Now I have tuned my MTA (Postfix with postscreen) to reject the majority of 
junk before it ever reaches SA.  See the archives for these Postscreen weighted 
RBLs if you are running Postfix.  With about 24 RBLs including invaluement, I 
am able to be aggressive with many RBLs adding up to a block threshold of 8 in 
postscreen.

On the other side of this, you have to setup postwhite to whitelist major mail 
providers like comcast.net, aol, google, yahoo.com, etc. and let SA score them.

Now I rarely get any reports of spam getting through unless it's from a 
compromised account.  These will always be difficult to block for zero-hour 
spam campaigns from botnets.

Also, setup the KAM.cf rules and extra signatures for ClamAV from Sanesecurity. 
 These often help with new spam campaigns.  I can post which signature DBs I am 
using if that would be helpful.

--
Dave

Re: [SOLVED] I'm an idiot

[jdow]

On 2017-07-07 03:38, Rainer Sokoll wrote:



Am 06.07.2017 um 18:27 schrieb Rainer Sokoll :


[...]


Hm, I got an email from cron:

---8<--
/etc/cron.daily/spamassassin:
error: unable to refresh mirrors file for channel updates.spamassassin.org, 
using old file
channel: could not find working mirror, channel failed
sa-update failed for unknown reasons
---8<--


After losing some hairs it turned out that there are no problems in DNS, no 
problems in my network.
/etc/cron.daily/spamassassin changes UID:GUID to debian-spamd:debian-spamd 
during run.
But /var/lib/spamassassin/ was owned by root:root :-(

Sorry for all the noise,

Rainer


On the other hand, FireFox reports:
This site can’t be reached

updates.spamassassin.org’s server DNS address could not be found.
Go to http://spamassassin.org/
Search Google for updates spamassassin org
ERR_NAME_NOT_RESOLVED

{o.o}   God never promised us only one problem at a time.

Re: Random word spams and wiki spams

[David Jones]

On 07/07/2017 11:04 AM, Charles Amstutz wrote:

Thank you everyone for the suggestions, I will look into it. One thing I've noticed is 
that sometimes it takes a day for any *BL's to pick up some of the spam, and by that 
time, the run could be done. Greylisting isn't an option. It sometimes feels like always 
reactive vs pro-active in filtering.  For example, I try to block the old runs of 
"Ford Warranties", write a few rules, then never receive them again :)

This is a slight over exaggeration, but close.



No. I completely understand.  A couple of years ago I was doing the same 
thing always reacting to new spam campaigns.  It took a lot of my time 
and I never felt like I was winning those one-day battles.


Now I have tuned my MTA (Postfix with postscreen) to reject the majority 
of junk before it ever reaches SA.  See the archives for these 
Postscreen weighted RBLs if you are running Postfix.  With about 24 RBLs 
including invaluement, I am able to be aggressive with many RBLs adding 
up to a block threshold of 8 in postscreen.


On the other side of this, you have to setup postwhite to whitelist 
major mail providers like comcast.net, aol, google, yahoo.com, etc. and 
let SA score them.


Now I rarely get any reports of spam getting through unless it's from a 
compromised account.  These will always be difficult to block for 
zero-hour spam campaigns from botnets.


Also, setup the KAM.cf rules and extra signatures for ClamAV from 
Sanesecurity.  These often help with new spam campaigns.  I can post 
which signature DBs I am using if that would be helpful.


--
Dave

RE: Random word spams and wiki spams

[Charles Amstutz]

Thank you everyone for the suggestions, I will look into it. One thing I've 
noticed is that sometimes it takes a day for any *BL's to pick up some of the 
spam, and by that time, the run could be done. Greylisting isn't an option. It 
sometimes feels like always reactive vs pro-active in filtering.  For example, 
I try to block the old runs of "Ford Warranties", write a few rules, then never 
receive them again :)

This is a slight over exaggeration, but close.  

Re: Random word spams and wiki spams

[David Jones]

On 07/07/2017 10:15 AM, Kevin A. McGrail wrote:

On 7/7/2017 9:06 AM, Charles Amstutz wrote:
I am new to the group, but have experience with writing some rules and 
some meta rules.


Has anyone come up with a good way to detect spam that has random 
words in paragraph forms (usually at the bottom of the message body) 
or they look like they copy parts from various wiki’s or other news 
sources?


That type of obfuscation is just a technique used by spammers. Typically 
there are other indicators that I would focus on.


In other words, analyzing the content might not help much but analyzing 
the pathway (how the email got to point B) might be.


It's often more helpful to use pastebin to post a full example with 
headers for discussions. Otherwise it's a bit vague to discuss.


Regards,
KAM


I agree.  Train them as spam in Bayesian.  Setup more RBL rules to 
augment the default SA RBLs, meta rules that hit combinations of 
existing SA rules for these emails, etc.


See the SA mailing list archives for the score.senderscore.org RBL as an 
example of a helpful RBL addition.


Invaluement RBL is well worth it's cost if you have a mail filtering 
platform with your own rbldnsd setup.  I am able to setup rules with 
scores well above 8.0 that cover a lot of edge case spam that don't hit 
other rules that add points.


--
Dave

Re: Random word spams and wiki spams

[Kevin A. McGrail]

On 7/7/2017 9:06 AM, Charles Amstutz wrote:
I am new to the group, but have experience with writing some rules and 
some meta rules.


Has anyone come up with a good way to detect spam that has random 
words in paragraph forms (usually at the bottom of the message body) 
or they look like they copy parts from various wiki’s or other news 
sources?


That type of obfuscation is just a technique used by spammers. Typically 
there are other indicators that I would focus on.


In other words, analyzing the content might not help much but analyzing 
the pathway (how the email got to point B) might be.


It's often more helpful to use pastebin to post a full example with 
headers for discussions. Otherwise it's a bit vague to discuss.


Regards,
KAM

Random word spams and wiki spams

[Charles Amstutz]

Hello,

I am new to the group, but have experience with writing some rules and some 
meta rules.

Has anyone come up with a good way to detect spam that has random words in 
paragraph forms (usually at the bottom of the message body) or they look like 
they copy parts from various wiki's or other news sources?

Thanks

Charles

Re: Body length tests

[Christian Laußat]

I think the difference is between body and rawbody rule:

rawbody: If there's encoding like quoted-printable or base64, the text 
parts are decoded, but you still get all the HTML tags and such.


body: If they exist, also html parts are decoded, so you just get the 
plain text content.


So it depends of what you would like to count, the length of the 
readable text or also the HTML encoding.


On 2012-07-07 04:27, Alex wrote:

Hi,

I'm having a problem with emails with short body text and a link to
malware that automatically downloads when the link is clicked. What's
the difference between the short body tests, besides the actual
character lengths:

72_active.cf
body__KAM_BODY_LENGTH_LT_128
eval:check_body_length('128')

describe__KAM_BODY_LENGTH_LT_128The length of the body
of the email is less than 128 bytes.

KAM.cf
meta__BODY_LE_200 (__RB_LE_200 == 1) && !__RB_GT_200
rawbody __RB_LE_200 /^.{2,200}$/s
tflags  __RB_LE_200 multiple maxhits=2

Here's one such example, if you're interested. The link is actually 
still valid.

https://pastebin.com/innRFvZt

It hit bayes00 and not many other rules. This one, or ones like it,
were hitting ANY_BOUNCE_MESSAGE (and FROM_NO_USER) in some variations
because the From field was either empty or missing entirely.


--
Christian Laußat

[SOLVED] I'm an idiot (was: Re: updates.spamassassin.org gone?)

[Rainer Sokoll]

> Am 06.07.2017 um 18:27 schrieb Rainer Sokoll :

[...]

> Hm, I got an email from cron:
> 
> ---8<--
> /etc/cron.daily/spamassassin:
> error: unable to refresh mirrors file for channel updates.spamassassin.org, 
> using old file
> channel: could not find working mirror, channel failed
> sa-update failed for unknown reasons
> ---8<--

After losing some hairs it turned out that there are no problems in DNS, no 
problems in my network.
/etc/cron.daily/spamassassin changes UID:GUID to debian-spamd:debian-spamd 
during run.
But /var/lib/spamassassin/ was owned by root:root :-(

Sorry for all the noise,

Rainer



signature.asc
Description: Message signed with OpenPGP