RE: MSBL Email Blocklist (EBL) SA usage query

[Peter]

Hi Kevin,

I am just trying this and getting the same error.  Did you work out how to
fix this?

Cheers,

Peter

*** REPLY SEPARATOR ***

On 25/10/2017 at 6:41 PM Kevin Miller wrote:
Implemented it on one of my tier 2 mx hosts.  No hits so far, but I’m not
sure if it’s working or not.  Running spamassasin –lint returns a
warning:
  root@mx2:/etc/spamassassin# spamassassin --lint
  Oct 25 09:39:35.403 [15095] warn: Use of uninitialized value in regexp
compilation at /etc/spamassassin/HashBL.pm line 52.

...Kevin
--
Kevin Miller
Network/email Administrator, CBJ MIS Dept.
155 South Seward Street
Juneau, Alaska 99801
Phone: (907) 586-0242, Fax: (907) 586-4588 Registered Linux User No: 307357

From: Michael Grant [mailto:michael.gr...@gmail.com]
Sent: Sunday, October 15, 2017 2:02 AM
To: SpamAssassin Users
Subject: MSBL Email Blocklist (EBL) SA usage query

Has anyone tried out the the MSBL Email Blocklist (EBL) HashBL.pm with
Spamassassin from msbl.org and possibly considered packaging this module
(available from this page: http://msbl.org/ebl-implementation.html) with
SpamAssassin (perhaps in a forthcoming release)?  rSpamD already has
internal support for the EBL. So I believe the MSBL folks are for this sort
of thing in general.

This plugin looks through the message (not just headers) for email
addresses which have been identified as email drop boxes for scams like 419
advance fee fraud.  It then looks hashes of these addresses up in a
blocklist.

I'm not affiliated with these folks.  I do however use this module in my
setup though and find it catches a bunch of things we wouldn't have
otherwise caught.

Michael Grant

Re: Scoring Philosophy?

[Bill Cole]

On 21 Nov 2017, at 16:01 (-0500), Jerry Malcolm wrote:

 I have the Bayesian filter working, with a simple way to train it.  
I have sent over 5000 training messages to it over the past 6-8 
months.


That's good, but only if it has a mix of ham and spam. If you only tell 
the Bayes filter about one or the other, it will work poorly. Also, 
while there is widely differing opinion on the "autolearn" 
functionality, I find it keeps my Bayes DB pretty accurate, with 
bayes_auto_learn_threshold_nonspam set to -0.1 (it defaults to 0.1) and 
bayes_auto_learn_threshold_spam to 8 (default is 12.)


On the larger issue of whether SpamAssassin is an "out-of-the-box" total 
solution for spam control, the simple answer is "no." No such thing 
exists or CAN exist. I've worked with about a dozen mail systems 
handling mail for about a hundred distinct domains (I'm old) and while 
they have had many commonalities, each has needed its own tweaking to 
optimize, and in cases where I've run multi-tenant systems, seemingly 
similar customers have required divergent filtering, i.e. multiple small 
businesses of similar scale in the same metropolitan area have each 
needed domain-specific filtering that their neighbors using the same 
infrastructure & services provider couldn't tolerate. The FUSSP is a 
myth. (see https://www.rhyolite.com/anti-spam/you-might-be.html for 
signs of FUSSP delusions.)


Beyond the fact that similar domains and even similar individuals can 
have starkly different anti-spam needs, there is a blind spot in 
SpamAssassin which is the result of its contributors generally 
practicing layered defense in such a way that they never even show to 
SpamAssassin a large fraction of the spam which targets them. SA will 
not catch some of the most blatant and even dangerous spam because it is 
easily caught by safe MTA (or pre-MTA, e.g. postscreen) anti-spam 
tactics or even network layer tools like the Spamhaus DROP and EDROP 
lists. If you do not use mechanisms that end the majority of SMTP 
sessions before the DATA phase, you will need to be especially careful 
about correct and customized configuration of SA.


One common area to be particularly careful about in configuring SA is 
network classification: the trusted_networks and internal_networks 
settings. If you do not set those correctly, you can end up never 
hitting any of the DNSBL rules or hitting them improperly because SA 
isn't working with the right Received header for a particular DNSBL. A 
related and increasingly common (dunno why) source of never hitting 
DNSBL rules is a form of firewall/router NAT sometimes called "Secure 
NAT" where inbound connections have their source IP's replaced with the 
IP of the device handling the NAT. This typically kills any ability of a 
MTA or a filter like SA to use DNSBLs or any other anti-spam tactic that 
requires knowing the client IP (or the client IP of the last 
external-client transport hop.)


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole

Re: Scoring Philosophy?

[David Jones]

On 11/21/2017 05:11 PM, Jerry Malcolm wrote:

On 11/21/2017 3:52 PM, Bowie Bailey wrote:

On 11/21/2017 4:01 PM, Jerry Malcolm wrote:
I have been using SpamAssassin in my hosting environment for several 
years.  It catches thousands of spam messages (thank you...). But my 
concern is that it doesn't catch a couple of hundred messages per 
day.  I have the Bayesian filter working, with a simple way to train 
it.  I have sent over 5000 training messages to it over the past 6-8 
months. I have set up a non-forwarding caching DNS, and the black 
list tests are working.


My question is with the scoring.  I understand the general theory of 
adding up 'votes' by all of the spam tests to determine if it's 
indeed spam.  But it appears that no one test, no matter how certain 
it is, has enough power to qualify the message as spam. The Bayesian 
filter can say it's 80-100% certain it's spam.  But some other test 
decides it's not and even sometimes has a negative number that 
subtracts the Bayesian score from the total.  But my biggest problem 
is that even if it's scored as coming from a BL URL, but if Bayesian 
doesn't also say it's spam, then it's apparently still not spam.  I 
spend a couple of hours every day trying to tell the Bayesian filter 
about today's new strains of spam that it hasn't yet seen.


Am I missing something obvious?  Is this just the way it works, and I 
should expect to have to run a couple of hundred missed spams through 
the Bayesian filter each day?  My threshold score was originally set 
to 5.0.  I don't even remember where that came from.  I dropped it to 
4.0 a couple of years ago, and that's where it is now.  But (see 
example output below) when BL says it's spam and adds 2.5, then 
Bayesian says it's 40-60% spam and adds 0.8, and it's got a small 
font and gets another 0.5, and all other tests are neutral... it's 
now 3.8 and STILL not spam with a threshold of 4.0.


Can someone tell me if this is by design and/or if my configuration 
should be adjusted?  I realize I can easily drop the threshold to 1.0 
or 2.0.  But that would probably just shift the problem to tons of 
false-positives which obviously is not a good solution.


The general philosophy is that no one rule should be able to mark a 
message as spam on its own.  However, quite a few of us have bumped 
the scores beyond that point for rules that we trust.  Most commonly 
the Bayes_99 or Bayes_99+Bayes_999 can be made to score 5+ points if 
your database is well trained.  You can also keep an eye on negative 
scoring rules that seem to hit too frequently on spams and bump the 
scores for those.


You can also add third-party rules.  I believe the only 
actively-maintained set at the moment is from KAM.


http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf
http://www.pccc.com/downloads/SpamAssassin/contrib/nonKAMrules.cf

I have both of these in use on my server with good results.

You can use trusted blacklists to block spam at the MTA before it even 
gets to SA.  I use zen.spamhaus.org for this on my server.


You didn't specify the version of SA you are running, but you should 
make sure you are upgraded to the newest version (3.4.1) for best 
results.


The amount of spam that gets through will be dependent on your 
mailflow.  I usually see 1 or 2 missed spams in my inbox per day. 
Today, the zen blacklist blocked 33 messages to my inbox and SA marked 
another 20 as spam while delivering 95 clean messages.  I think there 
might have been one spam that made it to my inbox.  I have the 
threshold set at 4.0 for my mailbox.



Hi Bowie,
Thanks for the quick response.   I'm a bit concerned about going in and 
playing around with scores, etc considering my minimal knowledge of the 
overall process at this point.  I'll download and install the KAM 
rules.  And I'll need to start logging my percentages of 
clean/spam/total per account and determine if my results are typical.  
But I'm just curious what others are using for their installation.  Is 
running SA out-of-the-box simply not done?  is it expected that all 
serious users will add 3rd party rule plugins and adjust scores?


Mail filtering is very dependent on your location, expected languages, 
and recipients.  SpamAssassin is pretty generic/conservative out of the 
box to fit in most environments safely.  You need to tune it a bit for 
your mail flow.


I'm currently being inundated with solar, skin tag removal, 3D organ 
printing, Shark Tank, and Russian brides spam messages.  Seems like some 
of these could be pretty obvious rules. No matter what I do with 
Bayesian training, they are still getting through.  Just out of 
curiosity, what rule(s), if any, should be expected to catch spam emails 
with subjects such as these? Or, said differently are these specific 
subject lines being caught by other users, and if so, what rules are 
catching them?  Is there some way I can quickly add a rule that says if 
the subject contains 'Skin Tags', score it to 5 

Re: Scoring Philosophy?

[Jerry Malcolm]

On 11/21/2017 3:52 PM, Bowie Bailey wrote:

On 11/21/2017 4:01 PM, Jerry Malcolm wrote:
I have been using SpamAssassin in my hosting environment for several 
years.  It catches thousands of spam messages (thank you...). But my 
concern is that it doesn't catch a couple of hundred messages per 
day.  I have the Bayesian filter working, with a simple way to train 
it.  I have sent over 5000 training messages to it over the past 6-8 
months. I have set up a non-forwarding caching DNS, and the black 
list tests are working.


My question is with the scoring.  I understand the general theory of 
adding up 'votes' by all of the spam tests to determine if it's 
indeed spam.  But it appears that no one test, no matter how certain 
it is, has enough power to qualify the message as spam. The Bayesian 
filter can say it's 80-100% certain it's spam.  But some other test 
decides it's not and even sometimes has a negative number that 
subtracts the Bayesian score from the total.  But my biggest problem 
is that even if it's scored as coming from a BL URL, but if Bayesian 
doesn't also say it's spam, then it's apparently still not spam.  I 
spend a couple of hours every day trying to tell the Bayesian filter 
about today's new strains of spam that it hasn't yet seen.


Am I missing something obvious?  Is this just the way it works, and I 
should expect to have to run a couple of hundred missed spams through 
the Bayesian filter each day?  My threshold score was originally set 
to 5.0.  I don't even remember where that came from.  I dropped it to 
4.0 a couple of years ago, and that's where it is now.  But (see 
example output below) when BL says it's spam and adds 2.5, then 
Bayesian says it's 40-60% spam and adds 0.8, and it's got a small 
font and gets another 0.5, and all other tests are neutral... it's 
now 3.8 and STILL not spam with a threshold of 4.0.


Can someone tell me if this is by design and/or if my configuration 
should be adjusted?  I realize I can easily drop the threshold to 1.0 
or 2.0.  But that would probably just shift the problem to tons of 
false-positives which obviously is not a good solution.


The general philosophy is that no one rule should be able to mark a 
message as spam on its own.  However, quite a few of us have bumped 
the scores beyond that point for rules that we trust.  Most commonly 
the Bayes_99 or Bayes_99+Bayes_999 can be made to score 5+ points if 
your database is well trained.  You can also keep an eye on negative 
scoring rules that seem to hit too frequently on spams and bump the 
scores for those.


You can also add third-party rules.  I believe the only 
actively-maintained set at the moment is from KAM.


http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf
http://www.pccc.com/downloads/SpamAssassin/contrib/nonKAMrules.cf

I have both of these in use on my server with good results.

You can use trusted blacklists to block spam at the MTA before it even 
gets to SA.  I use zen.spamhaus.org for this on my server.


You didn't specify the version of SA you are running, but you should 
make sure you are upgraded to the newest version (3.4.1) for best 
results.


The amount of spam that gets through will be dependent on your 
mailflow.  I usually see 1 or 2 missed spams in my inbox per day. 
Today, the zen blacklist blocked 33 messages to my inbox and SA marked 
another 20 as spam while delivering 95 clean messages.  I think there 
might have been one spam that made it to my inbox.  I have the 
threshold set at 4.0 for my mailbox.



Hi Bowie,
Thanks for the quick response.   I'm a bit concerned about going in and 
playing around with scores, etc considering my minimal knowledge of the 
overall process at this point.  I'll download and install the KAM 
rules.  And I'll need to start logging my percentages of 
clean/spam/total per account and determine if my results are typical.  
But I'm just curious what others are using for their installation.  Is 
running SA out-of-the-box simply not done?  is it expected that all 
serious users will add 3rd party rule plugins and adjust scores?  I'm 
currently being inundated with solar, skin tag removal, 3D organ 
printing, Shark Tank, and Russian brides spam messages.  Seems like some 
of these could be pretty obvious rules. No matter what I do with 
Bayesian training, they are still getting through.  Just out of 
curiosity, what rule(s), if any, should be expected to catch spam emails 
with subjects such as these? Or, said differently are these specific 
subject lines being caught by other users, and if so, what rules are 
catching them?  Is there some way I can quickly add a rule that says if 
the subject contains 'Skin Tags', score it to 5 (without getting into 
coding rule plugins, etc)?  Or is there some place that is creating 
these rules for 'pretty obvious' subject lines as the new strains of 
spam appear?


I still feel like I'm missing something obvious

Thanks again.

Jerry

On 11/21/2017 3:52 PM, Bowie 

Re: Scoring Philosophy?

[Bowie Bailey]

On 11/21/2017 4:01 PM, Jerry Malcolm wrote:
I have been using SpamAssassin in my hosting environment for several 
years.  It catches thousands of spam messages (thank you...).  But my 
concern is that it doesn't catch a couple of hundred messages per 
day.  I have the Bayesian filter working, with a simple way to train 
it.  I have sent over 5000 training messages to it over the past 6-8 
months. I have set up a non-forwarding caching DNS, and the black list 
tests are working.


My question is with the scoring.  I understand the general theory of 
adding up 'votes' by all of the spam tests to determine if it's indeed 
spam.  But it appears that no one test, no matter how certain it is, 
has enough power to qualify the message as spam. The Bayesian filter 
can say it's 80-100% certain it's spam.  But some other test decides 
it's not and even sometimes has a negative number that subtracts the 
Bayesian score from the total.  But my biggest problem is that even if 
it's scored as coming from a BL URL, but if Bayesian doesn't also say 
it's spam, then it's apparently still not spam.  I spend a couple of 
hours every day trying to tell the Bayesian filter about today's new 
strains of spam that it hasn't yet seen.


Am I missing something obvious?  Is this just the way it works, and I 
should expect to have to run a couple of hundred missed spams through 
the Bayesian filter each day?  My threshold score was originally set 
to 5.0.  I don't even remember where that came from.  I dropped it to 
4.0 a couple of years ago, and that's where it is now.  But (see 
example output below) when BL says it's spam and adds 2.5, then 
Bayesian says it's 40-60% spam and adds 0.8, and it's got a small font 
and gets another 0.5, and all other tests are neutral... it's now 3.8 
and STILL not spam with a threshold of 4.0.


Can someone tell me if this is by design and/or if my configuration 
should be adjusted?  I realize I can easily drop the threshold to 1.0 
or 2.0.  But that would probably just shift the problem to tons of 
false-positives which obviously is not a good solution.


The general philosophy is that no one rule should be able to mark a 
message as spam on its own.  However, quite a few of us have bumped the 
scores beyond that point for rules that we trust.  Most commonly the 
Bayes_99 or Bayes_99+Bayes_999 can be made to score 5+ points if your 
database is well trained.  You can also keep an eye on negative scoring 
rules that seem to hit too frequently on spams and bump the scores for 
those.


You can also add third-party rules.  I believe the only 
actively-maintained set at the moment is from KAM.


http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf
http://www.pccc.com/downloads/SpamAssassin/contrib/nonKAMrules.cf

I have both of these in use on my server with good results.

You can use trusted blacklists to block spam at the MTA before it even 
gets to SA.  I use zen.spamhaus.org for this on my server.


You didn't specify the version of SA you are running, but you should 
make sure you are upgraded to the newest version (3.4.1) for best results.


The amount of spam that gets through will be dependent on your 
mailflow.  I usually see 1 or 2 missed spams in my inbox per day. Today, 
the zen blacklist blocked 33 messages to my inbox and SA marked another 
20 as spam while delivering 95 clean messages.  I think there might have 
been one spam that made it to my inbox.  I have the threshold set at 4.0 
for my mailbox.


--
Bowie

Scoring Philosophy?

[Jerry Malcolm]

I have been using SpamAssassin in my hosting environment for several 
years.  It catches thousands of spam messages (thank you...).  But my 
concern is that it doesn't catch a couple of hundred messages per day.  
I have the Bayesian filter working, with a simple way to train it.  I 
have sent over 5000 training messages to it over the past 6-8 months. I 
have set up a non-forwarding caching DNS, and the black list tests are 
working.


My question is with the scoring.  I understand the general theory of 
adding up 'votes' by all of the spam tests to determine if it's indeed 
spam.  But it appears that no one test, no matter how certain it is, has 
enough power to qualify the message as spam. The Bayesian filter can say 
it's 80-100% certain it's spam.  But some other test decides it's not 
and even sometimes has a negative number that subtracts the Bayesian 
score from the total.  But my biggest problem is that even if it's 
scored as coming from a BL URL, but if Bayesian doesn't also say it's 
spam, then it's apparently still not spam.  I spend a couple of hours 
every day trying to tell the Bayesian filter about today's new strains 
of spam that it hasn't yet seen.


Am I missing something obvious?  Is this just the way it works, and I 
should expect to have to run a couple of hundred missed spams through 
the Bayesian filter each day?  My threshold score was originally set to 
5.0.  I don't even remember where that came from.  I dropped it to 4.0 a 
couple of years ago, and that's where it is now.  But (see example 
output below) when BL says it's spam and adds 2.5, then Bayesian says 
it's 40-60% spam and adds 0.8, and it's got a small font and gets 
another 0.5, and all other tests are neutral... it's now 3.8 and STILL 
not spam with a threshold of 4.0.


Can someone tell me if this is by design and/or if my configuration 
should be adjusted?  I realize I can easily drop the threshold to 1.0 or 
2.0.  But that would probably just shift the problem to tons of 
false-positives which obviously is not a good solution.


Thx.

Jerry

X-SpamAssassin_121: Content analysis details:   (3.8 points, 4.0 required)
X-SpamAssassin_122:
X-SpamAssassin_123:  pts rule name  description
X-SpamAssassin_124:  -- 
--
X-SpamAssassin_125:  2.5 URIBL_DBL_SPAM Contains a spam URL 
listed in the DBL blocklist

X-SpamAssassin_126: [URIs: lspdiscover.com]
X-SpamAssassin_127:  0.0 TVD_RCVD_IP4   Message was received 
from an IPv4 address
X-SpamAssassin_128:  0.0 TVD_RCVD_IP    Message was received 
from an IP address
X-SpamAssassin_129: -0.0 SPF_HELO_PASS  SPF: HELO matches SPF 
record
X-SpamAssassin_130:  0.8 BAYES_50   BODY: Bayes spam 
probability is 40 to 60%

X-SpamAssassin_131: [score: 0.5013]
X-SpamAssassin_132:  0.0 HTML_MESSAGE   BODY: HTML included in 
message
X-SpamAssassin_133:  0.5 JAM_SMALL_FONT_SIZE    RAW: Body of mail 
contains parts with very small

X-SpamAssassin_134: font
X-SpamAssassin_135:  0.0 T_REMOTE_IMAGE Message contains an 
external image

X-SpamAssassin_136:
X-SpamAssassin_999: --
X-Spam-Flag: NO
X-Spam-Status: No, hits=3.8 required=4.0
X-MessageIsSpamProbability: 0.0022784550478299674

Re: NOTE: Warning to Abusers of Update Servers

[RW]

On Tue, 21 Nov 2017 08:55:34 -0600
David Jones wrote:


> You are correct.  I haven't dug into the code to verify but it
> appears that 3.4.x sa-update does use the DNS TXT record to know when
> to download so it doesn't hurt anything to run this version hourly.


By the sound of it this warning doesn't apply at all to anyone with a
normal up to date installation.

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

My goal is to stop abuse without causing undue grief or fps.  It may come to 
more draconian steps as you suggest.
Regards,
KAM

On November 21, 2017 10:13:38 AM EST, AJ Weber  wrote:
>
>> The major offenders are sa-update 3.3.x and generic curl clients
>based 
>> on the user agent in the logs running from every minute to every 15 
>> minutes and blindly pulling down the same rulesets over and over.
>>
>My "vote" counts for very, very little, but since these clients already
>
>have the latest rules (multiple times, apparently), why not just block
>them?
>
>If they are actually monitoring their update scripts at all (seems 
>doubtful), it should get their heads out of the sand (was going to use
>a 
>similar metaphor but wanted to be nice).  They will probably look for a
>
>resolution and find these latest posts.
>
>If they're not monitoring their updates on any regular basis, it
>doesn't 
>seem like they care if they get them anyway.

Re: Understanding SPF-verified spam from dropbox

[Alex]

Hi,

On Mon, Nov 20, 2017 at 5:47 PM, Bill Cole
 wrote:
> On 20 Nov 2017, at 13:31, Alex wrote:
>
>> On Mon, Nov 20, 2017 at 12:58 PM, Axb  wrote:
>>>
>>> On 11/20/2017 06:26 PM, Alex wrote:


 Hi, we have an email that originated from email.dropbox.com and has a
 link to https://hyzas.xss.ht/ which is a "payload to test for
 Cross-site Scripting" from the XSS Hunter Team.
>
>
> Yeah, sure. Or maybe it's a malware vector masquerading as a harmless test.
> As the very large and very obfuscated script says in comments at the top:
>
>If you believe that this payload has been used to attempt to
>compromise your service without permission, please contact us
>using https://xsshunter.com/contact.
>
> The registration info for xss.ht includes an embedded 

Re: NOTE: Warning to Abusers of Update Servers

[AJ Weber]

The major offenders are sa-update 3.3.x and generic curl clients based 
on the user agent in the logs running from every minute to every 15 
minutes and blindly pulling down the same rulesets over and over.


My "vote" counts for very, very little, but since these clients already 
have the latest rules (multiple times, apparently), why not just block them?


If they are actually monitoring their update scripts at all (seems 
doubtful), it should get their heads out of the sand (was going to use a 
similar metaphor but wanted to be nice).  They will probably look for a 
resolution and find these latest posts.


If they're not monitoring their updates on any regular basis, it doesn't 
seem like they care if they get them anyway.

Re: NOTE: Warning to Abusers of Update Servers

[David Jones]

On 11/21/2017 03:56 AM, A. Schulze wrote:


Kevin A. McGrail:

If you are checking the SpamAssassin updates more than 2x a day, 
expect to be blocked in the very near future. We have people 
checking literally every minute and we only release rules currently 1x 
per day. There is no need to check this often!


I use sa-update to update all channels I use. From what I've seen on 
"sa-update --debug" the default is to test an dns-record.

( "dig 1.4.3.updates.spamassassin.org. txt" for the main channel )
If the DNS answer indicate the same version no HTTP requests are done.
So it SHOULD not hurt to call sa-update more frequently UNLESS one have 
specific scripts that directly connect to the updateservers via http.


is this assumtion right?

Andreas





You are correct.  I haven't dug into the code to verify but it appears 
that 3.4.x sa-update does use the DNS TXT record to know when to 
download so it doesn't hurt anything to run this version hourly.


The major offenders are sa-update 3.3.x and generic curl clients based 
on the user agent in the logs running from every minute to every 15 
minutes and blindly pulling down the same rulesets over and over.


Not sure why those 3.3.x don't just upgrade anyway since it's not that 
difficult to do.  At some point in the future the older 3.3.x will be 
considered end-of-life and no ruleset updates for them.  I am pretty 
sure they are already considered unsupported so if a ruleset 
incompatibility pops up, that could be the end of updates for those 
ancient versions.


--
David Jones

Re: Understanding SPF-verified spam from dropbox

[Bill Cole]

On 20 Nov 2017, at 13:31, Alex wrote:


On Mon, Nov 20, 2017 at 12:58 PM, Axb  wrote:

On 11/20/2017 06:26 PM, Alex wrote:


Hi, we have an email that originated from email.dropbox.com and has 
a

link to https://hyzas.xss.ht/ which is a "payload to test for
Cross-site Scripting" from the XSS Hunter Team.


Yeah, sure. Or maybe it's a malware vector masquerading as a harmless 
test. As the very large and very obfuscated script says in comments at 
the top:


   If you believe that this payload has been used to attempt to
   compromise your service without permission, please contact us
   using https://xsshunter.com/contact.

The registration info for xss.ht includes an embedded 

Re: Why doesn't HK_RANDOM_FROM trigger on this email address?

[Bill Cole]

On 20 Nov 2017, at 0:59 (-0500), Pedro David Marco wrote:


Well, F. W. Nietzsche never had kids

But almost never so many people have had the same father...  :-p
Now serious: Maybe you can add some more rules to deduce it may be a 
german email and score the RANDOM accordingly...


Not really useful, because this is a check of the From header. There are 
a large number of people with German surnames who don't use the German 
language or live in places where German is the primary language.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole

Re: NOTE: Warning to Abusers of Update Servers

[Eray Aslan]

On Tue, Nov 21, 2017 at 01:41:56PM +0100, Ralf Hildebrandt wrote:
> Yup, seen that on a stratum 1 NTP server - all clients seem to have
> the same config, thus querying my server by means of NAT instead of
> using an internal stratum 2 :(

Common, probably internal, recursive DNS server but not an NTP one.  I
would have thought that that was a common config. Must be a lot of
clients to make you notice.  Cable/mobile operator?  With mobiles behind
a NAT gateway?  Any details you can share just for curiosity's sake?

-- 
Eray

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 7:32 AM, Anthony Cartmell wrote:

would give admins enough of a clue to see if they were a culprit without
giving the actual IPs away?


It's a good idea.  I think removing the first octet would be enough 
obfuscation.


So these IPs in 21 days are the top 15 abusers.  With rule updates 
disabled, this traffic level indicates a problem.


 24441 XX.61.138.136
  20948 XX.137.36.178
  20270 XX.56.17.151
  18452 XX.76.211.56
  18141 XX.32.88.139
  14028 XX.69.200.153
  12539 XX.229.96.116
  12506 XX.221.192.173
  11568 XX.77.52.43
  10526 XX.163.197.66
  10079 XX.61.28.10
   8818 XX.204.24.253
   8606 XX.128.6.242
   8289 XX.210.201.60
   6640 XX.57.105.90
   5647 XX.233.160.53
   5491 XX.19.251.191
   5266 XX.231.133.170
   4315 XX.163.15.214
   2980 XX.63.124.11

Regards,
KAM

Re: NOTE: Warning to Abusers of Update Servers

[Ralf Hildebrandt]

* Kevin A. McGrail :
> On 11/21/2017 7:35 AM, Reindl Harald wrote:
> > or they have simply more than one machine behind a single outgoing IP
> Yes, we considered that too hence why we haven't just blocked things
> outright.

Yup, seen that on a stratum 1 NTP server - all clients seem to have
the same config, thus querying my server by means of NAT instead of
using an internal stratum 2 :(

-- 
Ralf Hildebrandt   Charite Universitätsmedizin Berlin
ralf.hildebra...@charite.deCampus Benjamin Franklin
https://www.charite.de Hindenburgdamm 30, 12203 Berlin
Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 7:35 AM, Reindl Harald wrote:

or they have simply more than one machine behind a single outgoing IP
Yes, we considered that too hence why we haven't just blocked things 
outright.


Regards,
KAM

Re: NOTE: Warning to Abusers of Update Servers

[Ralf Hildebrandt]

* Kevin A. McGrail :

> The Top 14 abusers account for nearly 30% of our update based on sampling
> one mirror for people downloading the files.

On my sanesecurity pattern mirror, I just blacklist/nullroute those idiots.

-- 
Ralf Hildebrandt   Charite Universitätsmedizin Berlin
ralf.hildebra...@charite.deCampus Benjamin Franklin
https://www.charite.de Hindenburgdamm 30, 12203 Berlin
Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

Re: NOTE: Warning to Abusers of Update Servers

[Anthony Cartmell]

>> I don't suppose you can list the offending IP addresses?
>>
>> Or partially-obfuscated IP addresses?
> 
> While I think it might be helpful to the administrators, I thought it
> might cross the line into publicly shaming people so I can't.

Understood.

Although perhaps listing them with just the least-significant bits, like:

  xxx.xxx.123.456

would give admins enough of a clue to see if they were a culprit without
giving the actual IPs away?

Anthony
-- 
www.fonant.com - Quality web sites
Tel. 01903 867 810
Fonant Ltd is registered in England and Wales, company No. 7006596
Registered office: Amelia House, Crescent Road, Worthing, West Sussex,
BN11 1QR

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 7:11 AM, Anthony Cartmell wrote:

I don't suppose you can list the offending IP addresses?

Or partially-obfuscated IP addresses?


While I think it might be helpful to the administrators, I thought it 
might cross the line into publicly shaming people so I can't.


Regards,

KAM

Re: NOTE: Warning to Abusers of Update Servers

[Anthony Cartmell]

> The Top 14 abusers account for nearly 30% of our update based on
> sampling one mirror for people downloading the files.

I don't suppose you can list the offending IP addresses?

Or partially-obfuscated IP addresses?

Anthony
-- 
www.fonant.com - Quality web sites
Tel. 01903 867 810
Fonant Ltd is registered in England and Wales, company No. 7006596
Registered office: Amelia House, Crescent Road, Worthing, West Sussex,
BN11 1QR

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 4:56 AM, A. Schulze wrote:
I use sa-update to update all channels I use. From what I've seen on 
"sa-update --debug" the default is to test an dns-record.

( "dig 1.4.3.updates.spamassassin.org. txt" for the main channel )
If the DNS answer indicate the same version no HTTP requests are done.
So it SHOULD not hurt to call sa-update more frequently UNLESS one 
have specific scripts that directly connect to the updateservers via 
http.


is this assumtion right?


Yes, that assumption is correct.  Hence why single IPs downloading files 
1 million times from a single mirror is a problem.  And no, that's not 
an exaggeration.


My conjecture is some people have setup minute cron jobs and the 
installation is failing so it is downloading over and over and over.  
It's a complete guess but I appreciate people speaking up.


Regards,

KAM

Re: AW: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 5:20 AM, Martin Puppe wrote:

Twice a day is obviously plenty for normal operation. But when I'm actively 
debugging an issue with sa-update, I might check more often.


Not even going to be a blip on the stats unless you have it automated.  
Don't sweat that!


--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: NOTE: Warning to Abusers of Update Servers

[Kevin A. McGrail]

On 11/21/2017 4:42 AM, Matthew Broadhead wrote:
I have a cron to check once per day.  Is that ok or do you think once 
per week is enough?
Hi Matt, Once per day would be appropriate.  In fact, once an hour would 
be fine.  The check should just hit DNS.


The Top 14 abusers account for nearly 30% of our update based on 
sampling one mirror for people downloading the files.


Conjecture is people have misconfigured systems that are failing and 
grabbing the files over and over and over again with cron jobs


Regards,
KAM

On 21/11/2017 03:04, Kevin A. McGrail wrote:

All,

If you are checking the SpamAssassin updates more than 2x a day, 
expect to be blocked in the very near future.  We have people 
checking literally every minute and we only release rules currently 
1x per day.  There is no need to check this often!


Regards,

KAM

AW: NOTE: Warning to Abusers of Update Servers

[Martin Puppe]

Twice a day is obviously plenty for normal operation. But when I'm actively 
debugging an issue with sa-update, I might check more often.

Regards,
Martin Puppe

-Ursprüngliche Nachricht-
Von: Kevin A. McGrail [mailto:kmcgr...@apache.org]
Gesendet: Dienstag, 21. November 2017 03:05
An: Spamassassin 
Betreff: NOTE: Warning to Abusers of Update Servers

All,

If you are checking the SpamAssassin updates more than 2x a day, expect to be 
blocked in the very near future.  We have people checking literally every 
minute and we only release rules currently 1x per day. There is no need to 
check this often!

Regards,

KAM

--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation Chair Emeritus 
Apache SpamAssassin Project








JAM Software GmbH
Managing Director: Joachim Marder
Am Wissenschaftspark 26 * 54296 Trier * Germany
Phone: +49 (0)651-145 653 -0 * Fax: +49 (0)651-145 653 -29
Commercial register number HRB 4920 (AG Wittlich) http://www.jam-software.com






JAM Software GmbH
Geschäftsführer: Joachim Marder
Am Wissenschaftspark 26 * 54296 Trier * Germany
Tel: 0651-145 653 -0 * Fax: 0651-145 653 -29
Handelsregister Nr. HRB 4920 (AG Wittlich) http://www.jam-software.de

Re: NOTE: Warning to Abusers of Update Servers

[A. Schulze]

Kevin A. McGrail:

If you are checking the SpamAssassin updates more than 2x a day,  
expect to be blocked in the very near future. We have people  
checking literally every minute and we only release rules currently  
1x per day. There is no need to check this often!


I use sa-update to update all channels I use. From what I've seen on  
"sa-update --debug" the default is to test an dns-record.

( "dig 1.4.3.updates.spamassassin.org. txt" for the main channel )
If the DNS answer indicate the same version no HTTP requests are done.
So it SHOULD not hurt to call sa-update more frequently UNLESS one  
have specific scripts that directly connect to the updateservers via  
http.


is this assumtion right?

Andreas

Re: NOTE: Warning to Abusers of Update Servers

[Matthew Broadhead]

I have a cron to check once per day.  Is that ok or do you think once 
per week is enough?


On 21/11/2017 03:04, Kevin A. McGrail wrote:

All,

If you are checking the SpamAssassin updates more than 2x a day, 
expect to be blocked in the very near future.  We have people checking 
literally every minute and we only release rules currently 1x per 
day.  There is no need to check this often!


Regards,

KAM