Re: Scoring Explanation Please

[Denny Jones via users]

 Thank you for the explanation. Makes sense now.



On Wednesday, August 30, 2023 at 02:55:50 PM CDT, Bill Cole 
 wrote:  
 
 On 2023-08-30 at 15:14:15 UTC-0400 (Wed, 30 Aug 2023 19:14:15 + 
(UTC))
Denny Jones via users 
is rumored to have said:

> Hello,
> I have looked high and low and can't find an explanation for 
> multi-level scoring:
> score SCC_CANSPAM_2    3.799    0.001    3.799    0.00
> What does this mean?
> In my simplistic way of doing things I would write this as:
> score SCC_CANSPAM_2 3.799

Try running this:

perldoc Mail::SpamAssassin::Conf


That provides you with a man-like interface for the configuration of 
SpamAssassin, extracted from the Mail::SpamAssassin::Conf perl module. 
Not very far into that document you will find:

        If four valid scores are listed, then the score that is used 
depends
        on how SpamAssassin is being used. The first score is used when 
both
        Bayes and network tests are disabled (score set 0). The second 
score
        is used when Bayes is disabled, but network tests are enabled 
(score
        set 1). The third score is used when Bayes is enabled and 
network
        tests are disabled (score set 2). The fourth score is used when
        Bayes is enabled and network tests are enabled (score set 3).

Very often, you will find the the automated rescoring system will emit 
what looks like a perverse set of scores with the 2 network-enabled 
scores at or near zero. That is an artifact of how rescoring is done 
combined with the fact that network tests are often a distillation of 
other people's recent spam detections.      Essentially a very 'small' 
rule is duplicative of the detection being effectively done by a network 
source.





-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
  

Re: Scoring Explanation Please

[Bill Cole]

On 2023-08-30 at 15:14:15 UTC-0400 (Wed, 30 Aug 2023 19:14:15 + 
(UTC))

Denny Jones via users 
is rumored to have said:


Hello,
I have looked high and low and can't find an explanation for 
multi-level scoring:

score SCC_CANSPAM_2    3.799    0.001    3.799    0.00
What does this mean?
In my simplistic way of doing things I would write this as:
score SCC_CANSPAM_2 3.799


Try running this:

perldoc Mail::SpamAssassin::Conf


That provides you with a man-like interface for the configuration of 
SpamAssassin, extracted from the Mail::SpamAssassin::Conf perl module. 
Not very far into that document you will find:


If four valid scores are listed, then the score that is used 
depends
on how SpamAssassin is being used. The first score is used when 
both
Bayes and network tests are disabled (score set 0). The second 
score
is used when Bayes is disabled, but network tests are enabled 
(score
set 1). The third score is used when Bayes is enabled and 
network

tests are disabled (score set 2). The fourth score is used when
Bayes is enabled and network tests are enabled (score set 3).

Very often, you will find the the automated rescoring system will emit 
what looks like a perverse set of scores with the 2 network-enabled 
scores at or near zero. That is an artifact of how rescoring is done 
combined with the fact that network tests are often a distillation of 
other people's recent spam detections.  Essentially a very 'small' 
rule is duplicative of the detection being effectively done by a network 
source.






--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: Scoring Explanation Please

[David B Funk]

Denny,

If you read the fine manual for the spamassassin configuration file, in section 
for 'score SYMBOLIC_TEST_NAME n.nn [ n.nn n.nn n.nn ]'


You'll see:

   If only one valid score is listed, then that score is always used for a test.

   If four valid scores are listed, then the score that is used depends on how 
SpamAssassin is being used. The first score is used when both Bayes and network 
tests are disabled (score set 0). The second score is used when Bayes is 
disabled, but network tests are enabled (score set 1). The third score is used 
when Bayes is enabled and network tests are disabled (score set 2). The fourth 
score is used when Bayes is enabled and network tests are enabled (score set 3).


So when there are four score values it will use the one relevant to your SA's 
operating condition.


EG: if the rule is senstive to the presence of network type tests, such as 
DNSRBLs, the score can be adjusted accordingly.



On Wed, 30 Aug 2023, Denny Jones via users wrote:


Hello,

I have looked high and low and can't find an explanation for multi-level 
scoring:

score SCC_CANSPAM_2    3.799    0.001    3.799    0.00

What does this mean?

In my simplistic way of doing things I would write this as:

score SCC_CANSPAM_2 3.799

Thanks for helping clear the mud in my mind!

Denny






--
Dave Funk   University of Iowa
 College of Engineering
319/335-5751   FAX: 319/384-05491256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Scoring Explanation Please

[Denny Jones via users]

Hello,
I have looked high and low and can't find an explanation for multi-level 
scoring:
score SCC_CANSPAM_2    3.799    0.001    3.799    0.00
What does this mean?
In my simplistic way of doing things I would write this as:
score SCC_CANSPAM_2 3.799

Thanks for helping clear the mud in my mind!
Denny

Re: Emails from gmail.com bypassing Spamassassin scoring

[Bill Cole]

On 2022-02-07 at 13:43:31 UTC-0500 (Mon, 07 Feb 2022 13:43:31 -0500)
Chad 
is rumored to have said:

> I have been getting numerous emails lately from various gmail.com accounts.   
>They are spam or phishing emails and today I got one that had a subject of 
> RECEIPT 5454 and only a JPG image of an invoice. There was no content in 
> the email.
>
>
>
> It bypassed Spamassassin scoring.  Do you know why or what setting I need 
> to set so EVERY email goes through Spamassassin scoring procedures?
>
>
>
> My email server is:mercury2022.mercuryemail.net
[...]
> Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com 
> [209.85.214.172])
>
> by mercury2022.mercuryemail.net (Postfix) with ESMTPS id 
> A5F7E8043D4A
>
> for ; Mon,  7 Feb 2022 10:44:18 -0500 
> (EST)

OK, so we know that your mail server is running Postfix but not how you've 
integrated SpamAssassin. There are many possibilities, with 2 independent 
attributes:


1. Interface to Postfix:
  a. content_filter setting to pipe mail to a bespoke script (maybe 
distro-provided)
  b. milter (amavis, spamass-milter, mimedefang, etc.)
  c. SMTP Proxy (usually amavis)
  d. FILTER action in an access map to a bespoke script.
  e. NONE: Integrated with a downstream delivery agent (e.g. Dovecot LMTP) or 
MUA.

2. Interface to SA:
  a. Load Mail::SpamAssassin Perl modules and use them directly
  b. Use a spamc binary built from the SA distribution to contact a local spamd 
instance
  c. Use a spamc binary built from the SA distribution to contact a remote 
spamd instance
  d. Use a custom implementation of the spamc protocol to contact a local spamd 
instance
  e. Use a custom implementation of the spamc protocol to contact a remote 
spamd instance
  f. Run the spamassassin script and handle its output.

So, yeah: 30 possible combinations. It is hard to say what is broken without 
knowing how you have SA working when it works. This sort of problem is never 
technically in SpamAssassin itself, as SpamAssassin itself doesn't include any 
software that could act as a gatekeeper.


-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: Emails from gmail.com bypassing Spamassassin scoring

[Chad]

Thank you for responding
You were correct it was the size limit that bypassed the scanning

I created a spamc.conf in the spam assassin folder with the “-s option” and 
increased the scanning size to avoid bypassing on smaller attachments.  




On Feb 7, 2022, at 5:24 PM, David B Funk  wrote:

How big was the message? (attached images can be pretty big).

Depending on the "glue" you use to connect your mail MTA to SA, it may have 
some kind of size restriction.

For example, the 'spamc' client has a 'max-size' parameter (which defaults to 
500KB). Any message larger than that size will not be passed to SA (IE it will 
skip scanning).

Does your MTA log the SA processing? Can you see any logged errors associated 
with that particular message?

On Mon, 7 Feb 2022, Chad wrote:

> All of the other emails that were sent before and after this particular email 
> have the X-Spam-Status and X-spam-Report scoring,
> 
> So Spamassassin was running correctly.
> 
> 
> 
> -Original Message-
> From: Marc 
> Date: Monday, February 7, 2022 at 1:49 PM
> To: Chad , "users@spamassassin.apache.org" 
> 
> Subject: RE: Emails from gmail.com bypassing Spamassassin scoring
> 
>> I have been getting numerous emails lately from various gmail.com
>> accounts.  They are spam or phishing emails and today I got one that
>> had a subject of RECEIPT 5454 and only a JPG image of an invoice.
>> There was no content in the email.
>> 
>> 
>> 
>> It bypassed Spamassassin scoring.  Do you know why or what setting I
>> need to set so EVERY email goes through Spamassassin scoring procedures?
>> 
>> 
> 
> I do not see X-Spam headers[1], so your spamassassin was not working?
> 
> 
> [1]
> X-Spam-Status: No, score=-0.4 required=3.0 tests=ALL_TRUSTED,SPF_NEUTRAL,
>TVD_SPACE_RATIO,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no
>version=3.4.6
> X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
>4422b522-8a2b-4864-9498-4f2d06aca485
> 

-- 
Dave Funk   University of Iowa
 College of Engineering
319/335-5751   FAX: 319/384-05491256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

smime.p7s
Description: S/MIME cryptographic signature

Re: Emails from gmail.com bypassing Spamassassin scoring

[David B Funk]

How big was the message? (attached images can be pretty big).

Depending on the "glue" you use to connect your mail MTA to SA, it may have some 
kind of size restriction.


For example, the 'spamc' client has a 'max-size' parameter (which defaults to 
500KB). Any message larger than that size will not be passed to SA (IE it will 
skip scanning).


Does your MTA log the SA processing? Can you see any logged errors associated 
with that particular message?


On Mon, 7 Feb 2022, Chad wrote:


All of the other emails that were sent before and after this particular email 
have the X-Spam-Status and X-spam-Report scoring,

So Spamassassin was running correctly.



-Original Message-
From: Marc 
Date: Monday, February 7, 2022 at 1:49 PM
To: Chad , "users@spamassassin.apache.org" 

Subject: RE: Emails from gmail.com bypassing Spamassassin scoring


I have been getting numerous emails lately from various gmail.com
accounts.  They are spam or phishing emails and today I got one that
had a subject of RECEIPT 5454 and only a JPG image of an invoice.
There was no content in the email.



It bypassed Spamassassin scoring.  Do you know why or what setting I
need to set so EVERY email goes through Spamassassin scoring procedures?




I do not see X-Spam headers[1], so your spamassassin was not working?


[1]
X-Spam-Status: No, score=-0.4 required=3.0 tests=ALL_TRUSTED,SPF_NEUTRAL,
TVD_SPACE_RATIO,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no
version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
4422b522-8a2b-4864-9498-4f2d06aca485



--
Dave Funk   University of Iowa
 College of Engineering
319/335-5751   FAX: 319/384-05491256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

RE: Emails from gmail.com bypassing Spamassassin scoring

[Marc]

> 
> All of the other emails that were sent before and after this particular
> email have the X-Spam-Status and X-spam-Report scoring,
> 
> So Spamassassin was running correctly.
> 

So something went wrong with this one. It should have headers, maybe some 
communication problem. I have configured the MTA to process the messages anyway 
if spamd is not available. You can also configure to bounce the message with an 
'Temporary unable to process'..

Re: Emails from gmail.com bypassing Spamassassin scoring

[Chad]

smime.p7m
Description: S/MIME encrypted message

RE: Emails from gmail.com bypassing Spamassassin scoring

[Marc]

> I have been getting numerous emails lately from various gmail.com
> accounts.  They are spam or phishing emails and today I got one that
> had a subject of RECEIPT 5454 and only a JPG image of an invoice.
> There was no content in the email.
> 
> 
> 
> It bypassed Spamassassin scoring.  Do you know why or what setting I
> need to set so EVERY email goes through Spamassassin scoring procedures?
> 
> 

I do not see X-Spam headers[1], so your spamassassin was not working?


[1]
X-Spam-Status: No, score=-0.4 required=3.0 tests=ALL_TRUSTED,SPF_NEUTRAL,
TVD_SPACE_RATIO,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no
version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
4422b522-8a2b-4864-9498-4f2d06aca485

Emails from gmail.com bypassing Spamassassin scoring

[Chad]

smime.p7m
Description: S/MIME encrypted message

Re: SPF_NONE scoring

[Dave Warren]

On 2021-11-30 12:24, Greg Troxel wrote:

Lots of people think SPF is silly.  And spammers spamming from a domain
they control can even dkim/dmarc.


Domain based reputation is an extremely powerful tool, but it is only 
useful when you know the actual sender of a message. The benefit isn't 
in blocklisting, it is enabling legitimate mail to get through while you 
can filter more aggressively.


This applies a lot less to smaller operations as you really need a large 
amount of data (and the skills to use it), but even at small scales 
being able to bypass spam filters for mail you know you want is 
incredibly useful, especially when you want mail from a particular 
company even though they use a garbage ESP or service provider that you 
would really rather block.

Re: SPF_NONE scoring

[Byung-Hee HWANG]

Hellow Greg,

Greg Troxel  writes:

> [...]
> Lots of people think SPF is silly.  And spammers spamming from a domain
> they control can even dkim/dmarc.   So I agree that actual data would be
> interesting.

I totally agree with you, thanks!

Sincerely, Byung-Hee

-- 
^고맙습니다 _地平天成_ 감사합니다_^))//

Re: SPF_NONE scoring

[Bill Cole]

On 2021-11-30 at 13:47:36 UTC-0500 (Tue, 30 Nov 2021 11:47:36 -0700)
Philip Prindeville 
is rumored to have said:


Hi,

I'm looking at the 0.001 scoring for SPF_NONE and scratching my head.  
This was discussed a bit in early 2015, but maybe it needs revisiting 
with new perspective.


Surely no one who cares about maintaining their reputation by 
protecting themselves against spoofing would fail to provide SPF 
records...


Surely no one who cares about the security of their email would run 
their own on-premises Exchange...


Having started my sysadmin career less than 30 years ago, I never have 
been exposed to an Internet where the dominant visible feature of my 
fellow admins has been operational competence. We're all a bunch of 
bozos making stupid mistakes...



So how is this score arrived at?


In theory, it is set in concert with all of the other default rules by 
periodic analyses of the scoring of spam and ham corpora submitted by 
members of the SA community. As a 'network' rule, it is only included in 
analysis weekly.


In practice, it is nailed down at a tiny non-zero value because 
otherwise it would not be "good enough" to publish and demand has been 
expressed for its publication.



And of Ham, how much of it has a valid SPF?


Recently: 90.1202%


And of Spam, how much of it lacks a valid SPF?


Recently: 65.3614%


Has anyone run some numbers?


Yes. See https://ruleqa.spamassassin.org/. The numbers above are drawn 
from the last "network masscheck" accessible there.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: SPF_NONE scoring

[Loren Wilton]

So how is this score arrived at?


I believe that scores of 0.001 are generally manually set, and not intended 
to be anything other than a visible marker that the rule hit. That is 
probably the case here.


   Loren

Re: SPF_NONE scoring

[Matija Nalis]

On Tue, Nov 30, 2021 at 11:47:36AM -0700, Philip Prindeville wrote:
> I'm looking at the 0.001 scoring for SPF_NONE and scratching my head.  This 
> was discussed a bit in early 2015, but maybe it needs revisiting with new 
> perspective.

SPF is double edged sword. Sure, when it great to authenticate
envelope senders when it works, but:

- when used in combination with mailing list, plain message
  forwarding etc. it will break with false positive, marking
  (for example) this perfectly valid message of mine as a fake.
  See https://en.wikipedia.org/wiki/Sender_Policy_Framework#FAIL_and_forwarding

  This is the reason why you can only really use it for "SPF OK"
  validation - "SPF FAIL" does not really tell you anything, as it
  will happen as often for forged senders, as for valid senders.

  This is why it will often end as "?all" or "~all" and not "-all"
  (and/or soft DMARC policies)

- Also, envelope sender (on which SPF operates) is something
  completely different thing from header "From:" which is what vast
  majority of users will see, so it does not provide protection which
  one might expect.
  See https://en.wikipedia.org/wiki/Sender_Policy_Framework#Header_limitations

  And this makes "SPF OK" much less useful then it sounds in theory.

- Then there are misconfigurations (hitting limit of max 10 DNS
  lookups, SPF records which were setup once but not kept up-to-date,
  etc).

Thus, SPF is IMHO not very usable for scoring on its own, but it does
have a useful purpose for creating custom SA rules and is often very
usable for short circuiting with whitelist_auth.

> Surely no one who cares about maintaining their reputation by protecting 
> themselves against spoofing would fail to provide SPF records...  

For example, I do not provide it on my few other e-mail accounts by
choice (especially most of them which deal with many mailing lists,
or with users which use non-SRS e-mail forwarding), as mere existence
of SPF there causes much more damage then the potential help it
brings.

> So how is this score arrived at?

That, I am not sure. Perhaps how well it is an indicator on
ham/spam corpuses run to determine scores in general in SA? 

> And of Ham, how much of it has a valid SPF?

For my recent hams, I get this:

714 SPF_PASS=
128 SPF_NONE=
 67 SPF_NEUTRAL_ALL=
  9 SPF_FAIL=
  1 SPF_SOFTFAIL=

So, about 1 message in 7 hams does not have SPF.

> And of Spam, how much of it lacks a valid SPF?

For recent spams that reach any kind of mailbox here (eg. not
hitting very-safe RBLs, and not having very high SA scores - ie. 
having at least a minimum of potential for being misclassified
non-spam):

   2291 SPF_PASS=
667 SPF_SOFTFAIL=
472 SPF_NONE=
353 SPF_FAIL=
154 SPF_NEUTRAL_ALL=
129 SPF_PERMERROR=
 53 SPF_NEUTRAL=
 17 SPF_TEMPERROR=

So, about 1 message in 9 spams does not have SPF.

In summary, there does not seem to be big difference between
adoption of SPF in spammers as opposed to legitimate users

-- 
Opinions above are GNU-copylefted.

Re: SPF_NONE scoring

[Greg Troxel]

Philip Prindeville  writes:

> I'm looking at the 0.001 scoring for SPF_NONE and scratching my head.  This 
> was discussed a bit in early 2015, but maybe it needs revisiting with new 
> perspective.
>
> Surely no one who cares about maintaining their reputation by
> protecting themselves against spoofing would fail to provide SPF
> records...  So how is this score arrived at?
>
> And of Ham, how much of it has a valid SPF?
>
> And of Spam, how much of it lacks a valid SPF?
>
> Has anyone run some numbers?

I see 0.001 as a score that says: this might be a spam sign, we don't
know, and this way it shows up in reports, without really affecting
anything.

Lots of people think SPF is silly.  And spammers spamming from a domain
they control can even dkim/dmarc.   So I agree that actual data would be
interesting.


signature.asc
Description: PGP signature

SPF_NONE scoring

[Philip Prindeville]

Hi,

I'm looking at the 0.001 scoring for SPF_NONE and scratching my head.  This was 
discussed a bit in early 2015, but maybe it needs revisiting with new 
perspective.

Surely no one who cares about maintaining their reputation by protecting 
themselves against spoofing would fail to provide SPF records...  So how is 
this score arrived at?

And of Ham, how much of it has a valid SPF?

And of Spam, how much of it lacks a valid SPF?

Has anyone run some numbers?

Thanks,

-Philip

Re: Scoring for "look alike" characters in subject?

[Kevin A. McGrail]

Hi Steve,

There are many rules that look at this.  The FUZZY Logic rules might help
and in the KAM ruleset, you'll see replace_tag lines and how they are used
in various places to shutdown spammers used to obfuscate words by using
other character sets and symbols.  You can find the KAM.cf ruleset on
mcgrail.com under downloads and there is an SA Channel for it as well.

regards,
KAM
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Mon, Mar 15, 2021 at 6:59 AM Steve Dondley  wrote:

> I'm noticing a fair amount of spam getting through using letters in the
> subject line that are outside the standard set of ASCII characters in an
> effort to bypass spam filters. For example, instead of a capital "R",
> there will be a letter that closely approximates a capital "R" but when
> you look closely at it, you'll see the bottom of the rounded part of the
> "R" never connects to the line running along the left side of the
> letter.
>
> I don't want to use a rule that is too-restrictive (like maybe banning
> all non-standard ascii characters) but I also want to increase the
> likelihood of email using these tactics getting flagged as spam.
>
> I'm new to spamasssassin so I'm not sure if a rule like this already
> exists or how I might go about finding this rule or what I should weight
> it. I'm wondering if others on the list have rules to address this same
> issue and can share their rule. Thanks.
>

Scoring for "look alike" characters in subject?

[Steve Dondley]

I'm noticing a fair amount of spam getting through using letters in the 
subject line that are outside the standard set of ASCII characters in an 
effort to bypass spam filters. For example, instead of a capital "R", 
there will be a letter that closely approximates a capital "R" but when 
you look closely at it, you'll see the bottom of the rounded part of the 
"R" never connects to the line running along the left side of the 
letter.


I don't want to use a rule that is too-restrictive (like maybe banning 
all non-standard ascii characters) but I also want to increase the 
likelihood of email using these tactics getting flagged as spam.


I'm new to spamasssassin so I'm not sure if a rule like this already 
exists or how I might go about finding this rule or what I should weight 
it. I'm wondering if others on the list have rules to address this same 
issue and can share their rule. Thanks.

Re: Scoring Based on IP Address

[John Hardin]

On Fri, 18 Dec 2020, @lbutlr wrote:


On 17 Dec 2020, at 16:19, Dave Wreski  wrote:

On 12/17/20 6:05 PM, Matt wrote:

Is there a way with spamassassin local.conf to add a higher score
based on source ip address or subnet?  Basically the last IP in
"Received:" header.
bad_subnet_add_20_points: 192.168.240.0/24
Raising the score if that IP appeared anywhere in headers or body
might work too.



Yes, but if you're effectively going to create a "poison pill" rule where any 
mail from a particular network is quarantined, you might be better of doing this at the 
firewall or in postfix directly and just rejecting it outright.

header __BAD_IP_RCVD  Received  =~ /192\.168\.240\.\d{1,3}/
body   __BAD_IP_BODY /192\.168\.240\.\d{1,3}/
rawbody __BAD_IP_RAWBODY /192\.168\.240\.\d{1,3}/
meta MY_BAD_SENDER __BAD_IP_RCVD || __BAD_IP_BODY || __BAD_IP_RAWBODY
score MY_BAD_SENDER 20
describe MY_BAD_SENDER Contains bad IP


Won't this match for that IP in ANY Received: header?


Yes. That's "deep inspection", and runs the risk of a hit on a legitimate 
"bad" IP in the sender's local network (assuming their MTA records the 
initial submission).


It would be better to check the last external IP in X-Spam-Relays-External:

  header __EXT_MTA_IP_BAD  X-Spam-Relays-External =~ /^\[ ip=192\.168\.240\.\d+ 
/


And, as Dave said, if you're going to poison pill based on the external 
MTA's IP address, then do it with an MTA IP rule or at the firewall, it's 
a lot easier (and lighter-weight) than all this SA stuff.


For example, in /etc/mail/access (for sendmail):

  93.159.212.159550 5.7.1 Spammed a mailing list - go away.
  65.49.16.2550 5.7.1 Open relay - go away.
  202.65.168.39 550 5.7.1 Seven 419 spams in one hour - go away.
  213.171.44.75 550 5.7.1 Open relay - email worms - go away.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.org pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
  does quite what I want. I wish Christopher Robin was here."
   -- Peter da Silva in a.s.r
---
 7 days until Christmas

Re: Scoring Based on IP Address

[@lbutlr]

On 17 Dec 2020, at 16:19, Dave Wreski  wrote:
> On 12/17/20 6:05 PM, Matt wrote:
>> Is there a way with spamassassin local.conf to add a higher score
>> based on source ip address or subnet?  Basically the last IP in
>> "Received:" header.
>> bad_subnet_add_20_points: 192.168.240.0/24
>> Raising the score if that IP appeared anywhere in headers or body
>> might work too.

> Yes, but if you're effectively going to create a "poison pill" rule where any 
> mail from a particular network is quarantined, you might be better of doing 
> this at the firewall or in postfix directly and just rejecting it outright.
> 
> header __BAD_IP_RCVD  Received  =~ /192\.168\.240\.\d{1,3}/
> body   __BAD_IP_BODY /192\.168\.240\.\d{1,3}/
> rawbody __BAD_IP_RAWBODY /192\.168\.240\.\d{1,3}/
> meta MY_BAD_SENDER __BAD_IP_RCVD || __BAD_IP_BODY || __BAD_IP_RAWBODY
> score MY_BAD_SENDER 20
> describe MY_BAD_SENDER Contains bad IP

Won't this match for that IP in ANY Received: header?

-- 
"How good bad music and bad reasons sound when we march against an
enemy." -  Friedrich Nietzsche

Re: Scoring Based on IP Address

[Dave Wreski]

Hi,

On 12/17/20 6:05 PM, Matt wrote:

Is there a way with spamassassin local.conf to add a higher score
based on source ip address or subnet?  Basically the last IP in
"Received:" header.

bad_subnet_add_20_points: 192.168.240.0/24

Raising the score if that IP appeared anywhere in headers or body
might work too.


Yes, but if you're effectively going to create a "poison pill" rule 
where any mail from a particular network is quarantined, you might be 
better of doing this at the firewall or in postfix directly and just 
rejecting it outright.


header __BAD_IP_RCVD  Received  =~ /192\.168\.240\.\d{1,3}/
body   __BAD_IP_BODY /192\.168\.240\.\d{1,3}/
rawbody __BAD_IP_RAWBODY /192\.168\.240\.\d{1,3}/
meta MY_BAD_SENDER __BAD_IP_RCVD || __BAD_IP_BODY || __BAD_IP_RAWBODY
score MY_BAD_SENDER 20
describe MY_BAD_SENDER Contains bad IP

Regards,
Dave

Scoring Based on IP Address

[Matt]

Is there a way with spamassassin local.conf to add a higher score
based on source ip address or subnet?  Basically the last IP in
"Received:" header.

bad_subnet_add_20_points: 192.168.240.0/24

Raising the score if that IP appeared anywhere in headers or body
might work too.

Re: Screwed-up scoring

[Linkcheck]

I read the thread. I didn't comment because it was obvious the rationals 
would lose and the unnecessary changes would go ahead. From that 
discussion I took away the thought that I had a long-ish breathing space 
which would allow me to update my complete mail server - OS, Postfix and 
all - and get rid of the now-likely-to-break spamassassin. I did not 
expect it to break within a few days!


I wonder how this fiasco will affect all those who do not audit this 
list - surely a large number? Or any who, due to the SPF failure 
yesterday, missed some of the list?


This whole affair has been badly mis-managed. No engineer should have 
behaved in this cavalier fashion for such a spurious, mis-informed 
reason and with such a short change-over perdiod.

Re: Screwed-up scoring

[Linkcheck]

Whether or not it's the ONLY one it should have been NONE. You claimed 
we would not have to change anything for at least a year - as I 
understodd it. Certainly you should not have broken existing installations!


I am running 3.4.2, dictated by my OS. I am quite happy running that 
version - at least, I was before the speciously argued changed that 
broke it.


What about all the other whitelist and blacklist nomenclature in the 
(pre-broken) version? I have altered scores for those below. Are they 
also broken? Or any other that I may have missed?


priority USER_IN_WHITELIST
priority USER_IN_DEF_WHITELIST
priority USER_IN_ALL_SPAM_TO
priority USER_IN_DKIM_WHITELIST
priority USER_IN_DEF_DKIM_WL
priority USER_IN_SPF_WHITELIST
priority USER_IN_DEF_SPF_WL
priority USER_IN_BLACKLIST
priority USER_IN_BLACKLIST_TO
shortcircuit USER_IN_WHITELIST
shortcircuit USER_IN_DEF_WHITELIST
shortcircuit USER_IN_ALL_SPAM_TO
shortcircuit SUBJECT_IN_WHITELIST
shortcircuit USER_IN_DKIM_WHITELIST
shortcircuit USER_IN_DEF_DKIM_WL
shortcircuit USER_IN_SPF_WHITELIST
shortcircuit USER_IN_DEF_SPF_WL
shortcircuit USER_IN_BLACKLIST
shortcircuit USER_IN_BLACKLIST_TO
shortcircuit SUBJECT_IN_BLACKLIST

Re: Screwed-up scoring

[Matus UHLAR - fantomas]

On Sunday 19 July 2020 at 17:44:27, Linkcheck wrote:

Thanks to those responsible for screwing up the scoring of my
spamassassin installation. It's been working well for years but now my
changes to scoring have been cancelled due to renaming
whitelist/blacklist to whatever.

I noticed it purely by accident this morning: USER_IN_WHITELIST_TO no
longer gave me the expected score because it has now been replaced by
USER_IN_WELCOMELIST_TO.


On 19.07.20 18:00, Antony Stone wrote:

I think you must quite possibly be the only person on this list who has not
noticed the 223 emails containing "IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK"
in the subject line over the past 9 days discussing precisely this change.


maybe he is just not running trunk and so ignored tha thread.
Or, maybe he expected tohe change only to take place in trunk.

any way, I would also expect this change happen after new SA release, or at
least after announce different than "... running trunk ..."

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Save the whales. Collect the whole set.

Re: Screwed-up scoring

[Martin Gregorie]

On Sun, 2020-07-19 at 20:27 -0400, Kevin A. McGrail wrote:
> On 7/19/2020 8:23 PM, Martin Gregorie wrote:
> > The only way I can see to prevent the name changes from affecting SA
> > users private rules is to duplicate the affected rules
> 
> Yeah, I just posted this idea on the dev list to use a meta like this
> which I think will allow it to work backwards to 3.3.x. Will that work
> for your install?
> 
Your suggested workround should work here although, because my private
rules don't reference any standard ruleset rules with names containing
'BLACKLIST' or 'WHITELIST', I'm not affected by these name changes:
thats pure luck.

Your idea is neater than my suggestion because it can't mess up private
rules that make use of numeric score values. However, both workrounds
will, I suspect, make standard rule maintenance more complex. What about
maintaining one format in ruleQA output and including a configurable
rule name conversion step in the rules update process? If that was
controlled by a new local.cf directive it should be a pretty small code
change.

Martin

Re: Screwed-up scoring

[jdow]

On 20200719 15:44:54, Luis E. Muñoz wrote:

On 19 Jul 2020, at 10:54, Kevin A. McGrail wrote:


Great question.  That's really a third party rule.  I would like to see it
change eventually but maybe that's another phase.  Thoughts?


My thoughts are to delay any further social/political motivated name changes 
until after the extents of the current process are fully completed and 
understood. At that time, I would also suggest bringing in the opinions of the 
people who will bear the larger extent of the implementation – the users 
themselves – as well as the allegedly aggravated people on whose behalf you seem 
to favor the change.


Best regards

-lem


There is something to be said for running SL 7.x rather than something fancy and 
newer. I get to see other people have problems and have plenty of time to deal 
with them before they hit me.


{^_-}

Re: Screwed-up scoring

[Kevin A. McGrail]

On 7/19/2020 8:23 PM, Martin Gregorie wrote:
> The only way I can see to prevent the name changes from affecting SA
> users private rules is to duplicate the affected rules

Yeah, I just posted this idea on the dev list to use a meta like this
which I think will allow it to work backwards to 3.3.x. Will that work
for your install?

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
  #bz7826 renames whitelist to welcomelist
  header USER_IN_WELCOMELIST_TO eval:check_to_in_welcomelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'welcomelist_to'
  tflags USER_IN_WELCOMELIST_TO userconf nice noautolearn
  score USER_IN_WELCOMELIST_TO  -6.0
else
  header USER_IN_WELCOMELIST_TO eval:check_to_in_whitelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'welcomelist_to'
  tflags USER_IN_WELCOMELIST_TO userconf nice noautolearn
  score USER_IN_WELCOMELIST_TO  -0.01

  meta USER_IN_WHITELIST_TO (USER_IN_WELCOMELIST_TO)
  describe USER_IN_WHITELIST_TO DEPRECATED: See
USER_IN_WELCOMELIST_TO
  tflags USER_IN_WHITELIST_TO   userconf nice noautolearn
  score USER_IN_WHITELIST_TO    -6.0
endif

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Screwed-up scoring

[Martin Gregorie]

On Sun, 2020-07-19 at 15:44 -0700, Luis E. Muñoz wrote:
> On 19 Jul 2020, at 10:54, Kevin A. McGrail wrote:
> 
> > Great question.  That's really a third party rule.  I would like to 
> > see it
> > change eventually but maybe that's another phase.  Thoughts?
> 
The only way I can see to prevent the name changes from affecting SA
users private rules is to duplicate the affected rules: one copy using
BLACKLIST/WHITELIST and the other using BANNEDLIST/WELCOMELIST and,
since both copies will fire, with their scores halved. This will allow
private rules to work as normal until the BLACK/WHITE rule names are
removed from the standard set: the overall score for a message will
remain unchanged.

The above should solve the problem for cases (the majority?) where the
private rules only care whether subrules fire or not. However, if
anybody's private rules compare subrule score values then the private
rules may fail completely unless rewritten.

Martin

Re: Screwed-up scoring

[Luis E. Muñoz]

On 19 Jul 2020, at 10:54, Kevin A. McGrail wrote:

Great question.  That's really a third party rule.  I would like to 
see it

change eventually but maybe that's another phase.  Thoughts?


My thoughts are to delay any further social/political motivated name 
changes until after the extents of the current process are fully 
completed and understood. At that time, I would also suggest bringing in 
the opinions of the people who will bear the larger extent of the 
implementation – the users themselves – as well as the allegedly 
aggravated people on whose behalf you seem to favor the change.


Best regards

-lem

Re: Screwed-up scoring

[Kevin A. McGrail]

Great question.  That's really a third party rule.  I would like to see it
change eventually but maybe that's another phase.  Thoughts?

On Sun, Jul 19, 2020, 13:17 Martin Gregorie  wrote:

> On Sun, 2020-07-19 at 11:59 -0400, Kevin A. McGrail wrote:
>
> > Whitelist will become welcomelist and blacklist will become
> > blocklist. Are you running a modern SA like 3.4.4?  If so, you should
> > be able to proactively add entries for this.
> >
> Just been grepping my local rules for WHITELIST and BLACKLIST without
> finding any that none are affected by those changes.
>
> Then I also grepped them for WHITE and BLACK and this time I saw that
> two of my local rules reference the standard URIBL_BLACK rule. Is this
> name likely to change?
>
> Martin
>
>
>

Re: Screwed-up scoring

[Martin Gregorie]

On Sun, 2020-07-19 at 11:59 -0400, Kevin A. McGrail wrote:

> Whitelist will become welcomelist and blacklist will become
> blocklist. Are you running a modern SA like 3.4.4?  If so, you should
> be able to proactively add entries for this.
> 
Just been grepping my local rules for WHITELIST and BLACKLIST without
finding any that none are affected by those changes.

Then I also grepped them for WHITE and BLACK and this time I saw that
two of my local rules reference the standard URIBL_BLACK rule. Is this
name likely to change?

Martin

Re: Screwed-up scoring

[Antony Stone]

On Sunday 19 July 2020 at 17:44:27, Linkcheck wrote:

> Thanks to those responsible for screwing up the scoring of my
> spamassassin installation. It's been working well for years but now my
> changes to scoring have been cancelled due to renaming
> whitelist/blacklist to whatever.
> 
> I noticed it purely by accident this morning: USER_IN_WHITELIST_TO no
> longer gave me the expected score because it has now been replaced by
> USER_IN_WELCOMELIST_TO.

I think you must quite possibly be the only person on this list who has not 
noticed the 223 emails containing "IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK" 
in the subject line over the past 9 days discussing precisely this change.

I sympathise with you - I really do - I do not agree with the changes which 
have occurred, the reasons for them, or the lack of discussion with the 
community before they were implemented, but I find the fact that you haven't 
noticed they have already been done and have been announced here quite 
remarkable.

> Can someone post a list of ALL the new names, with their originals, please?

Excellent request - I'm surprised that the powers-that-be who have implemented 
these changes haven't simply done this as a matter of course.

I see no mention of such a list in the bug report (how ironic that a bug 
report gets filed to announce the introduction of a bug into the software...) 
which was quoted in the original announcement of this fait accompli to the 
list:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7826


Regards,


Antony.

-- 
There's no such thing as bad weather - only the wrong clothes.

 - Billy Connolly

   Please reply to the list;
 please *don't* CC me.

Re: Screwed-up scoring

[Kevin A. McGrail]

On 7/19/2020 11:44 AM, Linkcheck wrote:
> Can someone post a list of ALL the new names, with their originals,
> please?

The only functionality changed so far is WHITELIST_TO which is now
WELCOMELIST_TO in the configuration options with backwards compatibility.

The stock rule that was USER_IN_WHITELIST_TO is now USER_IN_WELCOMELIST_TO

Whitelist will become welcomelist and blacklist will become blocklist. 
Are you running a modern SA like 3.4.4?  If so, you should be able to
proactively add entries for this.

Regards,

KAM

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Screwed-up scoring

[Linkcheck]

Thanks to those responsible for screwing up the scoring of my 
spamassassin installation. It's been working well for years but now my 
changes to scoring have been cancelled due to renaming 
whitelist/blacklist to whatever.


I noticed it purely by accident this morning: USER_IN_WHITELIST_TO no 
longer gave me the expected score because it has now been replaced by 
USER_IN_WELCOMELIST_TO. Great. I now have to dredge up some time from 
somewhere to change all the other scores that have been messed up, with 
only the vaguest clue as to what the names are likely to be.


Can someone post a list of ALL the new names, with their originals, please?

Re: Loads of recent low-scoring snowshoe spam

[John Hardin]

On Thu, 26 Sep 2019, Amir Caspi wrote:


On Sep 26, 2019, at 10:18 AM, John Hardin  wrote:


Some of those are following a pattern I've recently noticed - fairly obviously 
bogus spamvertising domain URLs with some .gov URLs thrown in as well. I'm 
assuming that's an attempt to leverage naïve domain whitelisting. One has a 
Humane Society URL, I presume the goal is similar.


Although they may not be in the spamples I provided, I've also seen .edu links.


Yeah, I'm starting to see those too. Added __URI_DOTEDU to see what it's 
worth.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Our politicians should bear in mind the fact that the American
  Revolution was touched off by the then-current government
  attempting to confiscate firearms from the people.
---
 3 days until the 78th anniversary of the massacre at Babi Yar
 Disarmament enables genocide - Registration enables disarmament

Re: Loads of recent low-scoring snowshoe spam

[Amir Caspi]

On Sep 26, 2019, at 10:18 AM, John Hardin  wrote:
> 
> Some of those are following a pattern I've recently noticed - fairly 
> obviously bogus spamvertising domain URLs with some .gov URLs thrown in as 
> well. I'm assuming that's an attempt to leverage naïve domain whitelisting. 
> One has a Humane Society URL, I presume the goal is similar.

Although they may not be in the spamples I provided, I've also seen .edu links. 
 And in today's spam I got a .gov.on.ca <http://gov.on.ca/> link.  So we might 
need some variants, but then again, I suspect these will require a lot of 
tuning to guard against FPs.

My new AC_ rules (particularly AC_LARGE_INDENT and AC_POST*EXTRAS) do really 
well locally, but not so much in masscheck ... but they hit otherwise very 
low-scoring spam.  I would request that someone more talented than I am look at 
tuning those against FPs, if they are willing...

Cheers.

--- Amir

Re: Loads of recent low-scoring snowshoe spam

[John Hardin]

On Wed, 25 Sep 2019, Amir Caspi wrote:


Just a few (of many) spamples here:
https://pastebin.com/wRFBSCEZ
https://pastebin.com/FUdFEdhT
https://pastebin.com/LkqSEdAh


Some of those are following a pattern I've recently noticed - fairly 
obviously bogus spamvertising domain URLs with some .gov URLs thrown in as 
well. I'm assuming that's an attempt to leverage naïve domain 
whitelisting. One has a Humane Society URL, I presume the goal is similar.


I added __URI_DOTGOV but the performance isn't that great at the moment. I 
expect the masscheck corpora aren't seeing a lot of these (yet?). It's 
possible some of the DOTGOV combinations would work better in the Real 
World than they currently are in masschecks...



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Are you a mildly tech-literate politico horrified by the level of
  ignorance demonstrated by lawmakers gearing up to regulate online
  technology they don't even begin to grasp? Cool. Now you have a
  tiny glimpse into a day in the life of a gun owner.   -- Sean Davis
---
 3 days until the 78th anniversary of the massacre at Babi Yar
 Disarmament enables genocide - Registration enables disarmament

Loads of recent low-scoring snowshoe spam

[Amir Caspi]

Hi all,

In recent weeks, my server has been getting hit with tons of snowshoe spam.  
Much of it is not getting filtered because even when it hits Bayes, it doesn't 
hit basically any other rules, and therefore is scoring just below 5 points.  
(Much of it hits only BAYES_50 and is therefore scoring even lower.)

Does anyone have any rules that can help hit these spams?  It seems like none 
of the default rules, nor KAM.cf, nor nonKAMrules.cf, are hitting these.  
Sometimes I'm lucky and Razor/DCC/Pyzor and/or URIBLs have already picked them 
up, throwing their score over threshold... but often I'm at the beginning of 
the queue and none of the hashes/BLs have gotten them yet.

(Reporting to SpamCop, it seems that almost all of this spam from today is 
coming from relays owned by sourcedns / liquidweb, and references URIs hosted 
by losangelesdedicated... although yesterday's spam came from a Romanian relay 
with URIs hosted by versaweb / fiberhub, so obviously there's no long-term 
pattern to the sources.)

Just a few (of many) spamples here:
https://pastebin.com/wRFBSCEZ
https://pastebin.com/FUdFEdhT
https://pastebin.com/LkqSEdAh

I've been testing some custom rules which are doing very well locally but which 
seem to have a high FP rate on masscheck, so would need some tuning before 
being included in the default rules, and I unfortunately haven't had time to do 
this tuning.  (If anyone wants to take a stab at it... the custom rules are 
AC_LOW_OPACITY, AC_POSTHTML_EXTRAS, AC_POSTIMG_EXTRAS, and AC_LARGE_INDENT. 
There is also AC_TINY_FONT but that seems to FP all over the place.)

Thanks in advance for any ideas/help... these have been really annoying my 
users.

Cheers.

--- Amir

Re: Scoring TLS.

[@lbutlr]

On 6 Sep 2019, at 14:37, @lbutlr  wrote:
> I do need to go through the logs again at some point and see how things are 
> shaping up. It would be interesting to see what the server-to-server 
> encryption looks like now for valid mail. I suspect that 1.1 has dropped to 
> near 0 and 1.0 is more spam than it was, but that’s just a guess.

I ran a quick check and less than 1% of my secure connections (700 out of 
74,000) are using TLSv1 instead of TLSv1.2, and more than half of those are 
from list servers. The rest are mostly unknown with a few named like 
blackboard,bet, shoran.io, and admiral.net.

I’m not blocking TLSv1 servers at this, but I am certainly considering adding a 
point or so in SA. That will not affect the mailing lists at all, but it might 
catch some of the other garbage.





-- 
LOOSE TEETH DON'T NEED MY HELP Bart chalkboard Ep. AABF16

Re: Scoring TLS.

[@lbutlr]

On 6 Sep 2019, at 14:14, Matus UHLAR - fantomas  wrote:
 TLSv1.0 is EOLed and should not be used nor supported.
> 
>> On 6 Sep 2019, at 01:57, Matus UHLAR - fantomas  wrote:
>>> well, if your clients (some old server installations) only support tls1.0, 
>>> it's better to allow it than forgint it to go plaintext or reject the mail 
>>> at all.
> 
>>> On 06.09.19 00:57, @lbutlr wrote:
>> I don’t agree. It is thinking like this that leads to people still wanting 
>> to use RC4-SHA or HTTP AUTH.
> 
> the alternative on server-server connection is no encryption at all.

Which is still going to be the case for a still significant percentage of 
connections. Used a deprecated end-of-life security shouldn’t be encouraged.

>>> http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html
> 
> On 06.09.19 11:50, @lbutlr wrote:
>> That is four years ago and largely covers maintaining support for the 16 
>> year-old Exchange 2003.
> 
> did tou intentionally skip the link that was an update to this one and only 
> one year old to blame me for the older one?

Of course not, the second one was a followup to the first one, which again was 
largely about Exchange 2003, so I didn’t think it really added anything and it 
was also still before the EOL for TLSv1.0.

>> The difference right now is that TLSv1.0 is end-of-life and has known flaws. 
>>  It should no more be used than MD5 or RC2.
>> 
>> However, I think here we were talking about TLS connections from sending 
>> servers; there TLSv1.0 is already basically unused.  You are more likely to 
>> not get an opportunistic encryption at all that TLSv1.
> 
> I'd be happy to see any statistics about this. Possibly in postfix list, if
> you can…

Your logs will be different than mine, I am sure. When last I checked for 
successfully submitted mails, unencrypted was more common that TLSv1.0, and 
that was … spring?

>51 version=TLSv1,
> 8 version=TLSv1.1,
>   539 version=TLSv1.2,
>92 version=TLSv1.3,

Most of my TLSv1 were connections that were rejected for high degrees of 
spammishness.

I do need to go through the logs again at some point and see how things are 
shaping up. It would be interesting to see what the server-to-server encryption 
looks like now for valid mail. I suspect that 1.1 has dropped to near 0 and 1.0 
is more spam than it was, but that’s just a guess.



-- 
'We get that in here some nights, when someone's had a few. Cosmic
speculation about whether the gods exist. Next thing, there's a bolt of
lightning through the door with a note wrapped round it saying, "Yes, we
do" and a pair of sandals with smoke coming out.' (Small Gods)

Re: Scoring TLS.

[Matus UHLAR - fantomas]

TLSv1.0 is EOLed and should not be used nor supported.



On 6 Sep 2019, at 01:57, Matus UHLAR - fantomas  wrote:

well, if your clients (some old server installations) only support tls1.0, it's 
better to allow it than forgint it to go plaintext or reject the mail at all.



On 06.09.19 00:57, @lbutlr wrote:

I don’t agree. It is thinking like this that leads to people still wanting to 
use RC4-SHA or HTTP AUTH.


the alternative on server-server connection is no encryption at all.


http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html


On 06.09.19 11:50, @lbutlr wrote:

That is four years ago and largely covers maintaining support for the 16 
year-old Exchange 2003.


did tou intentionally skip the link that was an update to this one and only
one year old to blame me for the older one?


The difference right now is that TLSv1.0 is end-of-life and has known
flaws.  It should no more be used than MD5 or RC2.

However, I think here we were talking about TLS connections from sending
servers; there TLSv1.0 is already basically unused.  You are more likely
to not get an opportunistic encryption at all that TLSv1.


I'd be happy to see any statistics about this. Possibly in postfix list, if
you can...

mine logs for seven weeks (since I upgraded to debian 10) say:

for the server side:

51 version=TLSv1,
 8 version=TLSv1.1,
   539 version=TLSv1.2,
92 version=TLSv1.3,

these are unique IP/version counts

for the client side:

 1 version=TLSv1,
 1 version=TLSv1.1,
24 version=TLSv1.2,
 5 version=TLSv1.3,

and there are unique server name/version counts

...(sorry) I don't exchange mail with too many sites on this server
maybe I could do more statistics at work. 


seems that betwen the tlsv1 sites is postfix-users mailing list ;-)


On 6 Sep 2019, at 00:51, Reio Remma  wrote:

I recently did an experiment where I stopped accepting incoming e-mail
without TLS.  This seemingly cut off about 95-99% of spam.  Unfortunately
there still seem to be a small percentage of servers sending without TLS,
so that was a no go.



I took that to mean the OP was not talking about submission from clients,
but incoming mail from other servers.


so did I. I don't allow submission clients to use weak encryption, unless
they really need to allow that.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Honk if you love peace and quiet.

Re: Scoring TLS.

[@lbutlr]

On 6 Sep 2019, at 01:57, Matus UHLAR - fantomas  wrote:
> On 06.09.19 00:57, @lbutlr wrote:
>> TLSv1.0 is EOLed and should not be used nor supported.
> 
> well, if your clients (some old server installations) only support tls1.0, 
> it's better to allow it than forgint it to go plaintext or reject the mail at 
> all.

I don’t agree. It is thinking like this that leads to people still wanting to 
use RC4-SHA or HTTP AUTH.

> http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html

That is four years ago and largely covers maintaining support for the 16 
year-old Exchange 2003.

The difference right now is that TLSv1.0 is end-of-life and has known flaws. It 
should no more be used than MD5 or RC2.

However, I think here we were talking about TLS connections from sending 
servers; there TLSv1.0 is already basically unused. You are more likely to not 
get an opportunistic encryption at all that TLSv1.

On 6 Sep 2019, at 00:51, Reio Remma  wrote:
> I recently did an experiment where I stopped accepting incoming e-mail 
> without TLS. This seemingly cut off about 95-99% of spam. Unfortunately there 
> still seem to be a small percentage of servers sending without TLS, so that 
> was a no go.


I took that to mean the OP was not talking about submission from clients, but 
incoming mail from other servers.



-- 
The trouble with being a god is that you've got no one to pray to.

Re: Scoring TLS.

[John Hardin]

On Fri, 6 Sep 2019, Reio Remma wrote:


Does the Received check only check the last untrusted relay?


No, the named header checks test all the headers having that name 
(presuming there are multiple present).


If you want to verify that TLS was used on the connection into your 
infrastructure, you're going to want to include a match on your MTA name 
in the header.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Are you a mildly tech-literate politico horrified by the level of
  ignorance demonstrated by lawmakers gearing up to regulate online
  technology they don't even begin to grasp? Cool. Now you have a
  tiny glimpse into a day in the life of a gun owner.   -- Sean Davis
---
 11 days until the 232nd anniversary of the signing of the U.S. Constitution

Re: Scoring TLS.

[Reio Remma]

On 06/09/2019 15:53, RW wrote:

On Fri, 6 Sep 2019 09:51:06 +0300
Reio Remma wrote:


Hello!

I recently did an experiment where I stopped accepting incoming
e-mail without TLS. This seemingly cut off about 95-99% of spam.
Unfortunately there still seem to be a small percentage of servers
sending without TLS, so that was a no go.

Now I've instead turned to SpamAssassin to score TLS.

header MR_RCVD_TLS  Received =~ / by \S+ \(OpenSMTPD\) with ESMTPS id
[a-z0-9]{8} \((TLSv\d+(?:[.]\d+)?):\S+:\d+:\S+\)/s

Does the Received check only check the last untrusted relay?


No that runs against all Received headers, you should make sure the
"by" part only matches your MX server.


Thanks a bunch for the info!

Reio

Re: Scoring TLS.

[RW]

On Fri, 6 Sep 2019 09:51:06 +0300
Reio Remma wrote:

> Hello!
> 
> I recently did an experiment where I stopped accepting incoming
> e-mail without TLS. This seemingly cut off about 95-99% of spam.
> Unfortunately there still seem to be a small percentage of servers
> sending without TLS, so that was a no go.
> 
> Now I've instead turned to SpamAssassin to score TLS.
> 
> header MR_RCVD_TLS  Received =~ / by \S+ \(OpenSMTPD\) with ESMTPS id 
> [a-z0-9]{8} \((TLSv\d+(?:[.]\d+)?):\S+:\d+:\S+\)/s
> 
> Does the Received check only check the last untrusted relay?


No that runs against all Received headers, you should make sure the
"by" part only matches your MX server.


I was going to suggest using ALL-EXTERNAL, but it looks like it's
broken. The headers aren't flattered, making it too awkward to be worth
using in most cases.

Re: Scoring TLS.

[Reio Remma]

On 06/09/2019 15:25, RW wrote:

On Fri, 6 Sep 2019 10:17:23 +0300
Reio Remma wrote:


On 06/09/2019 09:57, @lbutlr wrote:

On 6 Sep 2019, at 00:51, Reio Remma  wrote:

Even though I recall QMail having TLSv1 back when we were still
using it.

TLSv1.0 is EOLed and should not be used nor supported.

But yes, mailing lists are therein reason I a=have not gone 100%
TLS myself (it’s not just this one, sadly).

There is very little desired email that does not come from lists
that is not using TLS 1.1 or better (TLS 1.1 shouldn’t be used
either, but I see a fair amount of 1.1 still, or did last I looked
a few months ago).

Apache lists also seem to break DKIM with the subject and content
modifications. Not all lists do that and they behave well on that
front.

I don't know about other Apache lists, but this one doesn't - unless
the source does something silly like signing List-Id.


Oh, this is awkward. It seems I was looking at mail source of another 
list when I wrote that.


I eat my words!

Reio

Re: Scoring TLS.

[RW]

On Fri, 6 Sep 2019 10:17:23 +0300
Reio Remma wrote:

> On 06/09/2019 09:57, @lbutlr wrote:
> > On 6 Sep 2019, at 00:51, Reio Remma  wrote:  
> >> Even though I recall QMail having TLSv1 back when we were still
> >> using it.  
> > TLSv1.0 is EOLed and should not be used nor supported.
> >
> > But yes, mailing lists are therein reason I a=have not gone 100%
> > TLS myself (it’s not just this one, sadly).
> >
> > There is very little desired email that does not come from lists
> > that is not using TLS 1.1 or better (TLS 1.1 shouldn’t be used
> > either, but I see a fair amount of 1.1 still, or did last I looked
> > a few months ago). 
> 
> Apache lists also seem to break DKIM with the subject and content 
> modifications. Not all lists do that and they behave well on that
> front.

I don't know about other Apache lists, but this one doesn't - unless
the source does something silly like signing List-Id. 

Re: Scoring TLS.

[Matus UHLAR - fantomas]

On 6 Sep 2019, at 00:51, Reio Remma  wrote:
Even though I recall QMail having TLSv1 back when we were still 
using it.



On 06.09.19 00:57, @lbutlr wrote:

TLSv1.0 is EOLed and should not be used nor supported.



On 06/09/2019 10:57, Matus UHLAR - fantomas wrote:

well, if your clients (some old server installations) only support tls1.0,
it's better to allow it than forgint it to go plaintext or reject the mail
at all.

http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html

http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td96604.html

just FYI


On 06.09.19 11:03, Reio Remma wrote:
Much to my amazement the Postfix (that comes with CentOS 7 - v.2.10 
IIRC) defaults to using no TLS at all for outgoing mail. You need to 
manually enable opportunistic TLS.


I remember there were servers that announced using TLS but failed
configuring it, producing temporary error. 


For cases like this, you must be prepared to disable TLS for those
servers/domains, or be prepared to lose mail.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
A day without sunshine is like, night.

Re: Scoring TLS.

[Reio Remma]

On 06/09/2019 10:57, Matus UHLAR - fantomas wrote:

On 6 Sep 2019, at 00:51, Reio Remma  wrote:
Even though I recall QMail having TLSv1 back when we were still 
using it.


On 06.09.19 00:57, @lbutlr wrote:

TLSv1.0 is EOLed and should not be used nor supported.


well, if your clients (some old server installations) only support 
tls1.0,
it's better to allow it than forgint it to go plaintext or reject the 
mail

at all.

http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html 

http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td96604.html 



just FYI 


Much to my amazement the Postfix (that comes with CentOS 7 - v.2.10 
IIRC) defaults to using no TLS at all for outgoing mail. You need to 
manually enable opportunistic TLS.

Re: Scoring TLS.

[Matus UHLAR - fantomas]

On 6 Sep 2019, at 00:51, Reio Remma  wrote:

Even though I recall QMail having TLSv1 back when we were still using it.


On 06.09.19 00:57, @lbutlr wrote:

TLSv1.0 is EOLed and should not be used nor supported.


well, if your clients (some old server installations) only support tls1.0,
it's better to allow it than forgint it to go plaintext or reject the mail
at all.

http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td78583.html
http://postfix.1071664.n5.nabble.com/Update-to-recommended-TLS-settings-td96604.html

just FYI

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I drive way too fast to worry about cholesterol.

Re: Scoring TLS.

[Reio Remma]

On 06/09/2019 09:57, @lbutlr wrote:

On 6 Sep 2019, at 00:51, Reio Remma  wrote:

Even though I recall QMail having TLSv1 back when we were still using it.

TLSv1.0 is EOLed and should not be used nor supported.

But yes, mailing lists are therein reason I a=have not gone 100% TLS myself 
(it’s not just this one, sadly).

There is very little desired email that does not come from lists that is not 
using TLS 1.1 or better (TLS 1.1 shouldn’t be used either, but I see a fair 
amount of 1.1 still, or did last I looked a few months ago).



Apache lists also seem to break DKIM with the subject and content 
modifications. Not all lists do that and they behave well on that front.

Re: Scoring TLS.

[@lbutlr]

On 6 Sep 2019, at 00:51, Reio Remma  wrote:
> Even though I recall QMail having TLSv1 back when we were still using it.

TLSv1.0 is EOLed and should not be used nor supported.

But yes, mailing lists are therein reason I a=have not gone 100% TLS myself 
(it’s not just this one, sadly).

There is very little desired email that does not come from lists that is not 
using TLS 1.1 or better (TLS 1.1 shouldn’t be used either, but I see a fair 
amount of 1.1 still, or did last I looked a few months ago).



-- 
The easiest way to find something lost around the house is to buy a
replacement.

Scoring TLS.

[Reio Remma]

Hello!

I recently did an experiment where I stopped accepting incoming e-mail 
without TLS. This seemingly cut off about 95-99% of spam. Unfortunately 
there still seem to be a small percentage of servers sending without 
TLS, so that was a no go.


Now I've instead turned to SpamAssassin to score TLS.

header MR_RCVD_TLS  Received =~ / by \S+ \(OpenSMTPD\) with ESMTPS id 
[a-z0-9]{8} \((TLSv\d+(?:[.]\d+)?):\S+:\d+:\S+\)/s


Does the Received check only check the last untrusted relay?

The Apache lists seem to be using no TLS either. :)

Even though I recall QMail having TLSv1 back when we were still using it.

Thanks,
Reio

Re: Scoring by registrar?

[Grant Taylor]

On 7/1/19 4:32 PM, Sean Lynch wrote:
I think fast flux came up in reference to a speculation I'd made 
regarding why the spammers were using their own nameservers rather than 
Namecheap's.


Ah.

I don't think it's particularly off-base to refer to rapid registration 
of new domains as fast flux.


I can't agree to that.

Fast Flux is a technique used within a given domain name.  Not something 
that is done across domain names.


Infoblox has a good article that refers to changing IPs behind a domain. 
 This is decidedly not multiple domain names.


Link - What is a Fast Flux?
 - https://www.infoblox.com/glossary/fast-flux/

As for rapidly registering domains, I'm seeing an average of 106,608 new 
domains registered a day.  So, even if a bad actor registers 1,000 new 
domains, that's only 1% of the overall daily registration.


In fact, I'm pretty sure support for this, and slowness in taking down 
domains (though they do often take them down eventually at least), 
are why Namecheap is so popular.


That may very well be the case.  But I think that "fast flux" is the 
wrong term for it.


As I mentioned, filtering using fresh.fmb.la catches about 1/3 of the 
domains. Fortunately, since they're actually using their own servers and 
not a botnet, blocking their netblock catches the rest, though it's not 
my preference since it will cause collateral damage (even though 
registering with dnswl.org is an easy way around that), it's manual, and 
it only helps my 3 users. Incentivizing Namecheap to move faster on 
these would benefit a lot more people.


ACK



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Scoring by registrar?

[Sean Lynch]

On 7/1/19 3:13 PM, Grant Taylor wrote:

On 7/1/19 6:44 AM, micah anderson wrote:

This sounds like Fast Flux


How is this fast flux?

I thought fast flux was rapidly updating A records on the DNS server 
(for a given qname) or updating NS records with the registrar for a 
single given domain.


It sounds to me like Sean was talking about wanting to identify which of 
many domains were had a common registrar.  This doesn't sound like fast 
flux—as I understand it—to me.



Having such a list would be very helpful for dealing with fast flux.


How is what the OP's talking about related to fast flux?


I think fast flux came up in reference to a speculation I'd made 
regarding why the spammers were using their own nameservers rather than 
Namecheap's. I don't think it's particularly off-base to refer to rapid 
registration of new domains as fast flux. In fact, I'm pretty sure 
support for this, and slowness in taking down domains (though they do 
often take them down eventually at least), are why Namecheap is so popular.


As I mentioned, filtering using fresh.fmb.la catches about 1/3 of the 
domains. Fortunately, since they're actually using their own servers and 
not a botnet, blocking their netblock catches the rest, though it's not 
my preference since it will cause collateral damage (even though 
registering with dnswl.org is an easy way around that), it's manual, and 
it only helps my 3 users. Incentivizing Namecheap to move faster on 
these would benefit a lot more people.

Re: Scoring by registrar?

[Grant Taylor]

On 7/1/19 6:44 AM, micah anderson wrote:

This sounds like Fast Flux


How is this fast flux?

I thought fast flux was rapidly updating A records on the DNS server 
(for a given qname) or updating NS records with the registrar for a 
single given domain.


It sounds to me like Sean was talking about wanting to identify which of 
many domains were had a common registrar.  This doesn't sound like fast 
flux—as I understand it—to me.



Having such a list would be very helpful for dealing with fast flux.


How is what the OP's talking about related to fast flux?



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Scoring by registrar?

[Paul Stead]

On Mon, 1 Jul 2019 at 16:17, RW  wrote:

>
> On the site they have:
>
> Query   ResponseNameMeaning
> domain  127.2.0.2   fresh   Domain registered in last 7 days
> domain  127.2.0.14  fresh14 Domain registered in last 7-14 days
>
> there's no mention of the 127.2.0.28 result, but from the previous line
> it looks like NEWDOM28 would be 14-28.
>
>
This. I've updated the site to reflect the 127.2.0.28 return (NEWDOM28)

Paul

Re: Scoring by registrar?

[RW]

On Mon, 01 Jul 2019 07:45:23 -0700
Sean Lynch wrote:

> On July 1, 2019 7:22:58 AM PDT, micah anderson 
> wrote:
> >Sean Lynch  writes:
> >  
> >>>Having such a list would be very helpful for dealing with fast
> >>>flux.  
> >>
> >> SA already has this. It used fresh.fmb.la to detect domains  
> >registered within the past couple of weeks.
> >
> >It does? Do I need to enable something to get that?  
> 
> I got the test via sa-update, and it's a network check so they have
> to be enabled. Its the FROM_FMBLA_NEWDOM, FROM_FMBLA_NEWDOM14, and
> FROM_FMBLA_NEWDOM28 rules. Though since fresh.fmb.la only returns 0-7
> days and 7-14 days and I've only seen NEWDOM and NEWDOM28 fire I
> think NEWDOM28 may actually mean 7-14 days. Or the fresh.fmb.la docs
> are out of date. The maintainer is on this list and can probably
> comment.

On the site they have:

Query   ResponseNameMeaning
domain  127.2.0.2   fresh   Domain registered in last 7 days
domain  127.2.0.14  fresh14 Domain registered in last 7-14 days

there's no mention of the 127.2.0.28 result, but from the previous line
it looks like NEWDOM28 would be 14-28.

Re: Scoring by registrar?

[John Hardin]

On Mon, 1 Jul 2019, micah anderson wrote:


Grant Taylor  writes:

As a Namecheap customer, you are making me want to move. That is good,
but its also something you should consider, before you block the entire
registrar: there are a significant number of non-spamming Namecheap
customers that you would be cutting off if you did this. I understand
you want to put pressure on Namecheap, but the flip side of that is you
will be cutting yourself off from those domains in the process.


Note: I don't think "poison pill" treatment is being advocated here, just 
"another spam sign along with the rest"...



I think there are also lists of domains that have been recently
registered.  Which might help if the single use domains were recently
registered.


Having such a list would be very helpful for dealing with fast flux.


Day Old Bread et. al.

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The yardstick you should use when considering whether to support a
  given piece of legislation is "what if my worst enemy is chosen to
  administer this law?"
---
 3 days until the 243rd anniversary of the Declaration of Independence

Re: Scoring by registrar?

[Sean Lynch]

On July 1, 2019 7:22:58 AM PDT, micah anderson  wrote:
>Sean Lynch  writes:
>
>>>Having such a list would be very helpful for dealing with fast flux.
>>
>> SA already has this. It used fresh.fmb.la to detect domains
>registered within the past couple of weeks.
>
>It does? Do I need to enable something to get that?

I got the test via sa-update, and it's a network check so they have to be 
enabled. Its the FROM_FMBLA_NEWDOM, FROM_FMBLA_NEWDOM14, and 
FROM_FMBLA_NEWDOM28 rules. Though since fresh.fmb.la only returns 0-7 days and 
7-14 days and I've only seen NEWDOM and NEWDOM28 fire I think NEWDOM28 may 
actually mean 7-14 days. Or the fresh.fmb.la docs are out of date. The 
maintainer is on this list and can probably comment.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Scoring by registrar?

[micah anderson]

Sean Lynch  writes:

>>Having such a list would be very helpful for dealing with fast flux.
>
> SA already has this. It used fresh.fmb.la to detect domains registered within 
> the past couple of weeks.

It does? Do I need to enable something to get that?
-- 
micah

Re: Scoring by registrar?

[Sean Lynch]

On July 1, 2019 5:44:37 AM PDT, micah anderson  wrote:
>Grant Taylor  writes:
>
>>> A very large number (nearly all, in fact) of the spams I receive
>these 
>>> days involve domains registered with Namecheap. I've received
>hundreds 
>>> of spams involving .icu domains from what appear to be the same
>spammer. 
>>> I also receive a large number of scams impersonating Bitmain, again 
>>> using domains involving Namecheap.
>>
>> Is Namecheap just the registrar?  Or are they also hosting the DNS
>service?
>
>As a Namecheap customer, you are making me want to move. That is good,
>but its also something you should consider, before you block the entire
>registrar: there are a significant number of non-spamming Namecheap
>customers that you would be cutting off if you did this. I understand
>you want to put pressure on Namecheap, but the flip side of that is you
>will be cutting yourself off from those domains in the process.

Like all SA rules, registrar would be just one of many signals, so Namecheap 
customers would only be cut off if their emails or IPs seem spammy in other 
ways. And there's always the option of registering with dnswl.org.

>>> While Namecheap does suspend at least some domains within days of
>their 
>>> being used in a campaign, it's clear that these are being treated as
>
>>> single-use domains, so this has very little impact on the spammers.
>
>This sounds like Fast Flux - and it is not something that happens only
>on Namecheap.
>
>> I think there are also lists of domains that have been recently 
>> registered.  Which might help if the single use domains were recently
>
>> registered.
>
>Having such a list would be very helpful for dealing with fast flux.

SA already has this. It used fresh.fmb.la to detect domains registered within 
the past couple of weeks.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Scoring by registrar?

[micah anderson]

Grant Taylor  writes:

>> A very large number (nearly all, in fact) of the spams I receive these 
>> days involve domains registered with Namecheap. I've received hundreds 
>> of spams involving .icu domains from what appear to be the same spammer. 
>> I also receive a large number of scams impersonating Bitmain, again 
>> using domains involving Namecheap.
>
> Is Namecheap just the registrar?  Or are they also hosting the DNS service?

As a Namecheap customer, you are making me want to move. That is good,
but its also something you should consider, before you block the entire
registrar: there are a significant number of non-spamming Namecheap
customers that you would be cutting off if you did this. I understand
you want to put pressure on Namecheap, but the flip side of that is you
will be cutting yourself off from those domains in the process.

>> While Namecheap does suspend at least some domains within days of their 
>> being used in a campaign, it's clear that these are being treated as 
>> single-use domains, so this has very little impact on the spammers.

This sounds like Fast Flux - and it is not something that happens only
on Namecheap.

> I think there are also lists of domains that have been recently 
> registered.  Which might help if the single use domains were recently 
> registered.

Having such a list would be very helpful for dealing with fast flux.

-- 
micah

Re: Scoring by registrar?

[Paul Stead]

On Mon, 1 Jul 2019 at 06:38, Sean Lynch  wrote:

> It's pretty useful already. If you're able to get the name of the
> registrar from that service, I think it might make a useful spam signal
> since some registrars seem to be a lot more popular with spammers than
> others.
>

Not really, essentially it's access to the zonefile, so no more information
available that doing an "NS" DNS lookup

Re: Scoring by registrar?

[Sean Lynch]

On 6/30/19 9:41 PM, Paul Stead wrote:
On Sun, 30 Jun 2019 at 19:46, Sean Lynch > wrote:



On 6/30/19 11:40 AM, Grant Taylor wrote:
> On 6/30/19 12:05 PM, John Hardin wrote:
>> There's really no infrastructure for it. Somebody would have to
hook
>> into the registrar data feeds to collect it and publish it in a
>> usable form, and nobody has done so that I am aware of.
>
> Whois Domain Search has some information.
>
> Link - Whois Domain Search
>  - http://whoisds.com/
>
> They provide an API and an ability to download copies of their
database.
>
> I'm downloading their free newly registered domain list.  It's
only a
> list of domains registered in the last day and they have 10 (?)
days
> worth available for download.

I wonder if that's the list fresh.fmb.la  uses?


fresh.fmb.la  uses the CZDS service from ICANN to 
create the fresh list - is there anything I could do to make the BL 
more useful?


It's pretty useful already. If you're able to get the name of the 
registrar from that service, I think it might make a useful spam signal 
since some registrars seem to be a lot more popular with spammers than 
others.

Re: Scoring by registrar?

[Paul Stead]

On Sun, 30 Jun 2019 at 19:46, Sean Lynch  wrote:

>
> On 6/30/19 11:40 AM, Grant Taylor wrote:
> > On 6/30/19 12:05 PM, John Hardin wrote:
> >> There's really no infrastructure for it. Somebody would have to hook
> >> into the registrar data feeds to collect it and publish it in a
> >> usable form, and nobody has done so that I am aware of.
> >
> > Whois Domain Search has some information.
> >
> > Link - Whois Domain Search
> >  - http://whoisds.com/
> >
> > They provide an API and an ability to download copies of their database.
> >
> > I'm downloading their free newly registered domain list.  It's only a
> > list of domains registered in the last day and they have 10 (?) days
> > worth available for download.
>
> I wonder if that's the list fresh.fmb.la uses?
>

fresh.fmb.la uses the CZDS service from ICANN to create the fresh list - is
there anything I could do to make the BL more useful?

Paul

Re: Scoring by registrar?

[John Hardin]

On Sun, 30 Jun 2019, Sean Lynch wrote:


On June 30, 2019 11:20:33 AM PDT, John Hardin  wrote:


...and if the same IP address is a regular abuser that never sends any
legitimate traffic, tarpit them:

   http://www.impsec.org/~jhardin/antispam/spammer-firewall


I do like the idea of tarpitting spammers, because I want to drive up 
the cost of spamming. I haven't been able to find even anecdotal 
evidence that it causes them any genuine pain beyond just sleeping 
though since they tend to have very aggressive timeouts.


Anectodal tarpit evidence from a *very* small MTA:

25/tcp (smtp): 5 host(s), 98 connection(s)
  1 185.16.204.92
  6 193.56.28.33
 10 185.234.219.100
 20 37.72.168.198
 61 193.169.252.171

If enough people were doing this I believe it would have an impact.

postscreen's short sleep during its two-line greeting seems to cause a 
lot of spammers to hang up, or they try saying HELO too early and 
postscreen blocks them.


I do that, too. :)


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Are you a mildly tech-literate politico horrified by the level of
  ignorance demonstrated by lawmakers gearing up to regulate online
  technology they don't even begin to grasp? Cool. Now you have a
  tiny glimpse into a day in the life of a gun owner.   -- Sean Davis
---
 4 days until the 243rd anniversary of the Declaration of Independence

Re: Scoring by registrar?

[John Hardin]

On Sun, 30 Jun 2019, Grant Taylor wrote:


On 6/30/19 12:05 PM, John Hardin wrote:
There's really no infrastructure for it. Somebody would have to hook into 
the registrar data feeds to collect it and publish it in a usable form, and 
nobody has done so that I am aware of.


Whois Domain Search has some information.

Link - Whois Domain Search
- http://whoisds.com/

They provide an API and an ability to download copies of their database.

I'm downloading their free newly registered domain list.  It's only a list of 
domains registered in the last day and they have 10 (?) days worth available 
for download.


A decade ago I wrote a plugin that used whois to try to do this as an 
experiment. The big drawback is: actually doing this could easily be 
considered abuse of the whois system and could easily get you blacklisted. 
This is *not* recommended for production use.


   http://www.impsec.org/~jhardin/antispam/registrar_scoring/

This is just for illustration. I *strongly* discourage using this in 
anything other than a limited test environment (assuming it even still 
works).


Interesting.  I'll have to read and assimilate your work.  I'm sure I'll 
learn many things.  Thank you for sharing.  :-)


If I were ever to implement something like this, I would NOT blindly do the 
Whois query directly for each incoming email.  I would query a local service 
that cached information (as in committed to disk) and have that service fetch 
information about domains that it didn't have information on.


Which is what that does.

I might even make such a system periodically check to see if things like DNS 
servers had changed and then refresh the cache on demand as necessary.


I don't remember if I implemented cache expiry.

I agree that blindly and directly doing a Whois query for each and every 
incoming email would cause some people to get upset.  Not to mention the 
performance and latency implications.


Well, for each domain not seen [yet|recently].

If you had access to the registrar feeds you might be able to write 
something that used that data which would not be considered abusive.


I think that's exactly the type of data that Whois Domain Search is selling, 
and why they are selling it.


Right. I neglected to mention above that the data *was* available for $$$, 
as I presumed we were discussing this in the context of a free service.


Is there anybody in the SA user community who does have access to the raw 
registrar feeds?


I don't.  But I think Whois Domain Search offers trial options.

No, I'm not affiliated with Whois Domain Search.  I simply download their 
free list of domains registered yesterday each day.  }:-)  Not that I've 
actually done anything with that data yet.  But that's a different problem.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Are you a mildly tech-literate politico horrified by the level of
  ignorance demonstrated by lawmakers gearing up to regulate online
  technology they don't even begin to grasp? Cool. Now you have a
  tiny glimpse into a day in the life of a gun owner.   -- Sean Davis
---
 4 days until the 243rd anniversary of the Declaration of Independence

Re: Scoring by registrar?

[Sean Lynch]

On June 30, 2019 11:20:33 AM PDT, John Hardin  wrote:
>On Sun, 30 Jun 2019, Grant Taylor wrote:
>
>> On 6/30/19 10:51 AM, Martin Gregorie wrote:
>>> If you don't mind a delay in receiving mail from hosts you've never
>seen
>>> before, why not implement a greylister?
>>> 
>>> https://en.wikipedia.org/wiki/Greylisting
>>
>> I see your GreyListing and raise you NoListing:
>>
>> https://en.wikipedia.org/wiki/Nolisting
>>
>> TL;DR:  NoListing works by having an MX record that either does not
>respond 
>> to TCP connections for SMTP, or sends TCP Resets.  Thus causing RFC
>compliant 
>> DNS servers to move on to the next priority MX in short order.

NoListing concerns me for two reasons: first, it causes everyone to have to try 
twice regardless of reputation. Second, Bad Things will happen if I do anything 
punitive on the highest preference MX and my primary and secondary go down. 
With greylisting, I can at least whitelist anyone registered with dnswl.org, 
etc. A greylist server could also whitelist an entire domain once any of its 
servers passes, if SPF is set up.

>
>...and if the same IP address is a regular abuser that never sends any 
>legitimate traffic, tarpit them:
>
>http://www.impsec.org/~jhardin/antispam/spammer-firewall

I do like the idea of tarpitting spammers, because I want to drive up the cost 
of spamming. I haven't been able to find even anecdotal evidence that it causes 
them any genuine pain beyond just sleeping though since they tend to have very 
aggressive timeouts. postscreen's short sleep during its two-line greeting 
seems to cause a lot of spammers to hang up, or they try saying HELO too early 
and postscreen blocks them.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Scoring by registrar?

[Sean Lynch]

On 6/30/19 11:40 AM, Grant Taylor wrote:

On 6/30/19 12:05 PM, John Hardin wrote:
There's really no infrastructure for it. Somebody would have to hook 
into the registrar data feeds to collect it and publish it in a 
usable form, and nobody has done so that I am aware of.


Whois Domain Search has some information.

Link - Whois Domain Search
 - http://whoisds.com/

They provide an API and an ability to download copies of their database.

I'm downloading their free newly registered domain list.  It's only a 
list of domains registered in the last day and they have 10 (?) days 
worth available for download.


I wonder if that's the list fresh.fmb.la uses?



A decade ago I wrote a plugin that used whois to try to do this as an 
experiment. The big drawback is: actually doing this could easily be 
considered abuse of the whois system and could easily get you 
blacklisted. This is *not* recommended for production use.


   http://www.impsec.org/~jhardin/antispam/registrar_scoring/

This is just for illustration. I *strongly* discourage using this in 
anything other than a limited test environment (assuming it even 
still works).


Interesting.  I'll have to read and assimilate your work.  I'm sure 
I'll learn many things.  Thank you for sharing.  :-)


If I were ever to implement something like this, I would NOT blindly 
do the Whois query directly for each incoming email.  I would query a 
local service that cached information (as in committed to disk) and 
have that service fetch information about domains that it didn't have 
information on.


I might even make such a system periodically check to see if things 
like DNS servers had changed and then refresh the cache on demand as 
necessary.


I agree that blindly and directly doing a Whois query for each and 
every incoming email would cause some people to get upset.  Not to 
mention the performance and latency implications.


If you had access to the registrar feeds you might be able to write 
something that used that data which would not be considered abusive.


I think that's exactly the type of data that Whois Domain Search is 
selling, and why they are selling it.


Is there anybody in the SA user community who does have access to the 
raw registrar feeds?


I don't.  But I think Whois Domain Search offers trial options.

No, I'm not affiliated with Whois Domain Search.  I simply download 
their free list of domains registered yesterday each day.  }:-)  Not 
that I've actually done anything with that data yet.  But that's a 
different problem.


With fresh.fmb.la, the raw data is a little less useful unless you want 
better resolution than a week at a time. It might be useful for finding 
and reporting Bitmain lookalike domains before they get used in spam blasts.


I might find it worth it to sign up for one of their services if I can 
use it to offer some useful service such as a DNSBL to others. I'll need 
to check their subscriber agreement. Thanks for pointing it out!

Re: Scoring by registrar?

[Grant Taylor]

On 6/30/19 12:05 PM, John Hardin wrote:
There's really no infrastructure for it. Somebody would have to hook 
into the registrar data feeds to collect it and publish it in a usable 
form, and nobody has done so that I am aware of.


Whois Domain Search has some information.

Link - Whois Domain Search
 - http://whoisds.com/

They provide an API and an ability to download copies of their database.

I'm downloading their free newly registered domain list.  It's only a 
list of domains registered in the last day and they have 10 (?) days 
worth available for download.


A decade ago I wrote a plugin that used whois to try to do this as an 
experiment. The big drawback is: actually doing this could easily be 
considered abuse of the whois system and could easily get you 
blacklisted. This is *not* recommended for production use.


   http://www.impsec.org/~jhardin/antispam/registrar_scoring/

This is just for illustration. I *strongly* discourage using this in 
anything other than a limited test environment (assuming it even still 
works).


Interesting.  I'll have to read and assimilate your work.  I'm sure I'll 
learn many things.  Thank you for sharing.  :-)


If I were ever to implement something like this, I would NOT blindly do 
the Whois query directly for each incoming email.  I would query a local 
service that cached information (as in committed to disk) and have that 
service fetch information about domains that it didn't have information on.


I might even make such a system periodically check to see if things like 
DNS servers had changed and then refresh the cache on demand as necessary.


I agree that blindly and directly doing a Whois query for each and every 
incoming email would cause some people to get upset.  Not to mention the 
performance and latency implications.


If you had access to the registrar feeds you might be able to write 
something that used that data which would not be considered abusive.


I think that's exactly the type of data that Whois Domain Search is 
selling, and why they are selling it.


Is there anybody in the SA user community who does have access to the 
raw registrar feeds?


I don't.  But I think Whois Domain Search offers trial options.

No, I'm not affiliated with Whois Domain Search.  I simply download 
their free list of domains registered yesterday each day.  }:-)  Not 
that I've actually done anything with that data yet.  But that's a 
different problem.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Scoring by registrar?

[Sean Lynch]

On 6/30/19 11:05 AM, John Hardin wrote:

On Sun, 30 Jun 2019, Sean Lynch wrote:

A very large number (nearly all, in fact) of the spams I receive 
these days involve domains registered with Namecheap.


I'd like to add a spam score to any message using a domain registered 
with them.


Does such functionality already exist in SpamAssassin? Is there an 
RHSBL or some other simple mechanism I could use to look up the 
registrar for a domain?


There's really no infrastructure for it. Somebody would have to hook 
into the registrar data feeds to collect it and publish it in a usable 
form, and nobody has done so that I am aware of.


A decade ago I wrote a plugin that used whois to try to do this as an 
experiment. The big drawback is: actually doing this could easily be 
considered abuse of the whois system and could easily get you 
blacklisted. This is *not* recommended for production use.


  http://www.impsec.org/~jhardin/antispam/registrar_scoring/

This is just for illustration. I *strongly* discourage using this in 
anything other than a limited test environment (assuming it even still 
works).
I've been wary of just querying whois for precisely this reason. Maybe 
rate-limited queries along with greylisting to give time to do the lookup?


If you had access to the registrar feeds you might be able to write 
something that used that data which would not be considered abusive.


Is there anybody in the SA user community who does have access to the 
raw registrar feeds?


This would be lovely. Turning it into a DNS-based service would be even 
better!


Thanks for the response!

Re: Scoring by registrar?

[Sean Lynch]

On 6/30/19 11:00 AM, Grant Taylor wrote:

On 6/30/19 10:08 AM, Sean Lynch wrote:
Hi, everyone! I used to run my own mail servers back in the mid '90s 
and even worked as the postmaster for a regional ISP and worked on 
mail servers for some large corporations and even a small national 
ISP as a consultant. After a hiatus where I drank the hosted email 
kool-aid, I'm back to hosting my own email.


Welcome back to the fray.  :-)

At the moment I'm using a combination of SMTP-time DNSBL and other 
checks and SpamAssassin at delivery time for spam filtering. Very few 
spams are even making it to SpamAssassin, but many that do make it 
all the way through into my inbox.


:-(

A very large number (nearly all, in fact) of the spams I receive 
these days involve domains registered with Namecheap. I've received 
hundreds of spams involving .icu domains from what appear to be the 
same spammer. I also receive a large number of scams impersonating 
Bitmain, again using domains involving Namecheap.


Is Namecheap just the registrar?  Or are they also hosting the DNS 
service?


Ah, I should have mentioned that. Unfortunately, they're just the 
registrar. I suspect the spammers use DNS servers they can update 
quickly, but since it's slower to update NS records and glue records, 
the nameserver IPs and names might make interesting extra signals to 
score on.




While Namecheap does suspend at least some domains within days of 
their being used in a campaign, it's clear that these are being 
treated as single-use domains, so this has very little impact on the 
spammers. Since for whatever reason they're so attractive to spammers 
that they seem to be a nearly universal choice, at least for spams I 
get, I'd like to add a spam score to any message using a domain 
registered with them.


Does such functionality already exist in SpamAssassin? Is there an 
RHSBL or some other simple mechanism I could use to look up the 
registrar for a domain?


I'm not sure how to check for Namecheap as the domain registrar. I 
think it should be relatively easy to check if the Namecheap is being 
used for the DNS service by checking what DNS servers are used.  
Perhaps you could alter the score that way.


I think you could likely take this a step further and use something 
like BIND's features to alter responses to DNS queries based on the 
DNS server the information comes from.  Meaning you could break email 
from domains using specific DNS servers.  }:-) This means that you 
could configure your MTA to require valid DNS (which it should be 
doing anyway).  Thus your email server would not accept email from 
domains that use Namecheap DNS servers. }:-D


I think there are also lists of domains that have been recently 
registered.  Which might help if the single use domains were recently 
registered.


I do plan to set up a DNS server at some point in order to implement my 
own DNSBLs among other things.


About 1/3 of both the .icu and Bitmain spams do hit one of the 
FROM_FMBLA_NEWDOM rules. I've bumped the scores up for those so that any 
recently-registered .icu domain will always go to my junk folder.


One of my goals is to incentivize Namecheap to make themselves less 
attractive to spammers. Having one person use their being the registrar 
as a spam signal doesn't accomplish that, but inspiring many people to 
might.


Even better would be to use signals like that as an SMTP-time test so 
that senders will (hopefully) see a bounce message that says they need 
to register with dnswl.org if they want to be able to send email from a 
Namecheap-registered domain. I should probably investigate mtpolicyd a 
little more closely; right now I just use policyd-spf-python to reject 
any messages that fail SPF, but that catches almost nothing because the 
spammers who are able to get past the DNSBLs I use typically have set up 
all the right records for their throwaway domains, including SPF and DKIM.

Re: Scoring by registrar?

[John Hardin]

On Sun, 30 Jun 2019, Grant Taylor wrote:


On 6/30/19 10:51 AM, Martin Gregorie wrote:

If you don't mind a delay in receiving mail from hosts you've never seen
before, why not implement a greylister?

https://en.wikipedia.org/wiki/Greylisting


I see your GreyListing and raise you NoListing:

https://en.wikipedia.org/wiki/Nolisting

TL;DR:  NoListing works by having an MX record that either does not respond 
to TCP connections for SMTP, or sends TCP Resets.  Thus causing RFC compliant 
DNS servers to move on to the next priority MX in short order.


...and if the same IP address is a regular abuser that never sends any 
legitimate traffic, tarpit them:


   http://www.impsec.org/~jhardin/antispam/spammer-firewall

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The focus of our education system is
  the transfer of tax dollars between politicians and unions.
  Educating children is its waste product.   -- Frank Fleming
---
 4 days until the 243rd anniversary of the Declaration of Independence

Re: Scoring by registrar?

[Grant Taylor]

On 6/30/19 10:51 AM, Martin Gregorie wrote:

If you don't mind a delay in receiving mail from hosts you've never seen
before, why not implement a greylister?

https://en.wikipedia.org/wiki/Greylisting


I see your GreyListing and raise you NoListing:

https://en.wikipedia.org/wiki/Nolisting

TL;DR:  NoListing works by having an MX record that either does not 
respond to TCP connections for SMTP, or sends TCP Resets.  Thus causing 
RFC compliant DNS servers to move on to the next priority MX in short order.


I find that this cuts out a LOT of crap without most (if not all) of the 
problems generally associated with GreyListing.


 · It's stateless
 · It doesn't care where the retries come from
 · It's RFC compliant, no grey area
 · It allows fast retries.
· Nothing prevents the same server from trying the next MX immediately.
 · There aren't issues with "You must wait X number of minutes".
· There is no mechanism in SMTP to indicate how long to wait.
· Servers can try the next MX immediately

I also highly recommend something like Junk Email Filter's Project 
Tar(baby) as a high order MX.


Link - Project Tar
 - http://wiki.junkemailfilter.com/index.php/Project_tarbaby

While you're at it, consider using Junk Email Filter's Spam DNS Lists to 
filter bad actors learned via Project Tar.


Link - Spam DNS Lists
 - http://wiki.junkemailfilter.com/index.php/Spam_DNS_Lists



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Scoring by registrar?

[John Hardin]

On Sun, 30 Jun 2019, Sean Lynch wrote:

A very large number (nearly all, in fact) of the spams I receive these days 
involve domains registered with Namecheap.


I'd like to add a spam score to any message using a domain registered 
with them.


Does such functionality already exist in SpamAssassin? Is there an RHSBL or 
some other simple mechanism I could use to look up the registrar for a 
domain?


There's really no infrastructure for it. Somebody would have to hook into 
the registrar data feeds to collect it and publish it in a usable form, 
and nobody has done so that I am aware of.


A decade ago I wrote a plugin that used whois to try to do this as an 
experiment. The big drawback is: actually doing this could easily be 
considered abuse of the whois system and could easily get you blacklisted. 
This is *not* recommended for production use.


  http://www.impsec.org/~jhardin/antispam/registrar_scoring/

This is just for illustration. I *strongly* discourage using this in 
anything other than a limited test environment (assuming it even still 
works).


If you had access to the registrar feeds you might be able to write 
something that used that data which would not be considered abusive.


Is there anybody in the SA user community who does have access to the raw 
registrar feeds?



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  If Microsoft made hammers, everyone would whine about how poorly
  screws were designed and about how they are hard to hammer in, and
  wonder why it takes so long to paint a wall using the hammer.
---
 4 days until the 243rd anniversary of the Declaration of Independence

Re: Scoring by registrar?

[Grant Taylor]

On 6/30/19 10:08 AM, Sean Lynch wrote:
Hi, everyone! I used to run my own mail servers back in the mid '90s and 
even worked as the postmaster for a regional ISP and worked on mail 
servers for some large corporations and even a small national ISP as a 
consultant. After a hiatus where I drank the hosted email kool-aid, I'm 
back to hosting my own email.


Welcome back to the fray.  :-)

At the moment I'm using a combination of SMTP-time DNSBL and other 
checks and SpamAssassin at delivery time for spam filtering. Very 
few spams are even making it to SpamAssassin, but many that do make 
it all the way through into my inbox.


:-(

A very large number (nearly all, in fact) of the spams I receive these 
days involve domains registered with Namecheap. I've received hundreds 
of spams involving .icu domains from what appear to be the same spammer. 
I also receive a large number of scams impersonating Bitmain, again 
using domains involving Namecheap.


Is Namecheap just the registrar?  Or are they also hosting the DNS service?

While Namecheap does suspend at least some domains within days of their 
being used in a campaign, it's clear that these are being treated as 
single-use domains, so this has very little impact on the spammers. 
Since for whatever reason they're so attractive to spammers that they 
seem to be a nearly universal choice, at least for spams I get, I'd like 
to add a spam score to any message using a domain registered with them.


Does such functionality already exist in SpamAssassin? Is there an RHSBL 
or some other simple mechanism I could use to look up the registrar for 
a domain?


I'm not sure how to check for Namecheap as the domain registrar.  I 
think it should be relatively easy to check if the Namecheap is being 
used for the DNS service by checking what DNS servers are used.  Perhaps 
you could alter the score that way.


I think you could likely take this a step further and use something like 
BIND's features to alter responses to DNS queries based on the DNS 
server the information comes from.  Meaning you could break email from 
domains using specific DNS servers.  }:-)  This means that you could 
configure your MTA to require valid DNS (which it should be doing 
anyway).  Thus your email server would not accept email from domains 
that use Namecheap DNS servers.  }:-D


I think there are also lists of domains that have been recently 
registered.  Which might help if the single use domains were recently 
registered.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Scoring by registrar?

[Sean Lynch]

On 6/30/19 9:51 AM, Martin Gregorie wrote:

On Sun, 2019-06-30 at 09:08 -0700, Sean Lynch wrote:

A very large number (nearly all, in fact) of the spams I receive
these days involve domains registered with Namecheap. I've received
hundreds of spams involving .icu domains from what appear to be the
same spammer.


Write a local rule that adds points for mails from .icu


Such a rule already exists. I've bumped up its score already.




I also receive a large number of scams impersonating Bitmain, again
using domains involving Namecheap.


As above, but for Bitmain.


Thanks. I'm aware I can do this.




While Namecheap does suspend at least some domains within days of
their being used in a campaign, it's clear that these are being
treated as single-use domains, so this has very little impact on the
spammers. Since for whatever reason they're so attractive to spammers
that they seem to be a nearly universal choice, at least for spams I
get, I'd like to add a spam score to any message using a domain
registered with them.


If you don't mind a delay in receiving mail from hosts you've never seen
before, why not implement a greylister?

https://en.wikipedia.org/wiki/Greylisting


Thanks. I'm aware of greylisting already.



Does such functionality already exist in SpamAssassin?

Defining local rules has always been possible.


Thanks. I'm aware of this. I was asking what functionality exists, if 
any, for determining who a domain's registrar is.




Greylisters are used to front end your MTA, so work independently of
Spamassassin.

I find combinations of rules can be surprisingly specific, e.g. to catch
sales spam:

- write a rule that contains a list of selling terms with a very small
   positive score (0.001)
- write another rule that contains a list of products pushed by
   spammers, again with a very small positive score
- write a meta rule the triggers only when both the previous rules
   are hit and give it a significant score
   
If you avoid sales terms and product names/descriptions that are in

common use the meta rule will cause few false positives.


Thanks. As I said, been using SpamAssassin (and generally fighting spam) 
for years, so I'm already aware of this.


  
Martin

Re: Scoring by registrar?

[Martin Gregorie]

On Sun, 2019-06-30 at 09:08 -0700, Sean Lynch wrote:
> A very large number (nearly all, in fact) of the spams I receive
> these days involve domains registered with Namecheap. I've received
> hundreds of spams involving .icu domains from what appear to be the
> same spammer.
>
Write a local rule that adds points for mails from .icu  

> I also receive a large number of scams impersonating Bitmain, again 
> using domains involving Namecheap.
> 
As above, but for Bitmain.

> While Namecheap does suspend at least some domains within days of
> their being used in a campaign, it's clear that these are being
> treated as single-use domains, so this has very little impact on the
> spammers. Since for whatever reason they're so attractive to spammers
> that they seem to be a nearly universal choice, at least for spams I
> get, I'd like to add a spam score to any message using a domain
> registered with them.
> 
If you don't mind a delay in receiving mail from hosts you've never seen
before, why not implement a greylister?   

https://en.wikipedia.org/wiki/Greylisting

Does such functionality already exist in SpamAssassin?

>
Defining local rules has always been possible.

Greylisters are used to front end your MTA, so work independently of
Spamassassin.

I find combinations of rules can be surprisingly specific, e.g. to catch
sales spam:

- write a rule that contains a list of selling terms with a very small
  positive score (0.001)
- write another rule that contains a list of products pushed by
  spammers, again with a very small positive score
- write a meta rule the triggers only when both the previous rules
  are hit and give it a significant score
  
If you avoid sales terms and product names/descriptions that are in
common use the meta rule will cause few false positives.
 
Martin

Scoring by registrar?

[Sean Lynch]

Hi, everyone! I used to run my own mail servers back in the mid '90s and 
even worked as the postmaster for a regional ISP and worked on mail 
servers for some large corporations and even a small national ISP as a 
consultant. After a hiatus where I drank the hosted email kool-aid, I'm 
back to hosting my own email. At the moment I'm using a combination of 
SMTP-time DNSBL and other checks and SpamAssassin at delivery time for 
spam filtering. Very few spams are even making it to SpamAssassin, but 
many that do make it all the way through into my inbox.


A very large number (nearly all, in fact) of the spams I receive these 
days involve domains registered with Namecheap. I've received hundreds 
of spams involving .icu domains from what appear to be the same spammer. 
I also receive a large number of scams impersonating Bitmain, again 
using domains involving Namecheap.


While Namecheap does suspend at least some domains within days of their 
being used in a campaign, it's clear that these are being treated as 
single-use domains, so this has very little impact on the spammers. 
Since for whatever reason they're so attractive to spammers that they 
seem to be a nearly universal choice, at least for spams I get, I'd like 
to add a spam score to any message using a domain registered with them.


Does such functionality already exist in SpamAssassin? Is there an RHSBL 
or some other simple mechanism I could use to look up the registrar for 
a domain?

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[Dave Warren]

On 2019-05-14 09:17, John Hardin wrote:

On Tue, 14 May 2019, cyflhn wrote:

It has happened many times that the emails from our server were 
identified as

spam. I have checked the emails which were not identified as spam. But I
found that the SpamAssassin Scoring For MDAEMON_DNSBL is quite high, the
score of MDAEMON_DNSBL is always 4. I also checked the logs of 
SpamAssassin

and here are some messages:


Is this a local SA install, or some third party testing service? If the 
latter, who?



Performing DNS-BL lookup
* zen.spamhaus.org - passed
* bl.spamcop.net - passed
* bad.psky.me - failed - 198.54.117.200


That doesn't appear to be SA related. Is that just informational related 
data?



* 1.6 BAYES_50 BODY: Bayes spam probability is 40 to 60%
* [score: 0.5000]
* 4.0 MDAEMON_DNSBL MDaemon: marked by MDaemon\'s DNSBL
* 2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From



I still don't know what's reason for such a high score for MDAEMON_DNSBL


That rule is not in the base SA ruleset so we can't help you analyze it. 
I suggest you contact MDaemon to see why you're listed.


I've been aware on a family matter, but I can provide a bit of context 
about this particular rule. I previously worked with MDaemon (then Alt-N 
Technologies) and although this was some years ago I'm still familiar 
with the product and can help off-list if needed, feel free to reach out 
on or off list as applicable.


The "Performing DNS-BL lookup" header (above) shows the DNS-BLs which 
are configured in MDaemon and the results for each. bad.psky.me has not 
(to my knowledge) ever been a default in MDaemon.


Normally you should only use DNS-BLs for outright blocking at this stage 
(and let SpamAssassin's own DNS-BL functionality score) as this feature 
provides pre-DATA message rejection, but if you choose to accept 
messages that hit MDaemon's DNS-BL then you can pass points into 
SpamAssassin via the MDAEMON_DNSBL rule.


There are a few reasons for this, but mainly it comes down to the fact 
that MDaemon's DNS-BL implementation predated SpamAssassin being 
supported by MDaemon, and in the initial implementation there were a 
number of issues with SpamAssassin's implementation when running under 
Windows. These issues are long since resolved, but there is no incentive 
to remove the integration.


Removing the IP from bad.psky.me will cause the rule in SpamAssassin to 
disappear. Since bad.psky.me seems to be in a "list the world" phase the 
MDaemon administrator should completely remove this DNS-BL from the 
MDaemon configuration.


There is nothing a sender can do, only the receiving MDaemon server's 
administrator can make changes here.

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[RW]

On Tue, 14 May 2019 10:18:35 -0400
Kris Deugau wrote:

> Matus UHLAR - fantomas wrote:
> > On 14.05.19 06:18, cyflhn wrote:  
> >> but what about this one "FREEMAIL_FORGED_REPLYTO". why it got 2.1

> As for the score, it's autogenerated based on statistical analysis of
> a large corpus of mail along with most of the other stock rules: 
> https://wiki.apache.org/spamassassin/NightlyMassCheck

My understanding is that scores in 50_scores.cf haven't been optimized
for 10 years since the very fast perceptron fell-off and they had to
revert to the extremely slow genetic algorithm. The autogenerated
scores for a minority of the rules are in 72_scores.

This isn't entirely accurate because some rules are scored in both
files, but it gives a rough indication of the scale of problem. 
 
$ grep -E '^\s*score' 50_scores.cf| wc -l
 662

$ grep -E '^\s*score' 72_scores.cf| wc -l
 210

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[John Hardin]

On Tue, 14 May 2019, cyflhn wrote:


It has happened many times that the emails from our server were identified as
spam. I have checked the emails which were not identified as spam. But I
found that the SpamAssassin Scoring For MDAEMON_DNSBL is quite high, the
score of MDAEMON_DNSBL is always 4. I also checked the logs of SpamAssassin
and here are some messages:


Is this a local SA install, or some third party testing service? If the 
latter, who?



Performing DNS-BL lookup
* zen.spamhaus.org - passed
* bl.spamcop.net - passed
* bad.psky.me - failed - 198.54.117.200


That doesn't appear to be SA related. Is that just informational related 
data?



* 1.6 BAYES_50 BODY: Bayes spam probability is 40 to 60%
* [score: 0.5000]
* 4.0 MDAEMON_DNSBL MDaemon: marked by MDaemon\'s DNSBL
* 2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From



I still don't know what's reason for such a high score for MDAEMON_DNSBL


That rule is not in the base SA ruleset so we can't help you analyze it. I 
suggest you contact MDaemon to see why you're listed.



and why FREEMAIL_FORGED_REPLYTO  got 2.1 score?


The answer to that question for base SA rules is always: because it looks 
like a fairly good spam sign based on the masscheck corpora.


If those are emails you are generating, then why are you using a freemail 
address for the Reply-To: address them but not the From: address?


Providing the headers from such a message would make it easier for us to 
provide a more substantive analysis.



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Are you a mildly tech-literate politico horrified by the level of
  ignorance demonstrated by lawmakers gearing up to regulate online
  technology they don't even begin to grasp? Cool. Now you have a
  tiny glimpse into a day in the life of a gun owner.   -- Sean Davis
---
 Today: the 71st anniversary of Israel's independence

Re: SpamAssassin Scoring For No SPF Record

[Kevin A. McGrail]

+1.001.  Score it as a .001

On Tue, May 14, 2019, 10:18 Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:

> On 14 May 2019, at 11:03, RW wrote:
>
> > On Tue, 14 May 2019 01:32:43 -0400
> > Bill Cole wrote:
> >
> >> On 14 May 2019, at 0:41, Siddhesh wrote:
> >>
> >>> Hello Folks,
> >>>
> >>> Can we have a SpamAssassin score for a domain which doesn't have
> >>> SPF record set ??
> >>
> >> The rule SPF_NONE exists in the standard rules channel. It is
> >> currently scored at 0 by default.
> >
> > I don't see why it isn't scored at 0.001.
>
> The relevant comment in the 50_scores.cf file offers a dubious
> justification that scoring it would encourage spammers to publish valid
> SPF records just to avoid the rule. This would be a problem if SPF_PASS
> had a strong ham score and lots of spammers used domains theyb owned
> without SPF but I don't really see the logic when SPF_PASS is scored at
> -0.001 and not having any SPF record is no longer the mark of a spammer
> but rather a mark of mail apathy.
>
>

Re: SpamAssassin Scoring For No SPF Record

[Bill Cole]

On 14 May 2019, at 11:03, RW wrote:


On Tue, 14 May 2019 01:32:43 -0400
Bill Cole wrote:


On 14 May 2019, at 0:41, Siddhesh wrote:


Hello Folks,

Can we have a SpamAssassin score for a domain which doesn't have
SPF record set ??


The rule SPF_NONE exists in the standard rules channel. It is
currently scored at 0 by default.


I don't see why it isn't scored at 0.001.


The relevant comment in the 50_scores.cf file offers a dubious 
justification that scoring it would encourage spammers to publish valid 
SPF records just to avoid the rule. This would be a problem if SPF_PASS 
had a strong ham score and lots of spammers used domains theyb owned 
without SPF but I don't really see the logic when SPF_PASS is scored at 
-0.001 and not having any SPF record is no longer the mark of a spammer 
but rather a mark of mail apathy.

Re: SpamAssassin Scoring For No SPF Record

[RW]

On Tue, 14 May 2019 01:32:43 -0400
Bill Cole wrote:

> On 14 May 2019, at 0:41, Siddhesh wrote:
> 
> > Hello Folks,
> >
> > Can we have a SpamAssassin score for a domain which doesn't have
> > SPF record set ??  
> 
> The rule SPF_NONE exists in the standard rules channel. It is
> currently scored at 0 by default.

I don't see why it isn't scored at 0.001.

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[nektarios]

On Tue, 14 May 2019 05:49:24 -0700 (MST)
cyflhn  wrote:

> Thank you for your reply. But are you sure that it was caused by
> bad.psky.me? I could not find any useful information about
> bad.psky.me. It seems that it has been already shutdown.
> 
> 
> 
> --
> Sent from:
> http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Actually if you cannot remove your server from bad.psky.me service then
you need to actually contact the server that blocked to inform them
about the unreliable BL. No way round it.

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[RW]

On Tue, 14 May 2019 15:35:43 +0200
Matus UHLAR - fantomas wrote:

> On 14.05.19 06:18, cyflhn wrote:
> >but what about this one "FREEMAIL_FORGED_REPLYTO". why it got 2.1
> >score?  
> 
> this is standard rule where mail pretending to come from one freemail
> service really comes from another freemail service.


It's based  on a freemail Reply-To with a *non*-freemail From address.

It's a common pattern seen in fraud spams where a disposable account is
used for receiving replies.

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[Kris Deugau]

Matus UHLAR - fantomas wrote:

On 14.05.19 06:18, cyflhn wrote:

but what about this one "FREEMAIL_FORGED_REPLYTO". why it got 2.1 score?


this is standard rule where mail predenting to come from one freemail
service really comes from another freemail service.


Actually, unless I misread the rule, it's a message with a Reply-To 
pointing to a freemail service, where the message did not come from that 
specific service.  The actual origin could be anything from another 
freemail service to a small community ISP to a corporate groupware system.


As for the score, it's autogenerated based on statistical analysis of a 
large corpus of mail along with most of the other stock rules: 
https://wiki.apache.org/spamassassin/NightlyMassCheck


-kgd

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[Matus UHLAR - fantomas]

On 14.05.19 06:18, cyflhn wrote:

but what about this one "FREEMAIL_FORGED_REPLYTO". why it got 2.1 score?


this is standard rule where mail predenting to come from one freemail
service really comes from another freemail service.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[cyflhn]

but what about this one "FREEMAIL_FORGED_REPLYTO". why it got 2.1 score?



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[cyflhn]

Thank you for your reply. But are you sure that it was caused by bad.psky.me?
I could not find any useful information about bad.psky.me. It seems that it
has been already shutdown.



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SpamAssassin Scoring For MDAEMON_DNSBL

[Axb]

MDAEMON_DNSBL is not a stock SA rule.
This is a custom rule added by Alt-N/MDAEMON which uses SA.

You will need to contact them for further help


On 5/14/19 11:44 AM, cyflhn wrote:

It has happened many times that the emails from our server were identified as
spam. I have checked the emails which were not identified as spam. But I
found that the SpamAssassin Scoring For MDAEMON_DNSBL is quite high, the
score of MDAEMON_DNSBL is always 4. I also checked the logs of SpamAssassin
and here are some messages:

Performing DNS-BL lookup
* zen.spamhaus.org - passed
* bl.spamcop.net - passed
* bad.psky.me - failed - 198.54.117.200

* 1.6 BAYES_50 BODY: Bayes spam probability is 40 to 60%
* [score: 0.5000]
* 4.0 MDAEMON_DNSBL MDaemon: marked by MDaemon\'s DNSBL
* 2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From

The email sender has set up SPF record. I still don't know what's reason for
such a high score for MDAEMON_DNSBL  and why FREEMAIL_FORGED_REPLYTO  got
2.1 score?



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SpamAssassin Scoring For No SPF Record

[Kevin A. McGrail]

KAM.cf has lazy domain security rules as well.

On Tue, May 14, 2019, 01:33 Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:

> On 14 May 2019, at 0:41, Siddhesh wrote:
>
> > Hello Folks,
> >
> > Can we have a SpamAssassin score for a domain which doesn't have SPF
> > record set ??
>
> The rule SPF_NONE exists in the standard rules channel. It is currently
> scored at 0 by default. You are free to score it differently in your
> local configuration.  Good luck with that.
>
> --
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
>

SpamAssassin Scoring For MDAEMON_DNSBL

[cyflhn]

It has happened many times that the emails from our server were identified as
spam. I have checked the emails which were not identified as spam. But I
found that the SpamAssassin Scoring For MDAEMON_DNSBL is quite high, the
score of MDAEMON_DNSBL is always 4. I also checked the logs of SpamAssassin
and here are some messages:

Performing DNS-BL lookup 
* zen.spamhaus.org - passed
* bl.spamcop.net - passed
* bad.psky.me - failed - 198.54.117.200

* 1.6 BAYES_50 BODY: Bayes spam probability is 40 to 60%
* [score: 0.5000]
* 4.0 MDAEMON_DNSBL MDaemon: marked by MDaemon\'s DNSBL
* 2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From

The email sender has set up SPF record. I still don't know what's reason for
such a high score for MDAEMON_DNSBL  and why FREEMAIL_FORGED_REPLYTO  got
2.1 score?



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SpamAssassin Scoring For No SPF Record

[Bill Cole]

On 14 May 2019, at 0:41, Siddhesh wrote:


Hello Folks,

Can we have a SpamAssassin score for a domain which doesn't have SPF 
record set ??


The rule SPF_NONE exists in the standard rules channel. It is currently 
scored at 0 by default. You are free to score it differently in your 
local configuration.  Good luck with that.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)

SpamAssassin Scoring For No SPF Record

[Siddhesh]

Hello Folks,

Can we have a SpamAssassin score for a domain which doesn't have SPF 
record set ??


Regards,
Siddhesh Kadam

https://www.netcoresmartech.com/resources/ebooks/key-mobile-app-metrics?utm_source=email-disclaimer_medium=email_campaign=key-app-metrics