Upcoming KAM.cf Ruleset 20th Anniversary
Hi, very soon we will celebrate KAM.cf Ruleset 20th Anniversary, are there any stories about how you use the ruleset, any products that include the rules you are aware of, or other info about how it has helped with spam and email security ? Glad to receive any info or story about KAM.cf SpamAssassin ruleset. Giovanni OpenPGP_signature.asc Description: OpenPGP digital signature
Re: OFF-TOPIC ANNOUNCE: KAM Ruleset Turning PCCC Wild RBL Back On
With all respects, i agree with Bill... but suppose just Bill is wrong... Kam rules are free and show really huge quality, what is wrong about gently ask for cooperation if used in a commercial way? KAM++ Pedro. On Tuesday, March 21, 2023 at 06:18:38 PM GMT+1, Bill Cole wrote: On 2023-03-21 at 12:52:16 UTC-0400 (Tue, 21 Mar 2023 17:52:16 +0100) Benny Pedersen is rumored to have said: > Kevin A. McGrail skrev den 2023-03-21 17:27: > >> https://mcgrail.com/template/donate > > you know the rules to post commericial postings to public free > maillists ?, What rules exactly are you referring to? Please cite them precisely, in grammatically decipherable English. Note that if the 'rules' being cited are not on an ASF site, they do not apply here. The McGrail Foundation is not a commercial entity. That's why that page talks of donating rather than purchasing, and why it refers to a US tax code section. Noting the enhancement of a widely-used free service for SA users provided by a non-profit charitable foundation with in-kind support from a commercial entity (Linode, A.K.A. Amazon) is not a commercial posting. If you want kolektiva.social, it is over there... -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire
Re: OFF-TOPIC ANNOUNCE: KAM Ruleset Turning PCCC Wild RBL Back On
On 2023-03-21 at 12:52:16 UTC-0400 (Tue, 21 Mar 2023 17:52:16 +0100) Benny Pedersen is rumored to have said: Kevin A. McGrail skrev den 2023-03-21 17:27: https://mcgrail.com/template/donate you know the rules to post commericial postings to public free maillists ?, What rules exactly are you referring to? Please cite them precisely, in grammatically decipherable English. Note that if the 'rules' being cited are not on an ASF site, they do not apply here. The McGrail Foundation is not a commercial entity. That's why that page talks of donating rather than purchasing, and why it refers to a US tax code section. Noting the enhancement of a widely-used free service for SA users provided by a non-profit charitable foundation with in-kind support from a commercial entity (Linode, A.K.A. Amazon) is not a commercial posting. If you want kolektiva.social, it is over there... -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire
Re: OFF-TOPIC ANNOUNCE: KAM Ruleset Turning PCCC Wild RBL Back On
Kevin A. McGrail skrev den 2023-03-21 17:27: https://mcgrail.com/template/donate you know the rules to post commericial postings to public free maillists ?, rspamd did this abuse aswell, now thay have only non free irc support, and telegram more talk about linode ? :) mx ~ # dig -4 +short rs.dns-oarc.net txt rst.x487.rs.dns-oarc.net. rst.x461.x487.rs.dns-oarc.net. rst.x466.x461.x487.rs.dns-oarc.net. "172.104.150.56 DNS reply size limit is at least 487" "172.104.150.56 sent EDNS buffer size 512" great for dns i have a ticket to this problem at linode, there answer is try add new ips to your linode vps, you can be lucky a new one works :/ will it be possible to see kam channel could provide corpus data to spamassassin, its very low at moment
OFF-TOPIC ANNOUNCE: KAM Ruleset Turning PCCC Wild RBL Back On
Hello All, I am pleased to announce that users of the KAM ruleset will once again have the free use of the PCCC Wild RBL. The RBL was previously removed from use due to its popularity. Thanks go to Linode.com for donating the servers and as always thanks to PCCC for the datafeed. The KAM Ruleset will be celebrating the start of it's 20th year of publishing free rules and threat data in May 2023. If you are a commercial user, please consider a donation or long-term sponsorship: https://mcgrail.com/template/donate <https://mcgrail.com/template/donate> More Info: https://raptor.pccc.com/RBL <https://raptor.pccc.com/RBL> https://mcgrail.com/template/projects#KAM1 <https://mcgrail.com/template/projects#KAM1> Regards, KAM -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: How to verify specific commits are in current ruleset?
On 30 May 2019, at 10:57, Mike Ray wrote: Hello all- Been using spamassassin for awhile now, basically letting it run on auto-pilot and it's been great so far. However, after the recent __STYLE_GIBBERISH bug (https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7707), I need to have a little more understanding of SA. The LATEST problem is actually in a different bug that is classified as a "Security" bug, and so is not publicly visible. My biggest issue at the moment is that I saw John's message last night that said an update would be pushed out with yesterday's update (https://www.mail-archive.com/users@spamassassin.apache.org/msg104352.html). However, this morning, the only way I was able to verify that, was by looking at his change (https://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/jhardin/20_misc_testing.cf?r1=1857655&r2=1857654&pathrev=1857655) and comparing that to the code currently running on my mail servers. Is there any easier way to verify that a specific commit is in my currently running rule set? Yes. Compare the first line of /var/lib/spamassassin/3.004002/updates_spamassassin_org.cf to the revision number of the commit.
How to verify specific commits are in current ruleset?
Hello all- Been using spamassassin for awhile now, basically letting it run on auto-pilot and it's been great so far. However, after the recent __STYLE_GIBBERISH bug (https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7707), I need to have a little more understanding of SA. My biggest issue at the moment is that I saw John's message last night that said an update would be pushed out with yesterday's update (https://www.mail-archive.com/users@spamassassin.apache.org/msg104352.html). However, this morning, the only way I was able to verify that, was by looking at his change (https://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/jhardin/20_misc_testing.cf?r1=1857655&r2=1857654&pathrev=1857655) and comparing that to the code currently running on my mail servers. Is there any easier way to verify that a specific commit is in my currently running rule set? Mike Ray
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 11 Feb 2018, at 9:54 (-0500), Benny Pedersen wrote: first query would be valid for 300 secs, but that is imho still not free, problem is that keeping low ttls does not change how dns works, any auth dns servers will upate on soa serial anyway, the crime comes in when sa using remote dns servers that ignore soa serial updates in that case ttls would keep spammers listed for 300 secs only That's not how DNS TTLs work. When a record's TTL elapses in the local name cache, it is dropped. The next query for that name and record type causes the resolver to make another query to the authoritative nameservers, which will return the same record whose TTL expired unless it has been removed from the zone. No standards-conforming DNS resolver returns NXDOMAIN based on the lack of a non-expired record in its cache and an unchanged SOA serial above the name. That would make no sense at all and require many more SOA queries than actually happen. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Dave Warren skrev den 2018-02-06 20:39: How low are the TTLs? I'm seeing 300 seconds on 127.0.0.2 which is more than sufficient time for a single message to finish processing, such that multiple queries from one message would absolutely be cached (or more likely, the first would still be pending and the second would get the same answer as the first). first query would be valid for 300 secs, but that is imho still not free, problem is that keeping low ttls does not change how dns works, any auth dns servers will upate on soa serial anyway, the crime comes in when sa using remote dns servers that ignore soa serial updates in that case ttls would keep spammers listed for 300 secs only and thats why i say 300 secs helps spammers ;; ANSWER SECTION: 2.0.0.127.bb.barracudacentral.org. 300 IN A 127.0.0.2 Maybe the TTLs are different for other records? 300 is imho to low to anything thats called free i would like to accept free if it was 3600 I am also noticing very intermittent response times, sometimes taking over a second to get a response, other times taking under 50ms. rndc querylog is my friend i just like to start a debate on why 300 is accepted as free, it does matter for non datafeeds users, but for datafeeds it does not matter at all
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On Tue, 6 Feb 2018 11:38:42 -0500 Alex wrote: > On Tue, Feb 6, 2018 at 8:44 AM, David Jones wrote: ustomer's compromised accounts. > > > > Leave out the RCVD_IN_BRBL rule above and change the > > RCVD_IN_BRBL_LASTEXT score to 1.4 to keep things the same. > > If you think the RCVD_IN_BRBL rule is a good one, I'd like to use it, > and while I've implemented much of your approach, I can't implement > all of it. I've been doing deep XBL checks for years (they come free with the zen look-ups). Initially I found that they did have a low FP rate, but that's changed as more of my mail comes through mobile (cellular) networks that use NAT to support millions of users on thousands of IPv4 addresses. I'm seeing about 10% of ham submitted from these networks hitting my deep XBL rule. > Can I also ask again about reasonable RCVD_IN_LASHBACK and > RCVD_IN_LASHBACK_LASTEXT scores? If you're going to create deep versions of more than one list it doesn't make sense to score them individually. The dominant cause of FPs on such rules is dynamic IP address reuse, being in more than one list doesn't say anything about that. Allowing a large score to build-up from multiple deep rules is reckless.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Hi, >>> whitelist_auth *@bounce.mail.salesforce.com >>> whitelist_auth *@sendgrid.net >>> whitelist_auth *@*.mcdlv.net >> >> >> I've seen enough spam sent through all three - both by way of whole >> apparently spammer-owned accounts and cracked-but-otherwise-legitimate >> accounts - that I would never blanket-whitelist whole bulk email providers. >> >> Legitimate mail sent through them generally gets through anyway IME. > > An alternative is to use "def_whitelist_auth" instead of "whitelist_auth" > That gives a -7.5 point bump to usually good sources which may occasionally > get abused. > > That way if one of their accounts gets p0wned your anti-phish rules have a > chance of pulling the junk into the spam-tagged range. Phishing is a significant concern for us. Whether or not the phish came through the customer of one of these senders or through the senders themselves, whitelisting these senders only facilitates more phishes. There was a period when it was about one being reported by a particularly large customer per day. Telling my customers that we've contacted the provider and reported the spam just isn't good enough. We also received a phish through freshdesk.com which is on the def_whitelist. It's also on the DKIMWL_WL, subtracting another -3.5 points. It was also listed in RCVD_IN_HOSTKARMA_W, but also in LASHBACK_LASTEXT and invaluement, but not enough to compensate for the negative points. I suspect it was a compromised freshdesk trial account that was managed by amazonaws and sendgrid before passing through smtp.freshdesk.com, both of which weren't whitelisted at the time.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On Tue, 6 Feb 2018, Kris Deugau wrote:
Alex wrote:
These phishes we've received were all from otherwise trusted sources
like salesforce, amazonses and sendgrid. These are examples that I
believe were previously whitelisted because of having received a phish
through these systems but have no been disabled.
whitelist_auth *@bounce.mail.salesforce.com
whitelist_auth *@sendgrid.net
whitelist_auth *@*.mcdlv.net
I've seen enough spam sent through all three - both by way of whole
apparently spammer-owned accounts and cracked-but-otherwise-legitimate
accounts - that I would never blanket-whitelist whole bulk email providers.
Legitimate mail sent through them generally gets through anyway IME.
An alternative is to use "def_whitelist_auth" instead of "whitelist_auth"
That gives a -7.5 point bump to usually good sources which may occasionally get
abused.
That way if one of their accounts gets p0wned your anti-phish rules have a
chance of pulling the junk into the spam-tagged range.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 02/06/2018 01:28 PM, Alex wrote:
Hi,
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 &&
!RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal',
'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in
Barracuda
RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
You don't think these scores are a bit high for a normal installation?
The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
RCVD_IN_BRBL doesn't otherwise exist.
Also, does someone have a recommended score for the lashback RBL? I've
had it in testing for quite some time and would like to put it into
production with reasonable values...
Ok, ok. Uncle. (Waving white flag.) I have been sufficiently flogged so
I
have learned my lesson. :) This works in my highly customized SA
platform
where I have to do outbound mail filtering so deep Received header
checking
is valuable to block spam from my customer's compromised accounts.
Leave out the RCVD_IN_BRBL rule above and change the RCVD_IN_BRBL_LASTEXT
score to 1.4 to keep things the same.
I didn't mean to imply I don't agree or otherwise support your
approach. It was just unclear that this was in conjunction with that
approach of using higher spam rule scores to offset lower ham rule
scores or if it was recommended for everyone.
If you think the RCVD_IN_BRBL rule is a good one, I'd like to use it,
and while I've implemented much of your approach, I can't implement
all of it. My users raise holy hell when they receive even one phish
from an otherwise trustworthy source that's been whitelisted. It hits
on a ton of email at both ends of the spectrum - most are either very
low scoring or are already spam.
First let me say that my method for many whitelist_auth entries does not
allow for any phishing emails so if I find any of those, they do not get a
whitelist_auth entry. With a properly tuned MTA in front of SA, the only
phishing or blatant spam should be coming from compromised accounts or
zero-hour spam which are going to be difficult to block anyway.
Yes, I should have mentioned that as well. We're not whitelisting any
end-user level accounts.
These phishes we've received were all from otherwise trusted sources
like salesforce, amazonses and sendgrid. These are examples that I
believe were previously whitelisted because of having received a phish
through these systems but have no been disabled.
whitelist_auth *@bounce.mail.salesforce.com
whitelist_auth *@sendgrid.net
whitelist_auth *@*.mcdlv.net
I have never received phishing emails from trusted senders like those.
If I did, then they would not be considered a trusted sender. We have
received unsolicited junk emails from customers of theirs after someone
sold our email addresses to some so-called "double opt-in" email lists
that we never opted into. I have been reporting this abuse and those
rogue customers are immediately locked/blocked/disabled/etc. which helps
all of us. I have never received a malicious/infected/spoofing email
from those, just harmless junk email.
My method should only be whitelist_auth'ing system-generated/bulk emails
from reputable senders that handle abuse reports quickly and shouldn't match
compromised accounts and "freemail" domains. It will also match commonly
spoofed domains like fedex.com, ups.com, and banks to help block those
phishing emails.
This also requires supporting rules that add significant points based
on their name most simply, or other more complicated rules to catch
more sophisticated phish attempts. Spammers hitting our systems are
far more sophisticated than just using "UPS" in the subject or
pretending to be from ups.com.
Sure. What I have found effective is to train your Bayes with all
spoofing emails so high BAYES rule hits will add points. The authentic
emails will score very low from the whitelist_auth entry. Add in some
custom regex rules in the subject and body for variations of
misspellings of the spoofed brand or new zero-hour findings and this is
the best protection I have been able to come up with for new
spoofing/spam campaigns.
Can I also ask again about reasonable RCVD_IN_LASHBACK and
RCVD_IN_LASHBACK_LASTEXT scores?
It really depends on how much customization you have done to SA and how much
your mail flow can handle bumping up scores. If you do some log analysis
and find that RCVD_IN_LASHBACK and RCVD_IN_LASHBACK_LASTEXT are pretty
accurate for your mail flow, then you c
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Alex wrote: These phishes we've received were all from otherwise trusted sources like salesforce, amazonses and sendgrid. These are examples that I believe were previously whitelisted because of having received a phish through these systems but have no been disabled. whitelist_auth *@bounce.mail.salesforce.com whitelist_auth *@sendgrid.net whitelist_auth *@*.mcdlv.net I've seen enough spam sent through all three - both by way of whole apparently spammer-owned accounts and cracked-but-otherwise-legitimate accounts - that I would never blanket-whitelist whole bulk email providers. Legitimate mail sent through them generally gets through anyway IME. -kgd
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 2018-02-05 09:12, Benny Pedersen wrote: Kevin A. McGrail skrev den 2018-02-05 16:53: I don't think that will apply will it because it will be looking up something like 1.2.3.4.bb.barracuda.blah which isn't cached. the first qurry can make a qurry with very low ttl, so it would not be cached, that means number 2 query still mkae dns query to that zone :( How low are the TTLs? I'm seeing 300 seconds on 127.0.0.2 which is more than sufficient time for a single message to finish processing, such that multiple queries from one message would absolutely be cached (or more likely, the first would still be pending and the second would get the same answer as the first). ;; ANSWER SECTION: 2.0.0.127.bb.barracudacentral.org. 300 IN A 127.0.0.2 Maybe the TTLs are different for other records? I am also noticing very intermittent response times, sometimes taking over a second to get a response, other times taking under 50ms.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Hi,
> ifplugin Mail::SpamAssassin::Plugin::DNSEval
>
> header __RCVD_IN_BRBL eval:check_rbl('brbl',
> 'bb.barracudacentral.org')
> tflags __RCVD_IN_BRBL net
>
> header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
> '127.0.0.2')
> metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 &&
> !RCVD_IN_BRBL_LASTEXT
> describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
> bb.barracudacentral.org
> score RCVD_IN_BRBL1.2
> tflags RCVD_IN_BRBLnet
>
> header RCVD_IN_BRBL_LASTEXT
> eval:check_rbl('brbl-lastexternal',
> 'bb.barracudacentral.org')
> describeRCVD_IN_BRBL_LASTEXTLast external is listed in
> Barracuda
> RBL bb.barracudacentral.org
> score RCVD_IN_BRBL_LASTEXT2.2
> tflags RCVD_IN_BRBL_LASTEXTnet
>
> endif
You don't think these scores are a bit high for a normal installation?
The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
RCVD_IN_BRBL doesn't otherwise exist.
Also, does someone have a recommended score for the lashback RBL? I've
had it in testing for quite some time and would like to put it into
production with reasonable values...
>>>
>>> Ok, ok. Uncle. (Waving white flag.) I have been sufficiently flogged so
>>> I
>>> have learned my lesson. :) This works in my highly customized SA
>>> platform
>>> where I have to do outbound mail filtering so deep Received header
>>> checking
>>> is valuable to block spam from my customer's compromised accounts.
>>>
>>> Leave out the RCVD_IN_BRBL rule above and change the RCVD_IN_BRBL_LASTEXT
>>> score to 1.4 to keep things the same.
>>
>>
>> I didn't mean to imply I don't agree or otherwise support your
>> approach. It was just unclear that this was in conjunction with that
>> approach of using higher spam rule scores to offset lower ham rule
>> scores or if it was recommended for everyone.
>>
>> If you think the RCVD_IN_BRBL rule is a good one, I'd like to use it,
>> and while I've implemented much of your approach, I can't implement
>> all of it. My users raise holy hell when they receive even one phish
>> from an otherwise trustworthy source that's been whitelisted. It hits
>> on a ton of email at both ends of the spectrum - most are either very
>> low scoring or are already spam.
>>
>
> First let me say that my method for many whitelist_auth entries does not
> allow for any phishing emails so if I find any of those, they do not get a
> whitelist_auth entry. With a properly tuned MTA in front of SA, the only
> phishing or blatant spam should be coming from compromised accounts or
> zero-hour spam which are going to be difficult to block anyway.
Yes, I should have mentioned that as well. We're not whitelisting any
end-user level accounts.
These phishes we've received were all from otherwise trusted sources
like salesforce, amazonses and sendgrid. These are examples that I
believe were previously whitelisted because of having received a phish
through these systems but have no been disabled.
whitelist_auth *@bounce.mail.salesforce.com
whitelist_auth *@sendgrid.net
whitelist_auth *@*.mcdlv.net
> My method should only be whitelist_auth'ing system-generated/bulk emails
> from reputable senders that handle abuse reports quickly and shouldn't match
> compromised accounts and "freemail" domains. It will also match commonly
> spoofed domains like fedex.com, ups.com, and banks to help block those
> phishing emails.
This also requires supporting rules that add significant points based
on their name most simply, or other more complicated rules to catch
more sophisticated phish attempts. Spammers hitting our systems are
far more sophisticated than just using "UPS" in the subject or
pretending to be from ups.com.
>> Can I also ask again about reasonable RCVD_IN_LASHBACK and
>> RCVD_IN_LASHBACK_LASTEXT scores?
>>
>
> It really depends on how much customization you have done to SA and how much
> your mail flow can handle bumping up scores. If you do some log analysis
> and find that RCVD_IN_LASHBACK and RCVD_IN_LASHBACK_LASTEXT are pretty
> accurate for your mail flow, then you can bump it up like I have to 2.2 and
> 4.2 respectively.
>
> DISCLAIMER: I am not recommending this for everyone so no flaming. Set
> these scores low and test for a few weeks or months to see how your mail
> logs line up with real spam then increase the scores as you see fit. Again,
> I do outbound mail filtering for my customers so the deep Received header
> inspection is helpful to determine compromised accounts and keep my mail
> servers off of RBLs.
Are there any numbers available from anyone's masschecks?
Thank you as always for your support.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 02/06/2018 10:38 AM, Alex wrote:
Hi,
On Tue, Feb 6, 2018 at 8:44 AM, David Jones wrote:
On 02/05/2018 09:07 PM, Alex wrote:
Hi,
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT eval:check_rbl('brbl-lastexternal',
'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in
Barracuda
RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
You don't think these scores are a bit high for a normal installation?
The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
RCVD_IN_BRBL doesn't otherwise exist.
Also, does someone have a recommended score for the lashback RBL? I've
had it in testing for quite some time and would like to put it into
production with reasonable values...
Ok, ok. Uncle. (Waving white flag.) I have been sufficiently flogged so I
have learned my lesson. :) This works in my highly customized SA platform
where I have to do outbound mail filtering so deep Received header checking
is valuable to block spam from my customer's compromised accounts.
Leave out the RCVD_IN_BRBL rule above and change the RCVD_IN_BRBL_LASTEXT
score to 1.4 to keep things the same.
I didn't mean to imply I don't agree or otherwise support your
approach. It was just unclear that this was in conjunction with that
approach of using higher spam rule scores to offset lower ham rule
scores or if it was recommended for everyone.
If you think the RCVD_IN_BRBL rule is a good one, I'd like to use it,
and while I've implemented much of your approach, I can't implement
all of it. My users raise holy hell when they receive even one phish
from an otherwise trustworthy source that's been whitelisted. It hits
on a ton of email at both ends of the spectrum - most are either very
low scoring or are already spam.
First let me say that my method for many whitelist_auth entries does not
allow for any phishing emails so if I find any of those, they do not get
a whitelist_auth entry. With a properly tuned MTA in front of SA, the
only phishing or blatant spam should be coming from compromised accounts
or zero-hour spam which are going to be difficult to block anyway.
My method should only be whitelist_auth'ing system-generated/bulk emails
from reputable senders that handle abuse reports quickly and shouldn't
match compromised accounts and "freemail" domains. It will also match
commonly spoofed domains like fedex.com, ups.com, and banks to help
block those phishing emails.
Can I also ask again about reasonable RCVD_IN_LASHBACK and
RCVD_IN_LASHBACK_LASTEXT scores?
It really depends on how much customization you have done to SA and how
much your mail flow can handle bumping up scores. If you do some log
analysis and find that RCVD_IN_LASHBACK and RCVD_IN_LASHBACK_LASTEXT are
pretty accurate for your mail flow, then you can bump it up like I have
to 2.2 and 4.2 respectively.
DISCLAIMER: I am not recommending this for everyone so no flaming. Set
these scores low and test for a few weeks or months to see how your mail
logs line up with real spam then increase the scores as you see fit.
Again, I do outbound mail filtering for my customers so the deep
Received header inspection is helpful to determine compromised accounts
and keep my mail servers off of RBLs.
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_LASHBACK eval:check_rbl('lashback',
'ubl.unsubscore.com.')
describe __RCVD_IN_LASHBACK Received is listed in Lashback
ubl.unsubscore.com
tflags __RCVD_IN_LASHBACK net
header __RCVD_IN_LASHBACK_2eval:check_rbl_sub('lashback',
'127.0.0.2')
metaRCVD_IN_LASHBACK__RCVD_IN_LASHBACK_2 &&
!RCVD_IN_LASHBACK_LASTEXT
describeRCVD_IN_LASHBACKReceived is listed in Lashback
ubl.unsubscore.com
score RCVD_IN_LASHBACK2.2
tflags RCVD_IN_LASHBACKnet
header RCVD_IN_LASHBACK_LASTEXT eval:check_rbl('lashback-lastexternal',
'ubl.unsubscore.com.')
describe RCVD_IN_LASHBACK_LASTEXT Last external is listed in Lashback
ubl.unsubscore.com
score RCVD_IN_LASHBACK_LASTEXT4.2
tflags RCVD_IN_LASHBACK_LASTEXTnet
endif
--
David Jones
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Hi,
On Tue, Feb 6, 2018 at 8:44 AM, David Jones wrote:
> On 02/05/2018 09:07 PM, Alex wrote:
>>
>> Hi,
>>
>>> ifplugin Mail::SpamAssassin::Plugin::DNSEval
>>>
>>> header __RCVD_IN_BRBL eval:check_rbl('brbl',
>>> 'bb.barracudacentral.org')
>>> tflags __RCVD_IN_BRBL net
>>>
>>> header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
>>> '127.0.0.2')
>>> metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
>>> describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
>>> bb.barracudacentral.org
>>> score RCVD_IN_BRBL1.2
>>> tflags RCVD_IN_BRBLnet
>>>
>>> header RCVD_IN_BRBL_LASTEXT eval:check_rbl('brbl-lastexternal',
>>> 'bb.barracudacentral.org')
>>> describeRCVD_IN_BRBL_LASTEXTLast external is listed in
>>> Barracuda
>>> RBL bb.barracudacentral.org
>>> score RCVD_IN_BRBL_LASTEXT2.2
>>> tflags RCVD_IN_BRBL_LASTEXTnet
>>>
>>> endif
>>
>>
>> You don't think these scores are a bit high for a normal installation?
>> The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
>> RCVD_IN_BRBL doesn't otherwise exist.
>>
>> Also, does someone have a recommended score for the lashback RBL? I've
>> had it in testing for quite some time and would like to put it into
>> production with reasonable values...
>>
>
> Ok, ok. Uncle. (Waving white flag.) I have been sufficiently flogged so I
> have learned my lesson. :) This works in my highly customized SA platform
> where I have to do outbound mail filtering so deep Received header checking
> is valuable to block spam from my customer's compromised accounts.
>
> Leave out the RCVD_IN_BRBL rule above and change the RCVD_IN_BRBL_LASTEXT
> score to 1.4 to keep things the same.
I didn't mean to imply I don't agree or otherwise support your
approach. It was just unclear that this was in conjunction with that
approach of using higher spam rule scores to offset lower ham rule
scores or if it was recommended for everyone.
If you think the RCVD_IN_BRBL rule is a good one, I'd like to use it,
and while I've implemented much of your approach, I can't implement
all of it. My users raise holy hell when they receive even one phish
from an otherwise trustworthy source that's been whitelisted. It hits
on a ton of email at both ends of the spectrum - most are either very
low scoring or are already spam.
Can I also ask again about reasonable RCVD_IN_LASHBACK and
RCVD_IN_LASHBACK_LASTEXT scores?
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 02/05/2018 09:07 PM, Alex wrote:
Hi,
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT eval:check_rbl('brbl-lastexternal',
'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in Barracuda
RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
You don't think these scores are a bit high for a normal installation?
The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
RCVD_IN_BRBL doesn't otherwise exist.
Also, does someone have a recommended score for the lashback RBL? I've
had it in testing for quite some time and would like to put it into
production with reasonable values...
Ok, ok. Uncle. (Waving white flag.) I have been sufficiently flogged
so I have learned my lesson. :) This works in my highly customized SA
platform where I have to do outbound mail filtering so deep Received
header checking is valuable to block spam from my customer's compromised
accounts.
Leave out the RCVD_IN_BRBL rule above and change the
RCVD_IN_BRBL_LASTEXT score to 1.4 to keep things the same.
--
David Jones
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
David Jones skrev den 2018-02-05 15:09:
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 &&
!RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal', 'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in
Barracuda RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
On 05.02.18 16:26, Benny Pedersen wrote:
this rule makes 2 dns querys, waste of cpu performance, i would
suggest to drop the lastextnal,
this rule checks all untrusted IPs, while lastexternal uses only one (often
the most important - IP that sent mail to your systems).
also, because of caching, brbl-lastexternal result will be cached.
b.barracudacentral.org uses ttl of 300, which should be cached during the
mail processing.
if you want to spare DNS queries, leave only lastexternal.
so its only if ip is listed yes or
no, 50% dns querys saved, and still same hits on listed ips, why do
we need to help spammers ?
network checks including DNS lookups help much in spam processing, after
bayes they are second best mechanism to detect spam.
NOT using them is helping spammers.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Atheism is a non-prophet organization.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Hi,
> ifplugin Mail::SpamAssassin::Plugin::DNSEval
>
> header __RCVD_IN_BRBL eval:check_rbl('brbl',
> 'bb.barracudacentral.org')
> tflags __RCVD_IN_BRBL net
>
> header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
> '127.0.0.2')
> metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
> describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
> bb.barracudacentral.org
> score RCVD_IN_BRBL1.2
> tflags RCVD_IN_BRBLnet
>
> header RCVD_IN_BRBL_LASTEXT eval:check_rbl('brbl-lastexternal',
> 'bb.barracudacentral.org')
> describeRCVD_IN_BRBL_LASTEXTLast external is listed in Barracuda
> RBL bb.barracudacentral.org
> score RCVD_IN_BRBL_LASTEXT2.2
> tflags RCVD_IN_BRBL_LASTEXTnet
>
> endif
You don't think these scores are a bit high for a normal installation?
The current default score for RCVD_IN_BRBL_LASTEXT is 1.4 and
RCVD_IN_BRBL doesn't otherwise exist.
Also, does someone have a recommended score for the lashback RBL? I've
had it in testing for quite some time and would like to put it into
production with reasonable values...
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On Mon, 05 Feb 2018 17:12:08 +0100 Benny Pedersen wrote: > Kevin A. McGrail skrev den 2018-02-05 16:53: > > > I don't think that will apply will it because it will be looking up > > something like 1.2.3.4.bb.barracuda.blah which isn't cached. > > the first qurry can make a qurry with very low ttl, so it would not > be cached, that means number 2 query still mkae dns query to that > zone :( SA sends its DNS requests out early in rapid succession. The chances are that the local DNS cache would see the second request as a duplicate of a pending look-up. In that case caching is not needed. > > Anyway, we're debating a rule that's removed :) > lastexternal is still a mistake imho :=) lastexternal is correct, that's what RCVD_IN_BRBL_LASTEXT does. Making use of deep checks on lists containing dynamic addresses is risky, and likely to vary a lot between different mail flows. David's rules are not appropriate as a general replacement for RCVD_IN_BRBL_LASTEXT IMO.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 2/5/2018 11:32 AM, RW wrote: Just to clarify, there is no legal or moral obligation to do this, the 'bb' subdomain was created specifically so SA users wouldn't need to register. Anything you may read on the Barracuda site applies to the 'b' version. Barracuda has given no indication that anything has changed. I've been trying to reach Barracuda pretty doggedly about the issue for months about the issues with bb requiring registration hence the removal. If you have a contact, get in touch with them and we can relight the rule. I was hoping some discussion on list might get their attention. Regards, KAM
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On Mon, 5 Feb 2018 08:09:55 -0600 David Jones wrote: > Heads up! This RBL has been removed from the core SA ruleset. In 36 > to 48 hours sa-update will remove the RCVD_IN_BRBL_LASTEXT rule after > it has gone through the masscheck and rule promotion process. > > Details can be found here: > > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7417 > > To add this rule back manually, you should register here: > > http://barracudacentral.org/account/register Just to clarify, there is no legal or moral obligation to do this, the 'bb' subdomain was created specifically so SA users wouldn't need to register. Anything you may read on the Barracuda site applies to the 'b' version. Barracuda has given no indication that anything has changed. The issue is that some users have seen look-ups fail and attributed it to possible rate limiting on non-registered IP addresses. You should register if you use the 'b' list directly from an MTA. If you only run the 'bb' version from SA and use fixed IP addresses it *may* help with reliability if you register. If you run SA client side on a dynamic address there's no point in registering. > and add this to your local.cf: I would suggest people just copy the existing rule as it is. This list contains dynamic IP addresses so shouldn't be used for deep look-ups. > NOTE: recent masscheck processing shows RCVD_IN_BRBL_LASTEXT as the > most effective non-zero scoring rule to hit spam (43%) so it's > probably worth adding back to your local.cf on Wednesday or Thursday. If you don't rename the rule there's no need to wait. Note that once it's removed the scores for other rules will adjust to compensate for its absence, and this *may* lead to more FP's if it's left at its current score.
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Kevin A. McGrail skrev den 2018-02-05 16:53: I don't think that will apply will it because it will be looking up something like 1.2.3.4.bb.barracuda.blah which isn't cached. the first qurry can make a qurry with very low ttl, so it would not be cached, that means number 2 query still mkae dns query to that zone :( Anyway, we're debating a rule that's removed :) lastexternal is still a mistake imho :=) gosh i hate bind9 not have minimal ttl, good thing is that low ttl strikes back to badly configured dns zones :=)
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 02/05/2018 09:44 AM, Reindl Harald wrote:
Am 05.02.2018 um 16:36 schrieb David Jones:
On 02/05/2018 09:26 AM, Benny Pedersen wrote:
David Jones skrev den 2018-02-05 15:09:
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2 eval:check_rbl_sub('brbl',
'127.0.0.2')
meta RCVD_IN_BRBL __RCVD_IN_BRBL_2 &&
!RCVD_IN_BRBL_LASTEXT
describe RCVD_IN_BRBL Received is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL 1.2
tflags RCVD_IN_BRBL net
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal', 'bb.barracudacentral.org')
describe RCVD_IN_BRBL_LASTEXT Last external is listed in
Barracuda RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT 2.2
tflags RCVD_IN_BRBL_LASTEXT net
endif
this rule makes 2 dns querys, waste of cpu performance, i would
suggest to drop the lastextnal, so its only if ip is listed yes or
no, 50% dns querys saved, and still same hits on listed ips, why do
we need to help spammers ?
If you are running a local DNS cache like this list and the SA
documention recommends, does this really matter? My MTA should have
already queried this before SA does it so it should be in the local
DNS cache and not require a full recursive lookup from the SA rule above
1.2 poitns just because the IP of the previous hop is listet is pure
nonsense and it was even nosense as Barracuda Networks started with that
bullshit called "deep header inspection"
Barracuda and many other RBL's list here a ton of innocent IP's which
are nothing else than the endusers range which never should tocu an
inbound MX itself
so what the hell is the point that you give me 1.2 points because i use
as i should our MTA from my home-ip to send an ordianry mail?
It can be a sign of a compromised account. I just saw a compromised
account coming from Nigeria listed on BRBL through Office 365. Now that
O365 is finally adding the "x-originating-ip" header, we can detect
botnets sending via infected home computers.
Legit emails should have other rules subtracting points so a 1.2 should
not be a major factor in the score. My mail platform is scoring real
spam greater than 18 and usually in the 20's. I could leave this rule
out and be fine but most default SA instances would benefit from it. If
they want to lower the scores, then make them 0.2 and 1.2 then and use
them in meta rules.
--
David Jones
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 2/5/2018 10:36 AM, David Jones wrote: If you are running a local DNS cache like this list and the SA documention recommends, does this really matter? My MTA should have already queried this before SA does it so it should be in the local DNS cache and not require a full recursive lookup from the SA rule above. I don't think that will apply will it because it will be looking up something like 1.2.3.4.bb.barracuda.blah which isn't cached. Anyway, we're debating a rule that's removed :) Regards, KAM
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
David Jones skrev den 2018-02-05 16:36: If you are running a local DNS cache like this list and the SA documention recommends, does this really matter? My MTA should have already queried this before SA does it so it should be in the local DNS cache and not require a full recursive lookup from the SA rule above. yes it matters with low ttl, non datafeeds have low ttl, so it does not cache much on dns servers, sadly, this can be avoided in spamassassin without 2 dns querys it have always just be low priotet since we all assume datafedds where is does not matter This shouldn't be the case with a local DNS cache with the /etc/resolv.conf pointed to 127.0.0.1 or SA dns_server pointed to 127.0.0.1. already done here can lastexternal be solved to only use one query ? as is i think spamassassin should be changed so it can if not already on top of that bb. is for spamassassin, while b. was for mta stage, so 200% more querys to non datafeeds :/ it should have being one dns zone
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
On 02/05/2018 09:26 AM, Benny Pedersen wrote:
David Jones skrev den 2018-02-05 15:09:
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2 eval:check_rbl_sub('brbl',
'127.0.0.2')
meta RCVD_IN_BRBL __RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
describe RCVD_IN_BRBL Received is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL 1.2
tflags RCVD_IN_BRBL net
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal', 'bb.barracudacentral.org')
describe RCVD_IN_BRBL_LASTEXT Last external is listed in
Barracuda RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT 2.2
tflags RCVD_IN_BRBL_LASTEXT net
endif
this rule makes 2 dns querys, waste of cpu performance, i would suggest
to drop the lastextnal, so its only if ip is listed yes or no, 50% dns
querys saved, and still same hits on listed ips, why do we need to help
spammers ?
If you are running a local DNS cache like this list and the SA
documention recommends, does this really matter? My MTA should have
already queried this before SA does it so it should be in the local DNS
cache and not require a full recursive lookup from the SA rule above.
anyway i dont use it, above rule is only optimized for datafeeds, it
will drain non datafeed clients into hell
This shouldn't be the case with a local DNS cache with the
/etc/resolv.conf pointed to 127.0.0.1 or SA dns_server pointed to 127.0.0.1.
can lastexternal be solved to only use one query ?
as is i think spamassassin should be changed so it can if not already
--
David Jones
Re: Barracuda Reputation Block List (BRBL) removal from the SA ruleset
David Jones skrev den 2018-02-05 15:09:
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 &&
!RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal', 'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in
Barracuda RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
this rule makes 2 dns querys, waste of cpu performance, i would suggest
to drop the lastextnal, so its only if ip is listed yes or no, 50% dns
querys saved, and still same hits on listed ips, why do we need to help
spammers ?
anyway i dont use it, above rule is only optimized for datafeeds, it
will drain non datafeed clients into hell
can lastexternal be solved to only use one query ?
as is i think spamassassin should be changed so it can if not already
Barracuda Reputation Block List (BRBL) removal from the SA ruleset
Heads up! This RBL has been removed from the core SA ruleset. In 36 to
48 hours sa-update will remove the RCVD_IN_BRBL_LASTEXT rule after it
has gone through the masscheck and rule promotion process.
Details can be found here:
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7417
To add this rule back manually, you should register here:
http://barracudacentral.org/account/register
and add this to your local.cf:
ifplugin Mail::SpamAssassin::Plugin::DNSEval
header __RCVD_IN_BRBL eval:check_rbl('brbl',
'bb.barracudacentral.org')
tflags __RCVD_IN_BRBL net
header __RCVD_IN_BRBL_2eval:check_rbl_sub('brbl',
'127.0.0.2')
metaRCVD_IN_BRBL__RCVD_IN_BRBL_2 && !RCVD_IN_BRBL_LASTEXT
describeRCVD_IN_BRBLReceived is listed in Barracuda RBL
bb.barracudacentral.org
score RCVD_IN_BRBL1.2
tflags RCVD_IN_BRBLnet
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal', 'bb.barracudacentral.org')
describeRCVD_IN_BRBL_LASTEXTLast external is listed in
Barracuda RBL bb.barracudacentral.org
score RCVD_IN_BRBL_LASTEXT2.2
tflags RCVD_IN_BRBL_LASTEXTnet
endif
NOTE: recent masscheck processing shows RCVD_IN_BRBL_LASTEXT as the most
effective non-zero scoring rule to hit spam (43%) so it's probably worth
adding back to your local.cf on Wednesday or Thursday.
http://ruleqa.spamassassin.org/20180203-r1823022-n/RCVD_IN_BRBL_LASTEXT/detail
--
David Jones
Re: sa-update ruleset updates enabled again
On 11/19/2017 08:45 AM, David Mehler wrote: Hi, How does one get the new SA update rules? Thanks. Dave. Basically run the sa-update command. This should be cron'd or otherwise run automatically by whatever "glue" is calling Spamassassin. The "glue" could be an MTA like Postfix/Sendmail/Exim/procmail/fetchmail via a milter using spamd/spamc, a higher level program like amavis/mimedefang/MailScanner, or an MUA (mail client) like Thunderbird/Apple Mail. The higher-level "glue" will usually have their own way for running sa-update automatically for you in the background so just do some Googling based on your "glue." It shouldn't be run more frequently than about 4 hours since there are only 2 updates a day currently around 3 AM UTC and 9 AM UTC. https://wiki.apache.org/spamassassin/RuleUpdates On 11/19/17, David Jones wrote: On 11/18/2017 09:37 PM, John Hardin wrote: On Sun, 19 Nov 2017, Benny Pedersen wrote: David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. heads up :=) -- David Jones
Re: sa-update ruleset updates enabled again
On 11/18/2017 09:37 PM, John Hardin wrote: On Sun, 19 Nov 2017, Benny Pedersen wrote: David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. heads up :=) : delivery via smtp.ena.net[96.5.1.4]:25: host smtp.ena.net[96.5.1.4] said: 554 5.7.1 : Sender address rejected: Blocked TLD. Contact hostmas...@ena.com, supp...@ena.com or call (888)612-2880. (in reply to RCPT TO command) Right, published administrative contact addresses (particularly abuse@ and postmaster@) should have **no content filtering**. This is one of the more annoying things I run into when trying to be a good netizen, particularly when the domain gets all its email via Google. I have updated Postfix to not filter admin contacts so here comes the spam. :) I already had some local rules to subtract some points for these recipients in SA to make the blocking score 10.0 instead of the 6.0 default in MailScanner. -- David Jones
Re: sa-update ruleset updates enabled again
On 11/18/2017 09:46 AM, Benny Pedersen wrote: David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. may i ask why you tld block me ? sorry for asking here, private mails does not work I have allowed junc.eu. Normal mail flow on our mail filters doesn't receive any email from .eu so I allow them on-demand like this. Sorry about that. -- David Jones
Re: sa-update ruleset updates enabled again
On Sun, 19 Nov 2017, Benny Pedersen wrote: David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. heads up :=) : delivery via smtp.ena.net[96.5.1.4]:25: host smtp.ena.net[96.5.1.4] said: 554 5.7.1 : Sender address rejected: Blocked TLD. Contact hostmas...@ena.com, supp...@ena.com or call (888)612-2880. (in reply to RCPT TO command) Right, published administrative contact addresses (particularly abuse@ and postmaster@) should have **no content filtering**. This is one of the more annoying things I run into when trying to be a good netizen, particularly when the domain gets all its email via Google. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- If the rock of doom requires a gentle nudge away from Gaia to prevent a very bad day for Earthlings, NASA won’t be riding to the rescue. These days, NASA does dodgy weather research and outreach programs, not stuff in actual space with rockets piloted by flinty-eyed men called Buzz. -- Daily Bayonet --- 233 days since the first commercial re-flight of an orbital booster (SpaceX)
Re: sa-update ruleset updates enabled again
David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. heads up :=) : delivery via smtp.ena.net[96.5.1.4]:25: host smtp.ena.net[96.5.1.4] said: 554 5.7.1 : Sender address rejected: Blocked TLD. Contact hostmas...@ena.com, supp...@ena.com or call (888)612-2880. (in reply to RCPT TO command) i dont make long distance phone calls
Re: sa-update ruleset updates enabled again
David Jones skrev den 2017-11-18 16:26: Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. may i ask why you tld block me ? sorry for asking here, private mails does not work
sa-update ruleset updates enabled again
Heads up. DNS updates for sa-update have been enabled again. The next rules promotion will happen in about 11 hours around 2:30 AM UTC. -- David Jones
Re: Ruleset updates via nightly masscheck status
On 11/13/2017 02:33 PM, Tom Hendrikx wrote: On 28-10-17 15:20, David Jones wrote: On 10/27/2017 03:02 AM, Merijn van den Kroonenberg wrote: Please provide feedback in the next 48 hours -- positive or negative so I know we are good to enable DNS updates again on Sunday. After installing these rules, I'm seeing one warning in my log during spamassassin reload: Oct 27 09:48:24 myhostname spamd[16256]: rules: failed to run DKIM_VALID_EF test, skipping: Oct 27 09:48:24 myhostname spamd[16256]: (Can't locate object method "check_dkim_valid_envelopefrom" via package "Mail: [...]:SpamAssassin::PerMsgStatus" at (eval 1369) line 305. Oct 27 09:48:24 myhostname spamd[16256]: ) The DKIM_VALID_EF rule should not be published yet as it depends on a change in a Plugin. Tom, thank you for testing and providing feedback. I didn't notice this error because I had patched my DKIM.pm plugin for testing the new DKIM_VALID_EF rule (intended to be used in meta rules). I confirmed what you found on my default Fedora 26 installation. I have fixed the rulesets, specifically 25_dkim.cf and 50_scores.cf, to check for the SA version to remove this error and tested it. Monday's ruleset should have this fix after tomorrow's masscheck validates it. I will confirm Monday's ruleset has fixed this DKIM_VALID_EF error and let sa-update start updating again via DNS on Tuesday. If anyone else is testing the latest rulesets from the past couple of days, please provide feedback in the next 48 hours. And thank you for testing. Hi, I noticed that rule updates are still not live in DNS. Can I get an updated ruleset for additional testing somewhere, or are we going live? Regards, Tom So... Merijn has been helping me track down bugs. I apologize for the delay but every little tweak we make takes 1 to 3 days to test. We make some changes over the weekend that are showing good results. This morning we got rules promoted for the first time in about 4 weeks (new/different issue) so the masscheck tonight should generate a new ruleset. I will check things in the morning and send out an update. If we have a good 72_scores.cf then I will install the latest ruleset manually on my platforms. If everything looks fine, I will ask for volunteers again to do the same and check their scoring. -- David Jones
Re: Ruleset updates via nightly masscheck status
On 28-10-17 15:20, David Jones wrote: > On 10/27/2017 03:02 AM, Merijn van den Kroonenberg wrote: >> >>>> >>>> Please provide feedback in the next 48 hours -- positive or negative so >>>> I know we are good to enable DNS updates again on Sunday. >>>> >>> >>> After installing these rules, I'm seeing one warning in my log during >>> spamassassin reload: >>> >>> Oct 27 09:48:24 myhostname spamd[16256]: rules: failed to run >>> DKIM_VALID_EF test, skipping: >>> Oct 27 09:48:24 myhostname spamd[16256]: (Can't locate object method >>> "check_dkim_valid_envelopefrom" via package "Mail: >>> [...]:SpamAssassin::PerMsgStatus" at (eval 1369) line 305. >>> Oct 27 09:48:24 myhostname spamd[16256]: ) >> >> The DKIM_VALID_EF rule should not be published yet as it depends on a >> change in a Plugin. >> > > Tom, thank you for testing and providing feedback. I didn't notice this > error because I had patched my DKIM.pm plugin for testing the new > DKIM_VALID_EF rule (intended to be used in meta rules). I confirmed > what you found on my default Fedora 26 installation. > > I have fixed the rulesets, specifically 25_dkim.cf and 50_scores.cf, to > check for the SA version to remove this error and tested it. Monday's > ruleset should have this fix after tomorrow's masscheck validates it. > > I will confirm Monday's ruleset has fixed this DKIM_VALID_EF error and > let sa-update start updating again via DNS on Tuesday. > > If anyone else is testing the latest rulesets from the past couple of > days, please provide feedback in the next 48 hours. And thank you for > testing. Hi, I noticed that rule updates are still not live in DNS. Can I get an updated ruleset for additional testing somewhere, or are we going live? Regards, Tom
Re: Ruleset updates via nightly masscheck status
On 10/27/2017 03:02 AM, Merijn van den Kroonenberg wrote: Please provide feedback in the next 48 hours -- positive or negative so I know we are good to enable DNS updates again on Sunday. After installing these rules, I'm seeing one warning in my log during spamassassin reload: Oct 27 09:48:24 myhostname spamd[16256]: rules: failed to run DKIM_VALID_EF test, skipping: Oct 27 09:48:24 myhostname spamd[16256]: (Can't locate object method "check_dkim_valid_envelopefrom" via package "Mail: [...]:SpamAssassin::PerMsgStatus" at (eval 1369) line 305. Oct 27 09:48:24 myhostname spamd[16256]: ) The DKIM_VALID_EF rule should not be published yet as it depends on a change in a Plugin. Tom, thank you for testing and providing feedback. I didn't notice this error because I had patched my DKIM.pm plugin for testing the new DKIM_VALID_EF rule (intended to be used in meta rules). I confirmed what you found on my default Fedora 26 installation. I have fixed the rulesets, specifically 25_dkim.cf and 50_scores.cf, to check for the SA version to remove this error and tested it. Monday's ruleset should have this fix after tomorrow's masscheck validates it. I will confirm Monday's ruleset has fixed this DKIM_VALID_EF error and let sa-update start updating again via DNS on Tuesday. If anyone else is testing the latest rulesets from the past couple of days, please provide feedback in the next 48 hours. And thank you for testing. Any idea? This is ubuntu 16.04, latest ubuntu package (3.4.1-3) for spammassassin. Kind regards, Tom -- David Jones
Re: Ruleset updates via nightly masscheck status
>> >> Please provide feedback in the next 48 hours -- positive or negative so >> I know we are good to enable DNS updates again on Sunday. >> > > After installing these rules, I'm seeing one warning in my log during > spamassassin reload: > > Oct 27 09:48:24 myhostname spamd[16256]: rules: failed to run > DKIM_VALID_EF test, skipping: > Oct 27 09:48:24 myhostname spamd[16256]: (Can't locate object method > "check_dkim_valid_envelopefrom" via package "Mail: > [...]:SpamAssassin::PerMsgStatus" at (eval 1369) line 305. > Oct 27 09:48:24 myhostname spamd[16256]: ) The DKIM_VALID_EF rule should not be published yet as it depends on a change in a Plugin. > > Any idea? This is ubuntu 16.04, latest ubuntu package (3.4.1-3) for > spammassassin. > > > Kind regards, > Tom >
Re: Ruleset updates via nightly masscheck status
On 26-10-17 20:33, David Jones wrote: > On 10/26/2017 01:09 PM, David Jones wrote: >> On 10/25/2017 06:15 AM, David Jones wrote: >>> cd /tmp >>> wget http://sa-update.ena.com/1813149.tar.gz >>> wget http://sa-update.ena.com/1813149.tar.gz.sha1 >>> wget http://sa-update.ena.com/1813149.tar.gz.asc >>> sa-update -v --install 1813149.tar.gz > > Last night's run also successfully put the last known good 72_scores.cf > from March into the ruleset. > > Steps to manually installing last night's ruleset: > > cd /tmp > wget http://sa-update.ena.com/1813258.tar.gz > wget http://sa-update.ena.com/1813258.tar.gz.sha1 > wget http://sa-update.ena.com/1813258.tar.gz.asc > sa-update -v --install 1813258.tar.gz > > restart spamd, MailScanner, amavisd, mimedefang, etc. > > Please provide feedback in the next 48 hours -- positive or negative so > I know we are good to enable DNS updates again on Sunday. > After installing these rules, I'm seeing one warning in my log during spamassassin reload: Oct 27 09:48:24 myhostname spamd[16256]: rules: failed to run DKIM_VALID_EF test, skipping: Oct 27 09:48:24 myhostname spamd[16256]: (Can't locate object method "check_dkim_valid_envelopefrom" via package "Mail: [...]:SpamAssassin::PerMsgStatus" at (eval 1369) line 305. Oct 27 09:48:24 myhostname spamd[16256]: ) Any idea? This is ubuntu 16.04, latest ubuntu package (3.4.1-3) for spammassassin. Kind regards, Tom
Re: Ruleset updates via nightly masscheck status
On 10/26/2017 01:09 PM, David Jones wrote: On 10/25/2017 06:15 AM, David Jones wrote: cd /tmp wget http://sa-update.ena.com/1813149.tar.gz wget http://sa-update.ena.com/1813149.tar.gz.sha1 wget http://sa-update.ena.com/1813149.tar.gz.asc sa-update -v --install 1813149.tar.gz Last night's run also successfully put the last known good 72_scores.cf from March into the ruleset. Steps to manually installing last night's ruleset: cd /tmp wget http://sa-update.ena.com/1813258.tar.gz wget http://sa-update.ena.com/1813258.tar.gz.sha1 wget http://sa-update.ena.com/1813258.tar.gz.asc sa-update -v --install 1813258.tar.gz restart spamd, MailScanner, amavisd, mimedefang, etc. Please provide feedback in the next 48 hours -- positive or negative so I know we are good to enable DNS updates again on Sunday. I would like to enable DNS updates on Saturday evening (US time) so the sa-updates would start again on Sunday. P.S. Merijn van den Kroonenberg is helping me look at the root of the problem so we might be able to fix this properly in the next couple of weeks. -- David Jones
Re: Just interested: MIME validation ruleset and ASCII-0
30.05.2013 14:38, Simon Loewenthal kirjoitti: > > Hi there, > > The SA custom rulesets page refers to /MIME validation/ ruleset. This > is a small .cf file. I am interested in this rule: > > # ASCII-0 can crash mail clients. This is an absolute NO! > rawbody MIME_ASCII0 /\0/ > describeMIME_ASCII0 Message body contains ASCII-0 character > score MIME_ASCII0 1.5 > > > Does anyone know why this char should crash an email client? I did google a > little for ASCII charsets and this char, but I could not see how this might > cause a crash. I wonder if this is not caused by badly coded email client > implementations? > > Perhaps I am showing off my noob-ness :) > > Thanks! > Simon At least is a string terminator in C language library and most apps coded with it. May cause indefined behaviour. signature.asc Description: OpenPGP digital signature
Just interested: MIME validation ruleset and ASCII-0
Hi there, The SA custom rulesets page refers to _MIME validation_ ruleset. This is a small .cf file. I am interested in this rule: # ASCII-0 can crash mail clients. This is an absolute NO! rawbody MIME_ASCII0 // describe MIME_ASCII0 Message body contains ASCII-0 character score MIME_ASCII0 1.5 Does anyone know why this char should crash an email client? I did google a little for ASCII charsets and this char, but I could not see how this might cause a crash. I wonder if this is not caused by badly coded email client implementations? Perhaps I am showing off my noob-ness :) Thanks! Simon
Re: Using ZMI_GERMAN ruleset
Sorry for the delay. I don't read the list normally, so please always CC me if you want to reach me. On Mittwoch, 16. November 2011 Stefan Jakobs wrote: > the published ruleset in the update channel is much older than the > ruleset on the named website. > > # dig +short -t txt 2.3.3.70_zmi_german.cf.zmi.sa-update.dostech.net > txt "20100831" > > Is the update with sa-update still supported? I'm sorry, I've contacted "Daryl C. W. O'Shea" on Februar 12 and September 5, 2011, he wanted to look it up but it seems still old. @Daryl: Any chance you can fix that? If you tell me no, I can remove it from the list of possible updates. The correct way to get ZMI_GERMAN is currently per web on http://sa.zmi.at/rulesets/70_zmi_german.cf Version: 3.00.3 from 2011-12-19 If someone can describe exactly what to do to create my own channel with 10 minutes of work, I can do so, but I don't want to run around searching for that info, I'm too busy with other projects. I'd also need info what users would then need to do to get that updates. -- mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531 signature.asc Description: This is a digitally signed message part.
Re: Using ZMI_GERMAN ruleset
* Stefan Jakobs [2011-11-16 11:28]: > Hi list, > > the published ruleset in the update channel is much older than the ruleset on > the named website. > > # dig +short -t txt 2.3.3.70_zmi_german.cf.zmi.sa-update.dostech.net txt > "20100831" > > Is the update with sa-update still supported? Michael, could you answer that? Regards, Sebastian -- New GPG Key: 0x93A0B9CE (F4F6 B1A3 866B 26E9 450A 9D82 58A2 D94A 93A0 B9CE) Old GPG Key-ID: 0x76B79F20 (0x1B6034F476B79F20) 'Are you Death?' ... IT'S THE SCYTHE, ISN'T IT? PEOPLE ALWAYS NOTICE THE SCYTHE. -- Terry Pratchett, The Fifth Elephant
Re: Using ZMI_GERMAN ruleset
On Dienstag, 13. Dezember 2011 Axb wrote:
> patterns with >120 characters are not really efficient, in terms of
> speed and hit rate. They are very specific to certain campaigns and
> minimal template changes will render them useless as in:
>
> body __ZMIde_STOCK34 /Wir sind .{0,2}berzeugt, dass der
> Zeitpunkt sich an einem Unternehmen zu beteiligen, welches
> erfolgreich im Edelmetallhandel t.{0,2}tig ist, nicht besser sein
> k.{0,2}nnte/
>
> or
>
> body __ZMIde_SALE5 /In den letzten 5 Jahren hatte ich .{0,2}ber
> drei dutzend gut funktionierende Strategien, um die Zahl meiner
> Webseitenbesucher drastisch zu erh.{0,2}hen und dadurch meinen
> Umsatz anzukurbeln/
Since they get hits, no need to change them. Once I get reports about a
sentence that has been modified, I apply that to the rules. I need
feedback for those, of course.
And it should still be fast in terms of CPU as if there's (rule
__ZMIde_SALE5) no "In den l" in the message, the regex shouldn't have to
search too much, right? At least I'd guess it's an optimized search
which compares in 64bit steps, which is 8 chars at once?
Look for the "Krankenkassa" ruleset, this has been very active these
last weeks. All the time modifications from them, I get reports and
modify the rules accordingly.
And not to forget: Long sentences mean chance for a false positive drops
--
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc
it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531
signature.asc
Description: This is a digitally signed message part.
Re: Using ZMI_GERMAN ruleset
On 2011-12-13 8:34, Michael Monnerie wrote:
On Montag, 31. Oktober 2011 Axb wrote:
tried it and dumped due to low hit rate
stuff like
body ZMIde_JOBSEARCH6 /Dank sehr grossen Angagement, aber auch
der Umsetzung verschiedener Inovationen, konnte unsere Firma schon
nach vier Jahren auf die internationalen Ebene hinaufsteigen/
is not efficient
Its "efficient" in terms of "filtering only spam with zero false
positives", which is top priority for this ruleset. And you picked a
very old and very long rule. Most rules nowadays are just one or even
only part of a sentence, and it prooves very efficient. Stuff like the
__ZMIde_JOBEARN1-28 rules move false positives to 0, and I'm constantly
adding stuff.
I've now tried to remove all old cruft, that means single-line rules.
Rulesize went from 350KB to 296KB, that should save some RAM and CPU.
patterns with >120 characters are not really efficient, in terms of
speed and hit rate. They are very specific to certain campaigns and
minimal template changes will render them useless as in:
body __ZMIde_STOCK34 /Wir sind .{0,2}berzeugt, dass der Zeitpunkt
sich an einem Unternehmen zu beteiligen, welches erfolgreich im
Edelmetallhandel t.{0,2}tig ist, nicht besser sein k.{0,2}nnte/
or
body __ZMIde_SALE5 /In den letzten 5 Jahren hatte ich .{0,2}ber drei
dutzend gut funktionierende Strategien, um die Zahl meiner
Webseitenbesucher drastisch zu erh.{0,2}hen und dadurch meinen Umsatz
anzukurbeln/
Re: Using ZMI_GERMAN ruleset
On Montag, 31. Oktober 2011 Axb wrote: > tried it and dumped due to low hit rate > > stuff like > > body ZMIde_JOBSEARCH6 /Dank sehr grossen Angagement, aber auch > der Umsetzung verschiedener Inovationen, konnte unsere Firma schon > nach vier Jahren auf die internationalen Ebene hinaufsteigen/ > > is not efficient Its "efficient" in terms of "filtering only spam with zero false positives", which is top priority for this ruleset. And you picked a very old and very long rule. Most rules nowadays are just one or even only part of a sentence, and it prooves very efficient. Stuff like the __ZMIde_JOBEARN1-28 rules move false positives to 0, and I'm constantly adding stuff. I've now tried to remove all old cruft, that means single-line rules. Rulesize went from 350KB to 296KB, that should save some RAM and CPU. -- mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531 signature.asc Description: This is a digitally signed message part.
Re: Using ZMI_GERMAN ruleset
Hi list, the published ruleset in the update channel is much older than the ruleset on the named website. # dig +short -t txt 2.3.3.70_zmi_german.cf.zmi.sa-update.dostech.net txt "20100831" Is the update with sa-update still supported? Thanks and kind regards Stefan > Mo Okt 31 2011 14:43:22 CET von "Michael Monnerie" > Betreff: Using ZMI_GERMAN ruleset > > > > >Dear list, > > I'd like to receive some feedback on the usage of zmi_german. If you use > it, please report to spam-ger...@zmi.at and tell me what you think about > it. > > The ruleset is designed to filter only german spam, and is very safe. > Not a single report this year about FPs. If you didn't use it until now, > and get german spam, download it from > http://sa.zmi.at/rulesets/70_zmi_german.cf > > I'm seeking for people helping to cleanup and improve the filters. > Please contact me at spam-ger...@zmi.at >
Re: Using ZMI_GERMAN ruleset
On 2011-10-31 14:43, Michael Monnerie wrote: Dear list, I'd like to receive some feedback on the usage of zmi_german. If you use it, please report to spam-ger...@zmi.at and tell me what you think about it. The ruleset is designed to filter only german spam, and is very safe. Not a single report this year about FPs. If you didn't use it until now, and get german spam, download it from http://sa.zmi.at/rulesets/70_zmi_german.cf I'm seeking for people helping to cleanup and improve the filters. Please contact me at spam-ger...@zmi.at tried it and dumped due to low hit rate stuff like body ZMIde_JOBSEARCH6 /Dank sehr grossen Angagement, aber auch der Umsetzung verschiedener Inovationen, konnte unsere Firma schon nach vier Jahren auf die internationalen Ebene hinaufsteigen/ is not efficient
Using ZMI_GERMAN ruleset
Dear list, I'd like to receive some feedback on the usage of zmi_german. If you use it, please report to spam-ger...@zmi.at and tell me what you think about it. The ruleset is designed to filter only german spam, and is very safe. Not a single report this year about FPs. If you didn't use it until now, and get german spam, download it from http://sa.zmi.at/rulesets/70_zmi_german.cf I'm seeking for people helping to cleanup and improve the filters. Please contact me at spam-ger...@zmi.at -- mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531 signature.asc Description: This is a digitally signed message part.
Re: How to remove a domain from a stock or third-party 2tld ruleset?
Kris Deugau wrote: Karsten Bräckelmann wrote: Another approach, since I understand you want to query against a local URI DNSBL, is simply to use wildcard DNS entries. Thus, regardless of a 2tld listing and the resulting DNS lookup, it would return the same listing for the pure TLD and a second level TLD. Hmm. I hadn't thought of this, I'll give it a try and see if something chokes. Thanks! This seems to be a usable way to work around a domain in the stock util_rb_2tld lists. I added *.t35.com (made a convenient test case - actually listed locally with util_rb_2tld; going to remove it eventually) to our local URI blacklist, and while there have been missed spams with t35.com subdomains, none have shown up in the list to be added to the blacklist since I did so. -kgd
Re: How to remove a domain from a stock or third-party 2tld ruleset?
On 2010-05-28 23:57, Kris Deugau wrote: Karsten Bräckelmann wrote: On Wed, 2010-05-26 at 11:35 -0400, Kris Deugau wrote: Is there any way to take a domain listed with util_rb_2tld, and "un-2tld" it (similar to how you can unwhitelist stock whitelist entries if they don't work well with your mail)? IIRC this is not possible. Well, possible, but there's just no code to handle it. ;) Didn't think so, but... I recently came across a "free-subsite" domain that seems to be part of a cluster of **very** similar sites which I've given up listing subdomains for locally; instead I've added the TLDs to a local blacklist. For now I've just added a regular uri rule, but I'm pretty sure that won't scale, and it doesn't help with some of the automation I've been using to extract URIs not listed on any DNSBL yet from missed-spam reports. uri rules should work. I wouldn't worry about scaling too much, because the number of util_rb_2tld listings is limited. Another approach, since I understand you want to query against a local URI DNSBL, is simply to use wildcard DNS entries. Thus, regardless of a 2tld listing and the resulting DNS lookup, it would return the same listing for the pure TLD and a second level TLD. Hmm. I hadn't thought of this, I'll give it a try and see if something chokes. Thanks! let me guess... .co.cc ?
Re: How to remove a domain from a stock or third-party 2tld ruleset?
Karsten Bräckelmann wrote: On Wed, 2010-05-26 at 11:35 -0400, Kris Deugau wrote: Is there any way to take a domain listed with util_rb_2tld, and "un-2tld" it (similar to how you can unwhitelist stock whitelist entries if they don't work well with your mail)? IIRC this is not possible. Well, possible, but there's just no code to handle it. ;) Didn't think so, but... I recently came across a "free-subsite" domain that seems to be part of a cluster of **very** similar sites which I've given up listing subdomains for locally; instead I've added the TLDs to a local blacklist. For now I've just added a regular uri rule, but I'm pretty sure that won't scale, and it doesn't help with some of the automation I've been using to extract URIs not listed on any DNSBL yet from missed-spam reports. uri rules should work. I wouldn't worry about scaling too much, because the number of util_rb_2tld listings is limited. Another approach, since I understand you want to query against a local URI DNSBL, is simply to use wildcard DNS entries. Thus, regardless of a 2tld listing and the resulting DNS lookup, it would return the same listing for the pure TLD and a second level TLD. Hmm. I hadn't thought of this, I'll give it a try and see if something chokes. Thanks! -kgd
Re: How to remove a domain from a stock or third-party 2tld ruleset?
On Wed, 2010-05-26 at 11:35 -0400, Kris Deugau wrote:
> Is there any way to take a domain listed with util_rb_2tld, and
> "un-2tld" it (similar to how you can unwhitelist stock whitelist entries
> if they don't work well with your mail)?
IIRC this is not possible. Well, possible, but there's just no code to
handle it. ;)
> I recently came across a "free-subsite" domain that seems to be part of
> a cluster of **very** similar sites which I've given up listing
> subdomains for locally; instead I've added the TLDs to a local blacklist.
> For now I've just added a regular uri rule, but I'm pretty sure that
> won't scale, and it doesn't help with some of the automation I've been
> using to extract URIs not listed on any DNSBL yet from missed-spam reports.
uri rules should work. I wouldn't worry about scaling too much, because
the number of util_rb_2tld listings is limited.
Another approach, since I understand you want to query against a local
URI DNSBL, is simply to use wildcard DNS entries. Thus, regardless of a
2tld listing and the resulting DNS lookup, it would return the same
listing for the pure TLD and a second level TLD.
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
How to remove a domain from a stock or third-party 2tld ruleset?
Is there any way to take a domain listed with util_rb_2tld, and "un-2tld" it (similar to how you can unwhitelist stock whitelist entries if they don't work well with your mail)? I recently came across a "free-subsite" domain that seems to be part of a cluster of **very** similar sites which I've given up listing subdomains for locally; instead I've added the TLDs to a local blacklist. The domain that's in the stock 2tld list is bravepages.com; it seems to be Yet Another Face of 0catch.com, and I've seen these domains as well: 1accesshost.com bigheadhosting.net easyfreehosting.com envy.nu digitalzones.com And no doubt there are a fairly long list of others in the cluster. For now I've just added a regular uri rule, but I'm pretty sure that won't scale, and it doesn't help with some of the automation I've been using to extract URIs not listed on any DNSBL yet from missed-spam reports. -kgd
RE: GERMAN ruleset extended
> Dear users, > > I felt I didn't advertise our GERMAN ruleset since a long time: > > http://wiki.apache.org/spamassassin/CustomRulesets Good hint. Thank you. Giampaolo
GERMAN ruleset extended
Dear users, I felt I didn't advertise our GERMAN ruleset since a long time: http://wiki.apache.org/spamassassin/CustomRulesets Please feel free to implement it to catch out german spam. Should you use it and still receive german spam, please report that mail *including all headers* to spam-ger...@zmi.at I don't follow the list regularly, so please contact me there in case of problems. -- mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services http://proteger.at [gesprochen: Prot-e-schee] Tel: 0660 / 415 65 31 // Wir haben im Moment zwei Häuser zu verkaufen: // http://zmi.at/langegg/ // http://zmi.at/haus2009/ signature.asc Description: This is a digitally signed message part.
Re: SOUGHT ruleset FP
Hi, On Fri, Apr 16, 2010 at 01:53:55PM +0200, Karsten Bräckelmann wrote: > On Fri, 2010-04-16 at 12:20 +0100, Matthew Newton wrote: > > We had a legitimate e-mail hit the JM_SOUGHT_3 yesterday. It also > > hit a few other rules that pushed it over our reject threshold of > > 10, and easily over the 'junk mail folder' level of 5. > > > > I managed to get them to send me the message, and it hits rule > > __SEEK_5ID3LI "Conti nuum Intern ational Publishing" (spaces > > added!) which is the name of their company. > > Makes one wonder how that string ends up quite massively in spam traps. I did consider that. Without seeing the spam, of course, I can't say whether they are spamming or whether their name is being abused. All I have is a legitimate mail from them and a report that it is blocked. > > I know SOUGHT is an auto-generated ruleset; just wondering if > > there is there any way to remove false positives before the set is > > Yes. The Seek bits are cross-checked against a ham corpus, so the > easiest way is to inject an artificial ham message with the string in > question to get it off of the next run. OK, understood. > > generated? Otherwise I'll add local rules to compensate against > > this one. > > meta __SEEK_5ID3LI (0) > > The Seek ID is constant, and will be the same even with later Sought > runs, for a given string. Thought that might be the case from the codes - thanks for the confirmation. Cheers, Matthew -- Matthew Newton, Ph.D. Systems Architect (UNIX and Networks), Network Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253,
Re: SOUGHT ruleset FP
yep -- feel free to send me over copies of FP messages (or strings
that match them)
2010/4/16 Karsten Bräckelmann :
> On Fri, 2010-04-16 at 12:20 +0100, Matthew Newton wrote:
>> We had a legitimate e-mail hit the JM_SOUGHT_3 yesterday. It also
>> hit a few other rules that pushed it over our reject threshold of
>> 10, and easily over the 'junk mail folder' level of 5.
>>
>> I managed to get them to send me the message, and it hits rule
>> __SEEK_5ID3LI "Conti nuum Intern ational Publishing" (spaces
>> added!) which is the name of their company.
>
> Makes one wonder how that string ends up quite massively in spam traps.
>
>> I know SOUGHT is an auto-generated ruleset; just wondering if
>> there is there any way to remove false positives before the set is
>
> Yes. The Seek bits are cross-checked against a ham corpus, so the
> easiest way is to inject an artificial ham message with the string in
> question to get it off of the next run.
>
>> generated? Otherwise I'll add local rules to compensate against
>> this one.
>
> meta __SEEK_5ID3LI (0)
>
> The Seek ID is constant, and will be the same even with later Sought
> runs, for a given string.
>
> guenther
>
>
> --
> char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
>
>
Re: SOUGHT ruleset FP
On Fri, 2010-04-16 at 12:20 +0100, Matthew Newton wrote:
> We had a legitimate e-mail hit the JM_SOUGHT_3 yesterday. It also
> hit a few other rules that pushed it over our reject threshold of
> 10, and easily over the 'junk mail folder' level of 5.
>
> I managed to get them to send me the message, and it hits rule
> __SEEK_5ID3LI "Conti nuum Intern ational Publishing" (spaces
> added!) which is the name of their company.
Makes one wonder how that string ends up quite massively in spam traps.
> I know SOUGHT is an auto-generated ruleset; just wondering if
> there is there any way to remove false positives before the set is
Yes. The Seek bits are cross-checked against a ham corpus, so the
easiest way is to inject an artificial ham message with the string in
question to get it off of the next run.
> generated? Otherwise I'll add local rules to compensate against
> this one.
meta __SEEK_5ID3LI (0)
The Seek ID is constant, and will be the same even with later Sought
runs, for a given string.
guenther
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
SOUGHT ruleset FP
Hi, We had a legitimate e-mail hit the JM_SOUGHT_3 yesterday. It also hit a few other rules that pushed it over our reject threshold of 10, and easily over the 'junk mail folder' level of 5. I managed to get them to send me the message, and it hits rule __SEEK_5ID3LI "Conti nuum Intern ational Publishing" (spaces added!) which is the name of their company. I know SOUGHT is an auto-generated ruleset; just wondering if there is there any way to remove false positives before the set is generated? Otherwise I'll add local rules to compensate against this one. Thanks, Matthew -- Matthew Newton, Ph.D. Systems Architect (UNIX and Networks), Network Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253,
Re: Hijacked thread :) (was: ruleset for German...)
On Mon, 15 Mar 2010, Karsten Bräckelmann wrote: The TextCat plugin. Even part of stock SA, though not enabled by default. Supports per-user settings. (nod) For reasons specific to my MTA, I can't run SA 'per user', but I can choose the most common languages (en fr) in our system's mail and flag when neither of them are used (assigning UNWANTED_LANGUAGE_BODY a minimal score) - then the user can set a procmail delivery rule (quarantine when that rule is present in the X-Spam headers). It will do. :) But you just forked (to avoid the word hijacked) this thread, which is about a very specific, on-going spam run. The OP really doesn't want to identify German spam for scoring, cause that's likely his first language. ;) My bad. :) But my compliments on the OP's excellent English! :) - C
Re: [sa] Re: ruleset for German Bettchen and Schlafzimmer spam
On Mon, Mar 15, 2010 at 11:17:09PM +0100, Karsten Bräckelmann wrote: > On Mon, 2010-03-15 at 11:15 -0400, Charles Gregory wrote: > > H. I guess this goes back to my inquiry about the Brazilian spam > > > > I'm still looking for a way (hopefully) to simply identify the *language* > > of the mail (when not determined from CHARSET_FARAWAY rules), so that our > > users may opt-in for additional filtering based on language > > The TextCat plugin. Even part of stock SA, though not enabled by > default. Supports per-user settings. Though given the current bugs, I really would do any hard filtering..
Re: [sa] Re: ruleset for German Bettchen and Schlafzimmer spam
On Mon, 2010-03-15 at 11:15 -0400, Charles Gregory wrote:
> H. I guess this goes back to my inquiry about the Brazilian spam
>
> I'm still looking for a way (hopefully) to simply identify the *language*
> of the mail (when not determined from CHARSET_FARAWAY rules), so that our
> users may opt-in for additional filtering based on language
The TextCat plugin. Even part of stock SA, though not enabled by
default. Supports per-user settings.
But you just forked (to avoid the word hijacked) this thread, which is
about a very specific, on-going spam run. The OP really doesn't want to
identify German spam for scoring, cause that's likely his first
language. ;)
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: [sa] Re: ruleset for German Bettchen and Schlafzimmer spam
On Sun, 14 Mar 2010, Jörg Frings-Fürst wrote: take a look at http://wiki.apache.org/spamassassin/CustomRulesets and search to "German Language Ruleset". H. I guess this goes back to my inquiry about the Brazilian spam I'm still looking for a way (hopefully) to simply identify the *language* of the mail (when not determined from CHARSET_FARAWAY rules), so that our users may opt-in for additional filtering based on language - Charles
Re: ruleset for German Bettchen and Schlafzimmer spam
Am 15.03.2010 03:14, schrieb Marcus: > Am Sonntag, den 14.03.2010, 23:31 +0100 schrieb Kai Schaetzl: >> Marcus wrote on Sun, 14 Mar 2010 21:16:31 +0100: >> >>> The messages differ in subject and body. >> >> Do they? Hm, I must have overlooked this in your first message. Oh, wait, >> after reading it a second time I still can't see it. I think you must have >> forgotten to mention it. >> If you want help from the mailing list post some complete messages to >> pastebin. > > Seems to be that other german speaking members of this list know what > sort of spams I'm talking about. I got some helpful rulesets from them > for this case. Thanks! > > Kai, thank you for your constructive help and good night, > Marcus > Hi, i only get these from a forward service from gmx , they are allready marked then , i let spamassassin them autolearn by rule afterwards, so if they come next , they are noticed as spam so manual learning these mails should help as tmp workaround analyse the headers to find out which procedure fits best to catch/mark/reject them use more rulesets anyway as recommended before -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
Re: ruleset for German Bettchen and Schlafzimmer spam
Am Sonntag, den 14.03.2010, 23:31 +0100 schrieb Kai Schaetzl: > Marcus wrote on Sun, 14 Mar 2010 21:16:31 +0100: > > > The messages differ in subject and body. > > Do they? Hm, I must have overlooked this in your first message. Oh, wait, > after reading it a second time I still can't see it. I think you must have > forgotten to mention it. > If you want help from the mailing list post some complete messages to > pastebin. Seems to be that other german speaking members of this list know what sort of spams I'm talking about. I got some helpful rulesets from them for this case. Thanks! Kai, thank you for your constructive help and good night, Marcus
Re: ruleset for German Bettchen and Schlafzimmer spam
Marcus wrote on Sun, 14 Mar 2010 21:16:31 +0100: > The messages differ in subject and body. Do they? Hm, I must have overlooked this in your first message. Oh, wait, after reading it a second time I still can't see it. I think you must have forgotten to mention it. If you want help from the mailing list post some complete messages to pastebin. Kai -- Get your web at Conactive Internet Services: http://www.conactive.com
Re: ruleset for German Bettchen and Schlafzimmer spam
Am Sonntag, den 14.03.2010, 18:31 +0100 schrieb Kai Schaetzl: > Marcus wrote on Sun, 14 Mar 2010 17:10:08 +0100: > > > I'm getting a lot of these German 'Mehr aus dauer in Ihrem Bettchen' and > > 'Mehr ausdauer in Ihrem Schlafzimmer'. I've learnd my spamassassin for > > about a week. It decects some of them, but most are going through. All > > other kind of spam is detected very well. Did some know oder wrote a > > ruleset? > > what is so difficult to match against Bettchen or Schlafzimmer? The messages differ in subject and body. > I'd say > even a complete newbie will have this rule up and running after ten > minutes and reading the rules how-to on the wiki. Ciao!
Re: ruleset for German Bettchen and Schlafzimmer spam
Am Sonntag 14 März 2010 17:10:08 schrieb Marcus: > Hi, > > I'm getting a lot of these German 'Mehr aus dauer in Ihrem Bettchen' and > 'Mehr ausdauer in Ihrem Schlafzimmer'. I've learnd my spamassassin for > about a week. It decects some of them, but most are going through. All > other kind of spam is detected very well. Did some know oder wrote a > ruleset? > > Ciao, > Marcus > Hi Marcus, take a look at http://wiki.apache.org/spamassassin/CustomRulesets and search to "German Language Ruleset". CU Joerg Frings-Fuerst
Re: ruleset for German Bettchen and Schlafzimmer spam
Marcus wrote on Sun, 14 Mar 2010 17:10:08 +0100: > I'm getting a lot of these German 'Mehr aus dauer in Ihrem Bettchen' and > 'Mehr ausdauer in Ihrem Schlafzimmer'. I've learnd my spamassassin for > about a week. It decects some of them, but most are going through. All > other kind of spam is detected very well. Did some know oder wrote a > ruleset? what is so difficult to match against Bettchen or Schlafzimmer? I'd say even a complete newbie will have this rule up and running after ten minutes and reading the rules how-to on the wiki. Kai -- Get your web at Conactive Internet Services: http://www.conactive.com
ruleset for German Bettchen and Schlafzimmer spam
Hi, I'm getting a lot of these German 'Mehr aus dauer in Ihrem Bettchen' and 'Mehr ausdauer in Ihrem Schlafzimmer'. I've learnd my spamassassin for about a week. It decects some of them, but most are going through. All other kind of spam is detected very well. Did some know oder wrote a ruleset? Ciao, Marcus
Re: SpamAssassin Ruleset Generation
On Tue, 2009-10-06 at 13:50 -0700, an anonymous Nabble user wrote:
> Other than the sought rules, all the rules are manually generated?
Actually, as has been said, I believe all stock rules are manually
written. There are some third-party rule-sets out there that are auto
generated -- not limited to Sought.
> Is there any statistics on how frequently are new rules/regex adopted by
> spamassasssin? Who are the people who write them? Any details related to it?
Somehow this begs the question -- why?
Why are you asking? Why and what are you ultimately interested in?
And of course, did you even consider to dig through the SVN repo, some
docs on the wiki and to ask google? Most of this should be pretty easy
to find out if you're willing to read some.
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: SpamAssassin Ruleset Generation
poifgh wrote:
> I have a question about - understanding how are rulesets generated for
> spamassassin.
>
> For example - consider the rule in 20_drugs.cf :
> header SUBJECT_DRUG_GAP_C Subject =~
> /\bc.{0,2}i.{0,2}a.{0,2}l.{0,2}i.{0,2}s\b/i
> describe SUBJECT_DRUG_GAP_C Subject contains a gappy version of 'cialis'
>
> Who generated the regular expression
> "/\bc.{0,2}i.{0,2}a.{0,2}l.{0,2}i.{0,2}s\b/i"
>
Man, that's a good question. I wrote a large chunk of the rules in
20_drugs.cf, but not that one. ( I wrote the stuff near the bottom that
uses meta rules. ie: __DRUGS_ERECTILE1 through DRUGS_MANYKINDS,
originally distributed as a separate set called antidrug.cf). As I
recall, there were 2 other people making drug rules, but it's been a
LONG time, and I forget who did it. Those rules were written in the
2004-2006 time frame when pharmacy spams were just hammering the heck
outa everyone.
> a. Is it done manually with people writing regex to see how efficiently they
> capture spams?
>
Yes. Many hours of reading spams, studying them, testing various regex
tweaks, checking for false positives, etc, etc.
mass-check is your friend for this kind of stuff.
One post from when I was developing this as a stand-alone set:
http://mail-archives.apache.org/mod_mbox/spamassassin-users/200404.mbox/%3c6.0.0.22.0.20040428132346.029d9...@opal.evi-inc.com%3e
Note: the comcast link mentioned in that message should be considered
DEAD. The antidrug set is no longer maintained separately from the
mailline ruleset, and hasn't been for years.
If you want to break the rules down a bit, here's some tips:
The rules are in general designed to detect common methods to obscure
text by inserting spaces, punctuation, etc between letters, and possibly
substituting some of the letters for other similar looking characters.
(W4R3Z style, etc)
The simple format would be to think of it in groupings. You end up using
a repeating pattern of (some representation of a character)(some kind of
"gap" sequence)(character)(gap)...etc.
.{0,2} is a "gap sequence", although not one I prefer. I prefer
[_\W]{0,3} in most cases because it's a bit less FP-prone, but risks
missing things using small lower-case letters to gap.
You also get replacements for characters in some of those, like [A4]
instead of just A. Or, more elaborately.. [a4\xe0-\...@]
So this mess:
body __DRUGS_ERECTILE1
/(?:\b|\s)[_\W]{0,3}(?:\\\/|V)[_\W]{0,3}[ij1!|l\xEC\xED\xEE\xEF][_\W]{0,3}[a40\xe0-\...@][_\w]{0,3}[xyz]?[gj][_\W]{0,3}r[_\W]{0,3}[a40\xe0-\...@][_\w]{0,3}x?[_\W]{0,3}(?:\b|\s)/i
Could be broken down:
(?:\b|\s) - preamble, detecting space or word boundary.
[_\W]{0,3} - gap
(?:\\\/|V) - V
[_\W]{0,3} - gap
[ij1!|l\xEC\xED\xEE\xEF] - I
[_\W]{0,3} - gap
[a40\xe0-\...@] - A
[_\W]{0,3} - gap
[xyz]?[gj] - G (with optional extra garbage before it)
[_\W]{0,3} - gap
r- just R :-)
[_\W]{0,3} - gap
[a40\xe0-\...@] -A
[_\W]{0,3} - gap
x? - optional garbage
[_\W]{0,3} - gap
(?:\b|\s)- suffix, detecting space or word boundary.
Which detects weird spacings and substitutions in the word Viagra.
> But how are the rules generated themselves?
>
Mostly meatware, except the sought rules others have mentioned.
> Thnx
>
Re: SpamAssassin Ruleset Generation
On Tue, 6 Oct 2009, poifgh wrote: Other than the sought rules, all the rules are manually generated? Is there any statistics on how frequently are new rules/regex adopted by spamassasssin? Who are the people who write them? Any details related to it? Most of the rules are manually written by contributors such as myself. Some meta rules are generated by various means from existing rules - for example, the ADVANCE_FEE rules are generated using genetic algorithms to find effective combinations of simpler subrules that were manually generated. New rules are added whenever a contributor works on them, and this is generally based on when they have time to do so, when they have new ideas, and when new forms of spam appear. Indirect contributors will post rules to the users list and a contributor may add them to the rules sandbox for testing and eventual inclusion in the base ruleset. The CREDITS file in the sources should list all of the contributors. Some contributors may not have added their names to that file, though. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- 5 days since a sunspot last seen - EPA blames CO2 emissions
Re: SpamAssassin Ruleset Generation
Hi, > Other than the sought rules, all the rules are manually generated? Is there > any statistics on how frequently are new rules/regex adopted by > spamassasssin? Who are the people who write them? Any details related to Information on Justin Mason's SOUGHT rules is here: http://taint.org/2007/08/15/004348a.html Use sa-update to update your SA rules once or twice per day with the new stuff. His ongoing development work is here: http://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/jm/?sortby=date HTH, Alex
Re: SpamAssassin Ruleset Generation
poifgh wrote: > > > > Bowie Bailey wrote: >> >> >> >> http://www.google.com/search?q=spamassassin+sought >> > :-D - Thnx > > Other than the sought rules, all the rules are manually generated? Is there any statistics on how frequently are new rules/regex adopted by spamassasssin? Who are the people who write them? Any details related to it? thnx -- View this message in context: http://www.nabble.com/SpamAssassin-Ruleset-Generation-tp25773508p25776307.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: SpamAssassin Ruleset Generation
Bowie Bailey wrote: > > > > http://www.google.com/search?q=spamassassin+sought > :-D - Thnx -- View this message in context: http://www.nabble.com/SpamAssassin-Ruleset-Generation-tp25773508p25776303.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: SpamAssassin Ruleset Generation
poifgh wrote: > > RW-15 wrote: > >> On Tue, 6 Oct 2009 11:08:28 -0700 (PDT) >> poifgh wrote: >> >> >>> I have a question about - understanding how are rulesets generated for >>> ... >>> a. Is it done manually with people writing regex to see how >>> efficiently they capture spams? >>> b. Is there an algorithm that identifies large corpus of spam and the >>> comes up with these regex'es on its own? >>> c. Is it a combination of (a), (b)? >>> >> The optional sought rules are autogenerated, the rest are manual. >> > > Thnx - What are optional sought rules? > http://www.google.com/search?q=spamassassin+sought -- Bowie
Re: SpamAssassin Ruleset Generation
RW-15 wrote: > > On Tue, 6 Oct 2009 11:08:28 -0700 (PDT) > poifgh wrote: > >> >> I have a question about - understanding how are rulesets generated for >> ... >> a. Is it done manually with people writing regex to see how >> efficiently they capture spams? >> b. Is there an algorithm that identifies large corpus of spam and the >> comes up with these regex'es on its own? >> c. Is it a combination of (a), (b)? > > The optional sought rules are autogenerated, the rest are manual. > > Thnx - What are optional sought rules? -- View this message in context: http://www.nabble.com/SpamAssassin-Ruleset-Generation-tp25773508p25776105.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: SpamAssassin Ruleset Generation
On Tue, 6 Oct 2009 11:08:28 -0700 (PDT) poifgh wrote: > > I have a question about - understanding how are rulesets generated for > ... > a. Is it done manually with people writing regex to see how > efficiently they capture spams? > b. Is there an algorithm that identifies large corpus of spam and the > comes up with these regex'es on its own? > c. Is it a combination of (a), (b)? The optional sought rules are autogenerated, the rest are manual.
SpamAssassin Ruleset Generation
I have a question about - understanding how are rulesets generated for
spamassassin.
For example - consider the rule in 20_drugs.cf :
header SUBJECT_DRUG_GAP_C Subject =~
/\bc.{0,2}i.{0,2}a.{0,2}l.{0,2}i.{0,2}s\b/i
describe SUBJECT_DRUG_GAP_C Subject contains a gappy version of 'cialis'
Who generated the regular expression
"/\bc.{0,2}i.{0,2}a.{0,2}l.{0,2}i.{0,2}s\b/i"
a. Is it done manually with people writing regex to see how efficiently they
capture spams?
b. Is there an algorithm that identifies large corpus of spam and the comes
up with these regex'es on its own?
c. Is it a combination of (a), (b)?
I know scores for rules are generated using "a neural network trained with
error back propagation"
http://wiki.apache.org/spamassassin/HowScoresAreAssigned
But how are the rules generated themselves?
Thnx
--
View this message in context:
http://www.nabble.com/SpamAssassin-Ruleset-Generation-tp25773508p25773508.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Optional Tests in Main Ruleset?
On Thu, 2009-06-11 at 09:18 -0400, Charles Gregory wrote:
> Hallo!
>
> I've noticed a few rules now that seem to score *very* low.
> For example: DYN_RDNS_AND_INLINE_IMAGE=0.001
There are a lot of possible reasons for that, including informative only
rules (which are likely to have a description to that extent). Poor S/O
rating during the last GA run, but an otherwise potentially useful rule
would be another possible reason. Among others.
> Are these rules 'in development' and therefore not being assigned a
> significant score as of yet? Or, more interestingly, do they represent an
> 'optional' set of rules that can be 'activated' by raising their score,
> but which will otherwise *always* have a low score by default?
Rules in stock are not "in development". Optional rules are more likely
disabled by default, IMHO -- though on the other hand, every single rule
pretty much is optional, and you're free to tweak it.
Just checked current ruleqa results -- looks quite decent indeed.
Doesn't hit a lot of spam, but pretty accurate. Feel free to raise the
score if it would help with your spam, just keep in mind its potential
to cause FPs for you when you do.
> If the latter, then by chance is there a compact listing of these rules
> that I could review and 'turn on' as needed? Or do I just have to watch
> for them in the headers?
Sorry, you have to watch them. ;) There's no such list, and as I
mentioned before, there is a wide range of possible reasons.
Spam (and to a lesser extent, ham) does change, though, and a lot of the
current scores could be adjusted. Reports about FPs are the part usually
visible here. What's just not being reported are rules that are now much
more accurate and candidates for higher scores than they used to be.
guenther
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Optional Tests in Main Ruleset?
Hallo! I've noticed a few rules now that seem to score *very* low. For example: DYN_RDNS_AND_INLINE_IMAGE=0.001 Are these rules 'in development' and therefore not being assigned a significant score as of yet? Or, more interestingly, do they represent an 'optional' set of rules that can be 'activated' by raising their score, but which will otherwise *always* have a low score by default? If the latter, then by chance is there a compact listing of these rules that I could review and 'turn on' as needed? Or do I just have to watch for them in the headers? - Charles
Re: ruleset
JC Putter wrote: > where can i find more rulesets? using openprotect sare rules and > sought rulesets > That's about all there are... A few folks have odds and ends rules posted on their webpages/blogs/etc, but they're of mixed quality. Is there a particular reason your looking for more rulesets?
Re: ruleset
On 24.03.09 15:59, JC Putter wrote: > where can i find more rulesets? using openprotect sare rules and sought > rulesets build your own rulesets? SARE rulesets aren't updated anymore afaik (and thus number of false-positives is increasing). Do you have any problem that can't be solved by fine-tuning SA? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin, 1759
ruleset
where can i find more rulesets? using openprotect sare rules and sought rulesets __ Information from ESET NOD32 Antivirus, version of virus signature database 3957 (20090324) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: how to make a custom ruleset
Just fyi, this particular topic keeps getting raised here. It'd be
great if people would search the list archives. :)
One of the last times around:
http://www.nabble.com/forum/ViewPost.jtp?post=21296293&framed=y
In short, if you want to do this, write a plugin. REs are great until
you get complicated, like doing multiple headers and comparing
captured text.
SA used to have a rule that looked for from=to, but it didn't do well
and got removed. Some pointers in the above thread.
On Fri, Mar 6, 2009 at 2:44 PM, Mark Martinec wrote:
> Adi,
>
>> First, it read the sender, and put it into a variable
>> Then, it check, if the recipient is the same as that variable
>> if true, then give score 3.0
>
> The trick is to let a regexp see an entire mail header section.
> Unfortunately it means we can't reuse already parsed addresses
> in From and To header fields, but need to reparse it all in a regexp.
>
> The rules below comes close, but is not exact (the TOFROM rule
> only checks the first To). Mind the line wraps, there are three
> long lines, each starting by 'header':
>
>
> header SAME_FROMTO1 ALL =~ m{^From: (?: . | \n[\ \t] )* <\s*(.+)\s*> (?s:.*)
> ^To: (?: (?: "[^"]*" | \([^)]*\) |
> [\ \t]+ | \n[\ \t]+ )*? \1 [,(\ \t\n] | (?: . | \n[\ \t])* <\s*\1\s*>)}mix
> header SAME_FROMTO2 ALL =~ m{^From: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\
> \t]+ )* ([^<>,;\s...@[0-9a-z._-]+\.
> [a-z]{2,})\b (?s:.*) ^To: (?: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\ \t]+
> )*? \1 [,(\ \t\n] | (?: . | \n[\
> \t])* <\s*\1\s*>)}mix
> header SAME_TOFROM ALL =~ m{^To: (?: . | \n[\ \t] )* (?:\b|<)
> ([^<>,;\s...@[0-9a-z._-]+\.[a-z]{2,}) \b (?!\.)
> (?s:.*) ^From: (?: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\ \t]+ )*? \1 [,(\
> \t\n] | (?: . | \n[\ \t])*
> <\s*\1\s*>)}mix
> meta SAME_FROMTO SAME_FROMTO1 || SAME_FROMTO2 || SAME_TOFROM
> score SAME_FROMTO1 0.1
> score SAME_FROMTO2 0.1
> score SAME_TOFROM 0.1
> score SAME_FROMTO 1.5
>
>
> Mark
>
Re: how to make a custom ruleset
Adi,
> First, it read the sender, and put it into a variable
> Then, it check, if the recipient is the same as that variable
> if true, then give score 3.0
The trick is to let a regexp see an entire mail header section.
Unfortunately it means we can't reuse already parsed addresses
in From and To header fields, but need to reparse it all in a regexp.
The rules below comes close, but is not exact (the TOFROM rule
only checks the first To). Mind the line wraps, there are three
long lines, each starting by 'header':
header SAME_FROMTO1 ALL =~ m{^From: (?: . | \n[\ \t] )* <\s*(.+)\s*> (?s:.*)
^To: (?: (?: "[^"]*" | \([^)]*\) |
[\ \t]+ | \n[\ \t]+ )*? \1 [,(\ \t\n] | (?: . | \n[\ \t])* <\s*\1\s*>)}mix
header SAME_FROMTO2 ALL =~ m{^From: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\
\t]+ )* ([^<>,;\s...@[0-9a-z._-]+\.
[a-z]{2,})\b (?s:.*) ^To: (?: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\ \t]+ )*?
\1 [,(\ \t\n] | (?: . | \n[\
\t])* <\s*\1\s*>)}mix
header SAME_TOFROM ALL =~ m{^To: (?: . | \n[\ \t] )* (?:\b|<)
([^<>,;\s...@[0-9a-z._-]+\.[a-z]{2,}) \b (?!\.)
(?s:.*) ^From: (?: (?: "[^"]*" | \([^)]*\) | [\ \t]+ | \n[\ \t]+ )*? \1 [,(\
\t\n] | (?: . | \n[\ \t])*
<\s*\1\s*>)}mix
meta SAME_FROMTO SAME_FROMTO1 || SAME_FROMTO2 || SAME_TOFROM
score SAME_FROMTO1 0.1
score SAME_FROMTO2 0.1
score SAME_TOFROM 0.1
score SAME_FROMTO 1.5
Mark
Re: how to make a custom ruleset
On Fri, 6 Mar 2009, Adi Nugroho wrote: It is working well, but not global (just check for my.address, and not for everyone). Actually, it _is_ global, as it can only match on the domain name. Any mail from any user in your domain to any other user in your domain will hit this rule. Please help me again to teach me, how to make it a little bit different: First, it read the sender, and put it into a variable Then, it check, if the recipient is the same as that variable Unfortunately this cannot be done in SpamAssassin, and adding the ability to do that (have one rule depend on the actual text matched by another rule rather than just whether or not that rule matched _something_) is difficult and will likely not be done anytime soon. This particular test - whether from == to - is something that is best done by something outside SA. (1) configure your MTA to reject it at SMTP time (2) configure your MTA to reject mail claiming to be from your domain when it's coming in from outside your network (3) do something like write a procmail rule to check for that match and add a customer header if it is found, then have SA add points if that header is present (which, of course, depends on your using procmail to glue SA into your mail system) -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Failure to plan ahead on someone else's part does not constitute an emergency on my part. -- David W. Barts in a.s.r --- 2 days until Daylight Saving Time begins in U.S. - Spring Forward
RE: how to make a custom ruleset
Adi Nugroho wrote: > On Thursday 05 March 2009 23:44:39 Benny Pedersen wrote: > > header SELF_FROM From =~ /\...@my.address/i > > header SELF_TO To =~ /\...@my.address/i > > meta SELF (SELF_FROM && SELF_TO) > > describe SELF Trap mail with forged sender the same as recipient > > score SELF 3.0 > > Finally I understand above rule. > > First it check if the sender is my.address > then it check if the recipient is my.address > and finally, if both are true, then it give the score 3.0 Not quite. SELF_FROM and SELF_TO default to a score of 1. So it actually works like this: If the sender is my.address, score +1 If the recipient is my.address, score +1 If both are true, score +3 Final score = 5 If you don't want the sub-rules to score, you need to prefix the names with a double-underscore (this will also prevent them from showing in the rule summaries). header __SELF_FROM From =~ /\...@my.address/i header __SELF_TO To =~ /\...@my.address/i meta SELF (__SELF_FROM && __SELF_TO) describe SELF Trap mail with forged sender the same as recipient score SELF 3.0 -- Bowie
Re: how to make a custom ruleset
On Thursday 05 March 2009 23:44:39 Benny Pedersen wrote: > header SELF_FROM From =~ /\...@my.address/i > header SELF_TO To =~ /\...@my.address/i > meta SELF (SELF_FROM && SELF_TO) > describe SELF Trap mail with forged sender the same as recipient > score SELF 3.0 Finally I understand above rule. First it check if the sender is my.address then it check if the recipient is my.address and finally, if both are true, then it give the score 3.0 Thank you. It is working well, but not global (just check for my.address, and not for everyone). Please help me again to teach me, how to make it a little bit different: First, it read the sender, and put it into a variable Then, it check, if the recipient is the same as that variable if true, then give score 3.0 Thank you again :)
Re: how to make a custom ruleset
Adi Nugroho wrote on Fri, 6 Mar 2009 10:40:26 +0800: > Is there a howto about this ruleset? http://wiki.apache.org/spamassassin/WritingRules Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: how to make a custom ruleset
On Thursday 05 March 2009 23:44:39 Benny Pedersen wrote: > ups > > header SELF_FROM From =~ /\...@my.address/i > header SELF_TO To =~ /\...@my.address/i > meta SELF (SELF_FROM && SELF_TO) > describe SELF Trap mail with forged sender the same as recipient > score SELF 3.0 I have tried above syntax but failed. No mail identified as SELF. Is there a howto about this ruleset?
Re: how to make a custom ruleset
On Mar 5, 2009, at 7:28, Martin Gregorie wrote: On Thu, 2009-03-05 at 21:31 +0800, Adi Nugroho wrote: I found that a lot of spam is using recipient email address as the sender. (from a...@internux.co.id to a...@internux.co.id, or from i...@apache.org to i...@apache.org). The only disadvantage is that you'll label test messages as spam. If you allow address delimiters this is trivial to get around, just have the email their test to user+t...@example.com
Re: how to make a custom ruleset
On Thu, March 5, 2009 17:31, John Hardin wrote: >> header SELF_FROM From =~ /\...@my.address/i >> header SELF_TO To =~ /\...@my.address/i > > Are you sure you want to give 1 point to each of those cases in > addition to whatever points the meta adds? it was not me that maked the rules, just edit them :) > If not, then they should be named __SELF_FROM and __SELF_TO sure when do you stop CC me ? -- http://localhost/ 100% uptime and 100% mirrored :)
