SA Blacklist for Display Names?
Hi All, we get a lot of Spam with some bad Words in the Display name of the sender. With the blacklist_from command it is possible to filter by email-address, but what is the right notation for filtering the display name of the sender? can you please helb me? Thanks a lot, bye Daniel -- View this message in context: http://old.nabble.com/SA-Blacklist-for-Display-Names--tp27420107p27420107.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: SA Blacklist for Display Names?
On Tue, 2010-02-02 at 05:13 -0800, Daniel R. wrote: Hi All, we get a lot of Spam with some bad Words in the Display name of the sender. With the blacklist_from command it is possible to filter by email-address, but what is the right notation for filtering the display name of the sender? can you please helb me? At present you can only match strings in the From: header, so add a local rule in /etc/mail/spamassassin/local.cf something like this: describe XX_BADWORDS Bad words in the sender name header XX_BADWORDS From =~ /bad words pattern/i scoreXX_BADWORDS 5.0 See http://wiki.apache.org/spamassassin/WritingRules for a fuller explanation. Martin
Re: Problem with sa-blacklist
Michael Monnerie wrote: I can't reach Bill Stearns, so I try at this list: Dear Bill, I'm using the sa-blacklist.reject for postfix since a long time, but these last days your rsync doesn't work anymore: rsync: failed to connect to rsync.sa-blacklist.stearns.org: Connection timed out (110) So I had a look if something changed on http://www.sa-blacklist.stearns.org/sa-blacklist/ but obviously the information there is quite old: If I download the sa- blacklist.current.reject, it has a version of April: 200904171539 while my last rsync version is 200910142031 Any chance for a fix? mfg zmi SA-blacklist and sa-blacklist-uri are both dead as far as use within SpamAssassin goes. Although someone updated it in 2009, for all practical purposes it's use as a SA ruleset has been dead (or at least dying) since 2004. (when the WS sub-list of surbl.org was created) While it was an interesting case study, but it is *VERY* inefficient, and will kill most servers. Any use of it should be restricted to research purposes only (i.e.: reading the list manually to study patterns in emerging spam domains). It is too heavyweight to use under SpamAssassin. The plain sa-blacklist was not very effective, and consumed lots of memory (750MB per spamd instance?). This list worked on the From: address of the message, which spammers recycle very quickly. This means lots of addresses, a huge list, and very low hitrate due to low re-use. Plain and simple waste of memory to use it under SA. (although manually looking at the list does have some uses... as noted above..) The URI version has become the WS list over on surbl. This version had better hitrates, but the very large list consumed large amounts of memory too. Also, searching this huge list as a large number regular expressions is so computationally intensive that most systems can complete a DNS lookup against surbl.org before the regexes finish running. It is not unheard of for this ruleset to add 10 or more seconds to message processing, in addition to the over 1 gig of ram it consumes. Sure a more recent server with more CPU beef and fast ram could probably complete it in 3 seconds or so, but that is still slower than a DNS lookup. Most admins are not willing to devote several gigs of ram just for their SpamAssassin instances. I doubt you are either, so please don't use sa-blacklist. Unless you're looking to use it as a data set for analysis purposes, it is dead, and has been for a long time. The valuable parts have evolved into parts of SURBL, which is already in SpamAssassin, unless you're dealing with a version that is over 4 years old.
Re: Problem with sa-blacklist
On Samstag, 21. November 2009 Matt Kettler wrote: SA-blacklist and sa-blacklist-uri are both dead as far as use within SpamAssassin goes. Thank you for your answer, Matt. I have to apologize, I forgot to mention that I do not use that list in SA, for the reasons you listed. Instead, I used the postfix blacklist version of it, for a simple blacklist. Is there any replacement for it? mfg zmi -- // Michael Monnerie, Ing.BSc- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: curl -s http://zmi.at/zmi.asc | gpg --import // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 signature.asc Description: This is a digitally signed message part.
Problem with sa-blacklist
I can't reach Bill Stearns, so I try at this list: Dear Bill, I'm using the sa-blacklist.reject for postfix since a long time, but these last days your rsync doesn't work anymore: rsync: failed to connect to rsync.sa-blacklist.stearns.org: Connection timed out (110) So I had a look if something changed on http://www.sa-blacklist.stearns.org/sa-blacklist/ but obviously the information there is quite old: If I download the sa- blacklist.current.reject, it has a version of April: 200904171539 while my last rsync version is 200910142031 Any chance for a fix? mfg zmi -- // Michael Monnerie, Ing.BSc- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: curl -s http://zmi.at/zmi.asc | gpg --import // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4
Re: Problem with sa-blacklist
On 20.11.09 12:47, Michael Monnerie wrote: I can't reach Bill Stearns, so I try at this list: Dear Bill, I'm using the sa-blacklist.reject for postfix since a long time, but these last days your rsync doesn't work anymore: rsync: failed to connect to rsync.sa-blacklist.stearns.org: Connection timed out (110) do you use it at postfix level as regex filter? So I had a look if something changed on http://www.sa-blacklist.stearns.org/sa-blacklist/ but obviously the information there is quite old: If I download the sa- blacklist.current.reject, it has a version of April: 200904171539 while my last rsync version is 200910142031 Any chance for a fix? what about using URIBL/SURBL (or full SA with them) instead? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Saving Private Ryan... Private Ryan exists. Overwrite? (Y/N)
sa-blacklist
Having process load issues, I found that removing my two sa-blacklist rules took care of it. If fact, very good processing times now that they're gone. My question is, what I'm I missing? Spam filtering is doing a fine job since changes applied 24 hours ago. I run Postfix 2.2.8 with amavisd-new 2.3.3 that hands off to SA. The server is FreeBSD 5.4 with dual P4 processors with hyperthreading enabled and a gig of RAM using RAID5. My postfix/amavisd setup is using 4 processes at a time. I tried bumping this to 10 and my server will begin even hesitating on the shell prompt. This setup with 4 has run for a while very well, but then I added these sa-blacklist rules. Also, is hyperthreading a good thing? -- Robert
Re: sa-blacklist
Robert Fitzpatrick wrote: Having process load issues, I found that removing my two sa-blacklist rules took care of it. If fact, very good processing times now that they're gone. My question is, what I'm I missing? Spam filtering is doing a fine job since changes applied 24 hours ago. sa-blacklist is a large file, and consumes a LOT of memory and a reasonably large amount of processing time too. I run Postfix 2.2.8 with amavisd-new 2.3.3 that hands off to SA. The server is FreeBSD 5.4 with dual P4 processors with hyperthreading enabled and a gig of RAM using RAID5. My postfix/amavisd setup is using 4 processes at a time. I tried bumping this to 10 and my server will begin even hesitating on the shell prompt. How much free memory did you have when you were at 4 processes? How much when you had 10? I suspect when you were running with 10 you were going deep into your swapfile. Always double-check your free memory using free or top before increasing This setup with 4 has run for a while very well, but then I added these sa-blacklist rules. So don't add them... IMHO they're not nearly as useful as they are wasteful. If you've got tons of resources (3gig of ram 9ghz total cpu) it's probably worth it, but if not you need to test and see how well it works. Also, is hyperthreading a good thing? Sometimes, but not always. In some cases it can improve performance, but the Intel benchmarks showing 60% performance improvement are heavily tuned to perform best under HT. They're also avoiding the situations where HT actually impairs performance. The typical system sees somewhere between +20 and -15% performance improvement (note that -15% improvement is a detriment) depending on workload. Servers with large numbers of simultaneous processes all working at once tend to see the better end of that. Workstations grinding the heck out of the CPU with a single thread tend to be at the worst end of that. Take a look at: http://www.2cpu.com/articles/41_1.html For some examples of workloads that benefit (apache and mysql) and suffer (bladenc, kernel compiles, java VolanoMark) from the use of HT. With SA, you'll probably see benefit whenever there's more than (# of physical cpus) instances of SA actually processing mail at the same time. You'll probably see lower performance in the single-message-at-a time case. However, given that the single-message case reflects low-load times for your server, that should be OK, provided the high load end gets a good boost.
sa-blacklist from rulesdujour
Has this moved? Looks like a move error, but my config was update and
still and seems to download the recipes...getting a 302 'Found' message
from the web server and link works, but target says moved?
-- RANDOMVAL --
RULESET_NAME=RANDOMVAL
INDEX=11
CF_URL=http://www.sa-blacklist.stearns.org/sa-blacklist/random.current.cf
CF_FILE=random.current.cf
CF_NAME=William Stearn's RANDOM WORD Ruleset
PARSE_NEW_VER_SCRIPT=grep -i '^#release' | tail -1
CF_MUNGE_SCRIPT=
Old random.current.cf already existed
in /usr/local/etc/mail/spamassassin/RulesDuJour...
Retrieving file from
http://www.sa-blacklist.stearns.org/sa-blacklist/random.current.cf...
exec: curl -w %{http_code} --compressed -O -R -s -S
-z /usr/local/etc/mail/spamassassin/RulesDuJour/random.current.cf
http://www.sa-blacklist.stearns.org/sa-blacklist/random.current.cf 21
curl_output: 304
random.current.cf was up to date [skipped downloading of
http://www.sa-blacklist.stearns.org/sa-blacklist/random.current.cf ] ...
Installing new ruleset
from /usr/local/etc/mail/spamassassin/RulesDuJour/random.current.cf.2
Installing new version...
William Stearn's RANDOM WORD Ruleset has changed on esmtp.webtent.net.
Version line:
BUT...
Lint output: [5694] warn: config: failed to parse line, skipping: !
DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN
[5694] warn: config: failed to parse line, skipping: HTMLHEAD
[5694] warn: config: failed to parse line, skipping: TITLE302
Found/TITLE
[5694] warn: config: failed to parse line, skipping: /HEADBODY
[5694] warn: config: failed to parse line, skipping: H1Found/H1
[5694] warn: config: failed to parse line, skipping: The document has
moved A
HREF=http://www.sa-blacklist.stearns.org/sa-blacklist/random.current.cf;here/A.P
[5694] warn: config: failed to parse line, skipping: /BODY/HTML
[5694] warn: lint: 7 issues detected, please rerun with debug enabled
for more information
--
Robert
Re: [PLEASE NOTE] Change in location for sa-blacklist downloads
Good evening, all, This is a (shortened) repost of a sincere request to anyone using any of the sa-blacklist files. The URLs to those files have changed; please update your URLs in any automated download scripts. if you're using RulesDuJour, please get the latest version as Chris has already updated the URLs in that. On Sat, 17 Sep 2005, William Stearns wrote: (Summary - the sa-blacklist content is moving to new machines. If you're downloading any of the 15 versions of this list, you'll need to change the hostname you use in your download; see What you need to do below for instructions.) [snip history] What you need to do I've already set up new hostnames (*) from which the sa-blacklist files can be pulled. If you're getting any sa-blacklist files over http, please change the hostname you use to www.sa-blacklist.stearns.org. If you are using rsync to pull content, please use rsync.sa-blacklist.stearns.org. If you're using ftp, please use ftp.sa-blacklist.stearns.org. In other words, the exact same content should be viewable at http://www.sa-blacklist.stearns.org/sa-blacklist/ ftp://ftp.sa-blacklist.stearns.org/pub/wstearns/sa-blacklist/ rsync://rsync.sa-blacklist.stearns.org/wstearns/sa-blacklist/ (although this last one is commonly used by the rsync application and won't work in a web browser.) There's a real benefit to you in taking the time to make this switchover. My server was getting pegged for multiple minutes at the top of the hour, so you'll find your downloads are much faster. Because of the way the files are distributed, the content on the mirrors should always be as current as the ones on the main server. At some point in the near future, I'll be limiting access to or completely shutting down the old URLs, so it's to your advantage to switch over sooner rather than later. *smile* I'd sincerely appreciate it if you could check any automated download scripts or cron jobs and point them to these new hostnames. Sorry for the inconvenience, but because these URL's are only used for this content, you won't need to make this change again. As one last suggestion, you might want to consider using the ws.surbl.org dns lookup service which performs the same checks as sa-blacklist.current.uri.cf , but _much_ faster and with a _lot_ less memory. More information about this dns-based service is available at http://www.surbl.org/ . * These aliases will transparently pick a random server out of the available machines, spreading out the load. As more mirrors come online you'll be sent to them automatically. Cheers, - Bill --- It's a drop in the bucket, but if you get enough drops you get a bucket. -- Morgan Freeman, on the gulf coast disaster relief effort. -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --
Re: [PLEASE NOTE] Change in location for sa-blacklist downloads
William Stearns wrote: Good day, all, (Summary - the sa-blacklist content is moving to new machines. If you're downloading any of the 15 versions of this list, you'll need to change the hostname you use in your download; see What you need to do below for instructions.) Rules du Jour has been updated to point to the new domain. Chris Thielen signature.asc Description: OpenPGP digital signature
Re: [PLEASE NOTE] Change in location for sa-blacklist downloads
Good afternoon, Chris, On Mon, 19 Sep 2005, Chris Thielen wrote: William Stearns wrote: (Summary - the sa-blacklist content is moving to new machines. If you're downloading any of the 15 versions of this list, you'll need to change the hostname you use in your download; see What you need to do below for instructions.) Rules du Jour has been updated to point to the new domain. Thanks so much, Chris! Cheers, - Bill --- return(ECRAY); /* Program exited before being run */ -- Martin Mares [EMAIL PROTECTED] -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --
[PLEASE NOTE] Change in location for sa-blacklist downloads
Good day, all, (Summary - the sa-blacklist content is moving to new machines. If you're downloading any of the 15 versions of this list, you'll need to change the hostname you use in your download; see What you need to do below for instructions.) I had a chat with my ISP last week. They've known for a long time that the bandwidth spike at the top of very hour was my web server, but since they knew the sa-blacklist was hosted there and it was a public service project, they told me not to worry. Fast forward to last week. *smile* When I asked this new contact what amount of bandwidth my hosting contract would normally allow and how much bandwidth I'd actually been using over the last few months, he told me that I should be around 10G/month, but I've been using 1000G/month. Woah. Luckily, he wasn't asking me to pay 100X my current contract. *smile* They really have been great about it (I mean that sincerely), but both they and I know that's an unreasonable drain on their bandwidth and unfair to the other customers. To fix that, I'm transitioning the content to new machines with more available bandwidth. I owe a heartfelt thanks to Raymond, David, Panagiotis, Rob, Wim, Jeff, and Chris for offering to host the content at no cost on much faster lines than mine and offering suggestions on how to make the process more efficient. Their generousity makes it possible for me to continue providing this content. What you need to do I've already set up new hostnames (*) from which the sa-blacklist files can be pulled. If you're getting any sa-blacklist files over http, please change the hostname you use to www.sa-blacklist.stearns.org. If you are using rsync to pull content, please use rsync.sa-blacklist.stearns.org. If you're using ftp, please use ftp.sa-blacklist.stearns.org. In other words, the exact same content should be viewable at http://www.sa-blacklist.stearns.org/sa-blacklist/ ftp://ftp.sa-blacklist.stearns.org/pub/wstearns/sa-blacklist/ rsync://rsync.sa-blacklist.stearns.org/wstearns/sa-blacklist/ (although this last one is commonly used by the rsync application and won't work in a web browser.) There's a real benefit to you in taking the time to make this switchover. My server was getting pegged for multiple minutes at the top of the hour, so you'll find your downloads are much faster. Because of the way the files are distributed, the content on the mirrors should always be as current as the ones on the main server. At some point in the near future, I'll be limiting access to or completely shutting down the old URLs, so it's to your advantage to switch over sooner rather than later. *smile* I'd sincerely appreciate it if you could check any automated download scripts or cron jobs and point them to these new hostnames. Sorry for the inconvenience, but because these URL's are only used for this content, you won't need to make this change again. As one last suggestion, you might want to consider using the ws.surbl.org dns lookup service which performs the same checks as sa-blacklist.current.uri.cf , but _much_ faster and with a _lot_ less memory. More information about this dns-based service is available at http://www.surbl.org/ . Cheers, - Bill * These aliases will transparently pick a random server out of the available machines, spreading out the load. As more mirrors come online you'll be sent to them automatically. --- (Referring to the 32 bit system that feeds out files for kernel.org) We learned that the Linux load average rolls over at 1024. And we actually found this out empirically. -- Peter Anvin -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --
Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
I probably should have introduced this second SURBL list
that can be used together with or in place of sc.surbl.org
before mentioning that its name was changing from sa.surbl.org
to ws.surbl.org. :-) Note that the two lists have different
data sources, so strictly speaking one is not a replacement for
the other. They're two different lists. sc uses URI domains
from SpamCop reports. The data source for ws data is described
below. Both lists have merits and we'd encourage you to consider
trying both.
Here's an announcement with the additional update that
we've changed the *sample rule names* for the ws list to use
WS instead of SA:
__
http://www.surbl.org/ (with some live links)
More SURBL lists
In addition to the first SpamCop URI-derived SURBL sc.surbl.org, we
are pleased to host another RBL compatible with the SpamCopURI or
URIDNSBL SpamAssassin plugins, or any other software that can
check message body domains against a name-based RBL. Data for the
second SURBL ws.surbl.org comes from the domains in Bill Stearns'
SpamAssassin blacklist: sa-blacklist. This is a large list of
spam domains, including those found in spam message body URIs.
Both ws.surbl.org and sc.surbl.org SURBLs can be used in the same
SA installation by using two sets of rules.
An SA 2.63 rule and score using SpamCopURI (but not the SpamCop
data!) looks like this:
uri WS_URI_RBL eval:check_spamcop_uri_rbl('ws.surbl.org','127.0.0.2')
describe WS_URI_RBL URI's domain appears in spamcop database at ws.surbl.org
tflagsWS_URI_RBL net
score WS_URI_RBL 3.0
An SA 3.0 rule and score using URIBL's urirhsbl looks like this:
urirhsblURIBL_WS_SURBL ws.surbl.org. A
header URIBL_WS_SURBL eval:check_uridnsbl('URIBL_WS_SURBL')
describeURIBL_WS_SURBL Contains a URL listed in the WS SURBL blocklist
tflags URIBL_WS_SURBL net
score URIBL_WS_SURBL 3.0
More details about ws.surbl.org are available in the section
Additional SURBLs for spam URI testing (copied below).
Please note that the name of this list is being changed from
sa.surbl.org to ws.surbl.org. If you were using the old name in
your rules please update them to the new name.
...
Additional SURBLs for spam URI testing
Additional SURBLs that list domains occurring in spam message
bodies may be used with the same routines that use the
sc.surbl.org RBL.
sa-blacklist available as RBL: ws.surbl.org
In cooperation with Bill Stearns, SURBL is making his
sa-blacklist SpamAssassin blacklist available as the RBL
ws.surbl.org. It can be used in the same way as sc.surbl.org, for
example by adding urirhsbl and SpamCopURI rules as described in
the Quick Start section at the top of this document. Like sc,
ws.surbl.org is available through DNS and, for large-volume mail
servers, as rsynced BIND and rbldns zone files. Raymond
Dijkxhoorn has graciously agreed to host the ws.surbl.org zone
files from his rsync server along with sc.surbl.org's. Please
contact him at [EMAIL PROTECTED] for rsync access.
Both sc and ws RBLs can be used in the same installation. The
choice of using either or both or none is yours. Their data
differs somewhat, and we'll try to briefly describe and link some
of the differences here. Bill's list is rather large at about
9600 domains. It consists of domains found in spam message body
URIs and some spam sender and spam operator domains. Given that
the former are more relevant to isolate these days, most of the
recent additions to Bill's list have been URI domains. Those are
also the domains most relevant for use with the message body
checking approach which we propose throughout this site.
The data in sa-blacklist and therefore ws.surbl.org differ from
the SpamCop URI report data described above in that the list is
about ten times larger, more stable, and may have a slightly
higher false positive rate. Bill's policy for inclusion and
cleaning of the sa-blacklist is quite sound, however, so folks
should feel comfortable giving this list a try in addition to the
sc list. ws may currently detect some spam that sc misses, and
vice versa, but it's worth mentioning that the current sc is a
working prototype and that we expect the performance of sc to
improve as we tune the sc data engine further. sc just got out of
the gate, yet it already has some worthy competition in ws.
Thanks Bill!
Because ws is larger and more stable, the zone files for it gets
a six hour TTL compared to 10 minutes for sc. Due to the
differences between the time scales, sizes, and data sources of
ws and sc, we probably won't be offering a combined ws plus sc
list. For example it would be difficult to say what TTL a merged
list should get, and you probably would not want a megabyte plus
BIND zone file refreshing every 10 minutes. For those using
rsynced zone files that would probably not be an issue, but for
those using BIND, the DNS traffic quite well could be.
We encourage you to give ws.surbl.org a try.
Please note
Was there a BAD SA-BLACKLIST rule file published on oct 10 th or 11th
Hi! I'm using spamassassin 2.63-1, amavisd-new 20030616-p9 on RedHat 9.0 and it worked correctly But, today I received complaints from my users about unreceived mails on last monday Oct-11th. While looking at my postfix logfile, I noticed that the rule USER_IN_BLACKLIST has been triggered 2 times on THAT day! The daily average is normally 400. I'm using the rule /etc/mail/spamassassin/65_sa-blacklist.cf with a daily (04hAM) update from URL=http://www.stearns.org/sa-blacklist/sa-blacklist.current Of course, the entries of this file is NOW correct I'm assuming that, on October 10th-11th, there was a publication of some bad file Am I the only one who got that problem ? Is there some archive of that file ? Thanks, Eddy
