RE: DKIM Checks
From: Matt [mailto:lm7...@gmail.com] Sent: Wednesday, May 18, 2011 11:32 AM To: users Subject: DKIM Checks I am running spamassassin-3.2.5-1.el5 on 64 bit CentOS. sa-update -D seems to indicate that the DKIM libraries are installed. ... May 18 10:25:02.683 [15134] dbg: diag: [...] module installed: Mail::DKIM, version 0.39 ... Looking at the X-Spam-Report on various messages and I never see that its looked at. I see that SPF is checked and scored. Any idea why its not checking the DKIM signatures? Check the file v312.pre and see if the loadplugin line for DKIM is commented out. If it is, uncomment it.
RE: Fake MX
From: Bob Proulx [mailto:b...@proulx.com] Subject: Re: Fake MX [...] but that is distinct from being a tarpit, which is what I'm trying to clarify. A discussion around the definition of tarpit, and why tarbaby might be a suboptimal, though catchy, name? For the record a tarbaby: http://en.wikipedia.org/wiki/Tar_baby is something different from a tarpit: http://en.wikipedia.org/wiki/Tarpit_%28networking%29 Please, let's use the correct terminology. They really are pretty far from being interchangeable. I wonder if the OP was really referring to a honeypot?
RE: email address forgery
Are there domains that have actually defined SPF record type records? I haven’t been able to find any, but it could be the fault of the tools I’m using. L From: Noel Butler [mailto:noel.but...@ausics.net] Sent: Thursday, November 11, 2010 5:14 PM To: users@spamassassin.apache.org Subject: Re: email address forgery On Thu, 2010-11-11 at 10:07 -0500, Rob McEwen wrote: On 11/11/2010 9:11 AM, Jeremy Van Rooyen wrote: Can anybody explain to me how to do this and how would I be able to test it? Jeremy, I really like to use the following wizard to generate my SPF strings: http://www.openspf.org/ Scroll down to the section that says Deploying SPF, enter the domain name, and click GO. Then, on the next page, fine tune the answers to the various questions before submitting the info to generate your SPF string. Finally, go into your DNS server and, for that domain, add that string as a TXT record. *and* as an SPF record type, the TXT method is deprecated, but for time being it's good to use it since there are a lot, and I mean a LOT of outdated DNS servers around that do not support it even today, yes, the fault of the DNS server admin for running antiquated rubbish, but, there's just no telling some people to get with the times.
RE: DOS_OE_TO_MX
-Original Message- From: njjrdell [mailto:nruggi...@dellmagazines.net] Sent: Wednesday, September 29, 2010 11:32 AM To: users@spamassassin.apache.org Subject: Re: DOS_OE_TO_MX I'm pretty sure she would not send a GTUBE. Here is another from her Sep 28 08:35:26 nsmail spamd[207]: prefork: child states: II\n Sep 28 08:35:55 nsmail spamd[287]: spamd: connection from localhost [127.0.0.1] at port 50098\n Sep 28 08:35:55 nsmail spamd[287]: spamd: checking message 000b01cb5f6e$b1bbfe80$6629a...@traci for (unknown):500\n Sep 28 08:35:55 nsmail spamd[287]: spamd: identified spam (4006.3/5.0) for (unknown):500 in 1.0 seconds, 142218 bytes.\n Sep 28 08:35:55 nsmail spamd[287]: spamd: result: Y 4006 - AWL,BAYES_50,DATE_IN_FUTURE_12_24,DOS_OE_TO_MX scantime=1.0,size=142218,user=(unknown),uid=500,required_score=5.0,rhost=l ocalhost,raddr=127.0.0.1,rport=50098,mid=000b01cb5f6e$b1bbfe80$6629a...@t raci,bayes=0.483846,autolearn=no\n I never seen anything with such a score of 4006. DOS_OE_TO_MX is the rule that is consistent, so I was hoping to find out where it is to make sure nothing is scored wrong score DOS_OE_TO_MX 2.602 3.086 2.265 2.523
RE: DOS_OE_TO_MX
From: njjrdell [mailto:nruggi...@dellmagazines.net] Sent: Wednesday, September 29, 2010 12:05 PM To: users@spamassassin.apache.org Subject: RE: DOS_OE_TO_MX also, won't whitelisting her address open her up for spoofing? AWL has nothing to do with whitelist_from and other similar options. It's more of a score averager. http://wiki.apache.org/spamassassin/AutoWhitelist thanks for the scores. Now would that just go into /usr/local/share/spamassassin/50_scores.cf? and why would that score be missing. It's not missing. It is in /var/lib/spamassassin/3.003001/updates_spamassassin_org/50_scores.cf or some similar directory. To find your config directory path, try this: spamassassin -D config --lint Rosenbaum, Larry M. wrote: -Original Message- From: njjrdell [mailto:nruggi...@dellmagazines.net] Sent: Wednesday, September 29, 2010 11:32 AM To: users@spamassassin.apache.org Subject: Re: DOS_OE_TO_MX I'm pretty sure she would not send a GTUBE. Here is another from her Sep 28 08:35:26 nsmail spamd[207]: prefork: child states: II\n Sep 28 08:35:55 nsmail spamd[287]: spamd: connection from localhost [127.0.0.1] at port 50098\n Sep 28 08:35:55 nsmail spamd[287]: spamd: checking message 000b01cb5f6e$b1bbfe80$6629a...@traci for (unknown):500\n Sep 28 08:35:55 nsmail spamd[287]: spamd: identified spam (4006.3/5.0) for (unknown):500 in 1.0 seconds, 142218 bytes.\n Sep 28 08:35:55 nsmail spamd[287]: spamd: result: Y 4006 - AWL,BAYES_50,DATE_IN_FUTURE_12_24,DOS_OE_TO_MX scantime=1.0,size=142218,user=(unknown),uid=500,required_score=5.0,rhost=l ocalhost,raddr=127.0.0.1,rport=50098,mid=000b01cb5f6e$b1bbfe80$6629a...@t raci,bayes=0.483846,autolearn=no\n I never seen anything with such a score of 4006. DOS_OE_TO_MX is the rule that is consistent, so I was hoping to find out where it is to make sure nothing is scored wrong score DOS_OE_TO_MX 2.602 3.086 2.265 2.523 -- View this message in context: http://old.nabble.com/DOS_OE_TO_MX- tp29839497p29840133.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Moving from Solaris to Red Hat
We are currently running SA v3.3.1 on Solaris 9 and Solaris 10 and are planning to move to Red Hat. I don't have much experience with Red Hat (or Linux in general). Could you point me to some tips and documentation about installing and running SA on Red Hat? FYI, on Solaris I install by downloading the Mail-SpamAssassin-3.3.1.tar.bz2 file and running the build procedure, after installing the required Perl modules. Thanks, Larry
RE: Trouble whitelisting domain users with whitelist_from_rcvd
What is the best way to completely whitelist all internal emails so that there is no danger of any internal emails being blacklisted The best way is to not feed internal emails to SpamAssassin.
compiling: Illegal octal digit '9' ignored...
I'm seeing warnings like this when I run sa-compile:
Illegal octal digit '9' ignored at
/usr/local/lib/perl5/site_perl/5.10.1/Mail/SpamAssassin/Plugin/BodyRuleBaseExtractor.pm
line 1083, $fh line 5645.
Illegal octal digit '9' ignored at
/usr/local/lib/perl5/site_perl/5.10.1/Mail/SpamAssassin/Plugin/BodyRuleBaseExtractor.pm
line 1083, $fh line 6449.
Illegal octal digit '9' ignored at
/usr/local/lib/perl5/site_perl/5.10.1/Mail/SpamAssassin/Plugin/BodyRuleBaseExtractor.pm
line 1083, $fh line 7229.
I've done some digging and it looks like the rule2xs() function is trying to
parse stuff like this:
r \2512009 microsoft | unsubscribe | more newsletters | privac:FB_SOFTTABS
__FB_BCs __SEEK_YRQYH9,[l=1] __SUBSCRIPTION_INFO,[l=1]
r copyright \2512009 by nacha - the electronic payments
associ:LOTTO_AGENT,[l=1] T_LOTTO_DEPT,[l=1] __ATM_CARD,[l=1] __FEES,[l=1]
__LOTTO_WIN_01,[l=1] __SEEK_VZ7OQ6,[l=1] __YOUR_FUND,[l=1]
I think these are coming from lines like this:
sought_rules_yerp_org/20_sought.cf:body __SEEK_YRQYH9 /\x{a9}2009 Microsoft \|
Unsubscribe \| More Newsletters \| Privacy/
sought_rules_yerp_org/20_sought.cf:body __SEEK_VZ7OQ6 /Copyright \x{a9}2009 by
NACHA - The Electronic Payments Association/
So the year occurs right after the octal escape code for the copyright symbol,
and the fixup_re code is assuming the 2009 is part of the octal escape.
I suspect this error prevents the rule from being used. Is there a fix for
this?
System info:
SpamAssassin version 3.3.1
running on Perl version 5.10.1
SunOS email 5.9 Generic_118558-39 sun4u sparc SUNW,Sun-Fire-V210
RE: new PDF Launch malware exploit (with sample)
From: d.h...@yournetplus.com [mailto:d.h...@yournetplus.com] Sent: Wednesday, April 28, 2010 2:29 PM To: users@spamassassin.apache.org Subject: RE: new PDF Launch malware exploit (with sample) Quoting Rosenbaum, Larry M. rosenbau...@ornl.gov: Please don't send live malware samples to the list. Um... The OP did not send malware to the list. A link was supplied to the original message. You must have a scanner set up to follow links. That isn't a good idea, in my opinion. There was some code in the message, right after the Here's just the nifty Launch part paragraph. Perhaps it's not dangerous in a text message, but Forefront didn't like it anyway. -Original Message- From: Chip M. [mailto:sa_c...@iowahoneypot.com] Sent: Wednesday, April 28, 2010 2:01 PM To: users@spamassassin.apache.org Subject: new PDF Launch malware exploit (with sample) FILE QUARANTINED Microsoft Forefront Security for Exchange Server removed a file since it was found to be infected. File name: Body of Message Virus name: TrojanDropper:Win32/Pidrop.A
RE: new PDF Launch malware exploit (with sample)
Please don't send live malware samples to the list. -Original Message- From: Chip M. [mailto:sa_c...@iowahoneypot.com] Sent: Wednesday, April 28, 2010 2:01 PM To: users@spamassassin.apache.org Subject: new PDF Launch malware exploit (with sample) FILE QUARANTINED Microsoft Forefront Security for Exchange Server removed a file since it was found to be infected. File name: Body of Message Virus name: TrojanDropper:Win32/Pidrop.A
RE: More freemail URI spam
Generally speaking, anything deemed worthwhile is added to SA proper (unless there's a licensing question). The exceptions come from automated rules (like Sought, MBL, SARE 2tld, and Khop-sc-neighbors), 90_2tld.cf has been replaced by the official rule file 20_aux_tlds.cf. From the comments in that file: # This file replaces the SARE http://www.rulesemporium.com/rules/90_2tld.cf # which will be deprecated as from 2010-05-01
RE: 90_sare_freemail.cf.sare.sa-update.dostech.net
From: Yet Another Ninja [mailto:sa-l...@alexb.ch] On 3/4/2010 7:34 PM, Rosenbaum, Larry M. wrote: From: Karsten Bräckelmann [mailto:guent...@rudersport.de] On Thu, 2010-03-04 at 00:12 +0100, Yet Another Ninja wrote: On 3/3/2010 10:09 PM, Karsten Bräckelmann wrote: On Wed, 2010-03-03 at 15:38 -0500, Rosenbaum, Larry M. wrote: Is there still a reason for this update channel? 90_sare_freemail.cf.sare.sa-update.dostech.net Or is it now built in to SA v3.3.0? ^ 20_freemail.cf and 20_freemail_domains.cf ? 90_sare_freemail.cf is still supported by for ppl who haven't upgraded to SA 3.3.x Thanks for that addition and confirmation of status. :) The original question and hence my answer was specifically about 3.3.x, though, and whether it still is needed from external sources with that version. I'm doing the same additions to 20_freemail_domains.cf Later this year, 90_sare_freemail.cf, will become unsupported. Anybody using SA 3.3.x should drop 90_sare_freemail.cf usage. Thanks, but I'm confused, as there are domains in 90_sare_freemail.cf that are not currently in 20_freemail_domains.cf. Hi Larry... Never got around to do the diff... your msg triggered :-) Unless I borked it, it should now included the missing from 90_sare_freemail.cf I still don't see the domains in 20_freemail_domains.cf. Thanks, Larry
RE: 90_sare_freemail.cf.sare.sa-update.dostech.net
-Original Message- From: Karsten Bräckelmann [mailto:guent...@rudersport.de] Sent: Wednesday, March 03, 2010 6:19 PM To: users@spamassassin.apache.org Subject: Re: 90_sare_freemail.cf.sare.sa-update.dostech.net On Thu, 2010-03-04 at 00:12 +0100, Yet Another Ninja wrote: On 3/3/2010 10:09 PM, Karsten Bräckelmann wrote: On Wed, 2010-03-03 at 15:38 -0500, Rosenbaum, Larry M. wrote: Is there still a reason for this update channel? 90_sare_freemail.cf.sare.sa-update.dostech.net Or is it now built in to SA v3.3.0? ^ 20_freemail.cf and 20_freemail_domains.cf ? 90_sare_freemail.cf is still supported by for ppl who haven't upgraded to SA 3.3.x Thanks for that addition and confirmation of status. :) The original question and hence my answer was specifically about 3.3.x, though, and whether it still is needed from external sources with that version. I'm doing the same additions to 20_freemail_domains.cf Later this year, 90_sare_freemail.cf, will become unsupported. Anybody using SA 3.3.x should drop 90_sare_freemail.cf usage. Thanks, but I'm confused, as there are domains in 90_sare_freemail.cf that are not currently in 20_freemail_domains.cf. L
90_sare_freemail.cf.sare.sa-update.dostech.net
Is there still a reason for this update channel? 90_sare_freemail.cf.sare.sa-update.dostech.net Or is it now built in to SA v3.3.0?
90_sare_freemail.cf.sare.sa-update.dostech.net
Is there still a reason for this update channel? 90_sare_freemail.cf.sare.sa-update.dostech.net Or is it now built in to SA v3.3.0?
spamd: respawning server - why?
Yesterday one of our servers started having problems. I found the following messages in the syslog file: Jan 14 14:12:38 localhost spamd[20926]: spamd: respawning server at /usr/local/bin/spamd line 1080. Jan 14 14:12:38 localhost spamd[20927]: spamd: respawning server at /usr/local/bin/spamd line 1080. Jan 14 14:13:45 localhost spamd[21038]: spamd: respawning server at /usr/local/bin/spamd line 1080. Jan 14 14:13:45 localhost spamd[21056]: spamd: respawning server at /usr/local/bin/spamd line 1080. Jan 14 14:13:45 localhost spamd[21057]: spamd: respawning server at /usr/local/bin/spamd line 1080. Jan 14 15:17:46 localhost spamd[21726]: spamd: respawning server at /usr/local/bin/spamd line 1080. ..etc.. What causes this to happen? A reboot fixed the problem, but I want to make sure it doesn't happen again. SunOS ornl50 5.9 Generic_118558-39 sun4u sparc SUNW,Sun-Fire-V210 SpamAssassin Server version 3.2.5 running on Perl 5.8.8 with zlib support (Compress::Zlib 2.011)
Spamd startup locale question
SpamAssassin Server version 3.2.5 running on Perl 5.8.8 with zlib support (Compress::Zlib 2.011) SunOS ornl72 5.9 Generic_122300-07 sun4u sparc SUNW,Sun-Fire-V240 What causes the following error message when restarting spamd? perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LC_ALL = (unset), LANG = en_US are supported and installed on your system. perl: warning: Falling back to the standard locale (C). This happens on some, but not all, of our systems running spamd. All the startup files contain LANG=en_US export LANG Thanks, Larry
RE: spamassassin bug
check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164. What plugin do I need to have loaded to resolve this error? It looks like you are missing the v320.pre file, which contains loadplugin Mail::SpamAssassin::Plugin::Check along with several other important loadplugin lines.
RE: About upgrading
-Original Message- From: Bill Landry [mailto:b...@inetmsg.com] Sent: Sunday, January 10, 2010 12:42 PM To: users@spamassassin.apache.org Subject: Re: About upgrading LuKreme wrote: On 9-Jan-2010, at 21:23, Rosenbaum, Larry M. wrote: It's the number of seconds since the epoch (Jan 1, 1970). One easy way to convert it to a readable time is # perl -e 'print scalar localtime 1263044805, \n' Sat Jan 9 08:46:45 2010 Or even simpler: perl -le 'print scalar localtime 1263049538' Sat Jan 9 05:46:45 2010 % date -r 1263044805 Sat Jan 9 06:46:45 MST 2010 On Linux based systems: date -d @1263044805 Sat Jan 9 05:46:45 PST 2010 I like this output better than the perl output because it also includes the timezone. Excellent. Is there one that works on Solaris (other than the Perl version)?
RE: About upgrading
--Original Message- From: Alex [mailto:mysqlstud...@gmail.com] Sent: Saturday, January 09, 2010 9:13 PM To: SA Mailing list Subject: Re: About upgrading Hi, sa-learn --dump magic gives: 0.000 0 3 0 non-token data: bayes db version 0.000 0 57538 0 non-token data: nspam 0.000 0 74876 0 non-token data: nham 0.000 0 166338 0 non-token data: ntokens 0.000 0 1257478501 0 non-token data: oldest atime 0.000 0 1263049426 0 non-token data: newest atime 0.000 0 1263049538 0 non-token data: last journal sync atime 0.000 0 1263044805 0 non-token data: last expiry atime 0.000 0 5529600 0 non-token data: last expire atime delta 0.000 0 1868 0 non-token data: last expire reduction count Your database has 166338 tokens which is larger than the default bayes_expiry_max_db_size 15. The last expiration ran this morning at 8:46. You could try letting the bayes database get larger and turn off bayes_auto_expire. If you turn off bayes_auto_expire you'll have to add something to cron to periodically expire tokens. bayes_auto_expire is fine for lower volumes of email, but can get in the way with higher volumes. Can I ask how you calculate the actual time from that number? I suspect it's the epoch minus some division of 24hrs, but a quick search wasn't fruitful. It's the number of seconds since the epoch (Jan 1, 1970). One easy way to convert it to a readable time is # perl -e 'print scalar localtime 1263044805, \n' Sat Jan 9 08:46:45 2010
Solaris 10 requires --syslog-socket=native
I have just recently installed SA v3.3.0-rc1 on Solaris 10. I have discovered that in order for syslog logging to work, I have to start spamd with the switch --syslog-socket=native. It won't work if I set it to unix or inet or if I omit the switch entirely. As this is my first time running SpamAssassin on Solaris 10, I don't know if this discovery also applies to older SpamAssassin versions, but I suspect it does. I suggest the documentation be changed to reflect this, since currently it does not even mention native as a legitimate option. In the long term, perhaps native can be made the default, or have the code just not call setlogsock() if the --syslog-socket switch is absent. (I don't know if this is feasible with older versions of Sys::Syslog.) This is what the current docs say: --syslog-socket=*type* Specify how spamd should send messages to syslogd. The options are unix, inet or none. The default is to try unix first, falling back to inet if perl detects errors in its unix support. Some platforms, or versions of perl, are shipped with dysfunctional versions of the Sys::Syslog package which do not support some socket types, so you may need to set this. If you get error messages regarding __PATH_LOG or similar from spamd, try changing this setting.
Recommended version of re2c?
What is the recommended version of re2c to use with SpamAssassin v3.3.0? What about with v3.2.5? Thanks, Larry
RE: ANNOUNCE: Apache SpamAssassin 3.3.0-beta1 available
From: Mark Martinec [mailto:mark.martinec...@ijs.si]
Thanks for testing! Which version of a perl module Time::HiRes
do you have installed? See what is reported by:
$ perl -MTime::HiRes -le 'print Time::HiRes-VERSION'
1.9719
This is the version that came with Perl 5.10.1, and is the same version that's
on CPAN.
Could you please try upgrading this module if yours is rather old,
and see if that helps.
P.S., does the following change to t/timeout.t on your system
make any difference in test results?
--- timeout.t 2009-12-09 03:29:12.0 +0100
+++ timeout.t 2009-12-09 03:29:19.0 +0100
@@ -23,3 +23,3 @@
use strict;
-use Time::HiRes qw(time sleep);
+use Time::HiRes qw(time sleep alarm);
That didn't help.
BTW, has the clamav interface changed? I'm getting errors from this statement:
$permsgstatus-{main}-{conf}-{headers_spam}-{Virus} = $header;
Dec 9 13:46:03.893 [27455] warn: (Not a HASH reference at
/etc/mail/spamassassin/ornl_clamav.pm line 36.
Dec 9 13:46:03.893 [27455] warn: )
If I'm analyzing correctly, $permsgstatus-{main}-{conf}-{headers_spam} used
to be a hash ref, and now it's an array ref.
RE: ANNOUNCE: Apache SpamAssassin 3.3.0-beta1 available
From: Warren Togami [mailto:wtog...@redhat.com] Subject: ANNOUNCE: Apache SpamAssassin 3.3.0-beta1 available ... - if module Digest::SHA is not available, a module Digest::SHA1 will be used, but at least one of them must be installed; a DKIM plugin requires Digest::SHA (the older Digest::SHA1 does not support sha256 hashes), so in practice the Digest::SHA is required It appears that Net::DNS requires Digest::HMAC_MD5 and that Digest::HMAC_MD5 requires Digest::SHA1. So that for full functionality, both SHA and SHA1 are needed.
RE: ANNOUNCE: Apache SpamAssassin 3.3.0-beta1 available
SpamAssassin version 3.3.0-beta1 running on Perl version 5.10.1 Solaris 9 Sparc I am getting the following errors in make test: t/timeout.t ... 5/27 # Failed test 5 in t/timeout.t at line 63 t/timeout.t ... 7/27 # Failed test 7 in t/timeout.t at line 71 t/timeout.t ... 9/27 # Failed test 9 in t/timeout.t at line 79 t/timeout.t ... 11/27 # Failed test 11 in t/timeout.t at line 87 t/timeout.t ... 13/27 # Failed test 13 in t/timeout.t at line 95 t/timeout.t ... 16/27 # Failed test 16 in t/timeout.t at line 108 # Failed test 17 in t/timeout.t at line 109 # Failed test 18 in t/timeout.t at line 110 t/timeout.t ... 22/27 # Failed test 22 in t/timeout.t at line 122 # Failed test 24 in t/timeout.t at line 124 t/timeout.t ... 25/27 # Failed test 25 in t/timeout.t at line 129 # Failed test 27 in t/timeout.t at line 131 t/timeout.t ... Failed 12/27 subtests Thanks, Larry -Original Message- From: Warren Togami [mailto:wtog...@redhat.com] Sent: Sunday, December 06, 2009 10:01 PM To: SpamAssassin Users List; Development discussions related to Fedora Core Subject: ANNOUNCE: Apache SpamAssassin 3.3.0-beta1 available Apache SpamAssassin 3.3.0-beta1 is now available for testing.
DCC problems
Recently I've started seeing message like this in our log files: Jul 7 13:22:48 ornl73 dccifd[21907]: [ID 702911 mail.notice] no working DCC serversdcc1.dcc-servers.net dcc2.dcc-servers.net dcc3.dcc-servers.net ... at 64.124.52.232 208.201.249.2 Jul 7 13:22:48 ornl73 dccifd[21907]: [ID 702911 mail.error] no working DCC serversdcc1.dcc-servers.net dcc2.dcc-servers.net dcc3.dcc-servers.net ... at 64.124.52.232 208.201.249.2 Jul 7 13:22:52 ornl73 last message repeated 4 times Jul 7 13:22:55 ornl73 dccifd[21907]: [ID 702911 mail.error] continue not asking DCC 2045 seconds after failure Jul 7 13:23:00 ornl73 dccifd[21907]: [ID 702911 mail.error] continue not asking DCC 2040 seconds after failure I don't see any messages on the DCC home page http://www.rhyolite.com/dcc/ to indicate any problems with the DCC servers. Does anybody have any ideas what's wrong? Thanks, Larry
RE: DCC problems
Thanks. It was a firewall issue. From: Michael Scheidell [mailto:scheid...@secnap.net] Sent: Tuesday, July 07, 2009 2:54 PM To: Rosenbaum, Larry M. Cc: users@spamassassin.apache.org Subject: Re: DCC problems Rosenbaum, Larry M. wrote: Recently I've started seeing message like this in our log files: Jul 7 13:22:48 ornl73 dccifd[21907]: [ID 702911 mail.notice] no working DCC serversdcc1.dcc-servers.net dcc2.dcc-servers.net dcc3.dcc-servers.net ... at 64.124.52.232 208.201.249.2 Jul 7 13:22:48 ornl73 dccifd[21907]: [ID 702911 mail.error] no working DCC serversdcc1.dcc-servers.net dcc2.dcc-servers.net dcc3.dcc-servers.net ... at 64.124.52.232 208.201.249.2 Jul 7 13:22:52 ornl73 last message repeated 4 times Jul 7 13:22:55 ornl73 dccifd[21907]: [ID 702911 mail.error] continue not asking DCC 2045 seconds after failure Jul 7 13:23:00 ornl73 dccifd[21907]: [ID 702911 mail.error] continue not asking DCC 2040 seconds after failure Try doing a cdcc rtt then find out if someone monitoring the ornl.gov firewalls mucked with the udp settings. if you are using the public servers and are doing more then 100,000 'opts' a day, you might have gotten rate limited. oh, and if you are inside of a freebsd jail, you need version .111 or better of DCC. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 | SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 This email has been scanned and certified safe by SpammerTrap(r). For Information please see www.secnap.com/products/spammertrap/http://www.secnap.com/products/spammertrap/
RE: Plugin extracting text from docs
From: Jonas Eckerman [mailto:jonas_li...@frukt.org] Rosenbaum, Larry M. wrote: It appears that pdftohtml is only available as a Windows executable (on Sourceforge). If you want a precompiled executable it seems Windows is the only platform, but AFAICS the source code is also available at http://sourceforge.net/projects/pdftohtml/files/ I have found the Xpdf package, which pdftohtml is based on, has a pdftotext command line utility. If you build it with the --without-x option, you get just the command line utilities without the X-windows stuff, which eliminates the need to install a bunch of font software.
Bayes expiration logic
Has anybody considered revising the Bayes expiration logic? Maybe it's just our data that's weird, but the built-in expiration logic doesn't seem to work very well for us. Here are my observations: There's no point in checking anything older than oldest_atime. For this value and older, zero tokens will be expired. The current estimation pass logic goes back 256 days, even if the oldest atime is one week and the calculations have already started returning zeroes. If your target corresponds to a delta of more than a few days, you're unlikely to get very close to it because the estimation pass logic uses exponentially increasing intervals. There could be a big difference between 8 days and 16 days for delta. The initial guesstimate algorithm can choose a delta that's older than the oldest atime, which will result in the dreaded expired 0E0 tokens. Conversely, it can choose a delta so new that far too many tokens are expired. You're guaranteed to have at least 100,000 tokens left, but that's not good enough if you have set the max DB size to a million or more. I suggest using a binary search or perhaps linear interpolation. The starting endpoints would be 1) Oldest atime. We already know it will expire zero tokens. 2) 12 hours ago. Calculate the number of tokens expired for this value. If it expires too few, then use this as your delta (or quit if it expires less than 1000). If it expires too many, you have your two endpoints to begin the search. You can decide when to quit by closeness to the target, size of the interval, or number of iterations, or some combination. The only problem I've seen is that the token age distribution is nonlinear enough that there are some cases where linear interpolation doesn't converge very well, and I don't know the best way to introduce fudge factors to get around this.
RE: Plugin extracting text from docs
And, please tell me of problems. pdftohtml is imho not found in gentoo, but pdf2html is maybe the same ? It appears that pdftohtml is only available as a Windows executable (on Sourceforge). I need something that will run on Solaris.
RE: Plugin extracting text from docs (was: new spam using large images)
We can use antiword to render text from MSWord files, and unrtf to render text from RTF files. What is the best tool to render text from PDF files? (We are running Solaris 9) L -Original Message- From: Jonas Eckerman [mailto:jonas_li...@frukt.org] Sent: Wednesday, June 24, 2009 1:34 PM To: users@spamassassin.apache.org Subject: Plugin extracting text from docs (was: new spam using large images) Jason Haar wrote: Speaking of image/rtf/word attachment spam; is there any work going on to standardize this so that the textual output of such attachments could be fed back into SA? Just as a note: I'm currently working on a modular plugin for extracting text and add it to SA message parts. The plugin can use either external tools or it's own simple plugin modules. How to extract text from parts is configurable, and based on mime types and file names, so new formats can be added by simply configuring for new external tolls or creating a new plugin module. My *far* from finished module currently manages to extract text from Word documents (using antiword), OpenXML text documents (using a simple plugin) and RTF (using unrtf). I haven't tested where and how the extracted text is available to SpamAssassin yet (as noted, it's *far* from finished), but I am using set_rendered method as in the example, so it should work. ;-) Regards /Jonas -- Jonas Eckerman Fruktträdet Förbundet Sveriges Dövblinda http://www.fsdb.org/ http://www.frukt.org/ http://whatever.frukt.org/
RE: SORBS bites the dust
-Original Message- From: Matus UHLAR - fantomas [mailto:uh...@fantomas.sk] IMPORTANT: If sorbs does not get picked-up by a new host, will SA developers be ready to roll-out an SA update to remove the sorbs rules, so that we don't suffer a bunch of timeouts? Or how does that work? On 23.06.09 09:29, Jeff Moss wrote: WHAT? Sorbs and Spamhaus are polar opposites. Spamhaus is a great organization while SORBS is a POS that helped give all blacklists a bad name. sorbs makes good job, although there are some whiners not understanding the stuff... I don't know if SpamAssassin has ever used it. it still does: 50_scores.cf:score RCVD_IN_SORBS_BLOCK 0 # n=1 n=2 n=3 50_scores.cf:score RCVD_IN_SORBS_DUL 0 1.615 0 0.877 # n=0 n=2 50_scores.cf:score RCVD_IN_SORBS_HTTP 0 0.001 0 0.001 # n=0 n=2 50_scores.cf:score RCVD_IN_SORBS_MISC 0 0.001 0 0.353 # n=0 n=2 50_scores.cf:score RCVD_IN_SORBS_SMTP 0 # n=0 n=1 n=2 n=3 50_scores.cf:score RCVD_IN_SORBS_SOCKS 0 0.182 0 0.801 # n=0 n=2 50_scores.cf:score RCVD_IN_SORBS_WEB 0 1.117 0 0.619 # n=0 n=2 50_scores.cf:score RCVD_IN_SORBS_ZOMBIE 0 # n=0 n=1 n=2 n=3 Notice that the SORBS spam sources list (the one that charged a delisting fee) is not used. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Remember half the people you know are below average.
RE: new spam using large images
From: felic...@kluge.net On Behalf Of Theo Van Dinter On Fri, Jun 19, 2009 at 3:04 AM, Jason Haarjason.h...@trimble.co.nz wrote: Speaking of image/rtf/word attachment spam; is there any work going on to standardize this so that the textual output of such attachments could be fed back into SA? That functionality already exists (has for almost 3 years, actually), but as in the past (list archives) the documentation hasn't improved for it. :( Here's my last(?) post about it which has some sample code and everything: http://www.nabble.com/Re:-PDFText-Plugin-for-PDF-file-scoring---not- for-PDF-images-p11595641.html Thanks for the sample code. Once you get the $p object from $msg-find_parts(), how do you extract the contents of the message part to run it through antiword or whatever? L
419 scams in .doc and .rtf attachments
We get a significant number of 419 scam letters where the actual spam text is in a Word (.doc or .rtf) or PDF attachment. Example: http://pastebin.com/m4a161daa It would be really great if there was an SA plugin to extract the text from the attachment and then feed the text to the regular SA body rules. Has anybody looked at that possibility? Thanks, Larry
Do I need to adjust bayes_expiry_max_db_size?
We are running SpamAssassin version 3.2.5 running on Perl version 5.8.8 Solaris 9 Sparc with the MySQL Bayes store and autolearning. We are using bayes_expiry_max_db_size 100 Expiry is done manually once a day. Here is a typical output from expiry: Thu Mar 19 00:12:00 EDT 2009 Forcing Bayes expiry run [2541] dbg: bayes: using username: root [2541] dbg: bayes: database connection established [2541] dbg: bayes: found bayes db version 3 [2541] dbg: bayes: Using userid: 217 [2541] dbg: bayes: bayes journal sync starting [2541] dbg: bayes: bayes journal sync completed [2541] dbg: bayes: expiry starting [2541] dbg: bayes: expiry check keep size, 0.75 * max: 75 [2541] dbg: bayes: token count: 1792961, final goal reduction size: 1042961 [2541] dbg: bayes: first pass? current: 1237435937, Last: 1237349670, atime: 86400, count: 669993, newdelta: 55502, ratio: 1.55667447271837, period: 43200 [2541] dbg: bayes: can't use estimation method for expiry, unexpected result, calculating optimal atime delta (first pass) [2541] dbg: bayes: expiry max exponent: 9 [2541] dbg: bayes: atime token reduction [2541] dbg: bayes: === [2541] dbg: bayes: 43200 1144230 [2541] dbg: bayes: 86400 732048 [2541] dbg: bayes: 172800 0 [2541] dbg: bayes: 345600 0 [2541] dbg: bayes: 691200 0 [2541] dbg: bayes: 1382400 0 [2541] dbg: bayes: 2764800 0 [2541] dbg: bayes: 5529600 0 [2541] dbg: bayes: 11059200 0 [2541] dbg: bayes: 22118400 0 [2541] dbg: bayes: first pass decided on 86400 for atime delta [2541] dbg: bayes: expiry completed expired old bayes database entries in 172 seconds 1060954 entries kept, 732007 deleted token frequency: 1-occurrence tokens: 53.44% token frequency: less than 8 occurrences: 28.22% Thu Mar 19 00:15:09 EDT 2009 Done This is telling me that there are no tokens more than 2 days old. Is this good or bad? Should I be increasing the max DB size? Thanks, Larry Note: Here is the --dump magic output: 0.000 0 3 0 non-token data: bayes db version 0.000 07894739 0 non-token data: nspam 0.000 0 10477619 0 non-token data: nham 0.000 01428534 0 non-token data: ntokens 0.000 0 1237349612 0 non-token data: oldest atime 0.000 0 1237479369 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1237436073 0 non-token data: last expiry atime 0.000 0 86400 0 non-token data: last expire atime delta 0.000 0 732007 0 non-token data: last expire reduction count
RE: Hostname in X-Spam-Checker-Version Header
From: netz-haut - stephan seitz [mailto:s.se...@netz-haut.de] Sent: Tuesday, March 17, 2009 1:15 PM To: users@spamassassin.apache.org Subject: Hostname in X-Spam-Checker-Version Header Hi there, as shown in the manual, the X-Spam-Checker-Version header is not configurable for some reason. Is there some configuration magic to change just the hostname of this header? We're running a bunch of multi-homed mail servers and I just want to glue the shown name to the official hostname, which is also located locally (just another interface in the same hosts). Try report_hostname in local.cf.
freemail_re error
What is the significance of this message in the spamd log? Feb 27 12:56:25 localhost spamd[222]: config: dup unknown type freemail_re, Regexp (Somebody asked a similar question in November, but I didn't see an answer.) Thanks, Larry
RE: misc_10.cf
The information on the download page should be corrected to point to the right file. Or perhaps removed entirely: when you build from source, the build procedure prompts you for site contact information and puts it into the 10_default_prefs.cf file. It is also put into the sa-update script, so that the substitution is made whenever sa-update downloads a new version of 10_default_prefs.cf. (I don't know if this happens for CPAN or package installs.) -Original Message- From: Matt Kettler [mailto:mkettler...@verizon.net] Sent: Tuesday, February 10, 2009 1:16 AM To: RobertH Cc: users@spamassassin.apache.org Subject: Re: misc_10.cf RobertH wrote: Um, that's a file that comes with SA, and it is *NOT* user editable. Therefore, it's not an example, it is a standard config file that generates the default settings that you later over-ride with your local.cf. The 3.2.5 installation tarball will install the version of this file that is appropriate for 3.2.5, and sa-update may update it. matt, i am not seeing that file anywhere in my install and i am quite capable of using the locate command etc... Ahh, I forgot, 10_misc.cf has been renamed to 10_default_prefs.cf. My bad. Here's the 3.2 version. http://svn.apache.org/repos/asf/spamassassin/branches/3.2/rules/10_defa ult_prefs.cf It should, by default, be in /usr/share/spamassassin, along with the other files that create the default ruleset. Updated ones created by sa-update would be in /var/lib/spamassassin. You may want to template off that (see below) i am fairly certain i hand generated and installed via rpm generated by rpm -tb sa-tarballname.whateveritwas.somethingsomething something like that. on a centos aka redhat clone the misc_10.cf file looks pretty editable to me in some respects. If it looks editable, please note it contains this text near the top: # Please don't modify this file as your changes will be overwritten with # the next update. Use @@LOCAL_RULES_DIR@@/local.cf instead. # See 'perldoc Mail::SpamAssassin::Conf' for details. (see below for more clarification) i wouldnt have even have asked if i had not gone to spamassassin.apache.org and then clicked on downloads and on that page it says System Administrators Please create a local copy of the report_template text in a file named something like /etc/mail/spamassassin/10_local_report.cf, Ok, *that* you can do. You can, at the /etc/mail/spamassassin/ level create a file, with any name, that has the report_template parts of the file and edit that. This is, of course, creating a copy in your site rules dir, which is OK. I was trying to steer you away from the very common mistake of editing the base config files in /usr/share/spamassassin, as they get over-ridden, or obliterated, by sa-update runs. Editing files at the /usr/share/spamassassin or /var/lib/spamassassin level will just result in your changes being lost on the next sa-update run. Hence the warning.
RE: sa-update damages existing SA installation
From: Daryl C. W. O'Shea [mailto:spamassas...@dostech.ca] Sent: Saturday, December 20, 2008 2:48 AM On 19/12/2008 5:40 AM, Marcin Krol wrote: Daryl C. W. O'Shea wrote: do it all at once. See my SARE sa-update page for details: http://daryl.dostech.ca/sa-update/sare/sare-sa-update-howto.txt Are SARE rules still being updated a bit at least / are they still working? The only one really being updated is 90_2tld.cf: What do I need to put in my sa-update channel file to get updates for 90_2tld.cf? (I can't get to the howto web page above.)
Backup command for AWL?
The Bayes database can be backed up and restored with sa-learn --backup/--restore. Is there any similar way to back up and restore a MySQL-based AWL database? The check_whitelist command is only good for DBM files.
RE: Backup command for AWL?
From: Theo Van Dinter [mailto:[EMAIL PROTECTED] On Fri, Dec 05, 2008 at 11:58:26AM -0500, Rosenbaum, Larry M. wrote: The Bayes database can be backed up and restored with sa-learn -- backup/--restore. Is there any similar way to back up and restore a MySQL-based AWL database? The check_whitelist command is only good for DBM files. If you're using MySQL, why not just use the standard MySQL backup tools? ie: mysqldump, etc. That's a possibility. I'm currently doing that for AWL and Bayes, but I have a question. Why does the output from mysqldump get bigger every day, while the output from sa-learn --backup stays at about the same size? (The expiry job appears to be working properly.) It's like the bayes_token table is getting larger but still holding the same amount of data.
RE: why is SA testing my server in DNSBLs?
From: Brian J. Murrell [mailto:[EMAIL PROTECTED] Hi All, I was doing a bit of spamassassin -D testing with SA 3.2.4 and noticed that it's running my own mail server name through various DNSBL tests. Here are the headers of the particular message I am testing: From [EMAIL PROTECTED] Tue Dec 2 05:24:59 2008 Return-Path: [EMAIL PROTECTED] The checks it's doing below are all RHBL checks, so it's probably testing the Return-Path:. ... [29986] dbg: dns: launching DNS A query for linux.interlinx.bc.ca.rhsbl.ahbl.org. in background [29986] dbg: async: starting: DNSBL-A, dns:A:linux.interlinx.bc.ca.rhsbl.ahbl.org. (timeout 15.0s, min 3.0s) [29986] dbg: dns: checking A and MX for host linux.interlinx.bc.ca [29986] dbg: dns: launching DNS A query for linux.interlinx.bc.ca in background [29986] dbg: async: starting: NO_DNS_FOR_FROM, DNSBL-A, dns:A:linux.interlinx.bc.ca (timeout 15.0s, min 3.0s) [29986] dbg: dns: launching DNS MX query for linux.interlinx.bc.ca in background [29986] dbg: async: starting: NO_DNS_FOR_FROM, DNSBL-MX, dns:MX:linux.interlinx.bc.ca (timeout 15.0s, min 3.0s) ... [29986] dbg: dns: launching DNS A query for linux.interlinx.bc.ca.bl.open-whois.org. in background [29986] dbg: async: starting: DNSBL-A, dns:A:linux.interlinx.bc.ca.bl.open-whois.org. (timeout 15.0s, min 3.0s) ... [29986] dbg: dns: launching DNS A query for linux.interlinx.bc.ca.fulldom.rfc-ignorant.org. in background [29986] dbg: async: starting: DNSBL-A, dns:A:linux.interlinx.bc.ca.fulldom.rfc-ignorant.org. (timeout 15.0s, min 3.0s)
RE: optional modules
From: Stefan Jakobs [mailto:[EMAIL PROTECTED] INFO: SA version: 3.2.5, 3.002005, no optional modules: Sys::Hostname::Long What is the benefit of using this modul? Sys::Hostname::Long is required by Mail::SPF::Query. However, Mail::SPF::Query has been superseded by Mail::SPF, which doesn't require Sys::Hostname::Long. Mail::SPF Mail::SPF::Server Mail::SPF::Request Mail::SPF::Mech Mail::SPF::Mech::A Mail::SPF::Mech::PTR Mail::SPF::Mech::All Mail::SPF::Mech::Exists Mail::SPF::Mech::IP4 Mail::SPF::Mech::IP6 Mail::SPF::Mech::Include Mail::SPF::Mech::MX Mail::SPF::Mod Mail::SPF::Mod::Exp Mail::SPF::Mod::Redirect Mail::SPF::SenderIPAddrMech Mail::SPF::v1::Record Mail::SPF::v2::Record Mail::SPF::Query SPF, I know it. If you have Mail::SPF, you don't need Mail::SPF::Query (although the SpamAssassin build process may complain about it being missing). NetAddr::IP NetAddr::IP::Util auto::NetAddr::IP::Util::inet_n2dx auto::NetAddr::IP::Util::ipv6_n2d What are these modules doing, what is the benefit? Mail::SPF requires NetAddr::IP.
RE: SURBL Usage Policy change
Where is the price list? I haven't been able to find it. -Original Message- From: Joseph Brennan [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 12, 2008 12:25 PM To: users@spamassassin.apache.org Subject: Re: SURBL Usage Policy change Jeff Chan [EMAIL PROTECTED] wrote: Does anyone know how many non-profits have more than 1,000 users (i.e., users with mailboxes)? Most universities and colleges have many more than that. An undergrad-only school that admits only about 200 a year would pass that number, counting faculty and staff and the summer overlap of graduated and admitted student accounts. Requiring large organizations to use rsync and charging for it makes a lot of sense. How much, though... and we didn't budget this in when we estimated last spring, for the July-June fiscal year schools use... Joseph Brennan Columbia University Information Technology
bayes_token table too big?
SpamAssassin version 3.2.5, running on Perl version 5.8.8, Solaris 9 Using MySQL for Bayes database. I'm wondering if our Bayes token database is too big, and why. Based on some posts to this list, I decided to try converting our Bayes and AWL databases to InnoDB to improve performance. So I copied the database to a non-production MySQL server and tried to convert it there. It has taken 4 days to convert! I'm thinking something must be wrong. Here is the output I'm getting from our Bayes expire job: Tue Sep 30 00:12:00 EDT 2008 Forcing Bayes expiry run expired old bayes database entries in 193 seconds 104999743 entries kept, 147355 deleted token frequency: 1-occurrence tokens: 0.12% token frequency: less than 8 occurrences: 0.05% Tue Sep 30 00:15:28 EDT 2008 Done Wed Oct 1 00:12:00 EDT 2008 Forcing Bayes expiry run expired old bayes database entries in 210 seconds 105000814 entries kept, 242825 deleted token frequency: 1-occurrence tokens: 0.11% token frequency: less than 8 occurrences: 0.06% Wed Oct 1 00:15:47 EDT 2008 Done Thu Oct 2 00:12:00 EDT 2008 Forcing Bayes expiry run expired old bayes database entries in 206 seconds 105032264 entries kept, 239214 deleted token frequency: 1-occurrence tokens: 0.13% token frequency: less than 8 occurrences: 0.06% Thu Oct 2 00:15:39 EDT 2008 Done And here is the information from the local.cf file: bayes_expiry_max_db_size 50 So the config file says 500 thousand tokens, but the database has 105 million entries. Have I misunderstood something, or is expiry not working correctly?
RE: bayes_token table too big?
From: Theo Van Dinter [mailto:[EMAIL PROTECTED] On Mon, Oct 06, 2008 at 03:42:53PM -0400, Rosenbaum, Larry M. wrote: And here is the information from the local.cf file: bayes_expiry_max_db_size 50 So the config file says 500 thousand tokens, but the database has 105 million entries. Have I misunderstood something, or is expiry not working correctly? Do an expire run w/ -D bayes and show the expiry details. Mon Oct 6 16:11:00 EDT 2008 Forcing Bayes expiry run [25080] dbg: bayes: using username: root [25080] dbg: bayes: database connection established [25080] dbg: bayes: found bayes db version 3 [25080] dbg: bayes: Using userid: 1 [25080] dbg: bayes: bayes journal sync starting [25080] dbg: bayes: bayes journal sync completed [25080] dbg: bayes: expiry starting [25080] dbg: bayes: expiry check keep size, 0.75 * max: 375000 [25080] dbg: bayes: token count: 105095925, final goal reduction size: 104720925 [25080] dbg: bayes: first pass? current: 1223323871, Last: 1223266468, atime: 43200, count: 91425, newdelta: 37, ratio: 1145.42986054143, period: 43200 [25080] dbg: bayes: can't use estimation method for expiry, unexpected result, calculating optimal atime delta (first pass) [25080] dbg: bayes: expiry max exponent: 9 [25080] dbg: bayes: atime token reduction [25080] dbg: bayes: === [25080] dbg: bayes: 43200 69517 [25080] dbg: bayes: 86400 16821 [25080] dbg: bayes: 172800 6 [25080] dbg: bayes: 345600 6 [25080] dbg: bayes: 691200 6 [25080] dbg: bayes: 1382400 6 [25080] dbg: bayes: 2764800 6 [25080] dbg: bayes: 5529600 5 [25080] dbg: bayes: 11059200 3 [25080] dbg: bayes: 22118400 3 [25080] dbg: bayes: first pass decided on 43200 for atime delta [25080] dbg: bayes: expiry completed expired old bayes database entries in 118 seconds 105026416 entries kept, 69509 deleted token frequency: 1-occurrence tokens: 0.15% token frequency: less than 8 occurrences: 0.05% Mon Oct 6 16:13:09 EDT 2008 Done
RE: Skip scanning for large mails
From: mouss [mailto:[EMAIL PROTECTED] The samples I looked at could easily be stopped otherwise (I don't usuall get a lot of lottery mail with a large .tif from a gmail address!!). but it's not worth the pain. if spammers start sending large messages, things will change... We just received a 419 spam with a 642 KB JPG file. It would be nice if SpamAssassin could at least look at the text of messages like that.
RE: Skip scanning for large mails
From: mouss [mailto:[EMAIL PROTECTED] Rosenbaum, Larry M. wrote: From: mouss [mailto:[EMAIL PROTECTED] The samples I looked at could easily be stopped otherwise (I don't usuall get a lot of lottery mail with a large .tif from a gmail address!!). but it's not worth the pain. if spammers start sending large messages, things will change... We just received a 419 spam with a 642 KB JPG file. It would be nice if SpamAssassin could at least look at the text of messages like that. Can you post a copy on a web page, just to see if it can be stopped with MTA checks. http://pastebin.com/m479512c6
RE: Skip scanning for large mails
From: Evan Platt [mailto:[EMAIL PROTECTED] Rosenbaum, Larry M. wrote: We just received a 419 spam with a 642 KB JPG file. It would be nice if SpamAssassin could at least look at the text of messages like that. Wouldn't FuzzyOCR pick up on that? Not if spamc never passes it to spamd because it's over 256K.
SpamAssassin slowdown
SpamAssassin version 3.2.5 running on Perl version 5.8.8 Solaris 9 SPARC For the past few days I have noticed SpamAssassin takes much longer to process messages between about 10:00am and 1:00pm EDT. It doesn't appear to be a memory problem or a problem with our DNS server. To try to figure it out, I got a message which took 58 seconds to process and ran it through spamassassin -D, taking note of where the output paused for a long time. The results are shown below. Can anybody tell me what SpamAssassin is doing at that point? It says compiled body tests, but we are not running a compiled configuration. ... [13557] dbg: Botnet: RDNS is 'mail4.smartdeals-mail.com' [13557] dbg: Botnet: HELO is 'mail4.smartdeals-mail.com' [13557] dbg: Botnet: EnvelopeFrom is [EMAIL PROTECTED] [13557] dbg: Botnet: mail domain is smartdeals-mail.com [13557] dbg: Botnet: SOHO hit [13557] dbg: rules: ran eval rule BOTNET_SOHO == got hit (1) [13557] dbg: dkim: performing public key lookup and signature verification [13557] warn: Use of uninitialized value in sprintf at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Logger.pm line 213. [13557] dbg: dkim: signing identity: , d=smartdeals-mail.com, a=rsa-sha1, c=nofws [13557] dbg: dkim: public key lookup or verification failed: cannot change identity on Mail::DKIM::DkSignature at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Plugin/DKIM.pm line 366 [13557] dbg: dkim: policy: performing lookup [13557] dbg: dkim: policy result neutral: v=spf1 mx ip4:69.56.11.40/29 -all [13557] dbg: Botnet: checking NORDNS [13557] dbg: Botnet: no trusted relays [13557] dbg: Botnet: get_relay good RDNS [13557] dbg: Botnet: IP is '69.56.11.43' [13557] dbg: Botnet: RDNS is 'mail4.smartdeals-mail.com' [13557] dbg: Botnet: HELO is 'mail4.smartdeals-mail.com' [13557] dbg: Botnet: NORDNS miss [13557] dbg: spf: def_whitelist_from_spf: [EMAIL PROTECTED] is not in DEF_WHITELIST_FROM_SPF [13557] dbg: FreeMail: replyto: skipping, envelope sender looks bulk [13557] dbg: spf: whitelist_from_spf: [EMAIL PROTECTED] is not in user's WHITELIST_FROM_SPF [13557] dbg: async: select found no responses ready (t.o.=0.0) [13557] dbg: async: completed in 0.913 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:197.37.56.69 [13557] dbg: async: completed in 0.912 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:1.15.56.69 [13557] dbg: async: completed in 0.917 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:7.37.42.70 [13557] dbg: async: completed in 0.915 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:42.142.25.69 [13557] dbg: async: completed in 0.920 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:225.145.64.69 [13557] dbg: async: completed in 0.914 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:248.184.52.216 [13557] dbg: async: completed in 0.919 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:200.92.251.63 [13557] dbg: async: completed in 0.918 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:4.32.56.69 [13557] dbg: dns: harvested completed queries [13557] dbg: rules: running body tests; score so far=12.48 - Here is where output paused for a long time --- [13557] dbg: rules: compiled body tests [13557] dbg: rules: ran body rule __SARE_PHONE_NUM == got hit: 866-315-8447 [13557] dbg: rules: ran body rule FB_GET_MEDS == got hit: get a Medic [13557] dbg: rules: ran body rule __ORNL_LOTTO_BINGO_6 == got hit: email address [13557] dbg: rules: ran body rule __HAS_ANY_EMAIL == got hit: [EMAIL PROTECTED] [13557] dbg: rules: ran body rule __NONEMPTY_BODY == got hit: G [13557] dbg: rules: ran body rule __KAM_LOTTO3 == got hit: claim [13557] dbg: rules: ran body rule ACT_NOW_CAPS == got hit: Act Now [13557] dbg: rules: ran body rule __DOS_LINK == got hit: link [13557] dbg: rules: running uri tests; score so far=14.075 [13557] dbg: rules: compiled uri tests [13557] dbg: rules: ran uri rule __DOS_HAS_ANY_URI == got hit: m [13557] dbg: rules: ran uri rule __LOCAL_PP_NONPPURL == got hit: http://www.onewaytoadventure.com; [13557] dbg: pdfinfo: Identified 0 possible mime parts that need checked for PDF content [13557] dbg: pdfinfo: set_tag called for PDFCOUNT 0 [13557] dbg: pdfinfo: set_tag called for PDFIMGCOUNT 0 [13557] dbg: rules: ran eval rule __TAG_EXISTS_BODY == got hit (1) [13557] dbg: IXHASH: IxHash querying Server nospam.login-solutions.de [13557] dbg: IXHASH: Computed hash-value de2b33469f305b6e93687e3eb710db4d via method 1 [13557] dbg: IXHASH: Now checking de2b33469f305b6e93687e3eb710db4d.nospam.login-solutions.de ...
RE: Spam flooding recent days
From: Michał Jęczalik [mailto:[EMAIL PROTECTED] Subject: Spam flooding recent days Hello, I've noticed a huge increase of spam rate in past 2-3 weeks. Most of it are messages with some quite normal Subject:, often (but not neccesarily) referring to some fake event (i.e. some politician stabbed to death) and there's only a link, sometimes together with a single sentence, in the body It's called tabloid spam. http://redtape.msnbc.com/2008/07/no-presidential.html#posts
RE: Upgrade SpamAssassin failing
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5591 -Original Message- From: Asif Iqbal [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 01, 2008 3:59 PM To: users@spamassassin.apache.org Subject: Re: Upgrade SpamAssassin failing Still looking for some suggestion on this On Sun, Mar 30, 2008 at 11:59 AM, Asif Iqbal [EMAIL PROTECTED] wrote: I have tried to upgrade spamassassin twice with no avail. Current version running is 3.2.3 on Solaris 8 cpan install Mail::SpamAssassin Running install for module Mail::SpamAssassin Running make for J/JM/JMASON/Mail-SpamAssassin-3.2.4.tar.gz Is already unwrapped into directory /usr/local/cpan/build/Mail-SpamAssassin-3.2.4 Has already been processed within this session Running make test cp sa-compile blib/script/sa-compile /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/sa- compile cp spamc/spamc blib/script/spamc /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/spamc cp sa-learn blib/script/sa-learn /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/sa- learn cp spamassassin blib/script/spamassassin /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/spamassassin cp spamd/spamd blib/script/spamd /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/spamd /usr/bin/perl build/mkrules --exit_on_no_src --src rulesrc --out rules --manifest MANIFEST --manifestskip MANIFEST.SKIP no source directory found: exiting /usr/bin/perl build/preprocessor -Mvars -DVERSION=3.002004 -DPREFIX=/usr/local - DDEF_RULES_DIR=/usr/local/share/spamassassin -DLOCAL_RULES_DIR=/etc/mail/spamassassin -DLOCAL_STATE_DIR=/var/lib/spamassassin -DINSTALLSITELIB=/usr/local/lib/perl5/site_perl/5.8.0 -DCONTACT_ADDRESS=the administrator of that system -Msharpbang -Mconditional -DPERL_BIN=/usr/bin/perl -DPERL_WARN= -DPERL_TAINT= -m755 -isa-update.raw -osa-update cp sa-update blib/script/sa-update /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/sa- update PERL_DL_NONLAZY=1 /usr/bin/perl -MExtUtils::Command::MM -e test_harness(0, 'blib/lib', 'blib/arch') t/*.t t/basic_lintok t/basic_obj_api.ok t/bayesdbm..ok t/bayesdbm_flockok t/bayessdbm.ok t/bayessdbm_seen_delete.ok t/bayessql..skipped all skipped: no reason given t/blacklist_autolearn...ok t/body_mod..ok t/check_implemented.ok t/cidrs.ok t/config_errs...skipped all skipped: no reason given t/config_text...ok t/cpp_comments_in_spamc.ok t/date..ok t/db_awl_path...ok t/db_based_whitelistok t/db_based_whitelist_ipsok t/dcc...skipped all skipped: no reason given t/debug.ok t/desc_wrap.ok t/dkim..skipped all skipped: no reason given t/dnsbl.skipped all skipped: no reason given t/dnsbl_sc_meta.skipped all skipped: no reason given t/duplicatesok t/get_all_headers...ok t/get_headers...ok t/gtube.ok t/hashcash..ok t/html_colors...ok t/html_obfu.ok t/html_utf8.skipped all skipped: no reason given t/ifversion.ok t/ip_addrs..ok t/lang_lint.ok t/lang_pl_tests.couldn't set locale correctly t/lang_pl_tests.ok t/line_endings..ok t/lint_nocreate_prefs...ok t/memory_cycles.skipped all skipped: no reason given t/meta..ok t/metadata..ok t/mimeheaderok t/mimeparse.ok t/missing_hb_separator..ok t/mkrules...ok t/nonspam...ok t/pluginok t/plugin_file...ok t/plugin_priorities.ok t/prefs_include.ok t/prioritiesok t/razor2skipped all skipped: no reason given t/rcvd_parser...ok t/re_base_extractionok 1/115 100% Completed 18.86 rules/sec in 00m00s 100% Completed 2909.18 bases/sec in 00m00s t/re_base_extractionok 7/115 100% Completed 21.38 rules/sec in 00m00s 100% Completed 1984.06 bases/sec in 00m00s t/re_base_extractionok 10/115 100% Completed
RE: AWL scores high after receiving spam from myself?
From: Andreas Ntaflos [mailto:[EMAIL PROTECTED] To spamassassin this spam appears to come from myself. It scored a low AWL but over 16 points all in all so the next message received from [EMAIL PROTECTED] would certainly get high AWL score. My questions are these: did I get this right? Is that really what seems to be happening? If so, how do I handle such a scenario? When it is so easy to forge header fields does it even make sense to have an AWL that assigns scores based on where the mail *appears* to be coming from? The AWL classifies its history by both return address and IP. It sounds like in your case it is using the wrong IP, which may indicate problems with your trust path. Please see http://wiki.apache.org/spamassassin/TrustPath
RE: Rule for Russian character sets
From: Karsten Bräckelmann [mailto:[EMAIL PROTECTED] I've pointed it out before. Just use ok_locales, which is all about these char sets. No REs, almost no thinking required, no headache. A single line, and you're done. What's the best way to test the character set for use in a meta rule? We don't want to reject all messages with the Russian (Cyrillic) character set, but we may want to use something like if (character set is Russian) (body contains 'xyzzy') for instance. How would we test the character set?
RE: sa-compile format the standard now?
From: Bowie Bailey [mailto:[EMAIL PROTECTED] I would say that sa-compile is the preferred method due to its performance benefits. There aren't many (any?) drawbacks to using it. I don't use it here because it takes too long (over 20 minutes) to compile. (This is with SA v3.2.4, which is a big improvement over v3.2.3) L
RE: sa-compile format the standard now?
From: Bowie Bailey [mailto:[EMAIL PROTECTED] Rosenbaum, Larry M. wrote: From: Bowie Bailey [mailto:[EMAIL PROTECTED] I would say that sa-compile is the preferred method due to its performance benefits. There aren't many (any?) drawbacks to using it. I don't use it here because it takes too long (over 20 minutes) to compile. (This is with SA v3.2.4, which is a big improvement over v3.2.3) Why is this an issue? You only have to recompile the rules if there is an update and this is only once every week or two even when things are active. We have a lot of local rules that I fine-tune often, sometimes several times a day.
RE: Googlepages Livefilestore spams
Is it safe to use unbounded quantifiers like + and {2,} in uri rules? I avoid
them in regular body rules.
L
-Original Message-
From: Ben Lentz [mailto:[EMAIL PROTECTED]
Sent: Wednesday, January 09, 2008 10:56 PM
To: users@spamassassin.apache.org
Subject: Re: Googlepages Livefilestore spams
but this URI redirection stuff isn't very friendly
when used by a spammer.
Ben, the key is the btnI param, which maps to the I'm feeling
lucky
button.
This technique appeared last summer (I deployed my non-SA-based rule
on
03-Jul-2007).
Thank you, this is very valuable. I wonder if Google will ever consider
turning it off, since it's being abused.
For now, I'm going with:
uri GOOG_REDIR_SLASH
m{^https?://(?:\w+\.)*google\.(com|co\.uk|tw)/{2,}search}
score GOOG_REDIR_SLASH1.0
describeGOOG_REDIR_SLASHGoogle URL has extra slashes
after domain
uri GOOG_REDIR_LUCKY
m{^https?://(?:\w+\.)*google\.(com|co\.uk|tw)/+search.*btnI}
score GOOG_REDIR_LUCKY3.0
describeGOOG_REDIR_LUCKYGoogle URL uses I'm Feeling
Lucky for blind redirect
uri GOOG_PAGES
m{^https?://(?:\w+\.)*googlepages\.(com|co\.uk|tw)}
score GOOG_PAGES 2.0
describeGOOG_PAGES URL hosted at GooglePages
...seems pretty safe.
Mismatched URLs revisited
Some time ago (and more than once) there have been discussions on this list about email containing hyperlinks where the link text is a URL that doesn't match the URL in the link HREF, and the pros and cons of testing for and scoring these mismatched links. My management has raised this issue. My memory is hazy on what the final opinions were - it seems like this was initially discouraged, but later discussions may have been less discouraging. Could somebody point me to the threads where this is discussed? Also, does SpamAssassin currently contain any rules for this kind of testing, or are there third-party rules that do this? Thanks, Larry
whitelist_from_rcvd with numeric IP?
The documentation for whitelist_from_rcvd shows examples like this: whitelist_from_rcvd [EMAIL PROTECTED] example.com What if the sending server has no rDNS? Is there a way to use this feature with a numeric IP instead of a rDNS domain? If so, what is the syntax? Thanks, Larry
spamd children killed but don't die?
This morning on one of our servers, spamd was having problems. There were 8 spamd children running, but top showed only two of them were using any CPU time even though there was a backlog of messages to be processed. The log file included lines like this: Aug 24 08:49:22 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:23 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:32 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:41 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:49 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:51 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:49:59 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:50:00 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:50:04 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:50:06 localhost spamd[21051]: prefork: child states: BBBK Aug 24 08:50:12 localhost spamd[21051]: prefork: child states: BBBK which I think means 3 children busy, 5 children waiting to die. This (the multiple K children) had been going on for a few hours, which prevented new children from being spawned to handle the load. Restarting spamd via kill -HUP restored normal operation. Why were the killed processes not dying? System information: SunOS email 5.9 Generic_118558-39 sun4u sparc SUNW,Sun-Fire-V210 SpamAssassin Server version 3.2.3 running on Perl 5.8.8 with SSL support (IO::Socket::SSL 0.97) with zlib support (Compress::Zlib 1.41) Process information (combination of top and ps): Fri Aug 24 08:56:52 2007 last pid: 23996; load averages: 0.55, 0.54, 0.52 192 processes: 190 sleeping, 2 on cpu CPU states: 88.4% idle, 4.2% user, 3.4% kernel, 3.9% iowait, 0.0% swap Memory: 2048M real, 342M free, 1323M swap in use, 6109M swap free USER PID PPID STIME TIMESTATESIZE RES CPU spamd23740 2105108:55:39 0:17sleep 72M 59M 11.40% spamd20459 2105108:16:13 7:08cpu/1 77M 66M 6.27% root 21051 109:47:26 2:17sleep 66M 58M 0.03% spamd27830 2105103:39:24 5:28sleep 81M 70M 0.00% spamd27926 2105103:39:37 0:26sleep 76M 63M 0.00% spamd14411 2105100:53:09 0:11sleep 70M 51M 0.00% spamd22780 2105102:37:46 0:06sleep 71M 57M 0.00% spamd22775 2105102:37:32 0:04sleep 70M 56M 0.00% spamd22776 2105102:37:32 0:01sleep 68M 54M 0.00% spamd startup command: ulimit -n 256 spamd -d -u spamd -r $pidfile -x -m 8 --syslog=local2 --syslog-socket=inet -i -A $me,$em1,$em2,$em3,$em4
RE: warning - score undef for rule 'MISSING_SUBJECT'...
From: Leon Kolchinsky [mailto:[EMAIL PROTECTED] The first time I run sa-update after a v3.2.3 install, I get the following warnings: rules: score undef for rule 'MISSING_SUBJECT' in '' 'MISSING_SUBJECT' at ... The score MISSING_SUBJECT is removed from 3.1.x and 3.2.x now. You could check your local.cf (or in some .pre file) for this score and remove it. That's not what grep says: email# grep MISSING_SUBJECT /var/lib/spamassassin/3.002003/updates_spamassassin_org/* /var/lib/spamassassin/3.002003/updates_spamassassin_org/20_head_tests.cf :meta MISSING_SUBJECT !__HAS_SUBJECT /var/lib/spamassassin/3.002003/updates_spamassassin_org/20_head_tests.cf :describe MISSING_SUBJECT Missing Subject: header /var/lib/spamassassin/3.002003/updates_spamassassin_org/30_text_de.cf:la ng de describe MISSING_SUBJECT Betreff (Subject) fehlt /var/lib/spamassassin/3.002003/updates_spamassassin_org/50_scores.cf:sco re MISSING_SUBJECT 2.307 1.285 2.476 1.762 email# grep MISSING_SUBJECT /usr/local/share/spamassassin/*.cf /usr/local/share/spamassassin/20_head_tests.cf:meta MISSING_SUBJECT !__HAS_SUBJECT /usr/local/share/spamassassin/20_head_tests.cf:describe MISSING_SUBJECT Missing Subject: header /usr/local/share/spamassassin/30_text_de.cf:lang de describe MISSING_SUBJECT Betreff (Subject) fehlt /usr/local/share/spamassassin/50_scores.cf:score MISSING_SUBJECT 2.307 1.285 2.476 1.762 email# grep MISSING_SUBJECT /etc/mail/spamassassin/*.cf email#
warning - score undef for rule 'MISSING_SUBJECT'...
The first time I run sa-update after a v3.2.3 install, I get the following warnings: rules: score undef for rule 'MISSING_SUBJECT' in '' 'MISSING_SUBJECT' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. rules: score undef for rule 'EMPTY_MESSAGE' in '' 'EMPTY_MESSAGE' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. rules: score undef for rule 'NO_RECEIVED' in '' 'NO_RECEIVED' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. rules: score undef for rule 'MISSING_SUBJECT' in '' 'MISSING_SUBJECT' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. rules: score undef for rule 'EMPTY_MESSAGE' in '' 'EMPTY_MESSAGE' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. rules: score undef for rule 'NO_RECEIVED' in '' 'NO_RECEIVED' at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 2140. ... (repeated several times) The update succeeds anyway. What causes these warnings? Thanks, Larry
RE: ANNOUNCE: Apache SpamAssassin 3.2.3 available
I don't see it. How long does it take to get to the mirrors? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Mason Sent: Thursday, August 09, 2007 3:39 PM To: users@SpamAssassin.apache.org; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: ANNOUNCE: Apache SpamAssassin 3.2.3 available Apache SpamAssassin 3.2.3 is now available! This is a maintenance release of the 3.2.x branch. Downloads are available from: http://spamassassin.apache.org/downloads.cgi The release file will also be available via CPAN in the near future. md5sum of archive files: e9a5fd94dead0fca3f26fb3feb0c8e57 Mail-SpamAssassin-3.2.3.tar.bz2 2e356b70b9458b44a828c19f6e816521 Mail-SpamAssassin-3.2.3.tar.gz 6ea8ef7f37e4b305217fa8074dd2219e Mail-SpamAssassin-3.2.3.zip sha1sum of archive files: 53199e0218d2f93043fcdca4db3f164f1f9f7cbc Mail-SpamAssassin- 3.2.3.tar.bz2 93337a5cf6cc6f4980307c08ad65575fa08d1f54 Mail-SpamAssassin- 3.2.3.tar.gz 0eca91718518547323f43b5473d1362032edb592 Mail-SpamAssassin-3.2.3.zip
3.2.3 spamd_hup test failed
SpamAssassin v3.2.3, Perl 5.8.8, Solaris 9 What would cause this error? t/spamd_hup.ok 1/110# Failed test 5 in t/spamd_hup.t at line 40 # t/spamd_hup.t line 40 is: ok (-e $pid_file) or warn $pid_file does not exist post restart; log/spamd.pid does not exist post restart at t/spamd_hup.t line 40. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. Could not open pid file log/spamd.pid: No such file or directory Exiting subroutine via next at t/SATest.pm line 844. t/spamd_hup.FAILED tests 5, 7-110 Failed 105/110 tests, 4.55% okay The same test on a different, supposedly identical system passed. Also it passed when I ran it manually with prove -v t/spamd_hup.t.
iXhash zone transfer?
Is it possible to get zone transfers of the iXhash data?
What is bug 5548
In the SA 3.2.2 announcement was this item: - bug 5548: Certain mail input can take a long time to scan with 100% CPU utilisation, due to backtracking in a rule's regexp. fix but when I went to look up the bug, I didn't have permission. Could somebody give me more detail on this bug? In particular, we had a problem with v3.2.1 using a lot of CPU until we put the use bytes; hack (* see footnote) into Message.pm and I would like to know if 3.2.2 fixes this problem. * Note that the CPU usage was a problem even after we upgraded Perl to 5.8.8 to get rid of the UTF bug and associated log file messages.
Microsoft Antigen Spam Manager
I have been asked to look into Microsoft Antigen Spam Manager (and MS Forefront for Exchange Server). Does anybody have any information (or can point me to independent reviews) as to how good this product is and how it compares with SpamAssassin? Thanks ...
RE: URIBL_BLACK matching on messages with no URLs in them...
From: Jo Rhett [mailto:[EMAIL PROTECTED] SA doesn't just look for full URLs, it looks for things that could be hostnames ala copy www.example.com into your browser. This is fairly nonfunctional. I've been chasing around all sorts of FPs that seem to hit pretty much every message that comes to me with source code inside it, and you've probably nailed every one of them on the head. I didn't realize that they were related at the SA level. I need to completely disable this over-opportunistic behavior. 90% of my e-mails have either system output, or are concerning code segments or router interfaces, etc, etc. I need these mails to get through. At the very least, common collisions like script.pl need to be disabled. uridnsbl_skip_domain script.pl
RE: SA 3.2.1 not using SQL for bayes
From: Hamie [mailto:[EMAIL PROTECTED] Small problem with SA 3.2.1... I'm using a mysql database. The DB works fine for amavisd-new, and SA AWL. e.g. [438288] dbg: auto-whitelist: sql-based connected to DBI:mysql:sadb:ixx: [438288] dbg: auto-whitelist: sql-based using username: vscan01 [438288] dbg: auto-whitelist: sql-based get_addr_entry: found existing entry for [EMAIL PROTECTED]|ip=xx.xx [438288] dbg: auto-whitelist: sql-based [EMAIL PROTECTED]|ip=195.53 scores 21/8.128 [438288] dbg: auto-whitelist: AWL active, pre-score: -1.498, autolearn score: -1.498, mean: 0.387047619047619, IP: xx.xx.xx.xx [438288] dbg: auto-whitelist: sql-based add_score: new count: 22, new totscore: 6.63 for [EMAIL PROTECTED]|ip=xx.xx [438288] dbg: auto-whitelist: sql-based finish: disconnected from DBI:mysql:sadb: but the SA bayes SQL spits out the error [463326] dbg: plugin: loading Mail::SpamAssassin::BayesStore::SQL from @INC [463326] dbg: bayes: invalid config, must set bayes_sql_dsn config variable plugin: failed to create instance of plugin Mail::SpamAssassin::BayesStore::SQL: Which would be fine... Except my local.cf file (Which also defines the config for AWL) says bayes_sql_dsn DBI:mysql:sadb:xxx: bayes_sql_username spamassassin bayes_sql_password fred bayes_sql_override_usernamevscan01 bayes_store_module Mail::SpamAssassin::BayesStore::SQL I think you want this: bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
RE: make test dnsbl tests sporadically fail
I installed both patches and still get errors in some of the dnsbl tests. Here is a possibly relevant section of t/log/d.dns/1 from a system where the test succeeded: [27718] dbg: check: running tests for priority: 500 [27718] dbg: async: select found 1 socks ready [27718] dbg: uridnsbl: query for uribl-example-b.com took 4 seconds to look up (multi.surbl.org.:uribl-example-b.com) [27718] dbg: uridnsbl: query for uribl-example-a.com took 4 seconds to look up (multi.uribl.com.:uribl-example-a.com) ... [27718] dbg: uridnsbl: query for uribl-example-a.com took 4 seconds to look up (bl.open-whois.org.:uribl-example-a.com) [27718] dbg: async: queries completed: 73 started: 0 [27718] dbg: async: queries active: at Fri Jun 15 11:42:27 2007 [27718] dbg: dns: success for 0 of 73 queries [27718] dbg: rules: running head tests; score so far=18.85 And here is the corresponding log where the tests failed: [10362] dbg: check: running tests for priority: 500 [10362] dbg: async: select found no socks ready [10362] dbg: uridnsbl: query for uribl-example-b.com took 2 seconds to look up (multi.surbl.org.:uribl-example-b.com) [10362] dbg: uridnsbl: query for uribl-example-a.com took 2 seconds to look up (multi.uribl.com.:uribl-example-a.com) ... [10362] dbg: uridnsbl: query for uribl-example-a.com took 2 seconds to look up (bl.open-whois.org.:uribl-example-a.com) [10362] dbg: async: queries completed: 44 started: 0 [10362] dbg: async: queries active: DNSBL-A=10 DNSBL-TXT=19 at Fri Jun 15 11:59:06 2007 [10362] dbg: async: select found no socks ready [10362] dbg: async: queries completed: 0 started: 0 [10362] dbg: async: queries active: DNSBL-A=10 DNSBL-TXT=19 at Fri Jun 15 11:59:07 2007 [10362] dbg: async: select found no socks ready ... [10362] dbg: async: queries completed: 0 started: 0 [10362] dbg: async: queries active: DNSBL-A=10 DNSBL-TXT=19 at Fri Jun 15 11:59:27 2007 [10362] dbg: async: escaping: must have lost requests [10362] dbg: async: aborting remaining lookups [10362] dbg: dns: success for 44 of 73 queries [10362] dbg: rules: running head tests; score so far=14.85 So what is going on, and why aren't my socks ready? (Sounds like a laundry problem...) BTW, looking up 134.88.73.210.sb.dnsbltest.spamassassin.org (one of the failed lookups) from the command line returns a successful answer immediately. From: Randal, Phil [mailto:[EMAIL PROTECTED] Sent: Thursday, June 14, 2007 3:41 AM To: users@spamassassin.apache.org Subject: RE: make test dnsbl tests sporadically fail Possibly related to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 as discussed in the DNS tests getting aborted thread? Cheers, Phil -- Phil Randal Network Engineer Herefordshire Council Hereford, UK From: Rosenbaum, Larry M. [mailto:[EMAIL PROTECTED] Sent: 13 June 2007 22:01 To: users@spamassassin.apache.org Subject: make test dnsbl tests sporadically fail When I run make test for v3.2.1, why do some of the dnsbl tests sporadically fail? For instance: t/dnsbl.Not found: P_2 = dns:134.88.73.210.dnsbltest.spamassassin.org [127.0.0.4] # Failed test 1 in t/SATest.pm at line 635 Not found: P_7 = dns:134.88.73.210.sb.dnsbltest.spamassassin.org?type=TXT # Failed test 2 in t/SATest.pm at line 635 fail #2 Not found: P_4 = dns:14.35.17.212.dnsbltest.spamassassin.org [127.0.0.1] t/dnsbl.NOK 1# Failed test 3 in t/SATest.pm at line 635 fail #3 Not found: P_3 = dns:18.13.119.61.dnsbltest.spamassassin.org [127.0.0.12] # Failed test 4 in t/SATest.pm at line 635 fail #4 Not found: P_5 = dns:226.149.120.193.dnsbltest.spamassassin.org [127.0.0.1] # Failed test 5 in t/SATest.pm at line 635 fail #5 t/dnsbl.NOK 2 Not found: P_1 = dns:98.3.137.144.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 6 in t/SATest.pm at line 635 fail #6 Not found: P_6 = dns:example.com.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 7 in t/SATest.pm at line 635 fail #7 Not found: P_15 = DNSBL_RHS t/dnsbl.NOK 3# Failed test 8 in t/SATest.pm at line 635 fail #8 Not found: P_17 = DNSBL_SB_FLOAT t/dnsbl.NOK 4# Failed test 9 in t/SATest.pm at line 635 fail #9 Not found: P_18 = DNSBL_SB_STR # Failed test 10 in t/SATest.pm at line 635 fail #10 Not found: P_16 = DNSBL_SB_TIME # Failed test 11 in t/SATest.pm at line 635 fail #11 t/dnsbl.NOK 5 Not found: P_10 = DNSBL_TEST_DYNAMIC # Failed test 12 in t/SATest.pm at line 635 fail #12 Not found: P_12
v3.2.1 gives spamd: handle_user unable to find user:
SpamAssassin Server version 3.2.1
running on Perl 5.8.8
with zlib support (Compress::Zlib 2.004)
I've started seeing the spamd: handle_user unable to find user:
message in the spamd log file. This was not happening in v3.2.0. We
are starting spamd with this command:
spamd -d -u spamd -r $pidfile -x -m 12 --syslog=local2
--syslog-socket=inet -i -A $me,$em1,$em2,$em3,$em4
We are not using any kind of per-user configuration or per-user Bayes or
anything like that, and I don't expect the username that is running
spamc (on another machine) to exist on the spamd machine. What options
do I need to specify to suppress this error? Is spamd doing anything it
shouldn't do because of this error?
The change in behavior seems to be related to this change to the
got_user_header() function:
@@ -1886,9 +1911,12 @@
handle_user_setuid_with_ldap($current_user);
$setuid_to_user = 1;# as above
}
+else {
+ handle_user_setuid_basic($current_user);
+}
}
else {
-handle_user($current_user);
+handle_user_setuid_basic($current_user);
if ( $opt{'sql-config'} ) {
unless ( handle_user_sql($current_user) ) {
service_unavailable_error(Error fetching user preferences via
SQL);
make test dnsbl tests sporadically fail
When I run make test for v3.2.1, why do some of the dnsbl tests sporadically fail? For instance: t/dnsbl.Not found: P_2 = dns:134.88.73.210.dnsbltest.spamassassin.org [127.0.0.4] # Failed test 1 in t/SATest.pm at line 635 Not found: P_7 = dns:134.88.73.210.sb.dnsbltest.spamassassin.org?type=TXT # Failed test 2 in t/SATest.pm at line 635 fail #2 Not found: P_4 = dns:14.35.17.212.dnsbltest.spamassassin.org [127.0.0.1] t/dnsbl.NOK 1# Failed test 3 in t/SATest.pm at line 635 fail #3 Not found: P_3 = dns:18.13.119.61.dnsbltest.spamassassin.org [127.0.0.12] # Failed test 4 in t/SATest.pm at line 635 fail #4 Not found: P_5 = dns:226.149.120.193.dnsbltest.spamassassin.org [127.0.0.1] # Failed test 5 in t/SATest.pm at line 635 fail #5 t/dnsbl.NOK 2 Not found: P_1 = dns:98.3.137.144.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 6 in t/SATest.pm at line 635 fail #6 Not found: P_6 = dns:example.com.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 7 in t/SATest.pm at line 635 fail #7 Not found: P_15 = DNSBL_RHS t/dnsbl.NOK 3# Failed test 8 in t/SATest.pm at line 635 fail #8 Not found: P_17 = DNSBL_SB_FLOAT t/dnsbl.NOK 4# Failed test 9 in t/SATest.pm at line 635 fail #9 Not found: P_18 = DNSBL_SB_STR # Failed test 10 in t/SATest.pm at line 635 fail #10 Not found: P_16 = DNSBL_SB_TIME # Failed test 11 in t/SATest.pm at line 635 fail #11 t/dnsbl.NOK 5 Not found: P_10 = DNSBL_TEST_DYNAMIC # Failed test 12 in t/SATest.pm at line 635 fail #12 Not found: P_12 = DNSBL_TEST_RELAY # Failed test 13 in t/SATest.pm at line 635 fail #13 t/dnsbl.NOK 6 Not found: P_11 = DNSBL_TEST_SPAM # Failed test 14 in t/SATest.pm at line 635 fail #14 Not found: P_8 = DNSBL_TEST_TOP # Failed test 15 in t/SATest.pm at line 635 fail #15 Not found: P_9 = DNSBL_TEST_WHITELIST t/dnsbl.NOK 7# Failed test 16 in t/SATest.pm at line 635 fail #16 Not found: P_14 = DNSBL_TXT_RE # Failed test 17 in t/SATest.pm at line 635 fail #17 Not found: P_13 = DNSBL_TXT_TOP t/dnsbl.NOK 8# Failed test 18 in t/SATest.pm at line 635 fail #18 t/dnsbl.NOK 9Output can be examined in: log/d.dns/1 t/dnsbl.FAILED tests 1-18 Failed 18/23 tests, 21.74% okay If I run t/dnsbl.t later, a smaller number of the subtests fail. If I repeat it later, a different set of dnsbl subtests fail. There is nothing obviously wrong with the DNS server. What causes this problem?
RE: Error on startup after upgrade to 3.2.1:CompiledRegexps
From: Steven W. Orr [mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] ~]# rpm -Uvh /usr/src/redhat/RPMS/i386/spamassassin-3.2.1-1.i386.rpm /usr/src/redhat/RPMS/i386/perl-Mail-SpamAssassin-3.2.1-1.i386.rpm Preparing... ### [100%] 1:perl-Mail-SpamAssassin ### [ 50%] 2:spamassassin ### [100%] Stopping spamd: [ OK ] Starting spamd: [13775] error: Can't locate Mail/SpamAssassin/CompiledRegexps/body_0.pm in @INC (@INC contains: ... Am I concerned? Where is CompiledRegexps supposed to be and why is it not there? I think it means you have Rule2XSBody uncommented in v320.pre but you don't have a compiled ruleset. Perhaps you ran sa-compile under v3.2.0 but you haven't run it under v3.2.1.
RE: ANNOUNCE: Apache SpamAssassin 3.2.1 available
From: Duncan Hill [mailto:[EMAIL PROTECTED] On Tue, June 12, 2007 13:33, Justin Mason wrote: Daniel J McDonald writes: So, you can't build the RPM as root. I just added all of the various groups to my user, set up a user build directory tree, compiled it under my username and it tested fine, at least to the point that it normally bombs aha, that's it alright. could you open a bug to get that fixed? One argument is that you shouldn't be building RPMs as root since you don't know if the spec file is perfect and won't break outside of the semi-jail that RPM sets up. I've had at least one such RPM build when working with qmail, and it went and buggered up the build server by actually installing itself outside of the jail when building. Doing it as a regular reason will limit that kind of activity. Very interesting, but I ran into this problem on a Solaris system and I wasn't trying to build an RPM. I was just trying to build SA from source with the usual perl Makefile.PL make make test (this step gave errors when run as root) Does the same logic apply when RPMs are not involved?
RE: sa-compile and SARE
From: Doc Schneider [mailto:[EMAIL PROTECTED] Rosenbaum, Larry M. wrote: Does this fix the performance problems I was having, or does it just fix the UTF errors showing in the logs with Perl 5.8.8 ? You might try it and see if it helps with the performance. Since it does fix the UTF-8 issue it won't be doing as much logging and grinding. let us know if it helps. Performance is still bad unless I use bytes.
RE: sa-compile and SARE
-Original Message- From: Loren Wilton [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 22, 2007 1:36 PM To: users@spamassassin.apache.org Subject: Re: sa-compile and SARE So, can we now remove the use bytes; line? Should be able to. If this doesn't work we have some more files that need to be updated, and would like to know what they might be. Does this fix the performance problems I was having, or does it just fix the UTF errors showing in the logs with Perl 5.8.8 ?
RE: Spamassassin 3.2.0
From: Christopher X. Candreva [mailto:[EMAIL PROTECTED] On Sat, 19 May 2007, Ming Hou wrote: My issue Mail::DKIM and Mail::DomainKeys are required Crypt::OpenSSL::Random and Crypt::OpenSSL::RSA. But, I could not get Crypt::OpenSSL::Random to be built successfully because it always complained the following messages: Note (probably harmless): No library found for -lssl Note (probably harmless): No library found for -lcrypto Edit Makefile.PL and add -lssl and -lcrypto to the LIBS line. I also had to add an include line for the SSL headers. Then rerun Makefile.PL Or you can try the following: perl Makefile.PL INC='-I/usr/local/ssl/include' LIBS='-L/usr/local/ssl/lib -lssl -lcrypto' (all on one line)
RE: possible conflict in a SA setup between .pre and local.cf issue
From: Abba Communications [mailto:[EMAIL PROTECTED] I think I just noticed a conflict. Not sure if I made the mistake or not, yet I probably did. In my init.pre I have loadplugin Mail::SpamAssassin::Plugin::URIDNSBL uncommented and in my local.cf I have purposely set the config below. skip_rbl_checks 1 do these settings conflict? I believe they do, correct? No. skip_rbl_checks 1 does not turn off the URI DNSBL checks.
spamc -H favors one host (v3.2.0)
We have just upgraded from v3.1.8 to v3.2.0. We invoke spamc as follows: spamc -H -E -t 180 -s 20 -d spamd.ornl.gov # nslookup spamd.ornl.gov Name:spamd.ornl.gov Addresses: 160.91.4.92, 160.91.1.172 This used to connect equally to the two hosts, but now it makes almost all the connections to one host (.92). Has the host randomization logic changed? Is it broken?
RE: Poor performance with v3.2.0
Took 10 mins on my 2.8gh 512mb ram, with a bunch of sares rules. You using .12.0 of re2c? Yes. I think most of the time is spent in the rule extraction steps and the gcc compiles, and not in the re2c steps. (gcc is v3.4.6) Yes, you are right, after use warnings;. I ran SA3.2 on my site with use bytes; added, no problem so far. But it seems SA developers did not mention this, they might have their reasons (break normalize_charset for one reason). Yes, exactly -- breaking one of the major 3.2.0 features is not a good thing. :( Where can I find documentation on what normalize_charset does?
RE: Poor performance with v3.2.0
From: Loren Wilton [mailto:[EMAIL PROTECTED] Subject: Re: Poor performance with v3.2.0 It would be interesting on some system experiencing this slowdown to put 'use bytes' back into SA and see what happens with the performance. This wouldn't be any sort of a solution, but it would be an interesting data point. Interesting indeed. I added use bytes and performance is much improved. It's approximately back to where it was with v3.1.8. So what does this all mean? In case it matters, here's the output of perl -V: Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=solaris, osvers=2.9, archname=sun4-solaris uname='sunos email 5.9 generic_118558-39 sun4u sparc sunw,sun-fire-v210 ' config_args='-Dcc=gcc -d' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O', cppflags='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include' ccversion='', gccversion='3.4.6', gccosandvers='solaris2.9' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags =' -L/usr/local/lib ' libpth=/usr/local/lib /usr/lib /usr/ccs/lib libs=-lsocket -lnsl -ldl -lm -lc perllibs=-lsocket -lnsl -ldl -lm -lc libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' ' cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: PERL_MALLOC_WRAP USE_LARGE_FILES USE_PERLIO Built under solaris Compiled at May 4 2007 15:28:54 @INC: /usr/local/lib/perl5/5.8.8/sun4-solaris /usr/local/lib/perl5/5.8.8 /usr/local/lib/perl5/site_perl/5.8.8/sun4-solaris /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl/5.8.7/sun4-solaris /usr/local/lib/perl5/site_perl/5.8.7 /usr/local/lib/perl5/site_perl/5.8.5/sun4-solaris /usr/local/lib/perl5/site_perl/5.8.5 /usr/local/lib/perl5/site_perl .
RE: Poor performance with v3.2.0
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Did you have a massive volume of Malformed UTF-8 warning messages in the syslog output? No, I upgraded Perl to v5.8.8, which got rid of the warning messages but there was still a performance problem. Adding use bytes seems to have fixed the performance problem. I have a theory that this would indeed cause major slowdowns, since every warning message has to be transmitted via UDP to the syslogd daemon, who then writes it synchronously to disk. That is a pretty slow operation, and causes I/O.
RE: Poor performance with v3.2.0
-Original Message- From: Doc Schneider [mailto:[EMAIL PROTECTED] If he is getting the UTF-8 error, this would indeed be odd, since he is using perl-5.8.8 which supposedly handles those regexps which causes the error. What SARE rules are you running Larry? /usr/local/spamassassin/70_sare_adult.cf /usr/local/spamassassin/70_sare_bayes_poison_nxm.cf /usr/local/spamassassin/70_sare_evilnum0.cf /usr/local/spamassassin/70_sare_evilnum1.cf /usr/local/spamassassin/70_sare_evilnum2.cf /usr/local/spamassassin/70_sare_genlsubj0.cf /usr/local/spamassassin/70_sare_genlsubj1.cf /usr/local/spamassassin/70_sare_header0.cf /usr/local/spamassassin/70_sare_header1.cf /usr/local/spamassassin/70_sare_html0.cf /usr/local/spamassassin/70_sare_html1.cf /usr/local/spamassassin/70_sare_obfu.cf /usr/local/spamassassin/70_sare_oem.cf /usr/local/spamassassin/70_sare_random.cf /usr/local/spamassassin/70_sare_specific.cf /usr/local/spamassassin/70_sare_spoof.cf /usr/local/spamassassin/70_sare_stocks.cf /usr/local/spamassassin/70_sare_unsub.cf /usr/local/spamassassin/70_sare_uri0.cf /usr/local/spamassassin/70_sare_uri1.cf /usr/local/spamassassin/70_sare_whitelist_rcvd.cf /usr/local/spamassassin/70_sare_whitelist_spf.cf /usr/local/spamassassin/70_zmi_german.cf /usr/local/spamassassin/72_sare_bml_post25x.cf /usr/local/spamassassin/72_sare_redirect_post3.0.0.cf /usr/local/spamassassin/99_sare_fraud_post25x.cf also clamav, Botnet, and FuzzyOcr, and some local rules.
RE: Poor performance with v3.2.0
Bayes auto expiries (taking to long and getting killed)? I think that's a 600 second timeout. We're not using auto-expiry. Bayes expiry is being done with a batch job. It would be interesting on some system experiencing this slowdown to put 'use bytes' back into SA and see what happens with the performance. This wouldn't be any sort of a solution, but it would be an interesting data point. We have Perl v5.8.8, which doesn't have the UTF8 bug. Would it still be worth trying this experiment? If so, where do I put use bytes? And, God Blues you Theo! sa-compile (we have it working on FREEBSD for the non-sares rules) brings that performance back to 3.18 levels with all the new rules. sa-compile took 3 hours to run. (System is a SunFire v210 with 2 processors and 2 GB ram.)
Poor performance with v3.2.0
Running SpamAssassin v3.2.0 on Solaris 9, perl v5.8.8. I am getting really poor performance with v3.2.0 compared with v3.1.8. Average scan time per message is doubled (or worse). I'm also seeing messages like this in the log file (although not all the time): May 7 17:10:20 localhost spamd[19457]: plugin: eval failed: child processing timeout at /usr/local/bin/spamd line 1259, GEN390 line 377. May 7 17:11:24 localhost spamd[19447]: plugin: eval failed: child processing timeout at /usr/local/bin/spamd line 1259, GEN464 line 1097. Is there any way to tell what the timeout was waiting for, or why performance has gotten worse? I haven't changed the rules. Thanks, Larry
RE: Justa a small nag from 3.2.0...
From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Hi, list, I have a cron job running in order to learn from each user's HAM and SPAM subdirs. Whenever it runs, it complains like this: Subroutine FuzzyOcr::O_NONBLOCK redefined at /usr/share/perl/5.8/Exporter.pm line 65. at /usr/lib/perl/5.8/POSIX.pm line 19 Any hints on how to avoid this nag? Edit /usr/local/spamassassin/FuzzyOcr.pm and make the following change: use POSIX; --- use POSIX qw(SIGTERM);
RE: ANNOUNCE: Apache SpamAssassin 3.2.0-rc3 PRERELEASE available!
From: Justin Mason [mailto:[EMAIL PROTECTED] Subject: ANNOUNCE: Apache SpamAssassin 3.2.0-rc3 PRERELEASE available! Apache SpamAssassin 3.2.0-rc3 is now available! This is a *PRERELEASE*, not the full release of 3.2.0. I have found the following (non-fatal) issues with SA v3.2.0-rc3: 1) Now that spamc is linked with zlib, it doesn't link correctly on Solaris 8 where libz.so is in /usr/local/lib. Attempting to run spamc gives the following error: ld.so.1: ../spamc/spamc: fatal: libz.so: open failed: No such file or directory A workaround is to use the following make command # LD_RUN_PATH=/usr/local/lib make but it would be better if the configuration code supplied the right link options. 2) I tried to compile the rules to see how much speed increase I could get, but it didn't work. After running sa-compile and uncommenting the Rule2XSBody plugin, I got the following error: # spamassassin --lint ld.so.1: /usr/local/bin/spamassassin: fatal: relocation error: file /var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/CompiledR egexps/body_0/body_0.so: symbol Mail_SpamAssassin_CompiledRegexps_body_0_scan17: referenced symbol not found Killed This is with rc2 on Solaris 8. I haven't tried it with rc3 yet because it takes about 3 hours to compile on our development system. 3) I noticed the following rules were removed: SUBJECT_DRUG_GAP_P HTML_NONELEMENT_50_60 HTML_NONELEMENT_70_80 HTML_NONELEMENT_90_100 HTML_FONT_INVISIBLE HTML_BACKHAIR_8 RISK_FREE PORN_URL_SEX I was wondering why some of the HTML_NONELEMENT_nn_nn rules were removed and some were not. * bug 5236: Support Mail::SPF replacement for Mail::SPF::Query. Makefile.PL complains if Mail::SPF::Query is missing, even though it knows that Mail::SPF is preferred.
RE: ANNOUNCE: Apache SpamAssassin 3.2.0-rc3 PRERELEASE available! - UTF error
Testing v3.2.0-rc3 on Solaris 8 I'm finding zillions of the following errors in the log file: Apr 24 10:13:27 emaildev.ornl.gov spamd[12593]: Malformed UTF-8 character (unexpected non-continuation by te 0x00, immediately after start byte 0xce) in pattern match (m//) at /etc/mail/spamassassin/70_sare_obfu .cf, rule __SARE_OBFU_VISIT1, line 1, GEN174 line 390. Apr 24 10:13:27 emaildev.ornl.gov last message repeated 1 time Apr 24 10:13:28 emaildev.ornl.gov spamd[12593]: Malformed UTF-8 character (unexpected non-continuation by te 0x00, immediately after start byte 0xc4) in pattern match (m//) at /etc/mail/spamassassin/70_sare_obfu .cf, rule __SARE_OBFU_VISIT1, line 1, GEN174 line 390. Apr 24 10:13:28 emaildev.ornl.gov last message repeated 3 times ... Apr 24 10:39:13 emaildev.ornl.gov spamd[12592]: Malformed UTF-8 character (unexpected non-continuation byte 0x00, immediately after start byte 0xce) in pattern match (m//) at /etc/mail/spamassassin/72_sare_bml_post25x.cf, rule SARE_OBFUAUCTION, line 1, GEN192 line 832. Apr 24 10:39:13 emaildev.ornl.gov last message repeated 1 time Apr 24 10:39:13 emaildev.ornl.gov spamd[12592]: Malformed UTF-8 character (unexpected non-continuation byte 0x00, immediately after start byte 0xd0) in pattern match (m//) at /etc/mail/spamassassin/72_sare_bml_post25x.cf, rule SARE_OBFUAUCTION, line 1, GEN192 line 832. I don't see this error in v3.1.8. What is causing this error, and how do I fix it? Thanks, Larry
RE: ANNOUNCE: Apache SpamAssassin 3.2.0-rc3 PRERELEASE available!
2) I tried to compile the rules to see how much speed increase I could get, but it didn't work. After running sa-compile and uncommenting the Rule2XSBody plugin, I got the following error: # spamassassin --lint ld.so.1: /usr/local/bin/spamassassin: fatal: relocation error: file /var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/CompiledR egexps/body_0/body_0.so: symbol Mail_SpamAssassin_CompiledRegexps_body_0_scan17: referenced symbol not found Killed This is with rc2 on Solaris 8. I haven't tried it with rc3 yet because it takes about 3 hours to compile on our development system. Again, sounds like some kind of -R related issue. However, this *should* work since this module is simply built using the ExtUtils::MakeMaker support for building XS modules. Are you able to build XS modules (e.g. HTML::Parser) from source ok on that system? I have no trouble compiling other XS Perl modules. It doesn't seem to be a problem finding the dynamic libraries: # ldd /var/lib/spamassassin/compiled/3.002000/auto/Mail/ SpamAssassin/CompiledRegexps/body_0/body_0.so libc.so.1 = /usr/lib/libc.so.1 libdl.so.1 =/usr/lib/libdl.so.1 /usr/platform/SUNW,Ultra-5_10/lib/libc_psr.so.1 It's more a matter of finding the symbols: # ldd -r /var/lib/spamassassin/compiled/3.002000/auto/Mail/ SpamAssassin/CompiledRegexps/body_0/body_0.so libc.so.1 = /usr/lib/libc.so.1 libdl.so.1 =/usr/lib/libdl.so.1 /usr/platform/SUNW,Ultra-5_10/lib/libc_psr.so.1 symbol not found: main (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: PL_markstack_ptr (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: PL_stack_sp (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: PL_sv_yes (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: PL_stack_base (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_newSVpvn_share (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_av_push (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_sv_2pvutf8 (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_newAV (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_sv_2mortal (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Mail_SpamAssassin_CompiledRegexps_body_0_scan17 (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_croak (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_newRV (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_form (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_get_sv (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_sv_2pv_flags (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so) symbol not found: Perl_newXS (/var/lib/spamassassin/compiled/3.002000/auto/Mail/SpamAssassin/Compiled Regexps/body_0/body_0.so)
Bug 5420 - RE: 3.2.0-rc2?
Any chance of fixing Bug #5420 before the final release? http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5420 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 17, 2007 12:05 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; users@SpamAssassin.apache.org Subject: Re: 3.2.0-rc2? How's this working out? Any good/bad reports? --j. jm writes: Apache SpamAssassin 3.2.0-rc2 is now available! This is a *PRERELEASE*, not the full release of 3.2.0. Downloads are available from: http://people.apache.org/~jm/devel/ Downloading --- http://people.apache.org/~jm/devel/Mail-SpamAssassin-3.2.0-rc2.tar.bz2 http://people.apache.org/~jm/devel/Mail-SpamAssassin-3.2.0-rc2.tar.gz http://people.apache.org/~jm/devel/Mail-SpamAssassin-3.2.0-rc2.zip md5sum of archive files: fcc0242642826191a58d45bf5777e3b2 Mail-SpamAssassin-3.2.0-rc2.tar.bz2 c9cc09334b04bc76f08e22c1aee6d07e Mail-SpamAssassin-3.2.0-rc2.tar.gz 08f7a46d124e7abe50493ec4ddad7609 Mail-SpamAssassin-3.2.0-rc2.zip sha1sum of archive files: c592640242ef8f7b93a99235f836a0d33cadfa10 Mail-SpamAssassin-3.2.0- rc2.tar.bz2 d11350d3d418f75682b06098aeb438696faa688d Mail-SpamAssassin-3.2.0- rc2.tar.gz 43dd35eef6482cbd3176472e1ef0055eb4694dfe Mail-SpamAssassin-3.2.0- rc2.zip The release files also have a .asc accompanying them. The file serves as an external GPG signature for the given release file. The signing key is available via the wwwkeys.pgp.net key server, as well as http://spamassassin.apache.org/released/GPG-SIGNING-KEY The key information is: pub 1024D/265FA05B 2003-06-09 SpamAssassin Signing Key [EMAIL PROTECTED] Key fingerprint = 26C9 00A4 6DD4 0CD5 AD24 F6D7 DEE0 1987 265F A05B See the INSTALL and UPGRADE files in the distribution for important installation notes. Summary of major changes since 3.1.8 Changes to the core code: * new behavior for trusted_networks/internal_networks: the 127.* network is now always considered trusted and internal, regardless of configuration. * bug 3109: short-circuiting of 'definite ham' or 'definite spam' messages based on individual short-circuit rules using the 'shortcircuit' setting, by Dallas Engelken dallase /at/ uribl.com. * bug 5305: implement 'msa_networks', for ISPs to specify their Mail Submission Agents, and extend network trust accordingly. * bug 4636: Add support for charset normalization, so rules can be written in UTF-8 to match text in other charsets. * sa-compile: compilation of SpamAssassin rules into a fast parallel- matching DFA, implemented in native code. * tflags multiple: allow writing of rules that count multiple hits in a single message. * bug 4363: if a message uses CRLF for line endings, we should use it as well, otherwise stay with LF as usual; important for Windows users. * bug 4515: content preview was omitting first paragraph when no Subject: header was present. * The third-party modules used by sa-update are now required by the SpamAssassin package, instead of being optional. * Bug 5165: 'sa-update --checkonly' added to check for updates without applying them; thanks to anomie /at/ users.sourceforge.net * Bugs 4606, 4609: Adjust MIME parsing limits for nested multipart/* and message/rfc822 MIME parts. * bug 5295: add 'whitelist_auth', to whitelist addresses that send mail using sender-authorization systems like SPF, Domain Keys, and DKIM * Removed dependency on Text::Wrap CPAN module. * Received header parsing updates/fixes/additions. Spamc / spamd: * bug 4603: Mail::SpamAssassin::Spamd::Apache2 -- mod_perl2 module, implementing spamd as a mod_perl module, contributed as a Google Summer of Code project by Radoslaw Zielinski. * bug 3991: spamd can now listen on UNIX domain, TCP, and SSL sockets simultaneously. Command-line semantics extended slightly, although fully backwards compatibly; add the --ssl-port switch to allow TCP and SSL listening at the same time. * bug 3466: do Bayes expiration, if required, after results have been passed back to the client from spamd; this helps avoid client timeouts. * more complete IPv6 support. * spamc: Add '-K' switch, to ping spamd. * spamc: add '-z' switch, which compresses mails to be scanned using zlib compression; very useful for long-distance use of spamc over the internet. * bug 5296: spamc '--headers' switch, which scans messages and transmits back just rewritten headers. This is more bandwidth-efficient than the normal mode of scanning, but only works for 'report_safe 0'. * Bump spamd's protocol version to 1.4, to reflect new HEADERS verb used for '--headers'. Mail::SpamAssassin modules and API: * bug 4589: allow M::SA::Message to use IO::File objects to read in message (same as GLOB).
RE: Simple hosting user: can I install my own SA?
From: Andy Figueroa [mailto:[EMAIL PROTECTED] Woo Hoo - that's a good trick and good intel. Thinking ... Won't it still try to use /usr/share/spamassassin for rules and /var/lib/spamassassin for updates and /etc/mail/spamasassin for configuration and so forth? Can't do that without root access. I'm thinking out-loud here ... Thanks. You can get around that by building it with something like perl Makefile.PL PREFIX=/path/to/yourhomedir
RE: use or not use awl
From: Dave Koontz [mailto:[EMAIL PROTECTED] Not neccessarily. Put your awl on a sql database and add a timestamp column to the awl table, which gets automagically a new timestamp by the dbms each time a record is updated. The timestamp column type in Mysql is such a type. show create table awl: CREATE TABLE `awl` ( `username` varchar(100) collate latin1_german1_ci NOT NULL default '', `email` varchar(200) collate latin1_german1_ci NOT NULL default '', `ip` varchar(10) collate latin1_german1_ci NOT NULL default '', `count` int(11) default '0', `totscore` float default '0', `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`username`,`email`,`ip`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_german1_ci Then you can easily expire by date with a cron job, for example expire all that was not updated for the last 30 days: delete from awl where timestamp now() - interval 30 day Can you tell me how to do something similar for the bayes_seen table and MySQL? Thanks, Larry
BODY rule fails with double-spaced text
The doc for BODY rules says All HTML tags and line breaks will be
removed before matching. I was also told on this list that multiple
whitespace was compressed to single space characters. So if I have text
like this:
xyzzy
abcde
and the following rules:
bodyT_LMRTESTB1 /xyzzy abcde/
bodyT_LMRTESTB2 /xyzzy\s{1,4}abcde/
then both rules will match. However, if the text is double-spaced like
this:
xyzzy
abcde
then *neither* rule will match, even though I would have expected them
both to still match. Is this a designed feature or a bug?
RE: spam
Has anybody come up with a rule for these yet? I tried the following:
body ORNL_B0RKEN1 /^\d{3,5}\n{1,3}$/s
describe ORNL_B0RKEN1 B0rken spamware, message just contains a short
number
scoreORNL_B0RKEN1 1
This matches the spam message, but it also matches messages where the
number is followed by a blank line and more text, which is a false
positive. If I replace body with full, then it doesn't match the
spam message. I have also tried the following variations:
- using /s, /m, or neither switch
- using the ^ and $ anchors or the \A and \Z anchors
- using \n, \s, or neither (i.e. the pattern /^\d{3,5}$/)
In all cases I got the same results. What am I missing?
Thanks, Larry
-Original Message-
From: Nigel Frankcom [mailto:[EMAIL PROTECTED]
Posted At: Monday, December 04, 2006 8:02 PM
Posted To: sa-users
Conversation: spam
Subject: Re: spam
On Mon, 04 Dec 2006 16:35:33 -0800, Evan Platt
[EMAIL PROTECTED] wrote:
At 04:24 PM 12/4/2006, you wrote:
On Mon, 4 Dec 2006 16:11:28 -0800 (PST), san [EMAIL PROTECTED]
wrote:
Hi,
Am recieving a spam mails which is just having number on the body
just
like
1265 or 2196...
any thoughts how to stop this kind of spam..
thanks
san
Ditto
How in the hell does one write a rule for this sh*?
Maybe a rule if the message body is less than X characters?
I mean unless you expect lots of legitimate mail that says
Hello.
Good point; thanks.
Though I think I'll do one that picks only numerals. That said I'm
pretty sure there's a sare rule that covers this sort of thing
though I could easily be wrong; it wouldn't be the 1st time :-D
KR
Nigel
RE: new Botnet plugin version soon
From: Dennis Davis [mailto:[EMAIL PROTECTED] ... Question 2: someone asked why my module is Botnet instead of Mail::SpamAssassin::Plugin::Botnet. The answer is: when I first started this (and this is/was my first SA Plugin authoring attempt), I tried that and it didn't work. If someone wants to look at it, and figure out how to make that work I prefer to have all the SpamAssassin plugins grouped together where the default install puts them. This is in the directory: /usr/local/libdata/perl5/site_perl/Mail/SpamAssassin/Plugin/ I would prefer to use the xxx/site_perl/Mail/SpamAssassin/Plugin for plugins that are packaged with SpamAssassin, and that any added-in plugins that I install separately go into /etc/mail/spamassassin. I also see no advantage to moving the loadplugin statement into the init.pre file unless there are rules in other .cf files that depend on the plugin. In other words, it's fine the way it is.
RE: sa-update and 'doesnotexist'
From: Ben Lentz [mailto:[EMAIL PROTECTED] Greetings, List! I just upgraded from sa 3.1.3 to sa 3.1.6 and am having some weird problems with sa-update that I've never seen before. It would seem that my sys rules/default rules directory (/usr/share/spamassassin) is not being loaded by sa-update's internal lint test, but that my site rules directory (/etc/mail/spamassassin) is. And because by site rules directory has references to things in my sys rules directory, sa-update refuses to run because it thinks my rules are borked. Really, it's just not reading what's there. This is a known bug in 3.1.6. It is fixed in 3.1.7, which was just released today.
RE: spamd Sys-Hostname-Long-1.4 error
In Long.pm, go to line 91 and change this my $tmp = `hostname` . '.' . `domainname`; to this my $tmph = `hostname`; my $tmpd = `domainname`; my $tmp = $tmph.$tmpd; -Original Message- From: John Goubeaux [mailto:[EMAIL PROTECTED] Posted At: Friday, October 06, 2006 12:54 PM Posted To: sa-users Conversation: spamd Sys-Hostname-Long-1.4 error Subject: spamd Sys-Hostname-Long-1.4 error I am getting the following error generated from spamd, apparently due to the fact that the sys::hostname::long module behaves differently on different OSs in this case Solaris 8. While i realize this is not specific to spmassassin and a perl /OS issue was wondering if anyone running SA on Solaris has seen and been able to eliminate this error? SpamAssassin version 3.1.5 running on Perl version 5.8.5 Oct 5 09:28:21 kady spamd[870]: Insecure dependency in `` while running with -T switch at /usr/local/perl-5.8.5/lib/site_perl/5.8.5/Sys/Hostname/Long.pm line 91, GEN26746 line 77. any clues are appreciated ! -john -- John Goubeaux Systems Administrator Gevirtz Graduate School of Education UC Santa Barbara Phelps Hall 3534 805 893-8190
RE: 2 different scores?
From: Evan Platt [mailto:[EMAIL PROTECTED] ... I changed my procmailrc to; :0fw: spamassassin.lock | spamd -L -c -s 512000 Shouldn't that be spamc?
RE: sa-learn and Caught spams
From: Mike Woods [mailto:[EMAIL PROTECTED] The internet is a great place for raising more questions than it answers :D Given all the opinions I think I will move the caught spam's into the learning cycle however i'm also going to make sure that each spam is only ever fed through the system once, this wont be a problem since I already make use of their checksums to avoid duplicating files and I had intended to use it to remove old spam anyway. If you look at the X-Spam-Status header, it will tell you if the message was already autolearned: X-Spam-Status: Yes, score=19.5 required=5.0 tests=... (list of tests)... autolearn=spam version=3.1.5
RE: URIBL false matches
From: Mark G. Thomas [mailto:[EMAIL PROTECTED] Hi, I have a problem with incorrect URIBL hits on incoming forwarded messages that have been mangled by Lotus Notes. I have a customer with the domain name Yimaging.com. (Not really Y). ng.com is on the URIBL blacklist. I think for awhile it has been removed, but it's there again now. ... Is there some easy way I can exclude just the one domain name ng.com from being looked up at all, but otherwise still use the URIBL? uridnsbl_skip_domain ng.com
