Re: Is it possible to increase the Bayesian Score ?
Yes. Do it with due consideration. (I raised Bayes_99 to 5.0, for
example. At this location that is as sure an indication of spam as
I have present on the system. A score lower than Bayes_90, as out
of the box, seemed stupid. Bayes_99 fires on 0.08% of ham and 84.7%
of spam. Whitelist rules handle the chronically spammy Bayes rules
for the once a week or two mailings that look like spam but are not.
I also have a few good ham indicators with modestly low scores (they
are only good not great) which bring scores downwards so that missed
scores are very rare here. Quite often Bayes_99 is the only rule that
triggers, too.)
{^_^}
- Original Message -
From: Michael Ben-Nes [EMAIL PROTECTED]
Hello
Is it possible to increase the BAYES_XX score ?
Thanks
Re: Newbie looking for info...
Aaron Boyles wrote on Thu, 22 Dec 2005 09:34:09 -0500: Unfortunately, this would result in a third step in the SMTP process. Currently, the SMTP filter I run allows us to use our choice of virus scanner to check for viruses, monitor real-time traffic, and even chat back to a would-be hacker if they're screwing with the system manually, as well as back-up E-Mails for however long we need to, as well as all traffic that transpires in case we have to go back to a previous attack log for prosecution purposes. You don't need prosecution if no one can abuse you. MailScanner will provide all the above for you. Adding a spam filter at this point would just be the smart thing to do. The smart thing is to block as much unwanted traffic as you can before accepting it. Reduces spam influx by about 80% or more. Unfortunately, if we were to make a third server, we would then have this app receiving incoming SMTP traffic, doing its thing, then forwarding that on to the Spam Assassin server, That server is not necessary at all. then having THAT forward it on to the Exchange server. Again, keep in mind that I'm trying to keep this as ridiculously simple as possible for the people that'll have to actually implement it in my absence. Placing just one Linux box with MailScanner, SA and several virusscanners of your choice before the Exchange box *is* that simple. And if you use greylisting there won't be much spam left for SA anyway. You want it simple, I'm all for that. I think that your special Exchange sink solution is far more complicated/complex than this. It might be a genious piece of software, but it's not flexible since you produced it for a specific purpose and it's bound to Exchange which limits your options, anyway. Just administering the Exchange is more complex than administering the whole MailScanner box. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: I'm afraid I might have to report this list as a spam source
You are all speculating. No one knows why or if the original poster can't unsubscribe. And, frankly, it was the first posting of this kind I've ever seen. It's not a problem at all. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
SARE_URI_EQUALS false positives
I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is therefore skewing the scoring of some mail quite badly. The weird thing is that the uris that spamassassin is complaining about aren't uris at all. The mail in question is auto-created reports of cvs diffs, so it's slightly unusual. I've tried to condense the debug information. Here it is: This is some of the output from spamassassin -D false_positive [16733] dbg: uri: parsed uri found, updated.by=Mis [16733] dbg: uri: cleaned parsed uri, http://updated.by=Mis [16733] dbg: uri: cleaned parsed uri, updated.by=Mis [16733] dbg: uri: parsed uri found, http://updated.by=Mis [16733] dbg: uri: cleaned parsed uri, http://updated.by=Mis [16733] dbg: uri: parsed uri found, updated.by=Updated [16733] dbg: uri: cleaned parsed uri, updated.by=Updated [16733] dbg: uri: cleaned parsed uri, http://updated.by=Updated [16733] dbg: uri: parsed uri found, http://updated.by=Updated [16733] dbg: uri: cleaned parsed uri, http://updated.by=Updated These parsed uris are not links in the e-mail. They are just text. I've had a bit of a look at the regexps that spamassassin uses to work out what is a uri, and it seems that updated.by=Updated is treated as a uri because .by is a valid tld and spamassassin looks for schemeless uris, then prepends http:// for the tests. I'm running spamassassin 3.1.0 on perl 5.8.2. Does anyone have any suggestions, apart from simply reducing the score for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to guarantee that only real uris are parsed as such? Chris
Question re perl security fixes
Greetings all; Because FC2 seems to have fallen off the Fedora-Legacy radar, I dl'd the perl5.8.5 stuff for FC3, figuring if there was a dependency clash, rpm would fuss. It didn't, but I'm noting that when SA is searching for missing pkgs, the log does not indicate that it is searching in the new /usr/lib/perl5/site_perl/5.8.5 path, only in 5.8.0, 5.8.1, 5.8.2, 5.8.3. Is this something that would be fixed with a simple spamd restart or ?? -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Re: SARE_URI_EQUALS false positives
From: Chris Lear [EMAIL PROTECTED]
I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is
therefore skewing the scoring of some mail quite badly.
The weird thing is that the uris that spamassassin is complaining about
aren't uris at all. The mail in question is auto-created reports of cvs
diffs, so it's slightly unusual.
I've tried to condense the debug information. Here it is:
This is some of the output from spamassassin -D false_positive
[16733] dbg: uri: parsed uri found, updated.by=Mis
[16733] dbg: uri: cleaned parsed uri, http://updated.by=Mis
[16733] dbg: uri: cleaned parsed uri, updated.by=Mis
[16733] dbg: uri: parsed uri found, http://updated.by=Mis
[16733] dbg: uri: cleaned parsed uri, http://updated.by=Mis
[16733] dbg: uri: parsed uri found, updated.by=Updated
[16733] dbg: uri: cleaned parsed uri, updated.by=Updated
[16733] dbg: uri: cleaned parsed uri, http://updated.by=Updated
[16733] dbg: uri: parsed uri found, http://updated.by=Updated
[16733] dbg: uri: cleaned parsed uri, http://updated.by=Updated
These parsed uris are not links in the e-mail. They are just text.
I've had a bit of a look at the regexps that spamassassin uses to work
out what is a uri, and it seems that updated.by=Updated is treated as
a uri because .by is a valid tld and spamassassin looks for schemeless
uris, then prepends http:// for the tests.
I'm running spamassassin 3.1.0 on perl 5.8.2.
Does anyone have any suggestions, apart from simply reducing the score
for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to
guarantee that only real uris are parsed as such?
Before you drop the score precipitously check if there is some other
characteristic of the emails that trigger falsely which can be used to
apply a negative score. If there is such a characteristic then generate
the appropriate negative score. If not weigh how effective the rule is
for you. The version of sa-stats.pl that is on the SARE site helps
figure this out nicely.
That said it's close to a 50/50 rule that hits on very few messages
here so should have a low score. (It hit on 6 messages out of 75000.)
Cutting it out completely here seems like it would be effective TODAY.
That could change. At one time it was quite necessary. Spammer fads
change.)
{^_^}
Re: Question re perl security fixes
From: Gene Heskett [EMAIL PROTECTED]
Greetings all;
Because FC2 seems to have fallen off the Fedora-Legacy radar, I dl'd
the perl5.8.5 stuff for FC3, figuring if there was a dependency clash,
rpm would fuss.
It didn't, but I'm noting that when SA is searching for missing pkgs,
the log does not indicate that it is searching in the
new /usr/lib/perl5/site_perl/5.8.5 path, only in 5.8.0, 5.8.1, 5.8.2,
5.8.3.
Is this something that would be fixed with a simple spamd restart or ??
You can tell us easily enough. Try it. The cost of a simple
service spamassassin restart is remarkably small. (Or it SHOULD
be.)
{^_-}
Re: Question re perl security fixes
On Friday 23 December 2005 06:29, jdow wrote:
From: Gene Heskett [EMAIL PROTECTED]
Greetings all;
Because FC2 seems to have fallen off the Fedora-Legacy radar, I
dl'd the perl5.8.5 stuff for FC3, figuring if there was a
dependency clash, rpm would fuss.
It didn't, but I'm noting that when SA is searching for missing
pkgs, the log does not indicate that it is searching in the
new /usr/lib/perl5/site_perl/5.8.5 path, only in 5.8.0, 5.8.1,
5.8.2, 5.8.3.
Is this something that would be fixed with a simple spamd restart
or ??
You can tell us easily enough. Try it. The cost of a simple
service spamassassin restart is remarkably small. (Or it SHOULD
be.)
Looks like it did Joanne, I'm now seeing 5.8.5's in the log.
Thanks. Just trying to get this thing in shape for me to disappear for
a few days over the holidays.
{^_-}
--
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules. I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Re: SARE_URI_EQUALS false positives
* jdow wrote (23/12/05 11:26): From: Chris Lear [EMAIL PROTECTED] I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is therefore skewing the scoring of some mail quite badly. The weird thing is that the uris that spamassassin is complaining about aren't uris at all. The mail in question is auto-created reports of cvs diffs, so it's slightly unusual. [...] I've had a bit of a look at the regexps that spamassassin uses to work out what is a uri, and it seems that updated.by=Updated is treated as a uri because .by is a valid tld and spamassassin looks for schemeless uris, then prepends http:// for the tests. I'm running spamassassin 3.1.0 on perl 5.8.2. Does anyone have any suggestions, apart from simply reducing the score for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to guarantee that only real uris are parsed as such? Before you drop the score precipitously check if there is some other characteristic of the emails that trigger falsely which can be used to apply a negative score. If there is such a characteristic then generate the appropriate negative score. If not weigh how effective the rule is for you. The version of sa-stats.pl that is on the SARE site helps figure this out nicely. That said it's close to a 50/50 rule that hits on very few messages here so should have a low score. (It hit on 6 messages out of 75000.) Cutting it out completely here seems like it would be effective TODAY. That could change. At one time it was quite necessary. Spammer fads change.) I've reduced the score, and a quick check shows that that rule hits almost nothing anyway, so it's not a big problem. The bayes rules were keeping the false positives from doing much damage, anyway. But spamassassin uses uris for lots of things, and if it's commonly parsing (reasonably) normal text as uris, I would expect that to be a problem in more rules than just SARE_URI_EQUALS. Chris
Re: SARE_URI_EQUALS false positives
From: Chris Lear [EMAIL PROTECTED]
* jdow wrote (23/12/05 11:26):
From: Chris Lear [EMAIL PROTECTED]
I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is
therefore skewing the scoring of some mail quite badly.
The weird thing is that the uris that spamassassin is complaining about
aren't uris at all. The mail in question is auto-created reports of cvs
diffs, so it's slightly unusual.
[...]
I've had a bit of a look at the regexps that spamassassin uses to work
out what is a uri, and it seems that updated.by=Updated is treated as
a uri because .by is a valid tld and spamassassin looks for schemeless
uris, then prepends http:// for the tests.
I'm running spamassassin 3.1.0 on perl 5.8.2.
Does anyone have any suggestions, apart from simply reducing the score
for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to
guarantee that only real uris are parsed as such?
Before you drop the score precipitously check if there is some other
characteristic of the emails that trigger falsely which can be used to
apply a negative score. If there is such a characteristic then generate
the appropriate negative score. If not weigh how effective the rule is
for you. The version of sa-stats.pl that is on the SARE site helps
figure this out nicely.
That said it's close to a 50/50 rule that hits on very few messages
here so should have a low score. (It hit on 6 messages out of 75000.)
Cutting it out completely here seems like it would be effective TODAY.
That could change. At one time it was quite necessary. Spammer fads
change.)
I've reduced the score, and a quick check shows that that rule hits
almost nothing anyway, so it's not a big problem. The bayes rules were
keeping the false positives from doing much damage, anyway.
But spamassassin uses uris for lots of things, and if it's commonly
parsing (reasonably) normal text as uris, I would expect that to be a
problem in more rules than just SARE_URI_EQUALS.
That is a standalone rule.
And I do note that many of the SARE rules have severe problems in very
specific cases. There are some mailing lists that are not well filtered
for spam which have postings which trigger some of the too effective
to toss SARE rules. I've developed some massive meta rules to at least
partially get a handle on the problem. (A number of times XXX hit option
would be nice to have for this.)
{^_^}
Re: SARE_URI_EQUALS false positives
* jdow wrote (23/12/05 12:06): From: Chris Lear [EMAIL PROTECTED] * jdow wrote (23/12/05 11:26): From: Chris Lear [EMAIL PROTECTED] I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is therefore skewing the scoring of some mail quite badly. The weird thing is that the uris that spamassassin is complaining about aren't uris at all. The mail in question is auto-created reports of cvs diffs, so it's slightly unusual. [...] I've had a bit of a look at the regexps that spamassassin uses to work out what is a uri, and it seems that updated.by=Updated is treated as a uri because .by is a valid tld and spamassassin looks for schemeless uris, then prepends http:// for the tests. I'm running spamassassin 3.1.0 on perl 5.8.2. Does anyone have any suggestions, apart from simply reducing the score for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to guarantee that only real uris are parsed as such? Before you drop the score precipitously check if there is some other characteristic of the emails that trigger falsely which can be used to apply a negative score. If there is such a characteristic then generate the appropriate negative score. If not weigh how effective the rule is for you. The version of sa-stats.pl that is on the SARE site helps figure this out nicely. That said it's close to a 50/50 rule that hits on very few messages here so should have a low score. (It hit on 6 messages out of 75000.) Cutting it out completely here seems like it would be effective TODAY. That could change. At one time it was quite necessary. Spammer fads change.) I've reduced the score, and a quick check shows that that rule hits almost nothing anyway, so it's not a big problem. The bayes rules were keeping the false positives from doing much damage, anyway. But spamassassin uses uris for lots of things, and if it's commonly parsing (reasonably) normal text as uris, I would expect that to be a problem in more rules than just SARE_URI_EQUALS. That is a standalone rule. And I do note that many of the SARE rules have severe problems in very specific cases. There are some mailing lists that are not well filtered for spam which have postings which trigger some of the too effective to toss SARE rules. I've developed some massive meta rules to at least partially get a handle on the problem. (A number of times XXX hit option would be nice to have for this.) Sorry to go on, but I wonder whether you've missed by point. The SARE_URI_EQUALS rule is working fine. It just looks in the uris that spamassassin gives it, and complains when they contain =. The problem is that spamassassin is treating things that aren't uris as uris. So SARE_URI_EQUALS is working on dud data. In this specific case, the e-mail contains the text updated.by=Updated. This is not a uri, and nor should it be treated as one. But spamassassin thinks it is (becasue .by is a valid tld), so, as far as I can tell, *all* uri rules will check it. It so happens that SARE_URI_EQUALS hits in this case, but other uri rules are vulnerable to false positives if the uri parsing is wrong, aren't they? Chris
Re: SARE_URI_EQUALS false positives
updated.by - check http://www.tld.by/cgi-bin/registry.cgi You'll see that update.by is a registered domain! Therefore updated.by is indeed a URI. QED Paul Shupak [EMAIL PROTECTED]
Re: I'm afraid I might have to report this list as a spam source
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kai Schaetzl wrote: You are all speculating. No one knows why or if the original poster can't unsubscribe. I'll agree with that, to a point. And, frankly, it was the first posting of this kind I've ever seen. It's not a problem at all. I'll disagree with you here, I have had to contact the list-owner to get a dynamic address unsubscribed because when I tried the normal channels everything got bounced. Maybe this guy is just the first to complain out loud? Anyway, I'll second (third?) Jim Nasby's comments that: It's surprising to me that the SA lists aren't just run through SA. Spam making it past that is a good indication of where SA could be improved afterall. C. - -- Craig McLeanhttp://fukka.co.uk [EMAIL PROTECTED] Where the fun never starts Powered by FreeBSD, and GIN! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDrB+nMDDagS2VwJ4RAr3EAJ9cvML0MGnq6cYMHYn+TFETxWREowCfUCRL mmY3RsZCaMJVWmog7WPMot8= =Xjch -END PGP SIGNATURE-
RE: I'm afraid I might have to report this list as a spam source
-Original Message- From: Craig McLean [mailto:[EMAIL PROTECTED] Sent: 23 December 2005 16:03 To: users@spamassassin.apache.org Subject: Re: I'm afraid I might have to report this list as a spam source -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kai Schaetzl wrote: You are all speculating. No one knows why or if the original poster can't unsubscribe. I'll agree with that, to a point. And, frankly, it was the first posting of this kind I've ever seen. It's not a problem at all. I'll disagree with you here, I have had to contact the list-owner to get a dynamic address unsubscribed because when I tried the normal channels everything got bounced. Maybe this guy is just the first to complain out loud? Anyway, I'll second (third?) Jim Nasby's comments that: It's surprising to me that the SA lists aren't just run through SA. Spam making it past that is a good indication of where SA could be improved afterall. C. But of course when people drop examples etc it'll get blocked. I have the SA list whitelisted other wise it's FP all over the place. -- Martin Hepworth Snr Systems Administrator Solid State Logic Tel: +44 (0)1865 842300 ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote confirms that this email message has been swept for the presence of computer viruses and is believed to be clean. **
Re: I'm afraid I might have to report this list as a spam source
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Hepworth wrote: -Original Message- From: Craig McLean [mailto:[EMAIL PROTECTED] Sent: 23 December 2005 16:03 To: users@spamassassin.apache.org Subject: Re: I'm afraid I might have to report this list as a spam source -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kai Schaetzl wrote: You are all speculating. No one knows why or if the original poster can't unsubscribe. I'll agree with that, to a point. And, frankly, it was the first posting of this kind I've ever seen. It's not a problem at all. I'll disagree with you here, I have had to contact the list-owner to get a dynamic address unsubscribed because when I tried the normal channels everything got bounced. Maybe this guy is just the first to complain out loud? Anyway, I'll second (third?) Jim Nasby's comments that: It's surprising to me that the SA lists aren't just run through SA. Spam making it past that is a good indication of where SA could be improved afterall. C. But of course when people drop examples etc it'll get blocked. I have the SA list whitelisted other wise it's FP all over the place. As is the oft-repeated mantra of this list: SA doesn't block mail, it scores it. C. - -- Craig McLeanhttp://fukka.co.uk [EMAIL PROTECTED] Where the fun never starts Powered by FreeBSD, and GIN! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDrCVHMDDagS2VwJ4RAsDdAKD0rVshgzsCE1xzBlPpE9eSux7q+QCfbxJ3 XtA0kFwc1ZBBMaxNuEDAxXQ= =bu5v -END PGP SIGNATURE-
Re: Using Dig for RBL lookups.
CORRECTION OF MY PREVIOUS STATEMENTS: SBL doesn't appear to use a bitmask format as I suggested earlier. 127.0.0.6 would appear to be a valid answer for XBL only. It looks like they might use format 2 below, and SA's query implementation reflects this. Their choice of listing numbers suggests 1), but perhaps they were using that system and changed. Aaron Boyles wrote: Ahhh, so this isn't a standard format for all RBLs? Many, but not all. As I said before, each RBL has it's own formats, but *most* conform to the sendmail-style format. These are all NXDOMAIN for unlisted IPs, and 127.0.0.* for listed IPs. However, the lists generally start at 127.0.0.2, skipping the 127.0.0.1 loopback. So for most single-lists it's just a reverse-dotted-quad query for an A record and you get back NXDOMAIN or 127.0.0.2. Combined lists are a bit more complex. In general I've seen two common styles of response for combined lists. 1) using a bitmask like I thought SBL does, but it doesn't. In this style 2 = first list, 4 = second list, 6= first and second. I know multi.surbl.org's lists use this format, but that's a URIBL not a IP relay check. 2) returning multiple answers in a single response (this is valid), so the same lookup might return 127.0.0.2 and 127.0.0.3 to indicate listing in the first and second lists. combined.njabl.org and dnsbl.sorbs.net use this format. By the way, as a programmer who runs an IRC channel for a 3D Engine (TrueVision3D, Buy today!) I can say that as a rule, programmers tend to give the new guy a LOT of flack, especially when asking questions when they obviously know nothing about the subject (ie, me. Until yesterday, I didn't have the slightest clue how RBLs work.) Well, here, have some token flack :) You guys have been more than gracious, infinitely patient, and very accommodating. Most of my questions weren't even directly about SpamAssassin, but you guys have helped me through getting a very good feature added to my filter app. In appreciation, I'll be donating $50 to the ASF. Thank you very much for the hand-holding for the past two days! It's too bad more open source projects don't have such patient communities. Glad to be of help.
Re: sender-valid SMTP callbacks (Re: Does tuxorama.com sound fa miliar to anyone?)
François Conil said: snip it's kinda easy with postfix : http://www-personal.umich.edu/~malth/gaptuning/postfix/ I strongly advise to run it manually instead of via cron, since if the exchange server sh*t itself, the exchange_recipients list will contains nothing and all mails will be bounced :/ Hi François, I'm running my update via cron, but fearing the same issue I wrote a little shell script that calls the getadsmtp.pl script, after it creates the new list I do a line count on the new file and make sure it contains a minimum number (12000) of entries. If the file does not contain the required number of lines, then it shoots an email off to warn me that there were issues with the update and stops leaving the old postmapped file in place. If the required number of lines are in the file then it goes ahead and does a postmap on the raw file. Yeah, if it mangles the contents of the file somehow I will have problems, but I figure that most likely won't happen. This script has worked great for me and saved me once or twice in the last several months when the export failed. Plus I don't have to know whenever someone is hired or a new list, user, etc. has been created. Cheers, matt
Bayes Scores Skipped/Not Applied
Hi, I recently upgraded from spamassassin 3.0 to 3.1 and right away the amount of false negatives increased. I thought at first that it was because of the loss of dcc and razor (which surely is a factor), but on further investigation it appears that it is more related to the Bayes system. I have looked through the archives, but I am not finding what I need although it seems like this question has been asked more than once in different ways. I get almost no BAYES_xx or AWL scores on headers. I got one BAYES_00 tag betweem 18 December and 22 December and it is my understanding that I should see them on almost all emails. When I run spamassassin manually on a piece of spam, it applys a BAYES_xx score. Yet as far as I can tell the spamd daemon skips BAYES_xx scoring. It seems like this implies a simple *cf file issue. Spamassassin -D --lint shows no rules errors and the site directory shows up as /etc/mail/spamassassin which is where the local.cf file is. This is a sitewide installation. Among other things, debug output is included for both spamd and spamassassin- this may be a bit overkill, but I am trying to anticipate what things the group will ask for based on previous posts so I have included debugging as well a sample headers of a false negative that was clearly spam, the local.cf file from /etc/mail/spamassassin, permissions on the bayes db files, etc. Vital stats: Upgraded recently from spamassasin 3.0 to 3.1 mailscanner 4.48 Rules-de-jour *.cf files under /etc/mail/spamassassin Used sitewide in conjunction with Mailscanner perl 5.8.0 running on solaris 5.8 Here is sa-learn --dump magic: This shows that I have more than enough spam and ham 0.000 0 3 0 non-token data: bayes db version 0.000 0 3754 0 non-token data: nspam 0.000 0220 0 non-token data: nham 0.000 0 312279 0 non-token data: ntokens 0.000 0 1051829432 0 non-token data: oldest atime 0.000 0 1135374012 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1135373049 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count spamd runs as such: # ps -efa |grep spamd root 22145 21420 0 13:42:37 ?0:00 /usr/local/bin/perl -T /usr/local/bin/spamd -d -x -l -c --syslog-socket=inet /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0 0 0 0 use_bayes 1 use_bayes_rules 1 use_auto_whitelist 1 bayes_auto_learn1 bayes_auto_expire 1 bayes_expiry_max_db_size20 bayes_file_mode 0777 auto_whitelist_path/extra/system/spamassassin/autoDB/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /extra/system/spamassassin/autoDB/bayes bayes_ignore_header X-MailScanner bayes_ignore_header X-MailScanner-SpamCheck bayes_ignore_header X-MailScanner-SpamScore bayes_ignore_header X-MailScanner-Information # ls -lsag /extra/system/spamassassin/autoDB/ total 65893 3 drwxr-xr-x 3 root root 3072 Dec 23 11:16 ./ 1 drwxr-xr-x 6 root other 512 Nov 9 2004 ../ 16 -rw-rw-rw- 1 root other 24576 Dec 23 13:42 auto-whitelist 1 -rw--- 1 root other 6 Dec 23 13:42 auto-whitelist.mutex 23 -rw--- 1 root other 22572 Dec 23 14:13 bayes.mutex 312 -rw-rw-rw- 1 root other 352256 Dec 23 14:13 bayes_seen 8224 -rw-rw-rw- 1 root other10502144 Dec 23 14:13 bayes_toks 27792 -rw-rw-rw- 1 daemon mail 28430177 Dec 14 19:05 blacklist_mailspool 1 drwxrwxr-x 2 root users 512 Dec 23 11:09 delete/ 0 -rw-rw-rw- 1 daemon mail0 Dec 23 10:56 unwhitelist_mailspool 29520 -rw-rw-rw- 1 daemon mail 30198420 Dec 20 14:24 whitelist_mailspool Example e-mail header of false negative: Received: from i219-164-20-154.s02.a001.ap.plala.or.jp (i219-164-20-154.s02.a001.ap.plala.or.jp [219.164.20.154]) by [SNIP] (8.12.9/8.12.9) with SMTP id jBMCSafX001397 for [SNIP] ; Thu, 22 Dec 2005 04:28:37 -0800 (PST) Received: from [192.168.178.219] (port=29858 helo=smncs) by i219-164-20-154.s02.a001.ap.plala.or.jp with esmtp id 1EpPYF-0002uW-U5 for [EMAIL PROTECTED]; Thu, 22 Dec 2005 09:28:07 -0300 Date: Thu, 22 Dec 2005 21:28:28 +0900 From: [EMAIL PROTECTED] X-Mailer: The Bat! (v3.5) Professional Reply-To: [EMAIL PROTECTED] Organization: dyii X-Priority: 3 (Normal) Message-ID: [EMAIL PROTECTED] To: [SNIP] Subject: press release MIME-Version: 1.0 Content-Type: multipart/related; boundary==_869389e472dcb8f140f1bfe43211303b X-Spam: Not detected X-TSS-MailScanner-Information: See www.mailscanner.info for information X-TSS-MailScanner: Appears to be free
Re: SARE_URI_EQUALS false positives
Hello Chris, Friday, December 23, 2005, 3:04:29 AM, you wrote: CL I'm getting false positives for SARE_URI_EQUALS, which scores 5 and is CL therefore skewing the scoring of some mail quite badly. ... CL Does anyone have any suggestions, apart from simply reducing the CL score for SARE_URI_EQUALS? Is this a spamassassin bug, or is there CL no way to guarantee that only real uris are parsed as such? Send me a couple of sample emails with this problem so I can add them to my ham corpus, and SARE_URI_EQUALS will automagically drop its score to 1.666 (or lower). No SARE rule with ham scores more than 1.666. Best is to put them into an mbox file, zip, and email. Thanks. Bob Menschel
Re: Bayes Scores Skipped/Not Applied
John Urness wrote: /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0 0 0 0 That is very concerning. Why'd you do that? 99.9% of the time the proper fix is to declare a trusted_networks. Disabling this rule merely covers up one symptom of a very pervasive problem (errant trust). use_bayes 1 use_bayes_rules 1 use_auto_whitelist 1 bayes_auto_learn1 bayes_auto_expire 1 bayes_expiry_max_db_size20 bayes_file_mode 0777 auto_whitelist_path/extra/system/spamassassin/autoDB/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /extra/system/spamassassin/autoDB/bayes bayes_ignore_header X-MailScanner bayes_ignore_header X-MailScanner-SpamCheck bayes_ignore_header X-MailScanner-SpamScore bayes_ignore_header X-MailScanner-Information Wait, why the mailscanner ignores? Are you using mailscanner? If so, stop running spamd. MailScanner uses the perl API, so you don't need spamd, it's just wasting memory to run it.
Re: SARE_URI_EQUALS false positives
Does anyone have any suggestions, apart from simply reducing the score for SARE_URI_EQUALS? Is this a spamassassin bug, or is there no way to guarantee that only real uris are parsed as such? Several. 1.Change your report generator to remove the extraneous dot between updated and by. Or change it to the more common underscore, if you insist on these words being connected for some reason. 2.Put spaces around the equal sign. 3.If you are reluctant for the correct fix, drop the score on the uri_equals rule to 4 or maybe 3, depending on what else your report manages to hit. 4.You could submit a Bugzilla on the parsing of that phrase. But frankly I consider the bug in the report generation, not SA's parsing of strange syntax. Loren
Re: Bayes Scores Skipped/Not Applied
This seems strange: Here is sa-learn --dump magic: This shows that I have more than enough spam and ham 0.000 0 3 0 non-token data: bayes db version 0.000 0 3754 0 non-token data: nspam 0.000 0220 0 non-token data: nham 0.000 0 312279 0 non-token data: ntokens 0.000 0 1051829432 0 non-token data: oldest atime 0.000 0 1135374012 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1135373049 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count [19915] dbg: bayes: DB journal sync: last sync: 1105811470 [19915] dbg: bayes: corpus size: nspam = 153968, nham = 40588 [19915] dbg: bayes: score = 4.91619356335349e-10 [19915] dbg: bayes: DB expiry: tokens in DB: 1984629, Expiry max size: 15, Oldest atime: 1084312293, Newest atime: 0, Last expire: 1098687948, Current time: 1135280002 [19915] dbg: bayes: DB journal sync: last sync: 1105811470 As I read that, the bayes db has 3754 spam and 220 ham. But later in processing it has 153968 spam and 40558 ham! This makes me think you have two different bayes databases under two different users. Which would perhaps imply different user_prefs files, and one of them might not be enabling bayes. Loren
RE: Bayes Scores Skipped/Not Applied
Loren, You are seriously paying attention. I did the debugs yesterday and completely rebuilt the bayes db today using a whitelist and a blacklist mailspool so it is now a lot smaller since it lost a couple of years of autolearning when I started over... So it is actually from a site bayes database that was recreated and not an issue of multiple databases John -Original Message- From: Loren Wilton [mailto:[EMAIL PROTECTED] Sent: Friday, December 23, 2005 4:43 PM To: users@spamassassin.apache.org Subject: Re: Bayes Scores Skipped/Not Applied This seems strange: Here is sa-learn --dump magic: This shows that I have more than enough spam and ham 0.000 0 3 0 non-token data: bayes db version 0.000 0 3754 0 non-token data: nspam 0.000 0220 0 non-token data: nham 0.000 0 312279 0 non-token data: ntokens 0.000 0 1051829432 0 non-token data: oldest atime 0.000 0 1135374012 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1135373049 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count [19915] dbg: bayes: DB journal sync: last sync: 1105811470 [19915] dbg: bayes: corpus size: nspam = 153968, nham = 40588 [19915] dbg: bayes: score = 4.91619356335349e-10 [19915] dbg: bayes: DB expiry: tokens in DB: 1984629, Expiry max size: 15, Oldest atime: 1084312293, Newest atime: 0, Last expire: 1098687948, Current time: 1135280002 [19915] dbg: bayes: DB journal sync: last sync: 1105811470 As I read that, the bayes db has 3754 spam and 220 ham. But later in processing it has 153968 spam and 40558 ham! This makes me think you have two different bayes databases under two different users. Which would perhaps imply different user_prefs files, and one of them might not be enabling bayes. Loren
RE: Bayes Scores Skipped/Not Applied
Hi Matt, I stopped running spamd. The ALL_TRUSTED was letting a lot of junk get through and I saw a post that recommended 0 for the score to prevent false negatives. I have restored it to its original and added trusted networks (with a couple of subnets) as you suggest. I am still not seeing any BAYES scores. Here are a few examples: Message jBO1kTfX011690 from 141.157.60.60 ([EMAIL PROTECTED]) to tomsawyer.com is spam, SBL+XBL, spamcop.net, SpamAssassin (score=8.364, required 4, RATWARE_RCVD_PF 3.60, SARE_GETFCK 0.68, URIBL_JP_SURBL 4.09) Message jBO1hXfX008995 from 24.175.86.36 ([EMAIL PROTECTED]) to tomsawyer.com is spam, SpamAssassin (score=6.711, required 4, SPF_HELO_SOFTFAIL 2.43, SUBJ_ILLEGAL_CHARS 4.28) Message jBO1fYfX008953 from 222.47.203.164 ([EMAIL PROTECTED]) to tomsawyer.com is spam, SBL+XBL, SpamAssassin (score=10.678, required 4, FORGED_MUA_OUTLOOK 4.06, MSGID_SPAM_CAPS 4.40, SARE_RECV_IP_222032 2.22) Message jBO1kTfX011690 from 141.157.60.60 ([EMAIL PROTECTED]) to tomsawyer.com is spam, SBL+XBL, spamcop.net, SpamAssassin (score=8.364, required 4, RATWARE_RCVD_PF 3.60, SARE_GETFCK 0.68, URIBL_JP_SURBL 4.09) John -Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Sent: Friday, December 23, 2005 3:08 PM To: John Urness Cc: users@spamassassin.apache.org Subject: Re: Bayes Scores Skipped/Not Applied John Urness wrote: /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0 0 0 0 That is very concerning. Why'd you do that? 99.9% of the time the proper fix is to declare a trusted_networks. Disabling this rule merely covers up one symptom of a very pervasive problem (errant trust). use_bayes 1 use_bayes_rules 1 use_auto_whitelist 1 bayes_auto_learn1 bayes_auto_expire 1 bayes_expiry_max_db_size20 bayes_file_mode 0777 auto_whitelist_path /extra/system/spamassassin/autoDB/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /extra/system/spamassassin/autoDB/bayes bayes_ignore_header X-MailScanner bayes_ignore_header X-MailScanner-SpamCheck bayes_ignore_header X-MailScanner-SpamScore bayes_ignore_header X-MailScanner-Information Wait, why the mailscanner ignores? Are you using mailscanner? If so, stop running spamd. MailScanner uses the perl API, so you don't need spamd, it's just wasting memory to run it.
RE: Bayes Scores Skipped/Not Applied
Here is some debugging info from MailScanner. Starting MailScanner... In Debugging mode, not forking... debug: SpamAssassin version 3.0.1 debug: Score set 0 chosen. debug: running in taint mode? no debug: config: SpamAssassin failed to parse line, skipping: use_razor1 0 debug: SpamAssassin version 3.0.1 debug: Score set 0 chosen. Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 978. Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 980. debug: read_scoreonly_config: cannot open : No such file or directory [SNIP] /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.pm line 329. configuration file /usr/share/spamassassin/20_dnsbl_tests.cf requires version 3.001000 of SpamAssassin, but this is code version 3.01. Maybe you need to use the -C switch, or remove the old config files? Skipping this file at /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.pm line 329. configuration file /usr/share/spamassassin/20_drugs.cf requires version 3.001000 of SpamAssassin, but this is code version 3.01. Maybe you need to use the -C switch, or remove the old config files? Skipping this file at /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.pm line 329. [Snip] The parser/version errors are numerous and indicate SpamAssassin 3.0 is being called. Odd. More debug... debug: bayes: 19802 tie-ing to DB file R/O /extra/system/spamassassin/autoDB/bayes_toks debug: bayes: 19802 tie-ing to DB file R/O /extra/system/spamassassin/autoDB/bayes_seen debug: bayes: found bayes db version 3 debug: Score set 3 chosen. [Snip] So it looks like it is finding bayes. And scoring with it, but maybe its that it can't read the scoring from the cf files in /usr/share/spamassassin because a version conflict. Yes, going through the the log again I found that the bayes_cf file could not be parsed because an older version of spamassassin is at work. Presumably this is what no scores from Bayes are being used: configuration file /usr/share/spamassassin/23_bayes.cf requires version 3.001000 of SpamAssassin, but this is code version 3.01. Maybe you need to use the -C switch, or remove the old config files? Skipping this file at /usr/local/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.pm line 329. I installed it with make, make install, so assuming that is part of the problem, I am going to try upgrading with CPAN and see if that resolves the issue. John -Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Sent: Friday, December 23, 2005 3:08 PM To: John Urness Cc: users@spamassassin.apache.org Subject: Re: Bayes Scores Skipped/Not Applied John Urness wrote: /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0 0 0 0 That is very concerning. Why'd you do that? 99.9% of the time the proper fix is to declare a trusted_networks. Disabling this rule merely covers up one symptom of a very pervasive problem (errant trust). use_bayes 1 use_bayes_rules 1 use_auto_whitelist 1 bayes_auto_learn1 bayes_auto_expire 1 bayes_expiry_max_db_size20 bayes_file_mode 0777 auto_whitelist_path /extra/system/spamassassin/autoDB/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /extra/system/spamassassin/autoDB/bayes bayes_ignore_header X-MailScanner bayes_ignore_header X-MailScanner-SpamCheck bayes_ignore_header X-MailScanner-SpamScore bayes_ignore_header X-MailScanner-Information Wait, why the mailscanner ignores? Are you using mailscanner? If so, stop running spamd. MailScanner uses the perl API, so you don't need spamd, it's just wasting memory to run it.
RE: Bayes Scores Skipped/Not Applied: HAPPY RESOLUTION
Hi Matt, I resolved the issue. Thanks for pointing me in a different direction- the rubber has not been meeting the road for about a week on this issue! After upgrading using CPAN I am getting BAYES scores (among others from the /usr/share/spamassassin dir). So apparently it was an installation issue. Dec 23 18:45:12 unixserv0 MailScanner[24894]: Message jBO2iqfX025133 from 68.76.136.35 ([EMAIL PROTECTED]) to tomsawyer.com is spam, SBL+XBL, spamcop.net, SpamAssassin (score=36.46, required 4, autolearn=spam, BAYES_50 0.00, FORGED_RCVD_HELO 0.14, PYZOR_CHECK 3.70, RATWARE_NAME_ID 4.10, RCVD_IN_BL_SPAMCOP_NET 1.56, RCVD_IN_DSBL 2.60, RCVD_IN_NJABL_DUL 1.95, RCVD_IN_XBL 3.90, SAVE_THOUSANDS 0.40, SUBJECT_EXCESS_BASE64 0.45, TO_CC_NONE 0.13, URIBL_AB_SURBL 3.81, URIBL_JP_SURBL 4.09, URIBL_OB_SURBL 3.01, URIBL_SC_SURBL 4.50, URIBL_WS_SURBL 2.14) This leads me to a question about installation since in the INSTALL file it says: [unzip/untar the archive] cd Mail-SpamAssassin-* perl Makefile.PL [option: add -DSPAMC_SSL to $CFLAGS to build an SSL-enabled spamc] make make install This is how I normally done it over the past few years- install to a prefix in /usr/local and then create a symbolic link to /usr/local/spamassassin so that I can keep the previous version around in the event I need to recover from a bad upgrade. I don't recall ever having the difficulty that I had this time around. perl Makefile.PL --PREFIX=/usr/local/spamassassin-n.nn make make install ln -sf /usr/local/spamassassin-n.nn /usr/local/spamassassin The problem is that the installer did not put the perl modules in the site directory (/usr/perl5/site_perl/../... The spamassassin.pm module in the site dir now (after running CPAN) shows the updated version: #/usr/perl5# grep 3.00 /usr/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm $VERSION = 3.001000; # update after release (same format as perl $]) The SpamAssassin.pm file from my original make, make install is here, but obviously was not being used: # ls -lsa /usr/local/spamassassin/lib/perl5/site_perl/5.8.0/Mail total 53 1 drwxr-xr-x 3 root 512 Dec 12 16:08 ./ 1 drwxr-xr-x 4 root 512 Dec 12 16:08 ../ 1 drwxr-xr-x 10 root 1024 Dec 12 16:08 SpamAssassin/ 50 -r--r--r-- 1 root50780 Sep 13 19:07 SpamAssassin.pm This is also version 3.001.000 In the install directions, it does not say anything about building spamassassin and then moving perl modules manually. Am I missing something? John -Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Sent: Friday, December 23, 2005 3:08 PM To: John Urness Cc: users@spamassassin.apache.org Subject: Re: Bayes Scores Skipped/Not Applied John Urness wrote: /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0 0 0 0 That is very concerning. Why'd you do that? 99.9% of the time the proper fix is to declare a trusted_networks. Disabling this rule merely covers up one symptom of a very pervasive problem (errant trust). use_bayes 1 use_bayes_rules 1 use_auto_whitelist 1 bayes_auto_learn1 bayes_auto_expire 1 bayes_expiry_max_db_size20 bayes_file_mode 0777 auto_whitelist_path /extra/system/spamassassin/autoDB/auto-whitelist auto_whitelist_file_mode 0666 bayes_path /extra/system/spamassassin/autoDB/bayes bayes_ignore_header X-MailScanner bayes_ignore_header X-MailScanner-SpamCheck bayes_ignore_header X-MailScanner-SpamScore bayes_ignore_header X-MailScanner-Information Wait, why the mailscanner ignores? Are you using mailscanner? If so, stop running spamd. MailScanner uses the perl API, so you don't need spamd, it's just wasting memory to run it.
Spamassassin Debug
I'm trying to work through a problem where network tests don't seem to be working. Here's what I get from the debug on the command line: BTW: SA 3.0.1 invoked through amavisd on Mac OS X Server 10.4.3 saskatoon:~ mmuri$ sa-learn --dump magic ERROR: Bayes dump returned an error, please re-run with -D for more information saskatoon:~ mmuri$ sa-learn -D --dump magic debug: SpamAssassin version 3.0.1 debug: Score set 0 chosen. debug: running in taint mode? yes debug: Running in taint mode, removing unsafe env vars, and resetting PATH debug: PATH included '/bin', keeping. debug: PATH included '/sbin', keeping. debug: PATH included '/usr/bin', keeping. debug: PATH included '/usr/sbin', keeping. debug: Final PATH set to: /bin:/sbin:/usr/bin:/usr/sbin debug: using /etc/mail/spamassassin/init.pre for site rules init.pre debug: config: read file /etc/mail/spamassassin/init.pre debug: using //usr/share/spamassassin for default rules dir debug: config: read file //usr/share/spamassassin/10_misc.cf debug: config: read file //usr/share/spamassassin/20_anti_ratware.cf debug: config: read file //usr/share/spamassassin/20_body_tests.cf debug: config: read file //usr/share/spamassassin/20_compensate.cf debug: config: read file //usr/share/spamassassin/20_dnsbl_tests.cf debug: config: read file //usr/share/spamassassin/20_drugs.cf debug: config: read file //usr/share/spamassassin/20_fake_helo_tests.cf debug: config: read file //usr/share/spamassassin/20_head_tests.cf debug: config: read file //usr/share/spamassassin/20_html_tests.cf debug: config: read file //usr/share/spamassassin/20_meta_tests.cf debug: config: read file //usr/share/spamassassin/20_phrases.cf debug: config: read file //usr/share/spamassassin/20_porn.cf debug: config: read file //usr/share/spamassassin/20_ratware.cf debug: config: read file //usr/share/spamassassin/20_uri_tests.cf debug: config: read file //usr/share/spamassassin/23_bayes.cf debug: config: read file //usr/share/spamassassin/25_body_tests_es.cf debug: config: read file //usr/share/spamassassin/25_body_tests_pl.cf debug: config: read file //usr/share/spamassassin/25_hashcash.cf debug: config: read file //usr/share/spamassassin/25_head_tests_es.cf debug: config: read file //usr/share/spamassassin/25_head_tests_pl.cf debug: config: read file //usr/share/spamassassin/25_spf.cf debug: config: read file //usr/share/spamassassin/25_uribl.cf debug: config: read file //usr/share/spamassassin/30_text_de.cf debug: config: read file //usr/share/spamassassin/30_text_es.cf debug: config: read file //usr/share/spamassassin/30_text_fr.cf debug: config: read file //usr/share/spamassassin/30_text_it.cf debug: config: read file //usr/share/spamassassin/30_text_nl.cf debug: config: read file //usr/share/spamassassin/30_text_pl.cf debug: config: read file //usr/share/spamassassin/30_text_sk.cf debug: config: read file //usr/share/spamassassin/50_scores.cf debug: config: read file //usr/share/spamassassin/60_whitelist.cf debug: using //etc/mail/spamassassin for site rules dir debug: config: read file //etc/mail/spamassassin/local.cf debug: using /Network/Servers/saskatoon.myserver.com/Users/ mmuri/.spamassassin/user_prefs for user prefs file debug: config: read file /Network/Servers/saskatoon.myserver.com/ Users/mmuri/.spamassassin/user_prefs debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH (0x19250d8) debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH (0x1b09198) debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH (0x1a8bbfc) debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) implements 'parse_config' debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x1b09198) implements 'parse_config' debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) inhibited further callbacks ... last line repeated 33 times... debug: config: SpamAssassin failed to parse line, skipping: auto_learn 1 debug: config: SpamAssassin failed to parse line, skipping: safe_reporting 0 debug: config: SpamAssassin failed to parse line, skipping: use_terse_report 0 debug: config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail *** debug: config: SpamAssassin failed to parse line, skipping: rewrite_subject 0 debug: bayes: no dbs present, cannot tie DB R/O: /Network/Servers/ saskatoon.myserver.com/Users/mmuri/.spamassassin/bayes_toks debug: Score set 0 chosen. debug: bayes: no dbs present, cannot tie DB R/O: /Network/Servers/ saskatoon.myserver.com/Users/mmuri/.spamassassin/bayes_toks ERROR: Bayes dump returned an error, please re-run with -D for more information Miles Muri [EMAIL PROTECTED]
Re: Spamassassin Debug
I had a similar issue last week. I had inadvertantly set the DNS to an internal DNS server with forwarders to the outside, rather than to a real outside DNS server. Once I changed to a bona-fide outside DNS provider, all my network tests worked properly. You also need to fix those invalid settings in your local.cf, although they are not causing the network lookup problems. RO - Original Message - From: Miles Muri [EMAIL PROTECTED] To: users@spamassassin.apache.org Sent: Friday, December 23, 2005 8:00 PM Subject: Spamassassin Debug I'm trying to work through a problem where network tests don't seem to be working. Here's what I get from the debug on the command line: BTW: SA 3.0.1 invoked through amavisd on Mac OS X Server 10.4.3 saskatoon:~ mmuri$ sa-learn --dump magic ERROR: Bayes dump returned an error, please re-run with -D for more information saskatoon:~ mmuri$ sa-learn -D --dump magic debug: SpamAssassin version 3.0.1 debug: Score set 0 chosen. debug: running in taint mode? yes debug: Running in taint mode, removing unsafe env vars, and resetting PATH debug: PATH included '/bin', keeping. debug: PATH included '/sbin', keeping. debug: PATH included '/usr/bin', keeping. debug: PATH included '/usr/sbin', keeping. debug: Final PATH set to: /bin:/sbin:/usr/bin:/usr/sbin debug: using /etc/mail/spamassassin/init.pre for site rules init.pre debug: config: read file /etc/mail/spamassassin/init.pre debug: using //usr/share/spamassassin for default rules dir debug: config: read file //usr/share/spamassassin/10_misc.cf debug: config: read file //usr/share/spamassassin/20_anti_ratware.cf debug: config: read file //usr/share/spamassassin/20_body_tests.cf debug: config: read file //usr/share/spamassassin/20_compensate.cf debug: config: read file //usr/share/spamassassin/20_dnsbl_tests.cf debug: config: read file //usr/share/spamassassin/20_drugs.cf debug: config: read file //usr/share/spamassassin/20_fake_helo_tests.cf debug: config: read file //usr/share/spamassassin/20_head_tests.cf debug: config: read file //usr/share/spamassassin/20_html_tests.cf debug: config: read file //usr/share/spamassassin/20_meta_tests.cf debug: config: read file //usr/share/spamassassin/20_phrases.cf debug: config: read file //usr/share/spamassassin/20_porn.cf debug: config: read file //usr/share/spamassassin/20_ratware.cf debug: config: read file //usr/share/spamassassin/20_uri_tests.cf debug: config: read file //usr/share/spamassassin/23_bayes.cf debug: config: read file //usr/share/spamassassin/25_body_tests_es.cf debug: config: read file //usr/share/spamassassin/25_body_tests_pl.cf debug: config: read file //usr/share/spamassassin/25_hashcash.cf debug: config: read file //usr/share/spamassassin/25_head_tests_es.cf debug: config: read file //usr/share/spamassassin/25_head_tests_pl.cf debug: config: read file //usr/share/spamassassin/25_spf.cf debug: config: read file //usr/share/spamassassin/25_uribl.cf debug: config: read file //usr/share/spamassassin/30_text_de.cf debug: config: read file //usr/share/spamassassin/30_text_es.cf debug: config: read file //usr/share/spamassassin/30_text_fr.cf debug: config: read file //usr/share/spamassassin/30_text_it.cf debug: config: read file //usr/share/spamassassin/30_text_nl.cf debug: config: read file //usr/share/spamassassin/30_text_pl.cf debug: config: read file //usr/share/spamassassin/30_text_sk.cf debug: config: read file //usr/share/spamassassin/50_scores.cf debug: config: read file //usr/share/spamassassin/60_whitelist.cf debug: using //etc/mail/spamassassin for site rules dir debug: config: read file //etc/mail/spamassassin/local.cf debug: using /Network/Servers/saskatoon.myserver.com/Users/ mmuri/.spamassassin/user_prefs for user prefs file debug: config: read file /Network/Servers/saskatoon.myserver.com/ Users/mmuri/.spamassassin/user_prefs debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH (0x19250d8) debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH (0x1b09198) debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH (0x1a8bbfc) debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) implements 'parse_config' debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x1b09198) implements 'parse_config' debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) inhibited further callbacks ... last line repeated 33 times... debug: config: SpamAssassin failed to parse line, skipping: auto_learn 1 debug: config: SpamAssassin failed to parse line, skipping: safe_reporting 0 debug: config: SpamAssassin failed to parse line, skipping: use_terse_report 0 debug: config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail *** debug: config: SpamAssassin
Spamassassin debug - cont.
Sorry about that... pressed send before I had finished writing the message. regarding the debug output from the previous message, I have a few questions: 1) is the // in the default rules dir normal? Where do I fix this? 2) network tests don't seem to be working, I tried this test mentioned in the archives, and the surbl tests aren't happening: saskatoon:~ mmuri$ echo -e \n\nhttp://surbl-org-permanent-test-point- munged.com | spamassassin X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on myhost.myserver.com X-Spam-Level: X-Spam-Status: No, score=0.1 required=5.0 tests=ALL_TRUSTED,DOMAIN_RATIO, MISSING_DATE,MISSING_SUBJECT autolearn=no version=3.0.1 http://surbl-org-permanent-test-point-munged.com What does this line in the debug mean? I assume it is related to the lack of network tests... debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) inhibited further callbacks and why is it repeated 33 times? What do I do to fix it? (here's what I have done so far: Net::DNS didn't seem to be installed, so I used CPAN to get the latest version. amavisd.conf was changed to include $sa_local_tests_only = 0 (there seem to be amavisd.conf files at /etc/ amavisd.conf and at /etc/spam/clamav/amavisd.conf - both have been changed).) 3) this debug seems to be reading my local config and inserting default values where none exist, how should I invoke the -D switch to better debug at the sitewide level (I don't care about the user level at this point.) Miles Muri [EMAIL PROTECTED]
Re: Spamassassin Debug
At 11:00 PM 12/23/2005, Miles Muri wrote: I'm trying to work through a problem where network tests don't seem to be working. Here's what I get from the debug on the command line: BTW: SA 3.0.1 invoked through amavisd on Mac OS X Server 10.4.3 saskatoon:~ mmuri$ sa-learn --dump magic ERROR: Bayes dump returned an error, please re-run with -D for more information saskatoon:~ mmuri$ sa-learn -D --dump magic Based on the debug out, it sounds like your home directory of /Network/Servers/ saskatoon.myserver.com/Users/mmuri/.spamassassin/bayes_toks is unwritable. Check the permissions on the .spamassassin directory. You also have a LOT of errors in your configuration, fix them and run spamassassin --lint to make sure you got them all (lint should just run and exit without printing anything) debug: config: SpamAssassin failed to parse line, skipping: auto_learn 1 auto_learn was replaced with bayes_auto_learn in SA 2.50 debug: config: SpamAssassin failed to parse line, skipping: safe_reporting 0 That's safe_report, not safe_reporting debug: config: SpamAssassin failed to parse line, skipping: use_terse_report 0 Obsolete, this is superceded by the report_template commands debug: config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail *** debug: config: SpamAssassin failed to parse line, skipping: rewrite_subject 0 subject_tag and rewrite_subject was replaced with rewrite_header Subject in SA 3.0.0.
Re: Spamassassin debug - cont.
What does this line in the debug mean? I assume it is related to the lack of network tests... debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x19250d8) inhibited further callbacks It is essentially meaningless and is a normal operating sign. I believe it has been removed or changed in the lastest source. I'm not sure about the network tests, but that debug isn't related. I saw at the bottom of the debug that it had again chosen scoreset 0. The clearly means it knows it doesn't have network stuff enabled. Do you maybe have the 'no network tests' option on the command line (sorry, I froget which letter it is)? Loren
Re: Spamassassin Debug
Thanks, First, I don't know that the DNS server is the problem. I have caching DNS on the mailserver itself and it seems to work OK. I'm wondering if Net::DNS isn't working properly? It wasn't installed by default, so I used CPAN to get the latest version. How does a person go about testing it? For the local.cf settings, my local.cf file seems pretty inocuous: # # rewrite_subject 0 # report_safe 1 # trusted_networks 212.17.35. # Bayesian Auto Learn auto_learn 1 # Safe Reporting safe_reporting 0 # Full/Terse Reporting use_terse_report 0 # Subject Tag subject_tag *** Warning: Junk Mail *** # Rewrite the Subject rewrite_subject 0 # Use Bayesian Filtering use_bayes 1 # OK locals ok_locales en # OK languages ok_languages en fr de ja # Required hits to be marked as spam required_hits 5 Is that where you think the // is coming from? Thanks, Miles Muri [EMAIL PROTECTED] I had a similar issue last week. I had inadvertantly set the DNS to an internal DNS server with forwarders to the outside, rather than to a real outside DNS server. Once I changed to a bona-fide outside DNS provider, all my network tests worked properly. You also need to fix those invalid settings in your local.cf, although they are not causing the network lookup problems. RO
Re: Spamassassin Debug
Here's what --lint shows (the same errors you pointed out, thanks) saskatoon:/etc/mail/spamassassin root# spamassassin --lint config: SpamAssassin failed to parse line, skipping: auto_learn 1 config: SpamAssassin failed to parse line, skipping: safe_reporting 0 config: SpamAssassin failed to parse line, skipping: use_terse_report 0 config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail *** config: SpamAssassin failed to parse line, skipping: rewrite_subject 0 lint: 5 issues detected. please rerun with debug enabled for more information. I have fixed these now, but at least some of the local.cf settings seem to be overwritten when SA is invoked by amavisd. Here's the SA section in amavisd.conf # SpamAssassin settings # $sa_local_tests_only is passed to Mail::SpamAssassin::new as a value # of the option local_tests_only. See Mail::SpamAssassin man page. # If set to 1, no tests that require internet access will be performed. # $sa_local_tests_only = 0; # (default: false) #$sa_auto_whitelist = 1;# turn on AWL (default: false) $sa_mail_body_size_limit = 64*1024; # don't waste time on SA if mail is larger # (less than 1% of spam is 64k) # default: undef, no limitations # default values, can be overridden by more specific lookups, e.g. SQL $sa_tag_level_deflt = -999; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 5.0; # add 'spam detected' headers at that level $sa_kill_level_deflt = 22.0; #$sa_kill_level_deflt = $sa_tag2_level_deflt; # triggers spam evasive actions # at or above that level: bounce/reject/ drop, # quarantine, and adding mail address extension # # The $sa_tag_level_deflt, $sa_tag2_level_deflt and $sa_kill_level_deflt # may also be hashrefs to hash lookup tables, to make static per- recipient # settings possible without having to resort to SQL or LDAP lookups. # a quick reference: # tag_level controls adding the X-Spam-Status and X-Spam-Level headers, # tag2_level controls adding 'X-Spam-Flag: YES', and editing Subject, # kill_level controls 'evasive actions' (reject, quarantine, extensions); # it only makes sense to maintain the relationship: # tag_level = tag2_level = kill_level # string to prepend to Subject header field when message exceeds tag2 level $sa_spam_subject_tag = '*** JUNK MAIL ***'; # (defaults to undef, disables) # (only seen when spam is not to be rejected # and recipient is in local_domains*) $sa_spam_modifies_subj = 1; # may be a ref to a lookup table, default is true The config $sa_local_test_only was originally set to 1, but I changed that a while ago to no effect. I'm now digging through the debug output that I have to see if there are any other trails to follow. Many thanks, Miles Muri [EMAIL PROTECTED] At 11:00 PM 12/23/2005, Miles Muri wrote: I'm trying to work through a problem where network tests don't seem to be working. Here's what I get from the debug on the command line: BTW: SA 3.0.1 invoked through amavisd on Mac OS X Server 10.4.3 saskatoon:~ mmuri$ sa-learn --dump magic ERROR: Bayes dump returned an error, please re-run with -D for more information saskatoon:~ mmuri$ sa-learn -D --dump magic Based on the debug out, it sounds like your home directory of / Network/Servers/ saskatoon.myserver.com/Users/mmuri/.spamassassin/ bayes_toks is unwritable. Check the permissions on the .spamassassin directory. You also have a LOT of errors in your configuration, fix them and run spamassassin --lint to make sure you got them all (lint should just run and exit without printing anything) debug: config: SpamAssassin failed to parse line, skipping: auto_learn 1 auto_learn was replaced with bayes_auto_learn in SA 2.50 debug: config: SpamAssassin failed to parse line, skipping: safe_reporting 0 That's safe_report, not safe_reporting debug: config: SpamAssassin failed to parse line, skipping: use_terse_report 0 Obsolete, this is superceded by the report_template commands debug: config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail *** debug: config: SpamAssassin failed to parse line, skipping: rewrite_subject 0 subject_tag and rewrite_subject was replaced with rewrite_header Subject in SA 3.0.0.
Re: Spamassassin Debug
For the local.cf settings, my local.cf file seems pretty inocuous: Depends on precisely what you mean by 'inocuous'. At least four of those settings are invalid. Loren # # rewrite_subject 0 # report_safe 1 # trusted_networks 212.17.35. # Bayesian Auto Learn auto_learn 1 # Safe Reporting safe_reporting 0 # Full/Terse Reporting use_terse_report 0 # Subject Tag subject_tag *** Warning: Junk Mail *** # Rewrite the Subject rewrite_subject 0 # Use Bayesian Filtering use_bayes 1 # OK locals ok_locales en # OK languages ok_languages en fr de ja # Required hits to be marked as spam required_hits 5 Is that where you think the // is coming from? Thanks, Miles Muri [EMAIL PROTECTED] I had a similar issue last week. I had inadvertantly set the DNS to an internal DNS server with forwarders to the outside, rather than to a real outside DNS server. Once I changed to a bona-fide outside DNS provider, all my network tests worked properly. You also need to fix those invalid settings in your local.cf, although they are not causing the network lookup problems. RO
