Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
In debugging some Spamassassin configuration problems, I've found lots of Botnet-plugin-related errors in my logs - Use of uninitialized value in string eq at /var/spamassassin/local/Botnet.pm line 564. I found a January 17th, 2007 post on this @ http://archives.devshed.com/forums/networking-100/botnet-0-7-error-in-debug-log-2144795.html suggesting an imminent fix to this. But, the thread stopped there. Is there a current fix for this issue? Thanks, JTDeLys
Wrong RBL hits?
Hi I just noticed some inconsistency in a filtered spam on my server. The IPs in the reported RBL/WL don't match the IPs in the message header...?? I'm using SA 3.1.8 and amavisd-new SpamAssassin report (shortened): pts rule name description -- --- 0.1 FORGED_RCVD_HELO Received: contains a forged HELO 1.4 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) [SPF failed: Please see http://www.openspf.org/why.html?sender=agamemnon%40edomex.comip=213.203.223.10receiver=server.mindblow.ch] 1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see http://www.spamcop.net/bl.shtml?75.137.98.139] -0.1 RCVD_IN_DNSWL RBL: Received via whitelisted address, see http://www.dnswl.org/ [213.203.223.10 listed in list.dnswl.org] 1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is a abuseable web server [75.137.98.139 listed in dnsbl.sorbs.net] - BEGIN HEADERS - Return-Path: [EMAIL PROTECTED] X-Greylist: whitelisted by SQLgrey-1.6.7 Received: from gate01.nexlink.ch (gate01.nexlink.ch [80.86.198.160]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by server.mindblow.ch (Postfix) with ESMTP id 7CAD8D6A1 for [EMAIL PROTECTED]; Fri, 22 Jun 2007 13:04:15 +0200 (CEST) Received: from mail03.nexlink.ch ([10.51.9.3]) by gate01.nexlink.ch (8.13.1/8.13.1) with ESMTP id l5MB4Deu006418 for [EMAIL PROTECTED]; Fri, 22 Jun 2007 13:04:15 +0200 Received: from lb2 ([10.52.0.2] helo=mail.messaging.ch) by mail03.nexlink.ch with esmtp (Exim 4.63) (envelope-from [EMAIL PROTECTED]) id 1I1gw0-0001kB-UE; Fri, 22 Jun 2007 13:04:13 +0200 Received: from 24-151-201-36.dhcp.jcsn.tn.charter.com ([24.151.201.36]) by mail.messaging.ch with id Eb4G1X00H0ndMzs000; Fri, 22 Jun 2007 13:04:30 +0200 X-IMP: RBL SBL+XBL: 0.00,RBL SPAMCOP: 0.00,RBL SORBS: 0.10,RBL MAPS_ORDB: 0.00,URL RHS: 0.00,URL SURBL: 0.00,cmae[100|Undefined:Undefined] X-POSSIBLE-SPAM: 100 X-CLOUDMARK-SPAM-SCORE: 100.00 Date: Fri, 22 Jun 2007 17:02:13 +0500 From: Daniel [EMAIL PROTECTED] To: [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] Subject: [SPAM?] [DEL] Ich habe die beste Casino-Seite entdeckt ! MIME-Version: 1.0 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: 7bit -- END HEADERS -- But 213.203.223.10 is my.dynamic-net.ch which this mail didn't pass through And 75.137.98.139 is 75-137-98-139.dhcp.gnvl.sc.charter.com It seems to me, that two mail headers got confused .. maybe two lookups were performed simultaneously and the wrong results collected??? Matt
AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
same issue on my side...i'm also interested in a solution :-) Ove -Ursprüngliche Nachricht- Von: JT DeLys [mailto:[EMAIL PROTECTED] Gesendet: Montag, 25. Juni 2007 08:08 An: users@spamassassin.apache.org Betreff: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564 In debugging some Spamassassin configuration problems, I've found lots of Botnet-plugin-related errors in my logs - Use of uninitialized value in string eq at /var/spamassassin/local/Botnet.pm line 564. I found a January 17th, 2007 post on this @ http://archives.devshed.com/forums/networking-100/botnet-0-7-error-in-debug-log-2144795.html suggesting an imminent fix to this. But, the thread stopped there. Is there a current fix for this issue? Thanks, JTDeLys
Re: Botnet Score
Matt schrieb: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt i'm using the default 5 and until now i had one false positive (but bayes and awl saved it) thinking about it i might reduce the score to 3, but not lower because its really doing a great job over here arni
Re: Automatic Whitelist Generation - Why wouldn't this work?
On Mon, 2007-06-25 at 06:25 -0700, Marc Perkel wrote: Clarification. When I say that spammers can't spoof RNDS what I mean is that if you do a reverse lookup and get a spoofed name then when you look up the spoofed name it won't resolve back to the IP you looked up. I'm testing this idea now. Of course, that's what the botnet plugin does. But if you are looking for known ham sources, that's bonded sender or some such. They at least have a financial incentive to not send spam. For anyone else it's just a matter of when they get pwn3d next. -- Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX Austin Energy http://www.austinenergy.com
Re: Automatic Whitelist Generation - Why wouldn't this work?
On Mon, 25 Jun 2007, Marc Perkel wrote: Clarification. When I say that spammers can't spoof RNDS what I mean is that if you do a reverse lookup and get a spoofed name then when you look up the spoofed name it won't resolve back to the IP you looked up. I'm testing this idea now. RoadRunner Internet is already doing this. A customer of ours received a rejection message and this was within the content: 452 Too many recipients received this hour. Please see our rate limit policy at http://security.rr.com/spam.htm#ratelimit I can't to it myself here. I had it set once and by the end of a day, I had received a number of complaints from customers that they were not receiving messages from who they were before. Here I use Postfix and it is just a matter of throwing a switch so-to-speak to enable this feature.
Re: Botnet Score
Matt wrote: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt I have yet to see a hit, none so far in production (botnet been on for 5 days now). spamassassin -D --lint triggers one botnet hit, but not real spam for me :/
Re: Automatic Whitelist Generation - Why wouldn't this work?
Clarification. When I say that spammers can't spoof RNDS what I mean is that if you do a reverse lookup and get a spoofed name then when you look up the spoofed name it won't resolve back to the IP you looked up. I'm testing this idea now. Marc Perkel wrote: OK - here's an idea I'm rolling around in my brain and thinking this could work to massively automatically generate white lists of IP addresses from companies that generate no spam at all. This could be used not only to greatly reduce false positives, but also you reduce system load. Any IP listed is ham and no need for further testing. One thing that spammers can't spoof is RDNS. So if the RNDS of an IP is xxx.xxx.amd.com then we know the email is ham. Suppose that we start with a list of companies that we know that any email that comes from those hosts will always be ham then we can create a dynamically generated whitelist based on host IP addresses that come from the list. A query comes in to a specially written DNS server where the RNDS is looked up and it's xxx.ibm.com and ibm.com is in the list of blessed ham hosts. We would need a fast way of getting rid of the subhost part to do the lookup, stripping the xxx part off to get the domain, . We would then return a yes response and cache the data in a local database. The database could contain tens of thousands of domains that never send spam. How would we get this list? For now I'm doing it manually but it could possible be done by tracking ham and spam hist over time of verious IP addresses and looking for patterns of behavior that would indicate that indicate that the source is 100% clean. Of course this wouldn't solve domains like yahoo, hotmail, comcast, and other mixed source spam but it would allow a lot of email to be preclassified as ham without further testing. Who likes this idea?
Detecting the domain part of a host address?
What would be the method of detecting the domain part of a host address? For example: 82-46-151-246.cable.ubr04.perr.blueyonder.co.uk How would you write a perl script that would extract the blueyonder.co.uk part?
Re: Botnet Score
Matthias Haegele wrote: Jari Fredriksson schrieb: Matt wrote: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt I have yet to see a hit, none so far in production (botnet been on for 5 days now). Perhaps you use greylisting or similiar solutions already, or messages get blocked by Blacklists on MTA-Level? No, no such measures. But starting spamd -D tells this 24069] dbg: Botnet: checking BADDNS [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: BADDNS skipped [24069] dbg: Botnet: checking CLIENTWORDS [24069] dbg: Botnet: client words regexp is((\b|\d)(a|s|d(yn)?)?dsl(\b|\d))|((\b|\d)cable(\b|\d))|((\b|\d)catv(\b|\d))|((\b|\d)ddns(\b|\d))|((\b|\d)dhcp(\b) [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: CLIENTWORDS skipped [24069] dbg: Botnet: checking SERVERWORDS [24069] dbg: Botnet: server words list is((\b|\d)mail(\b|\d))|((\b|\d)mta(\b|\d))|((\b|\d)mx(\b|\d))|((\b|\d)relay(\b|\d))|((\b|\d)smtp(\b|\d))|((\b|\d)exc) [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: SERVERWORDS skipped [24069] dbg: Botnet: starting [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: skipping [24069] dbg: Botnet: checking IPINHOSTNAME [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: IPINHOSTNAME skipped [24069] dbg: Botnet: checking for CLIENT [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: CLIENT skipped [24069] dbg: Botnet: checking for SOHO server [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: SOHO skipped [24069] dbg: Botnet: checking NORDNS [24069] dbg: Botnet: no trusted relays [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: NORDNS skipped Seems that botnet disables itself? No trusted relays?
Re: Botnet Score
Jari Fredriksson schrieb: Matthias Haegele wrote: Jari Fredriksson schrieb: Matt wrote: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt I have yet to see a hit, none so far in production (botnet been on for 5 days now). Perhaps you use greylisting or similiar solutions already, or messages get blocked by Blacklists on MTA-Level? No, no such measures. But starting spamd -D tells this Seems that botnet disables itself? No trusted relays? 127.0.0.1 should be automatically trusted and you should add all your MX'es ip's so botnet can work properly arni
Re: Botnet Score
127.0.0.1 should be automatically trusted and you should add all your MX'es ip's so botnet can work properly Add to where? I have internal_networks and trusted_networks set up in local.cf
Re: Botnet Score
Jari Fredriksson schrieb: 127.0.0.1 should be automatically trusted and you should add all your MX'es ip's so botnet can work properly Add to where? I have internal_networks and trusted_networks set up in local.cf then that should be ok
Re: Detecting the domain part of a host address?
On Mon, Jun 25, 2007 at 06:30:19AM -0700, Marc Perkel wrote: What would be the method of detecting the domain part of a host address? 82-46-151-246.cable.ubr04.perr.blueyonder.co.uk How would you write a perl script that would extract the blueyonder.co.uk part? Use RegistrarBoundaries? :) -- Randomly Selected Tagline: /etc/fstab The file fstab resides in /etc. - man page for fstab pgpYnB96ujhsr.pgp Description: PGP signature
Re: Detecting the domain part of a host address?
Theo Van Dinter wrote: On Mon, Jun 25, 2007 at 06:30:19AM -0700, Marc Perkel wrote: What would be the method of detecting the domain part of a host address? 82-46-151-246.cable.ubr04.perr.blueyonder.co.uk How would you write a perl script that would extract the blueyonder.co.uk part? Use RegistrarBoundaries? :) ok - what's that? How would I write a perl script to do that?
Botnet + p0f (was: Botnet Score)
Mark Martinec wrote: The accuracy of botnet can be greatly enhanced it is when tamed down by p0f results (passive operating system fingerprinting). This is my experience as well. My Botnet scores looks like this currently: header BOTNET eval:botnet() score BOTNET 2.0 metaBOTNET_WINDOWS (BOTNET __OS_WINDOWS) score BOTNET_WINDOWS 1.0 header __OS_WINDOWSp0fIP2OS =~ /Windows/i The X-Amavis-OS-Fingerprint header field can be inserted by p0f+p0fanalyzer+amavisd (which I use), or by p0f+p0fanalyzer + p0f pluging for SA by Vincent Li Another alternative is my stuff at: http://whatever.frukt.org/p0fstats.text.shtml The stuff there uses UDP to send p0f info from the system running p0f (probably the firewall) to a collecting system that stores it in a database. It includes a perl module and a SpamAssassin plugin that can get info from the database, as well as some graph stuff. The SpamAssassin module is fairly new (about a year old), but the basic send/collect/store system has been in use for years here (though it has been modified and changed along the way). I have no idea wether my stuff is better, worse or just different than the stuff you mentioned above. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: Botnet Score
Mark Martinec schrieb: The accuracy of botnet can be greatly enhanced it is when tamed down by p0f results (passive operating system fingerprinting). I cant fully agree with that because allmost all xDSL or Cable users use some kind of hardware router which usually runs some kind of embedded unix or propetary system which will behave like unix. So from my experience you often see unix from the internet's point of view where its actually windows. arni
Re: Detecting the domain part of a host address?
On Mon, Jun 25, 2007 at 07:32:18AM -0700, Marc Perkel wrote: Use RegistrarBoundaries? :) ok - what's that? How would I write a perl script to do that? Take a look at Mail::SpamAssassin::Util::RegistrarBoundaries and the bits that call it. It doesn't have a POD, but it's an easy module to read. -- Randomly Selected Tagline: I am Mr Do. I am sedentary by nature, enjoying passive entertainment, eating when the mood takes me, and playing with my food. I try to avoid conflict, but when I'm angered, I can be a devil - if you force me to fight, I will crush you. With apples. - http://blog.ravenblack.net/quiz/videogame.pl?q=1a=11 pgp1K9z9IgnAs.pgp Description: PGP signature
Re: AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
Sorry, I've been having some issues at work for the last 6 or 7 months, that have kept me from working on the next version of Botnet. It's fixed in the version... I just haven't been able to get the new version out the door :-} Starckjohann, Ove wrote: same issue on my side...i'm also interested in a solution :-) Ove -Ursprüngliche Nachricht- Von: JT DeLys [mailto:[EMAIL PROTECTED] Gesendet: Montag, 25. Juni 2007 08:08 An: users@spamassassin.apache.org Betreff: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564 In debugging some Spamassassin configuration problems, I've found lots of Botnet-plugin-related errors in my logs - Use of uninitialized value in string eq at /var/spamassassin/local/Botnet.pm line 564. I found a January 17th, 2007 post on this @ http://archives.devshed.com/forums/networking-100/botnet-0-7-error-in-debug-log-2144795.html suggesting an imminent fix to this. But, the thread stopped there. Is there a current fix for this issue? Thanks, JTDeLys
Re: Botnet Score
Matthias Haegele wrote: Jari Fredriksson schrieb: Matt wrote: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt I have yet to see a hit, none so far in production (botnet been on for 5 days now). Perhaps you use greylisting or similiar solutions already, or messages get blocked by Blacklists on MTA-Level? In my experience, there are 3 things that have a really heavy overlap in effectiveness: 1) aggressive greet-pause/greeting-delay (say, 25+ seconds) 2) greylisting 3) Botnet Each one will leak a little bit that the others can catch, but generally speaking, if you're doing one, you wont see much benefit with the others. Since they happen in the above order, that means that the aggressive greet-pause will keep you from seeing as much benefits with the others. The advantage of lessening your reliance on the lower numbered techniques is: less severe impact from false-positives (a false-positive from greet-pause, on a host that refuses to wait out your delay duration, is effectively blacklisted from ever talking to you, for example; but a host that triggers Botnet, even if you have a score of 5, is just going to get put into your spam folder or quarantine -- no where near as bad). Then you add to that that since I last really analyzed this, pbl.spamhaus.org came into existence. That also seems to have some overlap with the purpose of Botnet. I'm not sure exactly how to add it to the above list, except that it comes before #3. So, if you're doing zen.spamhaus.org or pbl.spamhaus.org as a block list, some amount of greet-pause, AND greylisting ... then Botnet may only trigger on a few messages.
Re: Botnet Score
Jari Fredriksson wrote: Matthias Haegele wrote: Jari Fredriksson schrieb: Matt wrote: I have added botnet to my Spamassassin install. It seems to have helped quite a bit so far. I am just wandering about the 5 points it gives for a hit. Is that too much? Does it have alot of false positives or not? Matt I have yet to see a hit, none so far in production (botnet been on for 5 days now). Perhaps you use greylisting or similiar solutions already, or messages get blocked by Blacklists on MTA-Level? No, no such measures. But starting spamd -D tells this [24069] dbg: Botnet: All skipped/no untrusted [24069] dbg: Botnet: BADDNS skipped That means that the messages you're testing with are only coming from IP addresses you trust. Since Botnet skips looking at your own trusted relays, in trying to find the host that submitted the message to your group of systems, that means it's having the same effect as the all trusted rule. Basically Botnet is telling you this came from one of your own machines, and I'm assuming you don't have a locally installed botnet, thus I'm not going to waste time on figuring out anything for this message.
RE: AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
Sorry, I've been having some issues at work for the last 6 or 7 months, that have kept me from working on the next version of Botnet. It's fixed in the version... I just haven't been able to get the new version out the door :-} Mr. Rudd, Glad to see you coming out of the other side of the work tunnel... :-) Is there an expected ETA for the next Botnet release please? TIA - rh
Re: Mail not checked for spam in procmailrc
I am not sure if I understand what do you mean by this,
***You wrote
{^_^}
**
Thank you,
-Jai
jdow wrote:
From: Jai Rangi [EMAIL PROTECTED]
Hello All,
I am little confused here. I have this rule in my .procmailrc file.
:0f
* ^[F|f]rom:.*aleks\.com
*
^[m|M]essage-[i|I][D|d]:.*aleks\.com|^Received:.*(authenticated).*\.aleks\.com
| formail -AX-ALEKS-Spam: none
#:0fwE
:0fw
* 256000
* !^X-ALEKS-Spam: none
* !^FROM_DAEMON
| /usr/bin/spamc
So according to this rule every email should have tag X-ALEKS-Spam:
none or it should be checked for spam. Now I get few mail that dont
go through spam and do not get the No-Spam tag. For example this
Return-Path: [EMAIL PROTECTED]
X-Original-To: [EMAIL PROTECTED]
Delivered-To: [EMAIL PROTECTED]
Received: from ip26.aleks.com (ip26.aleks.com [216.34.240.160])
by localmail.lan.aleks.com (Postfix) with ESMTP id 0937560E72
for [EMAIL PROTECTED]; Thu, 21 Jun 2007 11:58:06 -0700 (PDT)
Received: from praznik-d.net (praznik-d.net [206.191.135.39])
by ip26.aleks.com (8.11.6/8.11.6) with SMTP id l5LIw3T09378
for [EMAIL PROTECTED]; Thu, 21 Jun 2007 11:58:03 -0700
Date: Thu, 21 Jun 2007 11:58:03 -0700
Message-Id: [EMAIL PROTECTED]
Received: (qmail 46857 invoked by uid 0); 21 Jun 2007 16:06:47 -
From: Cobra [EMAIL PROTECTED]
Subject: Affordable Health
To: [EMAIL PROTECTED]
MIME-Version: 1.0
Content-Type: MULTIPART/alternative;
BOUNDARY=0-1097643056-1182442005=:46707
X-route-head: verified/rgwl/ok/ref/aleks.com=clean
X-bounce-to: [EMAIL PROTECTED]
Can some one please give me some hint why this happened. Why this
email was not checked by spamc.
{^_^}
SA fails to search specified DATADIR for Distribution files
Exploring some problems I've been having, I have uploaded the latest SpamAssassin SVN source code. I'm specifying 'custom' locations for SpamAssassin local, distribution update files. Despite being correctly 'told' where to pick up the files, SA seems to ignore the DATADIR spec (where the Distribution files are), only checking in the LOCALSTATEDIR (where the UPdates are), and traversing no further. Can someone help find the problem? I've pasted some of the details below. Thanks, JTDeLys svn info Path: . URL: http://svn.apache.org/repos/asf/spamassassin/trunk Repository Root: http://svn.apache.org/repos/asf Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 Revision: 550563 Node Kind: directory Schedule: normal Last Changed Author: jm Last Changed Rev: 550555 Last Changed Date: 2007-06-25 10:20:48 -0700 (Mon, 25 Jun 2007) Reading PACKAGING, the wiki, and nformation from previous list posts, DATADIR (DEFRULESDIR): SpamAssassin's real logic lies in its shipped rule definitions and the corresponding scores. The files with these settings have to be saved somewhere, normally below PREFIX/share/spamassassin. The full path to that directory can be changed with this variable (DEFRULESDIR is a synonym). CONFDIR (LOCALRULESDIR): SpamAssassin looks for its config files in SYSCONFDIR/mail/spamassassin. LOCALSTATEDIR: sa-update will download rule updates into LOCALSTATEDIR/spamassassin. I've chosen to configure build using, perl Makefile.PL \ PREFIX=/usr/local/spamassassin \ DATADIR=/usr/local/etc/spamassassin/Distribution \ CONFDIR=/usr/local/etc/spamassassin/Local \ LOCALSTATEDIR=/usr/local/etc/spamassassin/Updates cd spamc perl version.h.pl After install, sa-update (of both Distribution SARE rules), and sa-compile, I have cd /usr/local/etc/spamassassin ls Distribution/ 10_default_prefs.cf20_vbounce.cf 30_text_it.cf 20_advance_fee.cf 23_bayes.cf 30_text_nl.cf 20_body_tests.cf 25_accessdb.cf30_text_pl.cf 20_compensate.cf 25_antivirus.cf 30_text_pt_br.cf 20_dnsbl_tests.cf 25_asn.cf 50_scores.cf 20_drugs.cf25_dcc.cf 60_awl.cf 20_dynrdns.cf 25_dkim.cf60_shortcircuit.cf 20_fake_helo_tests.cf 25_domainkeys.cf 60_whitelist.cf 20_head_tests.cf 25_hashcash.cf60_whitelist_dk.cf 20_html_tests.cf 25_pyzor.cf 60_whitelist_dkim.cf 20_imageinfo.cf25_razor2.cf 60_whitelist_spf.cf 20_meta_tests.cf 25_replace.cf 60_whitelist_subject.cf 20_net_tests.cf25_spf.cf languages 20_phrases.cf 25_textcat.cf sa-update-pubkey.txt 20_porn.cf 25_uribl.cf user_prefs.template 20_ratware.cf 30_text_de.cf 20_uri_tests.cf30_text_fr.cf ls -rd Updates/* Updates/compiled Updates/3.003000 Next, checking in the resulting man spamassassin, ... CONFIGURATION FILES ... Default configuration data is loaded from the first existing directory in: /usr/local/etc/spamassassin/Updates/3.003000 /usr/local/etc/spamassassin/Distribution /usr/local/spamassassin/share/spamassassin /usr/local/share/spamassassin /usr/share/spamassassin Site-specific configuration data is used to override any values which had already been set. This is loaded from the first existing directory in: /usr/local/etc/spamassassin/Local /usr/local/spamassassin/etc/mail/spamassassin /usr/local/spamassassin/etc/spamassassin /usr/local/etc/spamassassin /usr/pkg/etc/spamassassin /usr/etc/spamassassin /etc/mail/spamassassin /etc/spamassassin ... The problem is that spamassassin --lint --nocreate-prefs --debug /tmp/sa_debug_output.txt reports that, [24122] dbg: config: using /usr/local/etc/spamassassin/Local for site rules pre files [24122] dbg: config: read file /usr/local/etc/spamassassin/Local/init.pre [24122] dbg: config: using /usr/local/etc/spamassassin/Updates/3.003000 for sys rules pre files [24122] dbg: config: using /usr/local/etc/spamassassin/Updates/3.003000 for default rules dir and, apparently, never picks up the original .cf's from, /usr/local/etc/spamassassin/Distribution so, e.g., cd /usr/local/etc/spamassassin find . | grep -i 50_ ./Distribution/50_scores.cf never gets found/read -- DESPITE having assigned DATADIR=/usr/local/etc/spamassassin/Distribution It seems to me that, according to, Default configuration data is loaded from the first existing directory in: the Default config data is NOT picked up from the first INSTANCE of a given .cf in the search hierarchy, just from the first DIR listed there. -- Thanks, JTDeLys
Re: SA fails to search specified DATADIR for Distribution files
JT DeLys wrote: It seems to me that, according to, Default configuration data is loaded from the first existing directory in: the Default config data is NOT picked up from the first INSTANCE of a given .cf in the search hierarchy, just from the first DIR listed there. Yeah, as you quoted, the docs say first existing directory and not filename. The software, perhaps coincidentally, works the same way. I'm not sure what the problem is you are attempting to demonstrate... the update from updates.spamassassin.org contains everything you need from the default DATADIR. Daryl
Re: SA fails to search specified DATADIR for Distribution files
I'm not sure what the problem is you are attempting to demonstrate... the update from updates.spamassassin.org contains everything you need from the default DATADIR. The issue parallels a discussion I've been having in IRC. The problem is that I'm not seeing any SPF checks being done, despite the Plugin being enabled AND the rules/scores being defined in 50_scores.cf. 'felicity' suggested that's because the rules/scores are not being /seen/ -- and, checking with LINT, there's, indeed, no trace of those rules -- or ANYTHING from the Distribution files being found/read. Left to my own devices, I'm trying to figure out why not. Again, I've specified where the Distribution files are -- why are they not being found/read? You state, the update from updates.spamassassin.org contains everything you need from the default DATADIR. Checking my setup/install, cd /usr/local/etc/spamassassin grep -i spf `grep -rlni spf .` | grep -i score ./Distribution/50_scores.cf:score RCVD_IN_IADB_SPF 0 -0.001 0 -0.078 ./Distribution/50_scores.cf:ifplugin Mail::SpamAssassin::Plugin::SPF ./Distribution/50_scores.cf:score USER_IN_SPF_WHITELIST -100.000 ./Distribution/50_scores.cf:score USER_IN_DEF_SPF_WL -7.500 ./Distribution/50_scores.cf:score ENV_AND_HDR_SPF_MATCH -7.500 ./Distribution/50_scores.cf:endif # Mail::SpamAssassin::Plugin::SPF ./Distribution/50_scores.cf:# SPF ./Distribution/50_scores.cf:# Note that the benefit for a valid SPF record is deliberately minimal; it's ./Distribution/50_scores.cf:# likely that more spammers would quickly move to setting valid SPF records ./Distribution/50_scores.cf:ifplugin Mail::SpamAssassin::Plugin::SPF ./Distribution/50_scores.cf:score SPF_PASS -0.001 ./Distribution/50_scores.cf:score SPF_HELO_PASS -0.001 ./Distribution/50_scores.cf:score SPF_FAIL 2.600 0.992 1.669 0.693 ./Distribution/50_scores.cf:score SPF_HELO_FAIL 2.298 0.365 0.540 0.001 ./Distribution/50_scores.cf:score SPF_HELO_NEUTRAL 2.231 2.000 0.744 0.576 ./Distribution/50_scores.cf:score SPF_HELO_SOFTFAIL 2.599 1.533 1.427 0.841 ./Distribution/50_scores.cf:score SPF_NEUTRAL 2.199 1.210 0.756 0.686 ./Distribution/50_scores.cf:score SPF_SOFTFAIL 2.301 0.654 0.698 0.596 ./Distribution/50_scores.cf:endif # Mail::SpamAssassin::Plugin::SPF Everything relevant ONLY seems to be in Distribution/ -- nothing in Updates/. Checking where the updates SHOULD be, ls -d /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org returns, /usr/local/bin/ls: cannot access /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org: No such file or directory Well, that's a problem. looking at output of 'sa-update --debug', ... [18621] dbg: logger: adding facilities: all [18621] dbg: logger: logging level is DBG [18621] dbg: generic: SpamAssassin version 3.3.0-r543787 [18621] dbg: config: score set 0 chosen. [18621] dbg: dns: is Net::DNS::Resolver available? yes [18621] dbg: dns: Net::DNS version: 0.60 [18621] dbg: generic: sa-update version svn540382 [18621] dbg: generic: using update directory: /usr/local/etc/spamassassin/Updates/3.003000 ... [18621] dbg: channel: reading in channelfile /usr/local/etc/spamassassin/sa- update-channels.conf [18621] dbg: channel: adding updates.spamassassin.org [18621] dbg: channel: attempting channel updates.spamassassin.org [18621] dbg: channel: update directory /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org [18621] dbg: channel: channel cf file /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org.cf [18621] dbg: channel: channel pre file /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org.pre [18621] dbg: dns: query failed: 0.3.3.updates.spamassassin.org = NXDOMAIN [18621] dbg: channel: no updates available, skipping channel [18621] dbg: diag: updates complete, exiting with code 1 I notice the dns query FAIL. checking the man page, The default channel is updates.spamassassin.org, which has updated rules since the previous release. Net::DNS, which I think is responsble for those queries, /is/ working, as updates of other channels are working. Ideas? -- Thanks, JTDeLys
Re: SA fails to search specified DATADIR for Distribution files
JT DeLys wrote: Checking where the updates SHOULD be, ls -d /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org returns, /usr/local/bin/ls: cannot access /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org: No such file or directory Well, that's a problem. looking at output of 'sa-update --debug', ... [18621] dbg: logger: adding facilities: all [18621] dbg: logger: logging level is DBG [18621] dbg: generic: SpamAssassin version 3.3.0-r543787 [18621] dbg: config: score set 0 chosen. [18621] dbg: dns: is Net::DNS::Resolver available? yes [18621] dbg: dns: Net::DNS version: 0.60 [18621] dbg: generic: sa-update version svn540382 [18621] dbg: generic: using update directory: /usr/local/etc/spamassassin/Updates/3.003000 ... [18621] dbg: channel: reading in channelfile /usr/local/etc/spamassassin/sa-update-channels.conf [18621] dbg: channel: adding updates.spamassassin.org http://updates.spamassassin.org [18621] dbg: channel: attempting channel updates.spamassassin.org http://updates.spamassassin.org [18621] dbg: channel: update directory /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org [18621] dbg: channel: channel cf file /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org.cf [18621] dbg: channel: channel pre file /usr/local/etc/spamassassin/Updates/3.003000/updates_spamassassin_org.pre [18621] dbg: dns: query failed: 0.3.3.updates.spamassassin.org http://0.3.3.updates.spamassassin.org = NXDOMAIN [18621] dbg: channel: no updates available, skipping channel [18621] dbg: diag: updates complete, exiting with code 1 I notice the dns query FAIL. checking the man page, The default channel is updates.spamassassin.org http://updates.spamassassin.org, which has updated rules since the previous release. Net::DNS, which I think is responsble for those queries, /is/ working, as updates of other channels are working. Ideas? Your problem isn't anything to do with the datadir. It has to do with the fact sa-update isn't running. Furthermore, sa-update thinks your running version 3.3.0. The newest stable release is 3.2.1, so it looks to me that you're running a dev version.
Re: SA fails to search specified DATADIR for Distribution files
On Mon, Jun 25, 2007 at 02:47:03PM -0500, Richard Frovarp wrote: Your problem isn't anything to do with the datadir. It has to do with the fact sa-update isn't running. Furthermore, sa-update thinks your running version 3.3.0. The newest stable release is 3.2.1, so it looks to me that you're running a dev version. Yeah, we've been chatting about this in IRC. Basically, 3.3 has no updates available at all, but he's downloading updates from SARE, etc. There's a requirement that if you want the default rules, and you use sa-update, then you have to use the updates.spamassassin.org channel, and since it doesn't exist for 3.3, the install is in violation of that. I've asked that he open a bugzilla ticket asking for 3.3 updates, and in the mean time reverting to a non-development is a good idea for a non-testing environment. -- Randomly Selected Tagline: And, although some really nasty mind-games were played, no entities were physically harmed during the making of this interactive entertainment (except for the botched special-effect on the bunny rabbit that went so horribly wrong and really bummed everyone out, no thanks to Mr. Boomer). - From the 7th Guest pgpf0P2670GN0.pgp Description: PGP signature
Re: SA fails to search specified DATADIR for Distribution files
felicity has faster fingers. He beat me to the puchline. Thanks JTDeLys
Re: AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
Sorry, I've been having some issues at work for the last 6 or 7 months, that have kept me from working on the next version of Botnet. It's fixed in the version... I just haven't been able to get the new version out the door :-} I understand. Thanks. In the meantime, is Botnet /with/ these errors still reliable? Are these errors considered 'fatal' or badly skewing scoring? Or, can we just live with them for now with no particular damaging effects? -- Thanks, JTDeLys
Re: Botnet + p0f (was: Botnet Score)
On Mon, 25 Jun 2007, Jonas Eckerman wrote: Mark Martinec wrote: The accuracy of botnet can be greatly enhanced it is when tamed down by p0f results (passive operating system fingerprinting). This is my experience as well. My Botnet scores looks like this currently: header BOTNET eval:botnet() score BOTNET 2.0 metaBOTNET_WINDOWS (BOTNET __OS_WINDOWS) score BOTNET_WINDOWS 1.0 header __OS_WINDOWSp0fIP2OS =~ /Windows/i The X-Amavis-OS-Fingerprint header field can be inserted by p0f+p0fanalyzer+amavisd (which I use), or by p0f+p0fanalyzer + p0f pluging for SA by Vincent Li Another alternative is my stuff at: http://whatever.frukt.org/p0fstats.text.shtml The stuff there uses UDP to send p0f info from the system running p0f (probably the firewall) to a collecting system that stores it in a database. It includes a perl module and a SpamAssassin plugin that can get info from the database, as well as some graph stuff. The SpamAssassin module is fairly new (about a year old), but the basic send/collect/store system has been in use for years here (though it has been modified and changed along the way). I have no idea wether my stuff is better, worse or just different than the stuff you mentioned above. The p0f+p0fanalyzer+p0f plugin for SA is the same idea as yours, Mark Martinec's p0f-analyzer.pl script listen over udp and store fingerprint information in memory instead of database. my SA plugin simply extract the first untrusted relay ip and send query to p0f-analyzer.pl to collect the fingerprint information and add a metadata X-P0f-OS-Fingerprint. I have another SA plugin which send query to p0f unix socket, in this case, p0f-analyzer.pl is not needed, the drawback is SA has to run on MX host and the plugin has to do extra work to deal with machine endianess. http://bl0g.blogdns.com/spamassassin/p0f.tar the p0f-ppc.pm works on Linux PPC distritution,p0f-x86.pm works on Linux X86 distribution. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http: //whatever.frukt.org/ http: //www.fsdb.org/ http: //www.frukt.org/ !DSPAM:3363,467fd31d318231401698275! Vincent Li http://bl0g.blogdns.com
Re: Mail not checked for spam in procmailrc
Look at the line I underlined. Your rule decided you sent the email so
exempted it.
{^_^}
- Original Message -
From: Jai Rangi [EMAIL PROTECTED]
I am not sure if I understand what do you mean by this,
***You wrote
{^_^}
**
Thank you,
-Jai
jdow wrote:
From: Jai Rangi [EMAIL PROTECTED]
Message-Id: [EMAIL PROTECTED]
Will AWL score continue to drop over time once a message is in the AWL?
For awhile, I've not had spf-whitelisting working correctly. So, some messages have been getting too high spam scores, despite being spf-OK'd. Now, spf-whitelisting is working again. For a couple of messages from the spf-whitelisted domain, I got the expected bunch-of-hits , 'plus' ~100 pts for the spf. Good. Most recently, the messages are showing up ONLY with an in AWL score of ~ 2.6. No more hits on SPF rules. They're still getting through, and /are/ categorized as not-spam. Reading the documentation, AWL apparently ... Then, it combines this long-term average score ... Since the SPF whiteliesting rules are no longer hitting, /will/ the AWL score continue to drop over time? /I/ don't think so -- as a result of receiving/scoring these message incorrectly over time, I /think/ I've poisoned the scoring for them. Am I correct? If so, how do I start over scoring for this domain alone? Simply removing the domain from the AWL works only for 1-2 messages, then we're back to the same place. -- Thanks, JTDeLys
Re: AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
JT DeLys wrote: Sorry, I've been having some issues at work for the last 6 or 7 months, that have kept me from working on the next version of Botnet. It's fixed in the version... I just haven't been able to get the new version out the door :-} I understand. Thanks. In the meantime, is Botnet /with/ these errors still reliable? Are these errors considered 'fatal' or badly skewing scoring? Or, can we just live with them for now with no particular damaging effects? I still use the broken one in production. It only actually causes the error _sometimes_, and it doesn't appear to be fatal to the scanning process.
Re: AW: Old/Unresolved Botnet error, Use of uninitialized value in string eq at ... line 564
I still use the broken one in production. It only actually causes the error _sometimes_, and it doesn't appear to be fatal to the scanning process. Good enough! -- Thanks, JTDeLys
Re: SA fails to search specified DATADIR for Distribution files
JT DeLys wrote: Everything relevant ONLY seems to be in Distribution/ -- nothing in Updates/. The problem is that there currently is no update for the SVN version (3.3.0). If you were to use 3.2 it would work. Reverting the changes that broke 3.3 updates is on my list of things to do. Daryl
Re: SA fails to search specified DATADIR for Distribution files
If you were to use 3.2 it would work. I've already 'downgraded' to svn 3.2-branch, and you're right - It works! Reverting the changes that broke 3.3 updates is on my list of things to do. Great. Thanks. -- Thanks, JTDeLys
exposing rules
Is there a way to put into a header (or something) all the rules that here HIT in a message?
Re: Mail not checked for spam in procmailrc
Two things here,
1. I have two rule, (from and message_id) they both should match before
we add the tag X-ALEKS-Spam: none. Right?
2. Why I dont have tag in the header?
-Jai
:0f
* ^[F|f]rom:.*aleks\.com
*
^[m|M]essage-[i|I][D|d]:.*aleks\.com|^Received:.*(authenticated).*\.aleks\.com
| formail -AX-ALEKS-Spam: none
Does
jdow wrote:
Look at the line I underlined. Your rule decided you sent the email so
exempted it.
{^_^}
- Original Message - From: Jai Rangi [EMAIL PROTECTED]
I am not sure if I understand what do you mean by this,
***You wrote
{^_^}
**
Thank you,
-Jai
jdow wrote:
From: Jai Rangi [EMAIL PROTECTED]
Message-Id: [EMAIL PROTECTED]
Hello All,
I am little confused here. I have this rule in my .procmailrc file.
:0f
* ^[F|f]rom:.*aleks\.com
*
^[m|M]essage-[i|I][D|d]:.*aleks\.com|^Received:.*(authenticated).*\.aleks\.com
| formail -AX-ALEKS-Spam: none
#:0fwE
:0fw
* 256000
* !^X-ALEKS-Spam: none
* !^FROM_DAEMON
| /usr/bin/spamc
So according to this rule every email should have tag X-ALEKS-Spam:
none or it should be checked for spam. Now I get few mail that dont
go through spam and do not get the No-Spam tag. For example this
Return-Path: [EMAIL PROTECTED]
X-Original-To: [EMAIL PROTECTED]
Delivered-To: [EMAIL PROTECTED]
Received: from ip26.aleks.com (ip26.aleks.com [216.34.240.160])
by localmail.lan.aleks.com (Postfix) with ESMTP id 0937560E72
for [EMAIL PROTECTED]; Thu, 21 Jun 2007 11:58:06 -0700 (PDT)
Received: from praznik-d.net (praznik-d.net [206.191.135.39])
by ip26.aleks.com (8.11.6/8.11.6) with SMTP id l5LIw3T09378
for [EMAIL PROTECTED]; Thu, 21 Jun 2007 11:58:03 -0700
Date: Thu, 21 Jun 2007 11:58:03 -0700
Message-Id: [EMAIL PROTECTED]
Received: (qmail 46857 invoked by uid 0); 21 Jun 2007 16:06:47 -
From: Cobra [EMAIL PROTECTED]
Subject: Affordable Health
To: [EMAIL PROTECTED]
MIME-Version: 1.0
Content-Type: MULTIPART/alternative;
BOUNDARY=0-1097643056-1182442005=:46707
X-route-head: verified/rgwl/ok/ref/aleks.com=clean
X-bounce-to: [EMAIL PROTECTED]
Can some one please give me some hint why this happened. Why this
email was not checked by spamc.
Re: exposing rules
Tom Allison wrote: Is there a way to put into a header (or something) all the rules that here HIT in a message? By default this will be in X-Spam-Status. If they're not, can you let us know how you're calling spamassassin? Some tools, such as amavis and MailScanner generate their own headers and do not allow SA to insert its own.
Re: exposing rules
On Jun 25, 2007, at 7:42 PM, Matt Kettler wrote: Tom Allison wrote: Is there a way to put into a header (or something) all the rules that here HIT in a message? By default this will be in X-Spam-Status. If they're not, can you let us know how you're calling spamassassin? Some tools, such as amavis and MailScanner generate their own headers and do not allow SA to insert its own. Not that at all. I'm just new to using SA as perl modules and wasn't sure how it all worked.
utf8
I'm not sure how/if this is done. But I was wondering if anyone has looked into decoding all the charsets into utf8 for bayesian analysis. octets is not readily visible to the user the way it's done today.
forged_aol_tags
This is ham from an AOL user, did something change to make SA think it's forged? Begin forwarded message: From: [EMAIL PROTECTED] Date: June 25, 2007 9:07:00 PM PDT To: [EMAIL PROTECTED] Subject: Re: Sail Boat Fire. Return-Path: [EMAIL PROTECTED] Received: from murder ([unix socket]) by smtp.interstellar.com (Cyrus v2.2.12-OS X 10.4.8) with LMTPA; Mon, 25 Jun 2007 21:07:20 -0700 Received: from localhost (localhost [127.0.0.1]) by smtp.interstellar.com (Postfix) with ESMTP id 9CCED43FCEB for [EMAIL PROTECTED]; Mon, 25 Jun 2007 21:07:20 -0700 (PDT) Received: from smtp.interstellar.com ([127.0.0.1]) by localhost (interstellar.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TKWfYEQEA73g for [EMAIL PROTECTED]; Mon, 25 Jun 2007 21:07:14 -0700 (PDT) Received: from imo-m25.mx.aol.com (imo-m25.mx.aol.com [64.12.137.6]) by smtp.interstellar.com (Postfix) with ESMTP id 24F5B43FCDD for [EMAIL PROTECTED]; Mon, 25 Jun 2007 21:07:13 -0700 (PDT) Received: from [EMAIL PROTECTED] by imo-m25.mx.aol.com (mail_out_v38_r9.2.) id t.d02.12dfc8fd (52375) for [EMAIL PROTECTED]; Tue, 26 Jun 2007 00:07:00 -0400 (EDT) Received: from mblk-r36 (mblk-r36.mblk.aol.com [152.163.179.35]) by ciaaol-m02.mx.aol.com (v117.7) with ESMTP id MAILCIAAOLM029- cc97468090e43b6; Tue, 26 Jun 2007 00:07:00 -0400 Received: from 208.208.47.100 by mblk-r36.sysops.aol.com (152.163.179.35) with HTTP (WebMailUI); Tue, 26 Jun 2007 00:07:00 -0400 X-Sieve: CMU Sieve 2.2 X-Virus-Scanned: amavisd-new 2.5.1 (20070531) at interstellar.com X-Spam-Flag: NO X-Spam-Score: 0.831 X-Spam-Status: No, score=0.831 tagged_above=0 required=3 tests= [AWL=-0.454, BAYES_00=-2.599, DKIM_POLICY_SIGNSOME=0, DK_POLICY_SIGNSOME=0, FORGED_AOL_TAGS=2.488, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, SPF_PASS=-0.001] References: [EMAIL PROTECTED] [EMAIL PROTECTED] In-Reply-To: [EMAIL PROTECTED] X-Mb-Message-Source: WebUI Mime-Version: 1.0 X-Mb-Message-Type: User Content-Type: multipart/alternative; boundary= MB_8C985BE94819D23_AFC_6800_mblk-r36.sysops.aol.com X-Mailer: AOL WebMail 27618 Message-Id: [EMAIL PROTECTED] X-Aol-Ip: 152.163.179.35
