smf-sav, smf-zombie experience
I want to thank all for their input over the last several days while I was getting hammered with dictionary attacks, It's still on, but appears to be reduced on off-business hours. Has anyone any opinions or pointers on smf-sav and smf-zombie as milters to help reduce the onslaught passed to SA? TIA Pat... [EMAIL PROTECTED] CocoNet Corporation SW Florida's First ISP
How to upgrade spamassassin in Mandrake 10.1
Hi all, I try to upgrade my SA in mandrake 10.1. I've downloaded the latest SA and build the rpm. But, when I tried to upgrade it, it errored: rpm -Uvh spamassassin-3.1.7-1.i586.rpm perl-Mail-SpamAssassin-3.1.7-1.i586.rpm error: Failed dependencies: spamassassin = 3.0.4-0.1.101mdk is needed by (installed) spamassassin-spamd-3.0.4-0.1.101mdk perl-Mail-SpamAssassin = 3.0.4-0.1.101mdk is needed by (installed) spamassassin-tools-3.0.4-0.1.101mdk I notice that my mandrake 10.1 contains several rpms regarding SA: spamassassin-tools-3.0.4-0.1.101mdk spamassassin-3.0.4-0.1.101mdk spamassassin-spamd-3.0.4-0.1.101mdk spamassassin-spamc-3.0.4-0.1.101mdk Can someone help me how to upgrade it? Should I (forced) remove all previous SA? Thank you very much, -- Fajar Priyanto | Reg'd Linux User #327841 | Linux tutorial http://linux2.arinet.org 5:46pm up 0:11, 2.6.16.13-4-default GNU/Linux Let's use OpenOffice. http://www.openoffice.org pgpTVx8nf3QwD.pgp Description: PGP signature
Re: How to upgrade spamassassin in Mandrake 10.1
On Sun, Dec 03, 2006 at 05:50:12PM +0700, Fajar Priyanto wrote: I notice that my mandrake 10.1 contains several rpms regarding SA: spamassassin-tools-3.0.4-0.1.101mdk spamassassin-3.0.4-0.1.101mdk spamassassin-spamd-3.0.4-0.1.101mdk spamassassin-spamc-3.0.4-0.1.101mdk Can someone help me how to upgrade it? Should I (forced) remove all previous SA? I don't think you need to force the removal, but yes, I'd get rid of the Mandrake specific ones and then install the standard SA ones. -- Randomly Selected Tagline: MA Driving #6: Suicide Alley is the better of the roads. pgpUtUNwQddll.pgp Description: PGP signature
Re: How to upgrade spamassassin in Mandrake 10.1
On Sunday 03 December 2006 4:50 am, Fajar Priyanto wrote: Can someone help me how to upgrade it? Should I (forced) remove all previous SA? Thank you very much, I also run Mandrake 10.1 but I've always installed via CPAN, to me its much easier. -- Chris pgpGYZ8Talt3C.pgp Description: PGP signature
Re: How to upgrade spamassassin in Mandrake 10.1
On Sun, 2006-12-03 at 17:50 +0700, Fajar Priyanto wrote: Hi all, I try to upgrade my SA in mandrake 10.1. I've downloaded the latest SA and build the rpm. But, when I tried to upgrade it, it errored: rpm -Uvh spamassassin-3.1.7-1.i586.rpm perl-Mail-SpamAssassin-3.1.7-1.i586.rpm error: Failed dependencies: Can someone help me how to upgrade it? Should I (forced) remove all previous SA? You can find updated RPMs for 10.1 here: ftp://ftp.neocat.org/pub/RPMS/10.1/i586 These will match your current install, allowing a clean upgrade. Alternatively, you can remove (rpm -e) the current SA install (be sure to list all the installed RPMs on the command line) then install your newly built ones. -Bill
Re: Rewrite subject with score
[EMAIL PROTECTED] wrote: I have seen this in the past but now can not find those email on how to do this. What i want to do is rewrite the subject line so when it is thought to be spam, it will appear like this: [SPAM]score score=the score of the email thought to be spam. Can some please let me know how to do this. Chris http://wiki.apache.org/spamassassin/SubjectRewrite
Re: How to upgrade spamassassin in Mandrake 10.1
Fajar Priyanto wrote: Hi all, I try to upgrade my SA in mandrake 10.1. I've downloaded the latest SA and build the rpm. But, when I tried to upgrade it, it errored: rpm -Uvh spamassassin-3.1.7-1.i586.rpm perl-Mail-SpamAssassin-3.1.7-1.i586.rpm error: Failed dependencies: spamassassin = 3.0.4-0.1.101mdk is needed by (installed) spamassassin-spamd-3.0.4-0.1.101mdk perl-Mail-SpamAssassin = 3.0.4-0.1.101mdk is needed by (installed) spamassassin-tools-3.0.4-0.1.101mdk I notice that my mandrake 10.1 contains several rpms regarding SA: spamassassin-tools-3.0.4-0.1.101mdk spamassassin-3.0.4-0.1.101mdk spamassassin-spamd-3.0.4-0.1.101mdk spamassassin-spamc-3.0.4-0.1.101mdk Can someone help me how to upgrade it? Should I (forced) remove all previous SA? Thank you very much, Are you using a sql-based Bayes db? I found that the upgrade of perl-MailSpamAssassin failed with a MySQL bayes. When I removed the password for 'root'@'localhost', the upgrade succeeded. (I then put the password back.) -- Steve
Re: whitelist_from and whitelist_from_rcvd not working
Mark Adams wrote: Hi All, Spamassassin 3.1.4-1 Currently have entries like the following in the local.cf file whitelist_from [EMAIL PROTECTED] and whitelist_from [EMAIL PROTECTED] But mail is still picked up as spam for the [EMAIL PROTECTED] Have also tried the following; whitelist_from_rcvd [EMAIL PROTECTED] domain.com and whitelist_from_rcvd [EMAIL PROTECTED] domain.com But nothing seems to work? has anyone got any advice on this? do you have always_trust_envelope_sender 1 ?
Re: Best Choice for Bayes filtering on SpamAssassin
Noc Phibee wrote: Hi i have 6 servers running on spamassassin 3.1.7 (now after a upgrades). Actually, all have Bayes Filering with local Db (default db, not sql) I want know what is the best choice ? : - Default Db or MySQL db ? - 1 Bayes Db per server or 1 Bayes on Sql for all server (same database) My server receive 500 000/ 750 000 mails /days thanks bye MySQL DB, make sure you expire manually. Master/Slave config depends on how you learn, I learn on one machine, so that machine is the master and does the expiration while the others simply just 'look' at the data. I don't receive quite the load you do, but my boxes are quite undersized (anyone say P2? :-D ) I'm not sure if you are using AWL or user_prefs, but these are also in the same configuration. AWL connects to the master only however. I saw a marked in bayes performance when I changed from DB to MySQL. If all your servers are local to themselves, then master/slave config is the easiest to accomplish and as far as upgrades go, if all your boxes are the same (hardware wise) when you get one setup and tuned you'd simply image over the the other 4/5 in the chain. (Of course there are the box independent configs -- name ip etc. But I'm sure with that volume of mail, you have a system to keep all servers the 'same') -- Thanks, JamesDR smime.p7s Description: S/MIME Cryptographic Signature
Re: How to upgrade spamassassin in Mandrake 10.1
On Sunday 03 December 2006 23:33, Steven Stern wrote: Are you using a sql-based Bayes db? I found that the upgrade of perl-MailSpamAssassin failed with a MySQL bayes. When I removed the password for 'root'@'localhost', the upgrade succeeded. (I then put the password back.) No, I'm using ordinary SA. -- Fajar Priyanto | Reg'd Linux User #327841 | Linux tutorial http://linux2.arinet.org 1:41am up 8:05, 2.6.16.13-4-default GNU/Linux Let's use OpenOffice. http://www.openoffice.org pgpfBOT0O1kL0.pgp Description: PGP signature
Custom Rules
Hello, I have been asked by my boss to setup SpamAssassin on the corporate email server with the following rules. A single header should record the cumulative scores for the following: SPF record not available or not accurate for the sending server- 2 points Date in the mail header more than 10 minutes out of sync - 1 point Date in the mail header more than 30 mintues out of sync- 2 points From address contains only email address - 1 point for example flag these [EMAIL PROTECTED] or [EMAIL PROTECTED] not X X [EMAIL PROTECTED] Since I am new to SpamAssassin, I am not sure where to begin or if this is even possible. If someone could assist me in setting up these rules I would be greatful. Regards, Jaysen B. Johnson
Re: Custom Rules
On Sun, Dec 03, 2006 at 01:16:40PM -0800, Jaysen Johnson wrote: SPF record not available or not accurate for the sending server- 2 points Check out the current SPF rules. Not available may need some plugin changes. Date in the mail header more than 10 minutes out of sync - 1 point Date in the mail header more than 30 mintues out of sync- 2 points What does this mean? That the Date header, after timezone standardization, says the message is X minutes old? If so, that's going to be a bad rule since a mail can be delayed at any point during its travels to the destination. -- Randomly Selected Tagline: It's a chicken finger device.- Theo, looking at entree pgpsYkKg1UU7n.pgp Description: PGP signature
Re: Custom Rules
Jaysen Johnson wrote: Hello, I have been asked by my boss to setup SpamAssassin on the corporate email server with the following rules. A single header should record the cumulative scores for the following: SPF record not available or not accurate for the sending server- 2 points No. The current module just returns false if it can't find SPF results. You could submit at patch for /Mail/SpamAssassin/Plugin/SPF.pm to fix that. (I just might, since I agree with your logic but it's not as high on my list as other things) Date in the mail header more than 10 minutes out of sync - 1 point Date in the mail header more than 30 mintues out of sync- 2 points No. The rules which deal with dates are: describe DATE_IN_PAST_03_06Date: is 3 to 6 hours before Received: date describe DATE_IN_PAST_06_12Date: is 6 to 12 hours before Received: date describe DATE_IN_PAST_12_24Date: is 12 to 24 hours before Received: date ...etc And I doubt that a 10-minute variance will catch a lot of spam, really. It will absolutely catch a lot of ham, especially messages which are queued and sent later (person working disconnected on a laptop) From address contains only email address - 1 point for example flag these [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] or [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] not X X [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] score NO_REAL_NAME 1 There is no matching for From header mapping, but you can add your own header FROM_ADDRESS_EQ_REAL From =~ /^\s*([^@[EMAIL PROTECTED]@]+)\s+\1\s*$/i describe FROM_ADDRESS_EQ_REAL To: repeats address as real name score FROM_ADDRESS_EQ_REAL 1 -- Jo Rhett Network/Software Engineer Net Consonance
skipping SPF checks for authenticated users
So I saw on this list a comment about skipping SPF checks for authenticated users, to use LOCAL_AUTH_RCVD like so: header LOCAL_AUTH_RCVDReceived =~ /\(authenticated as [EMAIL PROTECTED]) by host.name.dom / Well, I got this working properly but I found that it doesn't do anything by itself. I can negative the SPF failure by using score LOCAL_AUTH_RCVD-10 But negating the score is very different from actually skipping SPF and DUL checks, which would save some processing that isn't useful. Grepping for LOCAL_AUTH_RCVD in the source code shows that nothing else looks for it. So this isn't how to deal with it properly it is a recipe for how to negate the score which is entirely different. Am I overlooking anything? Or do I need to change the code and submit a patch so that a person can optionally avoid doing DUL and SPF checks on authenticated e-mail? -- Jo Rhett Network/Software Engineer Net Consonance
sa-learn and autolearn - working or not?
Can you please check my SA kit to assure me that sa-learn is having the intended effect? I have ZERO (none, nil) instances where header shows autolearn= as any other value than autolearn=no. This leads me conclude that my sa-learn data is not being utilized by spamd?! Is there a log or other way to peek into SA to have it tell me whether it's got the sa-learn data in the bayes engine and IS USING THAT INFORMATION? * RedHat AS 4, spamd , qmail, qmail-scanner, clamav * spamd -V SpamAssassin Server version 3.1.7 running on Perl 5.8.5 with SSL support (IO::Socket::SSL 1.02) ps waux | grep -i spam root 3842 0.0 2.9 34520 30328 ? Ss Nov15 0:14 /usr/bin/spamd -x -H /home/spamd -d -s /var/log/spamd/spamd.log root 26510 0.0 4.0 45476 41316 ? SDec01 0:38 spamd child root 31622 0.0 3.2 36920 32688 ? S11:38 0:00 spamd child * sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 11737 0 non-token data: nspam 0.000 0 23457 0 non-token data: nham 0.000 0 142588 0 non-token data: ntokens 0.000 0 1164208327 0 non-token data: oldest atime 0.000 0 1165185233 0 non-token data: newest atime 0.000 0 1165184550 0 non-token data: last journal sync atime 0.000 0 1164928829 0 non-token data: last expiry atime 0.000 0 720391 0 non-token data: last expire atime delta 0.000 0 29443 0 non-token data: last expire reduction count ** ls -l /home/spamd/.spamassassin/ total 7544 -rw--- 1 root root 13248 Dec 3 16:55 bayes_journal -rw--- 1 root root 5124096 Dec 3 16:33 bayes_seen -rw--- 1 root root 5398528 Dec 3 16:33 bayes_toks * cat /etc/mail/spamassassin/local.cf | grep -v ^# required_score 6 rewrite_header Subject [SPAM] report_safe 0 use_pyzor 0 use_razor2 1 use_dcc 0 dcc_home /var/dcc skip_rbl_checks 0 rbl_timeout 3 score RCVD_IN_BL_SPAMCOP_NET 2 use_bayes 1 bayes_auto_learn 1 bayes_path /home/spamd/.spamassassin I don't see anything here that prevents autolearn results from being applied? HELP please! Most appreciated! Dave.
real or fake capital-one message
I got this in my inbox today, I believe it to be real, however I'll post the headers below. The reason I think it may be real is that there is some person out there named Carol Pollock who for some reason and some how is using the email address of [EMAIL PROTECTED] How, I haven't the faintest clue. Here are the headers: X-Spam-Virus: No X-Spam-Seen: Tokens 204 X-Spam-New: Tokens 293 X-Spam-Remote: Host localhost.localdomain X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on cpollock.localdomain X-Spam-Hammy: Tokens 56 X-Spam-Status: No, score=-105.3 required=5.0 tests=BAYES_00,HTML_MESSAGE, SPF_FAIL,SPF_HELO_PASS,USER_IN_WHITELIST autolearn=disabled version=3.1.7 X-Spam-Spammy: Tokens 5 X-Spam-Pyzor: Reported 0 times. X-Spam-Token: Summary Tokens: new, 89; hammy, 56; neutral, 143; spammy, 5. X-Spam-DCC: CollegeOfNewCaledonia cpollock 1189; Body=1 Fuz1=1 Fuz2=1 X-Spam-Untrusted: Relays [ ip=216.35.62.79 rdns=arm79.bigfootinteractive.com helo=bigfootinteractive.com by=mx-bracke.atl.sa.earthlink.net ident= envfrom= intl=0 id=1gQWIB30u3Nl34i6 auth= ] X-Spam-Level: X-Spam-RBL: Results dns:email.capitalone.com?type=MX [20 arm.bigfootinteractive.com.] dns:email.capitalone.com [206.132.3.45] Status: U Return-Path: [EMAIL PROTECTED] Received: from pop.earthlink.net [209.86.93.201] by localhost with POP3 (fetchmail-6.2.5) for [EMAIL PROTECTED] (single-drop); Sun, 03 Dec 2006 13:11:30 -0600 (CST) Received: from bigfootinteractive.com ([216.35.62.79]) by mx-bracke.atl.sa.earthlink.net (EarthLink SMTP Server) with SMTP id 1gQWIB30u3Nl34i6 for [EMAIL PROTECTED]; Sun, 3 Dec 2006 14:09:41 -0500 (EST) Reply-To: Capital One [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] X-BFI: TBTH0562119F1CA6AC909D05A5EBC0 Date: Sun, 03 Dec 2006 14:09:41 EST From: Capital One [EMAIL PROTECTED] Subject: Welcome to Capital One No Hassle Rewards To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: multipart/alternative; boundary=ABCD-TBTH0562119F1CA6AC909D05A5EBC0-EFGH X-ELNK-Info: sbv=0; sbrc=.0; sbf=00; sbw=000; X-SenderIP: 216.35.62.79 X-ASN: ASN-3561 X-CIDR: 216.32.0.0/14 X-UID: 24237 X-Length: 11032 [EMAIL PROTECTED] chris]$ nslookup 216.35.62.79 Server: 127.0.0.1 Address:127.0.0.1#53 Non-authoritative answer: 79.62.35.216.in-addr.arpa canonical name = 79.0/25.62.35.216.in-addr.arpa. 79.0/25.62.35.216.in-addr.arpa name = arm79.bigfootinteractive.com. I could of course throw this into my spam folder and report it with the rest or I could just delete it, however I'm curious as to whether its an actual message from them or not. It has a valid certificate issued by VeriSign OU = www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign OU = VeriSign International Server CA - Class 3 OU = VeriSign, Inc. O = VeriSign Trust Network 02/12/2006 18:00:00 (02/13/2006 00:00:00 GMT) 02/13/2007 17:59:59 (02/13/2007 23:59:59 GMT) I'm going to assume that its a vaild message and that she's again using my email address and that I'm getting some of her mail. This happened with Circuit City last month and I 'tried' talking to them about this but since their support apparently has been outsourced I got nowhere, the same as when I tried to talk to Earthlink about it. -- Chris pgpZyj3tcyIjB.pgp Description: PGP signature
Re: real or fake capital-one message
On Sun, 3 Dec 2006, Chris wrote:
I got this in my inbox today, I believe it to be real, however I'll post the
headers below. The reason I think it may be real is that there is some
person out there named Carol Pollock who for some reason and some how is
using the email address of [EMAIL PROTECTED] How, I haven't the
faintest clue. Here are the headers:
X-Spam-Untrusted: Relays [ ip=216.35.62.79
rdns=arm79.bigfootinteractive.com
helo=bigfootinteractive.com by=mx-bracke.atl.sa.earthlink.net ident=
envfrom= intl=0 id=1gQWIB30u3Nl34i6 auth= ]
X-Spam-Level:
X-Spam-RBL: Results dns:email.capitalone.com?type=MX [20
arm.bigfootinteractive.com.]
dns:email.capitalone.com [206.132.3.45]
Status: U
Return-Path: [EMAIL PROTECTED]
Received: from pop.earthlink.net [209.86.93.201]
by localhost with POP3 (fetchmail-6.2.5)
for [EMAIL PROTECTED] (single-drop); Sun, 03 Dec 2006 13:11:30
-0600 (CST)
Received: from bigfootinteractive.com ([216.35.62.79])
by mx-bracke.atl.sa.earthlink.net (EarthLink SMTP Server) with SMTP
[snip..]
I'd vote for this being a legit case of pilot error on the
original user's part. Much to their shame, CapitalOne -does- use
BFI for sending out many of their mailings.
I even had to go so far as to whitelist_from_rcvd [EMAIL PROTECTED]
sent via bigfootinteractive.com
Now to be fair, CapitalOne isn't the only culprit in this crime,
email.discovercard.com email.chase.com use BFI too.
Dave
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: real or fake capital-one message
Received: from bigfootinteractive.com ([216.35.62.79]) by mx-bracke.atl.sa.earthlink.net (EarthLink SMTP Server) with SMTP My first guess would be fake just from the headers. However, if it looks like legit opt-in stuff, then maybe it is. I suspect (assuming the person really exists) that their email address is similar to yours, and she fat-fingered you address instead of hers when entering the info on their web site. Then again, there are a whole lot of spammers that think I want property in Costa Rica and that my name is Jose Martinez. Loren
Re: Custom Rules
On Sun, 3 Dec 2006, Jaysen Johnson wrote: I have been asked by my boss to setup SpamAssassin on the corporate email server with the following rules. Date in the mail header more than 10 minutes out of sync - 1 point Date in the mail header more than 30 mintues out of sync - 2 points You need to gently adjust your boss' expectations for the promptness of email delivery. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Men by their constitutions are naturally divided in to two parties: 1. Those who fear and distrust the people and wish to draw all powers from them into the hands of the higher classes. 2. Those who identify themselves with the people, have confidence in them, cherish and consider them as the most honest and safe, although not the most wise, depository of the public interests. -- Thomas Jefferson --- 12 days until Bill of Rights day
Re: Best Choice for Bayes filtering on SpamAssassin
I use 4 in a load-balanced arrangement- They all share the same Mysql db. It is on another server. It works great and they all use the same bayes, awl and other mail settings used by policyd. Dave On Sat, 2006-12-02 at 19:24 +, Nigel Frankcom wrote: On Sat, 02 Dec 2006 18:31:47 +0100, Noc Phibee [EMAIL PROTECTED] wrote: Thanks to your answer Yes 6 server in load balancing with for all 70 concurrency incoming only for spam detect and 3 server for virus scan Michael Scheidell a écrit : -Original Message- From: Noc Phibee [mailto:[EMAIL PROTECTED] Sent: Saturday, December 02, 2006 8:35 AM To: users@spamassassin.apache.org Subject: Best Choice for Bayes filtering on SpamAssassin Hi i have 6 servers running on spamassassin 3.1.7 (now after a upgrades). Actually, all have Bayes Filering with local Db (default db, not sql) I want know what is the best choice ? : - Default Db or MySQL db ? MySql. Db can corrupt. Db WILL corrupt, and you can't replicate/share it. - 1 Bayes Db per server or 1 Bayes on Sql for all server (same database) If all 'round robin' mx, maybe 1 bayes on each server. If priority (mx 1, mx 2, mx... Etc) having one bayes per server will give a very jaded view of the world for server 6 (spammers go for highest # mx first. All it will get is spam) Also depends on why 6 servers, are all 6 the same? Load balancing? Failover? Backup mx's? different functions? (some do SA, some do cached dns, some do mysql, some do postfix?) My server receive 500 000/ 750 000 mails /days I have one getting 10MM per day. Configured right, you would really only need two servers, the other 4 make an update/configuration problem. With 3 servers, you could try mysql nbd database (I have not yet tried this) With 2, you could try mysql replication dual-master/slave(and deal with collisions, collision skips might not be a big deal) With 2, you might try memory devices, and 'mirror' the memory device which would hold the mysql server (I have not tried this, I don't think that a missing record or two on the bayes db is any bid deal) You COULD, once per day, just after expire, dump/load the Bayesian from 'master' to slave. Hi, I run multiple SA server fronts end with a single MySQL bayes backend and have done for a number of years. At some point I'll add Load Balancing to the SQL but at the moment it's on a stable box with little or nothing else to do. To date I've had no issue with it; though my mail throughput is a fraction of yours. The reason for multiple SA's is/was to cover downtime on any given server for maintenance. My MTA has a list of SA servers it will use in series; if 1 is unavailable it will got to 2 and so on. How this would work under the heavy loads you experience is open to debate. All I can say is that it's worked very well here. HTH Nigel
reporting joe-job bounces to razor/pyzor/dcc
I'm full of questions tonight. Looks like the joe-job against me is running full force again, thanks to the VBounce rule set they're not going into my spam folder as to be run against my reporting script. However, would it cause any harm if these were run against my other script which reports to razor/pyzor and dcc? -- Chris pgplU3gCVjMAe.pgp Description: PGP signature
Secure Quotes spam
Hi all, just a curiosity question: I seem to be getting an average of about 30 spams a week that all contain URLs that point to sites that look just like this (sample image, with several tabs with different URLs that point to identical copies of the same thing) http://www.doki-doki.net/~lamune/temp/spam1.png Does this look familiar to anyone? It seems pretty phishy to me, especially given that there's apparently no contact information on any of these pages. -Mike
Re: sa-learn and autolearn - working or not?
On Sun, Dec 03, 2006 at 05:09:11PM -0600, Dave Richardson wrote: intended effect? I have ZERO (none, nil) instances where header shows autolearn= as any other value than autolearn=no. This leads me conclude that my sa-learn data is not being utilized by spamd?! Define utilized. scanning and learning are different things. Is there a log or other way to peek into SA to have it tell me whether it's got the sa-learn data in the bayes engine and IS USING THAT INFORMATION? As with all things, run with -D. sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 11737 0 non-token data: nspam 0.000 0 23457 0 non-token data: nham 0.000 0 142588 0 non-token data: ntokens assuming this is the DB being accessed, it should be used for scanning. you'd see BAYES_* rule hits in the status header. bayes_path /home/spamd/.spamassassin that's an invalid path, it needs a file prefix. -- Randomly Selected Tagline: Today I set a motherboard on fire. Now the bizarre thing is that after the smoke cleared it still worked. - Alan Cox pgppxY4dJl98X.pgp Description: PGP signature
New Rule: OE_MULTIPART_RELATED
Hello list, For your consideration: header __MULTIPART_RELATED Content-Type =~ /multipart\/related/ meta OE_MULTIPART_RELATED (__OE_MUA __MULTIPART_RELATED) describe OE_MULTIPART_RELATED Possible image spam forged as from MS Outlook The false Positive rate on my corpus is 0.1%. I can't tell you about the false negative rate since I don't keep my spam (only my ham). This rule works very well on the pump-and-dump image spam that has been escaping my spamassassin installation for the last few months. Although Outlook Express is capable of generating messages with multipart/related MIME type, it only does that if the user creates an HTML message with inline images. This happens occasionally but rarely (hence the 0.1%). I expect the perceptron might give this rule a score of perhaps +0.5, which is not enough to catch the pump-and-dump image spam by itself, but works well in conjunction with Mail::SpamAssassin::Plugin::ImageInfo. Thoughts on this rule? --Ian Turner
How is LOCAL_AUTH_RCVD used?
Hi, I have a similar problem as the one recently reported by J. Rhett in thread skipping SPF checks for authenticated users. I'm trying to use Botnet plugin and make it not score for authenticated users; having the same for SPF and RBL would be even better. So the problem is that SA doesn't recognize that users are authenticated, I saw this document: http://wiki.apache.org/spamassassin/DynablockIssues which just says to add a LOCAL_AUTH_RCVD rule that matches your mail server, I did and it doesn't work as expected: SA matches the rule and adds a 1.0 score, the pseudo-header shows no authentication was recognized: dbg: metadata: X-Spam-Relays-Untrusted: [ ip=200.52.129.137 rdns=mail.legosoft.com.mx helo= by=cactus-soft.dyndns.org ident= [EMAIL PROTECTED] intl=0 id=J9POUJ-0001MC-JY auth= ] [ ip=189.149.70.163 rdns=dsl-189-149-70-163.prod-infinitum.com.mx helo=MARISELA by=mail.legosoft.com.mx ident= envfrom= intl=0 id=kB3G26P6019032 auth= ] Any help clarifying how the LOCAL_AUTH_RCVD rule is used, or an alternative to make SA recognize the authenticated user, will be appreciated. Using SA 3.1.7, under Solaris 9 with sendmail 8.13.8 and Windwos XP manually for testing. -- René Berber
SA 3.1.7 not picking up SQL-based Bayes
Hey folks, I'm finishing up a mailserver upgrade this weekend, and I notice that my new SQL-based install isn't picking up on user-based Bayes data. This is on a new, squeaky-clean OpenBSD 4.0-STABLE machine running on AMD64, using SpamAssassin 3.1.7 with perl 5.8.8. As per spamd -D info: 2006-12-03 22:41:53.760956500 [12889] dbg: config: retrieving prefs for [EMAIL PROTECTED] from SQL server OK, yay, spamd is picking up on the SQL userprefs. 2006-12-03 22:41:53.772480500 [12889] dbg: info: user has changed Not sure what this means? 2006-12-03 22:41:53.774209500 [12889] dbg: bayes: using username: [EMAIL PROTECTED] 2006-12-03 22:41:53.781308500 [12889] dbg: bayes: database connection established 2006-12-03 22:41:53.786485500 [12889] dbg: bayes: found bayes db version 3 2006-12-03 22:41:53.789654500 [12889] dbg: bayes: unable to initialize database for [EMAIL PROTECTED] user, aborting! 2006-12-03 22:41:54.117388500 [12889] dbg: bayes: not scoring message, returning undef 2006-12-03 22:41:54.118260500 [12889] dbg: bayes: opportunistic call attempt failed, DB not readable Uh. What does unable to initialize database mean? Spamd has already successfully connected to the PostgreSQL database above, right? So what does initializing database mean? My user_scores_sql_custom_query is as follows, if that makes a difference (not sure if that's consulted for Bayes data): user_scores_sql_custom_querySELECT preference, value FROM userpref WHERE username = _MAILBOX_ OR username = _USERNAME_ OR username = '$GLOBAL' ORDER BY user name ASC; To add insult to injury, learning spam and ham work just fine. It's just the Bayes scoring that seems to have issues. So. I'm at a loss at the moment... My SA install is doing well, but not as well as it should, if it's ignoring Bayes. What info can I pass along to help diagnose this problem? Thanks much! Benny -- If stupidity were a handicap, you'd have the best parking spot. --Bill Paul
Re: SA 3.1.7 not picking up SQL-based Bayes
C. Bensend wrote: Hey folks, I'm finishing up a mailserver upgrade this weekend, and I notice that my new SQL-based install isn't picking up on user-based Bayes data. This is on a new, squeaky-clean OpenBSD 4.0-STABLE machine running on AMD64, using SpamAssassin 3.1.7 with perl 5.8.8. As per spamd -D info: 2006-12-03 22:41:53.760956500 [12889] dbg: config: retrieving prefs for [EMAIL PROTECTED] from SQL server OK, yay, spamd is picking up on the SQL userprefs. 2006-12-03 22:41:53.772480500 [12889] dbg: info: user has changed Not sure what this means? 2006-12-03 22:41:53.774209500 [12889] dbg: bayes: using username: [EMAIL PROTECTED] 2006-12-03 22:41:53.781308500 [12889] dbg: bayes: database connection established 2006-12-03 22:41:53.786485500 [12889] dbg: bayes: found bayes db version 3 2006-12-03 22:41:53.789654500 [12889] dbg: bayes: unable to initialize database for [EMAIL PROTECTED] user, aborting! 2006-12-03 22:41:54.117388500 [12889] dbg: bayes: not scoring message, returning undef 2006-12-03 22:41:54.118260500 [12889] dbg: bayes: opportunistic call attempt failed, DB not readable Uh. What does unable to initialize database mean? Spamd has already successfully connected to the PostgreSQL database above, right? So what does initializing database mean? My user_scores_sql_custom_query is as follows, if that makes a difference (not sure if that's consulted for Bayes data): user_scores_sql_custom_querySELECT preference, value FROM userpref WHERE username = _MAILBOX_ OR username = _USERNAME_ OR username = '$GLOBAL' ORDER BY user name ASC; To add insult to injury, learning spam and ham work just fine. It's just the Bayes scoring that seems to have issues. So. I'm at a loss at the moment... My SA install is doing well, but not as well as it should, if it's ignoring Bayes. What info can I pass along to help diagnose this problem? I think its just a slightly confusing message. If you run: sa-learn -u [EMAIL PROTECTED] Does it show that you have 200 ham and 200 spam in the database? If so then there is a problem, if not you just need to train it some more. What the WARNING is telling you is that hey this database isn't ready for scoring so I'm not gonna use it. This is why learning works just fine. Finish training up the DB and see if it then starts working for you. Michael PS Possibly we should get the warning text changed a bit, feel free to open up a bug so we can track the work, thanks. Thanks much! Benny
Re: How is LOCAL_AUTH_RCVD used?
Rene, you can score the rule to be lower. For instance, a score of -10 will probably do what you need. It doesn't prevent the utilization, but does solve the problem of having your local users get +1 points for authenticating. The long-term problem needs a real fix. René Berber wrote: Hi, I have a similar problem as the one recently reported by J. Rhett in thread skipping SPF checks for authenticated users. I'm trying to use Botnet plugin and make it not score for authenticated users; having the same for SPF and RBL would be even better. So the problem is that SA doesn't recognize that users are authenticated, I saw this document: http://wiki.apache.org/spamassassin/DynablockIssues which just says to add a LOCAL_AUTH_RCVD rule that matches your mail server, I did and it doesn't work as expected: SA matches the rule and adds a 1.0 score, the pseudo-header shows no authentication was recognized: dbg: metadata: X-Spam-Relays-Untrusted: [ ip=200.52.129.137 rdns=mail.legosoft.com.mx helo= by=cactus-soft.dyndns.org ident= [EMAIL PROTECTED] intl=0 id=J9POUJ-0001MC-JY auth= ] [ ip=189.149.70.163 rdns=dsl-189-149-70-163.prod-infinitum.com.mx helo=MARISELA by=mail.legosoft.com.mx ident= envfrom= intl=0 id=kB3G26P6019032 auth= ] Any help clarifying how the LOCAL_AUTH_RCVD rule is used, or an alternative to make SA recognize the authenticated user, will be appreciated. Using SA 3.1.7, under Solaris 9 with sendmail 8.13.8 and Windwos XP manually for testing. -- Jo Rhett Network/Software Engineer Net Consonance
Re: SA 3.1.7 not picking up SQL-based Bayes
I think its just a slightly confusing message. If you run:
sa-learn -u [EMAIL PROTECTED]
Does it show that you have 200 ham and 200 spam in the database? If so
then there is a problem, if not you just need to train it some more.
What the WARNING is telling you is that hey this database isn't ready
for scoring so I'm not gonna use it. This is why learning works just
fine. Finish training up the DB and see if it then starts working for
you.
Michael
PS Possibly we should get the warning text changed a bit, feel free to
open up a bug so we can track the work, thanks.
Hi Michael,
Well, I have the following in the script that runs every now and
again, to execute sa-learn:
[EMAIL PROTECTED] ~]$ sa-learn --dump magic | grep non-token data: nham |
awk '{ print $3 }'
257526
[EMAIL PROTECTED] ~]$ sa-learn --dump magic | grep non-token data: nspam |
awk '{ print $3 }'
470150
I'm fairly sure I have enough ham and spam. :) Also, I'm watching
the PostgreSQL logfile when I do that, and it _is_ querying the
database.
Just for argument's sake, I checked for *BAYES* in the spamd logfile,
and I don't get a single hit. So, Bayes is definately not working
for _any_ of the accounts, not just mine. :(
Thanks for any insight,
Benny
--
If stupidity were a handicap, you'd have the best parking spot.
--Bill Paul
Re: SA 3.1.7 not picking up SQL-based Bayes
C. Bensend wrote:
I think its just a slightly confusing message. If you run:
sa-learn -u [EMAIL PROTECTED]
Does it show that you have 200 ham and 200 spam in the database? If so
then there is a problem, if not you just need to train it some more.
What the WARNING is telling you is that hey this database isn't ready
for scoring so I'm not gonna use it. This is why learning works just
fine. Finish training up the DB and see if it then starts working for
you.
Michael
PS Possibly we should get the warning text changed a bit, feel free to
open up a bug so we can track the work, thanks.
Hi Michael,
Well, I have the following in the script that runs every now and
again, to execute sa-learn:
[EMAIL PROTECTED] ~]$ sa-learn --dump magic | grep non-token data: nham |
awk '{ print $3 }'
257526
[EMAIL PROTECTED] ~]$ sa-learn --dump magic | grep non-token data: nspam |
awk '{ print $3 }'
470150
I'm fairly sure I have enough ham and spam. :) Also, I'm watching
the PostgreSQL logfile when I do that, and it _is_ querying the
database.
Ahh but you didn't run the command I asked you to run. You are passing
the user: [EMAIL PROTECTED] to SpamAssassin so it will use that as
the key for the database, running the command from the command like that
way is going to use your unix id as the key. I'm guessing you changed
something in your mail setup to start passing in @domain in addition to
the regular unix username.
Michael
Just for argument's sake, I checked for *BAYES* in the spamd logfile,
and I don't get a single hit. So, Bayes is definately not working
for _any_ of the accounts, not just mine. :(
Thanks for any insight,
Benny
Re: SA 3.1.7 not picking up SQL-based Bayes
Ahh but you didn't run the command I asked you to run. You are passing the user: [EMAIL PROTECTED] to SpamAssassin so it will use that as the key for the database, running the command from the command like that way is going to use your unix id as the key. I'm guessing you changed something in your mail setup to start passing in @domain in addition to the regular unix username. Actually, yes, I did, but I don't think it turned out like we were expecting (hence I didn't include it, I'm sorry): [EMAIL PROTECTED] ~]$ sa-learn -u [EMAIL PROTECTED] SpamAssassin version 3.1.7 Please select either --spam, --ham, --folders, --forget, --sync, --import, --dump, --clear, --backup or --restore Usage: sa-learn [options] [file]... sa-learn [options] --dump [ all | data | magic ] Options: --ham Learn messages as ham (non-spam) --spamLearn messages as spam --forget Forget a message --use-ignores Use bayes_ignore_from and bayes_ignore_to --syncSyncronize the database and the journal if needed --force-expireForce a database sync and expiry run --dbpath path Allows commandline override (in bayes_path form) for where to read the Bayes DB from --dump [all|data|magic] Display the contents of the Bayes database Takes optional argument for what to display --regexp reFor dump only, specifies which tokens to dump based on a regular expression. -f file, --folders=file Read list of files/directories from file --dir Ignored; historical compatability --fileIgnored; historical compatability --mboxInput sources are in mbox format --mbx Input sources are in mbx format --showdotsShow progress using dots --no-sync Skip syncronizing the database and journal after learning -L, --local Operate locally, no network accesses --import Migrate data from older version/non DB_File based databases --clear Wipe out existing database --backup Backup, to STDOUT, existing database --restore filename Restore a database from filename -u username, --username=username Override username taken from the runtime environment -C path, --configpath=path, --config-file=path Path to standard configuration dir -p prefs, --prefspath=file, --prefs-file=fileSet user preferences file --siteconfigpath=path Path for site configs (def: /etc/mail/spamassassin) -D, --debug-level Print debugging messages -V, --version Print version -h, --helpPrint usage message But regardless - won't the user_scores_sql_custom_query I posted handle that possibility? I am _so_ not an SQL guru, but it looks correct to me? I'm never afraid to admit a mistake, so if I'm smoking crack here, please step up and say so. :) Benny -- If stupidity were a handicap, you'd have the best parking spot. --Bill Paul
Re: SA 3.1.7 not picking up SQL-based Bayes
C. Bensend wrote: Ahh but you didn't run the command I asked you to run. You are passing the user: [EMAIL PROTECTED] to SpamAssassin so it will use that as the key for the database, running the command from the command like that way is going to use your unix id as the key. I'm guessing you changed something in your mail setup to start passing in @domain in addition to the regular unix username. Actually, yes, I did, but I don't think it turned out like we were expecting (hence I didn't include it, I'm sorry): [EMAIL PROTECTED] ~]$ sa-learn -u [EMAIL PROTECTED] add the rest of you --dump magic command to that. But regardless - won't the user_scores_sql_custom_query I posted handle that possibility? I am _so_ not an SQL guru, but it looks correct to me? I'm never afraid to admit a mistake, so if I'm smoking crack here, please step up and say so. :) That custom query has nothing to do with bayes or awl sql stuffs. Michael Benny
Re: SA 3.1.7 not picking up SQL-based Bayes
add the rest of you --dump magic command to that. Right. Duh me. Heh. The following was captured via -D: [20507] dbg: bayes: using username: [EMAIL PROTECTED] [20507] dbg: bayes: database connection established [20507] dbg: bayes: found bayes db version 3 [20507] dbg: bayes: unable to initialize database for [EMAIL PROTECTED] user, aborting! [20507] dbg: config: score set 0 chosen. [20507] dbg: bayes: database connection established [20507] dbg: bayes: found bayes db version 3 [20507] dbg: bayes: unable to initialize database for [EMAIL PROTECTED] user, aborting! ERROR: Bayes dump returned an error, please re-run with -D for more information That custom query has nothing to do with bayes or awl sql stuffs. Gotcha. Thanks. Thanks for taking a look at this, Michael, Benny -- If stupidity were a handicap, you'd have the best parking spot. --Bill Paul
Re: How is LOCAL_AUTH_RCVD used?
Jo Rhett wrote: Rene, you can score the rule to be lower. For instance, a score of -10 will probably do what you need. I know that, I saw your post and I have been changing scores to fine tune my installation (for instance I give -2.5 points to any SPF validated server). It doesn't prevent the utilization, but does solve the problem of having your local users get +1 points for authenticating. The long-term problem needs a real fix. Exactly, and the Wiki page I used probably needs to be corrected or updated. Thanks for your reply. [snip] -- René Berber
Re: New Rule: OE_MULTIPART_RELATED
Hello list, For your consideration: header __MULTIPART_RELATED Content-Type =~ /multipart\/related/ meta OE_MULTIPART_RELATED (__OE_MUA __MULTIPART_RELATED) describe OE_MULTIPART_RELATED Possible image spam forged as from MS Outlook The false Positive rate on my corpus is 0.1%. I can't tell you about the false negative rate since I don't keep my spam (only my ham). This rule works very well on the pump-and-dump image spam that has been escaping my spamassassin installation for the last few months. Although Outlook Express is capable of generating messages with multipart/related MIME type, it only does that if the user creates an HTML message with inline images. This happens occasionally but rarely (hence the 0.1%). I expect the perceptron might give this rule a score of perhaps +0.5, which is not enough to catch the pump-and-dump image spam by itself, but works well in conjunction with Mail::SpamAssassin::Plugin::ImageInfo. Thoughts on this rule? --Ian Turner Hi Ian, this would trap mail using outlook stationery. I dont really like it, but I get it in wanted mail. Generally I believe that rules scoring valid use of mail (cid addressing, mime types) should be avoided - unless you want to block, e.g., mails with images or mails sent from outlook generally Rather try to find a subtle difference in the way real outlook builds the message and the spammers do it, that would really reveal it is not from outlook Wolfgang Hamann
