Re: sa-learn not learning?
Hi John, Looks like yo replied directly to me. I couldn't find your reply on the list yet? At any rate... The Bayes DB has been learned and in effect for a long time - years before my time. No ID's have changed or the config that has caused this error. I add users to the whitelist - and use sa-learn - that's it. 1. [EMAIL PROTECTED] spam-email]$ sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 797361 0 non-token data: nspam 0.000 0 665377 0 non-token data: nham 0.000 0 186483 0 non-token data: ntokens 0.000 0 1203464108 0 non-token data: oldest atime 0.000 0 1203536991 0 non-token data: newest atime 0.000 0 1203536443 0 non-token data: last journal sync atime 0.000 0 1203507419 0 non-token data: last expiry atime 0.000 0 43200 0 non-token data: last expire atime delta 0.000 0 101794 0 non-token data: last expire reduction count 2. sa-learn running as amavis. [EMAIL PROTECTED] spam-email]$ id uid=503(amavis) gid=504(amavis) groups=504(amavis) 3. I think we are filtering with Spamd - how can I tell - in a config file or dir? (/etc/mail./spamassasin or /var/amavis/.spamassassin) I have both binaries... [EMAIL PROTECTED] spam-email]$ which spamd /usr/bin/spamd [EMAIL PROTECTED] spam-email]$ which spamc /usr/bin/spamc 4. [EMAIL PROTECTED] root]# ps axu | grep spamd root 18580 0.0 0.1 1736 588 pts/2S14:00 0:00 grep spamd Scott Pichelman Systems Administrator Weir Minerals North America 2701 S Stoughton Rd Madison WI 53716 USA T: +(00)1 608 226 5615 F: +(00)1 608 221 5807 M: +(00)1 608 279 5056 E: [EMAIL PROTECTED] W: www.weirminerals.com John Hardin [EMAIL PROTECTED] 02/20/2008 01:43 PM To pichels [EMAIL PROTECTED] cc users@spamassassin.apache.org Subject Re: sa-learn not learning? On Wed, 20 Feb 2008, pichels wrote: But, I've tried learning any email after I recieved the Perl error message and none are being learned? And why is the spam being scored wioth spamassassin? I don't understand? Could my Bayes DB need to be re-synced or forced to expire some dups or ? Note that bayes needs at least 200 spams and 200 hams before is starts scoring. Have you learned that many yet? If you have kept your training corpus, you could delete the bayes database files entirely and start training over from scratch. My users are getting the nice girl emails and they are not scoring as I've shown in my post - why? They score with spamassassin debug but are not being stopped by SA in my maillogs? That smells like a user ID problem. If the user ID that spamassassin/spamd is running under is different than the user ID you are running sa-learn under, the bayes databases are different - you're training a database that SA isn't looking at. Verify that you are training using the same user as the user spamassassin/spamd is running as to filter mail. Can I provide more details? What does sa-learn --dump magic report? How are you filtering messages? spamc+spamd? What user is spamd running as? What user are you running sa-learn as? What (if anything) does ps axu | grep spamd report? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- [Small arms] are fundamentally dangerous and their removal from the equation either by control, neutralisation or removal is essential. The first step is to gain information on their numbers and whereabouts. -- the UN, who doesn't want to confiscate guns --- 2 days until George Washington's 276th Birthday This document should only be read by those persons to whom it is addressed and is not intended to be relied upon by any person without subsequent written confirmation of its contents. Accordingly, our company disclaim all responsibility and accept no liability (including in negligence) for the consequences for any person acting, or refraining from acting, on such information prior to the receipt by those persons of subsequent written confirmation., If you have received this E-mail message in error, please notify us immediately by telephone. Please also destroy and delete the message from your computer., Any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this E-mail message is strictly prohibited.
Re: sa-learn not learning?
Hi, Scott, I'll give you my two cents here 2008/2/20, [EMAIL PROTECTED] [EMAIL PROTECTED]: Hi John, Looks like yo replied directly to me. I couldn't find your reply on the list yet? At any rate... The Bayes DB has been learned and in effect for a long time - years before my time. No ID's have changed or the config that has caused this error. I add users to the whitelist - and use sa-learn - that's it. 1. [EMAIL PROTECTED] spam-email]$ sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 797361 0 non-token data: nspam 0.000 0 665377 0 non-token data: nham 0.000 0 186483 0 non-token data: ntokens 0.000 0 1203464108 0 non-token data: oldest atime 0.000 0 1203536991 0 non-token data: newest atime 0.000 0 1203536443 0 non-token data: last journal sync atime 0.000 0 1203507419 0 non-token data: last expiry atime 0.000 0 43200 0 non-token data: last expire atime delta 0.000 0 101794 0 non-token data: last expire reduction count 2. sa-learn running as amavis. [EMAIL PROTECTED] spam-email]$ id uid=503(amavis) gid=504(amavis) groups=504(amavis) 3. I think we are filtering with Spamd - how can I tell - in a config file or dir? (/etc/mail./spamassasin or /var/amavis/.spamassassin) I have both binaries... [EMAIL PROTECTED] spam-email]$ which spamd /usr/bin/spamd [EMAIL PROTECTED] spam-email]$ which spamc /usr/bin/spamc 4. [EMAIL PROTECTED] root]# ps axu | grep spamd root 18580 0.0 0.1 1736 588 pts/2S14:00 0:00 grep spamd Amavis loads the pertinent SA routines and code by itself, it doesn't call SA OR Spamd at any moment. From what I've read, you SA-Amavis duo has been running from some time ago. Anyway, I recommend you read the HOWTO by Gary V. It has some interesting notes about the users under Amavis runs, and other valuable material. It's located here: http://www200.pair.com/mecham/spam/ You could try running Amavis in debug mode (i.e., stop amavis and from the command line type: *# amavisd debug*-*sa * That will show you how Amavis treats the message. I do also suggest rising the detail level in Amavis' logs Anyway, my answer is getting totally OT here. You might have more luck asking in the Amavis list. Hope this helps, Luis Scott Pichelman Systems Administrator Weir Minerals North America 2701 S Stoughton Rd Madison WI 53716 USA T: +(00)1 608 226 5615 F: +(00)1 608 221 5807 M: +(00)1 608 279 5056 E: [EMAIL PROTECTED] W: www.weirminerals.com *John Hardin [EMAIL PROTECTED]* 02/20/2008 01:43 PM To pichels [EMAIL PROTECTED] cc users@spamassassin.apache.org Subject Re: sa-learn not learning? On Wed, 20 Feb 2008, pichels wrote: But, I've tried learning any email after I recieved the Perl error message and none are being learned? And why is the spam being scored wioth spamassassin? I don't understand? Could my Bayes DB need to be re-synced or forced to expire some dups or ? Note that bayes needs at least 200 spams and 200 hams before is starts scoring. Have you learned that many yet? If you have kept your training corpus, you could delete the bayes database files entirely and start training over from scratch. My users are getting the nice girl emails and they are not scoring as I've shown in my post - why? They score with spamassassin debug but are not being stopped by SA in my maillogs? That smells like a user ID problem. If the user ID that spamassassin/spamd is running under is different than the user ID you are running sa-learn under, the bayes databases are different - you're training a database that SA isn't looking at. Verify that you are training using the same user as the user spamassassin/spamd is running as to filter mail. Can I provide more details? What does sa-learn --dump magic report? How are you filtering messages? spamc+spamd? What user is spamd running as? What user are you running sa-learn as? What (if anything) does ps axu | grep spamd report? -- John Hardin KA7OHZ http://www.impsec.org/~jhardin/http://www.impsec.org/%7Ejhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- [Small arms] are fundamentally dangerous and their removal from the equation either by control, neutralisation or removal is essential. The first step is to gain information on their numbers and whereabouts. -- the UN, who doesn't want to confiscate guns --- 2 days until George Washington's 276th Birthday This document should only be read
Re: sa-learn not learning?
On Wed, 20 Feb 2008, pichels wrote: Then, when I tried to run sa-learn - none of my messages are learning? [EMAIL PROTECTED] spam-email]$ sa-learn --spam mosconj-022008-1 Learned tokens from 0 message(s) (1 message(s) examined) That just means that sa-learn has already learned that message and is ignoring it. It does not by itself indicate a problem exists. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The third basic rule of firearms safety: Keep your booger hook off the bang switch! --- 2 days until George Washington's 276th Birthday
Re: sa-learn not learning?
Hi John, But, I've tried learning any email after I recieved the Perl error message and none are being learned? And why is the spam being scored wioth spamassassin? I don't understand? Could my Bayes DB need to be re-synced or forced to expire some dups or ? My users are getting the nice girl emails and they are not scoring as I've shown in my post - why? They score with spamassassin debug but are not being stopped by SA in my maillogs? Can I provide more details? Thanks, pichels John Hardin wrote: On Wed, 20 Feb 2008, pichels wrote: Then, when I tried to run sa-learn - none of my messages are learning? [EMAIL PROTECTED] spam-email]$ sa-learn --spam mosconj-022008-1 Learned tokens from 0 message(s) (1 message(s) examined) That just means that sa-learn has already learned that message and is ignoring it. It does not by itself indicate a problem exists. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The third basic rule of firearms safety: Keep your booger hook off the bang switch! --- 2 days until George Washington's 276th Birthday -- View this message in context: http://www.nabble.com/sa-learn-%22not%22-learning--tp15594114p15594744.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: sa-learn not learning?
On Wed, Feb 20, 2008 at 09:24:28AM -0800, pichels wrote: /usr/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Message/Node.pm line 119. Learned tokens from 0 message(s) (1 message(s) examined) So, I found a post that explained this was a bug and to pach the Node.pm file and so I patched it: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5823 bug 5823 isn't related to Message::Node [EMAIL PROTECTED] spam-email]$ sa-learn --spam mosconj-022008-1 Learned tokens from 0 message(s) (1 message(s) examined) It could mean that the message was already learned. However, why does SA with debug and lint score the email? * 2.5 MISSING_HB_SEP Missing blank line between message header and body It is exceedingly likely that your input message is malformed. If you can pastebot the message somewhere, we can look at it for you and let you know. -- Randomly Selected Tagline: That's why I don't like video's - no commercial to read through. - Robert Gombert pgpjjyMarwls4.pgp Description: PGP signature
Re: sa-learn not learning?
On Wed, 20 Feb 2008, pichels wrote: But, I've tried learning any email after I recieved the Perl error message and none are being learned? And why is the spam being scored wioth spamassassin? I don't understand? Could my Bayes DB need to be re-synced or forced to expire some dups or ? Note that bayes needs at least 200 spams and 200 hams before is starts scoring. Have you learned that many yet? If you have kept your training corpus, you could delete the bayes database files entirely and start training over from scratch. My users are getting the nice girl emails and they are not scoring as I've shown in my post - why? They score with spamassassin debug but are not being stopped by SA in my maillogs? That smells like a user ID problem. If the user ID that spamassassin/spamd is running under is different than the user ID you are running sa-learn under, the bayes databases are different - you're training a database that SA isn't looking at. Verify that you are training using the same user as the user spamassassin/spamd is running as to filter mail. Can I provide more details? What does sa-learn --dump magic report? How are you filtering messages? spamc+spamd? What user is spamd running as? What user are you running sa-learn as? What (if anything) does ps axu | grep spamd report? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- [Small arms] are fundamentally dangerous and their removal from the equation either by control, neutralisation or removal is essential. The first step is to gain information on their numbers and whereabouts. -- the UN, who doesn't want to confiscate guns --- 2 days until George Washington's 276th Birthday
Re: sa-learn not learning?
On Wed, 20 Feb 2008, [EMAIL PROTECTED] wrote: The Bayes DB has been learned and in effect for a long time - years before my time. No ID's have changed or the config that has caused this error. I add users to the whitelist - and use sa-learn - that's it. ok. 0.000 0 797361 0 non-token data: nspam 0.000 0 665377 0 non-token data: nham ok. 2. sa-learn running as amavis. [EMAIL PROTECTED] spam-email]$ id uid=503(amavis) gid=504(amavis) groups=504(amavis) 3. I think we are filtering with Spamd - how can I tell - in a config file or dir? (/etc/mail./spamassasin or /var/amavis/.spamassassin) I have both binaries... I'm not sure, I'm not familiar with how amavisd hooks into SA. 4. [EMAIL PROTECTED] root]# ps axu | grep spamd root 18580 0.0 0.1 1736 588 pts/2S14:00 0:00 grep spamd That seems to indicate spamd isn't in use. Hmm. Do you have autolearn turned on? Perhaps the messages were already automatically learned and are being ignored when you later try to manually train. You said those sa-learn errors started happening recently - was SA updated? Was perl updated? Grab a missed spam out of your mailbox and run it through SA with debug, then compare the score to what it got when it went through your MTA. As Theo noted, the earlier example looked misformatted enough (the MISSING_* hits) to push the score over the threshold. And given that the bug was not related to Node.pm, did you keep the unpatched Node.pm around? You might want to consider upgrading to 3.1.9, if you're not willing to go to 3.2.x yet... -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The fetters imposed on liberty at home have ever been forged out of the weapons provided for defense against real, pretended, or imaginary dangers from abroad. -- James Madison, 1799 --- 2 days until George Washington's 276th Birthday
Re: sa-learn not learning with sudo
On Sat, Apr 22, 2006 at 10:55:29AM +0200, Michael Monnerie wrote: ... # sudo -H -u vscan sa-learn --dump ... But when I do # su -l vscan ... # sudo -H -u vscan sa-learn --dump ... Now why is there a diff between sudo as a user or directly logging in as One of the differences will be all the commands in the User's shell-startup-Files! Those are ignored, if you run the command directly by sudo. It also depends on the version of 'sudo', because one of the latest changes *dropped* the HOME-Variable from the environment (at least if you run the command directly from sudo!). Lots of our automated cron-scripts suddenly failed by this 'security fix' and we had to replace OLD: sudo command NEW: sudo env HOME=$HOME command to 'bridge the gap' and re-use the *current* HOME 'inside of sudo'. May be the 'sudo -l vscan' also sets the missing HOME! YoursStucki (postmaster hit by the same? :-)
Re: sa-learn not learning with sudo
Michael Monnerie wrote: Hi, I've setup a new server, and made # sudo -H -u vscan sa-learn --ham --progress --no-sync --mbox /tmp/ham.txt 3.59 msgs/sec 31m43s DONE Learned tokens from 6842 message(s) (6842 message(s) examined) I don't know why it's not working, but the FIRST thing to check is what files are being updated, if any, when this command runs. Check both: /home/vscan/.spamassassin/bayes_db_toks /root/.spamassassin/bayes_db_toks (Note:, you changed the bayes path to cause this slightly abnormal naming. Normally the file would be bayes_toks. I'm really not sure why you over-rode bayes_path with such a subtle change from the default, but that is the result.) And the same for SPAM, and then I have this: # sudo -H -u vscan sa-learn --dump 0.000 0 0 0 non-token data: nspam 0.000 0 0 0 non-token data: nham But when I do # su -l vscan [EMAIL PROTECTED]:~ sa-learn --ham --progress --no-sync --mbox /tmp/ham.txt 6.32 msgs/sec 18m03s DONE Learned tokens from 6842 message(s) (6842 message(s) examined) I have: # sudo -H -u vscan sa-learn --dump 0.000 0 6726 0 non-token data: nspam 0.000 0 7023 0 non-token data: nham Now why is there a diff between sudo as a user or directly logging in as the user? My local.cf has this entries: loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold Do NOT put loadplugin statements in your local.cf. These belong in init.pre. bayes_path ~/.spamassassin/bayes_db bayes_expiry_max_db_size2000 bayes_file_mode 0777 Note: you don't need the 0777 normally unless you're forcing bayes path to a non-home-directory-relative path that multiple users share. ie: /var/spamassassin/bayes bayes_auto_expire 0 Note: I hope you have a cron job that kicks of sa-learn --force-expire. With bayes_auto_expire disabled there's nothing else to keep the bayes DB from growing without bound. bayes_auto_learn_threshold_spam 8.00 bayes_auto_learn_threshold_nonspam 0.1 mfg zmi
