Re: No magic, since clearing database
On 10/23/2012 at 8:24 PM, John Hardin jhar...@impsec.org wrote: On Tue, 23 Oct 2012, Joseph Acquisto wrote: On 10/23/2012 at 6:33 PM, Joseph Acquisto j...@j4computers.com wrote: On 10/23/2012 at 11:18 AM, John Hardin jhar...@impsec.org wrote: On Tue, 23 Oct 2012, Joseph Acquisto wrote: On 10/23/2012 at 6:02 AM, Joseph Acquisto j...@j4computers.com wrote: On 10/22/2012 at 8:30 PM, John Hardin jhar...@impsec.org wrote: On Mon, 22 Oct 2012, Joseph Acquisto wrote: I just noticed this in /var/log/messages: Oct 22 20:20:11 mybox spamd[31966]: config: SpamAssassin failed to parse line, /etc/mail/spamassassin/bayes_db/bayes/ is not valid for bayes_path, skipping: bayes_path /etc/mail/spamassassin/bayes_db/bayes/ Lose the trailing slash. This always bites people. What you're specifying is a directory _AND_ a filename prefix, not just a directory. No more error, but still updates /root/.spamassassin/bayes_stuff Huh. Indeed. It's at least creating the files now. I had created /etc/mail/spamassassin/bayes_db/bayes When I removed bayes (as a directory) . . . well . . .ahem. Hey, you're helping improve the wiki entry for this... :) I'm, uhhh, overjoyed . . . No, really. Yeah, that's the ticket. OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. I can sleep peacefully tonight. joe a.
URIBL_BLOCKED
Anybody else getting this this morning?
Re: [solved] SA-3.3.2 options max-spare and max-children doesn't work as i expect
W dniu 23.10.2012 22:24, RW pisze: Hi, On reading you your question more thoroughly I see that your main point was that you aren't getting as many processes as expected. The number of child processes isn't adjusted immediately, it's incremented or decremented when a child announces that it is idle. Testing with only six calls isn't enough to expect sensible results. What you need to do is hammer spamd with lots more spamc calls and watch the number of child processes evolve in real time - maybe have the background processes log the child count as each spamc process completes. Indeed, I've flooded spamd with many connections. In results I got as much spamd processess as I defined using -m option. Thanks for tip, now all is clear for me. Regards, Marcin
Re: SA wiki
On 10/23, Joseph Acquisto wrote: at http://wiki.apache.org/spamassassin/SiteWideBayesFeedback the link a cookbook to setup site wide ham/spam forwarding for postfix http://gtmp.org/publications/sa-postfix-en;, links to topic does not exist yet. It apparently got deleted. The page is available in archive.org, a very useful tool. Anybody can edit the wiki, just create an account and email the dev list asking for write access. This is mentioned at the bottom of the front page of the SA wiki, but I know it's not very obvious, I missed it myself. You could also try contacting the owner of gtmp.org. -- Just because you're offended, doesn't mean you're right. - Ricky Gervais http://www.ChaosReigns.com
Re: SA wiki
On 10/24/2012 9:31 AM, dar...@chaosreigns.com wrote: On 10/23, Joseph Acquisto wrote: at http://wiki.apache.org/spamassassin/SiteWideBayesFeedback the link a cookbook to setup site wide ham/spam forwarding for postfix http://gtmp.org/publications/sa-postfix-en;, links to topic does not exist yet. It apparently got deleted. The page is available in archive.org, a very useful tool. Anybody can edit the wiki, just create an account and email the dev list asking for write access. This is mentioned at the bottom of the front page of the SA wiki, but I know it's not very obvious, I missed it myself. You could also try contacting the owner of gtmp.org. Damnit, I'm sorry Darxus. I researched this yesterday and accidentally took it off-list. Thanks. Appears to be at http://gtmp.org/doku.php/publications:sa-postfix-en now I fixed the wiki. regards, KAM
Re: No magic, since clearing database
On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using?
Re: URIBL_BLOCKED
On 10/24/2012 6:37 AM, Jared Hall wrote: Anybody else getting this this morning? Need more information but off the cuff it sounds like you are blocked because you aren't using a locally cached copy of an RBL or you've exceed an RBL's free limits.
Re: BAYES_99 score
On 22/10/12 19:15, dar...@chaosreigns.com wrote: On 10/22, JP Kelly wrote: Should I set the BAYES_99 score high enough to trigger as spam? I get plenty of spam getting through which does not get caught because BAYES_99 is the only rule which fires and it is not set to score at or above the threshold. You could. Some people only use bayesian filtering, which would be similar. The important question is, how many false positives (non-spams flagged as spams) would that cause? SpamAssassin's automated scoring attempts to achieve 1 false positive in 2,500 non-spams, with a score threshold of 5.0. So if you don't have an absolute minimum of 2,500 representative non-spams to check for having hit BAYES_99, you risk increasing your false positives. But it's your risk to take. I have had very good success running adjusted scores for BAYES rules, but I am very careful how I train my bayes database. I've disabled auto-learning and only manually train on hand-checked ham and spam examples. Consequently, I find the extremes (BAYES_99 and BAYES_00) to be highly reliable indicators.
Re: No magic, since clearing database
Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a.
Re: BAYES_99 score
24.10.2012 18:19, Ned Slider kirjoitti: I have had very good success running adjusted scores for BAYES rules, but I am very careful how I train my bayes database. I've disabled auto-learning and only manually train on hand-checked ham and spam examples. Consequently, I find the extremes (BAYES_99 and BAYES_00) to be highly reliable indicators. I have never seen false BAYES_99, but false BAYES_00 is not that rare. -- You learn to write as if to someone else because NEXT YEAR YOU WILL BE SOMEONE ELSE. signature.asc Description: OpenPGP digital signature
Re: No magic, since clearing database
On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM
Re: No magic, since clearing database
Kevin A. McGrail kmcgr...@pccc.com 10/24/12 11:55 AM On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM Will try. How can I trap/redirect the -D output? In entirety? I use putty to access the box and cannot sroll back all the way to the beginning of output. Perhaps there is a secret to that, as well. joe a.
Re: No magic, since clearing database
On 10/24/2012 12:48 PM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 11:55 AM On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM Will try. How can I trap/redirect the -D output? In entirety? I use putty to access the box and cannot sroll back all the way to the beginning of output. Perhaps there is a secret to that, as well. Edit your Putty settings. In the Window section there is an option for Lines of scrollback. I set mine to 20. That way, I can always scroll back to see or copy whatever I need to. -- Bowie
Re: No magic, since clearing database
On Wed, 2012-10-24 at 12:48 -0400, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 11:55 AM On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM Will try. How can I trap/redirect the -D output? In entirety? I use putty to access the box and cannot sroll back all the way to the beginning of output. Perhaps there is a secret to that, as well. Run the sa-learn command like this: sa-learn -D your usual arguments 21 | tee logfile.txt and then use ftp or PuTTY's scp command to copy logfile.txt back to your PC. Martin
Re: No magic, since clearing database
On 10/24/2012 12:48 PM, Joseph Acquisto wrote:
How can I trap/redirect the -D output? In entirety? I use putty to
access the box and cannot sroll back all the way to the beginning of
output. Perhaps there is a secret to that, as well. joe a.
This is a bit basic but I would recommend you learn more about the pipe
command and input/output redirection in Unix.
Your mileage may vary but here's some real-world examples to get you
started.
view a file:
cat /tmp/file
view a file with pagination:
cat /tmp/file | more
redirect standard out to a file:
cat /tmp/file /tmp/file.bak
redirect standard out and standard error:
spamassassin -t /tmp/mboxfile -D 21
redirect standard out and standard error to a file:
spamassassin -t /tmp/mboxfile -D 21 /tmp/file.out
output to a file and view at the same time:
spamassassin -t /tmp/mboxfile -D 21 | tee /tmp/file.out
You can also redirect input.
Send a blank email redirecting /dev/null as the input:
mail -s 'test' kmcgr...@pccc.com /dev/null
With unix, you can do a lot of crazy things with pipes and I use it a
lot for general tasks.
For example, want the largest file in a dir?
ls -1s /var/spool/mail/ | sort -n
Want to get all the sub routines defined in a perl library listed
alphabetically?
grep sub Library_hsubox.pm | grep { | awk '{print $2}' | sort
Have a list of stuff with duplicates? Pipe through sort then uniq:
(Couldn't think of a good example without a dataset that has lots of
duplicates. But for example, you could grep an access log, use awk to
grab the IP, sort by IP and then use uniq to get a list of unique ip
addresses).
Here's a good free book: http://linux.101hacks.com/toc/
Regards
KAM
Re: No magic, since clearing database
How can I trap/redirect the -D output? In entirety? I use putty to access the box and cannot sroll back all the way to the beginning of output. Perhaps there is a secret to that, as well. Edit your Putty settings. In the Window section there is an option for Lines of scrollback. I set mine to 20. That way, I can always scroll back to see or copy whatever I need to. -- Bowie Huh. I see that on the putty I'm using here, but did not see it there. Ah, long days . . . joe a.
Re: No magic, since clearing database
tee ?? That's a new one on me. That's *two* things I've learned today! whew! Time for a nap. joe a. Martin Gregorie mar...@gregorie.org 10/24/12 1:11 PM On Wed, 2012-10-24 at 12:48 -0400, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 11:55 AM On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM Will try. How can I trap/redirect the -D output? In entirety? I use putty to access the box and cannot sroll back all the way to the beginning of output. Perhaps there is a secret to that, as well. Run the sa-learn command like this: sa-learn -D your usual arguments 21 | tee logfile.txt and then use ftp or PuTTY's scp command to copy logfile.txt back to your PC. Martin
Re: No magic, since clearing database
On 10/24/2012 1:33 PM, Joseph Acquisto wrote: tee ?? That's a new one on me. That's *two* things I've learned today! whew! Time for a nap. Used to be part of the GNU shell utils now part of coreutils http://savannah.gnu.org/projects/shellutils See http://www.gnu.org/software/coreutils/manual/html_node/index.html Regards, KAM
Re: No magic, since clearing database
On Wed, 24 Oct 2012, Joseph Acquisto wrote:
Kevin A. McGrail kmcgr...@pccc.com 10/24/12 11:55 AM
On 10/24/2012 11:25 AM, Joseph Acquisto wrote:
Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM
On 10/24/2012 6:09 AM, Joseph Acquisto wrote:
OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line.
It seems happy now. Funny, for some reason I thought that was default.
Guess not.
What version of SA are you using?
3.3.2 - I believe, came with opensuse 12.2
joe a.
According to the docs, Detect is the default. Might need a -D to see why
it's not detecting as mbox.
Can you open a bug please?
Regards,
KAM
Will try.
How can I trap/redirect the -D output? In entirety? I use putty to access the
box and cannot sroll back all the way to
the beginning of output. Perhaps there is a secret to that, as well.
joe a.
On a Unix system you can use the tee command to capture standard out to
a file as well as see it.
So something like:
sa-learn --ham -D --mbox /tmp/mailbox 21 | tee /tmp/transcript.txt
will run sa-learn --ham -D --mbox /tmp/mailbox , capturing all output
in a file and show it to you too.
Other option is to use the Unix script command to capture your
entire session in a file.
ssh to your system, do script /tmp/transcript.txt
it will give you a new shell; do your stuff you want to capture
sa-learn etc. Then do a ^D to end the script, and look at the
/tmp/transcript.txt file.
Just do -not- try to view the /tmp/transcript.txt while your script
capture is in progress.
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: BAYES_99 score
On 10/24/2012 8:35 AM, Jari Fredriksson wrote: 24.10.2012 18:19, Ned Slider kirjoitti: I have had very good success running adjusted scores for BAYES rules, but I am very careful how I train my bayes database. I've disabled auto-learning and only manually train on hand-checked ham and spam examples. Consequently, I find the extremes (BAYES_99 and BAYES_00) to be highly reliable indicators. I have never seen false BAYES_99, but false BAYES_00 is not that rare. I'm not sure what's going on, but i cleared Bayes, and set use_auto_learn 0 and then relearned from HAM/Spam messages, and checking for yesterday, I got 12 spam, every single one had BAYES_00 set. I do get a vast amount of spam coming in here, so that 12 is down from several hundred spam that got marked correctly.
Re: BAYES_99 score
On Wed, 24 Oct 2012, Cathryn Mataga wrote: On 10/24/2012 8:35 AM, Jari Fredriksson wrote: 24.10.2012 18:19, Ned Slider kirjoitti: I have had very good success running adjusted scores for BAYES rules, but I am very careful how I train my bayes database. I've disabled auto-learning and only manually train on hand-checked ham and spam examples. Consequently, I find the extremes (BAYES_99 and BAYES_00) to be highly reliable indicators. I have never seen false BAYES_99, but false BAYES_00 is not that rare. I'm not sure what's going on, but i cleared Bayes, and set use_auto_learn 0 and then relearned from HAM/Spam messages, and checking for yesterday, I got 12 spam, every single one had BAYES_00 set. Add those FNs to your spam corpus, and verify by hand every single message in your ham corpus. Then wipe and retrain again. If you get hams that score higher than BAYES_00 add them to your ham training corpus and train. If you get spams that score less than BAYES_99 add them to your spam corpus and train. The training for both of those is considered daily maintenance that should be scripted and run from cron, and doesn't involve a wipe of your database. If the FN spams are *extremely* short, they may be misclassified by Bayes. Were the FNs really short, like a message with just a URI in the body? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- You do not examine legislation in the light of the benefits it will convey if properly administered, but in the light of the wrongs it would do and the harms it would cause if improperly administered. -- Lyndon B. Johnson --- 146 days since the first successful private support mission to ISS (SpaceX)
Re: No magic, since clearing database
On 10/24/2012 at 11:54 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? Regards, KAM I have a debug output to file. Would someone like to look it over for obvious issues, before I attempt to open a bug? joe a.
Re: No magic, since clearing database
On Wed, 24 Oct 2012, Joseph Acquisto wrote: On 10/24/2012 at 11:54 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? I have a debug output to file. Would someone like to look it over for obvious issues, before I attempt to open a bug? Post it somewhere and send the URL to the list; there shouldn't be any sensitive information in the sa-learn debug log. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Political Correctness is a doctrine which is based on the premise that it is possible, through nothing more than a suitable choice of words, to pick up a turd by the clean end. --- 146 days since the first successful private support mission to ISS (SpaceX)
Re: No magic, since clearing database
On 10/24/2012 at 8:07 PM, John Hardin jhar...@impsec.org wrote: On Wed, 24 Oct 2012, Joseph Acquisto wrote: On 10/24/2012 at 11:54 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: On 10/24/2012 11:25 AM, Joseph Acquisto wrote: Kevin A. McGrail kmcgr...@pccc.com 10/24/12 9:52 AM On 10/24/2012 6:09 AM, Joseph Acquisto wrote: OBTW . . . fixed my starved db by adding --mbox to the sa-learn command line. It seems happy now. Funny, for some reason I thought that was default. Guess not. What version of SA are you using? 3.3.2 - I believe, came with opensuse 12.2 joe a. According to the docs, Detect is the default. Might need a -D to see why it's not detecting as mbox. Can you open a bug please? I have a debug output to file. Would someone like to look it over for obvious issues, before I attempt to open a bug? Post it somewhere and send the URL to the list; there shouldn't be any sensitive information in the sa-learn debug log. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Political Correctness is a doctrine which is based on the premise that it is possible, through nothing more than a suitable choice of words, to pick up a turd by the clean end. --- 146 days since the first successful private support mission to ISS (SpaceX) for the sa-learn with --mbox http://pastebin.com/T0MtyN2J for the sa-learn w/o --mbox http://pastebin.com/pD7kuEsZ expires in a day. joe a.
sa-update different rulesets
Evening, This might be particular to the Ubuntu spamassassin package, but I'm a little confused about sa-update and the channel files. I added sought dostech rulesets and updated them with sa-update. Will sa-update remember them and continue to update them daily? Does sa-update need to be told which rulesets to download? Debian/Ubuntu have a spamassassin script in /etc/cron.daily but I didn't see anything in it that was specific to the update channels. Cheers, -- jonathan
Re: sa-update different rulesets
To do sa-update with the default channel and the saught channel, I have a cron job that does: /usr/bin/sa-update --gpgkey 6C6191E3 --channel sought.rules.yerp.org --channel updates.spamassassin.org No, just grabbing a channel once will not cause sa-update to keep it up to date on its own afterward. On 10/25, Jonathan Nichols wrote: Evening, This might be particular to the Ubuntu spamassassin package, but I'm a little confused about sa-update and the channel files. I added sought dostech rulesets and updated them with sa-update. Will sa-update remember them and continue to update them daily? Does sa-update need to be told which rulesets to download? Debian/Ubuntu have a spamassassin script in /etc/cron.daily but I didn't see anything in it that was specific to the update channels. Cheers, -- jonathan -- I don't want to die... just yet... not while there's... women. - J. Matthew Root, 8/23/02 (http://www.jmrart.com/) http://www.ChaosReigns.com
