Re: How do I use sa-learn on IMAP folders on OSX?
I can't believe I forgot about them being hidden... there they are! thank you. robert - From my iPhone. > On 21 Dec 2015, at 6:43 pm, alarig wrote: > >> On Mon Dec 21 18:26:50 2015, Robert Chalmers wrote: >> and I can find the /var/mail/vhosts folders ok, but where are the other ones? > > Hi, > > Do you have a folder /var/mail/vhosts/$host/$addr/.Junk/ or something > like that? > > -- > alarig
Re: Is BAYES filtering working? Having doubts.
Good question. I'd like to know myself - From my iPhone. > On 29 Dec 2015, at 1:28 pm, Jude DaShiell wrote: > > With spamassassin, is it possible to have the filter show counts of number of > messages sent to spam, number of messages sent to ham, and total number of > messages processed that a user can check?On Mon, 28 Dec 2015, Bill Cole wrote: > >> Date: Mon, 28 Dec 2015 23:42:03 >> From: Bill Cole >> Reply-To: users@spamassassin.apache.org >> To: users@spamassassin.apache.org >> Subject: Re: Is BAYES filtering working? Having doubts. >>> On 28 Dec 2015, at 17:54, Peter L. Berghold wrote: >>> >>> The script that I use to pull the messages out of a >>> spam bucket invoking sa-learn runs as root which has permissions to read >>> from anywhere. The complication is the amavis does not have permissions >>> to read the Maildir files for trivial users like root does. >>> That said, I have some thoughts as how to solve that. >> >> In case your ideas don't work out... >> >> Useful facts: sa-learn reads stdin if you don't give it any file arguments >> and it can take mbox format as input. >> >> Using these facts, my learning script that runs as root and reads from >> multiple real users' Maildirs does this to learn ham: >> >> for AFILE in $HAMS ; do formail < $AFILE ; done| sudo -H -u $SAUSER sa-learn >> --ham --mbox >> >> Where $HAMS is the list of ham message files and $SAUSER is the user >> handling the system-wide BayesDB. I use formail there just to give each >> message a leading 'From ' line (i.e. mbox format) so that the whole bunch >> can be piped into a single sa-learn invocation. The alternative without >> formail would be to pipe each raw message into its own sa-learn. If you >> don't have sudo installed or don't like letting root use it, you can >> replicate the same effect with su in an uglier command line. > > -- >
Re: How do I actually add these descriptions then...
Thanks folks. As they are cosmetic, I'll worry about them later... I now know how to fix it up though, so it wont take long. I'll also check rule update is going on. thanks - From my iPhone. > On 8 Feb 2016, at 6:18 pm, John Hardin wrote: > >> On Mon, 8 Feb 2016, Reindl Harald wrote: >> >>> Am 08.02.2016 um 18:58 schrieb Robert Chalmers: >>> I have quite a list of these in the output from spam assassin -D ―lint >> >> they should be part of the rules itself and i don't understand why rule >> writes don't run "-D --lint" regulary *before* publish > > I do, every time before I check in, with various combinations of disabled > plugins. I don't worry that much about cosmetic warnings, I'm primarily > looking for things that will *kill* SA. That's bitten me before and it's > extremely embarrassing. > >> "describe RULNE_NAME description" in local.cf if you really find it worth >> instead write a bugreport > > DO NOT file a bug for those, they are cosmetic. Complain here on the users > list. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- > The ["assault weapons"] ban is the moral equivalent of banning red > cars because they look too fast. -- Steve Chapman, Chicago Tribune > --- > 4 days until Abraham Lincoln's and Charles Darwin's 207th Birthdays
Re: Can I drop ****** SPAM ******** not send it on?
I see. Hmmm. I have the system really screwed down tight, and understand how I can use the mail reading client to run a rule to divert such a message to a specific mailbox. I thought it may be possible to divert messages that do get marked as spam to be dumped. I can't see how some get through but they do, and as they are always spam, I'm happy to dump them. I have the system set to just reject nearly everything suspicious at the gate, but 1 or 2 still sneak through, so I'm just trying to not even see them in the mailboxes at all. I could put my configs up, but it's just clutter at this stage. thanks Robert - From my iPhone. > On 7 Mar 2016, at 5:44 pm, RW wrote: > > On Sun, 6 Mar 2016 07:35:37 + > rob...@chalmers.com.au wrote: > >> I'm trying to drop such messages, not have them still appear in my >> mailbox, but can't find a way? Any ideas? > > Are you sure you really want to do this? IMO it's a really bad idea. > > Rejecting or discarding very high-scoring spam is one-thing, but it's > sensible to file the lower-scoring spam into a folder somewhere. > > How to do any of this has nothing to do with SpamAssassin, so you need > say what you are currently doing with you mail.
SPF_TEMPERROR now firing
SPF_TEMPERROR now firing now scoring 1. Good. As I am still learning I now know something I didn't previously. Interesting responses here. - From my iPhone.
How do I use sa-learn on IMAP folders on OSX?
I understand the use of sa-learn, and all the examples, but how do I tell it
where my Spam folder is when I can’t find it myself?
This is my Dovecot configuration
mail_location = maildir:/var/mail/vhosts/%d/%n
mail_max_userip_connections = 30
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = /
and I can find the /var/mail/vhosts folders ok, but where are the other ones?
Robert Chalmers
rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio:
http://tinyurl.com/lwwddov
Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB
Storage made up of -
Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower
Bay
Looking for a way to dump spam assassin modified mail
I’m looking for a way to just dump mail that has the header modified with the * SPAM * assignment. I mean, not have the Client mail reader do it, just have either spamd, or postfix/dovecot dump it. I’m sure I’ve seen something about doing this, but can’t find it now…. lost in al the configurations. thanks Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: Looking for a way to dump spam assassin modified mail
That looks to be just what I want. I now have it running, so will see how it goes. Thanks for that. Much appreciated There are a few other really good options, but this one is nice and compact, no extra scripts. I’m running amavis-new, postfix with postscreen fairly heavily in use, dovecot, spamassassin and not much gets through, but enough to annoy me. Thanks to everyone for the pointers. very useful. Robert > On 21 Jan 2016, at 12:41, Robert Moskowitz wrote: > > I use amavis-new to do this: > > amavisd.conf > > $log_level = 1; # set the log level to one > $sa_tag_level_deflt = -999; # i want to see the headers so change to -99 > $sa_tag2_level_deflt = 5.0; # start with 5 > $sa_kill_level_deflt = 9; # change to 9 > $sa_dsn_cutoff_level = 9; # change to 9 > $sa_quarantine_cutoff_level = 50; # remove the starting # and change to 50 > $notify_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line > $forward_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line > $final_banned_destiny = D_BOUNCE; # change to D_DISCARD > > > > On 01/21/2016 07:25 AM, Robert Chalmers wrote: >> I’m looking for a way to just dump mail that has the header modified with >> the * SPAM * assignment. >> >> I mean, not have the Client mail reader do it, just have either spamd, or >> postfix/dovecot dump it. >> >> I’m sure I’ve seen something about doing this, but can’t find it >> now…. lost in al the configurations. >> >> thanks >> >> >> >> Robert Chalmers >> rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: >> http://tinyurl.com/lwwddov <http://tinyurl.com/lwwddov> >> Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. >> 2TB Storage made up of - >> Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. >> Lower Bay >> >> >> > Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Fixed? Re: Looking for a way to dump spam assassin modified mail
Just to let anyone else know who may be interested. This appears to have solved the last little bit of spam getting through, as well as removing email that had the addition to the Subject line of the SPAM** signal. I haven’t had ANY spam sneak through now since I implemented this. Thanks Robert M. Of course I have spam assassin itself running with the counts quite strict. I also have postscreen in postfix and other settings in there all quite rigid. and so on. amavisd.conf $log_level = 1; # set the log level to one $sa_tag_level_deflt = -999; # i want to see the headers so change to -99 $sa_tag2_level_deflt = 5.0; # start with 5 $sa_kill_level_deflt = 9; # change to 9 $sa_dsn_cutoff_level = 9; # change to 9 $sa_quarantine_cutoff_level = 50; # remove the starting # and change to 50 $notify_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line $forward_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line $final_banned_destiny = D_BOUNCE; # change to D_DISCARD > On 21 Jan 2016, at 12:57, Robert Chalmers wrote: > > That looks to be just what I want. I now have it running, so will see how it > goes. Thanks for that. Much appreciated > > There are a few other really good options, but this one is nice and compact, > no extra scripts. > > I’m running amavis-new, postfix with postscreen fairly heavily in use, > dovecot, spamassassin and not much gets through, but enough to annoy me. > > Thanks to everyone for the pointers. very useful. > > Robert > > >> On 21 Jan 2016, at 12:41, Robert Moskowitz > <mailto:r...@htt-consult.com>> wrote: >> >> I use amavis-new to do this: >> >> amavisd.conf >> >> $log_level = 1; # set the log level to one >> $sa_tag_level_deflt = -999; # i want to see the headers so change to -99 >> $sa_tag2_level_deflt = 5.0; # start with 5 >> $sa_kill_level_deflt = 9; # change to 9 >> $sa_dsn_cutoff_level = 9; # change to 9 >> $sa_quarantine_cutoff_level = 50; # remove the starting # and change to >> 50 >> $notify_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line >> $forward_method = 'smtp:[127.0.0.1]:10025'; # uncomment the line >> $final_banned_destiny = D_BOUNCE; # change to D_DISCARD >> >> >> >> On 01/21/2016 07:25 AM, Robert Chalmers wrote: >>> I’m looking for a way to just dump mail that has the header modified with >>> the * SPAM * assignment. >>> >>> I mean, not have the Client mail reader do it, just have either spamd, or >>> postfix/dovecot dump it. >>> >>> I’m sure I’ve seen something about doing this, but can’t find it >>> now…. lost in al the configurations. >>> >>> thanks >>> >>> >>> >>> Robert Chalmers >>> rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: >>> http://tinyurl.com/lwwddov <http://tinyurl.com/lwwddov> >>> Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. >>> 2TB Storage made up of - >>> Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. >>> Lower Bay >>> >>> >>> >> > > Robert Chalmers > rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: > http://tinyurl.com/lwwddov <http://tinyurl.com/lwwddov> > Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. > 2TB Storage made up of - > Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. > Lower Bay > > > Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Where to I add the -D option for psamassassin
That and other options I see for spamassassin at https://wiki.apache.org/spamassassin/AutolearningNotWorking "Again, use the "-D" flag to SpamAssassin, and you will see the score that is used to determine whether or not autolearning will be triggered" Where do I apply this? I have spamassassin set up with postfix on a mac. Robert Chalmers rob...@chalmers.com.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
How do I actually add these descriptions then...
I have quite a list of these in the output from spam assassin -D —lint Feb 8 17:44:07.199 [15545] dbg: config: warning: score set for non-existent rule DUP_SUSP_HDR Feb 8 17:44:07.205 [15545] dbg: config: warning: no description set for STOX_AND_PRICE Feb 8 17:44:07.206 [15545] dbg: config: warning: no description set for FSL_INTERIA_ABUSE Feb 8 17:44:07.206 [15545] dbg: config: warning: no description set for MID_DEGREES Feb 8 17:44:07.206 [15545] dbg: config: warning: no description set for HK_SCAM_N13 Feb 8 17:44:07.206 [15545] dbg: config: warning: no description set for HTML_TITLE_SUBJ_DIFF Feb 8 17:44:07.206 [15545] dbg: config: warning: no description set for STOCK_PRICES Feb 8 17:44:07.207 [15545] dbg: config: warning: no description set for LOTTERY_PH_004470 Feb 8 17:44:07.207 [15545] dbg: config: warning: no description set for REPLYTO_WITHOUT_TO_CC Feb 8 17:44:07.207 [15545] dbg: config: warning: no description set for FSL_HELO_SETUP Feb 8 17:44:07.207 [15545] dbg: config: warning: no description set for KB_RATWARE_OUTLOOK_MID thanks Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: How do I actually add these descriptions then...
: config: warning: no description set for DKIM_POLICY_SIGNSOME Feb 12 09:09:30.991 [73122] dbg: config: warning: no description set for FSL_INTERIA_ABUSE Feb 12 09:09:30.991 [73122] dbg: config: warning: no description set for L_SPAM_TOOL_13 Feb 12 09:09:30.992 [73122] dbg: config: warning: no description set for STOCK_PRICES Feb 12 09:09:30.992 [73122] dbg: config: warning: no description set for DOS_STOCK_BAT2 Feb 12 09:09:30.992 [73122] dbg: config: warning: no description set for DKIM_POLICY_TESTING Feb 12 09:09:30.992 [73122] dbg: config: warning: no description set for DKIM_VERIFIED Feb 12 09:09:30.992 [73122] dbg: config: warning: no description set for KB_RATWARE_OUTLOOK_MID Feb 12 09:09:30.993 [73122] dbg: config: warning: no description set for CTYPE_001C_B Feb 12 09:09:30.993 [73122] dbg: config: warning: no description set for SHORTENED_URL_SRC Feb 12 09:09:30.994 [73122] dbg: config: warning: no description set for TVD_PP_PHISH Feb 12 09:09:30.994 [73122] dbg: config: warning: no description set for MULTIPART_ALT_NON_TEXT Feb 12 09:09:30.994 [73122] dbg: config: warning: no description set for HELO_LH_LD Feb 12 09:09:30.994 [73122] dbg: config: warning: no description set for HTML_TITLE_SUBJ_DIFF Feb 12 09:09:30.994 [73122] dbg: config: warning: no description set for TVD_EB_PHISH Feb 12 09:09:30.995 [73122] dbg: config: warning: no description set for MSGID_DOLLARS_RANDOM Feb 12 09:09:30.995 [73122] dbg: config: warning: no description set for FSL_MIME_NO_TEXT Feb 12 09:09:30.996 [73122] dbg: config: warning: no description set for TVD_PH_BODY_META Feb 12 09:09:30.996 [73122] dbg: config: warning: no description set for CTYPE_001C_A Feb 12 09:09:30.996 [73122] dbg: config: warning: no description set for DCC_REPUT_13_19 Feb 12 09:09:30.997 [73122] dbg: config: warning: no description set for HK_SCAM_N13 Feb 12 09:09:30.997 [73122] dbg: config: warning: no description set for BUG6152_INVALID_DATE_TZ_ABSURD Feb 12 09:09:30.998 [73122] dbg: config: warning: no description set for KB_RATWARE_BOUNDARY Feb 12 09:09:30.998 [73122] dbg: config: warning: no description set for MSOE_MID_WRONG_CASE Feb 12 09:09:30.999 [73122] dbg: config: warning: no description set for HELO_LH_HOME Feb 12 09:09:30.999 [73122] dbg: config: warning: no description set for STOX_REPLY_TYPE_WITHOUT_QUOTES Feb 12 09:09:31.000 [73122] dbg: config: warning: no description set for KB_RATWARE_OUTLOOK_16 Feb 12 09:09:31.000 [73122] dbg: config: warning: no description set for MIME_BOUND_EQ_REL Feb 12 09:09:31.000 [73122] dbg: config: warning: no description set for HK_NAME_MR_MRS Feb 12 09:09:31.000 [73122] dbg: config: warning: no description set for HELO_OEM Feb 12 09:09:31.000 [73122] dbg: config: warning: no description set for FSL_HELO_BARE_IP_1 > On 10 Feb 2016, at 14:24, Reindl Harald wrote: > > > > Am 08.02.2016 um 19:18 schrieb John Hardin: >> On Mon, 8 Feb 2016, Reindl Harald wrote: >> >>> Am 08.02.2016 um 18:58 schrieb Robert Chalmers: >>>> I have quite a list of these in the output from spam assassin -D —lint >>> >>> they should be part of the rules itself and i don't understand why >>> rule writes don't run "-D --lint" regulary *before* publish >> >> I do, every time before I check in, with various combinations of >> disabled plugins. I don't worry that much about cosmetic warnings, I'm >> primarily looking for things that will *kill* SA. That's bitten me >> before and it's extremely embarrassing. >> >>> "describe RULNE_NAME description" in local.cf if you really find it >>> worth instead write a bugreport >> >> DO NOT file a bug for those, they are cosmetic. Complain here on the >> users list > > here we go: > > BASE64_LENGTH_78_79 > BUG6152_INVALID_DATE_TZ_ABSURD > CTYPE_001C_A > CTYPE_001C_B > CURR_PRICE > DKIM_POLICY_SIGNALL > DKIM_POLICY_SIGNSOME > DKIM_POLICY_TESTING > DKIM_VERIFIED > DOS_STOCK_BAT2 > FAKE_REPLY_C > FROM_MISSP_SPF_FAIL > FSL_FAKE_HOTMAIL_RVCD > FSL_HELO_BARE_IP_1 > FSL_HELO_BARE_IP_2 > FSL_HELO_DEVICE > FSL_HELO_FAKE > FSL_HELO_NON_FQDN_1 > FSL_HELO_SETUP > FSL_INTERIA_ABUSE > FSL_MIME_NO_TEXT > GEO_QUERY_STRING > HELO_FRIEND > HELO_LH_HOME > HELO_LH_LD > HELO_LOCALHOST > HELO_OEM > HIGH_CODEPAGE_URI > HK_LOTTO > HK_NAME_DR > HK_NAME_FM_MR_MRS > HK_NAME_MR_MRS > HK_SCAM_N1 > HK_SCAM_N15 > HK_SCAM_N2 > HK_SCAM_N3 > HK_SCAM_N8 > HTML_TITLE_SUBJ_DIFF > HTTPS_HTTP_MISMATCH > JM_I_FEEL_LUCKY > JM_RCVD_QMAILV1 > JM_TORA_XM > KB_DATE_CONTAINS_TAB > KB_FAKED_THE_BAT > KB_RATWARE_BOUNDARY > KB_RATWARE_MSGID > KB_RATWARE_OUTLOOK_08 > KB_RATWARE_OUTLOOK_12 > KB_RATWARE_OUTLOOK_16 > KB_RATWARE_OUTLOOK_MID > LIVEFILES
what is this telling me in spam assassin -D --lint ?
Is this a setting somewhere that I should have in place? Feb 12 09:09:30.509 [73122] dbg: config: fixed relative path: /opt/local/var/spamassassin/3.004001/updates_spamassassin_org/10_default_prefs.cf Feb 12 09:09:30.509 [73122] dbg: config: using "/opt/local/var/spamassassin/3.004001/updates_spamassassin_org/10_default_prefs.cf" for included file thanks Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. 2TB Storage made up of - Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
What do I do to fix this? bayes db update ignored: Permission denied
Should this set of directories be globally writable? Or writable by spamd? or postfix? spamd[298]: bayes: cannot write to /var/spamassassin/bayes_db/bayes_journal, bayes db update ignored: Permission denied Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: What do I do to fix this? bayes db update ignored: Permission denied
ok, I fixed it, but what is it recording ??? tail -f bayes_journal t 1457015338 ba89bf20a0 t 1457015338 f5539dc198 t 1457015338 11973086ed > On 3 Mar 2016, at 14:22, Robert Chalmers wrote: > > /var/spamassassin/bayes_db/bayes_journal Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: What do I do to fix this? bayes db update ignored: Permission denied
ah. So as in local.cf # Bayesian classifier auto-learning (default: 1) # bayes_auto_learn 1 bayes_path /var/spamassassin/bayes_db/bayes bayes_file_mode 0777 bayes_auto_learn_threshold_nonspam -0.001 bayes_auto_learn_threshold_spam 9.0 /var/spamassassin/bayes_db drwxr-xr-x 3 root wheel 102 3 Mar 14:37 . drwxr-xr-x 28 root wheel 952 23 Jan 15:58 .. drwxr-xr-x 5 root wheel 170 3 Mar 14:37 bayes_db -rw-rw-rw- 1 root wheel 2304 3 Mar 14:39 bayes_journal -rw-rw-rw- 1 root wheel 176128 3 Mar 14:32 bayes_seen -rw-rw-rw- 1 root wheel 3112960 3 Mar 14:32 bayes_toks > On 3 Mar 2016, at 14:39, Reindl Harald wrote: > > > > Am 03.03.2016 um 15:30 schrieb Robert Chalmers: >> ok, I fixed it, but what is it recording ??? >> >> tail -f bayes_journal >> t 1457015338 ba89bf20a0 >> t 1457015338 f5539dc198 >> t 1457015338 11973086ed > > http://spamassassin.apache.org/full/3.4.x/doc/sa-learn.html > > the timestamps when a token was last seen > > hence we disabled autolearning/autoexpire, train only by hand and have a > read-only namespace for the database folder and a rsyslog rule to supress > these warnings - minimizes disk IO and optimizes performance > > > Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: What do I do to fix this? bayes db update ignored: Permission denied
ok, I can see that. Interesting I missed it on the set up So, I’m running on OSX, and have to use plist files to start processes. The spamd owner is ‘spamuser’ - ( just because I did…. and as it’s not used outside that, I may as well leave it as such.) /opt/local/bin/daemondo --label=spamd --start-cmd /opt/local/libexec/perl5.22/spamd -l -u spamuser ; --pid=exec So anyway, on spamd restart, it all still appears to be working ok. Although I’m fully expecting something to come along and bite me. So what exactly is the “kludge” - given that mostly I followed the Wiki and various other setup guidelines? I’m not doing per user configs, but site wide. > On 3 Mar 2016, at 15:09, RW wrote: > > On Thu, 3 Mar 2016 14:46:33 + > Robert Chalmers wrote: > > >> >> /var/spamassassin/bayes_db >> >> drwxr-xr-x 3 root wheel 102 3 Mar 14:37 . >> drwxr-xr-x 28 root wheel 952 23 Jan 15:58 .. >> drwxr-xr-x 5 root wheel 170 3 Mar 14:37 bayes_db >> >> >> -rw-rw-rw- 1 root wheel 2304 3 Mar 14:39 bayes_journal >> -rw-rw-rw- 1 root wheel 176128 3 Mar 14:32 bayes_seen >> -rw-rw-rw- 1 root wheel 3112960 3 Mar 14:32 bayes_toks > > If spamd is running as user spamd (i.e. started as spamd -u spamd) the > files should be own by spamd. > > Don't run spamd without "-u" less you absolutely need to read per user > config from unix home directories. In that case use an sql database or > or leave the db files under ~/.spamassassin What you have there is a > dreadful kludge. > > And yes, I do know that it's suggested on the wiki. Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Re: What do I do to fix this? bayes db update ignored: Permission denied
ok, thanks all. I think I have it. zeus:bayes_db robert$ ls -la total 6528 drwxr-xr-x 5 spamuser wheel 170 3 Mar 15:35 . drwxrwxrwx 3 root wheel 102 3 Mar 14:37 .. -rw-rw 1 spamuser wheel49632 3 Mar 16:22 bayes_journal -rw-rw 1 spamuser wheel 176128 3 Mar 15:35 bayes_seen -rw-rw 1 spamuser wheel 3112960 3 Mar 15:35 bayes_toks Startup plist /opt/local/bin/daemondo --label=spamd --start-cmd /opt/local/libexec/perl5.22/spamd -l -u spamuser ; --pid=exec It’s all still working. So fortunately not too badly out of whack… thanks folks. Robert > On 3 Mar 2016, at 16:06, Tom Hendrikx wrote: > > > Hi, > > you probably messed up the permissions by running sa-learn or any other > tool that messes with the bayes files directly (i.e. not via spamd) as > root. > > Your changes work because they allow read/write access to anyone on the > system, which is not very secure. Best would be to do something like: > > chown spamuser:wheel > chmod 0660 > > Then restart spamd and see of it doesn't complain. This should allow > access for spamd and for users in the wheel group (administrative accounts). > > Regards, > Tom > > On 03-03-16 16:35, Robert Chalmers wrote: >> ok, I can see that. Interesting I missed it on the set up >> >> So, I’m running on OSX, and have to use plist files to start processes. >> The spamd owner is ‘spamuser’ - ( just because I did…. and as it’s not >> used outside that, I may as well leave it as such.) >> >> >> >> /opt/local/bin/daemondo >> --label=spamd >> --start-cmd >> /opt/local/libexec/perl5.22/spamd >> -l >> -u >> spamuser >> ; >> --pid=exec >> >> >> So anyway, on spamd restart, it all still appears to be working ok. >> Although I’m fully expecting something to come along and bite me. >> >> So what exactly is the “kludge” - given that mostly I followed the Wiki >> and various other setup guidelines? I’m not doing per user configs, but >> site wide. >> >> >> >> >> >>> On 3 Mar 2016, at 15:09, RW >> <mailto:rwmailli...@googlemail.com <mailto:rwmailli...@googlemail.com>>> >>> wrote: >>> >>> On Thu, 3 Mar 2016 14:46:33 + >>> Robert Chalmers wrote: >>> >>> >>>> >>>> /var/spamassassin/bayes_db >>>> >>>> drwxr-xr-x 3 root wheel 102 3 Mar 14:37 . >>>> drwxr-xr-x 28 root wheel 952 23 Jan 15:58 .. >>>> drwxr-xr-x 5 root wheel 170 3 Mar 14:37 bayes_db >>>> >>>> >>>> -rw-rw-rw- 1 root wheel 2304 3 Mar 14:39 bayes_journal >>>> -rw-rw-rw- 1 root wheel 176128 3 Mar 14:32 bayes_seen >>>> -rw-rw-rw- 1 root wheel 3112960 3 Mar 14:32 bayes_toks >>> >>> If spamd is running as user spamd (i.e. started as spamd -u spamd) the >>> files should be own by spamd. >>> >>> Don't run spamd without "-u" less you absolutely need to read per user >>> config from unix home directories. In that case use an sql database or >>> or leave the db files under ~/.spamassassin What you have there is a >>> dreadful kludge. >>> >>> And yes, I do know that it's suggested on the wiki. >> >> Robert Chalmers >> rob...@chalmers.com <mailto:rob...@chalmers.com> <mailto:rob...@chalmers.com >> <mailto:rob...@chalmers.com>>.au Quantum Radio: >> http://tinyurl.com/lwwddov <http://tinyurl.com/lwwddov> >> Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan >> 10.11. XCode 7.2.1 >> 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 >> HN-M101MBB. Lower Bay Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio: http://tinyurl.com/lwwddov Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Continuing - Re: How do I actually add these descriptions then...
I’ve added descriptions, grabbing the actual RULE name with awk, and creating
the list that way.
{
a=$12;
print "describe " a " Spam check applied.";
}
The result is like this.
describe LONG_TERM_PRICE Spam check applied.
describe MULTIPART_ALT_NON_TEXT Spam check applied.
describe TVD_IP_OCT Spam check applied.
describe HK_NAME_DR Spam check applied.
etc etc
So fine. Now spamassassin -D —lint returns this - and I know the names are in
the local.cf file.
, and apparently I now have two nonexistent rules.
This is the total list - BUT - each time I run the same command, it returns a
slightly different set of about 13 names???
Mar 7 15:04:18.096 [67806] dbg: config: warning: score set for non-existent
rule AXB_X_AOL_SEZ_S
Mar 7 15:04:18.096 [67806] dbg: config: warning: score set for non-existent
rule MALFORMED_FREEMAIL
Mar 7 15:04:18.104 [67806] dbg: config: warning: no description set for
STOX_REPLY_TYPE_WITHOUT_QUOTES
Mar 7 15:04:18.106 [67806] dbg: config: warning: no description set for
FSL_HELO_DEVICE
Mar 7 15:04:18.107 [67806] dbg: config: warning: no description set for
DKIM_POLICY_SIGNSOME
Mar 7 15:04:18.107 [67806] dbg: config: warning: no description set for
DKIM_POLICY_TESTING
Mar 7 15:04:18.108 [67806] dbg: config: warning: no description set for
KB_FAKED_THE_BAT
Mar 7 15:04:18.109 [67806] dbg: config: warning: no description set for
CURR_PRICE
Mar 7 15:04:18.110 [67806] dbg: config: warning: no description set for
LONG_TERM_PRICE
Mar 7 15:04:18.110 [67806] dbg: config: warning: no description set for
RCVD_IN_BRBL_LASTEXT
Mar 7 15:04:18.114 [67806] dbg: config: warning: no description set for
SHORT_TERM_PRICE
Mar 7 15:04:18.116 [67806] dbg: config: warning: no description set for
KB_DATE_CONTAINS_TAB
Mar 7 15:04:18.116 [67806] dbg: config: warning: no description set for
SB_GIF_AND_NO_URIS
Mar 7 15:04:18.118 [67806] dbg: config: warning: no description set for
BASE64_LENGTH_78_79
Mar 7 15:04:18.118 [67806] dbg: config: warning: no description set for
TVD_PH_BODY_META
thanks
Robert
> On 12 Feb 2016, at 09:11, Robert Chalmers wrote:
>
> Yes well, back again.
>
> sa-update is running, and supposedly updating rules.
> spam assassin -D —lint still shows a long list… however, no longer shows that
> first one from a while back…
>
>
> Feb 12 09:09:30.977 [73122] dbg: config: warning: no description set for
> HK_SCAM_N2
> Feb 12 09:09:30.977 [73122] dbg: config: warning: no description set for
> HK_SCAM_N3
> Feb 12 09:09:30.978 [73122] dbg: config: warning: no description set for
> BASE64_LENGTH_78_79
> Feb 12 09:09:30.979 [73122] dbg: config: warning: no description set for
> TVD_IP_OCT
> Feb 12 09:09:30.979 [73122] dbg: config: warning: no description set for
> RCVD_IN_MSPIKE_ZBI
> Feb 12 09:09:30.980 [73122] dbg: config: warning: no description set for
> JM_TORA_XM
> Feb 12 09:09:30.980 [73122] dbg: config: warning: no description set for
> STOX_REPLY_TYPE
> Feb 12 09:09:30.981 [73122] dbg: config: warning: no description set for
> FSL_HELO_NON_FQDN_1
> Feb 12 09:09:30.981 [73122] dbg: config: warning: no description set for
> JM_RCVD_QMAILV1
> Feb 12 09:09:30.981 [73122] dbg: config: warning: no description set for
> LOTTERY_1
> Feb 12 09:09:30.981 [73122] dbg: config: warning: no description set for
> TVD_FINGER_02
> Feb 12 09:09:30.982 [73122] dbg: config: warning: no description set for
> STOX_AND_PRICE
> Feb 12 09:09:30.982 [73122] dbg: config: warning: no description set for
> RCVD_IN_BRBL_LASTEXT
> Feb 12 09:09:30.982 [73122] dbg: config: warning: no description set for
> FSL_HELO_BARE_IP_2
> Feb 12 09:09:30.982 [73122] dbg: config: warning: no description set for
> HK_NAME_FM_MR_MRS
> Feb 12 09:09:30.982 [73122] dbg: config: warning: no description set for
> FSL_HELO_SETUP
> Feb 12 09:09:30.983 [73122] dbg: config: warning: no description set for
> DKIM_POLICY_SIGNALL
> Feb 12 09:09:30.983 [73122] dbg: config: warning: no description set for
> HELO_FRIEND
> Feb 12 09:09:30.983 [73122] dbg: config: warning: no description set for
> TVD_RCVD_SPACE_BRACKET
> Feb 12 09:09:30.984 [73122] dbg: config: warning: no description set for
> TVD_SPACE_RATIO
> Feb 12 09:09:30.984 [73122] dbg: config: warning: no description set for
> CURR_PRICE
> Feb 12 09:09:30.984 [73122] dbg: config: warning: no description set for
> FSL_FAKE_HOTMAIL_RVCD
> Feb 12 09:09:30.985 [73122] dbg: config: warning: no description set for
> FROM_MISSP_SPF_FAIL
> Feb 12 09:09:30.985 [73122] dbg: config: warning: no description set for
> HK_LOTTO
> Feb 12 09:09:30.985 [73122] dbg: config: warning: no description set for
> FAKE_REPLY_C
> Feb 12 09:09:30.985 [73122] dbg: config: warning: no description set for
> TVD_IP_HEX
> Feb 12 09:09:30.9
Re: Missed spam, suggestions?
8462326.442.19 42.60 0.0331
>
> Note how highly BAYES 00/99 ranked. What you don't see is that BAYES_50 is
> way down in the mud (below 50 rank).
>
> BTW, this is with a Bayes that is mostly fed via auto-learning. I occasionally
> hand feed corner cases that get mis-classified (usually things like phishes,
> or conference announcments that can look shakey).
>
>
> --
> Dave Funk University of Iowa
> College of Engineering
> 319/335-5751 FAX: 319/384-0549 1256 Seamans Center
> Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
> #include
> Better is not better, 'standard' is better. B{
Robert Chalmers
rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio:
http://tinyurl.com/lwwddov
Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11.
XCode 7.2.1
2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB.
Lower Bay
Re: Continuing - Re: How do I actually add these descriptions then...
Ok, thanks.
Everything is working now. and yes, spamassassin —lint returns nothing.
I had probably just got outputs confused somewhere along the line. The —lint -D
log now also shows no warnings.
thanks
Robert
> On 8 Mar 2016, at 13:27, RW wrote:
>
> On Tue, 8 Mar 2016 07:05:11 +
> rob...@chalmers.com.au wrote:
>
>> In the original email I posted, I'm asking two questions.
>> 1. I'm getting two warnings about nonexistent rules. Is this fixable?
>
> That's because you created descriptions for rules that don't exist.
> Either you did something wrong or the rules got removed.
>
>
>> And 2. why is lint reporting a random set of 13 missing descriptions
>> when I have actually put those descriptions into local.cf?
>>
>
> Works for me
>
> $ spamassassin --lint -D 2>&1 | awk '/ no description set for /{print
> "describe ",$12,"No description "}' > /tmp/descriptions.cf
>
> $ head -n5 /tmp/descriptions.cf
> describe HK_LOTTO No description
> describe SUBJ_GET_LAID No description
> describe TVD_SPACE_RATIO No description
> describe TVD_IP_OCT No description
> describe DKIM_POLICY_SIGNALL No description
>
> # cp /tmp/descriptions.cf /usr/local/etc/mail/spamassassin/
>
> $ spamassassin --lint -D 2>&1 | awk '/ no description set for /{print
> "describe ",$12,"No description "}' | wc -l
> 0
>
>
> Did you try spamassassin --lint ?
>
> It's hard to see genuine problems if you lint with debug on.
>
>
Robert Chalmers
rob...@chalmers.com <mailto:rob...@chalmers.com>.au Quantum Radio:
http://tinyurl.com/lwwddov
Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11.
XCode 7.2.1
2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB.
Lower Bay
Re: Missed spam, suggestions?
sa-stats.pl Sometimes part of the spamassassin package. You may have to search for it on your system, otherwise, it’s available via CPAN > On 10 Mar 2016, at 21:38, Erickarlo Porro wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 > 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 > 54.79 > 6 DKIM_VALID_AU 87114.83 26.71 7.99 48.01 > 7 BAYES_00 83904.65 25.72 1.84 > 46.24 > 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 40.62 > 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 4.39 > 37.00 > 10BAYES_50 62013.44 19.01 25.56 > 34.18 > > > Based upon your stats it looks like you need more Bayes training. Your Bayes > 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't > be in the top-10 at all. > (of course if you've only been training for a week that would explain it). > > For example, here's my top-10 hits (for a one month interval). > > TOP SPAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 > 2BAYES_99109138 32.98 82.450.01 0.9998 > 3BAYES_999 104903 31.70 79.250.01 0. > 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 > 5URIBL_BLACK 9084527.61 68.630.27 0.9942 > 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 > 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 > 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. > 9L_FROM_SPAMMER1k7366722.26 55.650.00 1. > 10T__RECEIVED_1 7241342.60 54.70 34.54 0.5135 > > OP HAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1BAYES_00182674 56.032.11 91.97 0.0150 > 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 > 3SPF_PASS136623 63.08 54.52 68.78 0.3457 > 4T_RP_MATCHES_RCV
Re: Missed spam, suggestions?
The sa-stats.pl I refer to is here. https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not the same as the ones shown in other posts. I don’t know what that is. and has an output like this. zeus:~ robert$ perl sa-stats.pl Report Title : SpamAssassin - Spam Statistics Report Date : 2016-03-11 Period Beginning : Fri 11 Mar 00:00:00 2016 Period Ending: Sat 12 Mar 00:00:00 2016 Reporting Period : 24.00 hrs -- Note: 'ham' = 'nonspam' Total spam detected: 22 ( 51.16%) Total ham accepted : 21 ( 48.84%) --- Total emails processed : 43 (2/hr) Average spam threshold :3.00 Average spam score :4.46 Average ham score : -2.10 Spam kbytes processed : 397 ( 17 kb/hr) Ham kbytes processed : 147 (6 kb/hr) Total kbytes processed : 545 ( 23 kb/hr) Spam analysis time : 339 s ( 14 s/hr) Ham analysis time : 366 s ( 15 s/hr) Total analysis time: 706 s ( 29 s/hr) Statistics by Hour Hour Spam Ham ---- 2016-03-11 00 0 ( 0%) 13 (100%) 2016-03-11 01 0 ( 0%) 0 ( 0%) 2016-03-11 02 2 (100%) 0 ( 0%) 2016-03-11 03 4 (100%) 0 ( 0%) 2016-03-11 04 4 ( 57%) 3 ( 42%) 2016-03-11 05 6 ( 75%) 2 ( 25%) 2016-03-11 06 6 (100%) 0 ( 0%) 2016-03-11 07 0 ( 0%) 3 (100%) 2016-03-11 08 0 ( 0%) 0 ( 0%) 2016-03-11 09 0 ( 0%) 0 ( 0%) 2016-03-11 10 0 ( 0%) 0 ( 0%) 2016-03-11 11 0 ( 0%) 0 ( 0%) 2016-03-11 12 0 ( 0%) 0 ( 0%) 2016-03-11 13 0 ( 0%) 0 ( 0%) 2016-03-11 14 0 ( 0%) 0 ( 0%) 2016-03-11 15 0 ( 0%) 0 ( 0%) 2016-03-11 16 0 ( 0%) 0 ( 0%) 2016-03-11 17 0 ( 0%) 0 ( 0%) 2016-03-11 18 0 ( 0%) 0 ( 0%) 2016-03-11 19 0 ( 0%) 0 ( 0%) 2016-03-11 20 0 ( 0%) 0 ( 0%) 2016-03-11 21 0 ( 0%) 0 ( 0%) 2016-03-11 22 0 ( 0%) 0 ( 0%) 2016-03-11 23 0 ( 0%) 0 ( 0%) Done. Report generated in 1 sec by sa-stats.pl, version 6256. > On 10 Mar 2016, at 21:38, Erickarlo Porro wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 > 5 RCVD_IN_DNSWL_NONE9941
Re: Missed spam, suggestions?
Sorry - I missed the post from dbfunk. I just saw it in the archive. sa-stats.pl is the program, and you have to feed it from spamd.log to get those stats. To get a spamd.log, you have to start spamd with this -s facility, --syslog=facility <> Specify the syslog facility to use (default: mail). If stderr is specified, output will be written to stderr. (This is useful if you're running spamd under the daemontools package.) With a facility of file, all output goes to spamd.log. facility is interpreted as a file name to log to if it contains any characters except a-z and 0-9. null disables logging completely (used internally). spamd -s /var/log/spamd.log # log to file /var/log/spamd.log > On 10 Mar 2016, at 21:38, Erickarlo Porro wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 > 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 > 54.79 > 6 DKIM_VALID_AU 87114.83 26.71 7.99 48.01 > 7 BAYES_00 83904.65 25.72 1.84 > 46.24 > 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 40.62 > 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 4.39 > 37.00 > 10BAYES_50 62013.44 19.01 25.56 > 34.18 > > > Based upon your stats it looks like you need more Bayes training. Your Bayes > 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't > be in the top-10 at all. > (of course if you've only been training for a week that would explain it). > > For example, here's my top-10 hits (for a one month interval). > > TOP SPAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 > 2BAYES_99109138 32.98 82.450.01 0.9998 > 3BAYES_999 104903 31.70 79.250.01 0. > 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 > 5URIBL_BLACK 9084527.61 68.630.27 0.9942 > 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 > 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 > 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. > 9L_FROM_SPAMMER1k7366722.
Re: Missed spam, suggestions?
Thanks, yes, confusion had set in there … now I’m on the right track It will however be handy to have both. Robert > On 11 Mar 2016, at 14:59, Dave Funk wrote: > > TL;DR > You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. > > This is confusing because there are two different programs named > "sa-stats.pl". > > The one that comes with SpamAssassin (what you're referring to) is an engine > stats reporting tool; does not do rule hits analysis. > > The tool that Charles Sprickman and I used is the one from Dallas Engelken. > See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers > be sure to search that page for reference to Dallas Engelken. > > > > On Fri, 11 Mar 2016, Robert Chalmers wrote: > >> The sa-stats.pl I refer to is here. >> https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not >> the same as the ones shown in other posts. I don’t know what >> that is. >> and has an output like this. >> zeus:~ robert$ perl sa-stats.pl >> Report Title : SpamAssassin - Spam Statistics >> Report Date : 2016-03-11 >> Period Beginning : Fri 11 Mar 00:00:00 2016 >> Period Ending: Sat 12 Mar 00:00:00 2016 >> Reporting Period : 24.00 hrs >> -- >> Note: 'ham' = 'nonspam' >> Total spam detected: 22 ( 51.16%) >> Total ham accepted : 21 ( 48.84%) >> --- >> Total emails processed : 43 (2/hr) >> Average spam threshold :3.00 >> Average spam score :4.46 >> Average ham score : -2.10 >> Spam kbytes processed : 397 ( 17 kb/hr) >> Ham kbytes processed : 147 (6 kb/hr) >> Total kbytes processed : 545 ( 23 kb/hr) >> Spam analysis time : 339 s ( 14 s/hr) >> Ham analysis time : 366 s ( 15 s/hr) >> Total analysis time: 706 s ( 29 s/hr) >> Statistics by Hour >> >> Hour Spam Ham >> ---- >> 2016-03-11 00 0 ( 0%) 13 (100%) >> 2016-03-11 01 0 ( 0%) 0 ( 0%) >> 2016-03-11 02 2 (100%) 0 ( 0%) >> 2016-03-11 03 4 (100%) 0 ( 0%) >> 2016-03-11 04 4 ( 57%) 3 ( 42%) >> 2016-03-11 05 6 ( 75%) 2 ( 25%) >> 2016-03-11 06 6 (100%) 0 ( 0%) >> 2016-03-11 07 0 ( 0%) 3 (100%) >> 2016-03-11 08 0 ( 0%) 0 ( 0%) >> 2016-03-11 09 0 ( 0%) 0 ( 0%) >> 2016-03-11 10 0 ( 0%) 0 ( 0%) >> 2016-03-11 11 0 ( 0%) 0 ( 0%) >> 2016-03-11 12 0 ( 0%) 0 ( 0%) >> 2016-03-11 13 0 ( 0%) 0 ( 0%) >> 2016-03-11 14 0 ( 0%) 0 ( 0%) >> 2016-03-11 15 0 ( 0%) 0 ( 0%) >> 2016-03-11 16 0 ( 0%) 0 ( 0%) >> 2016-03-11 17 0 ( 0%) 0 ( 0%) >> 2016-03-11 18 0 ( 0%) 0 ( 0%) >> 2016-03-11 19 0 ( 0%) 0 ( 0%) >> 2016-03-11 20 0 ( 0%) 0 ( 0%) >> 2016-03-11 21 0 ( 0%) 0 ( 0%) >> 2016-03-11 22 0 ( 0%) 0 ( 0%) >> 2016-03-11 23 0 ( 0%) 0 ( 0%) >> Done. Report generated in 1 sec by sa-stats.pl, version 6256. >> >> On 10 Mar 2016, at 21:38, Erickarlo Porro wrote: >> I would like to know how to get these stats too. >> From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March >> 08, 2016 5:25 AM >> To: users@spamassassin.apache.org >> Subject: Re: Missed spam, suggestions? >> Can I ask, how are you getting these stats please? >> Thanks >> On 8 Mar 2016, at 05:11, David B Funk >> wrote: >> On Mon, 7 Mar 2016, Charles Sprickman wrote: >> >> I’ve been running with some daily training for a little over a week and >> I’m seeing less spam in my inbox. I’ve >> seen a few things slip through because bayes tipped them below the >> default score, these were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM >>
Re: Missed spam, suggestions?
Just a note - that server address isn’t responding at the moment. Maybe later.Hopefully only temporary. > On 11 Mar 2016, at 14:59, Dave Funk wrote: > > TL;DR > You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. > > This is confusing because there are two different programs named > "sa-stats.pl". > > The one that comes with SpamAssassin (what you're referring to) is an engine > stats reporting tool; does not do rule hits analysis. > > The tool that Charles Sprickman and I used is the one from Dallas Engelken. > See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers > be sure to search that page for reference to Dallas Engelken. > > > > On Fri, 11 Mar 2016, Robert Chalmers wrote: > >> The sa-stats.pl I refer to is here. >> https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not >> the same as the ones shown in other posts. I don’t know what >> that is. >> and has an output like this. >> zeus:~ robert$ perl sa-stats.pl >> Report Title : SpamAssassin - Spam Statistics >> Report Date : 2016-03-11 >> Period Beginning : Fri 11 Mar 00:00:00 2016 >> Period Ending: Sat 12 Mar 00:00:00 2016 >> Reporting Period : 24.00 hrs >> -- >> Note: 'ham' = 'nonspam' >> Total spam detected: 22 ( 51.16%) >> Total ham accepted : 21 ( 48.84%) >> --- >> Total emails processed : 43 (2/hr) >> Average spam threshold :3.00 >> Average spam score :4.46 >> Average ham score : -2.10 >> Spam kbytes processed : 397 ( 17 kb/hr) >> Ham kbytes processed : 147 (6 kb/hr) >> Total kbytes processed : 545 ( 23 kb/hr) >> Spam analysis time : 339 s ( 14 s/hr) >> Ham analysis time : 366 s ( 15 s/hr) >> Total analysis time: 706 s ( 29 s/hr) >> Statistics by Hour >> >> Hour Spam Ham >> ---- >> 2016-03-11 00 0 ( 0%) 13 (100%) >> 2016-03-11 01 0 ( 0%) 0 ( 0%) >> 2016-03-11 02 2 (100%) 0 ( 0%) >> 2016-03-11 03 4 (100%) 0 ( 0%) >> 2016-03-11 04 4 ( 57%) 3 ( 42%) >> 2016-03-11 05 6 ( 75%) 2 ( 25%) >> 2016-03-11 06 6 (100%) 0 ( 0%) >> 2016-03-11 07 0 ( 0%) 3 (100%) >> 2016-03-11 08 0 ( 0%) 0 ( 0%) >> 2016-03-11 09 0 ( 0%) 0 ( 0%) >> 2016-03-11 10 0 ( 0%) 0 ( 0%) >> 2016-03-11 11 0 ( 0%) 0 ( 0%) >> 2016-03-11 12 0 ( 0%) 0 ( 0%) >> 2016-03-11 13 0 ( 0%) 0 ( 0%) >> 2016-03-11 14 0 ( 0%) 0 ( 0%) >> 2016-03-11 15 0 ( 0%) 0 ( 0%) >> 2016-03-11 16 0 ( 0%) 0 ( 0%) >> 2016-03-11 17 0 ( 0%) 0 ( 0%) >> 2016-03-11 18 0 ( 0%) 0 ( 0%) >> 2016-03-11 19 0 ( 0%) 0 ( 0%) >> 2016-03-11 20 0 ( 0%) 0 ( 0%) >> 2016-03-11 21 0 ( 0%) 0 ( 0%) >> 2016-03-11 22 0 ( 0%) 0 ( 0%) >> 2016-03-11 23 0 ( 0%) 0 ( 0%) >> Done. Report generated in 1 sec by sa-stats.pl, version 6256. >> >> On 10 Mar 2016, at 21:38, Erickarlo Porro wrote: >> I would like to know how to get these stats too. >> From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March >> 08, 2016 5:25 AM >> To: users@spamassassin.apache.org >> Subject: Re: Missed spam, suggestions? >> Can I ask, how are you getting these stats please? >> Thanks >> On 8 Mar 2016, at 05:11, David B Funk >> wrote: >> On Mon, 7 Mar 2016, Charles Sprickman wrote: >> >> I’ve been running with some daily training for a little over a week and >> I’m seeing less spam in my inbox. I’ve >> seen a few things slip through because bayes tipped them below the >> default score, these were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM >> %OFHAM >> >
Re: Missed spam, suggestions?
Found a copy here … http://www.impsec.org/~jhardin/antispam/sa-stats.pl So finally found the right one. It does seem to be all working ok - at least to my eye. ./Sa_Stats.pl --logdir /var/log --filename spamd.log --num 18 Email: 53 Autolearn:14 AvgScore: 1.02 AvgScanTime: 6.14 sec Spam:20 Autolearn: 0 AvgScore: 4.15 AvgScanTime: 5.29 sec Ham: 33 Autolearn:14 AvgScore: -0.88 AvgScanTime: 6.65 sec Time Spent Running SA: 0.09 hours Time Spent Processing Spam:0.03 hours Time Spent Processing Ham: 0.06 hours TOP SPAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM AVGSCO -- 1HTML_MESSAGE 2052.83 100.00 24.244.15 2SPF_PASS 1743.40 85.00 18.183.76 3DCC_CHECK 1539.62 75.00 18.184.33 4BAYES_50 1426.42 70.000.003.86 5RDNS_NONE 1324.53 65.000.004.15 6SPF_HELO_PASS 1324.53 65.000.004.00 7T_REMOTE_IMAGE 815.09 40.000.003.75 8DKIM_SIGNED 645.28 30.00 54.553.17 9BAYES_999 611.32 30.000.004.83 10BAYES_99611.32 30.000.004.83 11DKIM_VALID 645.28 30.00 54.553.17 12RP_MATCHES_RCVD 430.19 20.00 36.363.25 13DKIM_VALID_AU 437.74 20.00 48.483.00 14HTML_IMAGE_RATIO_02 3 5.66 15.000.003.67 15MPART_ALT_DIFF_COUNT3 5.66 15.000.006.67 16MPART_ALT_DIFF 2 3.77 10.000.006.50 17FROM_12LTRDOM 2 3.77 10.000.003.00 18MORE_SEX2 3.77 10.000.005.00 -- TOP HAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM AVGSCO -- 1BAYES_00 3260.380.00 96.97 -0.91 2HEADER_FROM_DIFFERENT_DOMAINS 2956.605.00 87.88 -0.83 3DKIM_VALID 1845.28 30.00 54.55 -0.78 4DKIM_SIGNED1845.28 30.00 54.55 -0.78 5DKIM_VALID_AU 1637.74 20.00 48.48 -0.88 6RP_MATCHES_RCVD1230.19 20.00 36.36 -1.08 7HTML_MESSAGE852.83 100.00 24.24 -0.88 8DCC_CHECK 639.62 75.00 18.180.17 9FREEMAIL_FORGED_FROMDOMAIN 611.320.00 18.18 -1.17 10SPF_PASS643.40 85.00 18.18 -1.17 11FREEMAIL_FROM 613.215.00 18.18 -1.17 12UNPARSEABLE_RELAY 3 5.660.009.09 -1.00 13DEAR_SOMETHING 2 3.770.006.060.50 14MSGID_FROM_MTA_HEADER 1 1.890.003.03 -1.00 15HTML_FONT_LOW_CONTRAST 1 5.66 10.003.030.00 16DKIM_ADSP_CUSTOM_MED1 1.890.003.03 -1.00 17BAYES_051 1.890.003.030.00 18ALL_TRUSTED 1 1.890.003.03 -2.00 -- > On 11 Mar 2016, at 15:33, Robert Chalmers wrote: > > > Just a note - that server address isn’t responding at the moment. Maybe > later.Hopefully only temporary. > > >> On 11 Mar 2016, at 14:59, Dave Funk > <mailto:dbf...@engineering.uiowa.edu>> wrote: >> >> TL;DR >> You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. >> >> This is confusing because there are two different programs named >> "sa-stats.pl". >> >> The one that comes with SpamAssassin (what you're referring to) is an engine >> stats reporting tool; does not do rule hits analysis. >> >> The tool that Charles Sprickman and I used is the one from Dallas Engelken. >> See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers >> <http
How do I tell if SPF plugin is loaded?
How do I tell if SPF is loaded ? Is there a command or a header to look for? # Requires the Mail::SpamAssassin::Plugin::SPF plugin be loaded. Thanks Robert Chalmers rob...@chalmers.com <mailto:rob...@chalmers.com>.au Mac mini 6.2 - 2012, Intel Core i7,2.3 GHz, Memory:16 GB. El-Capitan 10.11. XCode 7.2.1 2TB: Drive 0:HGST HTS721010A9E630. Upper bay. Drive 1:ST1000LM024 HN-M101MBB. Lower Bay
Why is Spamassassin not scoring these Fails
I’m trying to discover why T_SPF_TEMPERROR and the other below it are not scoring higher even though they are actually failing? This is the part from a spam message that is sneaking through. > X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on zeus.localhost > X-Spam-Level: * > X-Spam-Status: No, score=1.9 required=3.0 tests=BAYES_50,DKIM_SIGNED, > > HTML_MESSAGE,RDNS_DYNAMIC,T_DKIM_INVALID,T_SPF_HELO_TEMPERROR,T_SPF_TEMPERROR > autolearn=no autolearn_force=no version=3.4.1 > X-Spam-HAM-Report: > * 0.0 T_SPF_TEMPERROR SPF: test of record failed (temperror) > * 0.0 T_SPF_HELO_TEMPERROR SPF: test of HELO record failed (temperror) > * 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% > * [score: 0.4909] > * 0.0 HTML_MESSAGE BODY: HTML included in message > * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily > * valid > * 1.0 RDNS_DYNAMIC Delivered to internal network by host with > * dynamic-looking rDNS > * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid > Received: from sd-88929.dedibox.fr (163-172-19-195.rev.poneytelecom.eu > [163.172.19.195]) In local.cf I have > local.cf:score SPF_FAIL 2 > local.cf:score SPF_HELO_FAIL 2 > local.cf:score SPF_SOFTFAIL 2 > local.cf:score T_SPF_HELO_TEMPERROR 2 > local.cf:score T_SPF_TEMPERROR 2 I’m running with Postfix and Amavisd as well. Everything is working - except something I”ve not configured correctly in this area. Does anyone have any idea why this might be thanks Robert Chalmers
