Re: Scoring Issues
Thank you, Yes, DCC Razor and Pyzor are installed and running. I will look into your other suggestions and let you know. On 1/30/18 1:37 PM, David Jones wrote: On 01/30/2018 11:47 AM, Computer Bob wrote: Also: I modified the following SA local.cf items: --- # Add *SPAM* to the Subject header of spam e-mails # rewrite_header Subject *SPAM* < Uncommented # Use Bayesian classifier (default: 1) # use_bayes 1 < Uncommented # Bayesian classifier auto-learning (default: 1) # bayes_auto_learn 1 < Uncommented # Set headers which may provide inappropriate cues to the Bayesian # classifier # # bayes_ignore_header X-Bogosity # bayes_ignore_header X-Spam-Flag # bayes_ignore_header X-Spam-Status --- I added the following: --- #dcc use_dcc 1 dcc_path /usr/local/bin/dccproc #pyzor use_pyzor 1 pyzor_path /usr/bin/pyzor #razor use_razor2 1 razor_config /etc/razor/razor-agent.conf -- I also copied the current KAM.cf to the /etc/spamassassin folder. Any further suggestions ? Did you actually install DCC, Razor, and Pyzor? Are you seeing any DCC, RAZOR, and PYZOR rule hits in your mail logs? Train your Bayes properly so you see BAYES_XX hits in your mail logs and bump up your BAYES_XX scores a little on both ends. Search the SA archives for recent tuning suggestions: - Add senderscore.org RBL - Add Lashback RBL Adjust MailSpike scores on the whitelist (negative) side: http://mailspike.org/usage.html If you are running Postfix as your MTA definitely enable postscreen with RBL weighting: https://lists.gt.net/spamassassin/users/199347 Enable greylisting in your MTA like SQLgrey.
Re: Scoring Issues
On 01/30/2018 11:47 AM, Computer Bob wrote: Also: I modified the following SA local.cf items: --- # Add *SPAM* to the Subject header of spam e-mails # rewrite_header Subject *SPAM* < Uncommented # Use Bayesian classifier (default: 1) # use_bayes 1 < Uncommented # Bayesian classifier auto-learning (default: 1) # bayes_auto_learn 1 < Uncommented # Set headers which may provide inappropriate cues to the Bayesian # classifier # # bayes_ignore_header X-Bogosity # bayes_ignore_header X-Spam-Flag # bayes_ignore_header X-Spam-Status --- I added the following: --- #dcc use_dcc 1 dcc_path /usr/local/bin/dccproc #pyzor use_pyzor 1 pyzor_path /usr/bin/pyzor #razor use_razor2 1 razor_config /etc/razor/razor-agent.conf -- I also copied the current KAM.cf to the /etc/spamassassin folder. Any further suggestions ? Did you actually install DCC, Razor, and Pyzor? Are you seeing any DCC, RAZOR, and PYZOR rule hits in your mail logs? Train your Bayes properly so you see BAYES_XX hits in your mail logs and bump up your BAYES_XX scores a little on both ends. Search the SA archives for recent tuning suggestions: - Add senderscore.org RBL - Add Lashback RBL Adjust MailSpike scores on the whitelist (negative) side: http://mailspike.org/usage.html If you are running Postfix as your MTA definitely enable postscreen with RBL weighting: https://lists.gt.net/spamassassin/users/199347 Enable greylisting in your MTA like SQLgrey. -- David Jones
Re: Scoring Issues
Also: I modified the following SA local.cf items: --- # Add *SPAM* to the Subject header of spam e-mails # rewrite_header Subject *SPAM* < Uncommented # Use Bayesian classifier (default: 1) # use_bayes 1 < Uncommented # Bayesian classifier auto-learning (default: 1) # bayes_auto_learn 1 < Uncommented # Set headers which may provide inappropriate cues to the Bayesian # classifier # # bayes_ignore_header X-Bogosity # bayes_ignore_header X-Spam-Flag # bayes_ignore_header X-Spam-Status --- I added the following: --- #dcc use_dcc 1 dcc_path /usr/local/bin/dccproc #pyzor use_pyzor 1 pyzor_path /usr/bin/pyzor #razor use_razor2 1 razor_config /etc/razor/razor-agent.conf -- I also copied the current KAM.cf to the /etc/spamassassin folder. Any further suggestions ? On 1/30/18 11:31 AM, Computer Bob wrote: Follow-up, I did a dist-upgrade to Ubuntu 16.04 LTS and the process whacked the SA bad. Removal and purging of SA was necessary and a fresh reinstall brought it back. It is currently "factory fresh". Still my problems persist, I am pursuing this via the Amavis mail list as command line calls to SA seem to indicate that it is ok.
Re: Scoring Issues
Follow-up, I did a dist-upgrade to Ubuntu 16.04 LTS and the process whacked the SA bad. Removal and purging of SA was necessary and a fresh reinstall brought it back. It is currently "factory fresh". Still my problems persist, I am pursuing this via the Amavis mail list as command line calls to SA seem to indicate that it is ok.
Re: Scoring Issues
On 27/01/2018 19:29, Ralph Seichter wrote: I trust you are aware that you actually penalise senders which pass the SPF check if you use a greater-than-zero score? Minus signs matter. ;-) Sure it's a "penalization", but of an order of magnitude so little that a minus, albeit more logically correct, wouldn't really matter in the grand scheme of scoring. I merely need dkim and spf rules to exist to use them in meta rules. But yes, a minus would be better :)
Re: Scoring Issues
Daniele Duca skrev den 2018-01-27 11:35: You are spot on, spammers are much more competent in setting up spf/dkim than most of legit mail administrators. sadly true I personally score spf/dkim that passes at 0 and only penalize the fails score 0 is disable tag if it littery 0 i just whitelist spammers that does not spam
Re: Scoring Issues
On 27.01.18 16:32, Daniele Duca wrote: > > score SPF_PASS -0.001 > > score SPF_HELO_PASS -0.001 > > I know, I meant to write that I score them at 0.001 (no minus sign in > my case) but I'm lazy :) I trust you are aware that you actually penalise senders which pass the SPF check if you use a greater-than-zero score? Minus signs matter. ;-) -Ralph
Re: Scoring Issues
On 27/01/2018 14:01, David Jones wrote: If you set those to 0, then you could be disabling many other helpful meta rules that use them. It is recommended to set them to a very small non-zero number as others have said: score SPF_PASS -0.001 score SPF_HELO_PASS -0.001 I know, I meant to write that I score them at 0.001 (no minus sign in my case) but I'm lazy :)
Re: Scoring Issues
On 01/27/2018 04:35 AM, Daniele Duca wrote: On 26/01/2018 23:54, David B Funk wrote: Regardless, giving -1 score for SPF_PASS and another -1 for SPF_HELO_PASS is nontrivial DainBRamage. It's trivial for a spammer to set up SPF on a throw-away domain and thus waltz thru that kind of filtering. You are spot on, spammers are much more competent in setting up spf/dkim than most of legit mail administrators. I personally score spf/dkim that passes at 0 and only penalize the fails Daniele If you set those to 0, then you could be disabling many other helpful meta rules that use them. It is recommended to set them to a very small non-zero number as others have said: score SPF_PASS -0.001 score SPF_HELO_PASS -0.001 -- David Jones
Re: Scoring Issues
On 26/01/2018 23:54, David B Funk wrote: Regardless, giving -1 score for SPF_PASS and another -1 for SPF_HELO_PASS is nontrivial DainBRamage. It's trivial for a spammer to set up SPF on a throw-away domain and thus waltz thru that kind of filtering. On 27.01.18 11:35, Daniele Duca wrote: You are spot on, spammers are much more competent in setting up spf/dkim than most of legit mail administrators. I personally score spf/dkim that passes at 0 and only penalize the fails note that score of "0" disables a rule, so this disables rules that depend on SPF_PASS or SPF_HELO_PASS. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. My mind is like a steel trap - rusty and illegal in 37 states.
Re: Scoring Issues
On 26/01/2018 23:54, David B Funk wrote: Regardless, giving -1 score for SPF_PASS and another -1 for SPF_HELO_PASS is nontrivial DainBRamage. It's trivial for a spammer to set up SPF on a throw-away domain and thus waltz thru that kind of filtering. You are spot on, spammers are much more competent in setting up spf/dkim than most of legit mail administrators. I personally score spf/dkim that passes at 0 and only penalize the fails Daniele
Re: Scoring Issues
On 26.01.18 14:39, b...@inter-control.com wrote: I have an issue with my setup somehow and it may be in amavis-new, most spam gets detected and delt with, some gets through and the scoring seems odd. The headers that get through are usually along the lines of: X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=- required=5 tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, score SPF_PASS -0.001 score SPF_HELO_PASS -0.001 ...who the hell configured SPF_PASS and SPF_HELO_PASS to score -1? Neither of them is a sign of non-spam. in fact, spammers exploit this. SPF only talks about FORGERY (often spam sign), not about spamminess. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I drive way too fast to worry about cholesterol.
Re: Scoring Issues
On 26 Jan 2018, at 17:47 (-0500), Computer Bob wrote: My understanding is that spamassassin is configured for razor and uribl. amavisd-new is configured to call spamassassin so is spamassassin not doing the sub calls ? Not exactly. The command-line 'spamassassin' script is written in Perl and it uses various Perl modules in the Mail::SpamAssassin::* tree. Amavisd-new also uses Mail::SpamAssassin::* modules but it does NOT use the spamassassin script or any other command-line tool. The effect of this is that it is possible for amavisd-new and spamassassin to use different configurations for the Mail::SpamAssassin::* modules. it is clear that this is happening on your system. I see no docs on configuring razor directly in amavis. If you could tell me what to look for it would be appreciated. Unfortunately, I can't help with amavisd-new because I don't use it. However, it is certain that it is using its own oddball config because these scores are ridiculous: tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, It's madness to give SPF_HELO_PASS or SPF_PASS significant scores on their own. Neither should have a score outside of the -0.01 to 0.01 range: SPF is informative but not probative. These rules somehow got set intentionally to sabotage-level scores somewhere that only the amavisd-new process is looking. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Scoring Issues
On Fri, 26 Jan 2018, John Hardin wrote:
On Fri, 26 Jan 2018, b...@inter-control.com wrote:
Oh, here is the X-SPAM status from the command line:
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
M1-2.dettenwanger.inter-control.com
X-Spam-Flag: YES
X-Spam-Level: ***
X-Spam-Status: Yes, score=23.0 required=4.0 tests=DKIM_SIGNED,
RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_SBL_CSS,RDNS_NONE,T_DKIM_INVALID,
URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM autolearn=no
autolearn_force=no
version=3.4.0
MIME-Version: 1.0
Bob
RAZOR and URIBL hits.
Is amavis perhaps configured to disable network tests?
On 1/26/18 2:48 PM, David Jones wrote:
On 01/26/2018 02:39 PM, b...@inter-control.com wrote:
The headers that get through are usually along the lines of:
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=- required=5
tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1,
T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01]
autolearn=ham autolearn_force=no
Regardless, giving -1 score for SPF_PASS and another -1 for SPF_HELO_PASS
is nontrivial DainBRamage.
It's trivial for a spammer to set up SPF on a throw-away domain and thus waltz
thru that kind of filtering.
Who set up amavis with that kind of idiocy?
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: Scoring Issues
My understanding is that spamassassin is configured for razor and uribl. amavisd-new is configured to call spamassassin so is spamassassin not doing the sub calls ? I see no docs on configuring razor directly in amavis. If you could tell me what to look for it would be appreciated. On 1/26/18 4:20 PM, John Hardin wrote: On Fri, 26 Jan 2018, b...@inter-control.com wrote: Oh, here is the X-SPAM status from the command line: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on M1-2.dettenwanger.inter-control.com X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=23.0 required=4.0 tests=DKIM_SIGNED, RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_SBL_CSS,RDNS_NONE,T_DKIM_INVALID, URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 MIME-Version: 1.0 Bob RAZOR and URIBL hits. Is amavis perhaps configured to disable network tests? On 1/26/18 2:48 PM, David Jones wrote: On 01/26/2018 02:39 PM, b...@inter-control.com wrote: The headers that get through are usually along the lines of: X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=- required=5 tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Re: Scoring Issues
Ok, I will look now, what am I looking for ? On 1/26/18 4:20 PM, John Hardin wrote: On Fri, 26 Jan 2018, b...@inter-control.com wrote: Oh, here is the X-SPAM status from the command line: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on M1-2.dettenwanger.inter-control.com X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=23.0 required=4.0 tests=DKIM_SIGNED, RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_SBL_CSS,RDNS_NONE,T_DKIM_INVALID, URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 MIME-Version: 1.0 Bob RAZOR and URIBL hits. Is amavis perhaps configured to disable network tests? On 1/26/18 2:48 PM, David Jones wrote: On 01/26/2018 02:39 PM, b...@inter-control.com wrote: The headers that get through are usually along the lines of: X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=- required=5 tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Re: Scoring Issues
I did not think so, but will check another day. 15 hours is enough for today. On 1/26/18 4:20 PM, John Hardin wrote: On Fri, 26 Jan 2018, b...@inter-control.com wrote: Oh, here is the X-SPAM status from the command line: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on M1-2.dettenwanger.inter-control.com X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=23.0 required=4.0 tests=DKIM_SIGNED, RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_SBL_CSS,RDNS_NONE,T_DKIM_INVALID, URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 MIME-Version: 1.0 Bob RAZOR and URIBL hits. Is amavis perhaps configured to disable network tests? On 1/26/18 2:48 PM, David Jones wrote: On 01/26/2018 02:39 PM, b...@inter-control.com wrote: The headers that get through are usually along the lines of: X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=- required=5 tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Re: Scoring Issues
On Fri, 26 Jan 2018, b...@inter-control.com wrote: Oh, here is the X-SPAM status from the command line: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on M1-2.dettenwanger.inter-control.com X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=23.0 required=4.0 tests=DKIM_SIGNED, RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_SBL_CSS,RDNS_NONE,T_DKIM_INVALID, URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 MIME-Version: 1.0 Bob RAZOR and URIBL hits. Is amavis perhaps configured to disable network tests? On 1/26/18 2:48 PM, David Jones wrote: On 01/26/2018 02:39 PM, b...@inter-control.com wrote: The headers that get through are usually along the lines of: X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=- required=5 tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Venezuela is busy reaping the benefits of Socialism: in one year 75% of the population has, on average, lost 19 pounds due to insufficient food, and 82% of households are below the poverty line. (2016 Venezuelan "Living Conditions Survey") --- Tomorrow: Wolfgang Amadeus Mozart's 262nd Birthday
Re: Scoring Issues
On 01/26/2018 02:39 PM, b...@inter-control.com wrote:
Greetings to all,
I have an issue with my setup somehow and it may be in amavis-new, most
spam gets detected and delt with, some gets through and the scoring
seems odd.
The headers that get through are usually along the lines of:
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=- required=5
tests=[HTML_MESSAGE=0.001, SPF_HELO_PASS=-1, SPF_PASS=-1,
T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01]
autolearn=ham autolearn_force=no
If I run the email through on the command line with:
cat {mailfile} | spamassassin -D -t
it always scores correctly and considers it spam.
The example mail above actually scored 32.2 on the command line.
I am running:
Ubuntu 14.04.5
Postfix mail_version = 2.11.0 milter_macro_v = $mail_name $mail_version
amavisd-new-2.7.1 (20120429)
ClamAV 0.99.2/24255/Thu Jan 25 11:22:47 2018
Anti-Virus scanner version: 13.0.3114
SpamAssassin version 3.4.0
running on Perl version 5.18.2
I have looked over amavis-new configs and cannot find anything out of order.
I don't understand how can most get caught and some get treated as this ?
I must be missing something.
A couple of common possibilities going on here:
1. Make sure you run the command line above as the same user as
amavisd-new is using to ensure you are using the same SA configuration.
2. How long ago did it score -1.999? If hours have gone by, other
things like RBLs and DCC can start hitting and cause the score to now be
32.2. We would need to see the X-Spam-Status output of the 32.2 score
to have an idea.
--
David Jones
